2024-09-16T15:27:41.931 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T15:27:41.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T15:27:42.038 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357 branch: wip-athakkar-testing-2024-09-16-1454 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: athakkar@redhat.com first_in_suite: false job_id: '7907357' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-athakkar-testing-2024-09-16-1454 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-athakkar-testing-2024-09-16-1454 sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df owner: scheduled_avan@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3077 sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df sleep_before_teardown: 0 subset: 1/20 suite: fs:volumes suite_branch: wip-athakkar-testing-2024-09-16-1454 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_33f95d2ae1bcbe7eb04404303f179eeeaa6982df/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/Om5s4i2MbBgbwdRkb5h7rH8nEevdNdrHPfUh/1RkSWpY66oh6pLq9m2MCKnGik0MOTd15HdV0VZDH0MIlU+A= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNv9PK5xpaToQFRLTEv8GBPuuwtvO3nr3fXTNyciz2ntgM8zt86gJ1xlvMSbGAE5gH+h0fo9Ok1V2+/HbcvObZQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_15:21:40 tube: smithi user: avan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T15:27:42.038 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_33f95d2ae1bcbe7eb04404303f179eeeaa6982df/qa; will attempt to use it 2024-09-16T15:27:42.039 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_33f95d2ae1bcbe7eb04404303f179eeeaa6982df/qa/tasks 2024-09-16T15:27:42.039 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T15:27:42.040 INFO:teuthology.task.internal:Checking packages... 2024-09-16T15:27:42.060 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '33f95d2ae1bcbe7eb04404303f179eeeaa6982df' 2024-09-16T15:27:42.060 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T15:27:42.060 INFO:teuthology.packaging:ref: None 2024-09-16T15:27:42.061 INFO:teuthology.packaging:tag: None 2024-09-16T15:27:42.061 INFO:teuthology.packaging:branch: wip-athakkar-testing-2024-09-16-1454 2024-09-16T15:27:42.061 INFO:teuthology.packaging:sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:27:42.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-athakkar-testing-2024-09-16-1454 2024-09-16T15:27:42.196 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4938.g33f95d2a 2024-09-16T15:27:42.197 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T15:27:42.203 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T15:27:42.203 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T15:27:42.208 INFO:teuthology.task.internal:Saving configuration 2024-09-16T15:27:42.224 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T15:27:42.230 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T15:27:42.254 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 15:22:50.344463', 'locked_by': 'scheduled_avan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/Om5s4i2MbBgbwdRkb5h7rH8nEevdNdrHPfUh/1RkSWpY66oh6pLq9m2MCKnGik0MOTd15HdV0VZDH0MIlU+A='} 2024-09-16T15:27:42.277 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 15:22:50.343135', 'locked_by': 'scheduled_avan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNv9PK5xpaToQFRLTEv8GBPuuwtvO3nr3fXTNyciz2ntgM8zt86gJ1xlvMSbGAE5gH+h0fo9Ok1V2+/HbcvObZQ='} 2024-09-16T15:27:42.277 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T15:27:42.282 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-16T15:27:42.282 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-16T15:27:42.282 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T15:27:42.340 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f36d56efbe0>, signals=[15]) 2024-09-16T15:27:42.340 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T15:27:42.345 INFO:teuthology.task.internal:Opening connections... 2024-09-16T15:27:42.345 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:27:42.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:27:42.420 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:27:42.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:27:42.491 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T15:27:42.497 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-09-16T15:27:42.514 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-09-16T15:27:42.515 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-09-16T15:27:42.572 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-09-16T15:27:42.572 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T15:27:42.573 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T15:27:42.574 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-09-16T15:27:42.599 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-09-16T15:27:42.618 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-09-16T15:27:42.618 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-09-16T15:27:42.676 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T15:27:42.677 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-09-16T15:27:42.677 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T15:27:42.677 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T15:27:42.677 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T15:27:42.677 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-09-16T15:27:42.699 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T15:27:42.708 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T15:27:42.713 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T15:27:42.714 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-09-16T15:27:42.716 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-09-16T15:27:42.733 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T15:27:42.738 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T15:27:42.738 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-09-16T15:27:42.773 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-09-16T15:27:42.808 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T15:27:42.860 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T15:27:42.874 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-16T15:27:42.874 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-16T15:27:42.874 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-16T15:27:42.875 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T15:27:42.875 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T15:27:42.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:27:42.891 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-16T15:27:42.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:27:42.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-16T15:27:43.058 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:27:43.058 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-16T15:27:43.058 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-16T15:27:43.074 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:27:43.074 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-16T15:27:43.074 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-16T15:27:43.075 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-503.el9.x86_64 2024-09-16T15:27:43.075 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:27:43.076 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-16T15:27:43.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum upgrade -y linux-firmware 2024-09-16T15:27:43.093 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-503.el9.x86_64 2024-09-16T15:27:43.093 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:27:43.094 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-16T15:27:43.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum upgrade -y linux-firmware 2024-09-16T15:27:46.210 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.2 MB 00:01 2024-09-16T15:27:47.702 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-16T15:27:47.936 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-16T15:27:49.836 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-16T15:27:52.421 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-09-16T15:27:53.871 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-16T15:27:54.320 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-16T15:27:55.083 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-16T15:27:55.754 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 116 kB/s | 19 kB 00:00 2024-09-16T15:27:57.234 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-16T15:28:00.539 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T15:28:02.615 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-16T15:28:02.616 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-09-16T15:28:02.616 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-16T15:28:02.745 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-16T15:28:02.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:28:02.902 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-16T15:28:02.902 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-16T15:28:02.906 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi100.front.sepia.ceph.com to complete... 2024-09-16T15:28:03.015 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-16T15:28:05.182 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-16T15:28:05.183 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-16T15:28:05.183 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-16T15:28:05.293 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-16T15:28:05.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:28:05.450 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-16T15:28:05.450 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-16T15:28:05.453 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi060.front.sepia.ceph.com to complete... 2024-09-16T15:28:15.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16 15:28:15 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-16T15:28:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:FINISHED --2024-09-16 15:28:15-- 2024-09-16T15:28:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:Total wall clock time: 12s 2024-09-16T15:28:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:Downloaded: 1 files, 343M in 12s (28.3 MB/s) 2024-09-16T15:28:15.341 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-16T15:28:15.342 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-16T15:28:16.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16 15:28:16 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-16T15:28:16.267 INFO:teuthology.orchestra.run.smithi060.stderr:FINISHED --2024-09-16 15:28:16-- 2024-09-16T15:28:16.267 INFO:teuthology.orchestra.run.smithi060.stderr:Total wall clock time: 11s 2024-09-16T15:28:16.267 INFO:teuthology.orchestra.run.smithi060.stderr:Downloaded: 1 files, 343M in 11s (32.7 MB/s) 2024-09-16T15:28:16.269 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-16T15:28:16.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-16T15:28:17.461 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying... ######################################## 2024-09-16T15:28:17.605 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing... ######################################## 2024-09-16T15:28:17.717 INFO:teuthology.orchestra.run.smithi100.stdout:Updating / installing... 2024-09-16T15:28:18.477 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying... ######################################## 2024-09-16T15:28:18.605 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing... ######################################## 2024-09-16T15:28:18.711 INFO:teuthology.orchestra.run.smithi060.stdout:Updating / installing... 2024-09-16T15:28:25.670 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-16T15:28:27.038 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-16T15:29:05.645 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-16T15:29:05.646 DEBUG:teuthology.orchestra.run.smithi100:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-16T15:29:05.736 INFO:teuthology.orchestra.run.smithi100.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:05.736 INFO:teuthology.orchestra.run.smithi100.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:05.736 INFO:teuthology.orchestra.run.smithi100.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:05.736 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-16T15:29:05.737 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-16T15:29:05.738 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-16T15:29:05.739 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-16T15:29:05.740 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-16T15:29:05.741 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-16T15:29:05.742 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-16T15:29:05.743 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-16T15:29:05.744 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-16T15:29:05.745 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-16T15:29:05.746 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-16T15:29:05.747 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-16T15:29:05.748 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-16T15:29:05.749 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-16T15:29:05.750 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-16T15:29:05.751 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-16T15:29:05.752 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-16T15:29:05.753 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-16T15:29:05.754 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-16T15:29:05.755 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-16T15:29:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-16T15:29:05.757 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-16T15:29:05.758 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-16T15:29:05.759 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-16T15:29:05.760 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-16T15:29:05.761 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-16T15:29:05.762 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-16T15:29:05.763 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-16T15:29:05.764 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-16T15:29:05.765 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-16T15:29:05.766 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-16T15:29:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-16T15:29:05.768 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-16T15:29:05.769 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-16T15:29:05.770 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-16T15:29:05.771 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-16T15:29:05.772 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-16T15:29:05.772 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-16T15:29:05.772 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-16T15:29:05.772 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-16T15:29:05.772 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-16T15:29:05.772 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-16T15:29:05.772 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-16T15:29:05.773 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-16T15:29:05.774 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-16T15:29:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-16T15:29:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-16T15:29:05.777 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-16T15:29:05.778 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-16T15:29:05.779 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-16T15:29:05.780 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-16T15:29:05.781 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-16T15:29:05.782 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-16T15:29:05.783 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-16T15:29:05.784 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-16T15:29:05.785 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-16T15:29:05.786 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-16T15:29:05.787 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-16T15:29:05.788 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-16T15:29:05.789 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-16T15:29:05.790 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-16T15:29:05.791 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-16T15:29:05.792 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-16T15:29:05.793 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-16T15:29:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-16T15:29:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-16T15:29:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-16T15:29:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-16T15:29:05.798 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-16T15:29:05.799 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-16T15:29:05.800 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-16T15:29:05.801 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-16T15:29:05.802 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-16T15:29:05.803 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-16T15:29:05.804 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-16T15:29:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-16T15:29:05.806 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-16T15:29:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-16T15:29:05.808 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-16T15:29:05.809 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-16T15:29:05.810 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-16T15:29:05.811 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-16T15:29:05.812 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-16T15:29:05.813 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-16T15:29:05.814 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-16T15:29:05.815 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-16T15:29:05.816 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-16T15:29:05.817 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-16T15:29:05.818 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-16T15:29:05.819 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-16T15:29:05.820 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-16T15:29:05.821 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-16T15:29:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-16T15:29:05.823 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-16T15:29:05.824 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-16T15:29:05.825 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-16T15:29:05.826 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-16T15:29:05.827 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-16T15:29:05.828 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-16T15:29:05.829 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-16T15:29:05.830 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-16T15:29:05.831 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-16T15:29:05.832 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-16T15:29:05.833 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-16T15:29:05.834 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-16T15:29:05.835 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-16T15:29:05.836 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-16T15:29:05.837 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-16T15:29:05.838 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-16T15:29:05.839 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-16T15:29:05.840 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-16T15:29:05.841 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-16T15:29:05.842 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-16T15:29:05.843 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-16T15:29:05.844 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-16T15:29:05.845 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-16T15:29:05.846 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-16T15:29:05.847 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-16T15:29:05.848 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-16T15:29:05.849 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-16T15:29:05.850 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-16T15:29:05.851 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-16T15:29:05.852 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-16T15:29:05.853 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-16T15:29:05.854 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-16T15:29:05.855 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-16T15:29:05.856 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-16T15:29:05.857 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-16T15:29:05.858 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-16T15:29:05.859 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-16T15:29:05.860 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-16T15:29:05.861 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-16T15:29:05.862 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-16T15:29:05.863 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-16T15:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-16T15:29:05.865 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-16T15:29:05.866 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-16T15:29:05.867 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-16T15:29:05.868 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-16T15:29:05.869 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-16T15:29:05.870 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-16T15:29:05.871 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-16T15:29:05.872 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-16T15:29:05.873 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-16T15:29:05.874 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-16T15:29:05.875 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-16T15:29:05.876 INFO:teuthology.orchestra.run.smithi100.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T15:29:05.880 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:05.880 DEBUG:teuthology.orchestra.run.smithi100:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout:postinstall scriptlet (using /bin/sh): 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout:for file in vmlinuz System.map config; do 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout: fi 2024-09-16T15:29:05.904 INFO:teuthology.orchestra.run.smithi100.stdout:done 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:05.905 INFO:teuthology.orchestra.run.smithi100.stdout:fi 2024-09-16T15:29:05.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-09-16T15:29:05.991 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-09-16T15:29:05.991 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-09-16T15:29:05.991 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-09-16T15:29:05.991 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 82.el9 2024-09-16T15:29:05.991 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8274670 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-09-16T15:29:05.992 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-09-16T15:29:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:29:05.993 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T15:29:05.993 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T15:29:05.993 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T15:29:05.993 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-09-16T15:29:05.993 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:29:05.993 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T15:29:05.994 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T15:29:05.994 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:05.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T15:29:06.537 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-09-16T15:29:07.573 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-16T15:29:07.574 DEBUG:teuthology.orchestra.run.smithi060:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-16T15:29:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-16T15:29:07.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-16T15:29:07.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-16T15:29:07.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-16T15:29:07.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-16T15:29:07.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-16T15:29:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-16T15:29:07.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-16T15:29:07.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-16T15:29:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-16T15:29:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-16T15:29:07.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-16T15:29:07.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-16T15:29:07.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-16T15:29:07.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-16T15:29:07.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-16T15:29:07.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-16T15:29:07.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-16T15:29:07.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-16T15:29:07.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-16T15:29:07.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-16T15:29:07.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-16T15:29:07.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-16T15:29:07.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-16T15:29:07.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-16T15:29:07.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-16T15:29:07.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-16T15:29:07.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-16T15:29:07.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-16T15:29:07.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-16T15:29:07.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-16T15:29:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-16T15:29:07.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-16T15:29:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-16T15:29:07.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-16T15:29:07.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-16T15:29:07.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-16T15:29:07.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-16T15:29:07.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-16T15:29:07.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-16T15:29:07.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-16T15:29:07.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-16T15:29:07.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-16T15:29:07.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-16T15:29:07.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-16T15:29:07.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-16T15:29:07.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-16T15:29:07.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-16T15:29:07.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-16T15:29:07.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-16T15:29:07.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-16T15:29:07.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-16T15:29:07.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-16T15:29:07.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-16T15:29:07.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-16T15:29:07.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-16T15:29:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-16T15:29:07.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-16T15:29:07.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-16T15:29:07.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-16T15:29:07.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-16T15:29:07.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-16T15:29:07.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-16T15:29:07.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-16T15:29:07.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-16T15:29:07.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-16T15:29:07.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-16T15:29:07.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-16T15:29:07.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-16T15:29:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-16T15:29:07.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-16T15:29:07.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-16T15:29:07.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-16T15:29:07.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-16T15:29:07.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-16T15:29:07.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-16T15:29:07.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-16T15:29:07.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-16T15:29:07.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-16T15:29:07.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-16T15:29:07.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-16T15:29:07.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-16T15:29:07.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-16T15:29:07.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-16T15:29:07.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-16T15:29:07.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-16T15:29:07.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-16T15:29:07.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-16T15:29:07.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-16T15:29:07.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-16T15:29:07.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-16T15:29:07.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-16T15:29:07.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-16T15:29:07.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-16T15:29:07.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-16T15:29:07.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-16T15:29:07.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-16T15:29:07.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-16T15:29:07.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-16T15:29:07.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-16T15:29:07.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-16T15:29:07.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-16T15:29:07.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-16T15:29:07.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-16T15:29:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-16T15:29:07.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-16T15:29:07.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-16T15:29:07.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-16T15:29:07.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-16T15:29:07.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-16T15:29:07.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-16T15:29:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-16T15:29:07.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-16T15:29:07.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-16T15:29:07.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-16T15:29:07.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-16T15:29:07.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-16T15:29:07.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-16T15:29:07.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-16T15:29:07.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-16T15:29:07.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-16T15:29:07.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-16T15:29:07.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-16T15:29:07.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-16T15:29:07.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-16T15:29:07.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-16T15:29:07.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-16T15:29:07.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-16T15:29:07.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-16T15:29:07.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-16T15:29:07.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-16T15:29:07.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-16T15:29:07.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-16T15:29:07.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-16T15:29:07.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-16T15:29:07.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-16T15:29:07.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-16T15:29:07.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-16T15:29:07.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-16T15:29:07.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-16T15:29:07.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-16T15:29:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-16T15:29:07.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-16T15:29:07.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-16T15:29:07.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-16T15:29:07.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-16T15:29:07.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-16T15:29:07.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-16T15:29:07.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-16T15:29:07.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-16T15:29:07.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-16T15:29:07.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-16T15:29:07.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-16T15:29:07.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-16T15:29:07.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-16T15:29:07.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-16T15:29:07.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-16T15:29:07.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-16T15:29:07.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-16T15:29:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-16T15:29:07.782 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-16T15:29:07.783 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-16T15:29:07.784 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-16T15:29:07.785 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-16T15:29:07.786 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-16T15:29:07.787 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-16T15:29:07.787 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-16T15:29:07.787 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-16T15:29:07.787 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-16T15:29:07.787 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T15:29:07.790 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:07.790 DEBUG:teuthology.orchestra.run.smithi060:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-16T15:29:07.813 INFO:teuthology.orchestra.run.smithi060.stdout:postinstall scriptlet (using /bin/sh): 2024-09-16T15:29:07.813 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T15:29:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T15:29:07.813 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout:for file in vmlinuz System.map config; do 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: fi 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout:done 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-09-16T15:29:07.814 INFO:teuthology.orchestra.run.smithi060.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-16T15:29:07.815 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T15:29:07.815 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:07.815 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-09-16T15:29:07.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2024-09-16T15:29:07.885 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T15:29:07.888 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2024-09-16T15:29:07.888 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2024-09-16T15:29:07.888 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2024-09-16T15:29:07.888 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 82.el9 2024-09-16T15:29:07.888 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8274670 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2024-09-16T15:29:07.889 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:29:07.890 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T15:29:07.890 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T15:29:07.890 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T15:29:07.890 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2024-09-16T15:29:07.890 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:29:07.890 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T15:29:07.891 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T15:29:07.891 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:07.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T15:29:07.918 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-09-16T15:29:07.920 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-16T15:29:07.935 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.3FmEOcZ3lp 2024-09-16T15:29:07.935 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3FmEOcZ3lp 2024-09-16T15:29:08.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.3FmEOcZ3lp 2024-09-16T15:29:08.143 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.3FmEOcZ3lp is 6KB 2024-09-16T15:29:08.191 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.3FmEOcZ3lp 2024-09-16T15:29:08.206 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T15:29:08.270 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T15:29:08.270 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T15:29:08.270 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T15:29:08.270 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-16T15:29:08.271 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:08.428 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-09-16T15:29:08.449 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-09-16T15:29:09.837 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T15:29:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-09-16T15:29:09.876 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-09-16T15:29:09.910 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.5ro6qWB5Gb 2024-09-16T15:29:09.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5ro6qWB5Gb 2024-09-16T15:29:09.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.5ro6qWB5Gb 2024-09-16T15:29:10.115 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.5ro6qWB5Gb is 6KB 2024-09-16T15:29:10.162 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.5ro6qWB5Gb 2024-09-16T15:29:10.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T15:29:10.238 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T15:29:10.239 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T15:29:10.239 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T15:29:10.239 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-16T15:29:10.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:10.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-09-16T15:29:38.459 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-16T15:29:38.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T15:29:38.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:29:40.412 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T15:29:40.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-16T15:29:40.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:30:38.520 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T15:30:40.417 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T15:30:47.526 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T15:30:47.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:30:48.846 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-16T15:30:49.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-16T15:30:49.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:30:49.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T15:30:49.466 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-16T15:30:49.467 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-16T15:30:49.516 INFO:teuthology.orchestra.run.smithi100.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:30:49.517 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:30:49.517 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T15:30:49.517 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T15:30:49.871 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-16T15:30:50.385 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-16T15:30:50.385 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-16T15:30:50.385 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-16T15:30:50.419 INFO:teuthology.orchestra.run.smithi060.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:30:50.419 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:30:50.419 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T15:30:50.419 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T15:30:50.520 DEBUG:teuthology.parallel:result is None 2024-09-16T15:30:51.422 DEBUG:teuthology.parallel:result is None 2024-09-16T15:30:51.422 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T15:30:51.430 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T15:30:51.430 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T15:30:51.434 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T15:30:51.470 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T15:30:51.478 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T15:30:51.490 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T15:30:51.490 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T15:30:51.512 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T15:30:51.575 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T15:30:51.582 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T15:30:51.582 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T15:30:51.597 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T15:30:51.762 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T15:30:51.785 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T15:30:51.825 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T15:30:51.871 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T15:30:51.874 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T15:30:51.881 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T15:30:51.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T15:30:51.885 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T15:30:51.942 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T15:30:51.951 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T15:30:51.951 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T15:30:51.954 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T15:30:51.988 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T15:30:52.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T15:30:52.160 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T15:30:52.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T15:30:52.294 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:30:52.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T15:30:52.399 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T15:30:52.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T15:30:52.544 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T15:30:52.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T15:30:52.685 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:30:52.685 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T15:30:52.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-09-16T15:30:52.783 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-09-16T15:30:52.867 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:30:52.900 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:30:53.298 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T15:30:53.305 INFO:teuthology.task.internal:Starting timer... 2024-09-16T15:30:53.305 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T15:30:53.315 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T15:30:53.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-09-16T15:30:53.411 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-09-16T15:30:53.415 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-09-16T15:30:53.527 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-09-16T15:30:53.530 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T15:30:53.531 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-09-16T15:30:53.569 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-09-16T15:30:53.569 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-09-16T15:30:53.610 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-09-16T15:30:53.610 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive'} 2024-09-16T15:30:53.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T15:30:53.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:30:53.670 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T15:30:53.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:30:53.730 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T15:30:53.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-09-16T15:30:53.811 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-09-16T15:30:53.903 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T15:30:53.912 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T15:30:53.922 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T15:30:53.923 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi100.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T15:35:38.343 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com')] 2024-09-16T15:35:38.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-16T15:35:38.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:35:38.440 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-16T15:35:38.551 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-16T15:35:38.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T15:35:38.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:35:38.638 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-16T15:35:38.751 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-16T15:35:38.751 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T15:35:38.762 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T15:35:38.762 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T15:35:38.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:35:38.766 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T15:35:38.767 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:35:38.825 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T15:35:38.850 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T15:35:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T15:35:38.880 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T15:35:38.906 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-09-16T15:35:38.930 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-09-16T15:35:38.931 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-09-16T15:35:38.956 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-09-16T15:35:38.960 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T15:35:38.984 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T15:35:38.990 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T15:35:39.013 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T15:35:39.082 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:35:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:35:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-09-16T15:35:39.088 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:35:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:35:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:35:39.089 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:35:39.115 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:35:39.123 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:35:39.123 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-16T15:35:39.123 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:35:39.123 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:35:39.123 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:35:39.123 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:35:39.124 INFO:teuthology.run_tasks:Running task install... 2024-09-16T15:35:39.132 DEBUG:teuthology.task.install:project ceph 2024-09-16T15:35:39.132 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}} 2024-09-16T15:35:39.132 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'} 2024-09-16T15:35:39.132 INFO:teuthology.task.install:Using flavor: default 2024-09-16T15:35:39.138 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T15:35:39.138 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T15:35:39.138 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'tag': None, 'wait_for_package': False} 2024-09-16T15:35:39.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:35:39.141 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'tag': None, 'wait_for_package': False} 2024-09-16T15:35:39.141 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:35:39.297 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/ 2024-09-16T15:35:39.297 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4938.g33f95d2a 2024-09-16T15:35:39.317 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/ 2024-09-16T15:35:39.317 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4938.g33f95d2a 2024-09-16T15:35:39.426 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T15:35:39.426 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:35:39.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T15:35:39.429 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T15:35:39.429 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:35:39.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T15:35:39.487 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T15:35:39.487 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T15:35:39.488 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T15:35:39.489 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T15:35:39.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T15:35:39.587 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T15:35:39.665 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T15:35:39.667 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T15:35:39.721 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-09-16T15:35:39.722 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-09-16T15:35:39.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-09-16T15:35:39.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-09-16T15:35:39.996 INFO:teuthology.orchestra.run.smithi100.stdout:45 files removed 2024-09-16T15:35:40.008 INFO:teuthology.orchestra.run.smithi060.stdout:45 files removed 2024-09-16T15:35:40.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T15:35:40.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T15:35:40.710 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 293 kB/s | 84 kB 00:00 2024-09-16T15:35:40.748 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 282 kB/s | 84 kB 00:00 2024-09-16T15:35:41.020 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-16T15:35:41.073 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 86 kB/s | 24 kB 00:00 2024-09-16T15:35:41.311 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-16T15:35:41.373 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 7.4 kB/s | 1.9 kB 00:00 2024-09-16T15:35:43.115 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-16T15:35:45.284 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-16T15:35:45.770 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.2 MB 00:04 2024-09-16T15:35:47.604 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-16T15:35:50.193 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.5 MB 00:00 2024-09-16T15:35:51.910 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 19 kB 00:00 2024-09-16T15:35:52.538 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.5 MB 00:00 2024-09-16T15:35:53.623 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-16T15:35:54.222 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 19 kB 00:00 2024-09-16T15:35:55.609 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-16T15:36:00.245 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-09-16T15:36:02.007 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-16T15:36:02.866 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T15:36:02.867 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T15:36:02.879 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T15:36:02.880 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T15:36:02.881 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T15:36:02.881 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T15:36:02.883 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T15:36:02.886 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T15:36:02.887 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T15:36:02.956 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-16T15:36:02.979 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-09-16T15:36:02.979 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-16T15:36:02.979 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-09-16T15:36:02.979 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-16T15:36:02.979 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T15:36:02.979 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 6.6 k 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 5.5 M 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 900 k 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 145 k 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 1.5 M 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 154 k 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 14 M 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 7.4 M 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 49 k 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 11 M 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 46 M 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 285 k 2024-09-16T15:36:02.980 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 784 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 225 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 27 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 18 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 32 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 756 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 127 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T15:36:02.981 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 161 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 321 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 306 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 99 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 85 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.2 M 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 170 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T15:36:02.982 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.5 M 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.2 M 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T15:36:02.983 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 22 M 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 34 k 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 2.4 M 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 291 k 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 4.8 M 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 17 M 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 16 k 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 25 k 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T15:36:02.984 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 161 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 508 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T15:36:02.985 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 5.5 M 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T15:36:02.986 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 45 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 148 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T15:36:02.987 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T15:36:02.988 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T15:36:02.989 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T15:36:02.990 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T15:36:02.991 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout:Install 149 Packages 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 8 Packages 2024-09-16T15:36:02.992 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:36:02.993 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 235 M 2024-09-16T15:36:02.993 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-16T15:36:03.447 INFO:teuthology.orchestra.run.smithi060.stdout:(1/157): ceph-19.3.0-4938.g33f95d2a.el9.x86_64. 48 kB/s | 6.6 kB 00:00 2024-09-16T15:36:03.622 INFO:teuthology.orchestra.run.smithi060.stdout:(2/157): ceph-fuse-19.3.0-4938.g33f95d2a.el9.x8 5.0 MB/s | 900 kB 00:00 2024-09-16T15:36:03.734 INFO:teuthology.orchestra.run.smithi060.stdout:(3/157): ceph-immutable-object-cache-19.3.0-493 1.3 MB/s | 145 kB 00:00 2024-09-16T15:36:03.814 INFO:teuthology.orchestra.run.smithi060.stdout:(4/157): ceph-base-19.3.0-4938.g33f95d2a.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-16T15:36:03.906 INFO:teuthology.orchestra.run.smithi060.stdout:(5/157): ceph-mgr-19.3.0-4938.g33f95d2a.el9.x86 17 MB/s | 1.5 MB 00:00 2024-09-16T15:36:03.956 INFO:teuthology.orchestra.run.smithi060.stdout:(6/157): ceph-mds-19.3.0-4938.g33f95d2a.el9.x86 11 MB/s | 2.4 MB 00:00 2024-09-16T15:36:04.471 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T15:36:04.472 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T15:36:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T15:36:04.486 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T15:36:04.487 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T15:36:04.488 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T15:36:04.490 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T15:36:04.493 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T15:36:04.494 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T15:36:04.548 INFO:teuthology.orchestra.run.smithi060.stdout:(7/157): ceph-common-19.3.0-4938.g33f95d2a.el9. 17 MB/s | 22 MB 00:01 2024-09-16T15:36:04.558 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-16T15:36:04.574 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-16T15:36:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-09-16T15:36:04.574 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-16T15:36:04.574 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-09-16T15:36:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T15:36:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 6.6 k 2024-09-16T15:36:04.574 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 5.5 M 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 900 k 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 145 k 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 1.5 M 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 154 k 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 14 M 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 7.4 M 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 49 k 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 11 M 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 46 M 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 285 k 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 784 k 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 225 k 2024-09-16T15:36:04.575 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 27 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 18 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 32 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 756 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 127 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T15:36:04.576 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 161 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 321 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 306 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 99 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 85 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.2 M 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 170 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T15:36:04.577 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.5 M 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.2 M 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 22 M 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 34 k 2024-09-16T15:36:04.578 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 2.4 M 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 291 k 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 4.8 M 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 17 M 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 16 k 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 25 k 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T15:36:04.579 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 161 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 508 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 5.5 M 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T15:36:04.580 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T15:36:04.581 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 45 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 148 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T15:36:04.582 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T15:36:04.583 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T15:36:04.584 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T15:36:04.585 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T15:36:04.586 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout:Install 149 Packages 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 8 Packages 2024-09-16T15:36:04.587 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:36:04.588 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 235 M 2024-09-16T15:36:04.588 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-16T15:36:04.632 INFO:teuthology.orchestra.run.smithi060.stdout:(8/157): ceph-mon-19.3.0-4938.g33f95d2a.el9.x86 6.6 MB/s | 4.8 MB 00:00 2024-09-16T15:36:04.690 INFO:teuthology.orchestra.run.smithi060.stdout:(9/157): ceph-selinux-19.3.0-4938.g33f95d2a.el9 432 kB/s | 25 kB 00:00 2024-09-16T15:36:05.066 INFO:teuthology.orchestra.run.smithi060.stdout:(10/157): ceph-osd-19.3.0-4938.g33f95d2a.el9.x8 15 MB/s | 17 MB 00:01 2024-09-16T15:36:05.208 INFO:teuthology.orchestra.run.smithi060.stdout:(11/157): ceph-radosgw-19.3.0-4938.g33f95d2a.el 17 MB/s | 11 MB 00:00 2024-09-16T15:36:05.233 INFO:teuthology.orchestra.run.smithi060.stdout:(12/157): cephfs-mirror-19.3.0-4938.g33f95d2a.e 1.3 MB/s | 225 kB 00:00 2024-09-16T15:36:05.250 INFO:teuthology.orchestra.run.smithi060.stdout:(13/157): cephfs-shell-19.3.0-4938.g33f95d2a.el 648 kB/s | 27 kB 00:00 2024-09-16T15:36:05.275 INFO:teuthology.orchestra.run.smithi060.stdout:(14/157): libcephfs-devel-19.3.0-4938.g33f95d2a 759 kB/s | 32 kB 00:00 2024-09-16T15:36:05.309 INFO:teuthology.orchestra.run.smithi060.stdout:(15/157): libcephfs2-19.3.0-4938.g33f95d2a.el9. 13 MB/s | 756 kB 00:00 2024-09-16T15:36:05.334 INFO:teuthology.orchestra.run.smithi060.stdout:(16/157): libcephsqlite-19.3.0-4938.g33f95d2a.e 2.7 MB/s | 161 kB 00:00 2024-09-16T15:36:05.359 INFO:teuthology.orchestra.run.smithi060.stdout:(17/157): librados-devel-19.3.0-4938.g33f95d2a. 2.5 MB/s | 127 kB 00:00 2024-09-16T15:36:05.410 INFO:teuthology.orchestra.run.smithi060.stdout:(18/157): libradosstriper1-19.3.0-4938.g33f95d2 6.6 MB/s | 508 kB 00:00 2024-09-16T15:36:05.477 INFO:teuthology.orchestra.run.smithi060.stdout:(19/157): python3-ceph-argparse-19.3.0-4938.g33 676 kB/s | 45 kB 00:00 2024-09-16T15:36:05.560 INFO:teuthology.orchestra.run.smithi060.stdout:(20/157): python3-ceph-common-19.3.0-4938.g33f9 1.7 MB/s | 148 kB 00:00 2024-09-16T15:36:05.575 INFO:teuthology.orchestra.run.smithi100.stdout:(1/157): ceph-19.3.0-4938.g33f95d2a.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-09-16T15:36:05.635 INFO:teuthology.orchestra.run.smithi060.stdout:(21/157): librgw2-19.3.0-4938.g33f95d2a.el9.x86 20 MB/s | 5.5 MB 00:00 2024-09-16T15:36:05.661 INFO:teuthology.orchestra.run.smithi060.stdout:(22/157): python3-cephfs-19.3.0-4938.g33f95d2a. 1.6 MB/s | 161 kB 00:00 2024-09-16T15:36:05.686 INFO:teuthology.orchestra.run.smithi060.stdout:(23/157): python3-rados-19.3.0-4938.g33f95d2a.e 6.2 MB/s | 321 kB 00:00 2024-09-16T15:36:05.720 INFO:teuthology.orchestra.run.smithi060.stdout:(24/157): python3-rbd-19.3.0-4938.g33f95d2a.el9 5.1 MB/s | 306 kB 00:00 2024-09-16T15:36:05.754 INFO:teuthology.orchestra.run.smithi060.stdout:(25/157): python3-rgw-19.3.0-4938.g33f95d2a.el9 1.4 MB/s | 99 kB 00:00 2024-09-16T15:36:05.767 INFO:teuthology.orchestra.run.smithi100.stdout:(2/157): ceph-fuse-19.3.0-4938.g33f95d2a.el9.x8 4.6 MB/s | 900 kB 00:00 2024-09-16T15:36:05.795 INFO:teuthology.orchestra.run.smithi060.stdout:(26/157): rbd-fuse-19.3.0-4938.g33f95d2a.el9.x8 1.1 MB/s | 85 kB 00:00 2024-09-16T15:36:05.884 INFO:teuthology.orchestra.run.smithi100.stdout:(3/157): ceph-immutable-object-cache-19.3.0-493 1.2 MB/s | 145 kB 00:00 2024-09-16T15:36:06.040 INFO:teuthology.orchestra.run.smithi060.stdout:(27/157): rbd-mirror-19.3.0-4938.g33f95d2a.el9. 11 MB/s | 3.2 MB 00:00 2024-09-16T15:36:06.129 INFO:teuthology.orchestra.run.smithi060.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4938.g 386 kB/s | 34 kB 00:00 2024-09-16T15:36:06.206 INFO:teuthology.orchestra.run.smithi100.stdout:(4/157): ceph-mds-19.3.0-4938.g33f95d2a.el9.x86 7.4 MB/s | 2.4 MB 00:00 2024-09-16T15:36:06.213 INFO:teuthology.orchestra.run.smithi060.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4938.g33f95d2 1.8 MB/s | 154 kB 00:00 2024-09-16T15:36:06.309 INFO:teuthology.orchestra.run.smithi100.stdout:(5/157): ceph-base-19.3.0-4938.g33f95d2a.el9.x8 5.9 MB/s | 5.5 MB 00:00 2024-09-16T15:36:06.347 INFO:teuthology.orchestra.run.smithi060.stdout:(30/157): rbd-nbd-19.3.0-4938.g33f95d2a.el9.x86 309 kB/s | 170 kB 00:00 2024-09-16T15:36:06.735 INFO:teuthology.orchestra.run.smithi100.stdout:(6/157): ceph-mgr-19.3.0-4938.g33f95d2a.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-09-16T15:36:07.151 INFO:teuthology.orchestra.run.smithi100.stdout:(7/157): ceph-common-19.3.0-4938.g33f95d2a.el9. 12 MB/s | 22 MB 00:01 2024-09-16T15:36:07.452 INFO:teuthology.orchestra.run.smithi100.stdout:(8/157): ceph-mon-19.3.0-4938.g33f95d2a.el9.x86 4.2 MB/s | 4.8 MB 00:01 2024-09-16T15:36:07.505 INFO:teuthology.orchestra.run.smithi060.stdout:(31/157): ceph-test-19.3.0-4938.g33f95d2a.el9.x 16 MB/s | 46 MB 00:02 2024-09-16T15:36:07.510 INFO:teuthology.orchestra.run.smithi100.stdout:(9/157): ceph-selinux-19.3.0-4938.g33f95d2a.el9 434 kB/s | 25 kB 00:00 2024-09-16T15:36:07.556 INFO:teuthology.orchestra.run.smithi060.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4938.g33 5.7 MB/s | 291 kB 00:00 2024-09-16T15:36:07.714 INFO:teuthology.orchestra.run.smithi060.stdout:(33/157): ceph-mgr-dashboard-19.3.0-4938.g33f95 9.4 MB/s | 14 MB 00:01 2024-09-16T15:36:07.781 INFO:teuthology.orchestra.run.smithi060.stdout:(34/157): ceph-mgr-rook-19.3.0-4938.g33f95d2a.e 219 kB/s | 49 kB 00:00 2024-09-16T15:36:07.794 INFO:teuthology.orchestra.run.smithi100.stdout:(10/157): ceph-radosgw-19.3.0-4938.g33f95d2a.el 17 MB/s | 11 MB 00:00 2024-09-16T15:36:07.831 INFO:teuthology.orchestra.run.smithi060.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4938.g3 141 kB/s | 16 kB 00:00 2024-09-16T15:36:07.852 INFO:teuthology.orchestra.run.smithi100.stdout:(11/157): cephfs-mirror-19.3.0-4938.g33f95d2a.e 3.8 MB/s | 225 kB 00:00 2024-09-16T15:36:07.856 INFO:teuthology.orchestra.run.smithi060.stdout:(36/157): ceph-volume-19.3.0-4938.g33f95d2a.el9 3.7 MB/s | 285 kB 00:00 2024-09-16T15:36:07.911 INFO:teuthology.orchestra.run.smithi100.stdout:(12/157): cephfs-shell-19.3.0-4938.g33f95d2a.el 464 kB/s | 27 kB 00:00 2024-09-16T15:36:07.948 INFO:teuthology.orchestra.run.smithi060.stdout:(37/157): ceph-mgr-diskprediction-local-19.3.0- 4.6 MB/s | 7.4 MB 00:01 2024-09-16T15:36:07.961 INFO:teuthology.orchestra.run.smithi100.stdout:(13/157): libcephfs-devel-19.3.0-4938.g33f95d2a 638 kB/s | 32 kB 00:00 2024-09-16T15:36:07.999 INFO:teuthology.orchestra.run.smithi060.stdout:(38/157): cephfs-top-19.3.0-4938.g33f95d2a.el9. 129 kB/s | 18 kB 00:00 2024-09-16T15:36:08.032 INFO:teuthology.orchestra.run.smithi060.stdout:(39/157): cephadm-19.3.0-4938.g33f95d2a.el9.noa 3.8 MB/s | 784 kB 00:00 2024-09-16T15:36:08.083 INFO:teuthology.orchestra.run.smithi060.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-16T15:36:08.133 INFO:teuthology.orchestra.run.smithi060.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-16T15:36:08.183 INFO:teuthology.orchestra.run.smithi060.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-16T15:36:08.198 INFO:teuthology.orchestra.run.smithi100.stdout:(14/157): ceph-osd-19.3.0-4938.g33f95d2a.el9.x8 12 MB/s | 17 MB 00:01 2024-09-16T15:36:08.226 INFO:teuthology.orchestra.run.smithi060.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-16T15:36:08.237 INFO:teuthology.orchestra.run.smithi100.stdout:(15/157): libcephfs2-19.3.0-4938.g33f95d2a.el9. 2.7 MB/s | 756 kB 00:00 2024-09-16T15:36:08.262 INFO:teuthology.orchestra.run.smithi100.stdout:(16/157): libcephsqlite-19.3.0-4938.g33f95d2a.e 2.5 MB/s | 161 kB 00:00 2024-09-16T15:36:08.289 INFO:teuthology.orchestra.run.smithi100.stdout:(17/157): librados-devel-19.3.0-4938.g33f95d2a. 2.5 MB/s | 127 kB 00:00 2024-09-16T15:36:08.301 INFO:teuthology.orchestra.run.smithi060.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 912 kB/s | 106 kB 00:00 2024-09-16T15:36:08.326 INFO:teuthology.orchestra.run.smithi060.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-16T15:36:08.346 INFO:teuthology.orchestra.run.smithi100.stdout:(18/157): libradosstriper1-19.3.0-4938.g33f95d2 6.0 MB/s | 508 kB 00:00 2024-09-16T15:36:08.376 INFO:teuthology.orchestra.run.smithi060.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-09-16T15:36:08.405 INFO:teuthology.orchestra.run.smithi100.stdout:(19/157): python3-ceph-argparse-19.3.0-4938.g33 775 kB/s | 45 kB 00:00 2024-09-16T15:36:08.427 INFO:teuthology.orchestra.run.smithi060.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 997 kB/s | 125 kB 00:00 2024-09-16T15:36:08.463 INFO:teuthology.orchestra.run.smithi100.stdout:(20/157): python3-ceph-common-19.3.0-4938.g33f9 2.5 MB/s | 148 kB 00:00 2024-09-16T15:36:08.477 INFO:teuthology.orchestra.run.smithi060.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.4 MB/s | 215 kB 00:00 2024-09-16T15:36:08.522 INFO:teuthology.orchestra.run.smithi100.stdout:(21/157): python3-cephfs-19.3.0-4938.g33f95d2a. 2.7 MB/s | 161 kB 00:00 2024-09-16T15:36:08.569 INFO:teuthology.orchestra.run.smithi060.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-16T15:36:08.611 INFO:teuthology.orchestra.run.smithi060.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 801 kB/s | 107 kB 00:00 2024-09-16T15:36:08.616 INFO:teuthology.orchestra.run.smithi100.stdout:(22/157): librgw2-19.3.0-4938.g33f95d2a.el9.x86 17 MB/s | 5.5 MB 00:00 2024-09-16T15:36:08.627 INFO:teuthology.orchestra.run.smithi060.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-16T15:36:08.639 INFO:teuthology.orchestra.run.smithi100.stdout:(23/157): python3-rados-19.3.0-4938.g33f95d2a.e 2.7 MB/s | 321 kB 00:00 2024-09-16T15:36:08.661 INFO:teuthology.orchestra.run.smithi060.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-16T15:36:08.664 INFO:teuthology.orchestra.run.smithi100.stdout:(24/157): python3-rbd-19.3.0-4938.g33f95d2a.el9 6.0 MB/s | 306 kB 00:00 2024-09-16T15:36:08.678 INFO:teuthology.orchestra.run.smithi060.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-16T15:36:08.690 INFO:teuthology.orchestra.run.smithi100.stdout:(25/157): python3-rgw-19.3.0-4938.g33f95d2a.el9 1.9 MB/s | 99 kB 00:00 2024-09-16T15:36:08.703 INFO:teuthology.orchestra.run.smithi060.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-16T15:36:08.715 INFO:teuthology.orchestra.run.smithi100.stdout:(26/157): rbd-fuse-19.3.0-4938.g33f95d2a.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-16T15:36:08.720 INFO:teuthology.orchestra.run.smithi060.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-16T15:36:08.737 INFO:teuthology.orchestra.run.smithi060.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 338 kB/s | 11 kB 00:00 2024-09-16T15:36:08.774 INFO:teuthology.orchestra.run.smithi100.stdout:(27/157): rbd-nbd-19.3.0-4938.g33f95d2a.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-16T15:36:08.813 INFO:teuthology.orchestra.run.smithi060.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-09-16T15:36:08.838 INFO:teuthology.orchestra.run.smithi060.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2024-09-16T15:36:08.840 INFO:teuthology.orchestra.run.smithi100.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4938.g 517 kB/s | 34 kB 00:00 2024-09-16T15:36:08.855 INFO:teuthology.orchestra.run.smithi060.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-16T15:36:08.880 INFO:teuthology.orchestra.run.smithi060.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-16T15:36:08.915 INFO:teuthology.orchestra.run.smithi100.stdout:(29/157): rbd-mirror-19.3.0-4938.g33f95d2a.el9. 14 MB/s | 3.2 MB 00:00 2024-09-16T15:36:08.922 INFO:teuthology.orchestra.run.smithi060.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-16T15:36:08.941 INFO:teuthology.orchestra.run.smithi100.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4938.g33f95d2 1.5 MB/s | 154 kB 00:00 2024-09-16T15:36:08.947 INFO:teuthology.orchestra.run.smithi060.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-16T15:36:08.998 INFO:teuthology.orchestra.run.smithi060.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 633 kB/s | 47 kB 00:00 2024-09-16T15:36:09.031 INFO:teuthology.orchestra.run.smithi060.stdout:(64/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-16T15:36:09.065 INFO:teuthology.orchestra.run.smithi060.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-16T15:36:09.090 INFO:teuthology.orchestra.run.smithi060.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 662 kB/s | 38 kB 00:00 2024-09-16T15:36:09.115 INFO:teuthology.orchestra.run.smithi060.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-16T15:36:09.191 INFO:teuthology.orchestra.run.smithi060.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-16T15:36:09.266 INFO:teuthology.orchestra.run.smithi060.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-16T15:36:09.308 INFO:teuthology.orchestra.run.smithi060.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 7.6 MB/s | 910 kB 00:00 2024-09-16T15:36:09.324 INFO:teuthology.orchestra.run.smithi100.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 19 MB/s | 7.4 MB 00:00 2024-09-16T15:36:09.333 INFO:teuthology.orchestra.run.smithi060.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-16T15:36:09.383 INFO:teuthology.orchestra.run.smithi100.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4938.g33 4.9 MB/s | 291 kB 00:00 2024-09-16T15:36:09.400 INFO:teuthology.orchestra.run.smithi060.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 15 MB/s | 4.9 MB 00:00 2024-09-16T15:36:09.434 INFO:teuthology.orchestra.run.smithi100.stdout:(33/157): ceph-mgr-rook-19.3.0-4938.g33f95d2a.e 986 kB/s | 49 kB 00:00 2024-09-16T15:36:09.442 INFO:teuthology.orchestra.run.smithi060.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-16T15:36:09.467 INFO:teuthology.orchestra.run.smithi060.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-16T15:36:09.492 INFO:teuthology.orchestra.run.smithi100.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4938.g3 283 kB/s | 16 kB 00:00 2024-09-16T15:36:09.509 INFO:teuthology.orchestra.run.smithi060.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-16T15:36:09.551 INFO:teuthology.orchestra.run.smithi100.stdout:(35/157): ceph-volume-19.3.0-4938.g33f95d2a.el9 4.8 MB/s | 285 kB 00:00 2024-09-16T15:36:09.576 INFO:teuthology.orchestra.run.smithi060.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-16T15:36:09.626 INFO:teuthology.orchestra.run.smithi100.stdout:(36/157): cephadm-19.3.0-4938.g33f95d2a.el9.noa 10 MB/s | 784 kB 00:00 2024-09-16T15:36:09.652 INFO:teuthology.orchestra.run.smithi060.stdout:(77/157): postgresql-server-13.16-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-09-16T15:36:09.677 INFO:teuthology.orchestra.run.smithi060.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-09-16T15:36:09.694 INFO:teuthology.orchestra.run.smithi060.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-09-16T15:36:09.711 INFO:teuthology.orchestra.run.smithi060.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 574 kB/s | 19 kB 00:00 2024-09-16T15:36:09.736 INFO:teuthology.orchestra.run.smithi060.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-16T15:36:09.753 INFO:teuthology.orchestra.run.smithi060.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 831 kB/s | 35 kB 00:00 2024-09-16T15:36:09.793 INFO:teuthology.orchestra.run.smithi100.stdout:(37/157): ceph-mgr-dashboard-19.3.0-4938.g33f95 16 MB/s | 14 MB 00:00 2024-09-16T15:36:09.810 INFO:teuthology.orchestra.run.smithi100.stdout:(38/157): cephfs-top-19.3.0-4938.g33f95d2a.el9. 99 kB/s | 18 kB 00:00 2024-09-16T15:36:09.837 INFO:teuthology.orchestra.run.smithi060.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-16T15:36:09.862 INFO:teuthology.orchestra.run.smithi060.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-16T15:36:09.887 INFO:teuthology.orchestra.run.smithi060.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-16T15:36:09.894 INFO:teuthology.orchestra.run.smithi100.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 798 kB/s | 80 kB 00:00 2024-09-16T15:36:09.912 INFO:teuthology.orchestra.run.smithi060.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T15:36:09.929 INFO:teuthology.orchestra.run.smithi060.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-16T15:36:09.952 INFO:teuthology.orchestra.run.smithi100.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T15:36:09.955 INFO:teuthology.orchestra.run.smithi060.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-16T15:36:09.980 INFO:teuthology.orchestra.run.smithi060.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T15:36:09.994 INFO:teuthology.orchestra.run.smithi100.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-16T15:36:10.005 INFO:teuthology.orchestra.run.smithi060.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-09-16T15:36:10.020 INFO:teuthology.orchestra.run.smithi100.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-16T15:36:10.022 INFO:teuthology.orchestra.run.smithi060.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-16T15:36:10.040 INFO:teuthology.orchestra.run.smithi060.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-16T15:36:10.062 INFO:teuthology.orchestra.run.smithi100.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-09-16T15:36:10.073 INFO:teuthology.orchestra.run.smithi060.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-16T15:36:10.103 INFO:teuthology.orchestra.run.smithi100.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-16T15:36:10.129 INFO:teuthology.orchestra.run.smithi100.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-16T15:36:10.162 INFO:teuthology.orchestra.run.smithi100.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-16T15:36:10.187 INFO:teuthology.orchestra.run.smithi100.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-16T15:36:10.198 INFO:teuthology.orchestra.run.smithi060.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-16T15:36:10.215 INFO:teuthology.orchestra.run.smithi060.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 311 kB/s | 44 kB 00:00 2024-09-16T15:36:10.240 INFO:teuthology.orchestra.run.smithi060.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-16T15:36:10.263 INFO:teuthology.orchestra.run.smithi100.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-09-16T15:36:10.266 INFO:teuthology.orchestra.run.smithi060.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-16T15:36:10.283 INFO:teuthology.orchestra.run.smithi060.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-16T15:36:10.305 INFO:teuthology.orchestra.run.smithi100.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-09-16T15:36:10.308 INFO:teuthology.orchestra.run.smithi060.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-09-16T15:36:10.330 INFO:teuthology.orchestra.run.smithi100.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-16T15:36:10.333 INFO:teuthology.orchestra.run.smithi060.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T15:36:10.355 INFO:teuthology.orchestra.run.smithi100.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-16T15:36:10.359 INFO:teuthology.orchestra.run.smithi060.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-16T15:36:10.381 INFO:teuthology.orchestra.run.smithi100.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-16T15:36:10.384 INFO:teuthology.orchestra.run.smithi060.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T15:36:10.406 INFO:teuthology.orchestra.run.smithi100.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-16T15:36:10.409 INFO:teuthology.orchestra.run.smithi060.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-16T15:36:10.448 INFO:teuthology.orchestra.run.smithi100.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-16T15:36:10.515 INFO:teuthology.orchestra.run.smithi100.stdout:(55/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-09-16T15:36:10.540 INFO:teuthology.orchestra.run.smithi100.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-09-16T15:36:10.566 INFO:teuthology.orchestra.run.smithi100.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-16T15:36:10.591 INFO:teuthology.orchestra.run.smithi100.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-16T15:36:10.616 INFO:teuthology.orchestra.run.smithi100.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-16T15:36:10.658 INFO:teuthology.orchestra.run.smithi100.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-16T15:36:10.659 INFO:teuthology.orchestra.run.smithi060.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-16T15:36:10.700 INFO:teuthology.orchestra.run.smithi100.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-16T15:36:10.735 INFO:teuthology.orchestra.run.smithi060.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-16T15:36:10.747 INFO:teuthology.orchestra.run.smithi100.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 277 kB/s | 24 kB 00:00 2024-09-16T15:36:10.776 INFO:teuthology.orchestra.run.smithi100.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-16T15:36:10.777 INFO:teuthology.orchestra.run.smithi060.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-16T15:36:10.801 INFO:teuthology.orchestra.run.smithi100.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.3 MB/s | 292 kB 00:00 2024-09-16T15:36:10.811 INFO:teuthology.orchestra.run.smithi060.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 573 kB/s | 19 kB 00:00 2024-09-16T15:36:10.826 INFO:teuthology.orchestra.run.smithi100.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T15:36:10.852 INFO:teuthology.orchestra.run.smithi100.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-16T15:36:10.853 INFO:teuthology.orchestra.run.smithi060.stdout:(108/157): lua-devel-5.4.4-4.el9.x86_64.rpm 48 kB/s | 22 kB 00:00 2024-09-16T15:36:10.903 INFO:teuthology.orchestra.run.smithi060.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 523 kB/s | 26 kB 00:00 2024-09-16T15:36:10.937 INFO:teuthology.orchestra.run.smithi060.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-16T15:36:10.970 INFO:teuthology.orchestra.run.smithi060.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-16T15:36:10.985 INFO:teuthology.orchestra.run.smithi100.stdout:(67/157): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-16T15:36:11.012 INFO:teuthology.orchestra.run.smithi060.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-16T15:36:11.077 INFO:teuthology.orchestra.run.smithi100.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-16T15:36:11.121 INFO:teuthology.orchestra.run.smithi060.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-09-16T15:36:11.127 INFO:teuthology.orchestra.run.smithi100.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-16T15:36:11.169 INFO:teuthology.orchestra.run.smithi100.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-16T15:36:11.203 INFO:teuthology.orchestra.run.smithi100.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-16T15:36:11.213 INFO:teuthology.orchestra.run.smithi060.stdout:(114/157): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-16T15:36:11.246 INFO:teuthology.orchestra.run.smithi060.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-16T15:36:11.279 INFO:teuthology.orchestra.run.smithi100.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-16T15:36:11.305 INFO:teuthology.orchestra.run.smithi060.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-16T15:36:11.322 INFO:teuthology.orchestra.run.smithi060.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-09-16T15:36:11.339 INFO:teuthology.orchestra.run.smithi060.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 420 kB/s | 14 kB 00:00 2024-09-16T15:36:11.381 INFO:teuthology.orchestra.run.smithi060.stdout:(119/157): protobuf-compiler-3.14.0-13.el9.x86_ 889 kB/s | 863 kB 00:00 2024-09-16T15:36:11.382 INFO:teuthology.orchestra.run.smithi100.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-09-16T15:36:11.412 INFO:teuthology.orchestra.run.smithi100.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 657 kB/s | 87 kB 00:00 2024-09-16T15:36:11.414 INFO:teuthology.orchestra.run.smithi060.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-09-16T15:36:11.457 INFO:teuthology.orchestra.run.smithi060.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-16T15:36:11.482 INFO:teuthology.orchestra.run.smithi060.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.8 MB/s | 220 kB 00:00 2024-09-16T15:36:11.487 INFO:teuthology.orchestra.run.smithi100.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 818 kB/s | 61 kB 00:00 2024-09-16T15:36:11.524 INFO:teuthology.orchestra.run.smithi060.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-16T15:36:11.541 INFO:teuthology.orchestra.run.smithi060.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 649 kB/s | 11 kB 00:00 2024-09-16T15:36:11.566 INFO:teuthology.orchestra.run.smithi060.stdout:(125/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-09-16T15:36:11.579 INFO:teuthology.orchestra.run.smithi100.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-09-16T15:36:11.583 INFO:teuthology.orchestra.run.smithi060.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-09-16T15:36:11.600 INFO:teuthology.orchestra.run.smithi060.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 692 kB/s | 23 kB 00:00 2024-09-16T15:36:11.605 INFO:teuthology.orchestra.run.smithi100.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-16T15:36:11.617 INFO:teuthology.orchestra.run.smithi060.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 579 kB/s | 19 kB 00:00 2024-09-16T15:36:11.633 INFO:teuthology.orchestra.run.smithi060.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 589 kB/s | 20 kB 00:00 2024-09-16T15:36:11.659 INFO:teuthology.orchestra.run.smithi060.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T15:36:11.693 INFO:teuthology.orchestra.run.smithi060.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-16T15:36:11.760 INFO:teuthology.orchestra.run.smithi060.stdout:(132/157): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-09-16T15:36:11.776 INFO:teuthology.orchestra.run.smithi060.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-09-16T15:36:11.793 INFO:teuthology.orchestra.run.smithi060.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 1.0 MB/s | 16 kB 00:00 2024-09-16T15:36:11.819 INFO:teuthology.orchestra.run.smithi060.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-16T15:36:11.862 INFO:teuthology.orchestra.run.smithi060.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-16T15:36:11.886 INFO:teuthology.orchestra.run.smithi060.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 420 kB/s | 28 kB 00:00 2024-09-16T15:36:11.920 INFO:teuthology.orchestra.run.smithi060.stdout:(138/157): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-09-16T15:36:11.945 INFO:teuthology.orchestra.run.smithi060.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 372 kB/s | 31 kB 00:00 2024-09-16T15:36:11.970 INFO:teuthology.orchestra.run.smithi060.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-16T15:36:11.995 INFO:teuthology.orchestra.run.smithi060.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-16T15:36:12.021 INFO:teuthology.orchestra.run.smithi060.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-16T15:36:12.046 INFO:teuthology.orchestra.run.smithi060.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-16T15:36:12.064 INFO:teuthology.orchestra.run.smithi100.stdout:(78/157): ceph-test-19.3.0-4938.g33f95d2a.el9.x 10 MB/s | 46 MB 00:04 2024-09-16T15:36:12.071 INFO:teuthology.orchestra.run.smithi060.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-16T15:36:12.089 INFO:teuthology.orchestra.run.smithi100.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 488 kB/s | 248 kB 00:00 2024-09-16T15:36:12.097 INFO:teuthology.orchestra.run.smithi060.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-16T15:36:12.122 INFO:teuthology.orchestra.run.smithi060.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 400 kB/s | 20 kB 00:00 2024-09-16T15:36:12.131 INFO:teuthology.orchestra.run.smithi100.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 49 kB/s | 26 kB 00:00 2024-09-16T15:36:12.155 INFO:teuthology.orchestra.run.smithi060.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T15:36:12.173 INFO:teuthology.orchestra.run.smithi100.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 231 kB/s | 19 kB 00:00 2024-09-16T15:36:12.180 INFO:teuthology.orchestra.run.smithi060.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-16T15:36:12.214 INFO:teuthology.orchestra.run.smithi100.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 416 kB/s | 35 kB 00:00 2024-09-16T15:36:12.264 INFO:teuthology.orchestra.run.smithi060.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-16T15:36:12.265 INFO:teuthology.orchestra.run.smithi100.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-16T15:36:12.292 INFO:teuthology.orchestra.run.smithi100.stdout:(84/157): python3-mako-1.1.4-6.el9.noarch.rpm 860 kB/s | 172 kB 00:00 2024-09-16T15:36:12.315 INFO:teuthology.orchestra.run.smithi100.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-16T15:36:12.341 INFO:teuthology.orchestra.run.smithi100.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T15:36:12.382 INFO:teuthology.orchestra.run.smithi100.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 633 kB/s | 42 kB 00:00 2024-09-16T15:36:12.425 INFO:teuthology.orchestra.run.smithi100.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-16T15:36:12.440 INFO:teuthology.orchestra.run.smithi060.stdout:(150/157): librbd1-19.3.0-4938.g33f95d2a.el9.x8 13 MB/s | 3.2 MB 00:00 2024-09-16T15:36:12.450 INFO:teuthology.orchestra.run.smithi100.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-16T15:36:12.475 INFO:teuthology.orchestra.run.smithi100.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-09-16T15:36:12.490 INFO:teuthology.orchestra.run.smithi060.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 10 MB/s | 2.3 MB 00:00 2024-09-16T15:36:12.517 INFO:teuthology.orchestra.run.smithi100.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-09-16T15:36:12.557 INFO:teuthology.orchestra.run.smithi060.stdout:(152/157): librados2-19.3.0-4938.g33f95d2a.el9. 8.6 MB/s | 3.5 MB 00:00 2024-09-16T15:36:12.559 INFO:teuthology.orchestra.run.smithi100.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 643 kB/s | 54 kB 00:00 2024-09-16T15:36:12.599 INFO:teuthology.orchestra.run.smithi060.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 376 kB/s | 41 kB 00:00 2024-09-16T15:36:12.654 INFO:teuthology.orchestra.run.smithi100.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-16T15:36:12.657 INFO:teuthology.orchestra.run.smithi060.stdout:(154/157): NetworkManager-libnm-1.49.90-1.el9.x 8.3 MB/s | 1.8 MB 00:00 2024-09-16T15:36:12.683 INFO:teuthology.orchestra.run.smithi060.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 1.9 MB/s | 249 kB 00:00 2024-09-16T15:36:12.702 INFO:teuthology.orchestra.run.smithi100.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 294 kB/s | 42 kB 00:00 2024-09-16T15:36:12.743 INFO:teuthology.orchestra.run.smithi100.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 499 kB/s | 44 kB 00:00 2024-09-16T15:36:12.766 INFO:teuthology.orchestra.run.smithi060.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-16T15:36:12.769 INFO:teuthology.orchestra.run.smithi100.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-16T15:36:12.811 INFO:teuthology.orchestra.run.smithi100.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-09-16T15:36:12.833 INFO:teuthology.orchestra.run.smithi060.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-16T15:36:12.839 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-16T15:36:12.839 INFO:teuthology.orchestra.run.smithi060.stdout:Total 24 MB/s | 235 MB 00:09 2024-09-16T15:36:12.853 INFO:teuthology.orchestra.run.smithi100.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 569 kB/s | 47 kB 00:00 2024-09-16T15:36:12.878 INFO:teuthology.orchestra.run.smithi100.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.5 MB/s | 304 kB 00:00 2024-09-16T15:36:12.920 INFO:teuthology.orchestra.run.smithi100.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 868 kB/s | 58 kB 00:00 2024-09-16T15:36:12.945 INFO:teuthology.orchestra.run.smithi100.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-16T15:36:12.987 INFO:teuthology.orchestra.run.smithi100.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 955 kB/s | 64 kB 00:00 2024-09-16T15:36:13.029 INFO:teuthology.orchestra.run.smithi100.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-09-16T15:36:13.163 INFO:teuthology.orchestra.run.smithi100.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-09-16T15:36:13.371 INFO:teuthology.orchestra.run.smithi100.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-16T15:36:13.430 INFO:teuthology.orchestra.run.smithi100.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-09-16T15:36:13.457 INFO:teuthology.orchestra.run.smithi100.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 737 kB/s | 19 kB 00:00 2024-09-16T15:36:13.499 INFO:teuthology.orchestra.run.smithi100.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.4 MB/s | 308 kB 00:00 2024-09-16T15:36:13.524 INFO:teuthology.orchestra.run.smithi100.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T15:36:13.558 INFO:teuthology.orchestra.run.smithi100.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-16T15:36:13.591 INFO:teuthology.orchestra.run.smithi100.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-16T15:36:13.633 INFO:teuthology.orchestra.run.smithi100.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-16T15:36:13.750 INFO:teuthology.orchestra.run.smithi100.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-16T15:36:13.809 INFO:teuthology.orchestra.run.smithi100.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-16T15:36:13.834 INFO:teuthology.orchestra.run.smithi100.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T15:36:13.860 INFO:teuthology.orchestra.run.smithi100.stdout:(116/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-09-16T15:36:13.885 INFO:teuthology.orchestra.run.smithi100.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-16T15:36:13.927 INFO:teuthology.orchestra.run.smithi100.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.3 MB/s | 172 kB 00:00 2024-09-16T15:36:13.952 INFO:teuthology.orchestra.run.smithi100.stdout:(119/157): python3-certifi-2023.05.07-4.el9.noa 153 kB/s | 14 kB 00:00 2024-09-16T15:36:14.036 INFO:teuthology.orchestra.run.smithi100.stdout:(120/157): libarrow-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-09-16T15:36:14.061 INFO:teuthology.orchestra.run.smithi100.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 2.7 MB/s | 358 kB 00:00 2024-09-16T15:36:14.086 INFO:teuthology.orchestra.run.smithi100.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T15:36:14.120 INFO:teuthology.orchestra.run.smithi100.stdout:(123/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-09-16T15:36:14.145 INFO:teuthology.orchestra.run.smithi100.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-16T15:36:14.170 INFO:teuthology.orchestra.run.smithi100.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-16T15:36:14.196 INFO:teuthology.orchestra.run.smithi100.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-16T15:36:14.221 INFO:teuthology.orchestra.run.smithi100.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-16T15:36:14.252 INFO:teuthology.orchestra.run.smithi100.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 344 kB/s | 19 kB 00:00 2024-09-16T15:36:14.280 INFO:teuthology.orchestra.run.smithi100.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-09-16T15:36:14.313 INFO:teuthology.orchestra.run.smithi100.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-16T15:36:14.347 INFO:teuthology.orchestra.run.smithi100.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-16T15:36:14.389 INFO:teuthology.orchestra.run.smithi100.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-09-16T15:36:14.423 INFO:teuthology.orchestra.run.smithi100.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.0 MB/s | 1.0 MB 00:00 2024-09-16T15:36:14.448 INFO:teuthology.orchestra.run.smithi100.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-16T15:36:14.498 INFO:teuthology.orchestra.run.smithi100.stdout:(135/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.7 MB/s | 2.0 MB 00:00 2024-09-16T15:36:14.523 INFO:teuthology.orchestra.run.smithi100.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-16T15:36:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-16T15:36:14.574 INFO:teuthology.orchestra.run.smithi100.stdout:(138/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-09-16T15:36:14.591 INFO:teuthology.orchestra.run.smithi100.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-09-16T15:36:14.616 INFO:teuthology.orchestra.run.smithi100.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-09-16T15:36:14.658 INFO:teuthology.orchestra.run.smithi100.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 537 kB/s | 36 kB 00:00 2024-09-16T15:36:14.675 INFO:teuthology.orchestra.run.smithi100.stdout:(142/157): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-09-16T15:36:14.692 INFO:teuthology.orchestra.run.smithi100.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 984 kB/s | 74 kB 00:00 2024-09-16T15:36:14.717 INFO:teuthology.orchestra.run.smithi100.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-16T15:36:14.751 INFO:teuthology.orchestra.run.smithi100.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-16T15:36:14.776 INFO:teuthology.orchestra.run.smithi100.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 339 kB/s | 20 kB 00:00 2024-09-16T15:36:14.801 INFO:teuthology.orchestra.run.smithi100.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T15:36:14.827 INFO:teuthology.orchestra.run.smithi100.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-16T15:36:14.935 INFO:teuthology.orchestra.run.smithi100.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-16T15:36:14.940 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-16T15:36:15.102 INFO:teuthology.orchestra.run.smithi100.stdout:(150/157): librados2-19.3.0-4938.g33f95d2a.el9. 12 MB/s | 3.5 MB 00:00 2024-09-16T15:36:15.140 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-16T15:36:15.141 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-16T15:36:15.153 INFO:teuthology.orchestra.run.smithi100.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-09-16T15:36:15.195 INFO:teuthology.orchestra.run.smithi100.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 983 kB/s | 41 kB 00:00 2024-09-16T15:36:15.253 INFO:teuthology.orchestra.run.smithi100.stdout:(153/157): librbd1-19.3.0-4938.g33f95d2a.el9.x8 7.6 MB/s | 3.2 MB 00:00 2024-09-16T15:36:15.303 INFO:teuthology.orchestra.run.smithi100.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-16T15:36:15.362 INFO:teuthology.orchestra.run.smithi100.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 7.0 MB/s | 1.8 MB 00:00 2024-09-16T15:36:15.421 INFO:teuthology.orchestra.run.smithi100.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-09-16T15:36:15.479 INFO:teuthology.orchestra.run.smithi100.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 1.4 MB 00:00 2024-09-16T15:36:15.485 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-16T15:36:15.485 INFO:teuthology.orchestra.run.smithi100.stdout:Total 22 MB/s | 235 MB 00:10 2024-09-16T15:36:17.550 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-16T15:36:17.690 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-16T15:36:17.690 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-16T15:36:17.732 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-16T15:36:17.732 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-16T15:36:20.139 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-16T15:36:20.139 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-16T15:36:24.807 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-16T15:36:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-16T15:36:25.006 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-16T15:36:25.083 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-16T15:36:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-16T15:36:25.156 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-16T15:36:25.211 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-16T15:36:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-16T15:36:25.739 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:36:25.955 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:36:25.982 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:36:26.130 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:36:26.166 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 10/165 2024-09-16T15:36:26.203 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-16T15:36:26.446 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-16T15:36:26.519 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-16T15:36:26.546 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:36:26.724 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:36:26.755 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 15/165 2024-09-16T15:36:26.887 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 15/165 2024-09-16T15:36:26.890 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-16T15:36:26.914 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T15:36:26.958 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T15:36:27.050 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-16T15:36:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T15:36:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-16T15:36:27.351 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T15:36:27.391 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-16T15:36:27.460 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-16T15:36:27.493 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-16T15:36:27.546 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-16T15:36:27.546 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-16T15:36:27.662 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:36:27.789 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:36:27.914 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-16T15:36:28.027 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:36:28.068 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:36:28.199 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:36:28.228 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:36:28.237 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-16T15:36:28.278 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-16T15:36:28.323 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-16T15:36:28.341 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:36:28.378 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 10/165 2024-09-16T15:36:28.415 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-16T15:36:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-16T15:36:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-16T15:36:28.514 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-16T15:36:28.569 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-16T15:36:28.608 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-16T15:36:28.641 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-16T15:36:28.644 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-16T15:36:28.711 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-16T15:36:28.750 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:36:28.763 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-16T15:36:28.828 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-16T15:36:28.861 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-16T15:36:28.869 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:36:28.901 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 15/165 2024-09-16T15:36:28.908 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-16T15:36:28.974 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 15/165 2024-09-16T15:36:29.002 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T15:36:29.006 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-16T15:36:29.040 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9. 34/165 2024-09-16T15:36:29.047 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T15:36:29.085 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el 35/165 2024-09-16T15:36:29.112 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_6 36/165 2024-09-16T15:36:29.208 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-16T15:36:29.294 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-16T15:36:29.386 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T15:36:29.393 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-16T15:36:29.564 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-16T15:36:29.577 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T15:36:29.635 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-16T15:36:29.759 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-16T15:36:29.883 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:36:30.027 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:36:30.308 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:36:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-16T15:36:30.421 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-16T15:36:30.441 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-16T15:36:30.464 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-16T15:36:30.483 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-16T15:36:30.507 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-16T15:36:30.526 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-16T15:36:30.545 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_6 46/165 2024-09-16T15:36:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-16T15:36:30.614 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-16T15:36:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-16T15:36:30.733 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-16T15:36:30.797 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-16T15:36:30.819 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-16T15:36:30.836 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-16T15:36:30.868 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-16T15:36:30.872 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-16T15:36:30.958 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-16T15:36:30.985 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-16T15:36:31.030 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-16T15:36:31.064 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-16T15:36:31.066 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-16T15:36:31.113 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-16T15:36:31.150 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-16T15:36:31.186 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-16T15:36:31.208 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-16T15:36:31.223 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-16T15:36:31.245 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9. 34/165 2024-09-16T15:36:31.281 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el 35/165 2024-09-16T15:36:31.282 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-16T15:36:31.308 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_6 36/165 2024-09-16T15:36:31.322 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-16T15:36:31.368 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-16T15:36:31.418 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-16T15:36:31.421 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-16T15:36:31.477 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-16T15:36:31.566 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-16T15:36:31.600 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-16T15:36:31.651 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-16T15:36:31.694 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-16T15:36:31.726 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-16T15:36:31.789 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-16T15:36:31.825 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-16T15:36:31.868 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-16T15:36:31.910 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-16T15:36:31.948 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-16T15:36:31.972 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-16T15:36:32.086 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-16T15:36:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-16T15:36:32.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-16T15:36:32.514 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-16T15:36:32.551 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-16T15:36:32.603 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-16T15:36:32.606 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-16T15:36:32.658 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_6 46/165 2024-09-16T15:36:32.670 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-16T15:36:32.710 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-16T15:36:32.751 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-16T15:36:32.802 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-16T15:36:32.842 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-16T15:36:32.878 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-16T15:36:32.894 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-16T15:36:32.913 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-16T15:36:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-16T15:36:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-16T15:36:33.003 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-16T15:36:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-16T15:36:33.111 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-16T15:36:33.124 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-16T15:36:33.182 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-16T15:36:33.209 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-16T15:36:33.217 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-16T15:36:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-16T15:36:33.345 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-16T15:36:33.383 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-16T15:36:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-16T15:36:33.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-16T15:36:33.477 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-16T15:36:33.518 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-16T15:36:33.564 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-16T15:36:33.614 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-16T15:36:33.783 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-16T15:36:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-16T15:36:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-16T15:36:33.990 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-16T15:36:34.031 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-16T15:36:34.081 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-16T15:36:34.112 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-16T15:36:34.136 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-16T15:36:34.247 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-16T15:36:34.363 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-16T15:36:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-16T15:36:34.415 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-16T15:36:34.449 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-16T15:36:34.461 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-16T15:36:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-16T15:36:34.544 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-16T15:36:34.576 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-16T15:36:34.591 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-16T15:36:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-16T15:36:34.656 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-16T15:36:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-16T15:36:34.693 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-16T15:36:34.728 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-16T15:36:34.760 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-16T15:36:34.786 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-16T15:36:34.804 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-16T15:36:34.829 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-16T15:36:34.837 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-16T15:36:34.935 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-16T15:36:34.987 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-16T15:36:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-16T15:36:35.081 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-16T15:36:35.117 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-16T15:36:35.153 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-16T15:36:35.210 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-16T15:36:35.218 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-16T15:36:36.118 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-16T15:36:36.177 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-16T15:36:36.181 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-16T15:36:36.214 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-16T15:36:36.331 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-16T15:36:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-16T15:36:36.397 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-16T15:36:36.450 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-16T15:36:36.546 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-16T15:36:36.604 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-16T15:36:36.842 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-16T15:36:36.872 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-16T15:36:36.964 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-16T15:36:37.745 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-16T15:36:37.884 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-16T15:36:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-16T15:36:38.130 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-16T15:36:38.166 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-16T15:36:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-16T15:36:38.569 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-16T15:36:38.619 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 102/165 2024-09-16T15:36:38.741 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 102/165 2024-09-16T15:36:38.775 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 103/165 2024-09-16T15:36:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-16T15:36:39.392 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-16T15:36:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 106/165 2024-09-16T15:36:39.446 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-16T15:36:39.538 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 106/165 2024-09-16T15:36:39.554 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-16T15:36:39.565 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 107/165 2024-09-16T15:36:39.599 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-16T15:36:39.632 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-16T15:36:39.979 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-16T15:36:40.018 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 102/165 2024-09-16T15:36:40.113 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 102/165 2024-09-16T15:36:40.149 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 103/165 2024-09-16T15:36:40.198 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-16T15:36:40.738 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-16T15:36:40.871 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 106/165 2024-09-16T15:36:40.962 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 106/165 2024-09-16T15:36:40.989 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 107/165 2024-09-16T15:36:42.324 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:36:42.362 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:36:43.040 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:36:43.082 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 109/165 2024-09-16T15:36:43.204 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 109/165 2024-09-16T15:36:43.204 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T15:36:43.204 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T15:36:43.204 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:36:43.250 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 110/165 2024-09-16T15:36:43.537 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:36:43.578 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:36:44.225 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:36:44.264 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 109/165 2024-09-16T15:36:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 109/165 2024-09-16T15:36:44.405 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T15:36:44.405 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T15:36:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:36:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 110/165 2024-09-16T15:36:56.508 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 110/165 2024-09-16T15:36:56.509 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-09-16T15:36:56.509 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-09-16T15:36:56.509 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-09-16T15:36:56.509 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-09-16T15:36:56.509 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-09-16T15:36:56.509 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-09-16T15:36:56.509 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-09-16T15:36:56.509 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:36:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 111/165 2024-09-16T15:36:56.761 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 111/165 2024-09-16T15:36:56.761 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:36:56.761 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T15:36:56.761 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:36:57.126 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 110/165 2024-09-16T15:36:57.126 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-09-16T15:36:57.126 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-09-16T15:36:57.126 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-09-16T15:36:57.127 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-09-16T15:36:57.127 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-09-16T15:36:57.127 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-09-16T15:36:57.127 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-09-16T15:36:57.127 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:36:57.354 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 111/165 2024-09-16T15:36:57.389 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 111/165 2024-09-16T15:36:57.389 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:36:57.389 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T15:36:57.389 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:36:58.457 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 112/165 2024-09-16T15:36:58.516 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 112/165 2024-09-16T15:36:58.516 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:36:58.516 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T15:36:58.516 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T15:36:58.516 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T15:36:58.516 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:36:58.773 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 113/165 2024-09-16T15:36:58.826 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 113/165 2024-09-16T15:36:58.826 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:36:58.826 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T15:36:58.826 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T15:36:58.826 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T15:36:58.827 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:36:58.982 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 112/165 2024-09-16T15:36:59.039 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 112/165 2024-09-16T15:36:59.039 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:36:59.039 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T15:36:59.039 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T15:36:59.039 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T15:36:59.040 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:36:59.282 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 113/165 2024-09-16T15:36:59.332 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 113/165 2024-09-16T15:36:59.332 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:36:59.332 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T15:36:59.332 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T15:36:59.333 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T15:36:59.333 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:36:59.341 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 114/165 2024-09-16T15:36:59.393 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 114/165 2024-09-16T15:36:59.393 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:36:59.393 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T15:36:59.393 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T15:36:59.393 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T15:36:59.393 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:36:59.525 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-16T15:36:59.587 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-16T15:36:59.668 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-16T15:36:59.816 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 114/165 2024-09-16T15:36:59.868 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 114/165 2024-09-16T15:36:59.868 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:36:59.868 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T15:36:59.868 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T15:36:59.868 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T15:36:59.868 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:36:59.905 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-16T15:36:59.963 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-16T15:36:59.991 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-16T15:37:00.051 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-16T15:37:00.052 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-16T15:37:00.114 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-16T15:37:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-16T15:37:00.347 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-16T15:37:00.367 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-16T15:37:00.410 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-16T15:37:00.420 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-16T15:37:00.452 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-16T15:37:00.485 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-16T15:37:00.505 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-16T15:37:00.622 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-16T15:37:00.780 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-16T15:37:00.837 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-16T15:37:00.877 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-16T15:37:00.902 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-16T15:37:01.136 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 126/165 2024-09-16T15:37:01.168 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 126/165 2024-09-16T15:37:01.202 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.e 127/165 2024-09-16T15:37:01.286 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a. 128/165 2024-09-16T15:37:01.315 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 129/165 2024-09-16T15:37:01.610 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 126/165 2024-09-16T15:37:01.643 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 126/165 2024-09-16T15:37:01.685 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.e 127/165 2024-09-16T15:37:01.766 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a. 128/165 2024-09-16T15:37:01.799 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 129/165 2024-09-16T15:37:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 129/165 2024-09-16T15:37:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 130/165 2024-09-16T15:37:05.398 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 129/165 2024-09-16T15:37:05.464 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 130/165 2024-09-16T15:37:06.426 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 130/165 2024-09-16T15:37:06.505 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 131/165 2024-09-16T15:37:06.541 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 130/165 2024-09-16T15:37:06.572 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 131/165 2024-09-16T15:37:06.665 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 131/165 2024-09-16T15:37:06.722 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 131/165 2024-09-16T15:37:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el 132/165 2024-09-16T15:37:06.881 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 133/165 2024-09-16T15:37:06.902 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el 132/165 2024-09-16T15:37:06.933 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 133/165 2024-09-16T15:37:06.934 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:06.934 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T15:37:06.934 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T15:37:06.934 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T15:37:06.934 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:06.941 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 133/165 2024-09-16T15:37:06.992 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 133/165 2024-09-16T15:37:06.992 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:06.992 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T15:37:06.992 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T15:37:06.992 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T15:37:06.992 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:07.000 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 134/165 2024-09-16T15:37:07.026 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 134/165 2024-09-16T15:37:07.050 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 134/165 2024-09-16T15:37:07.076 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 134/165 2024-09-16T15:37:07.137 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 135/165 2024-09-16T15:37:07.183 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 135/165 2024-09-16T15:37:07.202 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 136/165 2024-09-16T15:37:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 136/165 2024-09-16T15:37:08.275 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-16T15:37:08.311 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 138/165 2024-09-16T15:37:08.324 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-16T15:37:08.361 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 138/165 2024-09-16T15:37:08.361 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:08.361 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T15:37:08.361 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:37:08.361 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:37:08.361 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4938.g33f95 139/165 2024-09-16T15:37:08.429 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 138/165 2024-09-16T15:37:08.461 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4938.g33f95 139/165 2024-09-16T15:37:08.461 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:08.461 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T15:37:08.461 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:08.482 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 138/165 2024-09-16T15:37:08.482 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:08.482 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T15:37:08.482 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:37:08.482 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:37:08.482 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:08.520 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 140/165 2024-09-16T15:37:08.527 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4938.g33f95 139/165 2024-09-16T15:37:08.571 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 140/165 2024-09-16T15:37:08.571 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:08.572 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T15:37:08.572 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T15:37:08.572 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T15:37:08.572 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:08.577 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4938.g33f95 139/165 2024-09-16T15:37:08.577 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:08.577 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T15:37:08.577 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:08.661 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 140/165 2024-09-16T15:37:08.713 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 140/165 2024-09-16T15:37:08.713 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:08.713 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T15:37:08.713 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T15:37:08.713 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T15:37:08.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:08.921 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 141/165 2024-09-16T15:37:08.973 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 141/165 2024-09-16T15:37:08.973 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:08.973 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T15:37:08.973 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T15:37:08.973 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T15:37:08.973 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:09.079 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 141/165 2024-09-16T15:37:09.131 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 141/165 2024-09-16T15:37:09.131 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:09.131 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T15:37:09.131 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T15:37:09.131 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T15:37:09.131 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:13.660 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 142/165 2024-09-16T15:37:13.701 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 143/165 2024-09-16T15:37:13.744 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 144/165 2024-09-16T15:37:13.774 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-16T15:37:13.809 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-16T15:37:14.059 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 147/165 2024-09-16T15:37:14.106 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-16T15:37:14.134 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 142/165 2024-09-16T15:37:14.135 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_ 149/165 2024-09-16T15:37:14.167 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-16T15:37:14.184 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 143/165 2024-09-16T15:37:14.242 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-16T15:37:14.245 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 144/165 2024-09-16T15:37:14.281 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-16T15:37:14.317 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-16T15:37:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 147/165 2024-09-16T15:37:14.630 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-16T15:37:14.659 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_ 149/165 2024-09-16T15:37:14.692 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-16T15:37:14.791 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-16T15:37:14.957 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:37:15.009 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:37:15.052 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:37:15.636 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:37:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:37:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:37:16.304 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 153/165 2024-09-16T15:37:16.415 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-16T15:37:16.466 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-16T15:37:16.568 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-16T15:37:16.601 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-16T15:37:16.635 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-16T15:37:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-16T15:37:16.671 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T15:37:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T15:37:16.796 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T15:37:16.904 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T15:37:16.929 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-16T15:37:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:37:16.945 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:37:17.105 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 153/165 2024-09-16T15:37:17.181 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:37:17.181 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T15:37:17.205 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-16T15:37:17.270 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-16T15:37:17.299 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T15:37:17.374 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:37:17.391 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-16T15:37:17.416 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-16T15:37:17.441 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-16T15:37:17.478 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-16T15:37:17.478 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T15:37:17.576 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:37:17.577 INFO:teuthology.orchestra.run.smithi100.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-16T15:37:17.577 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:17.627 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T15:37:17.627 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T15:37:17.744 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T15:37:17.769 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-16T15:37:17.786 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:37:17.786 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:37:18.029 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:37:18.030 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T15:37:18.173 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T15:37:18.289 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:37:18.534 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:37:18.535 INFO:teuthology.orchestra.run.smithi060.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-16T15:37:18.535 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 1/165 2024-09-16T15:37:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 2/165 2024-09-16T15:37:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 3/165 2024-09-16T15:37:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 4/165 2024-09-16T15:37:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4938.g33f95 5/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 6/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 7/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 10/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 11/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 12/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 13/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_ 15/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 16/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 17/165 2024-09-16T15:37:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_6 18/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 19/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 20/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el 21/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9. 22/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_6 23/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 24/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 25/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 26/165 2024-09-16T15:37:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 27/165 2024-09-16T15:37:23.768 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 28/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 29/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a. 30/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 31/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 32/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 33/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el 34/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 35/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.e 36/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 37/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 38/165 2024-09-16T15:37:23.769 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 39/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-16T15:37:23.770 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-16T15:37:23.771 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-16T15:37:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-16T15:37:23.773 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-16T15:37:23.774 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-16T15:37:23.775 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-16T15:37:23.776 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-16T15:37:23.777 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 150/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 152/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-16T15:37:23.778 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-16T15:37:23.779 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-16T15:37:23.779 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-16T15:37:23.779 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-16T15:37:23.779 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-16T15:37:24.551 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 1/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 2/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 3/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 4/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4938.g33f95 5/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 6/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 7/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 10/165 2024-09-16T15:37:24.552 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 11/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 12/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 13/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_ 15/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 16/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 17/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_6 18/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 19/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 20/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el 21/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9. 22/165 2024-09-16T15:37:24.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_6 23/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 24/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 25/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 26/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 27/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 28/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 29/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a. 30/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 31/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 32/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 33/165 2024-09-16T15:37:24.554 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el 34/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 35/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.e 36/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 37/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 38/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 39/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-16T15:37:24.555 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-16T15:37:24.556 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-16T15:37:24.557 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-16T15:37:24.558 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-16T15:37:24.559 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-16T15:37:24.560 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-16T15:37:24.561 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-16T15:37:24.562 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-16T15:37:24.563 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-16T15:37:24.564 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 150/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 152/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-16T15:37:24.565 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-16T15:37:24.566 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-16T15:37:24.566 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-16T15:37:24.566 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-16T15:37:24.566 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-16T15:37:24.566 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-16T15:37:25.947 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:37:25.947 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:25.947 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T15:37:25.948 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T15:37:25.949 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T15:37:25.949 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T15:37:25.949 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.949 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.949 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.949 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.949 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.949 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.950 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.950 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.950 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.950 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.950 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.950 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.950 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.950 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.951 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.951 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.951 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.951 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.951 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.951 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.951 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.951 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T15:37:25.952 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T15:37:25.953 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T15:37:25.954 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T15:37:25.955 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T15:37:25.956 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T15:37:25.957 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T15:37:25.958 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T15:37:25.959 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.960 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.961 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:25.961 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T15:37:25.961 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T15:37:25.961 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T15:37:25.961 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T15:37:25.961 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T15:37:25.961 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T15:37:25.961 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T15:37:25.962 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:25.962 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-16T15:37:26.637 DEBUG:teuthology.parallel:result is None 2024-09-16T15:37:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:37:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:26.794 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-09-16T15:37:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T15:37:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T15:37:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T15:37:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T15:37:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.795 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.796 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T15:37:26.797 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T15:37:26.798 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T15:37:26.799 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.800 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T15:37:26.801 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T15:37:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T15:37:26.804 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.805 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T15:37:26.806 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T15:37:26.807 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:26.808 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-16T15:37:27.569 DEBUG:teuthology.parallel:result is None 2024-09-16T15:37:27.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:37:27.715 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T15:37:27.764 INFO:teuthology.orchestra.run.smithi060.stdout:19.3.0-4938.g33f95d2a.el9 2024-09-16T15:37:27.765 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4938.g33f95d2a.el9 2024-09-16T15:37:27.765 INFO:teuthology.task.install:The correct ceph version 19.3.0-4938.g33f95d2a is installed. 2024-09-16T15:37:27.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:37:27.906 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T15:37:27.952 INFO:teuthology.orchestra.run.smithi100.stdout:19.3.0-4938.g33f95d2a.el9 2024-09-16T15:37:27.952 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4938.g33f95d2a.el9 2024-09-16T15:37:27.952 INFO:teuthology.task.install:The correct ceph version 19.3.0-4938.g33f95d2a is installed. 2024-09-16T15:37:27.954 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T15:37:27.954 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:27.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T15:37:28.017 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:37:28.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T15:37:28.075 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T15:37:28.075 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:28.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T15:37:28.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T15:37:28.232 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:37:28.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T15:37:28.288 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T15:37:28.389 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T15:37:28.390 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:28.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T15:37:28.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T15:37:28.550 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:37:28.550 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T15:37:28.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T15:37:28.701 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T15:37:28.702 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:28.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T15:37:28.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T15:37:28.857 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:37:28.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T15:37:28.914 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T15:37:29.030 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T15:37:29.105 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T15:37:29.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /var/log/ceph 2024-09-16T15:37:29.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-09-16T15:37:29.164 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T15:37:29.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T15:37:29.168 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T15:37:29.223 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T15:37:29.223 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T15:37:29.227 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T15:37:29.289 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-16T15:37:29.289 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-16T15:37:29.289 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-16T15:37:29.289 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:29.289 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T15:37:29.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T15:37:29.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-16T15:37:29.553 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:37:29.553 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T15:37:29.588 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T15:37:29.756 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-16T15:37:29.816 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T15:37:29.816 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T15:37:29.816 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357', 'branch': 'wip-athakkar-testing-2024-09-16-1454', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': 'athakkar@redhat.com', 'first_in_suite': False, 'job_id': '7907357', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-athakkar-testing-2024-09-16-1454'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-athakkar-testing-2024-09-16-1454', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}}, 'owner': 'scheduled_avan@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3077, 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'sleep_before_teardown': 0, 'subset': '1/20', 'suite': 'fs:volumes', 'suite_branch': 'wip-athakkar-testing-2024-09-16-1454', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_33f95d2ae1bcbe7eb04404303f179eeeaa6982df/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/Om5s4i2MbBgbwdRkb5h7rH8nEevdNdrHPfUh/1RkSWpY66oh6pLq9m2MCKnGik0MOTd15HdV0VZDH0MIlU+A=', 'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNv9PK5xpaToQFRLTEv8GBPuuwtvO3nr3fXTNyciz2ntgM8zt86gJ1xlvMSbGAE5gH+h0fo9Ok1V2+/HbcvObZQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_15:21:40', 'tube': 'smithi', 'user': 'avan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T15:37:29.817 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T15:37:29.821 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T15:37:29.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T15:37:29.896 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T15:37:29.951 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:29.951 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T15:37:29.987 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T15:37:29.987 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-09-16T15:37:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T15:37:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:37:30.062 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-16T15:37:30.062 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:37:30.062 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:37:30.062 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-16 15:37:18.895906085 +0000 2024-09-16T15:37:30.062 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-16 15:35:15.435946924 +0000 2024-09-16T15:37:30.063 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-16 15:35:15.435946924 +0000 2024-09-16T15:37:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-16 15:35:15.435946924 +0000 2024-09-16T15:37:30.063 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T15:37:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:37:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:37:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000250349 s, 2.0 MB/s 2024-09-16T15:37:30.157 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T15:37:30.194 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-16 15:37:18.895906085 +0000 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-16 15:35:15.905942439 +0000 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-16 15:35:15.905942439 +0000 2024-09-16T15:37:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-16 15:35:15.905942439 +0000 2024-09-16T15:37:30.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T15:37:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:37:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:37:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000180046 s, 2.8 MB/s 2024-09-16T15:37:30.364 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T15:37:30.401 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-16 15:37:18.896906076 +0000 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-16 15:35:16.357938126 +0000 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-16 15:35:16.357938126 +0000 2024-09-16T15:37:30.476 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-16 15:35:16.357938126 +0000 2024-09-16T15:37:30.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T15:37:30.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:37:30.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:37:30.568 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000175401 s, 2.9 MB/s 2024-09-16T15:37:30.571 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T15:37:30.607 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-16 15:37:18.897906066 +0000 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-16 15:35:16.845933469 +0000 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-16 15:35:16.845933469 +0000 2024-09-16T15:37:30.684 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-16 15:35:16.845933469 +0000 2024-09-16T15:37:30.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T15:37:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:37:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:37:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000175277 s, 2.9 MB/s 2024-09-16T15:37:30.779 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T15:37:30.816 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:30.816 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:37:30.816 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T15:37:30.853 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T15:37:30.854 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-09-16T15:37:30.929 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T15:37:30.929 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:37:30.929 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 637 Links: 1 2024-09-16T15:37:30.929 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:37:30.930 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:37:30.930 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-16 15:37:17.964014017 +0000 2024-09-16T15:37:30.930 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-16 15:35:07.875931974 +0000 2024-09-16T15:37:30.930 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-16 15:35:07.875931974 +0000 2024-09-16T15:37:30.930 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-16 15:35:07.875931974 +0000 2024-09-16T15:37:30.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T15:37:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-16T15:37:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-16T15:37:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000259485 s, 2.0 MB/s 2024-09-16T15:37:31.025 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T15:37:31.061 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-09-16T15:37:31.136 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T15:37:31.136 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:37:31.136 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 656 Links: 1 2024-09-16T15:37:31.136 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:37:31.137 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:37:31.137 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-16 15:37:17.965013992 +0000 2024-09-16T15:37:31.137 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-16 15:35:08.353919897 +0000 2024-09-16T15:37:31.137 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-16 15:35:08.353919897 +0000 2024-09-16T15:37:31.137 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-16 15:35:08.353919897 +0000 2024-09-16T15:37:31.137 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T15:37:31.228 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-16T15:37:31.229 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-16T15:37:31.229 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000298255 s, 1.7 MB/s 2024-09-16T15:37:31.231 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T15:37:31.267 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 675 Links: 1 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-16 15:37:17.966013966 +0000 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-16 15:35:08.847907416 +0000 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-16 15:35:08.847907416 +0000 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-16 15:35:08.847907416 +0000 2024-09-16T15:37:31.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T15:37:31.435 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-16T15:37:31.435 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-16T15:37:31.435 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000318467 s, 1.6 MB/s 2024-09-16T15:37:31.438 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T15:37:31.474 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-09-16T15:37:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T15:37:31.549 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:37:31.549 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 695 Links: 1 2024-09-16T15:37:31.550 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:37:31.550 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:37:31.550 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-16 15:37:17.966013966 +0000 2024-09-16T15:37:31.550 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-16 15:35:09.296896072 +0000 2024-09-16T15:37:31.550 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-16 15:35:09.296896072 +0000 2024-09-16T15:37:31.550 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-16 15:35:09.296896072 +0000 2024-09-16T15:37:31.550 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T15:37:31.641 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-16T15:37:31.641 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-16T15:37:31.641 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000318628 s, 1.6 MB/s 2024-09-16T15:37:31.643 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T15:37:31.679 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:31.679 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T15:37:31.679 INFO:tasks.ceph:Generating config... 2024-09-16T15:37:31.680 INFO:tasks.ceph:[client] pid file = 2024-09-16T15:37:31.680 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-16T15:37:31.680 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-16T15:37:31.680 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-16T15:37:31.681 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-16T15:37:31.681 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T15:37:31.681 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T15:37:31.681 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T15:37:31.681 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T15:37:31.681 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T15:37:31.681 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-16T15:37:31.681 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T15:37:31.681 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T15:37:31.681 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-16T15:37:31.681 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T15:37:31.681 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T15:37:31.681 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T15:37:31.681 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T15:37:31.681 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T15:37:31.682 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T15:37:31.682 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T15:37:31.682 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T15:37:31.682 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T15:37:31.682 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T15:37:31.682 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T15:37:31.682 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T15:37:31.682 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T15:37:31.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T15:37:31.793 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T15:37:31.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T15:37:31.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:37:31.944 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.60'), ('mon.b', '172.21.15.100'), ('mon.c', '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]')] 2024-09-16T15:37:31.944 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-16T15:37:31.945 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:31.945 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T15:37:31.980 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.60 --add b 172.21.15.100 --addv c '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: generated fsid b577f92e-7c09-4d32-a34c-b870818f4d85 2024-09-16T15:37:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-09-16T15:37:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 0 2024-09-16T15:37:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:fsid b577f92e-7c09-4d32-a34c-b870818f4d85 2024-09-16T15:37:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:last_changed 2024-09-16T15:37:32.097839+0000 2024-09-16T15:37:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-09-16T15:37:32.097839+0000 2024-09-16T15:37:32.098 INFO:teuthology.orchestra.run.smithi060.stdout:min_mon_release 17 (quincy) 2024-09-16T15:37:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:election_strategy: 1 2024-09-16T15:37:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-16T15:37:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2024-09-16T15:37:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-09-16T15:37:32.099 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T15:37:32.101 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T15:37:32.135 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b577f92e-7c09-4d32-a34c-b870818f4d85... 2024-09-16T15:37:32.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T15:37:32.179 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T15:37:32.273 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-09-16T15:37:32.273 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-09-16T15:37:32.273 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T15:37:32.273 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-09-16T15:37:32.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.273 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T15:37:32.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T15:37:32.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on skipped message = False 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = b577f92e-7c09-4d32-a34c-b870818f4d85 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-09-16T15:37:32.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-09-16T15:37:32.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-09-16T15:37:32.278 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-09-16T15:37:32.279 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 30 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: debug finisher = 20 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-09-16T15:37:32.280 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T15:37:32.281 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: mon down mkfs grace = 300 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-09-16T15:37:32.282 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-09-16T15:37:32.283 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T15:37:32.284 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-09-16T15:37:32.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-09-16T15:37:32.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on skipped message = False 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = b577f92e-7c09-4d32-a34c-b870818f4d85 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.288 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-16T15:37:32.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:37:32.290 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 30 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: debug finisher = 20 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-09-16T15:37:32.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: mon down mkfs grace = 300 2024-09-16T15:37:32.292 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-09-16T15:37:32.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-09-16T15:37:32.302 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T15:37:32.302 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T15:37:32.401 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T15:37:32.401 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:32.401 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T15:37:32.436 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:32.436 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T15:37:32.510 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:37:32.510 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:32.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T15:37:32.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:37:32.629 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:37:32.629 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:32.704 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:37:32.704 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:37:32.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T15:37:32.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:37:32.783 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:37:32.783 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:32.858 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T15:37:32.858 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T15:37:32.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T15:37:32.964 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T15:37:32.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T15:37:33.088 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T15:37:33.095 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T15:37:33.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T15:37:33.200 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T15:37:33.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-16T15:37:33.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T15:37:33.365 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T15:37:33.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-16T15:37:33.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T15:37:33.530 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T15:37:33.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-16T15:37:33.590 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-16T15:37:33.694 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-16T15:37:33.699 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-16T15:37:33.754 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T15:37:33.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T15:37:33.837 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T15:37:33.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-16T15:37:33.948 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-16T15:37:33.973 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T15:37:34.055 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T15:37:34.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-16T15:37:34.162 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-16T15:37:34.189 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T15:37:34.190 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T15:37:34.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T15:37:34.247 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:34.247 INFO:tasks.ceph:role: osd.0 2024-09-16T15:37:34.247 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:37:34.247 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T15:37:34.302 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:37:34.302 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:37:34.302 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:37:34.302 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:37:34.302 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:37:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:37:34.303 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:37:34.303 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:37:34.303 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:37:34.303 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:37:34.336 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-16T15:37:34.339 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-16T15:37:34.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T15:37:34.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T15:37:34.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T15:37:34.541 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:34.541 INFO:tasks.ceph:role: osd.1 2024-09-16T15:37:34.541 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:37:34.541 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T15:37:34.596 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:37:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:37:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:37:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:37:34.596 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:37:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:37:34.597 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:37:34.597 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:37:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:37:34.597 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:37:34.630 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-16T15:37:34.634 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-16T15:37:34.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T15:37:34.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T15:37:34.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T15:37:34.832 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:34.832 INFO:tasks.ceph:role: osd.2 2024-09-16T15:37:34.832 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:37:34.832 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T15:37:34.887 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:37:34.887 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:37:34.888 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:37:34.888 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:37:34.888 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:37:34.888 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:37:34.888 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:37:34.888 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:37:34.888 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:37:34.888 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:37:34.921 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-16T15:37:34.924 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-16T15:37:34.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T15:37:35.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T15:37:35.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T15:37:35.121 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:35.121 INFO:tasks.ceph:role: osd.3 2024-09-16T15:37:35.121 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:37:35.121 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:37:35.177 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:37:35.178 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:37:35.212 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-16T15:37:35.215 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-16T15:37:35.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T15:37:35.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T15:37:35.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:35.453+0000 7f8fd9033640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T15:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:35.454+0000 7f8fd9033640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T15:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:35.454+0000 7f8fd9033640 -1 bdev(0x55df661c9800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-16T15:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:35.454+0000 7f8fd9033640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T15:37:38.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T15:37:38.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:38.647+0000 7fb889213640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T15:37:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:38.648+0000 7fb889213640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T15:37:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:38.648+0000 7fb889213640 -1 bdev(0x56497ec95800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-16T15:37:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:38.648+0000 7fb889213640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T15:37:42.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T15:37:42.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:42.319+0000 7f6ee6213640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T15:37:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:42.320+0000 7f6ee6213640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T15:37:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:42.320+0000 7f6ee6213640 -1 bdev(0x55901dcf1800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-16T15:37:42.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:42.320+0000 7f6ee6213640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T15:37:45.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T15:37:45.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:45.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:45.468+0000 7f286fbb2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T15:37:45.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:45.468+0000 7f286fbb2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T15:37:45.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:45.468+0000 7f286fbb2640 -1 bdev(0x559e6447b800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-16T15:37:45.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:37:45.469+0000 7f286fbb2640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T15:37:48.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T15:37:48.550 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T15:37:48.610 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:48.610 INFO:tasks.ceph:role: osd.4 2024-09-16T15:37:48.610 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:37:48.610 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T15:37:48.675 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:37:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:37:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:37:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:37:48.675 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:37:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:37:48.675 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:37:48.675 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:37:48.676 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:37:48.676 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:37:49.081 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-16T15:37:49.089 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-16T15:37:49.089 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T15:37:49.207 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T15:37:49.269 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T15:37:49.323 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:49.324 INFO:tasks.ceph:role: osd.5 2024-09-16T15:37:49.324 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:37:49.324 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T15:37:49.402 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:37:49.403 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:37:49.759 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-16T15:37:49.763 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-16T15:37:49.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T15:37:49.847 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T15:37:49.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T15:37:49.964 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:49.964 INFO:tasks.ceph:role: osd.6 2024-09-16T15:37:49.964 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:37:49.965 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:37:50.028 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:37:50.408 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-16T15:37:50.412 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-16T15:37:50.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T15:37:50.516 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T15:37:50.580 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T15:37:50.637 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:37:50.637 INFO:tasks.ceph:role: osd.7 2024-09-16T15:37:50.638 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:37:50.638 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T15:37:50.703 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:37:50.703 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:37:50.703 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:37:50.704 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:37:50.704 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:37:50.704 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:37:50.704 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:37:50.704 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:37:50.704 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:37:50.704 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:37:51.093 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-16T15:37:51.097 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-16T15:37:51.097 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T15:37:51.212 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T15:37:51.275 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:51.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:51.372+0000 7f94348b0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T15:37:51.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:51.374+0000 7f94348b0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T15:37:51.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:51.374+0000 7f94348b0640 -1 bdev(0x55d5df959800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-16T15:37:51.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:51.374+0000 7f94348b0640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T15:37:54.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T15:37:54.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:54.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:54.801+0000 7fb25f743640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T15:37:54.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:54.801+0000 7fb25f743640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T15:37:54.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:54.801+0000 7fb25f743640 -1 bdev(0x5606b5feb800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-16T15:37:54.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:54.802+0000 7fb25f743640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T15:37:57.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T15:37:57.864 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:37:57.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:57.953+0000 7f6d49064640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T15:37:57.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:57.953+0000 7f6d49064640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T15:37:57.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:57.953+0000 7f6d49064640 -1 bdev(0x55979048b800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-16T15:37:57.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:37:57.953+0000 7f6d49064640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T15:37:59.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:37:59.823 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:00.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T15:38:01.041 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:01.130+0000 7f7c3423b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T15:38:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:01.131+0000 7f7c3423b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T15:38:01.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:01.131+0000 7f7c3423b640 -1 bdev(0x555db8d3d800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:01.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:01.131+0000 7f7c3423b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T15:38:04.146 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T15:38:04.202 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T15:38:04.202 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:04.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T15:38:04.265 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:04.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-16T15:38:04.364 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:04.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-16T15:38:04.462 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:04.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T15:38:04.558 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:04.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T15:38:04.654 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:04.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T15:38:04.749 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:04.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T15:38:04.844 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:04.844 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T15:38:04.904 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:04.904 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-16T15:38:05.001 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:05.001 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-16T15:38:05.097 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:05.097 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T15:38:05.192 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:05.192 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T15:38:05.288 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:05.288 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T15:38:05.383 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:05.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T15:38:05.479 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:05.479 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T15:38:05.516 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:05.516 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-16T15:38:05.594 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:05.594 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T15:38:05.629 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:05.629 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-16T15:38:05.703 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T15:38:05.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T15:38:05.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T15:38:05.758 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.y] 2024-09-16T15:38:05.758 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw== 2024-09-16T15:38:05.758 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-09-16T15:38:05.758 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw== 2024-09-16T15:38:05.758 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-09-16T15:38:05.758 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA== 2024-09-16T15:38:05.758 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.0] 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ== 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.1] 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ== 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.2] 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ== 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.3] 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw== 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.x] 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ== 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA== 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw== 2024-09-16T15:38:05.759 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.4] 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g== 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.5] 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ== 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.6] 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ== 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.7] 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ== 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout:[client.2] 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA== 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout:[client.1] 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw== 2024-09-16T15:38:05.760 INFO:teuthology.orchestra.run.smithi060.stdout:[client.3] 2024-09-16T15:38:05.761 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw== 2024-09-16T15:38:05.794 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw== 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw== 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA== 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ== 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ== 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ== 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw== 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-09-16T15:38:05.795 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ== 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA== 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw== 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g== 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ== 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ== 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ== 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-16T15:38:05.796 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:38:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2024-09-16T15:38:05.797 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA== 2024-09-16T15:38:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2024-09-16T15:38:05.797 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw== 2024-09-16T15:38:05.797 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2024-09-16T15:38:05.797 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw== 2024-09-16T15:38:05.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:38:05.806 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:38:05.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:05.936 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:06.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:06.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:06.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.136 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.233 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.428 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:38:06.525 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:38:06.618 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:06.622 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:06.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:06.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:06.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.821 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:06.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:07.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:07.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:07.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:07.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:07.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:07.211 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:07.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:07.312 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:07.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:07.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:07.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:07.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:07.608 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T15:38:07.609 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T15:38:07.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:38:07.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T15:38:07.828 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T15:38:07.883 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:38:07.992 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T15:38:08.048 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T15:38:08.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:38:08.210 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T15:38:08.266 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:08.271 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:08.304 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T15:38:08.305 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T15:38:08.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T15:38:08.346 INFO:tasks.ceph.mon.a:Started 2024-09-16T15:38:08.347 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T15:38:08.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T15:38:08.350 INFO:tasks.ceph.mon.b:Started 2024-09-16T15:38:08.351 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T15:38:08.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T15:38:08.355 INFO:tasks.ceph.mon.c:Started 2024-09-16T15:38:08.356 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T15:38:08.356 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T15:38:08.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T15:38:08.359 INFO:tasks.ceph.mgr.y:Started 2024-09-16T15:38:08.360 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T15:38:08.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T15:38:08.363 INFO:tasks.ceph.mgr.x:Started 2024-09-16T15:38:08.363 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-16T15:38:08.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-16T15:38:08.368 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-16T15:38:08.368 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-16T15:38:08.414 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-16T15:38:08.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-16T15:38:08.460 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-16T15:38:08.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-16T15:38:08.508 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-16T15:38:08.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-16T15:38:08.555 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-16T15:38:08.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-16T15:38:08.601 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-16T15:38:08.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-16T15:38:08.648 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-16T15:38:08.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-16T15:38:08.697 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-16T15:38:08.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-16T15:38:08.708 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-16T15:38:08.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-16T15:38:08.758 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-16T15:38:08.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-16T15:38:08.811 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-16T15:38:08.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-16T15:38:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.842+0000 7fdc78a00640 1 -- 172.21.15.60:0/4075324203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc740cb4a0 msgr2=0x7fdc741500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:08.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.842+0000 7fdc78a00640 1 --2- 172.21.15.60:0/4075324203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc740cb4a0 0x7fdc741500a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400d750 tx=0x7fdc6400dc10 comp rx=0 tx=0).stop 2024-09-16T15:38:08.855 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-16T15:38:08.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-16T15:38:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.843+0000 7fdc78a00640 1 -- 172.21.15.60:0/4075324203 shutdown_connections 2024-09-16T15:38:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.843+0000 7fdc78a00640 1 --2- 172.21.15.60:0/4075324203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc7415eba0 0x7fdc74150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.843+0000 7fdc78a00640 1 --2- 172.21.15.60:0/4075324203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc740cbf90 0x7fdc740c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.843+0000 7fdc78a00640 1 --2- 172.21.15.60:0/4075324203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc740cb4a0 0x7fdc741500a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.843+0000 7fdc78a00640 1 -- 172.21.15.60:0/4075324203 >> 172.21.15.60:0/4075324203 conn(0x7fdc740d0b20 msgr2=0x7fdc740b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.847+0000 7fdc78a00640 1 -- 172.21.15.60:0/4075324203 shutdown_connections 2024-09-16T15:38:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.847+0000 7fdc78a00640 1 -- 172.21.15.60:0/4075324203 wait complete. 2024-09-16T15:38:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.845+0000 7f18c0e00640 1 -- 172.21.15.60:0/1371574342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1894005080 con 0x7f18bc15a520 2024-09-16T15:38:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.847+0000 7f18b2000640 1 -- 172.21.15.60:0/1371574342 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f18b402bc10 con 0x7f18bc15a520 2024-09-16T15:38:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.847+0000 7f18b2000640 1 -- 172.21.15.60:0/1371574342 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18b402c300 con 0x7f18bc15a520 2024-09-16T15:38:08.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.849+0000 7f18b2000640 1 -- 172.21.15.60:0/1371574342 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f18b404c020 con 0x7f18bc15a520 2024-09-16T15:38:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.879+0000 7f8f63600640 1 -- 172.21.15.60:0/3017999082 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f8f5c0b9f30 con 0x7f8f5c161590 2024-09-16T15:38:08.889 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-16T15:38:08.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-16T15:38:08.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.891+0000 7f8f58c00640 1 -- 172.21.15.60:0/3017999082 <== mon.2 v2:172.21.15.100:3301/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f8f480039b0 con 0x7f8f5c161590 2024-09-16T15:38:08.900 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-16T15:38:08.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-16T15:38:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.891+0000 7f8f58c00640 1 -- 172.21.15.60:0/3017999082 <== mon.2 v2:172.21.15.100:3301/0 9 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7f8f48003c20 con 0x7f8f5c161590 2024-09-16T15:38:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.893+0000 7f8f63600640 1 -- 172.21.15.60:0/3017999082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f5c161590 msgr2=0x7f8f5c204920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.893+0000 7f8f63600640 1 --2- 172.21.15.60:0/3017999082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f5c161590 0x7f8f5c204920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009950 tx=0x7f8f480046c0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.899+0000 7f8f63600640 1 -- 172.21.15.60:0/3017999082 shutdown_connections 2024-09-16T15:38:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.899+0000 7f8f63600640 1 --2- 172.21.15.60:0/3017999082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f5c161590 0x7f8f5c204920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.899+0000 7f8f63600640 1 --2- 172.21.15.60:0/3017999082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f5c0cfaf0 0x7f8f5c200ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.899+0000 7f8f63600640 1 --2- 172.21.15.60:0/3017999082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f5c0b5630 0x7f8f5c15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.899+0000 7f8f63600640 1 -- 172.21.15.60:0/3017999082 >> 172.21.15.60:0/3017999082 conn(0x7f8f5c0d3fb0 msgr2=0x7f8f5c0d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.891+0000 7f18b2000640 1 -- 172.21.15.60:0/1371574342 <== mon.1 v2:172.21.15.100:3300/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f18b402c770 con 0x7f18bc15a520 2024-09-16T15:38:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.899+0000 7f8f63600640 1 -- 172.21.15.60:0/3017999082 shutdown_connections 2024-09-16T15:38:08.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.899+0000 7f8f63600640 1 -- 172.21.15.60:0/3017999082 wait complete. 2024-09-16T15:38:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.903+0000 7fe1d4a00640 1 -- 172.21.15.60:0/3748343836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d00cbdd0 msgr2=0x7fe1d0070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:08.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.903+0000 7fe1d4a00640 1 --2- 172.21.15.60:0/3748343836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d00cbdd0 0x7fe1d0070090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009830 tx=0x7fe1bc02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.914 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-16T15:38:08.914 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-16T15:38:08.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.904+0000 7fe1d4a00640 1 -- 172.21.15.60:0/3748343836 shutdown_connections 2024-09-16T15:38:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.904+0000 7fe1d4a00640 1 --2- 172.21.15.60:0/3748343836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d00cbdd0 0x7fe1d0070090 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.904+0000 7fe1d4a00640 1 -- 172.21.15.60:0/3748343836 >> 172.21.15.60:0/3748343836 conn(0x7fe1d00cf9f0 msgr2=0x7fe1d00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.907+0000 7fe1d4a00640 1 -- 172.21.15.60:0/3748343836 shutdown_connections 2024-09-16T15:38:08.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.908+0000 7fe1d4a00640 1 -- 172.21.15.60:0/3748343836 wait complete. 2024-09-16T15:38:08.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.911+0000 7fe1d4a00640 1 Processor -- start 2024-09-16T15:38:08.934 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-16T15:38:08.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-16T15:38:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.919+0000 7fe1d4a00640 1 -- start start 2024-09-16T15:38:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.928+0000 7fe1d4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d006ef60 0x7fe1d01f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:08.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.928+0000 7fe1d4a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1d00cbdd0 0x7fe1d00b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.928+0000 7fe1d4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d0161560 0x7fe1d00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.929+0000 7fe1cfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d0161560 0x7fe1d00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.929+0000 7fe1d4a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00aea70 con 0x7fe1d0161560 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.929+0000 7fe1cfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d0161560 0x7fe1d00ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53960/0 (socket says 172.21.15.60:53960) 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.929+0000 7fe1cfe00640 1 -- 172.21.15.60:0/4028948831 learned_addr learned my addr 172.21.15.60:0/4028948831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.931+0000 7fe1cea00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1d00cbdd0 0x7fe1d00b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.932+0000 7fe1cf400640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d006ef60 0x7fe1d01f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.933+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00ae8f0 con 0x7fe1d006ef60 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.933+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d00aebf0 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.934+0000 7fe1cea00640 1 -- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d006ef60 msgr2=0x7fe1d01f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:08.951 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-16T15:38:08.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.938+0000 7fe1cea00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d006ef60 0x7fe1d01f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.938+0000 7fe1cea00640 1 -- 172.21.15.60:0/4028948831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d0161560 msgr2=0x7fe1d00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.938+0000 7fe1cea00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d0161560 0x7fe1d00ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.938+0000 7fe1cea00640 1 -- 172.21.15.60:0/4028948831 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d00b45c0 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.938+0000 7fe1cea00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1d00cbdd0 0x7fe1d00b5360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc031000 tx=0x7fe1bc0329e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.939+0000 7fe1c7400640 1 -- 172.21.15.60:0/4028948831 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc040070 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.939+0000 7fe1c7400640 1 -- 172.21.15.60:0/4028948831 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fe1bc0052f0 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.939+0000 7fe1c7400640 1 -- 172.21.15.60:0/4028948831 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0047a0 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.939+0000 7fe1cfe00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d0161560 0x7fe1d00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.939+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d01f5a80 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.939+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d00b4b60 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.946+0000 7fe1c7400640 1 -- 172.21.15.60:0/4028948831 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1bc039040 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.946+0000 7fe1c7400640 1 -- 172.21.15.60:0/4028948831 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1bc031d30 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.947+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d00b01a0 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.947+0000 7fe1cf400640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d006ef60 0x7fe1d01f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.950+0000 7fe1c7400640 1 -- 172.21.15.60:0/4028948831 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe1bc049060 con 0x7fe1d00cbdd0 2024-09-16T15:38:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.985+0000 7fd35f800640 1 -- 172.21.15.60:0/3404863460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 msgr2=0x7fd35815fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.985+0000 7fd35f800640 1 --2- 172.21.15.60:0/3404863460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 0x7fd35815fce0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd348009830 tx=0x7fd34803b540 comp rx=0 tx=0).stop 2024-09-16T15:38:09.000 DEBUG:tasks.ceph:set 18 configs 2024-09-16T15:38:09.000 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.001 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.001 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.001 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.001 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.995+0000 7fd35f800640 1 -- 172.21.15.60:0/3404863460 shutdown_connections 2024-09-16T15:38:09.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.995+0000 7fd35f800640 1 --2- 172.21.15.60:0/3404863460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 0x7fd35815fce0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.995+0000 7fd35f800640 1 -- 172.21.15.60:0/3404863460 >> 172.21.15.60:0/3404863460 conn(0x7fd3580b1840 msgr2=0x7fd3580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.999+0000 7fd35f800640 1 -- 172.21.15.60:0/3404863460 shutdown_connections 2024-09-16T15:38:09.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:08.999+0000 7fd35f800640 1 -- 172.21.15.60:0/3404863460 wait complete. 2024-09-16T15:38:09.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.002+0000 7fd35f800640 1 Processor -- start 2024-09-16T15:38:09.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.010+0000 7fd35f800640 1 -- start start 2024-09-16T15:38:09.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.011+0000 7fd35f800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35806fba0 0x7fd35814c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.011+0000 7fd35f800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3580af170 0x7fd358158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.011+0000 7fd35f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 0x7fd35814d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.019+0000 7fd35f800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd35806ff80 con 0x7fd35806fba0 2024-09-16T15:38:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.019+0000 7fd35f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580cf0d0 con 0x7fd3580b2b40 2024-09-16T15:38:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.019+0000 7fd35f800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd358070100 con 0x7fd3580af170 2024-09-16T15:38:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.019+0000 7fd35d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 0x7fd35814d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.019+0000 7fd35d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 0x7fd35814d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49598/0 (socket says 172.21.15.60:49598) 2024-09-16T15:38:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.019+0000 7fd35d800640 1 -- 172.21.15.60:0/769510775 learned_addr learned my addr 172.21.15.60:0/769510775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.019+0000 7fd357e00640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3580af170 0x7fd358158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.023+0000 7fd35d800640 1 -- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3580af170 msgr2=0x7fd358158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.023+0000 7fd35d800640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3580af170 0x7fd358158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.023+0000 7fd35d800640 1 -- 172.21.15.60:0/769510775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35806fba0 msgr2=0x7fd35814c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.023+0000 7fd35ce00640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35806fba0 0x7fd35814c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.023+0000 7fd35d800640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35806fba0 0x7fd35814c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.023+0000 7fd35d800640 1 -- 172.21.15.60:0/769510775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35815a270 con 0x7fd3580b2b40 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.023+0000 7fd357e00640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3580af170 0x7fd358158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.023+0000 7fd35ce00640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35806fba0 0x7fd35814c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.027+0000 7fd35d800640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 0x7fd35814d760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd34803bb30 tx=0x7fd348043f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.038+0000 7fd354c00640 1 -- 172.21.15.60:0/769510775 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348051070 con 0x7fd3580b2b40 2024-09-16T15:38:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.038+0000 7fd354c00640 1 -- 172.21.15.60:0/769510775 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fd348003800 con 0x7fd3580b2b40 2024-09-16T15:38:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.038+0000 7fd354c00640 1 -- 172.21.15.60:0/769510775 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348003c50 con 0x7fd3580b2b40 2024-09-16T15:38:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.038+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd358158a70 con 0x7fd3580b2b40 2024-09-16T15:38:09.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.038+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd358157a50 con 0x7fd3580b2b40 2024-09-16T15:38:09.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.044+0000 7fd354c00640 1 -- 172.21.15.60:0/769510775 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd34804a040 con 0x7fd3580b2b40 2024-09-16T15:38:09.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.044+0000 7fd354c00640 1 -- 172.21.15.60:0/769510775 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd348043020 con 0x7fd3580b2b40 2024-09-16T15:38:09.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.045+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd330005290 con 0x7fd3580b2b40 2024-09-16T15:38:09.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.050+0000 7fd354c00640 1 -- 172.21.15.60:0/769510775 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd34805b060 con 0x7fd3580b2b40 2024-09-16T15:38:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.058+0000 7f9cbfe00640 1 -- 172.21.15.60:0/582451984 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8007170 msgr2=0x7f9cb80512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.058+0000 7f9cbfe00640 1 --2- 172.21.15.60:0/582451984 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8007170 0x7f9cb80512f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800ea80 tx=0x7f9cac02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.062+0000 7f9cbfe00640 1 -- 172.21.15.60:0/582451984 shutdown_connections 2024-09-16T15:38:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.062+0000 7f9cbfe00640 1 --2- 172.21.15.60:0/582451984 >> v2:172.21.15.60:3300/0 conn(0x7f9cb8051830 0x7f9cb8039490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.062+0000 7f9cbfe00640 1 --2- 172.21.15.60:0/582451984 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8007170 0x7f9cb80512f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.062+0000 7f9cbfe00640 1 --2- 172.21.15.60:0/582451984 >> v2:172.21.15.100:3300/0 conn(0x7f9cb80de9c0 0x7f9cb8050db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.062+0000 7f9cbfe00640 1 -- 172.21.15.60:0/582451984 >> 172.21.15.60:0/582451984 conn(0x7f9cb801d950 msgr2=0x7f9cb8054740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.072+0000 7f18c0e00640 1 -- 172.21.15.60:0/1371574342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f1894005370 con 0x7f18bc15a520 2024-09-16T15:38:09.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.070+0000 7f9cbfe00640 1 -- 172.21.15.60:0/582451984 shutdown_connections 2024-09-16T15:38:09.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.070+0000 7f9cbfe00640 1 -- 172.21.15.60:0/582451984 wait complete. 2024-09-16T15:38:09.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.073+0000 7f9cbfe00640 1 Processor -- start 2024-09-16T15:38:09.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.073+0000 7f9cbfe00640 1 -- start start 2024-09-16T15:38:09.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.076+0000 7f9cbfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb8007170 0x7f9cb80df250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.077+0000 7f9cbfe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8051830 0x7f9cb81791a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.080+0000 7f18b2000640 1 -- 172.21.15.60:0/1371574342 <== mon.1 v2:172.21.15.100:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18b4033640 con 0x7f18bc15a520 2024-09-16T15:38:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.077+0000 7f9cbe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb8007170 0x7f9cb80df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.077+0000 7f9cbe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb8007170 0x7f9cb80df250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49610/0 (socket says 172.21.15.60:49610) 2024-09-16T15:38:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.080+0000 7f9cbe000640 1 -- 172.21.15.60:0/2390674615 learned_addr learned my addr 172.21.15.60:0/2390674615 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.080+0000 7f9cbfe00640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb80de9c0 0x7f9cb80dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.081+0000 7f9cbfe00640 1 -- 172.21.15.60:0/2390674615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb80def20 con 0x7f9cb80de9c0 2024-09-16T15:38:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.081+0000 7f9cbfe00640 1 -- 172.21.15.60:0/2390674615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb80deda0 con 0x7f9cb8007170 2024-09-16T15:38:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.081+0000 7f9cbd600640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8051830 0x7f9cb81791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.081+0000 7f9cbfe00640 1 -- 172.21.15.60:0/2390674615 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb80df0a0 con 0x7f9cb8051830 2024-09-16T15:38:09.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.082+0000 7f18b2000640 1 -- 172.21.15.60:0/1371574342 <== mon.1 v2:172.21.15.100:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f18b40338e0 con 0x7f18bc15a520 2024-09-16T15:38:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.084+0000 7fe1c7400640 1 -- 172.21.15.60:0/4028948831 <== mon.2 v2:172.21.15.100:3301/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1bc004e60 con 0x7fe1d00cbdd0 2024-09-16T15:38:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.082+0000 7f9cbe000640 1 -- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8051830 msgr2=0x7f9cb81791a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.082+0000 7f9cbe000640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8051830 0x7f9cb81791a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.082+0000 7f9cbe000640 1 -- 172.21.15.60:0/2390674615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb80de9c0 msgr2=0x7f9cb80dbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.082+0000 7f9cbe000640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb80de9c0 0x7f9cb80dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.083+0000 7f9cbe000640 1 -- 172.21.15.60:0/2390674615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb80e0470 con 0x7f9cb8007170 2024-09-16T15:38:09.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.085+0000 7f9cbd600640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8051830 0x7f9cb81791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.085+0000 7f9cbe000640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb8007170 0x7f9cb80df250 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000e9f0 tx=0x7f9cb000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.088+0000 7f9cb6a00640 1 -- 172.21.15.60:0/2390674615 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000cd80 con 0x7f9cb8007170 2024-09-16T15:38:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.088+0000 7f9cb6a00640 1 -- 172.21.15.60:0/2390674615 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cb0004020 con 0x7f9cb8007170 2024-09-16T15:38:09.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.088+0000 7f9cb6a00640 1 -- 172.21.15.60:0/2390674615 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0004450 con 0x7f9cb8007170 2024-09-16T15:38:09.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.088+0000 7f9cbfe00640 1 -- 172.21.15.60:0/2390674615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb80df8f0 con 0x7f9cb8007170 2024-09-16T15:38:09.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.088+0000 7f9cbfe00640 1 -- 172.21.15.60:0/2390674615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb80dfeb0 con 0x7f9cb8007170 2024-09-16T15:38:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.089+0000 7f18c0e00640 1 -- 172.21.15.60:0/1371574342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18bc15a520 msgr2=0x7f18bc15a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.089+0000 7f18c0e00640 1 --2- 172.21.15.60:0/1371574342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18bc15a520 0x7f18bc15a9a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18b402b370 tx=0x7f18b4004ab0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.089+0000 7f18c0e00640 1 -- 172.21.15.60:0/1371574342 shutdown_connections 2024-09-16T15:38:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.090+0000 7f18c0e00640 1 --2- 172.21.15.60:0/1371574342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18bc15a520 0x7f18bc15a9a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.090+0000 7f18c0e00640 1 --2- 172.21.15.60:0/1371574342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18bc0ad7c0 0x7f18bc0aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.090+0000 7f18c0e00640 1 --2- 172.21.15.60:0/1371574342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18a00052e0 0x7f18bc14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.090+0000 7f18c0e00640 1 -- 172.21.15.60:0/1371574342 >> 172.21.15.60:0/1371574342 conn(0x7f18bc0b1840 msgr2=0x7f18bc072d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.090+0000 7f18c0e00640 1 -- 172.21.15.60:0/1371574342 shutdown_connections 2024-09-16T15:38:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.092+0000 7f18c0e00640 1 -- 172.21.15.60:0/1371574342 wait complete. 2024-09-16T15:38:09.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.094+0000 7f9cb6a00640 1 -- 172.21.15.60:0/2390674615 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9cb00053c0 con 0x7f9cb8007170 2024-09-16T15:38:09.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.095+0000 7f9cb6a00640 1 -- 172.21.15.60:0/2390674615 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9cb0003ca0 con 0x7f9cb8007170 2024-09-16T15:38:09.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.097+0000 7f9cabe00640 1 -- 172.21.15.60:0/2390674615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c94005290 con 0x7f9cb8007170 2024-09-16T15:38:09.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.100+0000 7fd354c00640 1 -- 172.21.15.60:0/769510775 <== mon.1 v2:172.21.15.100:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd348042e50 con 0x7fd3580b2b40 2024-09-16T15:38:09.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.105+0000 7f9cb6a00640 1 -- 172.21.15.60:0/2390674615 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9cb0026020 con 0x7f9cb8007170 2024-09-16T15:38:09.135 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.140+0000 7f8bd7e00640 1 -- 172.21.15.60:0/1368340781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd016ef50 msgr2=0x7f8bd0165380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.140+0000 7f8bd7e00640 1 --2- 172.21.15.60:0/1368340781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd016ef50 0x7f8bd0165380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0097b0 tx=0x7f8bbc02a460 comp rx=0 tx=0).stop 2024-09-16T15:38:09.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.140+0000 7f8bd7e00640 1 -- 172.21.15.60:0/1368340781 shutdown_connections 2024-09-16T15:38:09.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.140+0000 7f8bd7e00640 1 --2- 172.21.15.60:0/1368340781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd016ef50 0x7f8bd0165380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.140+0000 7f8bd7e00640 1 -- 172.21.15.60:0/1368340781 >> 172.21.15.60:0/1368340781 conn(0x7f8bd00b1840 msgr2=0x7f8bd00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.141+0000 7f8bd7e00640 1 -- 172.21.15.60:0/1368340781 shutdown_connections 2024-09-16T15:38:09.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.142+0000 7f8bd7e00640 1 -- 172.21.15.60:0/1368340781 wait complete. 2024-09-16T15:38:09.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.144+0000 7f8bd7e00640 1 Processor -- start 2024-09-16T15:38:09.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.145+0000 7f8bd7e00640 1 -- start start 2024-09-16T15:38:09.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.148+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd006f510 0x7f8bd02049b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bd008b3b0 0x7f8bd015b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd7e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bd016ef50 0x7f8bd015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd006f510 0x7f8bd02049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd006f510 0x7f8bd02049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49626/0 (socket says 172.21.15.60:49626) 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd5400640 1 -- 172.21.15.60:0/906944725 learned_addr learned my addr 172.21.15.60:0/906944725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd0176be0 con 0x7f8bd008b3b0 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd5e00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bd016ef50 0x7f8bd015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd00cf4f0 con 0x7f8bd006f510 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.152+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd0176d60 con 0x7f8bd016ef50 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.153+0000 7f8bd5e00640 1 -- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd006f510 msgr2=0x7f8bd02049b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.153+0000 7f8bd5e00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd006f510 0x7f8bd02049b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.153+0000 7f8bd5e00640 1 -- 172.21.15.60:0/906944725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bd008b3b0 msgr2=0x7f8bd015b610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.156+0000 7f8bd4a00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bd008b3b0 0x7f8bd015b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.156+0000 7f8bd5e00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bd008b3b0 0x7f8bd015b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.156+0000 7f8bd5e00640 1 -- 172.21.15.60:0/906944725 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd0163600 con 0x7f8bd016ef50 2024-09-16T15:38:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.156+0000 7f8bd5400640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd006f510 0x7f8bd02049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.156+0000 7f8bd4a00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bd008b3b0 0x7f8bd015b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:09.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.158+0000 7f8bd5e00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bd016ef50 0x7f8bd015e1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c820 tx=0x7f8bc800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.158+0000 7f8bcd600640 1 -- 172.21.15.60:0/906944725 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8028070 con 0x7f8bd016ef50 2024-09-16T15:38:09.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.162+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd0163320 con 0x7f8bd016ef50 2024-09-16T15:38:09.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.162+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd02078d0 con 0x7f8bd016ef50 2024-09-16T15:38:09.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.162+0000 7f8bcd600640 1 -- 172.21.15.60:0/906944725 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bc8003ba0 con 0x7f8bd016ef50 2024-09-16T15:38:09.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.164+0000 7f8bcd600640 1 -- 172.21.15.60:0/906944725 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8028070 con 0x7f8bd016ef50 2024-09-16T15:38:09.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.165+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd0165380 con 0x7f8bd016ef50 2024-09-16T15:38:09.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.165+0000 7f8bcd600640 1 -- 172.21.15.60:0/906944725 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8bc802b070 con 0x7f8bd016ef50 2024-09-16T15:38:09.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.166+0000 7f8bcd600640 1 -- 172.21.15.60:0/906944725 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8bc800a960 con 0x7f8bd016ef50 2024-09-16T15:38:09.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.168+0000 7f8bcd600640 1 -- 172.21.15.60:0/906944725 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8bc8032080 con 0x7f8bd016ef50 2024-09-16T15:38:09.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.191+0000 7fd397e00640 1 -- 172.21.15.60:0/729982543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2420 msgr2=0x7fd3980b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.191+0000 7fd397e00640 1 --2- 172.21.15.60:0/729982543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2420 0x7fd3980b5240 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd384009860 tx=0x7fd38402a510 comp rx=0 tx=0).stop 2024-09-16T15:38:09.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.192+0000 7fd397e00640 1 -- 172.21.15.60:0/729982543 shutdown_connections 2024-09-16T15:38:09.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.192+0000 7fd397e00640 1 --2- 172.21.15.60:0/729982543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2420 0x7fd3980b5240 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.192+0000 7fd397e00640 1 -- 172.21.15.60:0/729982543 >> 172.21.15.60:0/729982543 conn(0x7fd39809d0b0 msgr2=0x7fd3980d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.194+0000 7fd397e00640 1 -- 172.21.15.60:0/729982543 shutdown_connections 2024-09-16T15:38:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.197+0000 7fd397e00640 1 -- 172.21.15.60:0/729982543 wait complete. 2024-09-16T15:38:09.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.208+0000 7fd397e00640 1 Processor -- start 2024-09-16T15:38:09.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.211+0000 7fd397e00640 1 -- start start 2024-09-16T15:38:09.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.212+0000 7f149e200640 1 -- 172.21.15.60:0/3321447674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f147c004d40 msgr2=0x7f147c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.212+0000 7f149e200640 1 --2- 172.21.15.60:0/3321447674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f147c004d40 0x7f147c0051c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1490004510 tx=0x7f147c005830 comp rx=0 tx=0).stop 2024-09-16T15:38:09.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.213+0000 7f1497e00640 1 -- 172.21.15.60:0/3321447674 reap_dead start 2024-09-16T15:38:09.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.213+0000 7f149e200640 1 -- 172.21.15.60:0/3321447674 shutdown_connections 2024-09-16T15:38:09.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.213+0000 7f149e200640 1 -- 172.21.15.60:0/3321447674 >> 172.21.15.60:0/3321447674 conn(0x7f14980b15f0 msgr2=0x7f14980b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.216+0000 7f149e200640 1 -- 172.21.15.60:0/3321447674 shutdown_connections 2024-09-16T15:38:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.217+0000 7f149e200640 1 -- 172.21.15.60:0/3321447674 wait complete. 2024-09-16T15:38:09.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.217+0000 7fd397e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3980b4d40 0x7fd3980b9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.217+0000 7fd397e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3980d2420 0x7fd39815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.217+0000 7fd397e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2800 0x7fd3980ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.218+0000 7fd396000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3980d2420 0x7fd39815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.218+0000 7fd396000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3980d2420 0x7fd39815c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49648/0 (socket says 172.21.15.60:49648) 2024-09-16T15:38:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.218+0000 7fd397400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2800 0x7fd3980ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.218+0000 7fd396a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3980b4d40 0x7fd3980b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.219+0000 7fd397400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2800 0x7fd3980ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43834/0 (socket says 172.21.15.60:43834) 2024-09-16T15:38:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.219+0000 7fd396a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3980b4d40 0x7fd3980b9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54040/0 (socket says 172.21.15.60:54040) 2024-09-16T15:38:09.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.219+0000 7fd396000640 1 -- 172.21.15.60:0/571627657 learned_addr learned my addr 172.21.15.60:0/571627657 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.219+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd39815fc80 con 0x7fd3980b4d40 2024-09-16T15:38:09.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.219+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd39815fe00 con 0x7fd3980d2420 2024-09-16T15:38:09.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.219+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd39815ff80 con 0x7fd3980d2800 2024-09-16T15:38:09.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.221+0000 7f149e200640 1 Processor -- start 2024-09-16T15:38:09.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.220+0000 7fd396a00640 1 -- 172.21.15.60:0/571627657 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2800 msgr2=0x7fd3980ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.221+0000 7fd396a00640 1 --2- 172.21.15.60:0/571627657 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2800 0x7fd3980ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.221+0000 7fd396a00640 1 -- 172.21.15.60:0/571627657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3980d2420 msgr2=0x7fd39815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.221+0000 7fd396a00640 1 --2- 172.21.15.60:0/571627657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3980d2420 0x7fd39815c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.221+0000 7fd396a00640 1 -- 172.21.15.60:0/571627657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3980cac80 con 0x7fd3980b4d40 2024-09-16T15:38:09.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.221+0000 7f149e200640 1 -- start start 2024-09-16T15:38:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.223+0000 7f149e200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f147c004d40 0x7f149814ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.223+0000 7f149e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14980b5160 0x7f14980c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.223+0000 7f149e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14980c4980 0x7f14980c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.223+0000 7f149e200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f149806f1c0 con 0x7f14980c4980 2024-09-16T15:38:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.223+0000 7f149e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f149806f040 con 0x7f14980b5160 2024-09-16T15:38:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.223+0000 7f149e200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f149806f340 con 0x7f147c004d40 2024-09-16T15:38:09.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.225+0000 7fd396000640 1 --2- 172.21.15.60:0/571627657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3980d2420 0x7fd39815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:09.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.225+0000 7f1497e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14980c4980 0x7f14980c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.225+0000 7f1496a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14980b5160 0x7f14980c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.225+0000 7fd396a00640 1 --2- 172.21.15.60:0/571627657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3980b4d40 0x7fd3980b9410 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd39806fb50 tx=0x7fd384032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.226+0000 7f1496a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14980b5160 0x7f14980c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49656/0 (socket says 172.21.15.60:49656) 2024-09-16T15:38:09.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.226+0000 7f1496a00640 1 -- 172.21.15.60:0/1288270555 learned_addr learned my addr 172.21.15.60:0/1288270555 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.226+0000 7fd38f400640 1 -- 172.21.15.60:0/571627657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384040070 con 0x7fd3980b4d40 2024-09-16T15:38:09.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.227+0000 7f1497400640 1 --2- 172.21.15.60:0/1288270555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f147c004d40 0x7f149814ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.227+0000 7fd38f400640 1 -- 172.21.15.60:0/571627657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd384005350 con 0x7fd3980b4d40 2024-09-16T15:38:09.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.229+0000 7f1497e00640 1 -- 172.21.15.60:0/1288270555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f147c004d40 msgr2=0x7f149814ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.229+0000 7f1497e00640 1 --2- 172.21.15.60:0/1288270555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f147c004d40 0x7f149814ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.228+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39815d4e0 con 0x7fd3980b4d40 2024-09-16T15:38:09.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.229+0000 7f1497e00640 1 -- 172.21.15.60:0/1288270555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14980b5160 msgr2=0x7f14980c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.233+0000 7f1497e00640 1 --2- 172.21.15.60:0/1288270555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14980b5160 0x7f14980c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.233+0000 7f1497e00640 1 -- 172.21.15.60:0/1288270555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14980c6d00 con 0x7f14980c4980 2024-09-16T15:38:09.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.233+0000 7f1497e00640 1 --2- 172.21.15.60:0/1288270555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14980c4980 0x7f14980c4e30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1490001ca0 tx=0x7f1490004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.228+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39815da60 con 0x7fd3980b4d40 2024-09-16T15:38:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.230+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39815dca0 con 0x7fd3980b4d40 2024-09-16T15:38:09.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.234+0000 7fd38f400640 1 -- 172.21.15.60:0/571627657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3840047a0 con 0x7fd3980b4d40 2024-09-16T15:38:09.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.238+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fe1d00d0900 con 0x7fe1d00cbdd0 2024-09-16T15:38:09.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.239+0000 7fd38f400640 1 -- 172.21.15.60:0/571627657 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd384039040 con 0x7fd3980b4d40 2024-09-16T15:38:09.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.235+0000 7f148f400640 1 -- 172.21.15.60:0/1288270555 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880013a0 con 0x7f14980c4980 2024-09-16T15:38:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.235+0000 7f148f400640 1 -- 172.21.15.60:0/1288270555 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1490001cd0 con 0x7f14980c4980 2024-09-16T15:38:09.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.235+0000 7f148f400640 1 -- 172.21.15.60:0/1288270555 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490007030 con 0x7f14980c4980 2024-09-16T15:38:09.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.245+0000 7fd38f400640 1 -- 172.21.15.60:0/571627657 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd384031db0 con 0x7fd3980b4d40 2024-09-16T15:38:09.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.246+0000 7fd38f400640 1 -- 172.21.15.60:0/571627657 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd39815dca0 con 0x7fd3980b4d40 2024-09-16T15:38:09.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.249+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14980c6fe0 con 0x7f14980c4980 2024-09-16T15:38:09.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.249+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149814f380 con 0x7f14980c4980 2024-09-16T15:38:09.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.255+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14980740e0 con 0x7f14980c4980 2024-09-16T15:38:09.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.256+0000 7fe1c7400640 1 -- 172.21.15.60:0/4028948831 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7fe1bc031750 con 0x7fe1d00cbdd0 2024-09-16T15:38:09.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.259+0000 7f148f400640 1 -- 172.21.15.60:0/1288270555 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1490007560 con 0x7f14980c4980 2024-09-16T15:38:09.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.259+0000 7f148f400640 1 -- 172.21.15.60:0/1288270555 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1490049c90 con 0x7f14980c4980 2024-09-16T15:38:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.264+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1d00cbdd0 msgr2=0x7fe1d00b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.264+0000 7fe1d4a00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1d00cbdd0 0x7fe1d00b5360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc031000 tx=0x7fe1bc0329e0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.264+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 shutdown_connections 2024-09-16T15:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.268+0000 7fe1d4a00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1d0161560 0x7fe1d00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.268+0000 7fe1d4a00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1d00cbdd0 0x7fe1d00b5360 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.268+0000 7fe1d4a00640 1 --2- 172.21.15.60:0/4028948831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1d006ef60 0x7fe1d01f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.268+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 >> 172.21.15.60:0/4028948831 conn(0x7fe1d00cf9f0 msgr2=0x7fe1d0072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.268+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 shutdown_connections 2024-09-16T15:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.268+0000 7fe1d4a00640 1 -- 172.21.15.60:0/4028948831 wait complete. 2024-09-16T15:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.272+0000 7f148f400640 1 -- 172.21.15.60:0/1288270555 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1490044060 con 0x7f14980c4980 2024-09-16T15:38:09.319 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.349+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fd330005580 con 0x7fd3580b2b40 2024-09-16T15:38:09.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.359+0000 7f9cabe00640 1 -- 172.21.15.60:0/2390674615 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f9c94005580 con 0x7f9cb8007170 2024-09-16T15:38:09.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.364+0000 7fd354c00640 1 -- 172.21.15.60:0/769510775 <== mon.1 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7fd348003df0 con 0x7fd3580b2b40 2024-09-16T15:38:09.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.368+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f8bd008bc30 con 0x7f8bd016ef50 2024-09-16T15:38:09.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 msgr2=0x7fd35814d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 0x7fd35814d760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd34803bb30 tx=0x7fd348043f00 comp rx=0 tx=0).stop 2024-09-16T15:38:09.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 shutdown_connections 2024-09-16T15:38:09.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3580b2b40 0x7fd35814d760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3580af170 0x7fd358158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 --2- 172.21.15.60:0/769510775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd35806fba0 0x7fd35814c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 >> 172.21.15.60:0/769510775 conn(0x7fd3580b1840 msgr2=0x7fd358150e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 shutdown_connections 2024-09-16T15:38:09.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.372+0000 7fd35f800640 1 -- 172.21.15.60:0/769510775 wait complete. 2024-09-16T15:38:09.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.374+0000 7f9cb6a00640 1 -- 172.21.15.60:0/2390674615 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f9cb00047c0 con 0x7f9cb8007170 2024-09-16T15:38:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.378+0000 7fa6f5800640 1 -- 172.21.15.60:0/782528705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e80e2a60 msgr2=0x7fa6e8037940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.378+0000 7fa6f5800640 1 --2- 172.21.15.60:0/782528705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e80e2a60 0x7fa6e8037940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8038100 tx=0x7fa6e402a610 comp rx=0 tx=0).stop 2024-09-16T15:38:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.381+0000 7fa6f5800640 1 -- 172.21.15.60:0/782528705 shutdown_connections 2024-09-16T15:38:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.381+0000 7fa6f5800640 1 --2- 172.21.15.60:0/782528705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6e803a940 0x7fa6e80e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.381+0000 7fa6f5800640 1 --2- 172.21.15.60:0/782528705 >> v2:172.21.15.60:3300/0 conn(0x7fa6e80e39e0 0x7fa6e803a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.381+0000 7fa6f5800640 1 --2- 172.21.15.60:0/782528705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e80e2a60 0x7fa6e8037940 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8038100 tx=0x7fa6e402a610 comp rx=0 tx=0).stop 2024-09-16T15:38:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.381+0000 7fa6f5800640 1 -- 172.21.15.60:0/782528705 >> 172.21.15.60:0/782528705 conn(0x7fa6e8055540 msgr2=0x7fa6e8055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.382+0000 7f9cabe00640 1 -- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb8007170 msgr2=0x7f9cb80df250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.382+0000 7f9cabe00640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb8007170 0x7f9cb80df250 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000e9f0 tx=0x7f9cb000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.382+0000 7f9cabe00640 1 -- 172.21.15.60:0/2390674615 shutdown_connections 2024-09-16T15:38:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.382+0000 7f9cabe00640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9cb80de9c0 0x7f9cb80dbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.382+0000 7f9cabe00640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9cb8051830 0x7f9cb81791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.382+0000 7f9cabe00640 1 --2- 172.21.15.60:0/2390674615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9cb8007170 0x7f9cb80df250 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.382+0000 7f9cabe00640 1 -- 172.21.15.60:0/2390674615 >> 172.21.15.60:0/2390674615 conn(0x7f9cb801d950 msgr2=0x7f9cb80dcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.382+0000 7f9cabe00640 1 -- 172.21.15.60:0/2390674615 shutdown_connections 2024-09-16T15:38:09.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.384+0000 7f9cabe00640 1 -- 172.21.15.60:0/2390674615 wait complete. 2024-09-16T15:38:09.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.388+0000 7f8bcd600640 1 -- 172.21.15.60:0/906944725 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f8bc800ab60 con 0x7f8bd016ef50 2024-09-16T15:38:09.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.389+0000 7fa6f5800640 1 -- 172.21.15.60:0/782528705 shutdown_connections 2024-09-16T15:38:09.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.389+0000 7fa6f5800640 1 -- 172.21.15.60:0/782528705 wait complete. 2024-09-16T15:38:09.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.395+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bd016ef50 msgr2=0x7f8bd015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.395+0000 7f8bd7e00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bd016ef50 0x7f8bd015e1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c820 tx=0x7f8bc800cce0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.399+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 shutdown_connections 2024-09-16T15:38:09.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.399+0000 7f8bd7e00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bd016ef50 0x7f8bd015e1c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.399+0000 7f8bd7e00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bd008b3b0 0x7f8bd015b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.399+0000 7f8bd7e00640 1 --2- 172.21.15.60:0/906944725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bd006f510 0x7f8bd02049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.399+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 >> 172.21.15.60:0/906944725 conn(0x7f8bd00b1840 msgr2=0x7f8bd00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.399+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 shutdown_connections 2024-09-16T15:38:09.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.399+0000 7f8bd7e00640 1 -- 172.21.15.60:0/906944725 wait complete. 2024-09-16T15:38:09.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.392+0000 7fa6f5800640 1 Processor -- start 2024-09-16T15:38:09.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.405+0000 7fa6f5800640 1 -- start start 2024-09-16T15:38:09.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.407+0000 7fa6f5800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e803a940 0x7fa6e81839a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.407+0000 7fa6f5800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6e80e39e0 0x7fa6e8186d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.407+0000 7fa6f5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e8009890 0x7fa6e8009e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.413+0000 7fa6ef400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6e80e39e0 0x7fa6e8186d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.413+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e803a940 0x7fa6e81839a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.414+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e803a940 0x7fa6e81839a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54056/0 (socket says 172.21.15.60:54056) 2024-09-16T15:38:09.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.414+0000 7fa6ef400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6e80e39e0 0x7fa6e8186d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43852/0 (socket says 172.21.15.60:43852) 2024-09-16T15:38:09.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.414+0000 7fa6efe00640 1 -- 172.21.15.60:0/2150857660 learned_addr learned my addr 172.21.15.60:0/2150857660 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.414+0000 7fa6f4e00640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e8009890 0x7fa6e8009e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.414+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e80e6ca0 con 0x7fa6e803a940 2024-09-16T15:38:09.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.414+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e8006950 con 0x7fa6e8009890 2024-09-16T15:38:09.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.414+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e80e6e20 con 0x7fa6e80e39e0 2024-09-16T15:38:09.420 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.418+0000 7fa6f4e00640 1 -- 172.21.15.60:0/2150857660 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6e80e39e0 msgr2=0x7fa6e8186d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.418+0000 7fa6f4e00640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6e80e39e0 0x7fa6e8186d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.418+0000 7fa6f4e00640 1 -- 172.21.15.60:0/2150857660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e803a940 msgr2=0x7fa6e81839a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.418+0000 7fa6f4e00640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e803a940 0x7fa6e81839a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.418+0000 7fa6f4e00640 1 -- 172.21.15.60:0/2150857660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e80df7f0 con 0x7fa6e8009890 2024-09-16T15:38:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.419+0000 7fa6efe00640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e803a940 0x7fa6e81839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.420+0000 7fa6f4e00640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e8009890 0x7fa6e8009e30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0077330 tx=0x7fa6f0069c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.420+0000 7fa6e3e00640 1 -- 172.21.15.60:0/2150857660 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0098070 con 0x7fa6e8009890 2024-09-16T15:38:09.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.421+0000 7fa6e3e00640 1 -- 172.21.15.60:0/2150857660 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f00737d0 con 0x7fa6e8009890 2024-09-16T15:38:09.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.421+0000 7fa6e3e00640 1 -- 172.21.15.60:0/2150857660 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f006ce60 con 0x7fa6e8009890 2024-09-16T15:38:09.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.421+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e80dfab0 con 0x7fa6e8009890 2024-09-16T15:38:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.421+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e80e0070 con 0x7fa6e8009890 2024-09-16T15:38:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.422+0000 7fa6e3e00640 1 -- 172.21.15.60:0/2150857660 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6f009b070 con 0x7fa6e8009890 2024-09-16T15:38:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.422+0000 7fa6e3e00640 1 -- 172.21.15.60:0/2150857660 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa6f009dd50 con 0x7fa6e8009890 2024-09-16T15:38:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.423+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e8053d70 con 0x7fa6e8009890 2024-09-16T15:38:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.427+0000 7fd8f8400640 1 -- 172.21.15.60:0/2418937994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 msgr2=0x7fd8f00d2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.427+0000 7fd8f8400640 1 --2- 172.21.15.60:0/2418937994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 0x7fd8f00d2030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009830 tx=0x7fd8dc02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.427+0000 7fd8f8400640 1 -- 172.21.15.60:0/2418937994 shutdown_connections 2024-09-16T15:38:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.427+0000 7fd8f8400640 1 --2- 172.21.15.60:0/2418937994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 0x7fd8f00d2030 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.427+0000 7fd8f8400640 1 --2- 172.21.15.60:0/2418937994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f00b5630 0x7fd8f00cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.427+0000 7fd8f8400640 1 -- 172.21.15.60:0/2418937994 >> 172.21.15.60:0/2418937994 conn(0x7fd8f00d3fb0 msgr2=0x7fd8f00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.425+0000 7fa6e3e00640 1 -- 172.21.15.60:0/2150857660 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa6e8053d70 con 0x7fa6e8009890 2024-09-16T15:38:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.428+0000 7fd37fe00640 1 -- 172.21.15.60:0/735271834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd364004800 msgr2=0x7fd364004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.428+0000 7fd37fe00640 1 --2- 172.21.15.60:0/735271834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd364004800 0x7fd364004c80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd3801628c0 tx=0x7fd364005830 comp rx=0 tx=0).stop 2024-09-16T15:38:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.429+0000 7fd37f400640 1 -- 172.21.15.60:0/735271834 reap_dead start 2024-09-16T15:38:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.429+0000 7fd37fe00640 1 -- 172.21.15.60:0/735271834 shutdown_connections 2024-09-16T15:38:09.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.429+0000 7fd37fe00640 1 -- 172.21.15.60:0/735271834 >> 172.21.15.60:0/735271834 conn(0x7fd3800b2670 msgr2=0x7fd3800b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.430+0000 7fd8f8400640 1 -- 172.21.15.60:0/2418937994 shutdown_connections 2024-09-16T15:38:09.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.430+0000 7fd8f8400640 1 -- 172.21.15.60:0/2418937994 wait complete. 2024-09-16T15:38:09.434 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.432+0000 7fd37fe00640 1 -- 172.21.15.60:0/735271834 shutdown_connections 2024-09-16T15:38:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.435+0000 7fd8f8400640 1 Processor -- start 2024-09-16T15:38:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.435+0000 7fd8f8400640 1 -- start start 2024-09-16T15:38:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.432+0000 7fd37fe00640 1 -- 172.21.15.60:0/735271834 wait complete. 2024-09-16T15:38:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.435+0000 7fd37fe00640 1 Processor -- start 2024-09-16T15:38:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.437+0000 7fd37fe00640 1 -- start start 2024-09-16T15:38:09.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.438+0000 7f8d6b200640 1 -- 172.21.15.60:0/4028515410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0cbca0 msgr2=0x7f8d6c0725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.438+0000 7f8d6b200640 1 --2- 172.21.15.60:0/4028515410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0cbca0 0x7f8d6c0725d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0c6b60 tx=0x7f8d5c02a400 comp rx=0 tx=0).stop 2024-09-16T15:38:09.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7f8d6b200640 1 -- 172.21.15.60:0/4028515410 shutdown_connections 2024-09-16T15:38:09.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7f8d6b200640 1 --2- 172.21.15.60:0/4028515410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0cbca0 0x7f8d6c0725d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7f8d6b200640 1 --2- 172.21.15.60:0/4028515410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d6c0ca150 0x7f8d6c0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7fd8f8400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 0x7fd8f00b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7f8d6b200640 1 --2- 172.21.15.60:0/4028515410 >> v2:172.21.15.60:3300/0 conn(0x7f8d6c0a04f0 0x7f8d6c0a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3801fe800 0x7fd380201390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.438+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2663543373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a1c005210 msgr2=0x7f0a1c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b6590 0x7fd38015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3802018d0 0x7fd38015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7f8d6b200640 1 -- 172.21.15.60:0/4028515410 >> 172.21.15.60:0/4028515410 conn(0x7f8d6c0cfb00 msgr2=0x7f8d6c0cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7fd8f8400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f0161590 0x7fd8f01f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.439+0000 7f0a3cc00640 1 --2- 172.21.15.60:0/2663543373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a1c005210 0x7f0a1c00b9c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a28006ed0 tx=0x7f0a1c005830 comp rx=0 tx=0).stop 2024-09-16T15:38:09.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37ea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3801fe800 0x7fd380201390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3802018d0 0x7fd38015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3802018d0 0x7fd38015c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54140/0 (socket says 172.21.15.60:54140) 2024-09-16T15:38:09.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37f400640 1 -- 172.21.15.60:0/1018873125 learned_addr learned my addr 172.21.15.60:0/1018873125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37ea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3801fe800 0x7fd380201390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43906/0 (socket says 172.21.15.60:43906) 2024-09-16T15:38:09.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd38006f830 con 0x7fd3802018d0 2024-09-16T15:38:09.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd38006f6b0 con 0x7fd3800b6590 2024-09-16T15:38:09.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd38006f9b0 con 0x7fd3801fe800 2024-09-16T15:38:09.444 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd37e000640 1 --2- 172.21.15.60:0/1018873125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b6590 0x7fd38015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7f0a36800640 1 -- 172.21.15.60:0/2663543373 reap_dead start 2024-09-16T15:38:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2663543373 shutdown_connections 2024-09-16T15:38:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2663543373 >> 172.21.15.60:0/2663543373 conn(0x7f0a380cfa20 msgr2=0x7f0a380cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.440+0000 7fd8f8400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f0159d50 0x7fd8f01fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7fd37e000640 1 -- 172.21.15.60:0/1018873125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3801fe800 msgr2=0x7fd380201390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7fd37e000640 1 --2- 172.21.15.60:0/1018873125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3801fe800 0x7fd380201390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7fd37e000640 1 -- 172.21.15.60:0/1018873125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3802018d0 msgr2=0x7fd38015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7fd37e000640 1 --2- 172.21.15.60:0/1018873125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3802018d0 0x7fd38015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7f8d6b200640 1 -- 172.21.15.60:0/4028515410 shutdown_connections 2024-09-16T15:38:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7fd37e000640 1 -- 172.21.15.60:0/1018873125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd380201de0 con 0x7fd3800b6590 2024-09-16T15:38:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2663543373 shutdown_connections 2024-09-16T15:38:09.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7f8d6b200640 1 -- 172.21.15.60:0/4028515410 wait complete. 2024-09-16T15:38:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2663543373 wait complete. 2024-09-16T15:38:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd37e000640 1 --2- 172.21.15.60:0/1018873125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b6590 0x7fd38015c1e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd36c032000 tx=0x7fd36c004da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7fd8f5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 0x7fd8f00b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.441+0000 7fd8f5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f0161590 0x7fd8f01f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd8f5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f0161590 0x7fd8f01f8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49710/0 (socket says 172.21.15.60:49710) 2024-09-16T15:38:09.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd8f5000640 1 -- 172.21.15.60:0/31178903 learned_addr learned my addr 172.21.15.60:0/31178903 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd8f6400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f0159d50 0x7fd8f01fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00d26f0 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd376a00640 1 -- 172.21.15.60:0/1018873125 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c005460 con 0x7fd3800b6590 2024-09-16T15:38:09.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd376a00640 1 -- 172.21.15.60:0/1018873125 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd36c0056f0 con 0x7fd3800b6590 2024-09-16T15:38:09.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd376a00640 1 -- 172.21.15.60:0/1018873125 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c03d680 con 0x7fd3800b6590 2024-09-16T15:38:09.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00d2570 con 0x7fd8f0161590 2024-09-16T15:38:09.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.442+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f00d2870 con 0x7fd8f0159d50 2024-09-16T15:38:09.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.443+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3802020c0 con 0x7fd3800b6590 2024-09-16T15:38:09.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.443+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3800c4a90 con 0x7fd3800b6590 2024-09-16T15:38:09.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.444+0000 7fd376a00640 1 -- 172.21.15.60:0/1018873125 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd36c03dbb0 con 0x7fd3800b6590 2024-09-16T15:38:09.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.444+0000 7fd376a00640 1 -- 172.21.15.60:0/1018873125 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd36c048400 con 0x7fd3800b6590 2024-09-16T15:38:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.445+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd380202300 con 0x7fd3800b6590 2024-09-16T15:38:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.445+0000 7fd8f5a00640 1 -- 172.21.15.60:0/31178903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f0159d50 msgr2=0x7fd8f01fc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.445+0000 7fd8f5a00640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f0159d50 0x7fd8f01fc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.445+0000 7fd8f5a00640 1 -- 172.21.15.60:0/31178903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f0161590 msgr2=0x7fd8f01f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.445+0000 7fd8f5a00640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f0161590 0x7fd8f01f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.445+0000 7fd8f5a00640 1 -- 172.21.15.60:0/31178903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f0159110 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.446+0000 7fd8f5000640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f0161590 0x7fd8f01f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:09.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.445+0000 7f8d6b200640 1 Processor -- start 2024-09-16T15:38:09.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.445+0000 7f8d6b200640 1 -- start start 2024-09-16T15:38:09.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.446+0000 7fd8f6400640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f0159d50 0x7fd8f01fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:09.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.446+0000 7fd8f5a00640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 0x7fd8f00b8070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c3c0 tx=0x7fd8e400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.446+0000 7f0a3cc00640 1 Processor -- start 2024-09-16T15:38:09.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.447+0000 7fd8ed600640 1 -- 172.21.15.60:0/31178903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4017070 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.447+0000 7fd8ed600640 1 -- 172.21.15.60:0/31178903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8e40035f0 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.447+0000 7fd8ed600640 1 -- 172.21.15.60:0/31178903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4003a40 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.447+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f01fccb0 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.447+0000 7f8d6b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d6c0a04f0 0x7f8d6c0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.447+0000 7f8d6b200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d6c0ca150 0x7f8d6c0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.448+0000 7f8d69400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d6c0ca150 0x7f8d6c0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.448+0000 7f8d69400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d6c0ca150 0x7f8d6c0b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43942/0 (socket says 172.21.15.60:43942) 2024-09-16T15:38:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.448+0000 7f8d69400640 1 -- 172.21.15.60:0/587213330 learned_addr learned my addr 172.21.15.60:0/587213330 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.448+0000 7f8d6b200640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0b0060 0x7f8d6c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.447+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f01fcf90 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7fd376a00640 1 -- 172.21.15.60:0/1018873125 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd380202300 con 0x7fd3800b6590 2024-09-16T15:38:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0cea70 con 0x7f8d6c0a04f0 2024-09-16T15:38:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0ce8f0 con 0x7f8d6c0b0060 2024-09-16T15:38:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c0cebf0 con 0x7f8d6c0ca150 2024-09-16T15:38:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6a800640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0b0060 0x7f8d6c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6a800640 1 -- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d6c0ca150 msgr2=0x7f8d6c0b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6a800640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d6c0ca150 0x7f8d6c0b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6a800640 1 -- 172.21.15.60:0/587213330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d6c0a04f0 msgr2=0x7f8d6c0c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6a800640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d6c0a04f0 0x7f8d6c0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.449+0000 7f8d6a800640 1 -- 172.21.15.60:0/587213330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c0b2f10 con 0x7f8d6c0b0060 2024-09-16T15:38:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.450+0000 7fd8ed600640 1 -- 172.21.15.60:0/31178903 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd8e401a070 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.451+0000 7f8d6a800640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0b0060 0x7f8d6c156580 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0a0f90 tx=0x7f8d5c033ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.450+0000 7fd8ed600640 1 -- 172.21.15.60:0/31178903 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd8e4012740 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.451+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f00d4ad0 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.448+0000 7f0a3cc00640 1 -- start start 2024-09-16T15:38:09.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.451+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fd3980b5240 con 0x7fd3980b4d40 2024-09-16T15:38:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.451+0000 7f8d62000640 1 -- 172.21.15.60:0/587213330 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c031b50 con 0x7f8d6c0b0060 2024-09-16T15:38:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.451+0000 7f8d62000640 1 -- 172.21.15.60:0/587213330 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d5c031de0 con 0x7f8d6c0b0060 2024-09-16T15:38:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.452+0000 7f8d62000640 1 -- 172.21.15.60:0/587213330 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c005280 con 0x7f8d6c0b0060 2024-09-16T15:38:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.452+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c0b31b0 con 0x7f8d6c0b0060 2024-09-16T15:38:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.452+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c0b3770 con 0x7f8d6c0b0060 2024-09-16T15:38:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f8d62000640 1 -- 172.21.15.60:0/587213330 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d5c0047a0 con 0x7f8d6c0b0060 2024-09-16T15:38:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f8d62000640 1 -- 172.21.15.60:0/587213330 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8d5c0335e0 con 0x7f8d6c0b0060 2024-09-16T15:38:09.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d44005290 con 0x7f8d6c0b0060 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.450+0000 7f0a3cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a1c005210 0x7f0a38151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.450+0000 7f0a3cc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a3814fa90 0x7f0a3814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.450+0000 7f0a3cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a38150430 0x7f0a380c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.450+0000 7f0a3cc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3806f0d0 con 0x7f0a1c005210 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.450+0000 7f0a3cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3806ef50 con 0x7f0a38150430 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.450+0000 7f0a3cc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0a380cc210 con 0x7f0a3814fa90 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.452+0000 7f0a35400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a3814fa90 0x7f0a3814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.452+0000 7f0a35400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a3814fa90 0x7f0a3814fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43944/0 (socket says 172.21.15.60:43944) 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.452+0000 7f0a35400640 1 -- 172.21.15.60:0/2929377234 learned_addr learned my addr 172.21.15.60:0/2929377234 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.452+0000 7f0a36800640 1 --2- 172.21.15.60:0/2929377234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a38150430 0x7f0a380c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.453+0000 7f0a36800640 1 -- 172.21.15.60:0/2929377234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a3814fa90 msgr2=0x7f0a3814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.453+0000 7f0a36800640 1 --2- 172.21.15.60:0/2929377234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a3814fa90 0x7f0a3814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.453+0000 7f0a36800640 1 -- 172.21.15.60:0/2929377234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a1c005210 msgr2=0x7f0a38151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.453+0000 7f0a36800640 1 --2- 172.21.15.60:0/2929377234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a1c005210 0x7f0a38151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.453+0000 7f0a36800640 1 -- 172.21.15.60:0/2929377234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a381508b0 con 0x7f0a38150430 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.453+0000 7f0a36800640 1 --2- 172.21.15.60:0/2929377234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a38150430 0x7f0a380c51a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00c0e0 tx=0x7f0a28006280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.453+0000 7f999cc00640 1 -- 172.21.15.60:0/1410588526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9998161460 msgr2=0x7f99980d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.453+0000 7f999cc00640 1 --2- 172.21.15.60:0/1410588526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9998161460 0x7f99980d21b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f99980d0180 tx=0x7f998802a510 comp rx=0 tx=0).stop 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f999cc00640 1 -- 172.21.15.60:0/1410588526 shutdown_connections 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f999cc00640 1 --2- 172.21.15.60:0/1410588526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9998161460 0x7f99980d21b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f999cc00640 1 -- 172.21.15.60:0/1410588526 >> 172.21.15.60:0/1410588526 conn(0x7f999809d0b0 msgr2=0x7f99980d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f0a2e000640 1 -- 172.21.15.60:0/2929377234 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28006af0 con 0x7f0a38150430 2024-09-16T15:38:09.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f0a2e000640 1 -- 172.21.15.60:0/2929377234 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a28046030 con 0x7f0a38150430 2024-09-16T15:38:09.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f0a2e000640 1 -- 172.21.15.60:0/2929377234 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28004020 con 0x7f0a38150430 2024-09-16T15:38:09.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f999cc00640 1 -- 172.21.15.60:0/1410588526 shutdown_connections 2024-09-16T15:38:09.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.454+0000 7f999cc00640 1 -- 172.21.15.60:0/1410588526 wait complete. 2024-09-16T15:38:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.455+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a380c3550 con 0x7f0a38150430 2024-09-16T15:38:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.455+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a380c3af0 con 0x7f0a38150430 2024-09-16T15:38:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.456+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a38160040 con 0x7f0a38150430 2024-09-16T15:38:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.456+0000 7fd8ed600640 1 -- 172.21.15.60:0/31178903 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd8f00d4ad0 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.457+0000 7f0a2e000640 1 -- 172.21.15.60:0/2929377234 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0a280035f0 con 0x7f0a38150430 2024-09-16T15:38:09.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.457+0000 7f0a2e000640 1 -- 172.21.15.60:0/2929377234 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a2804b5b0 con 0x7f0a38150430 2024-09-16T15:38:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.457+0000 7f8d62000640 1 -- 172.21.15.60:0/587213330 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8d5c03f030 con 0x7f8d6c0b0060 2024-09-16T15:38:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.457+0000 7f999cc00640 1 Processor -- start 2024-09-16T15:38:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7fd38f400640 1 -- 172.21.15.60:0/571627657 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7fd384049060 con 0x7fd3980b4d40 2024-09-16T15:38:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.458+0000 7f999cc00640 1 -- start start 2024-09-16T15:38:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f999cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99980b4d40 0x7f999815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f999cc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99980b5270 0x7f99980caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f999cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9998161460 0x7f9998158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f999cc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f99980cfbc0 con 0x7f99980b4d40 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f999cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99980d1220 con 0x7f9998161460 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f999cc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f99980d26f0 con 0x7f99980b5270 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f9996a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99980b5270 0x7f99980caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f9996a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99980b5270 0x7f99980caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43958/0 (socket says 172.21.15.60:43958) 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.460+0000 7f9996a00640 1 -- 172.21.15.60:0/250812748 learned_addr learned my addr 172.21.15.60:0/250812748 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.461+0000 7f9997400640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99980b4d40 0x7f999815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.462+0000 7f9996a00640 1 -- 172.21.15.60:0/250812748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9998161460 msgr2=0x7f9998158d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.463+0000 7f9996a00640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9998161460 0x7f9998158d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.463+0000 7f9996a00640 1 -- 172.21.15.60:0/250812748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99980b4d40 msgr2=0x7f999815a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.463+0000 7f9996a00640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99980b4d40 0x7f999815a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.463+0000 7f9996a00640 1 -- 172.21.15.60:0/250812748 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99980cb730 con 0x7f99980b5270 2024-09-16T15:38:09.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.463+0000 7f9997400640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99980b4d40 0x7f999815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:09.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.464+0000 7f9996a00640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99980b5270 0x7f99980caec0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f998400c4d0 tx=0x7f998400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.464+0000 7f998f400640 1 -- 172.21.15.60:0/250812748 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99840052d0 con 0x7f99980b5270 2024-09-16T15:38:09.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.464+0000 7f998f400640 1 -- 172.21.15.60:0/250812748 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9984005470 con 0x7f99980b5270 2024-09-16T15:38:09.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.464+0000 7f998f400640 1 -- 172.21.15.60:0/250812748 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984012870 con 0x7f99980b5270 2024-09-16T15:38:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.465+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3980b4d40 msgr2=0x7fd3980b9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.465+0000 7fd397e00640 1 --2- 172.21.15.60:0/571627657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3980b4d40 0x7fd3980b9410 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd39806fb50 tx=0x7fd384032b90 comp rx=0 tx=0).stop 2024-09-16T15:38:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.465+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999815af20 con 0x7f99980b5270 2024-09-16T15:38:09.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.466+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f999815b4e0 con 0x7f99980b5270 2024-09-16T15:38:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.466+0000 7f0a2e000640 1 -- 172.21.15.60:0/2929377234 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0a28050030 con 0x7f0a38150430 2024-09-16T15:38:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.466+0000 7f998f400640 1 -- 172.21.15.60:0/250812748 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9984012a10 con 0x7f99980b5270 2024-09-16T15:38:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.466+0000 7f998f400640 1 -- 172.21.15.60:0/250812748 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f998401b8a0 con 0x7f99980b5270 2024-09-16T15:38:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.467+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99980d3a60 con 0x7f99980b5270 2024-09-16T15:38:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.467+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 shutdown_connections 2024-09-16T15:38:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.467+0000 7fd397e00640 1 --2- 172.21.15.60:0/571627657 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3980d2800 0x7fd3980ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.467+0000 7fd397e00640 1 --2- 172.21.15.60:0/571627657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3980d2420 0x7fd39815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.467+0000 7fd397e00640 1 --2- 172.21.15.60:0/571627657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3980b4d40 0x7fd3980b9410 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.467+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 >> 172.21.15.60:0/571627657 conn(0x7fd39809d0b0 msgr2=0x7fd3980d3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.467+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 shutdown_connections 2024-09-16T15:38:09.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.467+0000 7fd397e00640 1 -- 172.21.15.60:0/571627657 wait complete. 2024-09-16T15:38:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.470+0000 7f998f400640 1 -- 172.21.15.60:0/250812748 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f99980d3a60 con 0x7f99980b5270 2024-09-16T15:38:09.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.470+0000 7fe40f000640 1 -- 172.21.15.60:0/2293180122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 msgr2=0x7fe4080b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.470+0000 7fe40f000640 1 --2- 172.21.15.60:0/2293180122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 0x7fe4080b3ec0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009830 tx=0x7fe3f402a540 comp rx=0 tx=0).stop 2024-09-16T15:38:09.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.471+0000 7fe40f000640 1 -- 172.21.15.60:0/2293180122 shutdown_connections 2024-09-16T15:38:09.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.471+0000 7fe40f000640 1 --2- 172.21.15.60:0/2293180122 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4080b4400 0x7fe40806e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.471+0000 7fe40f000640 1 --2- 172.21.15.60:0/2293180122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 0x7fe4080b3ec0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.471+0000 7fe40f000640 1 -- 172.21.15.60:0/2293180122 >> 172.21.15.60:0/2293180122 conn(0x7fe4080b1840 msgr2=0x7fe4080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.471+0000 7fe40f000640 1 -- 172.21.15.60:0/2293180122 shutdown_connections 2024-09-16T15:38:09.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.471+0000 7fe40f000640 1 -- 172.21.15.60:0/2293180122 wait complete. 2024-09-16T15:38:09.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.474+0000 7fe40f000640 1 Processor -- start 2024-09-16T15:38:09.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.479+0000 7fe40f000640 1 -- start start 2024-09-16T15:38:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.481+0000 7fe40f000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4400 0x7fe40815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.482+0000 7fe40f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 0x7fe4080aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.482+0000 7fe40f000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4081437c0 0x7fe4080aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.483+0000 7fe40d000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4081437c0 0x7fe4080aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.483+0000 7fe40d000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4081437c0 0x7fe4080aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43970/0 (socket says 172.21.15.60:43970) 2024-09-16T15:38:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.483+0000 7fe407e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4400 0x7fe40815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.483+0000 7fe407e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4400 0x7fe40815aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54170/0 (socket says 172.21.15.60:54170) 2024-09-16T15:38:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.483+0000 7fe407400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 0x7fe4080aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.483+0000 7fe407400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 0x7fe4080aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49762/0 (socket says 172.21.15.60:49762) 2024-09-16T15:38:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.483+0000 7fe40d000640 1 -- 172.21.15.60:0/3051373915 learned_addr learned my addr 172.21.15.60:0/3051373915 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.484+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe40806ef20 con 0x7fe4080b4400 2024-09-16T15:38:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.484+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe40806eda0 con 0x7fe4080b4bd0 2024-09-16T15:38:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.484+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe40806f0a0 con 0x7fe4081437c0 2024-09-16T15:38:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.485+0000 7fe407e00640 1 -- 172.21.15.60:0/3051373915 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4081437c0 msgr2=0x7fe4080aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.485+0000 7fe407e00640 1 --2- 172.21.15.60:0/3051373915 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4081437c0 0x7fe4080aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.485+0000 7fe407e00640 1 -- 172.21.15.60:0/3051373915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 msgr2=0x7fe4080aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.485+0000 7fe407e00640 1 --2- 172.21.15.60:0/3051373915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 0x7fe4080aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.485+0000 7fe407e00640 1 -- 172.21.15.60:0/3051373915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4080a9790 con 0x7fe4080b4400 2024-09-16T15:38:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.486+0000 7fe407400640 1 --2- 172.21.15.60:0/3051373915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 0x7fe4080aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.486+0000 7fe407e00640 1 --2- 172.21.15.60:0/3051373915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4400 0x7fe40815aa20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c3f0 tx=0x7fe3f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.487+0000 7fe3ffe00640 1 -- 172.21.15.60:0/3051373915 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8011790 con 0x7fe4080b4400 2024-09-16T15:38:09.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.487+0000 7fe3ffe00640 1 -- 172.21.15.60:0/3051373915 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3f8017070 con 0x7fe4080b4400 2024-09-16T15:38:09.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.488+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4080aaf60 con 0x7fe4080b4400 2024-09-16T15:38:09.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.488+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4080ab990 con 0x7fe4080b4400 2024-09-16T15:38:09.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.490+0000 7fe3ffe00640 1 -- 172.21.15.60:0/3051373915 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8011ce0 con 0x7fe4080b4400 2024-09-16T15:38:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.491+0000 7fe3ffe00640 1 -- 172.21.15.60:0/3051373915 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe3f801c460 con 0x7fe4080b4400 2024-09-16T15:38:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.492+0000 7fe3ffe00640 1 -- 172.21.15.60:0/3051373915 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe3f8012880 con 0x7fe4080b4400 2024-09-16T15:38:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.494+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e0005290 con 0x7fe4080b4400 2024-09-16T15:38:09.497 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.498+0000 7fe3ffe00640 1 -- 172.21.15.60:0/3051373915 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe4080a9790 con 0x7fe4080b4400 2024-09-16T15:38:09.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.512+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f149814fc30 con 0x7f14980c4980 2024-09-16T15:38:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.519+0000 7f148f400640 1 -- 172.21.15.60:0/1288270555 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7f1490044060 con 0x7f14980c4980 2024-09-16T15:38:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14980c4980 msgr2=0x7f14980c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 --2- 172.21.15.60:0/1288270555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14980c4980 0x7f14980c4e30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1490001ca0 tx=0x7f1490004160 comp rx=0 tx=0).stop 2024-09-16T15:38:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 shutdown_connections 2024-09-16T15:38:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 --2- 172.21.15.60:0/1288270555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14980c4980 0x7f14980c4e30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 --2- 172.21.15.60:0/1288270555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14980b5160 0x7f14980c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 --2- 172.21.15.60:0/1288270555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f147c004d40 0x7f149814ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 >> 172.21.15.60:0/1288270555 conn(0x7f14980b15f0 msgr2=0x7f1498075900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 shutdown_connections 2024-09-16T15:38:09.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.523+0000 7f149e200640 1 -- 172.21.15.60:0/1288270555 wait complete. 2024-09-16T15:38:09.558 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.590+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fa6e80e6550 con 0x7fa6e8009890 2024-09-16T15:38:09.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.595+0000 7fa6e3e00640 1 -- 172.21.15.60:0/2150857660 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v13)=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7fa6f009d440 con 0x7fa6e8009890 2024-09-16T15:38:09.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.597+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e8009890 msgr2=0x7fa6e8009e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.597+0000 7fa6f5800640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e8009890 0x7fa6e8009e30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0077330 tx=0x7fa6f0069c90 comp rx=0 tx=0).stop 2024-09-16T15:38:09.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.600+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 shutdown_connections 2024-09-16T15:38:09.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.600+0000 7fa6f5800640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6e8009890 0x7fa6e8009e30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.600+0000 7fa6f5800640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6e80e39e0 0x7fa6e8186d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.600+0000 7fa6f5800640 1 --2- 172.21.15.60:0/2150857660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6e803a940 0x7fa6e81839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.600+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 >> 172.21.15.60:0/2150857660 conn(0x7fa6e8055540 msgr2=0x7fa6e80ddca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.600+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 shutdown_connections 2024-09-16T15:38:09.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.600+0000 7fa6f5800640 1 -- 172.21.15.60:0/2150857660 wait complete. 2024-09-16T15:38:09.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.609+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f8d44005580 con 0x7f8d6c0b0060 2024-09-16T15:38:09.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.610+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fd8f00d2cf0 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.614+0000 7fd8ed600640 1 -- 172.21.15.60:0/31178903 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v14)=0 v14) ==== 125+0+0 (secure 0 0 0) 0x7fd8e4021070 con 0x7fd8f00cfaf0 2024-09-16T15:38:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.615+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fd38015e020 con 0x7fd3800b6590 2024-09-16T15:38:09.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.616+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 msgr2=0x7fd8f00b8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.616+0000 7fd8f8400640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 0x7fd8f00b8070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c3c0 tx=0x7fd8e400c7c0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.616+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 shutdown_connections 2024-09-16T15:38:09.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.616+0000 7fd8f8400640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f0159d50 0x7fd8f01fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.616+0000 7fd8f8400640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f0161590 0x7fd8f01f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.616+0000 7fd8f8400640 1 --2- 172.21.15.60:0/31178903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f00cfaf0 0x7fd8f00b8070 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.616+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 >> 172.21.15.60:0/31178903 conn(0x7fd8f00d3fb0 msgr2=0x7fd8f01623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.616+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 shutdown_connections 2024-09-16T15:38:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.617+0000 7fd8f8400640 1 -- 172.21.15.60:0/31178903 wait complete. 2024-09-16T15:38:09.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.618+0000 7f8d62000640 1 -- 172.21.15.60:0/587213330 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f8d5c031770 con 0x7f8d6c0b0060 2024-09-16T15:38:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.620+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0b0060 msgr2=0x7f8d6c156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.620+0000 7f8d6b200640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0b0060 0x7f8d6c156580 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0a0f90 tx=0x7f8d5c033ba0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.621+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 shutdown_connections 2024-09-16T15:38:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.621+0000 7f8d6b200640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d6c0b0060 0x7f8d6c156580 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.621+0000 7f8d6b200640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d6c0ca150 0x7f8d6c0b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.621+0000 7f8d6b200640 1 --2- 172.21.15.60:0/587213330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d6c0a04f0 0x7f8d6c0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.621+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 >> 172.21.15.60:0/587213330 conn(0x7f8d6c0cfb00 msgr2=0x7f8d6c0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.621+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 shutdown_connections 2024-09-16T15:38:09.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.621+0000 7f8d6b200640 1 -- 172.21.15.60:0/587213330 wait complete. 2024-09-16T15:38:09.623 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.623+0000 7fd376a00640 1 -- 172.21.15.60:0/1018873125 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7fd36c04d070 con 0x7fd3800b6590 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.624+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b6590 msgr2=0x7fd38015c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.624+0000 7fd37fe00640 1 --2- 172.21.15.60:0/1018873125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b6590 0x7fd38015c1e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd36c032000 tx=0x7fd36c004da0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.625+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 shutdown_connections 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.625+0000 7fd37fe00640 1 --2- 172.21.15.60:0/1018873125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3802018d0 0x7fd38015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.625+0000 7fd37fe00640 1 --2- 172.21.15.60:0/1018873125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b6590 0x7fd38015c1e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.625+0000 7fd37fe00640 1 --2- 172.21.15.60:0/1018873125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3801fe800 0x7fd380201390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.625+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 >> 172.21.15.60:0/1018873125 conn(0x7fd3800b2670 msgr2=0x7fd380158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.625+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 shutdown_connections 2024-09-16T15:38:09.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.625+0000 7fd37fe00640 1 -- 172.21.15.60:0/1018873125 wait complete. 2024-09-16T15:38:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.637+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f99980d21b0 con 0x7f99980b5270 2024-09-16T15:38:09.643 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.645 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.644+0000 7f998f400640 1 -- 172.21.15.60:0/250812748 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f99980d3a60 con 0x7f99980b5270 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.645+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99980b5270 msgr2=0x7f99980caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.645+0000 7f999cc00640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99980b5270 0x7f99980caec0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f998400c4d0 tx=0x7f998400c990 comp rx=0 tx=0).stop 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.646+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 shutdown_connections 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.646+0000 7f999cc00640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9998161460 0x7f9998158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.646+0000 7f999cc00640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99980b5270 0x7f99980caec0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.646+0000 7f999cc00640 1 --2- 172.21.15.60:0/250812748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99980b4d40 0x7f999815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.646+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 >> 172.21.15.60:0/250812748 conn(0x7f999809d0b0 msgr2=0x7f99980b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.646+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 shutdown_connections 2024-09-16T15:38:09.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.646+0000 7f999cc00640 1 -- 172.21.15.60:0/250812748 wait complete. 2024-09-16T15:38:09.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.648+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fe3e0005580 con 0x7fe4080b4400 2024-09-16T15:38:09.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.650+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f0a380c9d90 con 0x7f0a38150430 2024-09-16T15:38:09.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.650+0000 7fe3ffe00640 1 -- 172.21.15.60:0/3051373915 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v18)=0 v18) ==== 120+0+0 (secure 0 0 0) 0x7fe3f8012420 con 0x7fe4080b4400 2024-09-16T15:38:09.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4400 msgr2=0x7fe40815aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 --2- 172.21.15.60:0/3051373915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4400 0x7fe40815aa20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c3f0 tx=0x7fe3f800c8b0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 shutdown_connections 2024-09-16T15:38:09.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 --2- 172.21.15.60:0/3051373915 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4081437c0 0x7fe4080aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 --2- 172.21.15.60:0/3051373915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080b4bd0 0x7fe4080aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 --2- 172.21.15.60:0/3051373915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4400 0x7fe40815aa20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 >> 172.21.15.60:0/3051373915 conn(0x7fe4080b1840 msgr2=0x7fe408072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 shutdown_connections 2024-09-16T15:38:09.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.652+0000 7fe40f000640 1 -- 172.21.15.60:0/3051373915 wait complete. 2024-09-16T15:38:09.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.655+0000 7f0a2e000640 1 -- 172.21.15.60:0/2929377234 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f0a280062f0 con 0x7f0a38150430 2024-09-16T15:38:09.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a38150430 msgr2=0x7f0a380c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 --2- 172.21.15.60:0/2929377234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a38150430 0x7f0a380c51a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00c0e0 tx=0x7f0a28006280 comp rx=0 tx=0).stop 2024-09-16T15:38:09.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 shutdown_connections 2024-09-16T15:38:09.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 --2- 172.21.15.60:0/2929377234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0a38150430 0x7f0a380c51a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 --2- 172.21.15.60:0/2929377234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0a3814fa90 0x7f0a3814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 --2- 172.21.15.60:0/2929377234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0a1c005210 0x7f0a38151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 >> 172.21.15.60:0/2929377234 conn(0x7f0a380cfa20 msgr2=0x7f0a380afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 shutdown_connections 2024-09-16T15:38:09.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.657+0000 7f0a3cc00640 1 -- 172.21.15.60:0/2929377234 wait complete. 2024-09-16T15:38:09.671 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.676 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.676 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi060') 2024-09-16T15:38:09.676 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T15:38:09.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T15:38:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.871+0000 7f36a5c00640 1 -- 172.21.15.60:0/3411369043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36a0161460 msgr2=0x7f36a00d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.871+0000 7f36a5c00640 1 --2- 172.21.15.60:0/3411369043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36a0161460 0x7f36a00d21b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3690009830 tx=0x7f369002a470 comp rx=0 tx=0).stop 2024-09-16T15:38:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.871+0000 7f36a5c00640 1 -- 172.21.15.60:0/3411369043 shutdown_connections 2024-09-16T15:38:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.871+0000 7f36a5c00640 1 --2- 172.21.15.60:0/3411369043 >> v2:172.21.15.100:3300/0 conn(0x7f36a00b4d40 0x7f36a00d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.871+0000 7f36a5c00640 1 --2- 172.21.15.60:0/3411369043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36a0161460 0x7f36a00d21b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.871+0000 7f36a5c00640 1 -- 172.21.15.60:0/3411369043 >> 172.21.15.60:0/3411369043 conn(0x7f36a009d0b0 msgr2=0x7f36a00d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.871+0000 7f36a5c00640 1 -- 172.21.15.60:0/3411369043 shutdown_connections 2024-09-16T15:38:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.871+0000 7f36a5c00640 1 -- 172.21.15.60:0/3411369043 wait complete. 2024-09-16T15:38:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.873+0000 7f36a5c00640 1 Processor -- start 2024-09-16T15:38:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.873+0000 7f36a5c00640 1 -- start start 2024-09-16T15:38:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f36a5c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36a00b4d40 0x7f36a00cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f36a5c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a00b5270 0x7f36a01f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f36a5c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0161460 0x7f36a0157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f36a5c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00d1220 con 0x7f36a0161460 2024-09-16T15:38:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f36a5c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00b3ac0 con 0x7f36a00b5270 2024-09-16T15:38:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f36a5c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f36a01f55e0 con 0x7f36a00b4d40 2024-09-16T15:38:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f3697400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a00b5270 0x7f36a01f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f36a5200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0161460 0x7f36a0157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f3697400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a00b5270 0x7f36a01f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49784/0 (socket says 172.21.15.60:49784) 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f369fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36a00b4d40 0x7f36a00cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.874+0000 7f3697400640 1 -- 172.21.15.60:0/3329713378 learned_addr learned my addr 172.21.15.60:0/3329713378 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f36a5200640 1 -- 172.21.15.60:0/3329713378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36a00b4d40 msgr2=0x7f36a00cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f36a5200640 1 --2- 172.21.15.60:0/3329713378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36a00b4d40 0x7f36a00cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f36a5200640 1 -- 172.21.15.60:0/3329713378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a00b5270 msgr2=0x7f36a01f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f36a5200640 1 --2- 172.21.15.60:0/3329713378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a00b5270 0x7f36a01f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f36a5200640 1 -- 172.21.15.60:0/3329713378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a00cb480 con 0x7f36a0161460 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f3697400640 1 --2- 172.21.15.60:0/3329713378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a00b5270 0x7f36a01f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f36a5200640 1 --2- 172.21.15.60:0/3329713378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0161460 0x7f36a0157090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f369800cad0 tx=0x7f369800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f369cc00640 1 -- 172.21.15.60:0/3329713378 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698016020 con 0x7f36a0161460 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f369cc00640 1 -- 172.21.15.60:0/3329713378 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f369800ace0 con 0x7f36a0161460 2024-09-16T15:38:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f369cc00640 1 -- 172.21.15.60:0/3329713378 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698012680 con 0x7f36a0161460 2024-09-16T15:38:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a00cb760 con 0x7f36a0161460 2024-09-16T15:38:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.875+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a00cbd20 con 0x7f36a0161460 2024-09-16T15:38:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.876+0000 7f369cc00640 1 -- 172.21.15.60:0/3329713378 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3698019020 con 0x7f36a0161460 2024-09-16T15:38:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.876+0000 7f369cc00640 1 -- 172.21.15.60:0/3329713378 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f369801bcc0 con 0x7f36a0161460 2024-09-16T15:38:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.876+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a00b47a0 con 0x7f36a0161460 2024-09-16T15:38:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.879+0000 7f369cc00640 1 -- 172.21.15.60:0/3329713378 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3698020020 con 0x7f36a0161460 2024-09-16T15:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.947+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f36a015eb10 con 0x7f36a0161460 2024-09-16T15:38:09.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.949+0000 7f369cc00640 1 -- 172.21.15.60:0/3329713378 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3698012de0 con 0x7f36a0161460 2024-09-16T15:38:09.950 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-09-16T15:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.950+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0161460 msgr2=0x7f36a0157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.950+0000 7f36a5c00640 1 --2- 172.21.15.60:0/3329713378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0161460 0x7f36a0157090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f369800cad0 tx=0x7f369800cf90 comp rx=0 tx=0).stop 2024-09-16T15:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.951+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 shutdown_connections 2024-09-16T15:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.951+0000 7f36a5c00640 1 --2- 172.21.15.60:0/3329713378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36a0161460 0x7f36a0157090 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.951+0000 7f36a5c00640 1 --2- 172.21.15.60:0/3329713378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36a00b5270 0x7f36a01f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.951+0000 7f36a5c00640 1 --2- 172.21.15.60:0/3329713378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36a00b4d40 0x7f36a00cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.951+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 >> 172.21.15.60:0/3329713378 conn(0x7f36a009d0b0 msgr2=0x7f36a006d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.951+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 shutdown_connections 2024-09-16T15:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:09.951+0000 7f36a5c00640 1 -- 172.21.15.60:0/3329713378 wait complete. 2024-09-16T15:38:09.966 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T15:38:09.967 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T15:38:09.967 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:09.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T15:38:10.026 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:10.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T15:38:10.121 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:10.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T15:38:10.217 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:10.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T15:38:10.314 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:10.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T15:38:10.370 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:10.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T15:38:10.466 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:10.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T15:38:10.561 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:38:10.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T15:38:10.659 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 0de59e14-e6eb-4c11-8c21-84177ef27af2 0 2024-09-16T15:38:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.852+0000 7f767f200640 1 -- 172.21.15.100:0/1666806355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76780cf830 msgr2=0x7f76780b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.852+0000 7f767f200640 1 --2- 172.21.15.100:0/1666806355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76780cf830 0x7f76780b33b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7668009860 tx=0x7f766802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:38:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.852+0000 7f767f200640 1 -- 172.21.15.100:0/1666806355 shutdown_connections 2024-09-16T15:38:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.852+0000 7f767f200640 1 --2- 172.21.15.100:0/1666806355 >> v2:172.21.15.60:3300/0 conn(0x7f76780d2a00 0x7f76780b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.852+0000 7f767f200640 1 --2- 172.21.15.100:0/1666806355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76780cf830 0x7f76780b33b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.852+0000 7f767f200640 1 -- 172.21.15.100:0/1666806355 >> 172.21.15.100:0/1666806355 conn(0x7f767809d0b0 msgr2=0x7f76780d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.852+0000 7f767f200640 1 -- 172.21.15.100:0/1666806355 shutdown_connections 2024-09-16T15:38:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.852+0000 7f767f200640 1 -- 172.21.15.100:0/1666806355 wait complete. 2024-09-16T15:38:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.853+0000 7f767f200640 1 Processor -- start 2024-09-16T15:38:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.854+0000 7f767f200640 1 -- start start 2024-09-16T15:38:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.854+0000 7f767f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76780cf830 0x7f7678156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.854+0000 7f767f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76780d2a00 0x7f76781f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.854+0000 7f767f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7678161460 0x7f76780b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.854+0000 7f767f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f76780b56d0 con 0x7f7678161460 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.854+0000 7f767f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76780b5150 con 0x7f76780cf830 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.854+0000 7f767f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f76780b3e30 con 0x7f76780d2a00 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76780cf830 0x7f7678156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767d400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76780d2a00 0x7f76781f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76780cf830 0x7f7678156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49472/0 (socket says 172.21.15.100:49472) 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 -- 172.21.15.100:0/3424904403 learned_addr learned my addr 172.21.15.100:0/3424904403 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767e800640 1 --2- 172.21.15.100:0/3424904403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7678161460 0x7f76780b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 -- 172.21.15.100:0/3424904403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76780d2a00 msgr2=0x7f76781f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 --2- 172.21.15.100:0/3424904403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76780d2a00 0x7f76781f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 -- 172.21.15.100:0/3424904403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7678161460 msgr2=0x7f76780b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 --2- 172.21.15.100:0/3424904403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7678161460 0x7f76780b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 -- 172.21.15.100:0/3424904403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767815abf0 con 0x7f76780cf830 2024-09-16T15:38:10.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.855+0000 7f767de00640 1 --2- 172.21.15.100:0/3424904403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76780cf830 0x7f7678156f90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f767000cb40 tx=0x7f767000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:10.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.856+0000 7f766e000640 1 -- 172.21.15.100:0/3424904403 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670016020 con 0x7f76780cf830 2024-09-16T15:38:10.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.856+0000 7f766e000640 1 -- 172.21.15.100:0/3424904403 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f767000adb0 con 0x7f76780cf830 2024-09-16T15:38:10.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.856+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767815ae90 con 0x7f76780cf830 2024-09-16T15:38:10.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.856+0000 7f766e000640 1 -- 172.21.15.100:0/3424904403 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670012680 con 0x7f76780cf830 2024-09-16T15:38:10.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.856+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781f7e40 con 0x7f76780cf830 2024-09-16T15:38:10.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.856+0000 7f766e000640 1 -- 172.21.15.100:0/3424904403 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7670019020 con 0x7f76780cf830 2024-09-16T15:38:10.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.856+0000 7f766e000640 1 -- 172.21.15.100:0/3424904403 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f767001bd60 con 0x7f76780cf830 2024-09-16T15:38:10.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.857+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76780b42f0 con 0x7f76780cf830 2024-09-16T15:38:10.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.859+0000 7f766e000640 1 -- 172.21.15.100:0/3424904403 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7670020020 con 0x7f76780cf830 2024-09-16T15:38:10.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.934+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0de59e14-e6eb-4c11-8c21-84177ef27af2", "id": 0} v 0) -- 0x7f76780b6040 con 0x7f76780cf830 2024-09-16T15:38:10.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.937+0000 7f766e000640 1 -- 172.21.15.100:0/3424904403 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0de59e14-e6eb-4c11-8c21-84177ef27af2", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f7670004a40 con 0x7f76780cf830 2024-09-16T15:38:10.939 INFO:teuthology.orchestra.run.smithi100.stdout:0 2024-09-16T15:38:10.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.938+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76780cf830 msgr2=0x7f7678156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:10.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.938+0000 7f767f200640 1 --2- 172.21.15.100:0/3424904403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76780cf830 0x7f7678156f90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f767000cb40 tx=0x7f767000a060 comp rx=0 tx=0).stop 2024-09-16T15:38:10.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.939+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 shutdown_connections 2024-09-16T15:38:10.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.939+0000 7f767f200640 1 --2- 172.21.15.100:0/3424904403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7678161460 0x7f76780b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:10.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.939+0000 7f767f200640 1 --2- 172.21.15.100:0/3424904403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76780d2a00 0x7f76781f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:10.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.939+0000 7f767f200640 1 --2- 172.21.15.100:0/3424904403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76780cf830 0x7f7678156f90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:10.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.939+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 >> 172.21.15.100:0/3424904403 conn(0x7f767809d0b0 msgr2=0x7f76780b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:10.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.939+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 shutdown_connections 2024-09-16T15:38:10.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:10.939+0000 7f767f200640 1 -- 172.21.15.100:0/3424904403 wait complete. 2024-09-16T15:38:10.955 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 8a091819-c752-4c9d-87dd-3df656f8eb0a 1 2024-09-16T15:38:11.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.110+0000 7f6857800640 1 -- 172.21.15.100:0/3904915138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500b0b30 msgr2=0x7f68500af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.110+0000 7f6857800640 1 --2- 172.21.15.100:0/3904915138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500b0b30 0x7f68500af320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68500b1440 tx=0x7f684802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.110+0000 7f6857800640 1 -- 172.21.15.100:0/3904915138 shutdown_connections 2024-09-16T15:38:11.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.110+0000 7f6857800640 1 --2- 172.21.15.100:0/3904915138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500b0b30 0x7f68500af320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.110+0000 7f6857800640 1 --2- 172.21.15.100:0/3904915138 >> v2:172.21.15.60:3300/0 conn(0x7f68500d2960 0x7f6850160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.110+0000 7f6857800640 1 -- 172.21.15.100:0/3904915138 >> 172.21.15.100:0/3904915138 conn(0x7f68500d1e40 msgr2=0x7f68500d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:11.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.110+0000 7f6857800640 1 -- 172.21.15.100:0/3904915138 shutdown_connections 2024-09-16T15:38:11.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.110+0000 7f6857800640 1 -- 172.21.15.100:0/3904915138 wait complete. 2024-09-16T15:38:11.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.111+0000 7f6857800640 1 Processor -- start 2024-09-16T15:38:11.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.112+0000 7f6857800640 1 -- start start 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.112+0000 7f6857800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68500b0b30 0x7f68500c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.112+0000 7f6857800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500d00b0 0x7f68500c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f6857800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68500d2960 0x7f68500c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f6857800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f68500af860 con 0x7f68500b0b30 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f6857800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68500d06e0 con 0x7f68500d00b0 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f6857800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f68500af9e0 con 0x7f68500d2960 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f684fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500d00b0 0x7f68500c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f684fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500d00b0 0x7f68500c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49490/0 (socket says 172.21.15.100:49490) 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f684fe00640 1 -- 172.21.15.100:0/1558937140 learned_addr learned my addr 172.21.15.100:0/1558937140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-16T15:38:11.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f6855800640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68500d2960 0x7f68500c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f6854e00640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68500b0b30 0x7f68500c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f684fe00640 1 -- 172.21.15.100:0/1558937140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68500d2960 msgr2=0x7f68500c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f684fe00640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68500d2960 0x7f68500c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f684fe00640 1 -- 172.21.15.100:0/1558937140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68500b0b30 msgr2=0x7f68500c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f684fe00640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68500b0b30 0x7f68500c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.113+0000 7f684fe00640 1 -- 172.21.15.100:0/1558937140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68500cbae0 con 0x7f68500d00b0 2024-09-16T15:38:11.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.114+0000 7f684fe00640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500d00b0 0x7f68500c7bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f683c00e790 tx=0x7f683c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:11.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.114+0000 7f684cc00640 1 -- 172.21.15.100:0/1558937140 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c00eea0 con 0x7f68500d00b0 2024-09-16T15:38:11.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.114+0000 7f684cc00640 1 -- 172.21.15.100:0/1558937140 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f683c00cdb0 con 0x7f68500d00b0 2024-09-16T15:38:11.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.114+0000 7f684cc00640 1 -- 172.21.15.100:0/1558937140 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c010690 con 0x7f68500d00b0 2024-09-16T15:38:11.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.114+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6850158800 con 0x7f68500d00b0 2024-09-16T15:38:11.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.114+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68500c9390 con 0x7f68500d00b0 2024-09-16T15:38:11.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.114+0000 7f684cc00640 1 -- 172.21.15.100:0/1558937140 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f683c010830 con 0x7f68500d00b0 2024-09-16T15:38:11.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.114+0000 7f684cc00640 1 -- 172.21.15.100:0/1558937140 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f683c0217a0 con 0x7f68500d00b0 2024-09-16T15:38:11.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.115+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685006a2d0 con 0x7f68500d00b0 2024-09-16T15:38:11.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.117+0000 7f684cc00640 1 -- 172.21.15.100:0/1558937140 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f685006a2d0 con 0x7f68500d00b0 2024-09-16T15:38:11.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.192+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8a091819-c752-4c9d-87dd-3df656f8eb0a", "id": 1} v 0) -- 0x7f68500af320 con 0x7f68500d00b0 2024-09-16T15:38:11.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.196+0000 7f684cc00640 1 -- 172.21.15.100:0/1558937140 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8a091819-c752-4c9d-87dd-3df656f8eb0a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f683c010cb0 con 0x7f68500d00b0 2024-09-16T15:38:11.198 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:38:11.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.197+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500d00b0 msgr2=0x7f68500c7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.197+0000 7f6857800640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500d00b0 0x7f68500c7bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f683c00e790 tx=0x7f683c00ec50 comp rx=0 tx=0).stop 2024-09-16T15:38:11.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.198+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 shutdown_connections 2024-09-16T15:38:11.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.198+0000 7f6857800640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68500d2960 0x7f68500c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.198+0000 7f6857800640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68500d00b0 0x7f68500c7bf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.198+0000 7f6857800640 1 --2- 172.21.15.100:0/1558937140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68500b0b30 0x7f68500c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.198+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 >> 172.21.15.100:0/1558937140 conn(0x7f68500d1e40 msgr2=0x7f685006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:11.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.198+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 shutdown_connections 2024-09-16T15:38:11.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.198+0000 7f6857800640 1 -- 172.21.15.100:0/1558937140 wait complete. 2024-09-16T15:38:11.214 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new cd71dc0c-6837-4b8c-8380-6000e5cff822 2 2024-09-16T15:38:11.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.366+0000 7fa37de00640 1 -- 172.21.15.100:0/956326419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa378161a10 msgr2=0x7fa3780d0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.366+0000 7fa37de00640 1 --2- 172.21.15.100:0/956326419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa378161a10 0x7fa3780d0e90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa3780b56e0 tx=0x7fa36002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.366+0000 7fa37de00640 1 -- 172.21.15.100:0/956326419 shutdown_connections 2024-09-16T15:38:11.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.366+0000 7fa37de00640 1 --2- 172.21.15.100:0/956326419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa378161a10 0x7fa3780d0e90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.366+0000 7fa37de00640 1 --2- 172.21.15.100:0/956326419 >> v2:172.21.15.60:3300/0 conn(0x7fa378161630 0x7fa3780b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.366+0000 7fa37de00640 1 -- 172.21.15.100:0/956326419 >> 172.21.15.100:0/956326419 conn(0x7fa3780d3fe0 msgr2=0x7fa3780d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:11.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.366+0000 7fa37de00640 1 -- 172.21.15.100:0/956326419 shutdown_connections 2024-09-16T15:38:11.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.366+0000 7fa37de00640 1 -- 172.21.15.100:0/956326419 wait complete. 2024-09-16T15:38:11.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.368+0000 7fa37de00640 1 Processor -- start 2024-09-16T15:38:11.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.368+0000 7fa37de00640 1 -- start start 2024-09-16T15:38:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa37de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3780b2d10 0x7fa378155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa37de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa378161630 0x7fa3780b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa37de00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa378161a10 0x7fa3780b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa37de00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780d2770 con 0x7fa378161630 2024-09-16T15:38:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa37de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780d25f0 con 0x7fa3780b2d10 2024-09-16T15:38:11.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa37de00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa3780d13d0 con 0x7fa378161a10 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa376a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3780b2d10 0x7fa378155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa36fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa378161630 0x7fa3780b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa377400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa378161a10 0x7fa3780b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa376a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3780b2d10 0x7fa378155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49500/0 (socket says 172.21.15.100:49500) 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.369+0000 7fa376a00640 1 -- 172.21.15.100:0/3522858461 learned_addr learned my addr 172.21.15.100:0/3522858461 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36fe00640 1 -- 172.21.15.100:0/3522858461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa378161a10 msgr2=0x7fa3780b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36fe00640 1 --2- 172.21.15.100:0/3522858461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa378161a10 0x7fa3780b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36fe00640 1 -- 172.21.15.100:0/3522858461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3780b2d10 msgr2=0x7fa378155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36fe00640 1 --2- 172.21.15.100:0/3522858461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3780b2d10 0x7fa378155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36fe00640 1 -- 172.21.15.100:0/3522858461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3781f4fb0 con 0x7fa378161630 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36fe00640 1 --2- 172.21.15.100:0/3522858461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa378161630 0x7fa3780b8680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa360031000 tx=0x7fa360033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:11.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36f400640 1 -- 172.21.15.100:0/3522858461 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360031d60 con 0x7fa378161630 2024-09-16T15:38:11.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36f400640 1 -- 172.21.15.100:0/3522858461 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa360033e10 con 0x7fa378161630 2024-09-16T15:38:11.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3780ca810 con 0x7fa378161630 2024-09-16T15:38:11.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.370+0000 7fa36f400640 1 -- 172.21.15.100:0/3522858461 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360005280 con 0x7fa378161630 2024-09-16T15:38:11.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.371+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3780b8160 con 0x7fa378161630 2024-09-16T15:38:11.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.371+0000 7fa36f400640 1 -- 172.21.15.100:0/3522858461 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3600047a0 con 0x7fa378161630 2024-09-16T15:38:11.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.371+0000 7fa36f400640 1 -- 172.21.15.100:0/3522858461 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fa3600333f0 con 0x7fa378161630 2024-09-16T15:38:11.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.372+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3780b37b0 con 0x7fa378161630 2024-09-16T15:38:11.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.374+0000 7fa36f400640 1 -- 172.21.15.100:0/3522858461 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa3781f4fb0 con 0x7fa378161630 2024-09-16T15:38:11.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.447+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cd71dc0c-6837-4b8c-8380-6000e5cff822", "id": 2} v 0) -- 0x7fa37815e650 con 0x7fa378161630 2024-09-16T15:38:11.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.449+0000 7fa36f400640 1 -- 172.21.15.100:0/3522858461 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cd71dc0c-6837-4b8c-8380-6000e5cff822", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fa36003f030 con 0x7fa378161630 2024-09-16T15:38:11.450 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-09-16T15:38:11.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.450+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa378161630 msgr2=0x7fa3780b8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.450+0000 7fa37de00640 1 --2- 172.21.15.100:0/3522858461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa378161630 0x7fa3780b8680 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa360031000 tx=0x7fa360033c20 comp rx=0 tx=0).stop 2024-09-16T15:38:11.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.450+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 shutdown_connections 2024-09-16T15:38:11.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.450+0000 7fa37de00640 1 --2- 172.21.15.100:0/3522858461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa378161a10 0x7fa3780b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.450+0000 7fa37de00640 1 --2- 172.21.15.100:0/3522858461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa378161630 0x7fa3780b8680 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.450+0000 7fa37de00640 1 --2- 172.21.15.100:0/3522858461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3780b2d10 0x7fa378155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.450+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 >> 172.21.15.100:0/3522858461 conn(0x7fa3780d3fe0 msgr2=0x7fa37815c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:11.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.450+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 shutdown_connections 2024-09-16T15:38:11.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.451+0000 7fa37de00640 1 -- 172.21.15.100:0/3522858461 wait complete. 2024-09-16T15:38:11.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4 3 2024-09-16T15:38:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.620+0000 7fb30fe00640 1 -- 172.21.15.100:0/2496283605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb308145750 msgr2=0x7fb308073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.620+0000 7fb30fe00640 1 --2- 172.21.15.100:0/2496283605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb308145750 0x7fb308073440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009860 tx=0x7fb2fc02a510 comp rx=0 tx=0).stop 2024-09-16T15:38:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.620+0000 7fb30fe00640 1 -- 172.21.15.100:0/2496283605 shutdown_connections 2024-09-16T15:38:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.620+0000 7fb30fe00640 1 --2- 172.21.15.100:0/2496283605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb308145750 0x7fb308073440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.620+0000 7fb30fe00640 1 -- 172.21.15.100:0/2496283605 >> 172.21.15.100:0/2496283605 conn(0x7fb3080b1840 msgr2=0x7fb3080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.620+0000 7fb30fe00640 1 -- 172.21.15.100:0/2496283605 shutdown_connections 2024-09-16T15:38:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.620+0000 7fb30fe00640 1 -- 172.21.15.100:0/2496283605 wait complete. 2024-09-16T15:38:11.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.622+0000 7fb30fe00640 1 Processor -- start 2024-09-16T15:38:11.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.622+0000 7fb30fe00640 1 -- start start 2024-09-16T15:38:11.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb30806efa0 0x7fb308141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080cccd0 0x7fb3081f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb308145750 0x7fb3081f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb3080ce6d0 con 0x7fb30806efa0 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3080ce550 con 0x7fb308145750 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb3080ce850 con 0x7fb3080cccd0 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30ca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080cccd0 0x7fb3081f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb30806efa0 0x7fb308141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30ca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080cccd0 0x7fb3081f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:48816/0 (socket says 172.21.15.100:48816) 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30ca00640 1 -- 172.21.15.100:0/2962213374 learned_addr learned my addr 172.21.15.100:0/2962213374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30de00640 1 --2- 172.21.15.100:0/2962213374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb308145750 0x7fb3081f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30ca00640 1 -- 172.21.15.100:0/2962213374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb308145750 msgr2=0x7fb3081f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30ca00640 1 --2- 172.21.15.100:0/2962213374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb308145750 0x7fb3081f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30ca00640 1 -- 172.21.15.100:0/2962213374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb30806efa0 msgr2=0x7fb308141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30ca00640 1 --2- 172.21.15.100:0/2962213374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb30806efa0 0x7fb308141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.623+0000 7fb30ca00640 1 -- 172.21.15.100:0/2962213374 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3081f4650 con 0x7fb3080cccd0 2024-09-16T15:38:11.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.624+0000 7fb30ca00640 1 --2- 172.21.15.100:0/2962213374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080cccd0 0x7fb3081f50c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400c490 tx=0x7fb2f400c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:11.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.624+0000 7fb305600640 1 -- 172.21.15.100:0/2962213374 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4016020 con 0x7fb3080cccd0 2024-09-16T15:38:11.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.624+0000 7fb305600640 1 -- 172.21.15.100:0/2962213374 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2f4012940 con 0x7fb3080cccd0 2024-09-16T15:38:11.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.624+0000 7fb305600640 1 -- 172.21.15.100:0/2962213374 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4012d90 con 0x7fb3080cccd0 2024-09-16T15:38:11.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.624+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3081f86d0 con 0x7fb3080cccd0 2024-09-16T15:38:11.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.624+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3081f4ba0 con 0x7fb3080cccd0 2024-09-16T15:38:11.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.624+0000 7fb305600640 1 -- 172.21.15.100:0/2962213374 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb2f4019020 con 0x7fb3080cccd0 2024-09-16T15:38:11.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.624+0000 7fb305600640 1 -- 172.21.15.100:0/2962213374 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fb2f401a3b0 con 0x7fb3080cccd0 2024-09-16T15:38:11.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.625+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3080ced00 con 0x7fb3080cccd0 2024-09-16T15:38:11.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.627+0000 7fb305600640 1 -- 172.21.15.100:0/2962213374 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb2f4020070 con 0x7fb3080cccd0 2024-09-16T15:38:11.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.701+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4", "id": 3} v 0) -- 0x7fb3081f4d70 con 0x7fb3080cccd0 2024-09-16T15:38:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.705+0000 7fb305600640 1 -- 172.21.15.100:0/2962213374 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fb2f401a980 con 0x7fb3080cccd0 2024-09-16T15:38:11.706 INFO:teuthology.orchestra.run.smithi100.stdout:3 2024-09-16T15:38:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080cccd0 msgr2=0x7fb3081f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 --2- 172.21.15.100:0/2962213374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080cccd0 0x7fb3081f50c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400c490 tx=0x7fb2f400c950 comp rx=0 tx=0).stop 2024-09-16T15:38:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 shutdown_connections 2024-09-16T15:38:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 --2- 172.21.15.100:0/2962213374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb308145750 0x7fb3081f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 --2- 172.21.15.100:0/2962213374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3080cccd0 0x7fb3081f50c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 --2- 172.21.15.100:0/2962213374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb30806efa0 0x7fb308141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 >> 172.21.15.100:0/2962213374 conn(0x7fb3080b1840 msgr2=0x7fb30806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 shutdown_connections 2024-09-16T15:38:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.706+0000 7fb30fe00640 1 -- 172.21.15.100:0/2962213374 wait complete. 2024-09-16T15:38:11.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new a60837b5-a8b6-4837-827a-53041d9b4d6b 4 2024-09-16T15:38:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.873+0000 7fd69fe00640 1 -- 172.21.15.100:0/3175367602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b3a00 msgr2=0x7fd6a00b3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.873+0000 7fd69fe00640 1 --2- 172.21.15.100:0/3175367602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b3a00 0x7fd6a00b3e90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd698009ca0 tx=0x7fd69802ab50 comp rx=0 tx=0).stop 2024-09-16T15:38:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.873+0000 7fd69fe00640 1 -- 172.21.15.100:0/3175367602 shutdown_connections 2024-09-16T15:38:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.873+0000 7fd69fe00640 1 --2- 172.21.15.100:0/3175367602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b3a00 0x7fd6a00b3e90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.873+0000 7fd69fe00640 1 --2- 172.21.15.100:0/3175367602 >> v2:172.21.15.60:3300/0 conn(0x7fd6a00b4e70 0x7fd6a00b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.873+0000 7fd69fe00640 1 -- 172.21.15.100:0/3175367602 >> 172.21.15.100:0/3175367602 conn(0x7fd6a00d3e20 msgr2=0x7fd6a00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.873+0000 7fd69fe00640 1 -- 172.21.15.100:0/3175367602 shutdown_connections 2024-09-16T15:38:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.873+0000 7fd69fe00640 1 -- 172.21.15.100:0/3175367602 wait complete. 2024-09-16T15:38:11.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.875+0000 7fd69fe00640 1 Processor -- start 2024-09-16T15:38:11.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.875+0000 7fd69fe00640 1 -- start start 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.875+0000 7fd69fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6a00b3a00 0x7fd6a00cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd69fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b4e70 0x7fd6a0158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd69fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6a00d2000 0x7fd6a00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd69fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a00b1a10 con 0x7fd6a00b3a00 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd69fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a00b1890 con 0x7fd6a00d2000 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd69fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a00b1b90 con 0x7fd6a00b4e70 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd697e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b4e70 0x7fd6a0158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd69f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6a00d2000 0x7fd6a00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd697e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b4e70 0x7fd6a0158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:48828/0 (socket says 172.21.15.100:48828) 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd697e00640 1 -- 172.21.15.100:0/1471501961 learned_addr learned my addr 172.21.15.100:0/1471501961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-16T15:38:11.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd69ea00640 1 --2- 172.21.15.100:0/1471501961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6a00b3a00 0x7fd6a00cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd697e00640 1 -- 172.21.15.100:0/1471501961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6a00d2000 msgr2=0x7fd6a00ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd697e00640 1 --2- 172.21.15.100:0/1471501961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6a00d2000 0x7fd6a00ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd697e00640 1 -- 172.21.15.100:0/1471501961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6a00b3a00 msgr2=0x7fd6a00cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd697e00640 1 --2- 172.21.15.100:0/1471501961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6a00b3a00 0x7fd6a00cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.876+0000 7fd697e00640 1 -- 172.21.15.100:0/1471501961 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a01588b0 con 0x7fd6a00b4e70 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.877+0000 7fd697e00640 1 --2- 172.21.15.100:0/1471501961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b4e70 0x7fd6a0158370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd68800ea20 tx=0x7fd68800eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.877+0000 7fd697400640 1 -- 172.21.15.100:0/1471501961 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68800cd80 con 0x7fd6a00b4e70 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.877+0000 7fd697400640 1 -- 172.21.15.100:0/1471501961 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd688010430 con 0x7fd6a00b4e70 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.877+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a01f58e0 con 0x7fd6a00b4e70 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.877+0000 7fd697400640 1 -- 172.21.15.100:0/1471501961 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688010840 con 0x7fd6a00b4e70 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.877+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a00ccb40 con 0x7fd6a00b4e70 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.877+0000 7fd697400640 1 -- 172.21.15.100:0/1471501961 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6880109e0 con 0x7fd6a00b4e70 2024-09-16T15:38:11.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.877+0000 7fd697400640 1 -- 172.21.15.100:0/1471501961 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fd688018dd0 con 0x7fd6a00b4e70 2024-09-16T15:38:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.878+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a00b3e90 con 0x7fd6a00b4e70 2024-09-16T15:38:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.880+0000 7fd697400640 1 -- 172.21.15.100:0/1471501961 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd6a01588b0 con 0x7fd6a00b4e70 2024-09-16T15:38:11.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.954+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a60837b5-a8b6-4837-827a-53041d9b4d6b", "id": 4} v 0) -- 0x7fd6a00a1d50 con 0x7fd6a00b4e70 2024-09-16T15:38:11.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.959+0000 7fd697400640 1 -- 172.21.15.100:0/1471501961 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a60837b5-a8b6-4837-827a-53041d9b4d6b", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fd688010cd0 con 0x7fd6a00b4e70 2024-09-16T15:38:11.960 INFO:teuthology.orchestra.run.smithi100.stdout:4 2024-09-16T15:38:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b4e70 msgr2=0x7fd6a0158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:11.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 --2- 172.21.15.100:0/1471501961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b4e70 0x7fd6a0158370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd68800ea20 tx=0x7fd68800eee0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 shutdown_connections 2024-09-16T15:38:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 --2- 172.21.15.100:0/1471501961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6a00d2000 0x7fd6a00ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 --2- 172.21.15.100:0/1471501961 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6a00b4e70 0x7fd6a0158370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 --2- 172.21.15.100:0/1471501961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6a00b3a00 0x7fd6a00cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 >> 172.21.15.100:0/1471501961 conn(0x7fd6a00d3e20 msgr2=0x7fd6a0161620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 shutdown_connections 2024-09-16T15:38:11.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:11.960+0000 7fd69fe00640 1 -- 172.21.15.100:0/1471501961 wait complete. 2024-09-16T15:38:11.976 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new e6b32c2e-58f9-4cca-b0fe-efb31f753ae3 5 2024-09-16T15:38:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.128+0000 7f412ea00640 1 -- 172.21.15.100:0/2101325544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280b2690 msgr2=0x7f41280b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.128+0000 7f412ea00640 1 --2- 172.21.15.100:0/2101325544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280b2690 0x7f41280b3bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4120009be0 tx=0x7f412002a9f0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.129+0000 7f412ea00640 1 -- 172.21.15.100:0/2101325544 shutdown_connections 2024-09-16T15:38:12.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.129+0000 7f412ea00640 1 --2- 172.21.15.100:0/2101325544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280b2690 0x7f41280b3bd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.129+0000 7f412ea00640 1 -- 172.21.15.100:0/2101325544 >> 172.21.15.100:0/2101325544 conn(0x7f41280d3e20 msgr2=0x7f41280d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.129+0000 7f412ea00640 1 -- 172.21.15.100:0/2101325544 shutdown_connections 2024-09-16T15:38:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.129+0000 7f412ea00640 1 -- 172.21.15.100:0/2101325544 wait complete. 2024-09-16T15:38:12.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.130+0000 7f412ea00640 1 Processor -- start 2024-09-16T15:38:12.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.130+0000 7f412ea00640 1 -- start start 2024-09-16T15:38:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412ea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41280b2690 0x7f41280ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41280b4e70 0x7f41280caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280d2000 0x7f41280cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412ea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4128169160 con 0x7f41280b4e70 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41280b43a0 con 0x7f41280d2000 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412ea00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f41281692e0 con 0x7f41280b2690 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412d600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41280b2690 0x7f41280ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280d2000 0x7f41280cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.131+0000 7f412d600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41280b2690 0x7f41280ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:48830/0 (socket says 172.21.15.100:48830) 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412d600640 1 -- 172.21.15.100:0/2405319195 learned_addr learned my addr 172.21.15.100:0/2405319195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412cc00640 1 --2- 172.21.15.100:0/2405319195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41280b4e70 0x7f41280caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412e000640 1 -- 172.21.15.100:0/2405319195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41280b2690 msgr2=0x7f41280ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412e000640 1 --2- 172.21.15.100:0/2405319195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41280b2690 0x7f41280ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412e000640 1 -- 172.21.15.100:0/2405319195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41280b4e70 msgr2=0x7f41280caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412e000640 1 --2- 172.21.15.100:0/2405319195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41280b4e70 0x7f41280caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412e000640 1 -- 172.21.15.100:0/2405319195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41280cd650 con 0x7f41280d2000 2024-09-16T15:38:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412e000640 1 --2- 172.21.15.100:0/2405319195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280d2000 0x7f41280cb430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4120009000 tx=0x7f4120030df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:12.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f4125600640 1 -- 172.21.15.100:0/2405319195 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120047070 con 0x7f41280d2000 2024-09-16T15:38:12.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f4125600640 1 -- 172.21.15.100:0/2405319195 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4120004040 con 0x7f41280d2000 2024-09-16T15:38:12.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.132+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41280cd930 con 0x7f41280d2000 2024-09-16T15:38:12.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.133+0000 7f4125600640 1 -- 172.21.15.100:0/2405319195 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412002f400 con 0x7f41280d2000 2024-09-16T15:38:12.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.133+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41280cdef0 con 0x7f41280d2000 2024-09-16T15:38:12.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.133+0000 7f4125600640 1 -- 172.21.15.100:0/2405319195 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f412004a070 con 0x7f41280d2000 2024-09-16T15:38:12.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.133+0000 7f4125600640 1 -- 172.21.15.100:0/2405319195 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f412004cce0 con 0x7f41280d2000 2024-09-16T15:38:12.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.134+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41280d23e0 con 0x7f41280d2000 2024-09-16T15:38:12.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.136+0000 7f4125600640 1 -- 172.21.15.100:0/2405319195 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4120033020 con 0x7f41280d2000 2024-09-16T15:38:12.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.209+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6b32c2e-58f9-4cca-b0fe-efb31f753ae3", "id": 5} v 0) -- 0x7f412815e5a0 con 0x7f41280d2000 2024-09-16T15:38:12.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.213+0000 7f4125600640 1 -- 172.21.15.100:0/2405319195 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6b32c2e-58f9-4cca-b0fe-efb31f753ae3", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f41200044b0 con 0x7f41280d2000 2024-09-16T15:38:12.215 INFO:teuthology.orchestra.run.smithi100.stdout:5 2024-09-16T15:38:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.214+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280d2000 msgr2=0x7f41280cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.214+0000 7f412ea00640 1 --2- 172.21.15.100:0/2405319195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280d2000 0x7f41280cb430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4120009000 tx=0x7f4120030df0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.215+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 shutdown_connections 2024-09-16T15:38:12.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.215+0000 7f412ea00640 1 --2- 172.21.15.100:0/2405319195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41280d2000 0x7f41280cb430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.215+0000 7f412ea00640 1 --2- 172.21.15.100:0/2405319195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41280b4e70 0x7f41280caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.215+0000 7f412ea00640 1 --2- 172.21.15.100:0/2405319195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41280b2690 0x7f41280ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.215+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 >> 172.21.15.100:0/2405319195 conn(0x7f41280d3e20 msgr2=0x7f41280cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.215+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 shutdown_connections 2024-09-16T15:38:12.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.215+0000 7f412ea00640 1 -- 172.21.15.100:0/2405319195 wait complete. 2024-09-16T15:38:12.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new b57116a4-56ce-4fc4-87b1-30c4dd0e4827 6 2024-09-16T15:38:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.381+0000 7fd76ac00640 1 -- 172.21.15.100:0/1151072407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 msgr2=0x7fd764073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.381+0000 7fd76ac00640 1 --2- 172.21.15.100:0/1151072407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 0x7fd764073440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd7640d0cd0 tx=0x7fd75402a560 comp rx=0 tx=0).stop 2024-09-16T15:38:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.382+0000 7fd76ac00640 1 -- 172.21.15.100:0/1151072407 shutdown_connections 2024-09-16T15:38:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.382+0000 7fd76ac00640 1 --2- 172.21.15.100:0/1151072407 >> v2:172.21.15.60:3300/0 conn(0x7fd7640cccd0 0x7fd7640cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.382+0000 7fd76ac00640 1 --2- 172.21.15.100:0/1151072407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 0x7fd764073440 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.382+0000 7fd76ac00640 1 -- 172.21.15.100:0/1151072407 >> 172.21.15.100:0/1151072407 conn(0x7fd7640b1840 msgr2=0x7fd7640b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.382+0000 7fd76ac00640 1 -- 172.21.15.100:0/1151072407 shutdown_connections 2024-09-16T15:38:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.382+0000 7fd76ac00640 1 -- 172.21.15.100:0/1151072407 wait complete. 2024-09-16T15:38:12.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.383+0000 7fd76ac00640 1 Processor -- start 2024-09-16T15:38:12.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.383+0000 7fd76ac00640 1 -- start start 2024-09-16T15:38:12.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd76ac00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd76406efa0 0x7fd7641f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd76ac00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7640cccd0 0x7fd76415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd76ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 0x7fd7641e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd76ac00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7640ce480 con 0x7fd7640cccd0 2024-09-16T15:38:12.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd76ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd7640ce300 con 0x7fd764145750 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd76ac00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd7640ce600 con 0x7fd76406efa0 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd768c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 0x7fd7641e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd763e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd76406efa0 0x7fd7641f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd768c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 0x7fd7641e5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49548/0 (socket says 172.21.15.100:49548) 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.384+0000 7fd768c00640 1 -- 172.21.15.100:0/821569080 learned_addr learned my addr 172.21.15.100:0/821569080 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd75be00640 1 --2- 172.21.15.100:0/821569080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7640cccd0 0x7fd76415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd768c00640 1 -- 172.21.15.100:0/821569080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd76406efa0 msgr2=0x7fd7641f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd768c00640 1 --2- 172.21.15.100:0/821569080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd76406efa0 0x7fd7641f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd768c00640 1 -- 172.21.15.100:0/821569080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7640cccd0 msgr2=0x7fd76415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd768c00640 1 --2- 172.21.15.100:0/821569080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7640cccd0 0x7fd76415e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd768c00640 1 -- 172.21.15.100:0/821569080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7641f5d00 con 0x7fd764145750 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd768c00640 1 --2- 172.21.15.100:0/821569080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 0x7fd7641e5f00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00eb40 tx=0x7fd75c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:12.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd760c00640 1 -- 172.21.15.100:0/821569080 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c004b20 con 0x7fd764145750 2024-09-16T15:38:12.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd760c00640 1 -- 172.21.15.100:0/821569080 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd75c004ce0 con 0x7fd764145750 2024-09-16T15:38:12.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.385+0000 7fd760c00640 1 -- 172.21.15.100:0/821569080 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c005280 con 0x7fd764145750 2024-09-16T15:38:12.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.386+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7641f4f60 con 0x7fd764145750 2024-09-16T15:38:12.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.386+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7641f5500 con 0x7fd764145750 2024-09-16T15:38:12.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.386+0000 7fd760c00640 1 -- 172.21.15.100:0/821569080 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd75c0057b0 con 0x7fd764145750 2024-09-16T15:38:12.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.386+0000 7fd760c00640 1 -- 172.21.15.100:0/821569080 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fd75c021a30 con 0x7fd764145750 2024-09-16T15:38:12.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.387+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7640cdbb0 con 0x7fd764145750 2024-09-16T15:38:12.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.389+0000 7fd760c00640 1 -- 172.21.15.100:0/821569080 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd75c021440 con 0x7fd764145750 2024-09-16T15:38:12.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.463+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b57116a4-56ce-4fc4-87b1-30c4dd0e4827", "id": 6} v 0) -- 0x7fd7640ccab0 con 0x7fd764145750 2024-09-16T15:38:12.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.467+0000 7fd760c00640 1 -- 172.21.15.100:0/821569080 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b57116a4-56ce-4fc4-87b1-30c4dd0e4827", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fd75c027030 con 0x7fd764145750 2024-09-16T15:38:12.468 INFO:teuthology.orchestra.run.smithi100.stdout:6 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 msgr2=0x7fd7641e5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 --2- 172.21.15.100:0/821569080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 0x7fd7641e5f00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00eb40 tx=0x7fd75c00c2d0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 shutdown_connections 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 --2- 172.21.15.100:0/821569080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd764145750 0x7fd7641e5f00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 --2- 172.21.15.100:0/821569080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7640cccd0 0x7fd76415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 --2- 172.21.15.100:0/821569080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd76406efa0 0x7fd7641f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 >> 172.21.15.100:0/821569080 conn(0x7fd7640b1840 msgr2=0x7fd7640b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 shutdown_connections 2024-09-16T15:38:12.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.468+0000 7fd76ac00640 1 -- 172.21.15.100:0/821569080 wait complete. 2024-09-16T15:38:12.484 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new beb26a63-23f6-424a-b72c-d48de6bc032b 7 2024-09-16T15:38:12.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.636+0000 7f435d400640 1 -- 172.21.15.100:0/2057768443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b43c0 msgr2=0x7f43580d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.636+0000 7f435d400640 1 --2- 172.21.15.100:0/2057768443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b43c0 0x7f43580d2830 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4350005fd0 tx=0x7f435002a820 comp rx=0 tx=0).stop 2024-09-16T15:38:12.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.636+0000 7f435d400640 1 -- 172.21.15.100:0/2057768443 shutdown_connections 2024-09-16T15:38:12.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.636+0000 7f435d400640 1 --2- 172.21.15.100:0/2057768443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b43c0 0x7f43580d2830 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.636+0000 7f435d400640 1 --2- 172.21.15.100:0/2057768443 >> v2:172.21.15.60:3300/0 conn(0x7f4358161460 0x7f43580d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.636+0000 7f435d400640 1 -- 172.21.15.100:0/2057768443 >> 172.21.15.100:0/2057768443 conn(0x7f435809d0b0 msgr2=0x7f43580d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:12.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.636+0000 7f435d400640 1 -- 172.21.15.100:0/2057768443 shutdown_connections 2024-09-16T15:38:12.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.637+0000 7f435d400640 1 -- 172.21.15.100:0/2057768443 wait complete. 2024-09-16T15:38:12.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.639+0000 7f435d400640 1 Processor -- start 2024-09-16T15:38:12.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.639+0000 7f435d400640 1 -- start start 2024-09-16T15:38:12.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f435d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43580b43c0 0x7f435815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f435d400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b5480 0x7f43581f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f435d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4358161460 0x7f435809db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f435d400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f43580cf0b0 con 0x7f43580b43c0 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f435d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43580cef30 con 0x7f4358161460 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f435d400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f43580cf230 con 0x7f43580b5480 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f434fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b5480 0x7f43581f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f4357e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43580b43c0 0x7f435815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f434fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b5480 0x7f43581f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:48850/0 (socket says 172.21.15.100:48850) 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f434fe00640 1 -- 172.21.15.100:0/89565185 learned_addr learned my addr 172.21.15.100:0/89565185 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.641+0000 7f435ca00640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4358161460 0x7f435809db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f434fe00640 1 -- 172.21.15.100:0/89565185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4358161460 msgr2=0x7f435809db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f434fe00640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4358161460 0x7f435809db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f434fe00640 1 -- 172.21.15.100:0/89565185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43580b43c0 msgr2=0x7f435815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f434fe00640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43580b43c0 0x7f435815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f434fe00640 1 -- 172.21.15.100:0/89565185 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43581f41a0 con 0x7f43580b5480 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f4357e00640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43580b43c0 0x7f435815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f434fe00640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b5480 0x7f43581f4c60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f434000c430 tx=0x7f434000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f4354c00640 1 -- 172.21.15.100:0/89565185 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340017070 con 0x7f43580b5480 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f4354c00640 1 -- 172.21.15.100:0/89565185 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43400035f0 con 0x7f43580b5480 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435809e3e0 con 0x7f43580b5480 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f4354c00640 1 -- 172.21.15.100:0/89565185 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340003ac0 con 0x7f43580b5480 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.642+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43581f4460 con 0x7f43580b5480 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.643+0000 7f4354c00640 1 -- 172.21.15.100:0/89565185 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f434001a070 con 0x7f43580b5480 2024-09-16T15:38:12.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.643+0000 7f4354c00640 1 -- 172.21.15.100:0/89565185 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f4340003c60 con 0x7f43580b5480 2024-09-16T15:38:12.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.643+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43580cf6e0 con 0x7f43580b5480 2024-09-16T15:38:12.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.646+0000 7f4354c00640 1 -- 172.21.15.100:0/89565185 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4340021020 con 0x7f43580b5480 2024-09-16T15:38:12.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.720+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "beb26a63-23f6-424a-b72c-d48de6bc032b", "id": 7} v 0) -- 0x7f435809e900 con 0x7f43580b5480 2024-09-16T15:38:12.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.724+0000 7f4354c00640 1 -- 172.21.15.100:0/89565185 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "beb26a63-23f6-424a-b72c-d48de6bc032b", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f434001c440 con 0x7f43580b5480 2024-09-16T15:38:12.726 INFO:teuthology.orchestra.run.smithi100.stdout:7 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b5480 msgr2=0x7f43581f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b5480 0x7f43581f4c60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f434000c430 tx=0x7f434000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 shutdown_connections 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4358161460 0x7f435809db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43580b5480 0x7f43581f4c60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 --2- 172.21.15.100:0/89565185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43580b43c0 0x7f435815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 >> 172.21.15.100:0/89565185 conn(0x7f435809d0b0 msgr2=0x7f43580d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 shutdown_connections 2024-09-16T15:38:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:12.725+0000 7f435d400640 1 -- 172.21.15.100:0/89565185 wait complete. 2024-09-16T15:38:12.741 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T15:38:12.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T15:38:12.746 INFO:tasks.ceph.osd.0:Started 2024-09-16T15:38:12.746 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T15:38:12.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T15:38:12.751 INFO:tasks.ceph.osd.1:Started 2024-09-16T15:38:12.751 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T15:38:12.751 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T15:38:12.755 INFO:tasks.ceph.osd.2:Started 2024-09-16T15:38:12.755 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T15:38:12.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T15:38:12.760 INFO:tasks.ceph.osd.3:Started 2024-09-16T15:38:12.761 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T15:38:12.761 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T15:38:12.764 INFO:tasks.ceph.osd.4:Started 2024-09-16T15:38:12.764 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T15:38:12.765 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T15:38:12.768 INFO:tasks.ceph.osd.5:Started 2024-09-16T15:38:12.768 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T15:38:12.768 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T15:38:12.773 INFO:tasks.ceph.osd.6:Started 2024-09-16T15:38:12.773 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T15:38:12.773 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T15:38:12.777 INFO:tasks.ceph.osd.7:Started 2024-09-16T15:38:12.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:12.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.984+0000 7fa718e00640 1 -- 172.21.15.60:0/171631721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f8003c10 msgr2=0x7fa6f8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.984+0000 7fa718e00640 1 --2- 172.21.15.60:0/171631721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6f8003c10 0x7fa6f8004010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa704006f80 tx=0x7fa6f8005830 comp rx=0 tx=0).stop 2024-09-16T15:38:12.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.984+0000 7fa712800640 1 -- 172.21.15.60:0/171631721 reap_dead start 2024-09-16T15:38:12.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.984+0000 7fa718e00640 1 -- 172.21.15.60:0/171631721 shutdown_connections 2024-09-16T15:38:12.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.984+0000 7fa718e00640 1 -- 172.21.15.60:0/171631721 >> 172.21.15.60:0/171631721 conn(0x7fa7140b1950 msgr2=0x7fa7140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:12.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.984+0000 7fa718e00640 1 -- 172.21.15.60:0/171631721 shutdown_connections 2024-09-16T15:38:12.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.984+0000 7fa718e00640 1 -- 172.21.15.60:0/171631721 wait complete. 2024-09-16T15:38:12.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.986+0000 7fa718e00640 1 Processor -- start 2024-09-16T15:38:12.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.986+0000 7fa718e00640 1 -- start start 2024-09-16T15:38:12.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa718e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6f8003c10 0x7fa7141564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa718e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa714158ac0 0x7fa7141f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa718e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714154b10 0x7fa714154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:12.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa718e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa714074840 con 0x7fa714158ac0 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa718e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7140746c0 con 0x7fa714154b10 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa718e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa71406f160 con 0x7fa6f8003c10 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa70be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa714158ac0 0x7fa7141f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa70be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa714158ac0 0x7fa7141f8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54214/0 (socket says 172.21.15.60:54214) 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa70be00640 1 -- 172.21.15.60:0/2768191956 learned_addr learned my addr 172.21.15.60:0/2768191956 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa711e00640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6f8003c10 0x7fa7141564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa712800640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714154b10 0x7fa714154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa70be00640 1 -- 172.21.15.60:0/2768191956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6f8003c10 msgr2=0x7fa7141564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa70be00640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6f8003c10 0x7fa7141564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa70be00640 1 -- 172.21.15.60:0/2768191956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714154b10 msgr2=0x7fa714154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa70be00640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714154b10 0x7fa714154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.987+0000 7fa70be00640 1 -- 172.21.15.60:0/2768191956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa714156a20 con 0x7fa714158ac0 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.988+0000 7fa70be00640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa714158ac0 0x7fa7141f8180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc001db0 tx=0x7fa6fc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.988+0000 7fa70a000640 1 -- 172.21.15.60:0/2768191956 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc011700 con 0x7fa714158ac0 2024-09-16T15:38:12.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.988+0000 7fa70a000640 1 -- 172.21.15.60:0/2768191956 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6fc016020 con 0x7fa714158ac0 2024-09-16T15:38:12.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.988+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7141597f0 con 0x7fa714158ac0 2024-09-16T15:38:12.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.988+0000 7fa70a000640 1 -- 172.21.15.60:0/2768191956 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc011b40 con 0x7fa714158ac0 2024-09-16T15:38:12.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.988+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa714159230 con 0x7fa714158ac0 2024-09-16T15:38:12.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.989+0000 7fa70a000640 1 -- 172.21.15.60:0/2768191956 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6fc01b7a0 con 0x7fa714158ac0 2024-09-16T15:38:12.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.989+0000 7fa70a000640 1 -- 172.21.15.60:0/2768191956 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa6fc010a90 con 0x7fa714158ac0 2024-09-16T15:38:12.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.989+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71406f6c0 con 0x7fa714158ac0 2024-09-16T15:38:12.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:12.993+0000 7fa70a000640 1 -- 172.21.15.60:0/2768191956 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa6fc020020 con 0x7fa714158ac0 2024-09-16T15:38:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.058+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7141595d0 con 0x7fa714158ac0 2024-09-16T15:38:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.059+0000 7fa70a000640 1 -- 172.21.15.60:0/2768191956 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa6fc01bd20 con 0x7fa714158ac0 2024-09-16T15:38:13.060 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:13.060 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:12.722418+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa714158ac0 msgr2=0x7fa7141f8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa714158ac0 0x7fa7141f8180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc001db0 tx=0x7fa6fc0039c0 comp rx=0 tx=0).stop 2024-09-16T15:38:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 shutdown_connections 2024-09-16T15:38:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa714154b10 0x7fa714154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa714158ac0 0x7fa7141f8180 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 --2- 172.21.15.60:0/2768191956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6f8003c10 0x7fa7141564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 >> 172.21.15.60:0/2768191956 conn(0x7fa7140b1950 msgr2=0x7fa7140af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 shutdown_connections 2024-09-16T15:38:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:13.060+0000 7fa718e00640 1 -- 172.21.15.60:0/2768191956 wait complete. 2024-09-16T15:38:13.076 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T15:38:13.076 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T15:38:13.882 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:14.023 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.022+0000 7f508ec00640 1 -- 172.21.15.60:0/2246597234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b2de0 msgr2=0x7f50880b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:14.023 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.022+0000 7f508ec00640 1 --2- 172.21.15.60:0/2246597234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b2de0 0x7f50880b3920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f508808cee0 tx=0x7f507402a5f0 comp rx=0 tx=0).stop 2024-09-16T15:38:14.024 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.022+0000 7f508ec00640 1 -- 172.21.15.60:0/2246597234 shutdown_connections 2024-09-16T15:38:14.024 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.022+0000 7f508ec00640 1 --2- 172.21.15.60:0/2246597234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b2de0 0x7f50880b3920 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:14.024 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.022+0000 7f508ec00640 1 --2- 172.21.15.60:0/2246597234 >> v2:172.21.15.100:3300/0 conn(0x7f50880d0790 0x7f50880d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:14.024 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.022+0000 7f508ec00640 1 -- 172.21.15.60:0/2246597234 >> 172.21.15.60:0/2246597234 conn(0x7f508809d110 msgr2=0x7f50880d3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:14.024 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.022+0000 7f508ec00640 1 -- 172.21.15.60:0/2246597234 shutdown_connections 2024-09-16T15:38:14.024 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.022+0000 7f508ec00640 1 -- 172.21.15.60:0/2246597234 wait complete. 2024-09-16T15:38:14.024 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.023+0000 7f508ec00640 1 Processor -- start 2024-09-16T15:38:14.024 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.024+0000 7f508ec00640 1 -- start start 2024-09-16T15:38:14.026 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.024+0000 7f508ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50880b2de0 0x7f50880cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:14.026 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.024+0000 7f508ec00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b3e60 0x7f5088160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:14.026 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.024+0000 7f508ec00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50880d0790 0x7f50881f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:14.026 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.024+0000 7f508ec00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f50880b4b00 con 0x7f50880b3e60 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.024+0000 7f508ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f50880b4980 con 0x7f50880b2de0 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.024+0000 7f508ec00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f50880cf830 con 0x7f50880d0790 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b3e60 0x7f5088160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b3e60 0x7f5088160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54232/0 (socket says 172.21.15.60:54232) 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 -- 172.21.15.60:0/3280027691 learned_addr learned my addr 172.21.15.60:0/3280027691 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508d800640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50880b2de0 0x7f50880cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508e200640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50880d0790 0x7f50881f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 -- 172.21.15.60:0/3280027691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50880d0790 msgr2=0x7f50881f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50880d0790 0x7f50881f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 -- 172.21.15.60:0/3280027691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50880b2de0 msgr2=0x7f50880cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50880b2de0 0x7f50880cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:14.027 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 -- 172.21.15.60:0/3280027691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508815f550 con 0x7f50880b3e60 2024-09-16T15:38:14.028 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.025+0000 7f508ce00640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b3e60 0x7f5088160310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5074000c00 tx=0x7f5074003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:14.029 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.026+0000 7f5085600640 1 -- 172.21.15.60:0/3280027691 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507403f070 con 0x7f50880b3e60 2024-09-16T15:38:14.029 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.026+0000 7f5085600640 1 -- 172.21.15.60:0/3280027691 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50740047a0 con 0x7f50880b3e60 2024-09-16T15:38:14.029 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.026+0000 7f5085600640 1 -- 172.21.15.60:0/3280027691 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074035030 con 0x7f50880b3e60 2024-09-16T15:38:14.029 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.026+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508815f830 con 0x7f50880b3e60 2024-09-16T15:38:14.029 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.026+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881f5660 con 0x7f50880b3e60 2024-09-16T15:38:14.029 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.027+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50880b3920 con 0x7f50880b3e60 2024-09-16T15:38:14.029 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.027+0000 7f5085600640 1 -- 172.21.15.60:0/3280027691 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 2) ==== 50665+0+0 (secure 0 0 0) 0x7f5074042070 con 0x7f50880b3e60 2024-09-16T15:38:14.030 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.028+0000 7f5085600640 1 -- 172.21.15.60:0/3280027691 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5074072fb0 con 0x7f50880b3e60 2024-09-16T15:38:14.031 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.030+0000 7f5085600640 1 -- 172.21.15.60:0/3280027691 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f507403b460 con 0x7f50880b3e60 2024-09-16T15:38:14.096 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.095+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50880b6530 con 0x7f50880b3e60 2024-09-16T15:38:14.097 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.096+0000 7f5085600640 1 -- 172.21.15.60:0/3280027691 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5074042360 con 0x7f50880b3e60 2024-09-16T15:38:14.097 INFO:teuthology.misc.health.smithi060.stdout: 2024-09-16T15:38:14.097 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":10,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:12.722418+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:14.098 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b3e60 msgr2=0x7f5088160310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:14.098 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b3e60 0x7f5088160310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5074000c00 tx=0x7f5074003820 comp rx=0 tx=0).stop 2024-09-16T15:38:14.098 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 shutdown_connections 2024-09-16T15:38:14.099 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50880d0790 0x7f50881f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:14.099 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50880b3e60 0x7f5088160310 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:14.099 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 --2- 172.21.15.60:0/3280027691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50880b2de0 0x7f50880cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:14.099 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 >> 172.21.15.60:0/3280027691 conn(0x7f508809d110 msgr2=0x7f50880b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:14.099 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 shutdown_connections 2024-09-16T15:38:14.099 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:14.097+0000 7f508ec00640 1 -- 172.21.15.60:0/3280027691 wait complete. 2024-09-16T15:38:14.110 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T15:38:14.275 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-09-16T15:38:14.274+0000 7f4b60ba8640 -1 Falling back to public interface 2024-09-16T15:38:14.305 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-09-16T15:38:14.304+0000 7fa06080f640 -1 Falling back to public interface 2024-09-16T15:38:14.309 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-09-16T15:38:14.308+0000 7f2922a10640 -1 Falling back to public interface 2024-09-16T15:38:14.312 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-09-16T15:38:14.311+0000 7f46f800d640 -1 Falling back to public interface 2024-09-16T15:38:14.317 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T15:38:14.316+0000 7fbddbfdb640 -1 Falling back to public interface 2024-09-16T15:38:14.340 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-09-16T15:38:14.339+0000 7f97a4734640 -1 Falling back to public interface 2024-09-16T15:38:14.354 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-09-16T15:38:14.353+0000 7fbf9c224640 -1 Falling back to public interface 2024-09-16T15:38:14.716 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:38:14.715+0000 7fa535400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T15:38:14.717 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:38:14.715+0000 7fa535400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T15:38:14.896 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-09-16T15:38:14.895+0000 7f7b49a19640 -1 Falling back to public interface 2024-09-16T15:38:17.907 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-09-16T15:38:17.906+0000 7f4b60ba8640 -1 osd.1 0 log_to_monitors true 2024-09-16T15:38:17.981 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-09-16T15:38:17.979+0000 7f2922a10640 -1 osd.7 0 log_to_monitors true 2024-09-16T15:38:18.084 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-09-16T15:38:18.083+0000 7fbf9c224640 -1 osd.3 0 log_to_monitors true 2024-09-16T15:38:18.317 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T15:38:18.315+0000 7fbddbfdb640 -1 osd.4 0 log_to_monitors true 2024-09-16T15:38:18.485 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-09-16T15:38:18.484+0000 7fa06080f640 -1 osd.2 0 log_to_monitors true 2024-09-16T15:38:18.520 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-09-16T15:38:18.519+0000 7f46f800d640 -1 osd.0 0 log_to_monitors true 2024-09-16T15:38:18.805 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-09-16T15:38:18.804+0000 7f97a4734640 -1 osd.5 0 log_to_monitors true 2024-09-16T15:38:19.135 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-09-16T15:38:19.134+0000 7f7b49a19640 -1 osd.6 0 log_to_monitors true 2024-09-16T15:38:20.027 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-09-16T15:38:20.025+0000 7f4b5ba00640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T15:38:20.038 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-09-16T15:38:20.037+0000 7f4b51a00640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:38:20.917 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:21.161 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.160+0000 7f2a15400640 1 -- 172.21.15.60:0/3524559829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a101597d0 msgr2=0x7f2a10159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:21.161 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.160+0000 7f2a15400640 1 --2- 172.21.15.60:0/3524559829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a101597d0 0x7f2a10159c30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2a100cf8d0 tx=0x7f2a0802b7e0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.162 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.160+0000 7f2a15400640 1 -- 172.21.15.60:0/3524559829 shutdown_connections 2024-09-16T15:38:21.162 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.160+0000 7f2a15400640 1 --2- 172.21.15.60:0/3524559829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a101597d0 0x7f2a10159c30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.162 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.160+0000 7f2a15400640 1 --2- 172.21.15.60:0/3524559829 >> v2:172.21.15.60:3300/0 conn(0x7f2a10070600 0x7f2a1015b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.162 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.160+0000 7f2a15400640 1 -- 172.21.15.60:0/3524559829 >> 172.21.15.60:0/3524559829 conn(0x7f2a100cfb80 msgr2=0x7f2a100cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:21.162 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.160+0000 7f2a15400640 1 -- 172.21.15.60:0/3524559829 shutdown_connections 2024-09-16T15:38:21.162 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.161+0000 7f2a15400640 1 -- 172.21.15.60:0/3524559829 wait complete. 2024-09-16T15:38:21.165 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.163+0000 7f2a15400640 1 Processor -- start 2024-09-16T15:38:21.165 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.164+0000 7f2a15400640 1 -- start start 2024-09-16T15:38:21.167 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.166+0000 7f2a15400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a10070600 0x7f2a10157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:21.168 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.166+0000 7f2a15400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a100cf4f0 0x7f2a100af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:21.168 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a0e800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a10070600 0x7f2a10157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:21.168 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a100cf4f0 0x7f2a100af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:21.168 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a0e800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a10070600 0x7f2a10157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56076/0 (socket says 172.21.15.60:56076) 2024-09-16T15:38:21.168 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a100cf4f0 0x7f2a100af5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44036/0 (socket says 172.21.15.60:44036) 2024-09-16T15:38:21.168 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a0e800640 1 -- 172.21.15.60:0/1832096433 learned_addr learned my addr 172.21.15.60:0/1832096433 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:21.169 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a15400640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a101597d0 0x7f2a10158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1015a500 con 0x7f2a101597d0 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1015a380 con 0x7f2a100cf4f0 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2a100ca1d0 con 0x7f2a10070600 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.167+0000 7f2a0f200640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a101597d0 0x7f2a10158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.168+0000 7f2a0e800640 1 -- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a100cf4f0 msgr2=0x7f2a100af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.168+0000 7f2a0e800640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a100cf4f0 0x7f2a100af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.168+0000 7f2a0e800640 1 -- 172.21.15.60:0/1832096433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a101597d0 msgr2=0x7f2a10158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.168+0000 7f2a0e800640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a101597d0 0x7f2a10158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.168+0000 7f2a0e800640 1 -- 172.21.15.60:0/1832096433 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a101f4da0 con 0x7f2a10070600 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.168+0000 7f2a0f200640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a101597d0 0x7f2a10158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:21.170 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.169+0000 7f2a0e800640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a10070600 0x7f2a10157e00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000c7c0 tx=0x7f2a0000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:21.171 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.169+0000 7f2a06a00640 1 -- 172.21.15.60:0/1832096433 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00003da0 con 0x7f2a10070600 2024-09-16T15:38:21.171 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.169+0000 7f2a06a00640 1 -- 172.21.15.60:0/1832096433 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a00005070 con 0x7f2a10070600 2024-09-16T15:38:21.171 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.169+0000 7f2a06a00640 1 -- 172.21.15.60:0/1832096433 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a000054a0 con 0x7f2a10070600 2024-09-16T15:38:21.171 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.169+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a101f5080 con 0x7f2a10070600 2024-09-16T15:38:21.171 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.169+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a101f5640 con 0x7f2a10070600 2024-09-16T15:38:21.172 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.171+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1015b0e0 con 0x7f2a10070600 2024-09-16T15:38:21.174 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.173+0000 7f2a06a00640 1 -- 172.21.15.60:0/1832096433 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 4) ==== 101231+0+0 (secure 0 0 0) 0x7f2a00005640 con 0x7f2a10070600 2024-09-16T15:38:21.176 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.175+0000 7f2a06a00640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29ec078810 0x7f29ec07acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:21.176 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.175+0000 7f2a06a00640 1 -- 172.21.15.60:0/1832096433 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f2a000998a0 con 0x7f2a10070600 2024-09-16T15:38:21.177 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.176+0000 7f2a0de00640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29ec078810 0x7f29ec07acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:21.178 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.177+0000 7f2a0de00640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29ec078810 0x7f29ec07acd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f29f80095f0 tx=0x7f29f8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:21.185 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.184+0000 7f2a06a00640 1 -- 172.21.15.60:0/1832096433 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2a00062680 con 0x7f2a10070600 2024-09-16T15:38:21.495 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-09-16T15:38:21.494+0000 7fa05b200640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T15:38:21.503 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-09-16T15:38:21.502+0000 7fa051200640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:38:21.509 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-09-16T15:38:21.508+0000 7f46f2a00640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T15:38:21.513 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-09-16T15:38:21.512+0000 7fbf97400640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T15:38:21.517 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-09-16T15:38:21.516+0000 7f46e8a00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:38:21.522 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-09-16T15:38:21.521+0000 7fbf8ca00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:38:21.544 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.543+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a1015a6d0 con 0x7f2a10070600 2024-09-16T15:38:21.545 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.544+0000 7f2a06a00640 1 -- 172.21.15.60:0/1832096433 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7543 (secure 0 0 0) 0x7f2a000670b0 con 0x7f2a10070600 2024-09-16T15:38:21.545 INFO:teuthology.misc.health.smithi060.stdout: 2024-09-16T15:38:21.545 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":14,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:20.974265+0000","last_up_change":"2024-09-16T15:38:20.974265+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29ec078810 msgr2=0x7f29ec07acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29ec078810 0x7f29ec07acd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f29f80095f0 tx=0x7f29f8005f90 comp rx=0 tx=0).stop 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a10070600 msgr2=0x7f2a10157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a10070600 0x7f2a10157e00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000c7c0 tx=0x7f2a0000cc80 comp rx=0 tx=0).stop 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 shutdown_connections 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29ec078810 0x7f29ec07acd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a101597d0 0x7f2a10158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a100cf4f0 0x7f2a100af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.548 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.546+0000 7f2a15400640 1 --2- 172.21.15.60:0/1832096433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a10070600 0x7f2a10157e00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:21.549 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.547+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 >> 172.21.15.60:0/1832096433 conn(0x7f2a100cfb80 msgr2=0x7f2a1015c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:21.549 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.547+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 shutdown_connections 2024-09-16T15:38:21.549 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:21.547+0000 7f2a15400640 1 -- 172.21.15.60:0/1832096433 wait complete. 2024-09-16T15:38:21.560 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-16T15:38:21.854 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T15:38:21.853+0000 7fbdd7800640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T15:38:21.869 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-09-16T15:38:21.867+0000 7fbdcce00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:38:21.924 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-09-16T15:38:21.923+0000 7f291d200640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T15:38:21.931 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-09-16T15:38:21.929+0000 7f2913200640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:38:21.957 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-09-16T15:38:21.955+0000 7f979f600640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T15:38:21.966 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-09-16T15:38:21.963+0000 7f9795600640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:38:22.358 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-09-16T15:38:22.357+0000 7f7b44200640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T15:38:22.364 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-09-16T15:38:22.363+0000 7f7b3a200640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:38:28.366 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:28.519 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.517+0000 7ff306200640 1 -- 172.21.15.60:0/1142018949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e4003c10 msgr2=0x7ff2e4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:28.519 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.517+0000 7ff306200640 1 --2- 172.21.15.60:0/1142018949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e4003c10 0x7ff2e4004010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009b80 tx=0x7ff2e4005830 comp rx=0 tx=0).stop 2024-09-16T15:38:28.519 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.517+0000 7ff2ffe00640 1 -- 172.21.15.60:0/1142018949 reap_dead start 2024-09-16T15:38:28.519 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.517+0000 7ff306200640 1 -- 172.21.15.60:0/1142018949 shutdown_connections 2024-09-16T15:38:28.519 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.517+0000 7ff306200640 1 -- 172.21.15.60:0/1142018949 >> 172.21.15.60:0/1142018949 conn(0x7ff3000cfb80 msgr2=0x7ff3000cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:28.519 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.518+0000 7ff306200640 1 -- 172.21.15.60:0/1142018949 shutdown_connections 2024-09-16T15:38:28.519 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.518+0000 7ff306200640 1 -- 172.21.15.60:0/1142018949 wait complete. 2024-09-16T15:38:28.520 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.519+0000 7ff306200640 1 Processor -- start 2024-09-16T15:38:28.520 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.519+0000 7ff306200640 1 -- start start 2024-09-16T15:38:28.522 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.520+0000 7ff306200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4003c10 0x7ff3000b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:28.522 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.520+0000 7ff306200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000b4e60 0x7ff3000b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:28.522 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.520+0000 7ff306200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300153c10 0x7ff300154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:28.522 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.520+0000 7ff306200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000ae9d0 con 0x7ff3000b4e60 2024-09-16T15:38:28.522 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.520+0000 7ff306200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000ae850 con 0x7ff300153c10 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.520+0000 7ff306200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff300073e40 con 0x7ff2e4003c10 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.520+0000 7ff2f6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000b4e60 0x7ff3000b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.520+0000 7ff2f6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000b4e60 0x7ff3000b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57376/0 (socket says 172.21.15.60:57376) 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2f6a00640 1 -- 172.21.15.60:0/1241027349 learned_addr learned my addr 172.21.15.60:0/1241027349 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2ffe00640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300153c10 0x7ff300154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2ff400640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4003c10 0x7ff3000b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2f6a00640 1 -- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4003c10 msgr2=0x7ff3000b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2f6a00640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4003c10 0x7ff3000b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2f6a00640 1 -- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300153c10 msgr2=0x7ff300154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2f6a00640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300153c10 0x7ff300154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2f6a00640 1 -- 172.21.15.60:0/1241027349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3000b3060 con 0x7ff3000b4e60 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2ffe00640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300153c10 0x7ff300154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:28.523 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.521+0000 7ff2f6a00640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000b4e60 0x7ff3000b52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8001eb0 tx=0x7ff2e800ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:28.524 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.522+0000 7ff2f7e00640 1 -- 172.21.15.60:0/1241027349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8015070 con 0x7ff3000b4e60 2024-09-16T15:38:28.524 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.522+0000 7ff2f7e00640 1 -- 172.21.15.60:0/1241027349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2e8010980 con 0x7ff3000b4e60 2024-09-16T15:38:28.525 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.522+0000 7ff2f7e00640 1 -- 172.21.15.60:0/1241027349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8010d90 con 0x7ff3000b4e60 2024-09-16T15:38:28.525 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.522+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff300150850 con 0x7ff3000b4e60 2024-09-16T15:38:28.525 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.522+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff300150e10 con 0x7ff3000b4e60 2024-09-16T15:38:28.525 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.523+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff300074360 con 0x7ff3000b4e60 2024-09-16T15:38:28.525 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.524+0000 7ff2f7e00640 1 -- 172.21.15.60:0/1241027349 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff2e8018070 con 0x7ff3000b4e60 2024-09-16T15:38:28.526 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.525+0000 7ff2f7e00640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff2dc078850 0x7ff2dc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:28.526 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.525+0000 7ff2f7e00640 1 -- 172.21.15.60:0/1241027349 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7ff2e809aa20 con 0x7ff3000b4e60 2024-09-16T15:38:28.526 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.525+0000 7ff2ff400640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff2dc078850 0x7ff2dc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:28.527 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.526+0000 7ff2ff400640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff2dc078850 0x7ff2dc07ad10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0035000 tx=0x7ff2f002d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:28.531 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.530+0000 7ff2f7e00640 1 -- 172.21.15.60:0/1241027349 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff2e80633d0 con 0x7ff3000b4e60 2024-09-16T15:38:28.730 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.729+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff300072770 con 0x7ff3000b4e60 2024-09-16T15:38:28.731 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.730+0000 7ff2f7e00640 1 -- 172.21.15.60:0/1241027349 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12896 (secure 0 0 0) 0x7ff2e8062cd0 con 0x7ff3000b4e60 2024-09-16T15:38:28.731 INFO:teuthology.misc.health.smithi060.stdout: 2024-09-16T15:38:28.731 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":19,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:26.518675+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff2dc078850 msgr2=0x7ff2dc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff2dc078850 0x7ff2dc07ad10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0035000 tx=0x7ff2f002d040 comp rx=0 tx=0).stop 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000b4e60 msgr2=0x7ff3000b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000b4e60 0x7ff3000b52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8001eb0 tx=0x7ff2e800ea00 comp rx=0 tx=0).stop 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 shutdown_connections 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff2dc078850 0x7ff2dc07ad10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff300153c10 0x7ff300154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3000b4e60 0x7ff3000b52e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 --2- 172.21.15.60:0/1241027349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e4003c10 0x7ff3000b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 >> 172.21.15.60:0/1241027349 conn(0x7ff3000cfb80 msgr2=0x7ff3000c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:28.734 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 shutdown_connections 2024-09-16T15:38:28.735 INFO:teuthology.misc.health.smithi060.stderr:2024-09-16T15:38:28.732+0000 7ff306200640 1 -- 172.21.15.60:0/1241027349 wait complete. 2024-09-16T15:38:28.746 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T15:38:28.747 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T15:38:28.747 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:38:28.747 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:38:28.751 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:38:28.751 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:38:28.751 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:38:28.754 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:38:28.755 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:38:28.755 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:38:28.759 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:38:28.759 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:38:28.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:38:28.763 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:38:28.763 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-16T15:38:28.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:38:28.881 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:38:28.883 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:38:28.898 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:38:28.899 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:38:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.933+0000 7fa7de200640 1 -- 172.21.15.60:0/1608863958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7bc005210 msgr2=0x7fa7bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.933+0000 7fa7de200640 1 --2- 172.21.15.60:0/1608863958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7bc005210 0x7fa7bc00b9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009ed0 tx=0x7fa7bc005830 comp rx=0 tx=0).stop 2024-09-16T15:38:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.933+0000 7fa7de200640 1 -- 172.21.15.60:0/1608863958 shutdown_connections 2024-09-16T15:38:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.933+0000 7fa7de200640 1 --2- 172.21.15.60:0/1608863958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7bc005210 0x7fa7bc00b9c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.933+0000 7fa7de200640 1 -- 172.21.15.60:0/1608863958 >> 172.21.15.60:0/1608863958 conn(0x7fa7d80b1950 msgr2=0x7fa7d80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.933+0000 7fa7de200640 1 -- 172.21.15.60:0/1608863958 shutdown_connections 2024-09-16T15:38:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.933+0000 7fa7de200640 1 -- 172.21.15.60:0/1608863958 wait complete. 2024-09-16T15:38:28.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.935+0000 7fa7de200640 1 Processor -- start 2024-09-16T15:38:28.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.935+0000 7fa7de200640 1 -- start start 2024-09-16T15:38:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7de200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7bc005210 0x7fa7d809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7de200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8137db0 0x7fa7d814b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7de200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80ad600 0x7fa7d80cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7de200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d806fbe0 con 0x7fa7d80ad600 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7de200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d806fa60 con 0x7fa7d8137db0 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7de200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d806fd60 con 0x7fa7bc005210 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80ad600 0x7fa7d80cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8137db0 0x7fa7d814b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7bc005210 0x7fa7d809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7bc005210 0x7fa7d809aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:41304/0 (socket says 172.21.15.60:41304) 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.936+0000 7fa7d7400640 1 -- 172.21.15.60:0/184147667 learned_addr learned my addr 172.21.15.60:0/184147667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cea00640 1 -- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7bc005210 msgr2=0x7fa7d809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cea00640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7bc005210 0x7fa7d809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cea00640 1 -- 172.21.15.60:0/184147667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80ad600 msgr2=0x7fa7d80cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cea00640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80ad600 0x7fa7d80cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cea00640 1 -- 172.21.15.60:0/184147667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d8138250 con 0x7fa7d8137db0 2024-09-16T15:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7d7e00640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80ad600 0x7fa7d80cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cea00640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8137db0 0x7fa7d814b800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0003d80 tx=0x7fa7c0001800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cfe00640 1 -- 172.21.15.60:0/184147667 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0015070 con 0x7fa7d8137db0 2024-09-16T15:38:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cfe00640 1 -- 172.21.15.60:0/184147667 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7c0011860 con 0x7fa7d8137db0 2024-09-16T15:38:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d8138530 con 0x7fa7d8137db0 2024-09-16T15:38:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7cfe00640 1 -- 172.21.15.60:0/184147667 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0011cb0 con 0x7fa7d8137db0 2024-09-16T15:38:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.937+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d8138af0 con 0x7fa7d8137db0 2024-09-16T15:38:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.938+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d80cc970 con 0x7fa7d8137db0 2024-09-16T15:38:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.939+0000 7fa7cfe00640 1 -- 172.21.15.60:0/184147667 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa7c0018070 con 0x7fa7d8137db0 2024-09-16T15:38:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.940+0000 7fa7cfe00640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa7b4078880 0x7fa7b407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.940+0000 7fa7cfe00640 1 -- 172.21.15.60:0/184147667 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa7c009b2e0 con 0x7fa7d8137db0 2024-09-16T15:38:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.941+0000 7fa7d7400640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa7b4078880 0x7fa7b407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:28.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.941+0000 7fa7d7400640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa7b4078880 0x7fa7b407ad40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8000c00 tx=0x7fa7c80068a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:28.946+0000 7fa7cfe00640 1 -- 172.21.15.60:0/184147667 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa7c0063530 con 0x7fa7d8137db0 2024-09-16T15:38:29.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.183+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7d80cd430 con 0x7fa7d8137db0 2024-09-16T15:38:29.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.184+0000 7fa7cfe00640 1 -- 172.21.15.60:0/184147667 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fa7c0067f60 con 0x7fa7d8137db0 2024-09-16T15:38:29.185 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:29.185 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"btime":"2024-09-16T15:38:28:889067+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T15:38:29.185 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa7b4078880 msgr2=0x7fa7b407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa7b4078880 0x7fa7b407ad40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8000c00 tx=0x7fa7c80068a0 comp rx=0 tx=0).stop 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8137db0 msgr2=0x7fa7d814b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8137db0 0x7fa7d814b800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0003d80 tx=0x7fa7c0001800 comp rx=0 tx=0).stop 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 shutdown_connections 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa7b4078880 0x7fa7b407ad40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7d80ad600 0x7fa7d80cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d8137db0 0x7fa7d814b800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 --2- 172.21.15.60:0/184147667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7bc005210 0x7fa7d809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 >> 172.21.15.60:0/184147667 conn(0x7fa7d80b1950 msgr2=0x7fa7d815c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 shutdown_connections 2024-09-16T15:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:29.186+0000 7fa7de200640 1 -- 172.21.15.60:0/184147667 wait complete. 2024-09-16T15:38:29.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:29.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:31.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 -- 172.21.15.60:0/553747148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5006cb70 msgr2=0x7fea5015cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 --2- 172.21.15.60:0/553747148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5006cb70 0x7fea5015cd60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea5006fa90 tx=0x7fea3802a550 comp rx=0 tx=0).stop 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 -- 172.21.15.60:0/553747148 shutdown_connections 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 --2- 172.21.15.60:0/553747148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea50070f50 0x7fea50071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 --2- 172.21.15.60:0/553747148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea5006cb70 0x7fea5015cd60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 --2- 172.21.15.60:0/553747148 >> v2:172.21.15.100:3300/0 conn(0x7fea500ce310 0x7fea50160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 -- 172.21.15.60:0/553747148 >> 172.21.15.60:0/553747148 conn(0x7fea500b2b90 msgr2=0x7fea500b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 -- 172.21.15.60:0/553747148 shutdown_connections 2024-09-16T15:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.376+0000 7fea56600640 1 -- 172.21.15.60:0/553747148 wait complete. 2024-09-16T15:38:31.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.378+0000 7fea56600640 1 Processor -- start 2024-09-16T15:38:31.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.378+0000 7fea56600640 1 -- start start 2024-09-16T15:38:31.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.379+0000 7fea56600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5006cb70 0x7fea500ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:31.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.379+0000 7fea56600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea50070f50 0x7fea50151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:31.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.379+0000 7fea56600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea500ce310 0x7fea500c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:31.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.379+0000 7fea56600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fea500ccd10 con 0x7fea500ce310 2024-09-16T15:38:31.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.379+0000 7fea56600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea500ce6f0 con 0x7fea50070f50 2024-09-16T15:38:31.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.379+0000 7fea56600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fea500cce90 con 0x7fea5006cb70 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.379+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea500ce310 0x7fea500c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.379+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea500ce310 0x7fea500c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39324/0 (socket says 172.21.15.60:39324) 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea4fe00640 1 -- 172.21.15.60:0/3485510988 learned_addr learned my addr 172.21.15.60:0/3485510988 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea4f400640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5006cb70 0x7fea500ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea46a00640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea50070f50 0x7fea50151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea4fe00640 1 -- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5006cb70 msgr2=0x7fea500ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea4fe00640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5006cb70 0x7fea500ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea4fe00640 1 -- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea50070f50 msgr2=0x7fea50151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea4fe00640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea50070f50 0x7fea50151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea4fe00640 1 -- 172.21.15.60:0/3485510988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea500cf2e0 con 0x7fea500ce310 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea46a00640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea50070f50 0x7fea50151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:31.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.380+0000 7fea4fe00640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea500ce310 0x7fea500c7d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea4800eaa0 tx=0x7fea4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:31.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.381+0000 7fea47e00640 1 -- 172.21.15.60:0/3485510988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4800cdb0 con 0x7fea500ce310 2024-09-16T15:38:31.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.381+0000 7fea47e00640 1 -- 172.21.15.60:0/3485510988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea48004c10 con 0x7fea500ce310 2024-09-16T15:38:31.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.381+0000 7fea47e00640 1 -- 172.21.15.60:0/3485510988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48005020 con 0x7fea500ce310 2024-09-16T15:38:31.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.381+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea500cf5c0 con 0x7fea500ce310 2024-09-16T15:38:31.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.381+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea500ceff0 con 0x7fea500ce310 2024-09-16T15:38:31.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.382+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea50071350 con 0x7fea500ce310 2024-09-16T15:38:31.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.383+0000 7fea47e00640 1 -- 172.21.15.60:0/3485510988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fea480051c0 con 0x7fea500ce310 2024-09-16T15:38:31.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.384+0000 7fea47e00640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fea2c078850 0x7fea2c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:31.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.384+0000 7fea47e00640 1 -- 172.21.15.60:0/3485510988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fea4809af50 con 0x7fea500ce310 2024-09-16T15:38:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.385+0000 7fea4f400640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fea2c078850 0x7fea2c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.385+0000 7fea4f400640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fea2c078850 0x7fea2c07ad10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea4000bad0 tx=0x7fea4000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.389+0000 7fea47e00640 1 -- 172.21.15.60:0/3485510988 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fea48016030 con 0x7fea500ce310 2024-09-16T15:38:31.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.627+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5015ceb0 con 0x7fea500ce310 2024-09-16T15:38:31.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.627+0000 7fea47e00640 1 -- 172.21.15.60:0/3485510988 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7fea480632b0 con 0x7fea500ce310 2024-09-16T15:38:31.628 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:31.628 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"btime":"2024-09-16T15:38:29:890710+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4402,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T15:38:31.629 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2024-09-16T15:38:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.629+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fea2c078850 msgr2=0x7fea2c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.629+0000 7fea56600640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fea2c078850 0x7fea2c07ad10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea4000bad0 tx=0x7fea4000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.629+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea500ce310 msgr2=0x7fea500c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.629+0000 7fea56600640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea500ce310 0x7fea500c7d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea4800eaa0 tx=0x7fea4800ef60 comp rx=0 tx=0).stop 2024-09-16T15:38:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.630+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 shutdown_connections 2024-09-16T15:38:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.630+0000 7fea56600640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fea2c078850 0x7fea2c07ad10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.630+0000 7fea56600640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea500ce310 0x7fea500c7d10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.630+0000 7fea56600640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea50070f50 0x7fea50151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.630+0000 7fea56600640 1 --2- 172.21.15.60:0/3485510988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea5006cb70 0x7fea500ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.630+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 >> 172.21.15.60:0/3485510988 conn(0x7fea500b2b90 msgr2=0x7fea500cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.630+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 shutdown_connections 2024-09-16T15:38:31.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.630+0000 7fea56600640 1 -- 172.21.15.60:0/3485510988 wait complete. 2024-09-16T15:38:31.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:38:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.810+0000 7f7112e00640 1 -- 172.21.15.60:0/4269085556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b5370 msgr2=0x7f710c0b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.810+0000 7f7112e00640 1 --2- 172.21.15.60:0/4269085556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b5370 0x7f710c0b3bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f710c0d4ea0 tx=0x7f70f802a510 comp rx=0 tx=0).stop 2024-09-16T15:38:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.810+0000 7f7112e00640 1 -- 172.21.15.60:0/4269085556 shutdown_connections 2024-09-16T15:38:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.810+0000 7f7112e00640 1 --2- 172.21.15.60:0/4269085556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b5370 0x7f710c0b3bc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.810+0000 7f7112e00640 1 -- 172.21.15.60:0/4269085556 >> 172.21.15.60:0/4269085556 conn(0x7f710c0d3df0 msgr2=0x7f710c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.810+0000 7f7112e00640 1 -- 172.21.15.60:0/4269085556 shutdown_connections 2024-09-16T15:38:31.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.810+0000 7f7112e00640 1 -- 172.21.15.60:0/4269085556 wait complete. 2024-09-16T15:38:31.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.812+0000 7f7112e00640 1 Processor -- start 2024-09-16T15:38:31.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.812+0000 7f7112e00640 1 -- start start 2024-09-16T15:38:31.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7112e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b4100 0x7f710c0d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:31.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7112e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f710c0b5370 0x7f710c14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:31.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7112e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f710c161440 0x7f710c1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:31.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7112e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f710c0b2210 con 0x7f710c0b4100 2024-09-16T15:38:31.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7112e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f710c0b2090 con 0x7f710c161440 2024-09-16T15:38:31.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7112e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f710c0b2390 con 0x7f710c0b5370 2024-09-16T15:38:31.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7111a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b4100 0x7f710c0d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:31.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7111a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b4100 0x7f710c0d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39334/0 (socket says 172.21.15.60:39334) 2024-09-16T15:38:31.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7111a00640 1 -- 172.21.15.60:0/3763951723 learned_addr learned my addr 172.21.15.60:0/3763951723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:31.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7111000640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f710c0b5370 0x7f710c14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:31.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.814+0000 7f7112400640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f710c161440 0x7f710c1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7111a00640 1 -- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f710c0b5370 msgr2=0x7f710c14f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7111a00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f710c0b5370 0x7f710c14f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7111a00640 1 -- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f710c161440 msgr2=0x7f710c1f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7111a00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f710c161440 0x7f710c1f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7111a00640 1 -- 172.21.15.60:0/3763951723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c15baf0 con 0x7f710c0b4100 2024-09-16T15:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7112400640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f710c161440 0x7f710c1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7111000640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f710c0b5370 0x7f710c14f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7111a00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b4100 0x7f710c0d1140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f710000c400 tx=0x7f710000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7109600640 1 -- 172.21.15.60:0/3763951723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100017070 con 0x7f710c0b4100 2024-09-16T15:38:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7109600640 1 -- 172.21.15.60:0/3763951723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71000053c0 con 0x7f710c0b4100 2024-09-16T15:38:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c15ac70 con 0x7f710c0b4100 2024-09-16T15:38:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7109600640 1 -- 172.21.15.60:0/3763951723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100003850 con 0x7f710c0b4100 2024-09-16T15:38:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.815+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c0d1680 con 0x7f710c0b4100 2024-09-16T15:38:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.816+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710c0b2840 con 0x7f710c0b4100 2024-09-16T15:38:31.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.818+0000 7f7109600640 1 -- 172.21.15.60:0/3763951723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f710001a070 con 0x7f710c0b4100 2024-09-16T15:38:31.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.819+0000 7f7109600640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f70ec078850 0x7f70ec07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:31.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.819+0000 7f7109600640 1 -- 172.21.15.60:0/3763951723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f7100099940 con 0x7f710c0b4100 2024-09-16T15:38:31.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.819+0000 7f7111000640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f70ec078850 0x7f70ec07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:31.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.820+0000 7f7111000640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f70ec078850 0x7f70ec07ad10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70f8034000 tx=0x7f70f8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:31.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:31.824+0000 7f7109600640 1 -- 172.21.15.60:0/3763951723 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f71000622f0 con 0x7f710c0b4100 2024-09-16T15:38:32.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.023+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f710c06e900 con 0x7f710c0b4100 2024-09-16T15:38:32.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.024+0000 7f7109600640 1 -- 172.21.15.60:0/3763951723 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f7100061bf0 con 0x7f710c0b4100 2024-09-16T15:38:32.025 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:32.025 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:38:32.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:38:32.025 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:38:32.025 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:38:32.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:38:32.025 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:38:32.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f70ec078850 msgr2=0x7f70ec07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f70ec078850 0x7f70ec07ad10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70f8034000 tx=0x7f70f8004730 comp rx=0 tx=0).stop 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b4100 msgr2=0x7f710c0d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b4100 0x7f710c0d1140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f710000c400 tx=0x7f710000c8c0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 shutdown_connections 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f70ec078850 0x7f70ec07ad10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f710c161440 0x7f710c1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f710c0b5370 0x7f710c14f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 --2- 172.21.15.60:0/3763951723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f710c0b4100 0x7f710c0d1140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 >> 172.21.15.60:0/3763951723 conn(0x7f710c0d3df0 msgr2=0x7f710c160350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 shutdown_connections 2024-09-16T15:38:32.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.026+0000 7f7112e00640 1 -- 172.21.15.60:0/3763951723 wait complete. 2024-09-16T15:38:32.043 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:38:32.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:38:32.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.207+0000 7f5df8a00640 1 -- 172.21.15.60:0/3660306394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df4074460 msgr2=0x7f5df40cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:32.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.207+0000 7f5df8a00640 1 --2- 172.21.15.60:0/3660306394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df4074460 0x7f5df40cc5b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005fd0 tx=0x7f5dec02aa90 comp rx=0 tx=0).stop 2024-09-16T15:38:32.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.208+0000 7f5df8a00640 1 -- 172.21.15.60:0/3660306394 shutdown_connections 2024-09-16T15:38:32.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.208+0000 7f5df8a00640 1 --2- 172.21.15.60:0/3660306394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df4074460 0x7f5df40cc5b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.208+0000 7f5df8a00640 1 --2- 172.21.15.60:0/3660306394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df415bd60 0x7f5df40cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.208+0000 7f5df8a00640 1 -- 172.21.15.60:0/3660306394 >> 172.21.15.60:0/3660306394 conn(0x7f5df40b18e0 msgr2=0x7f5df40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.208+0000 7f5df8a00640 1 -- 172.21.15.60:0/3660306394 shutdown_connections 2024-09-16T15:38:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.208+0000 7f5df8a00640 1 -- 172.21.15.60:0/3660306394 wait complete. 2024-09-16T15:38:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.209+0000 7f5df8a00640 1 Processor -- start 2024-09-16T15:38:32.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.209+0000 7f5df8a00640 1 -- start start 2024-09-16T15:38:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.210+0000 7f5df8a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df406e910 0x7f5df41475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.210+0000 7f5df8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df4074460 0x7f5df4156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.210+0000 7f5df8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df415bd60 0x7f5df41572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.210+0000 7f5df8a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40ccc70 con 0x7f5df415bd60 2024-09-16T15:38:32.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.210+0000 7f5df8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40ccaf0 con 0x7f5df4074460 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.210+0000 7f5df8a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5df40ccdf0 con 0x7f5df406e910 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.210+0000 7f5df2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df415bd60 0x7f5df41572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.210+0000 7f5df2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df415bd60 0x7f5df41572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39352/0 (socket says 172.21.15.60:39352) 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5df2800640 1 -- 172.21.15.60:0/1320395394 learned_addr learned my addr 172.21.15.60:0/1320395394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5debe00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df4074460 0x7f5df4156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5df1e00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df406e910 0x7f5df41475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5df2800640 1 -- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df406e910 msgr2=0x7f5df41475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5df2800640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df406e910 0x7f5df41475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5df2800640 1 -- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df4074460 msgr2=0x7f5df4156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5df2800640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df4074460 0x7f5df4156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5df2800640 1 -- 172.21.15.60:0/1320395394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df41f4f90 con 0x7f5df415bd60 2024-09-16T15:38:32.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.211+0000 7f5df2800640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df415bd60 0x7f5df41572f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5dec006010 tx=0x7f5dec003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.212+0000 7f5dea000640 1 -- 172.21.15.60:0/1320395394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec046070 con 0x7f5df415bd60 2024-09-16T15:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.212+0000 7f5dea000640 1 -- 172.21.15.60:0/1320395394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dec003e50 con 0x7f5df415bd60 2024-09-16T15:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.212+0000 7f5dea000640 1 -- 172.21.15.60:0/1320395394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0044b0 con 0x7f5df415bd60 2024-09-16T15:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.212+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df41f5230 con 0x7f5df415bd60 2024-09-16T15:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.212+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df41f57f0 con 0x7f5df415bd60 2024-09-16T15:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.213+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df4069360 con 0x7f5df415bd60 2024-09-16T15:38:32.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.214+0000 7f5dea000640 1 -- 172.21.15.60:0/1320395394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5dec049070 con 0x7f5df415bd60 2024-09-16T15:38:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.215+0000 7f5dea000640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5dd0078850 0x7f5dd007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.215+0000 7f5dea000640 1 -- 172.21.15.60:0/1320395394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f5dec0bb4e0 con 0x7f5df415bd60 2024-09-16T15:38:32.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.215+0000 7f5df1e00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5dd0078850 0x7f5dd007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:32.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.216+0000 7f5df1e00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5dd0078850 0x7f5dd007ad10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5df41f8340 tx=0x7f5de400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:32.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.223+0000 7f5dea000640 1 -- 172.21.15.60:0/1320395394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5dec032030 con 0x7f5df415bd60 2024-09-16T15:38:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.419+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5df40cbcc0 con 0x7f5df415bd60 2024-09-16T15:38:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.905+0000 7f5dea000640 1 -- 172.21.15.60:0/1320395394 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f5dec0837b0 con 0x7f5df415bd60 2024-09-16T15:38:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:38:32.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5dd0078850 msgr2=0x7f5dd007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:32.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5dd0078850 0x7f5dd007ad10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5df41f8340 tx=0x7f5de400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df415bd60 msgr2=0x7f5df41572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df415bd60 0x7f5df41572f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5dec006010 tx=0x7f5dec003ce0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 shutdown_connections 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5dd0078850 0x7f5dd007ad10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5df415bd60 0x7f5df41572f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5df4074460 0x7f5df4156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 --2- 172.21.15.60:0/1320395394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5df406e910 0x7f5df41475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 >> 172.21.15.60:0/1320395394 conn(0x7f5df40b18e0 msgr2=0x7f5df40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.908+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 shutdown_connections 2024-09-16T15:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:32.909+0000 7f5df8a00640 1 -- 172.21.15.60:0/1320395394 wait complete. 2024-09-16T15:38:32.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:38:33.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.096+0000 7ff6ad800640 1 -- 172.21.15.60:0/236043561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a8161760 msgr2=0x7ff6a80ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:33.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.096+0000 7ff6ad800640 1 --2- 172.21.15.60:0/236043561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a8161760 0x7ff6a80ce4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff698009830 tx=0x7ff69802a510 comp rx=0 tx=0).stop 2024-09-16T15:38:33.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.096+0000 7ff6ad800640 1 -- 172.21.15.60:0/236043561 shutdown_connections 2024-09-16T15:38:33.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.096+0000 7ff6ad800640 1 --2- 172.21.15.60:0/236043561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6a80cec60 0x7ff6a80cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.096+0000 7ff6ad800640 1 --2- 172.21.15.60:0/236043561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a8161760 0x7ff6a80ce4a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.096+0000 7ff6ad800640 1 -- 172.21.15.60:0/236043561 >> 172.21.15.60:0/236043561 conn(0x7ff6a80b1840 msgr2=0x7ff6a80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:33.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.096+0000 7ff6ad800640 1 -- 172.21.15.60:0/236043561 shutdown_connections 2024-09-16T15:38:33.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.096+0000 7ff6ad800640 1 -- 172.21.15.60:0/236043561 wait complete. 2024-09-16T15:38:33.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.098+0000 7ff6ad800640 1 Processor -- start 2024-09-16T15:38:33.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.098+0000 7ff6ad800640 1 -- start start 2024-09-16T15:38:33.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6ad800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6a80af190 0x7ff6a8147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:33.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6ad800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a80cec60 0x7ff6a80b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:33.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6ad800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a8161760 0x7ff6a80b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6ad800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8162310 con 0x7ff6a8161760 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6ad800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a8162190 con 0x7ff6a80cec60 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6ad800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a8162490 con 0x7ff6a80af190 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a8161760 0x7ff6a80b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a8161760 0x7ff6a80b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39366/0 (socket says 172.21.15.60:39366) 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a7400640 1 -- 172.21.15.60:0/3066254575 learned_addr learned my addr 172.21.15.60:0/3066254575 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a6a00640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6a80af190 0x7ff6a8147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a6000640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a80cec60 0x7ff6a80b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a7400640 1 -- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6a80af190 msgr2=0x7ff6a8147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a7400640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6a80af190 0x7ff6a8147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a7400640 1 -- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a80cec60 msgr2=0x7ff6a80b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a7400640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a80cec60 0x7ff6a80b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.099+0000 7ff6a7400640 1 -- 172.21.15.60:0/3066254575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a8158670 con 0x7ff6a8161760 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.100+0000 7ff6a6000640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a80cec60 0x7ff6a80b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.100+0000 7ff6a7400640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a8161760 0x7ff6a80b0fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000eb10 tx=0x7ff6a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.100+0000 7ff69ea00640 1 -- 172.21.15.60:0/3066254575 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0004b20 con 0x7ff6a8161760 2024-09-16T15:38:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.100+0000 7ff69ea00640 1 -- 172.21.15.60:0/3066254575 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6a0004ce0 con 0x7ff6a8161760 2024-09-16T15:38:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.100+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a8158950 con 0x7ff6a8161760 2024-09-16T15:38:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.100+0000 7ff69ea00640 1 -- 172.21.15.60:0/3066254575 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0005280 con 0x7ff6a8161760 2024-09-16T15:38:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.100+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a8158f10 con 0x7ff6a8161760 2024-09-16T15:38:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.101+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a81629a0 con 0x7ff6a8161760 2024-09-16T15:38:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.103+0000 7ff69ea00640 1 -- 172.21.15.60:0/3066254575 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff6a0005420 con 0x7ff6a8161760 2024-09-16T15:38:33.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.103+0000 7ff69ea00640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff688078850 0x7ff68807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.104+0000 7ff69ea00640 1 -- 172.21.15.60:0/3066254575 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7ff6a009b520 con 0x7ff6a8161760 2024-09-16T15:38:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.104+0000 7ff6a6a00640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff688078850 0x7ff68807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:33.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.105+0000 7ff6a6a00640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff688078850 0x7ff68807ad10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff698034000 tx=0x7ff698004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:33.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.109+0000 7ff69ea00640 1 -- 172.21.15.60:0/3066254575 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff6a0063c50 con 0x7ff6a8161760 2024-09-16T15:38:33.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.302+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff6a8073430 con 0x7ff6a8161760 2024-09-16T15:38:33.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.912+0000 7ff69ea00640 1 -- 172.21.15.60:0/3066254575 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7ff6a0063550 con 0x7ff6a8161760 2024-09-16T15:38:33.914 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.916+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff688078850 msgr2=0x7ff68807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.916+0000 7ff6ad800640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff688078850 0x7ff68807ad10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff698034000 tx=0x7ff698004730 comp rx=0 tx=0).stop 2024-09-16T15:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.916+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a8161760 msgr2=0x7ff6a80b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.916+0000 7ff6ad800640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a8161760 0x7ff6a80b0fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000eb10 tx=0x7ff6a000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.917+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 shutdown_connections 2024-09-16T15:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.917+0000 7ff6ad800640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff688078850 0x7ff68807ad10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.917+0000 7ff6ad800640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6a8161760 0x7ff6a80b0fd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.917+0000 7ff6ad800640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6a80cec60 0x7ff6a80b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.917+0000 7ff6ad800640 1 --2- 172.21.15.60:0/3066254575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6a80af190 0x7ff6a8147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:33.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.917+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 >> 172.21.15.60:0/3066254575 conn(0x7ff6a80b1840 msgr2=0x7ff6a8074090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:33.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.918+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 shutdown_connections 2024-09-16T15:38:33.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:33.918+0000 7ff6ad800640 1 -- 172.21.15.60:0/3066254575 wait complete. 2024-09-16T15:38:33.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:38:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.119+0000 7f31a3e00640 1 -- 172.21.15.60:0/419737097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31a4161460 msgr2=0x7f31a40d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.119+0000 7f31a3e00640 1 --2- 172.21.15.60:0/419737097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31a4161460 0x7f31a40d22f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3194005fd0 tx=0x7f319402a540 comp rx=0 tx=0).stop 2024-09-16T15:38:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.119+0000 7f31a3e00640 1 -- 172.21.15.60:0/419737097 shutdown_connections 2024-09-16T15:38:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.119+0000 7f31a3e00640 1 --2- 172.21.15.60:0/419737097 >> v2:172.21.15.100:3300/0 conn(0x7f31a40b43c0 0x7f31a40d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.119+0000 7f31a3e00640 1 --2- 172.21.15.60:0/419737097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31a4161460 0x7f31a40d22f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.119+0000 7f31a3e00640 1 -- 172.21.15.60:0/419737097 >> 172.21.15.60:0/419737097 conn(0x7f31a409d0b0 msgr2=0x7f31a40d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.119+0000 7f31a3e00640 1 -- 172.21.15.60:0/419737097 shutdown_connections 2024-09-16T15:38:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.119+0000 7f31a3e00640 1 -- 172.21.15.60:0/419737097 wait complete. 2024-09-16T15:38:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.121+0000 7f31a3e00640 1 Processor -- start 2024-09-16T15:38:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.121+0000 7f31a3e00640 1 -- start start 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a40b43c0 0x7f31a414bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31a40b5480 0x7f31a40b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31a4161460 0x7f31a40b9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40cf0b0 con 0x7f31a4161460 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40cef30 con 0x7f31a40b43c0 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f31a40cf230 con 0x7f31a40b5480 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31a4161460 0x7f31a40b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31a4161460 0x7f31a40b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39374/0 (socket says 172.21.15.60:39374) 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3400640 1 -- 172.21.15.60:0/3945825986 learned_addr learned my addr 172.21.15.60:0/3945825986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a2000640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31a40b5480 0x7f31a40b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a2a00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a40b43c0 0x7f31a414bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3400640 1 -- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31a40b5480 msgr2=0x7f31a40b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3400640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31a40b5480 0x7f31a40b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3400640 1 -- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a40b43c0 msgr2=0x7f31a414bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3400640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a40b43c0 0x7f31a414bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.122+0000 7f31a3400640 1 -- 172.21.15.60:0/3945825986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a414c460 con 0x7f31a4161460 2024-09-16T15:38:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.123+0000 7f31a3400640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31a4161460 0x7f31a40b9380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f319c00eab0 tx=0x7f319c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.123+0000 7f319aa00640 1 -- 172.21.15.60:0/3945825986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c00cea0 con 0x7f31a4161460 2024-09-16T15:38:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.123+0000 7f319aa00640 1 -- 172.21.15.60:0/3945825986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f319c004c10 con 0x7f31a4161460 2024-09-16T15:38:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.123+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a41f8a30 con 0x7f31a4161460 2024-09-16T15:38:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.123+0000 7f319aa00640 1 -- 172.21.15.60:0/3945825986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c005020 con 0x7f31a4161460 2024-09-16T15:38:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.123+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a41f8ff0 con 0x7f31a4161460 2024-09-16T15:38:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.124+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a40cf750 con 0x7f31a4161460 2024-09-16T15:38:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.126+0000 7f319aa00640 1 -- 172.21.15.60:0/3945825986 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f319c0051c0 con 0x7f31a4161460 2024-09-16T15:38:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.127+0000 7f319aa00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3184078850 0x7f318407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.127+0000 7f319aa00640 1 -- 172.21.15.60:0/3945825986 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f319c09b630 con 0x7f31a4161460 2024-09-16T15:38:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.128+0000 7f31a2a00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3184078850 0x7f318407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.128+0000 7f31a2a00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3184078850 0x7f318407ad10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3194033000 tx=0x7f3194004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.132+0000 7f319aa00640 1 -- 172.21.15.60:0/3945825986 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f319c0636a0 con 0x7f31a4161460 2024-09-16T15:38:34.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.367+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f31a406aaf0 con 0x7f31a4161460 2024-09-16T15:38:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.915+0000 7f319aa00640 1 -- 172.21.15.60:0/3945825986 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f319c0680d0 con 0x7f31a4161460 2024-09-16T15:38:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T15:38:34.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.920+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3184078850 msgr2=0x7f318407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:34.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.920+0000 7f31a3e00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3184078850 0x7f318407ad10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3194033000 tx=0x7f3194004730 comp rx=0 tx=0).stop 2024-09-16T15:38:34.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.920+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31a4161460 msgr2=0x7f31a40b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:34.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.920+0000 7f31a3e00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31a4161460 0x7f31a40b9380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f319c00eab0 tx=0x7f319c00ef70 comp rx=0 tx=0).stop 2024-09-16T15:38:34.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.920+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 shutdown_connections 2024-09-16T15:38:34.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.920+0000 7f31a3e00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3184078850 0x7f318407ad10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:34.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.920+0000 7f31a3e00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31a4161460 0x7f31a40b9380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:34.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.921+0000 7f31a3e00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31a40b5480 0x7f31a40b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:34.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.921+0000 7f31a3e00640 1 --2- 172.21.15.60:0/3945825986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31a40b43c0 0x7f31a414bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:34.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.921+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 >> 172.21.15.60:0/3945825986 conn(0x7f31a409d0b0 msgr2=0x7f31a415f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.926+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 shutdown_connections 2024-09-16T15:38:34.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:34.929+0000 7f31a3e00640 1 -- 172.21.15.60:0/3945825986 wait complete. 2024-09-16T15:38:34.962 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.122+0000 7f9895a00640 1 -- 172.21.15.60:0/359496717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98901612d0 msgr2=0x7f98900b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.122+0000 7f9895a00640 1 --2- 172.21.15.60:0/359496717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98901612d0 0x7f98900b4e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9880005fd0 tx=0x7f988002a540 comp rx=0 tx=0).stop 2024-09-16T15:38:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.123+0000 7f9895a00640 1 -- 172.21.15.60:0/359496717 shutdown_connections 2024-09-16T15:38:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.123+0000 7f9895a00640 1 --2- 172.21.15.60:0/359496717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98900d4980 0x7f98900d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.123+0000 7f9895a00640 1 --2- 172.21.15.60:0/359496717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98901612d0 0x7f98900b4e00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.123+0000 7f9895a00640 1 -- 172.21.15.60:0/359496717 >> 172.21.15.60:0/359496717 conn(0x7f98900d3df0 msgr2=0x7f98900d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.123+0000 7f9895a00640 1 -- 172.21.15.60:0/359496717 shutdown_connections 2024-09-16T15:38:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.123+0000 7f9895a00640 1 -- 172.21.15.60:0/359496717 wait complete. 2024-09-16T15:38:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.124+0000 7f9895a00640 1 Processor -- start 2024-09-16T15:38:35.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.125+0000 7f9895a00640 1 -- start start 2024-09-16T15:38:35.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.125+0000 7f9895a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98900d4980 0x7f98900c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.125+0000 7f9895a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98901612d0 0x7f989006d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.125+0000 7f9895a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98901616b0 0x7f98901578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.125+0000 7f9895a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f98900b1c00 con 0x7f98900d4980 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.125+0000 7f9895a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98900b1a80 con 0x7f98901612d0 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.125+0000 7f9895a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f98900b1d80 con 0x7f98901616b0 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98900d4980 0x7f98900c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98900d4980 0x7f98900c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39392/0 (socket says 172.21.15.60:39392) 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 -- 172.21.15.60:0/355689941 learned_addr learned my addr 172.21.15.60:0/355689941 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988f400640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98901612d0 0x7f989006d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f9895000640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98901616b0 0x7f98901578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 -- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98901616b0 msgr2=0x7f98901578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98901616b0 0x7f98901578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 -- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98901612d0 msgr2=0x7f989006d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98901612d0 0x7f989006d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 -- 172.21.15.60:0/355689941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989006de30 con 0x7f98900d4980 2024-09-16T15:38:35.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.126+0000 7f988fe00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98900d4980 0x7f98900c8620 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9880009850 tx=0x7f98800038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:35.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.127+0000 7f9887e00640 1 -- 172.21.15.60:0/355689941 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988003f070 con 0x7f98900d4980 2024-09-16T15:38:35.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.127+0000 7f9887e00640 1 -- 172.21.15.60:0/355689941 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98800047a0 con 0x7f98900d4980 2024-09-16T15:38:35.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.127+0000 7f9887e00640 1 -- 172.21.15.60:0/355689941 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880035030 con 0x7f98900d4980 2024-09-16T15:38:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.127+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989006e0d0 con 0x7f98900d4980 2024-09-16T15:38:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.127+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989015bae0 con 0x7f98900d4980 2024-09-16T15:38:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.128+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98900b4e00 con 0x7f98900d4980 2024-09-16T15:38:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.129+0000 7f9887e00640 1 -- 172.21.15.60:0/355689941 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9880042070 con 0x7f98900d4980 2024-09-16T15:38:35.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.130+0000 7f9887e00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9870078850 0x7f987007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.130+0000 7f9887e00640 1 -- 172.21.15.60:0/355689941 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f98800bb720 con 0x7f98900d4980 2024-09-16T15:38:35.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.130+0000 7f988f400640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9870078850 0x7f987007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.131+0000 7f988f400640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9870078850 0x7f987007ad10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f98900ce5c0 tx=0x7f987c00b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.134+0000 7f9887e00640 1 -- 172.21.15.60:0/355689941 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9880083730 con 0x7f98900d4980 2024-09-16T15:38:35.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.334+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98900d09a0 con 0x7f98900d4980 2024-09-16T15:38:35.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.335+0000 7f9887e00640 1 -- 172.21.15.60:0/355689941 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16678 (secure 0 0 0) 0x7f9880088160 con 0x7f98900d4980 2024-09-16T15:38:35.336 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:35.337 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:34.368817+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:35.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.337+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9870078850 msgr2=0x7f987007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.337+0000 7f9895a00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9870078850 0x7f987007ad10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f98900ce5c0 tx=0x7f987c00b390 comp rx=0 tx=0).stop 2024-09-16T15:38:35.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.337+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98900d4980 msgr2=0x7f98900c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.337+0000 7f9895a00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98900d4980 0x7f98900c8620 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9880009850 tx=0x7f98800038a0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.338+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 shutdown_connections 2024-09-16T15:38:35.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.338+0000 7f9895a00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9870078850 0x7f987007ad10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.338+0000 7f9895a00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98901616b0 0x7f98901578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.338+0000 7f9895a00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98901612d0 0x7f989006d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.338+0000 7f9895a00640 1 --2- 172.21.15.60:0/355689941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98900d4980 0x7f98900c8620 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.338+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 >> 172.21.15.60:0/355689941 conn(0x7f98900d3df0 msgr2=0x7f98900ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:35.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.338+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 shutdown_connections 2024-09-16T15:38:35.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.338+0000 7f9895a00640 1 -- 172.21.15.60:0/355689941 wait complete. 2024-09-16T15:38:35.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:35.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.514+0000 7fec7f200640 1 -- 172.21.15.60:0/3082940541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec800b5370 msgr2=0x7fec800b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.514+0000 7fec7f200640 1 --2- 172.21.15.60:0/3082940541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec800b5370 0x7fec800b3bc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec68009860 tx=0x7fec6802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.514+0000 7fec7f200640 1 -- 172.21.15.60:0/3082940541 shutdown_connections 2024-09-16T15:38:35.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.514+0000 7fec7f200640 1 --2- 172.21.15.60:0/3082940541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec800b5370 0x7fec800b3bc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.514+0000 7fec7f200640 1 -- 172.21.15.60:0/3082940541 >> 172.21.15.60:0/3082940541 conn(0x7fec800d3df0 msgr2=0x7fec800d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:35.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.514+0000 7fec7f200640 1 -- 172.21.15.60:0/3082940541 shutdown_connections 2024-09-16T15:38:35.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.514+0000 7fec7f200640 1 -- 172.21.15.60:0/3082940541 wait complete. 2024-09-16T15:38:35.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.516+0000 7fec7f200640 1 Processor -- start 2024-09-16T15:38:35.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.516+0000 7fec7f200640 1 -- start start 2024-09-16T15:38:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec800b4100 0x7fec801f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec800b5370 0x7fec80154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80161440 0x7fec80157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fec800b2210 con 0x7fec800b5370 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec800b2090 con 0x7fec80161440 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec800b2390 con 0x7fec800b4100 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec77e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec800b5370 0x7fec80154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec77e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec800b5370 0x7fec80154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39402/0 (socket says 172.21.15.60:39402) 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec77e00640 1 -- 172.21.15.60:0/2602665438 learned_addr learned my addr 172.21.15.60:0/2602665438 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7de00640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec800b4100 0x7fec801f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7e800640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80161440 0x7fec80157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec77e00640 1 -- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec800b4100 msgr2=0x7fec801f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec77e00640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec800b4100 0x7fec801f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec77e00640 1 -- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80161440 msgr2=0x7fec80157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec77e00640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80161440 0x7fec80157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec77e00640 1 -- 172.21.15.60:0/2602665438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec80155600 con 0x7fec800b5370 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.517+0000 7fec7e800640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80161440 0x7fec80157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.518+0000 7fec77e00640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec800b5370 0x7fec80154c50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fec68009980 tx=0x7fec68003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.518+0000 7fec76000640 1 -- 172.21.15.60:0/2602665438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68005c90 con 0x7fec800b5370 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.518+0000 7fec76000640 1 -- 172.21.15.60:0/2602665438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec68005020 con 0x7fec800b5370 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.518+0000 7fec76000640 1 -- 172.21.15.60:0/2602665438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68005410 con 0x7fec800b5370 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.518+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec801558e0 con 0x7fec800b5370 2024-09-16T15:38:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.518+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec80155ea0 con 0x7fec800b5370 2024-09-16T15:38:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.519+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec800b27b0 con 0x7fec800b5370 2024-09-16T15:38:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.520+0000 7fec76000640 1 -- 172.21.15.60:0/2602665438 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fec68039040 con 0x7fec800b5370 2024-09-16T15:38:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.522+0000 7fec76000640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fec5c078850 0x7fec5c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.522+0000 7fec76000640 1 -- 172.21.15.60:0/2602665438 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fec680bb8f0 con 0x7fec800b5370 2024-09-16T15:38:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.522+0000 7fec7de00640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fec5c078850 0x7fec5c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.523+0000 7fec7de00640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fec5c078850 0x7fec5c07ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fec70001120 tx=0x7fec70009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.526+0000 7fec76000640 1 -- 172.21.15.60:0/2602665438 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fec68040030 con 0x7fec800b5370 2024-09-16T15:38:35.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.723+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec800760b0 con 0x7fec800b5370 2024-09-16T15:38:35.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.724+0000 7fec76000640 1 -- 172.21.15.60:0/2602665438 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16678 (secure 0 0 0) 0x7fec68083f50 con 0x7fec800b5370 2024-09-16T15:38:35.725 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:35.725 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:34.368817+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:35.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fec5c078850 msgr2=0x7fec5c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fec5c078850 0x7fec5c07ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fec70001120 tx=0x7fec70009040 comp rx=0 tx=0).stop 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec800b5370 msgr2=0x7fec80154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec800b5370 0x7fec80154c50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fec68009980 tx=0x7fec68003c10 comp rx=0 tx=0).stop 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 shutdown_connections 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fec5c078850 0x7fec5c07ad10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec80161440 0x7fec80157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec800b5370 0x7fec80154c50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 --2- 172.21.15.60:0/2602665438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec800b4100 0x7fec801f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 >> 172.21.15.60:0/2602665438 conn(0x7fec800d3df0 msgr2=0x7fec8006ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 shutdown_connections 2024-09-16T15:38:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.726+0000 7fec7f200640 1 -- 172.21.15.60:0/2602665438 wait complete. 2024-09-16T15:38:35.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:38:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.906+0000 7f67a2200640 1 -- 172.21.15.60:0/864802520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c15f260 msgr2=0x7f679c072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.906+0000 7f67a2200640 1 --2- 172.21.15.60:0/864802520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c15f260 0x7f679c072340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f678c005fd0 tx=0x7f678c02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.906+0000 7f67a2200640 1 -- 172.21.15.60:0/864802520 shutdown_connections 2024-09-16T15:38:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.906+0000 7f67a2200640 1 --2- 172.21.15.60:0/864802520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c15f260 0x7f679c072340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.906+0000 7f67a2200640 1 -- 172.21.15.60:0/864802520 >> 172.21.15.60:0/864802520 conn(0x7f679c0d1bf0 msgr2=0x7f679c0d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.906+0000 7f67a2200640 1 -- 172.21.15.60:0/864802520 shutdown_connections 2024-09-16T15:38:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.906+0000 7f67a2200640 1 -- 172.21.15.60:0/864802520 wait complete. 2024-09-16T15:38:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.907+0000 7f67a2200640 1 Processor -- start 2024-09-16T15:38:35.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.908+0000 7f67a2200640 1 -- start start 2024-09-16T15:38:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.908+0000 7f67a2200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c0d0400 0x7f679c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.908+0000 7f67a2200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f679c15f260 0x7f679c09d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679c15f640 0x7f679c06dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a2200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f679c0af230 con 0x7f679c0d0400 2024-09-16T15:38:35.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f679c0af0b0 con 0x7f679c15f640 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a2200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f679c0af3b0 con 0x7f679c15f260 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c0d0400 0x7f679c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c0d0400 0x7f679c15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39420/0 (socket says 172.21.15.60:39420) 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a0e00640 1 -- 172.21.15.60:0/2772141108 learned_addr learned my addr 172.21.15.60:0/2772141108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f6793e00640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f679c15f260 0x7f679c09d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a1800640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679c15f640 0x7f679c06dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a0e00640 1 -- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f679c15f260 msgr2=0x7f679c09d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a0e00640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f679c15f260 0x7f679c09d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a0e00640 1 -- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679c15f640 msgr2=0x7f679c06dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a0e00640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679c15f640 0x7f679c06dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a0e00640 1 -- 172.21.15.60:0/2772141108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c15ea40 con 0x7f679c0d0400 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f6793e00640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f679c15f260 0x7f679c09d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.909+0000 7f67a1800640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679c15f640 0x7f679c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.910+0000 7f67a0e00640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c0d0400 0x7f679c15e500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f678c006010 tx=0x7f678c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.910+0000 7f6799600640 1 -- 172.21.15.60:0/2772141108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c03f070 con 0x7f679c0d0400 2024-09-16T15:38:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.910+0000 7f6799600640 1 -- 172.21.15.60:0/2772141108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f678c004890 con 0x7f679c0d0400 2024-09-16T15:38:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.910+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679c06e640 con 0x7f679c0d0400 2024-09-16T15:38:35.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.910+0000 7f6799600640 1 -- 172.21.15.60:0/2772141108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c035030 con 0x7f679c0d0400 2024-09-16T15:38:35.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.910+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679c09bbc0 con 0x7f679c0d0400 2024-09-16T15:38:35.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.911+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679c072340 con 0x7f679c0d0400 2024-09-16T15:38:35.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.912+0000 7f6799600640 1 -- 172.21.15.60:0/2772141108 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f678c042070 con 0x7f679c0d0400 2024-09-16T15:38:35.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.914+0000 7f6799600640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6778078850 0x7f677807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:35.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.914+0000 7f6799600640 1 -- 172.21.15.60:0/2772141108 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f678c0bb8e0 con 0x7f679c0d0400 2024-09-16T15:38:35.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.914+0000 7f6793e00640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6778078850 0x7f677807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:35.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.915+0000 7f6793e00640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6778078850 0x7f677807ad10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6784007fd0 tx=0x7f6784009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:35.918+0000 7f6799600640 1 -- 172.21.15.60:0/2772141108 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f678c083eb0 con 0x7f679c0d0400 2024-09-16T15:38:36.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.155+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f679c0b7b70 con 0x7f679c0d0400 2024-09-16T15:38:36.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.937+0000 7f6799600640 1 -- 172.21.15.60:0/2772141108 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f678c0837b0 con 0x7f679c0d0400 2024-09-16T15:38:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.940+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6778078850 msgr2=0x7f677807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.940+0000 7f67a2200640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6778078850 0x7f677807ad10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6784007fd0 tx=0x7f6784009040 comp rx=0 tx=0).stop 2024-09-16T15:38:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.940+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c0d0400 msgr2=0x7f679c15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.940+0000 7f67a2200640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c0d0400 0x7f679c15e500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f678c006010 tx=0x7f678c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:38:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.941+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 shutdown_connections 2024-09-16T15:38:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.941+0000 7f67a2200640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6778078850 0x7f677807ad10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.941+0000 7f67a2200640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f679c15f640 0x7f679c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.941+0000 7f67a2200640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f679c15f260 0x7f679c09d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.941+0000 7f67a2200640 1 --2- 172.21.15.60:0/2772141108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f679c0d0400 0x7f679c15e500 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.941+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 >> 172.21.15.60:0/2772141108 conn(0x7f679c0d1bf0 msgr2=0x7f679c0b6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.941+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 shutdown_connections 2024-09-16T15:38:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:36.941+0000 7f67a2200640 1 -- 172.21.15.60:0/2772141108 wait complete. 2024-09-16T15:38:36.960 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:38:36.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:38:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.126+0000 7fed1da00640 1 -- 172.21.15.60:0/294440892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed180cccd0 msgr2=0x7fed180cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.126+0000 7fed1da00640 1 --2- 172.21.15.60:0/294440892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed180cccd0 0x7fed180cdbb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fed10005fd0 tx=0x7fed1002a820 comp rx=0 tx=0).stop 2024-09-16T15:38:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.126+0000 7fed1da00640 1 -- 172.21.15.60:0/294440892 shutdown_connections 2024-09-16T15:38:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.126+0000 7fed1da00640 1 --2- 172.21.15.60:0/294440892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed180cccd0 0x7fed180cdbb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.126+0000 7fed1da00640 1 --2- 172.21.15.60:0/294440892 >> v2:172.21.15.100:3300/0 conn(0x7fed18145750 0x7fed18073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.126+0000 7fed1da00640 1 -- 172.21.15.60:0/294440892 >> 172.21.15.60:0/294440892 conn(0x7fed180b1840 msgr2=0x7fed180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.126+0000 7fed1da00640 1 -- 172.21.15.60:0/294440892 shutdown_connections 2024-09-16T15:38:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.126+0000 7fed1da00640 1 -- 172.21.15.60:0/294440892 wait complete. 2024-09-16T15:38:37.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.128+0000 7fed1da00640 1 Processor -- start 2024-09-16T15:38:37.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.128+0000 7fed1da00640 1 -- start start 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed1da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806efa0 0x7fed18142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed1da00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed180cccd0 0x7fed181433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed1da00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18145750 0x7fed180c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed1da00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fed180ce530 con 0x7fed18145750 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed1da00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed180ce3b0 con 0x7fed1806efa0 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed1da00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fed180ce6b0 con 0x7fed180cccd0 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed17400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18145750 0x7fed180c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed17400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18145750 0x7fed180c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39432/0 (socket says 172.21.15.60:39432) 2024-09-16T15:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed17400640 1 -- 172.21.15.60:0/3039572951 learned_addr learned my addr 172.21.15.60:0/3039572951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed16a00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806efa0 0x7fed18142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.129+0000 7fed16000640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed180cccd0 0x7fed181433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed17400640 1 -- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed180cccd0 msgr2=0x7fed181433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed17400640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed180cccd0 0x7fed181433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed17400640 1 -- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806efa0 msgr2=0x7fed18142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed17400640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806efa0 0x7fed18142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed17400640 1 -- 172.21.15.60:0/3039572951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed18138000 con 0x7fed18145750 2024-09-16T15:38:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed17400640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18145750 0x7fed180c4ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed10006010 tx=0x7fed10032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed0ea00640 1 -- 172.21.15.60:0/3039572951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10037040 con 0x7fed18145750 2024-09-16T15:38:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed0ea00640 1 -- 172.21.15.60:0/3039572951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed10042070 con 0x7fed18145750 2024-09-16T15:38:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed0ea00640 1 -- 172.21.15.60:0/3039572951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed100042a0 con 0x7fed18145750 2024-09-16T15:38:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed181f4930 con 0x7fed18145750 2024-09-16T15:38:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.130+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed181f4ef0 con 0x7fed18145750 2024-09-16T15:38:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.131+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed180ceb60 con 0x7fed18145750 2024-09-16T15:38:37.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.133+0000 7fed0ea00640 1 -- 172.21.15.60:0/3039572951 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fed10004440 con 0x7fed18145750 2024-09-16T15:38:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.133+0000 7fed0ea00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf8078850 0x7fecf807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.134+0000 7fed0ea00640 1 -- 172.21.15.60:0/3039572951 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fed100bb490 con 0x7fed18145750 2024-09-16T15:38:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.134+0000 7fed16a00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf8078850 0x7fecf807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.135+0000 7fed16a00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf8078850 0x7fecf807ad10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed1815d880 tx=0x7fed08005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:37.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.138+0000 7fed0ea00640 1 -- 172.21.15.60:0/3039572951 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fed10083420 con 0x7fed18145750 2024-09-16T15:38:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.369+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fed180b1250 con 0x7fed18145750 2024-09-16T15:38:37.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.944+0000 7fed0ea00640 1 -- 172.21.15.60:0/3039572951 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fed10087e50 con 0x7fed18145750 2024-09-16T15:38:37.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf8078850 msgr2=0x7fecf807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf8078850 0x7fecf807ad10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed1815d880 tx=0x7fed08005f90 comp rx=0 tx=0).stop 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18145750 msgr2=0x7fed180c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18145750 0x7fed180c4ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed10006010 tx=0x7fed10032c90 comp rx=0 tx=0).stop 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 shutdown_connections 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf8078850 0x7fecf807ad10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18145750 0x7fed180c4ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed180cccd0 0x7fed181433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 --2- 172.21.15.60:0/3039572951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806efa0 0x7fed18142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 >> 172.21.15.60:0/3039572951 conn(0x7fed180b1840 msgr2=0x7fed180afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 shutdown_connections 2024-09-16T15:38:37.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:37.947+0000 7fed1da00640 1 -- 172.21.15.60:0/3039572951 wait complete. 2024-09-16T15:38:37.966 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:38:37.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:38:38.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.129+0000 7f8398000640 1 -- 172.21.15.60:0/2845049887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8378004800 msgr2=0x7f8378004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.129+0000 7f8398000640 1 --2- 172.21.15.60:0/2845049887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8378004800 0x7f8378004c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f837c006f90 tx=0x7f8378005830 comp rx=0 tx=0).stop 2024-09-16T15:38:38.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.129+0000 7f8398000640 1 -- 172.21.15.60:0/2845049887 shutdown_connections 2024-09-16T15:38:38.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.129+0000 7f8398000640 1 --2- 172.21.15.60:0/2845049887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8378004800 0x7f8378004c80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.129+0000 7f8398000640 1 -- 172.21.15.60:0/2845049887 >> 172.21.15.60:0/2845049887 conn(0x7f83900d3fe0 msgr2=0x7f83900d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.129+0000 7f8398000640 1 -- 172.21.15.60:0/2845049887 shutdown_connections 2024-09-16T15:38:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.129+0000 7f8398000640 1 -- 172.21.15.60:0/2845049887 wait complete. 2024-09-16T15:38:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.131+0000 7f8398000640 1 Processor -- start 2024-09-16T15:38:38.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.131+0000 7f8398000640 1 -- start start 2024-09-16T15:38:38.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8398000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8378004800 0x7f83900cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8398000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83901f58e0 0x7f83900cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8398000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83900b9770 0x7f83901f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8398000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f83900d2720 con 0x7f83901f58e0 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8398000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83900d25a0 con 0x7f83900b9770 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8398000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f83900d28a0 con 0x7f8378004800 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8394c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83901f58e0 0x7f83900cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8394c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83901f58e0 0x7f83900cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39446/0 (socket says 172.21.15.60:39446) 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8394c00640 1 -- 172.21.15.60:0/718668968 learned_addr learned my addr 172.21.15.60:0/718668968 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8395600640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8378004800 0x7f83900cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.132+0000 7f8396000640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83900b9770 0x7f83901f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f8394c00640 1 -- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8378004800 msgr2=0x7f83900cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f8394c00640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8378004800 0x7f83900cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f8394c00640 1 -- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83900b9770 msgr2=0x7f83901f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f8394c00640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83900b9770 0x7f83901f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f8394c00640 1 -- 172.21.15.60:0/718668968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83901f5e30 con 0x7f83901f58e0 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f8394c00640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83901f58e0 0x7f83900cd630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f837c00c350 tx=0x7f837c006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f838d600640 1 -- 172.21.15.60:0/718668968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c032030 con 0x7f83901f58e0 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f838d600640 1 -- 172.21.15.60:0/718668968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837c003af0 con 0x7f83901f58e0 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f838d600640 1 -- 172.21.15.60:0/718668968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c02e460 con 0x7f83901f58e0 2024-09-16T15:38:38.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83900cb690 con 0x7f83901f58e0 2024-09-16T15:38:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.133+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83900cbc50 con 0x7f83901f58e0 2024-09-16T15:38:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.135+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83900d3ba0 con 0x7f83901f58e0 2024-09-16T15:38:38.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.136+0000 7f838d600640 1 -- 172.21.15.60:0/718668968 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f837c035020 con 0x7f83901f58e0 2024-09-16T15:38:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.137+0000 7f838d600640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8370078850 0x7f837007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.137+0000 7f838d600640 1 -- 172.21.15.60:0/718668968 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f837c0bafe0 con 0x7f83901f58e0 2024-09-16T15:38:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.137+0000 7f8395600640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8370078850 0x7f837007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.138+0000 7f8395600640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8370078850 0x7f837007ad10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8390052890 tx=0x7f8384005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.142+0000 7f838d600640 1 -- 172.21.15.60:0/718668968 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f83900d3ba0 con 0x7f83901f58e0 2024-09-16T15:38:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.373+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f839008e070 con 0x7f83901f58e0 2024-09-16T15:38:38.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.374+0000 7f838d600640 1 -- 172.21.15.60:0/718668968 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f839008e070 con 0x7f83901f58e0 2024-09-16T15:38:38.376 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:38.376 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:38:34.368784+0000","modified":"2024-09-16T15:38:37.370114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-16T15:38:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-09-16T15:38:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8370078850 msgr2=0x7f837007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8370078850 0x7f837007ad10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8390052890 tx=0x7f8384005f90 comp rx=0 tx=0).stop 2024-09-16T15:38:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83901f58e0 msgr2=0x7f83900cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83901f58e0 0x7f83900cd630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f837c00c350 tx=0x7f837c006ca0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 shutdown_connections 2024-09-16T15:38:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8370078850 0x7f837007ad10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83900b9770 0x7f83901f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83901f58e0 0x7f83900cd630 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 --2- 172.21.15.60:0/718668968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8378004800 0x7f83900cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 >> 172.21.15.60:0/718668968 conn(0x7f83900d3fe0 msgr2=0x7f839015c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.376+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 shutdown_connections 2024-09-16T15:38:38.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.377+0000 7f8398000640 1 -- 172.21.15.60:0/718668968 wait complete. 2024-09-16T15:38:38.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.552+0000 7fbe5cc00640 1 -- 172.21.15.60:0/3532925113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe580b5370 msgr2=0x7fbe580b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.552+0000 7fbe5cc00640 1 --2- 172.21.15.60:0/3532925113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe580b5370 0x7fbe580b3bc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe580d4ea0 tx=0x7fbe4002a510 comp rx=0 tx=0).stop 2024-09-16T15:38:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.552+0000 7fbe5cc00640 1 -- 172.21.15.60:0/3532925113 shutdown_connections 2024-09-16T15:38:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.552+0000 7fbe5cc00640 1 --2- 172.21.15.60:0/3532925113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe580b5370 0x7fbe580b3bc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.552+0000 7fbe5cc00640 1 -- 172.21.15.60:0/3532925113 >> 172.21.15.60:0/3532925113 conn(0x7fbe580d3df0 msgr2=0x7fbe580d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.552+0000 7fbe5cc00640 1 -- 172.21.15.60:0/3532925113 shutdown_connections 2024-09-16T15:38:38.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.552+0000 7fbe5cc00640 1 -- 172.21.15.60:0/3532925113 wait complete. 2024-09-16T15:38:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.554+0000 7fbe5cc00640 1 Processor -- start 2024-09-16T15:38:38.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.554+0000 7fbe5cc00640 1 -- start start 2024-09-16T15:38:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe5cc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe580b4100 0x7fbe581f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe5cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe580b5370 0x7fbe58155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe5cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58161440 0x7fbe58156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe5cc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbe580b2210 con 0x7fbe580b5370 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe5cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe580b2090 con 0x7fbe58161440 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe5cc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbe580b2390 con 0x7fbe580b4100 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe4ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe580b5370 0x7fbe58155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe4ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe580b5370 0x7fbe58155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39458/0 (socket says 172.21.15.60:39458) 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe4ea00640 1 -- 172.21.15.60:0/69735680 learned_addr learned my addr 172.21.15.60:0/69735680 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe57400640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe580b4100 0x7fbe581f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe57e00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58161440 0x7fbe58156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe4ea00640 1 -- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe580b4100 msgr2=0x7fbe581f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe4ea00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe580b4100 0x7fbe581f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe4ea00640 1 -- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58161440 msgr2=0x7fbe58156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe4ea00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58161440 0x7fbe58156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe4ea00640 1 -- 172.21.15.60:0/69735680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe581564a0 con 0x7fbe580b5370 2024-09-16T15:38:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.555+0000 7fbe57e00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58161440 0x7fbe58156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.556+0000 7fbe4ea00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe580b5370 0x7fbe58155f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe4002ab00 tx=0x7fbe40032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.556+0000 7fbe4fe00640 1 -- 172.21.15.60:0/69735680 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40040070 con 0x7fbe580b5370 2024-09-16T15:38:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.556+0000 7fbe4fe00640 1 -- 172.21.15.60:0/69735680 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe40005310 con 0x7fbe580b5370 2024-09-16T15:38:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.556+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe580b70b0 con 0x7fbe580b5370 2024-09-16T15:38:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.556+0000 7fbe4fe00640 1 -- 172.21.15.60:0/69735680 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe400047a0 con 0x7fbe580b5370 2024-09-16T15:38:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.556+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe580b7670 con 0x7fbe580b5370 2024-09-16T15:38:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.557+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe580b2840 con 0x7fbe580b5370 2024-09-16T15:38:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.558+0000 7fbe4fe00640 1 -- 172.21.15.60:0/69735680 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fbe40039040 con 0x7fbe580b5370 2024-09-16T15:38:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.559+0000 7fbe4fe00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbe34078850 0x7fbe3407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.559+0000 7fbe4fe00640 1 -- 172.21.15.60:0/69735680 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbe400bbba0 con 0x7fbe580b5370 2024-09-16T15:38:38.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.560+0000 7fbe57400640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbe34078850 0x7fbe3407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.560+0000 7fbe57400640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbe34078850 0x7fbe3407ad10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe580ce0f0 tx=0x7fbe48009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:38.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.564+0000 7fbe4fe00640 1 -- 172.21.15.60:0/69735680 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbe581564a0 con 0x7fbe580b5370 2024-09-16T15:38:38.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.761+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe5815e520 con 0x7fbe580b5370 2024-09-16T15:38:38.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.762+0000 7fbe4fe00640 1 -- 172.21.15.60:0/69735680 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7fbe4009de20 con 0x7fbe580b5370 2024-09-16T15:38:38.764 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:38.764 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:35.924959+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:38.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbe34078850 msgr2=0x7fbe3407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbe34078850 0x7fbe3407ad10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe580ce0f0 tx=0x7fbe48009040 comp rx=0 tx=0).stop 2024-09-16T15:38:38.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe580b5370 msgr2=0x7fbe58155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe580b5370 0x7fbe58155f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe4002ab00 tx=0x7fbe40032c10 comp rx=0 tx=0).stop 2024-09-16T15:38:38.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 shutdown_connections 2024-09-16T15:38:38.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbe34078850 0x7fbe3407ad10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58161440 0x7fbe58156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe580b5370 0x7fbe58155f60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 --2- 172.21.15.60:0/69735680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe580b4100 0x7fbe581f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 >> 172.21.15.60:0/69735680 conn(0x7fbe580d3df0 msgr2=0x7fbe5806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:38.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 shutdown_connections 2024-09-16T15:38:38.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.765+0000 7fbe5cc00640 1 -- 172.21.15.60:0/69735680 wait complete. 2024-09-16T15:38:38.781 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:38:38.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:38:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.941+0000 7f6744c00640 1 -- 172.21.15.60:0/4239590951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 msgr2=0x7f6724003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.941+0000 7f6744c00640 1 --2- 172.21.15.60:0/4239590951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 0x7f6724003e70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6730009b20 tx=0x7f6724005830 comp rx=0 tx=0).stop 2024-09-16T15:38:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.941+0000 7f673e800640 1 -- 172.21.15.60:0/4239590951 reap_dead start 2024-09-16T15:38:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.941+0000 7f6744c00640 1 -- 172.21.15.60:0/4239590951 shutdown_connections 2024-09-16T15:38:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.941+0000 7f6744c00640 1 -- 172.21.15.60:0/4239590951 >> 172.21.15.60:0/4239590951 conn(0x7f67400b1950 msgr2=0x7f67400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.941+0000 7f6744c00640 1 -- 172.21.15.60:0/4239590951 shutdown_connections 2024-09-16T15:38:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.941+0000 7f6744c00640 1 -- 172.21.15.60:0/4239590951 wait complete. 2024-09-16T15:38:38.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.943+0000 7f6744c00640 1 Processor -- start 2024-09-16T15:38:38.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.943+0000 7f6744c00640 1 -- start start 2024-09-16T15:38:38.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f6744c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 0x7f674014f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f6744c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400cb740 0x7f67400cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f6744c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67400b3d60 0x7f67400b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f6744c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f67400745c0 con 0x7f6724003a50 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f6744c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6740074440 con 0x7f67400b3d60 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f6744c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6740074740 con 0x7f67400cb740 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f673de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 0x7f674014f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f673de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 0x7f674014f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35092/0 (socket says 172.21.15.60:35092) 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f673de00640 1 -- 172.21.15.60:0/464425719 learned_addr learned my addr 172.21.15.60:0/464425719 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f673e800640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67400b3d60 0x7f67400b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.944+0000 7f6735400640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400cb740 0x7f67400cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f673de00640 1 -- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400cb740 msgr2=0x7f67400cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f673de00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400cb740 0x7f67400cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f673de00640 1 -- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67400b3d60 msgr2=0x7f67400b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f673de00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67400b3d60 0x7f67400b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f673de00640 1 -- 172.21.15.60:0/464425719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67400b2fa0 con 0x7f6724003a50 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f673de00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 0x7f674014f610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6730036040 tx=0x7f6730033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f6736a00640 1 -- 172.21.15.60:0/464425719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730043070 con 0x7f6724003a50 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f6736a00640 1 -- 172.21.15.60:0/464425719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6730004340 con 0x7f6724003a50 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f6736a00640 1 -- 172.21.15.60:0/464425719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730038040 con 0x7f6724003a50 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.945+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67400b3280 con 0x7f6724003a50 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.946+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67400b3840 con 0x7f6724003a50 2024-09-16T15:38:38.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.947+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67400b2440 con 0x7f6724003a50 2024-09-16T15:38:38.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.950+0000 7f6736a00640 1 -- 172.21.15.60:0/464425719 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f673002b870 con 0x7f6724003a50 2024-09-16T15:38:38.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.951+0000 7f6736a00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f671c078880 0x7f671c07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:38.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.951+0000 7f6736a00640 1 -- 172.21.15.60:0/464425719 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f67300bafc0 con 0x7f6724003a50 2024-09-16T15:38:38.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.951+0000 7f6735400640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f671c078880 0x7f671c07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:38.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.952+0000 7f6735400640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f671c078880 0x7f671c07ad40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6728000fb0 tx=0x7f6728006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:38.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:38.953+0000 7f6736a00640 1 -- 172.21.15.60:0/464425719 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f673004c020 con 0x7f6724003a50 2024-09-16T15:38:39.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.145+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67400b2d50 con 0x7f671c078880 2024-09-16T15:38:39.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.151+0000 7f6736a00640 1 -- 172.21.15.60:0/464425719 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241439 (secure 0 0 0) 0x7f67400b2d50 con 0x7f671c078880 2024-09-16T15:38:39.155 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:38:39.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f671c078880 msgr2=0x7f671c07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f671c078880 0x7f671c07ad40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6728000fb0 tx=0x7f6728006720 comp rx=0 tx=0).stop 2024-09-16T15:38:39.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 msgr2=0x7f674014f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 0x7f674014f610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6730036040 tx=0x7f6730033670 comp rx=0 tx=0).stop 2024-09-16T15:38:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 shutdown_connections 2024-09-16T15:38:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f671c078880 0x7f671c07ad40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67400b3d60 0x7f67400b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67400cb740 0x7f67400cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 --2- 172.21.15.60:0/464425719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6724003a50 0x7f674014f610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 >> 172.21.15.60:0/464425719 conn(0x7f67400b1950 msgr2=0x7f674006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 shutdown_connections 2024-09-16T15:38:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.155+0000 7f6744c00640 1 -- 172.21.15.60:0/464425719 wait complete. 2024-09-16T15:38:39.172 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T15:38:37.708693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236028,"kb_used_data":2656,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738692,"statfs":{"total":773094113280,"available":772852420608,"internally_reserved":0,"allocated":2719744,"data_stored":1658860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.799006"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329296+0000","last_change":"2024-09-16T15:38:34.934287+0000","last_active":"2024-09-16T15:38:36.329296+0000","last_peered":"2024-09-16T15:38:36.329296+0000","last_clean":"2024-09-16T15:38:36.329296+0000","last_became_active":"2024-09-16T15:38:34.934004+0000","last_became_peered":"2024-09-16T15:38:34.934004+0000","last_unstale":"2024-09-16T15:38:36.329296+0000","last_undegraded":"2024-09-16T15:38:36.329296+0000","last_fullsized":"2024-09-16T15:38:36.329296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:43.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328795+0000","last_change":"2024-09-16T15:38:34.951427+0000","last_active":"2024-09-16T15:38:36.328795+0000","last_peered":"2024-09-16T15:38:36.328795+0000","last_clean":"2024-09-16T15:38:36.328795+0000","last_became_active":"2024-09-16T15:38:34.949484+0000","last_became_peered":"2024-09-16T15:38:34.949484+0000","last_unstale":"2024-09-16T15:38:36.328795+0000","last_undegraded":"2024-09-16T15:38:36.328795+0000","last_fullsized":"2024-09-16T15:38:36.328795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:54.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329086+0000","last_change":"2024-09-16T15:38:34.934519+0000","last_active":"2024-09-16T15:38:36.329086+0000","last_peered":"2024-09-16T15:38:36.329086+0000","last_clean":"2024-09-16T15:38:36.329086+0000","last_became_active":"2024-09-16T15:38:34.932223+0000","last_became_peered":"2024-09-16T15:38:34.932223+0000","last_unstale":"2024-09-16T15:38:36.329086+0000","last_undegraded":"2024-09-16T15:38:36.329086+0000","last_fullsized":"2024-09-16T15:38:36.329086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.673101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913370+0000","last_change":"2024-09-16T15:38:32.913370+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913370+0000","last_undegraded":"2024-09-16T15:38:32.913370+0000","last_fullsized":"2024-09-16T15:38:32.913370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.911636+0000","last_change":"2024-09-16T15:38:32.911636+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.911636+0000","last_undegraded":"2024-09-16T15:38:32.911636+0000","last_fullsized":"2024-09-16T15:38:32.911636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913455+0000","last_change":"2024-09-16T15:38:32.913455+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913455+0000","last_undegraded":"2024-09-16T15:38:32.913455+0000","last_fullsized":"2024-09-16T15:38:32.913455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.918819+0000","last_change":"2024-09-16T15:38:32.918819+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.918819+0000","last_undegraded":"2024-09-16T15:38:32.918819+0000","last_fullsized":"2024-09-16T15:38:32.918819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328641+0000","last_change":"2024-09-16T15:38:34.933709+0000","last_active":"2024-09-16T15:38:36.328641+0000","last_peered":"2024-09-16T15:38:36.328641+0000","last_clean":"2024-09-16T15:38:36.328641+0000","last_became_active":"2024-09-16T15:38:34.932126+0000","last_became_peered":"2024-09-16T15:38:34.932126+0000","last_unstale":"2024-09-16T15:38:36.328641+0000","last_undegraded":"2024-09-16T15:38:36.328641+0000","last_fullsized":"2024-09-16T15:38:36.328641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:44.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.912626+0000","last_change":"2024-09-16T15:38:32.912626+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.912626+0000","last_undegraded":"2024-09-16T15:38:32.912626+0000","last_fullsized":"2024-09-16T15:38:32.912626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.918741+0000","last_change":"2024-09-16T15:38:32.918741+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.918741+0000","last_undegraded":"2024-09-16T15:38:32.918741+0000","last_fullsized":"2024-09-16T15:38:32.918741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328973+0000","last_change":"2024-09-16T15:38:33.942385+0000","last_active":"2024-09-16T15:38:36.328973+0000","last_peered":"2024-09-16T15:38:36.328973+0000","last_clean":"2024-09-16T15:38:36.328973+0000","last_became_active":"2024-09-16T15:38:33.941960+0000","last_became_peered":"2024-09-16T15:38:33.941960+0000","last_unstale":"2024-09-16T15:38:36.328973+0000","last_undegraded":"2024-09-16T15:38:36.328973+0000","last_fullsized":"2024-09-16T15:38:36.328973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:08.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.911273+0000","last_change":"2024-09-16T15:38:32.911273+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.911273+0000","last_undegraded":"2024-09-16T15:38:32.911273+0000","last_fullsized":"2024-09-16T15:38:32.911273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.909327+0000","last_change":"2024-09-16T15:38:32.909327+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.909327+0000","last_undegraded":"2024-09-16T15:38:32.909327+0000","last_fullsized":"2024-09-16T15:38:32.909327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.919716+0000","last_change":"2024-09-16T15:38:32.919716+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.919716+0000","last_undegraded":"2024-09-16T15:38:32.919716+0000","last_fullsized":"2024-09-16T15:38:32.919716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.912147+0000","last_change":"2024-09-16T15:38:32.912147+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.912147+0000","last_undegraded":"2024-09-16T15:38:32.912147+0000","last_fullsized":"2024-09-16T15:38:32.912147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.921200+0000","last_change":"2024-09-16T15:38:32.921200+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.921200+0000","last_undegraded":"2024-09-16T15:38:32.921200+0000","last_fullsized":"2024-09-16T15:38:32.921200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913725+0000","last_change":"2024-09-16T15:38:32.913725+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913725+0000","last_undegraded":"2024-09-16T15:38:32.913725+0000","last_fullsized":"2024-09-16T15:38:32.913725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.921148+0000","last_change":"2024-09-16T15:38:32.921148+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.921148+0000","last_undegraded":"2024-09-16T15:38:32.921148+0000","last_fullsized":"2024-09-16T15:38:32.921148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329039+0000","last_change":"2024-09-16T15:38:33.940469+0000","last_active":"2024-09-16T15:38:36.329039+0000","last_peered":"2024-09-16T15:38:36.329039+0000","last_clean":"2024-09-16T15:38:36.329039+0000","last_became_active":"2024-09-16T15:38:33.939752+0000","last_became_peered":"2024-09-16T15:38:33.939752+0000","last_unstale":"2024-09-16T15:38:36.329039+0000","last_undegraded":"2024-09-16T15:38:36.329039+0000","last_fullsized":"2024-09-16T15:38:36.329039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:31.202600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.921207+0000","last_change":"2024-09-16T15:38:32.921207+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.921207+0000","last_undegraded":"2024-09-16T15:38:32.921207+0000","last_fullsized":"2024-09-16T15:38:32.921207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:38:32.916445+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:38:32.916445+0000","last_peered":"2024-09-16T15:38:32.916445+0000","last_clean":"2024-09-16T15:38:32.916445+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:38:32.916445+0000","last_undegraded":"2024-09-16T15:38:32.916445+0000","last_fullsized":"2024-09-16T15:38:32.916445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.918710+0000","last_change":"2024-09-16T15:38:32.918710+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.918710+0000","last_undegraded":"2024-09-16T15:38:32.918710+0000","last_fullsized":"2024-09-16T15:38:32.918710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913512+0000","last_change":"2024-09-16T15:38:32.913512+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913512+0000","last_undegraded":"2024-09-16T15:38:32.913512+0000","last_fullsized":"2024-09-16T15:38:32.913512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.921214+0000","last_change":"2024-09-16T15:38:32.921214+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.921214+0000","last_undegraded":"2024-09-16T15:38:32.921214+0000","last_fullsized":"2024-09-16T15:38:32.921214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328622+0000","last_change":"2024-09-16T15:38:34.932401+0000","last_active":"2024-09-16T15:38:36.328622+0000","last_peered":"2024-09-16T15:38:36.328622+0000","last_clean":"2024-09-16T15:38:36.328622+0000","last_became_active":"2024-09-16T15:38:34.931203+0000","last_became_peered":"2024-09-16T15:38:34.931203+0000","last_unstale":"2024-09-16T15:38:36.328622+0000","last_undegraded":"2024-09-16T15:38:36.328622+0000","last_fullsized":"2024-09-16T15:38:36.328622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:24.483689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.918708+0000","last_change":"2024-09-16T15:38:32.918708+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.918708+0000","last_undegraded":"2024-09-16T15:38:32.918708+0000","last_fullsized":"2024-09-16T15:38:32.918708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328940+0000","last_change":"2024-09-16T15:38:33.936405+0000","last_active":"2024-09-16T15:38:36.328940+0000","last_peered":"2024-09-16T15:38:36.328940+0000","last_clean":"2024-09-16T15:38:36.328940+0000","last_became_active":"2024-09-16T15:38:33.935318+0000","last_became_peered":"2024-09-16T15:38:33.935318+0000","last_unstale":"2024-09-16T15:38:36.328940+0000","last_undegraded":"2024-09-16T15:38:36.328940+0000","last_fullsized":"2024-09-16T15:38:36.328940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:27.366083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.914271+0000","last_change":"2024-09-16T15:38:32.914271+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.914271+0000","last_undegraded":"2024-09-16T15:38:32.914271+0000","last_fullsized":"2024-09-16T15:38:32.914271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328724+0000","last_change":"2024-09-16T15:38:34.928319+0000","last_active":"2024-09-16T15:38:36.328724+0000","last_peered":"2024-09-16T15:38:36.328724+0000","last_clean":"2024-09-16T15:38:36.328724+0000","last_became_active":"2024-09-16T15:38:34.928058+0000","last_became_peered":"2024-09-16T15:38:34.928058+0000","last_unstale":"2024-09-16T15:38:36.328724+0000","last_undegraded":"2024-09-16T15:38:36.328724+0000","last_fullsized":"2024-09-16T15:38:36.328724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:00.238312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.929048+0000","last_change":"2024-09-16T15:38:33.931369+0000","last_active":"2024-09-16T15:38:36.929048+0000","last_peered":"2024-09-16T15:38:36.929048+0000","last_clean":"2024-09-16T15:38:36.929048+0000","last_became_active":"2024-09-16T15:38:33.928339+0000","last_became_peered":"2024-09-16T15:38:33.928339+0000","last_unstale":"2024-09-16T15:38:36.929048+0000","last_undegraded":"2024-09-16T15:38:36.929048+0000","last_fullsized":"2024-09-16T15:38:36.929048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:44.073989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.911314+0000","last_change":"2024-09-16T15:38:32.911314+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.911314+0000","last_undegraded":"2024-09-16T15:38:32.911314+0000","last_fullsized":"2024-09-16T15:38:32.911314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.915071+0000","last_change":"2024-09-16T15:38:32.915071+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.915071+0000","last_undegraded":"2024-09-16T15:38:32.915071+0000","last_fullsized":"2024-09-16T15:38:32.915071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.947052+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.946593+0000","last_became_peered":"2024-09-16T15:38:33.946593+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:37.618001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.943983+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.943683+0000","last_became_peered":"2024-09-16T15:38:33.943683+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.572327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.915381+0000","last_change":"2024-09-16T15:38:32.915381+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.915381+0000","last_undegraded":"2024-09-16T15:38:32.915381+0000","last_fullsized":"2024-09-16T15:38:32.915381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.916045+0000","last_change":"2024-09-16T15:38:32.916045+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.916045+0000","last_undegraded":"2024-09-16T15:38:32.916045+0000","last_fullsized":"2024-09-16T15:38:32.916045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.915100+0000","last_change":"2024-09-16T15:38:32.915100+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.915100+0000","last_undegraded":"2024-09-16T15:38:32.915100+0000","last_fullsized":"2024-09-16T15:38:32.915100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913456+0000","last_change":"2024-09-16T15:38:32.913456+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913456+0000","last_undegraded":"2024-09-16T15:38:32.913456+0000","last_fullsized":"2024-09-16T15:38:32.913456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.909818+0000","last_change":"2024-09-16T15:38:32.909818+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.909818+0000","last_undegraded":"2024-09-16T15:38:32.909818+0000","last_fullsized":"2024-09-16T15:38:32.909818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":62477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":778240,"data_stored":647028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":75687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:38:39.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:38:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.371+0000 7f58dfe00640 1 -- 172.21.15.60:0/624086912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 msgr2=0x7f58d80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.371+0000 7f58dfe00640 1 --2- 172.21.15.60:0/624086912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 0x7f58d80b2440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009830 tx=0x7f58c402a410 comp rx=0 tx=0).stop 2024-09-16T15:38:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.372+0000 7f58dfe00640 1 -- 172.21.15.60:0/624086912 shutdown_connections 2024-09-16T15:38:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.372+0000 7f58dfe00640 1 --2- 172.21.15.60:0/624086912 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d8073fb0 0x7f58d8074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.372+0000 7f58dfe00640 1 --2- 172.21.15.60:0/624086912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 0x7f58d80b2440 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.372+0000 7f58dfe00640 1 -- 172.21.15.60:0/624086912 >> 172.21.15.60:0/624086912 conn(0x7f58d80b1950 msgr2=0x7f58d80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.372+0000 7f58dfe00640 1 -- 172.21.15.60:0/624086912 shutdown_connections 2024-09-16T15:38:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.372+0000 7f58dfe00640 1 -- 172.21.15.60:0/624086912 wait complete. 2024-09-16T15:38:39.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.373+0000 7f58dfe00640 1 Processor -- start 2024-09-16T15:38:39.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.373+0000 7f58dfe00640 1 -- start start 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.374+0000 7f58dfe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d8073fb0 0x7f58d81492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.374+0000 7f58dfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 0x7f58d814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.374+0000 7f58dfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d815f5a0 0x7f58d81585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.374+0000 7f58dfe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f58d806f2e0 con 0x7f58d815f5a0 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.374+0000 7f58dfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58d806f160 con 0x7f58d80ccf00 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.374+0000 7f58dfe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f58d806f460 con 0x7f58d8073fb0 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d815f5a0 0x7f58d81585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 0x7f58d814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:39.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dd400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d8073fb0 0x7f58d81492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d815f5a0 0x7f58d81585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35108/0 (socket says 172.21.15.60:35108) 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 -- 172.21.15.60:0/1472811759 learned_addr learned my addr 172.21.15.60:0/1472811759 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 0x7f58d814f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56668/0 (socket says 172.21.15.60:56668) 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 -- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d8073fb0 msgr2=0x7f58d81492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d8073fb0 0x7f58d81492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 -- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 msgr2=0x7f58d814f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 0x7f58d814f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 -- 172.21.15.60:0/1472811759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d8159120 con 0x7f58d815f5a0 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dca00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 0x7f58d814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.375+0000 7f58dde00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d815f5a0 0x7f58d81585b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58d000eb10 tx=0x7f58d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.376+0000 7f58d5600640 1 -- 172.21.15.60:0/1472811759 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0004b20 con 0x7f58d815f5a0 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.376+0000 7f58d5600640 1 -- 172.21.15.60:0/1472811759 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58d0004ce0 con 0x7f58d815f5a0 2024-09-16T15:38:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.376+0000 7f58d5600640 1 -- 172.21.15.60:0/1472811759 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0005280 con 0x7f58d815f5a0 2024-09-16T15:38:39.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.376+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d80ce7c0 con 0x7f58d815f5a0 2024-09-16T15:38:39.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.376+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d80ced80 con 0x7f58d815f5a0 2024-09-16T15:38:39.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.377+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d80cef80 con 0x7f58d815f5a0 2024-09-16T15:38:39.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.378+0000 7f58d5600640 1 -- 172.21.15.60:0/1472811759 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f58d0005420 con 0x7f58d815f5a0 2024-09-16T15:38:39.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.379+0000 7f58d5600640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58b8078850 0x7f58b807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:39.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.379+0000 7f58d5600640 1 -- 172.21.15.60:0/1472811759 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f58d009b520 con 0x7f58d815f5a0 2024-09-16T15:38:39.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.379+0000 7f58dd400640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58b8078850 0x7f58b807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:39.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.380+0000 7f58dd400640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58b8078850 0x7f58b807ad10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58cc001a00 tx=0x7f58cc001990 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:39.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.383+0000 7f58d5600640 1 -- 172.21.15.60:0/1472811759 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f58d80cef80 con 0x7f58d815f5a0 2024-09-16T15:38:39.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.573+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58d806f9f0 con 0x7f58b8078850 2024-09-16T15:38:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.578+0000 7f58d5600640 1 -- 172.21.15.60:0/1472811759 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241439 (secure 0 0 0) 0x7f58d806f9f0 con 0x7f58b8078850 2024-09-16T15:38:39.581 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:38:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58b8078850 msgr2=0x7f58b807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58b8078850 0x7f58b807ad10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58cc001a00 tx=0x7f58cc001990 comp rx=0 tx=0).stop 2024-09-16T15:38:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d815f5a0 msgr2=0x7f58d81585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d815f5a0 0x7f58d81585b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58d000eb10 tx=0x7f58d000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 shutdown_connections 2024-09-16T15:38:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58b8078850 0x7f58b807ad10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d815f5a0 0x7f58d81585b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d80ccf00 0x7f58d814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 --2- 172.21.15.60:0/1472811759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d8073fb0 0x7f58d81492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 >> 172.21.15.60:0/1472811759 conn(0x7f58d80b1950 msgr2=0x7f58d80b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 shutdown_connections 2024-09-16T15:38:39.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.581+0000 7f58dfe00640 1 -- 172.21.15.60:0/1472811759 wait complete. 2024-09-16T15:38:39.598 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T15:38:37.708693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236028,"kb_used_data":2656,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738692,"statfs":{"total":773094113280,"available":772852420608,"internally_reserved":0,"allocated":2719744,"data_stored":1658860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.799006"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329296+0000","last_change":"2024-09-16T15:38:34.934287+0000","last_active":"2024-09-16T15:38:36.329296+0000","last_peered":"2024-09-16T15:38:36.329296+0000","last_clean":"2024-09-16T15:38:36.329296+0000","last_became_active":"2024-09-16T15:38:34.934004+0000","last_became_peered":"2024-09-16T15:38:34.934004+0000","last_unstale":"2024-09-16T15:38:36.329296+0000","last_undegraded":"2024-09-16T15:38:36.329296+0000","last_fullsized":"2024-09-16T15:38:36.329296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:43.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328795+0000","last_change":"2024-09-16T15:38:34.951427+0000","last_active":"2024-09-16T15:38:36.328795+0000","last_peered":"2024-09-16T15:38:36.328795+0000","last_clean":"2024-09-16T15:38:36.328795+0000","last_became_active":"2024-09-16T15:38:34.949484+0000","last_became_peered":"2024-09-16T15:38:34.949484+0000","last_unstale":"2024-09-16T15:38:36.328795+0000","last_undegraded":"2024-09-16T15:38:36.328795+0000","last_fullsized":"2024-09-16T15:38:36.328795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:54.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329086+0000","last_change":"2024-09-16T15:38:34.934519+0000","last_active":"2024-09-16T15:38:36.329086+0000","last_peered":"2024-09-16T15:38:36.329086+0000","last_clean":"2024-09-16T15:38:36.329086+0000","last_became_active":"2024-09-16T15:38:34.932223+0000","last_became_peered":"2024-09-16T15:38:34.932223+0000","last_unstale":"2024-09-16T15:38:36.329086+0000","last_undegraded":"2024-09-16T15:38:36.329086+0000","last_fullsized":"2024-09-16T15:38:36.329086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.673101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913370+0000","last_change":"2024-09-16T15:38:32.913370+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913370+0000","last_undegraded":"2024-09-16T15:38:32.913370+0000","last_fullsized":"2024-09-16T15:38:32.913370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.911636+0000","last_change":"2024-09-16T15:38:32.911636+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.911636+0000","last_undegraded":"2024-09-16T15:38:32.911636+0000","last_fullsized":"2024-09-16T15:38:32.911636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913455+0000","last_change":"2024-09-16T15:38:32.913455+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913455+0000","last_undegraded":"2024-09-16T15:38:32.913455+0000","last_fullsized":"2024-09-16T15:38:32.913455+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.918819+0000","last_change":"2024-09-16T15:38:32.918819+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.918819+0000","last_undegraded":"2024-09-16T15:38:32.918819+0000","last_fullsized":"2024-09-16T15:38:32.918819+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328641+0000","last_change":"2024-09-16T15:38:34.933709+0000","last_active":"2024-09-16T15:38:36.328641+0000","last_peered":"2024-09-16T15:38:36.328641+0000","last_clean":"2024-09-16T15:38:36.328641+0000","last_became_active":"2024-09-16T15:38:34.932126+0000","last_became_peered":"2024-09-16T15:38:34.932126+0000","last_unstale":"2024-09-16T15:38:36.328641+0000","last_undegraded":"2024-09-16T15:38:36.328641+0000","last_fullsized":"2024-09-16T15:38:36.328641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:44.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.912626+0000","last_change":"2024-09-16T15:38:32.912626+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.912626+0000","last_undegraded":"2024-09-16T15:38:32.912626+0000","last_fullsized":"2024-09-16T15:38:32.912626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.918741+0000","last_change":"2024-09-16T15:38:32.918741+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.918741+0000","last_undegraded":"2024-09-16T15:38:32.918741+0000","last_fullsized":"2024-09-16T15:38:32.918741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328973+0000","last_change":"2024-09-16T15:38:33.942385+0000","last_active":"2024-09-16T15:38:36.328973+0000","last_peered":"2024-09-16T15:38:36.328973+0000","last_clean":"2024-09-16T15:38:36.328973+0000","last_became_active":"2024-09-16T15:38:33.941960+0000","last_became_peered":"2024-09-16T15:38:33.941960+0000","last_unstale":"2024-09-16T15:38:36.328973+0000","last_undegraded":"2024-09-16T15:38:36.328973+0000","last_fullsized":"2024-09-16T15:38:36.328973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:08.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.911273+0000","last_change":"2024-09-16T15:38:32.911273+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.911273+0000","last_undegraded":"2024-09-16T15:38:32.911273+0000","last_fullsized":"2024-09-16T15:38:32.911273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.909327+0000","last_change":"2024-09-16T15:38:32.909327+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.909327+0000","last_undegraded":"2024-09-16T15:38:32.909327+0000","last_fullsized":"2024-09-16T15:38:32.909327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.919716+0000","last_change":"2024-09-16T15:38:32.919716+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.919716+0000","last_undegraded":"2024-09-16T15:38:32.919716+0000","last_fullsized":"2024-09-16T15:38:32.919716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.912147+0000","last_change":"2024-09-16T15:38:32.912147+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.912147+0000","last_undegraded":"2024-09-16T15:38:32.912147+0000","last_fullsized":"2024-09-16T15:38:32.912147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.921200+0000","last_change":"2024-09-16T15:38:32.921200+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.921200+0000","last_undegraded":"2024-09-16T15:38:32.921200+0000","last_fullsized":"2024-09-16T15:38:32.921200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913725+0000","last_change":"2024-09-16T15:38:32.913725+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913725+0000","last_undegraded":"2024-09-16T15:38:32.913725+0000","last_fullsized":"2024-09-16T15:38:32.913725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.921148+0000","last_change":"2024-09-16T15:38:32.921148+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.921148+0000","last_undegraded":"2024-09-16T15:38:32.921148+0000","last_fullsized":"2024-09-16T15:38:32.921148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329039+0000","last_change":"2024-09-16T15:38:33.940469+0000","last_active":"2024-09-16T15:38:36.329039+0000","last_peered":"2024-09-16T15:38:36.329039+0000","last_clean":"2024-09-16T15:38:36.329039+0000","last_became_active":"2024-09-16T15:38:33.939752+0000","last_became_peered":"2024-09-16T15:38:33.939752+0000","last_unstale":"2024-09-16T15:38:36.329039+0000","last_undegraded":"2024-09-16T15:38:36.329039+0000","last_fullsized":"2024-09-16T15:38:36.329039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:31.202600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.921207+0000","last_change":"2024-09-16T15:38:32.921207+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.921207+0000","last_undegraded":"2024-09-16T15:38:32.921207+0000","last_fullsized":"2024-09-16T15:38:32.921207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:38:32.916445+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:38:32.916445+0000","last_peered":"2024-09-16T15:38:32.916445+0000","last_clean":"2024-09-16T15:38:32.916445+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:38:32.916445+0000","last_undegraded":"2024-09-16T15:38:32.916445+0000","last_fullsized":"2024-09-16T15:38:32.916445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.918710+0000","last_change":"2024-09-16T15:38:32.918710+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.918710+0000","last_undegraded":"2024-09-16T15:38:32.918710+0000","last_fullsized":"2024-09-16T15:38:32.918710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913512+0000","last_change":"2024-09-16T15:38:32.913512+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913512+0000","last_undegraded":"2024-09-16T15:38:32.913512+0000","last_fullsized":"2024-09-16T15:38:32.913512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.921214+0000","last_change":"2024-09-16T15:38:32.921214+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.921214+0000","last_undegraded":"2024-09-16T15:38:32.921214+0000","last_fullsized":"2024-09-16T15:38:32.921214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328622+0000","last_change":"2024-09-16T15:38:34.932401+0000","last_active":"2024-09-16T15:38:36.328622+0000","last_peered":"2024-09-16T15:38:36.328622+0000","last_clean":"2024-09-16T15:38:36.328622+0000","last_became_active":"2024-09-16T15:38:34.931203+0000","last_became_peered":"2024-09-16T15:38:34.931203+0000","last_unstale":"2024-09-16T15:38:36.328622+0000","last_undegraded":"2024-09-16T15:38:36.328622+0000","last_fullsized":"2024-09-16T15:38:36.328622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:24.483689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.918708+0000","last_change":"2024-09-16T15:38:32.918708+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.918708+0000","last_undegraded":"2024-09-16T15:38:32.918708+0000","last_fullsized":"2024-09-16T15:38:32.918708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328940+0000","last_change":"2024-09-16T15:38:33.936405+0000","last_active":"2024-09-16T15:38:36.328940+0000","last_peered":"2024-09-16T15:38:36.328940+0000","last_clean":"2024-09-16T15:38:36.328940+0000","last_became_active":"2024-09-16T15:38:33.935318+0000","last_became_peered":"2024-09-16T15:38:33.935318+0000","last_unstale":"2024-09-16T15:38:36.328940+0000","last_undegraded":"2024-09-16T15:38:36.328940+0000","last_fullsized":"2024-09-16T15:38:36.328940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:27.366083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.914271+0000","last_change":"2024-09-16T15:38:32.914271+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.914271+0000","last_undegraded":"2024-09-16T15:38:32.914271+0000","last_fullsized":"2024-09-16T15:38:32.914271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328724+0000","last_change":"2024-09-16T15:38:34.928319+0000","last_active":"2024-09-16T15:38:36.328724+0000","last_peered":"2024-09-16T15:38:36.328724+0000","last_clean":"2024-09-16T15:38:36.328724+0000","last_became_active":"2024-09-16T15:38:34.928058+0000","last_became_peered":"2024-09-16T15:38:34.928058+0000","last_unstale":"2024-09-16T15:38:36.328724+0000","last_undegraded":"2024-09-16T15:38:36.328724+0000","last_fullsized":"2024-09-16T15:38:36.328724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:00.238312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.929048+0000","last_change":"2024-09-16T15:38:33.931369+0000","last_active":"2024-09-16T15:38:36.929048+0000","last_peered":"2024-09-16T15:38:36.929048+0000","last_clean":"2024-09-16T15:38:36.929048+0000","last_became_active":"2024-09-16T15:38:33.928339+0000","last_became_peered":"2024-09-16T15:38:33.928339+0000","last_unstale":"2024-09-16T15:38:36.929048+0000","last_undegraded":"2024-09-16T15:38:36.929048+0000","last_fullsized":"2024-09-16T15:38:36.929048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:44.073989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.911314+0000","last_change":"2024-09-16T15:38:32.911314+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.911314+0000","last_undegraded":"2024-09-16T15:38:32.911314+0000","last_fullsized":"2024-09-16T15:38:32.911314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.915071+0000","last_change":"2024-09-16T15:38:32.915071+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.915071+0000","last_undegraded":"2024-09-16T15:38:32.915071+0000","last_fullsized":"2024-09-16T15:38:32.915071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.947052+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.946593+0000","last_became_peered":"2024-09-16T15:38:33.946593+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:37.618001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.943983+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.943683+0000","last_became_peered":"2024-09-16T15:38:33.943683+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.572327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.915381+0000","last_change":"2024-09-16T15:38:32.915381+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.915381+0000","last_undegraded":"2024-09-16T15:38:32.915381+0000","last_fullsized":"2024-09-16T15:38:32.915381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.916045+0000","last_change":"2024-09-16T15:38:32.916045+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.916045+0000","last_undegraded":"2024-09-16T15:38:32.916045+0000","last_fullsized":"2024-09-16T15:38:32.916045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.915100+0000","last_change":"2024-09-16T15:38:32.915100+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.915100+0000","last_undegraded":"2024-09-16T15:38:32.915100+0000","last_fullsized":"2024-09-16T15:38:32.915100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.913456+0000","last_change":"2024-09-16T15:38:32.913456+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.913456+0000","last_undegraded":"2024-09-16T15:38:32.913456+0000","last_fullsized":"2024-09-16T15:38:32.913456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:38:33.906366+0000","last_change":"2024-09-16T15:38:33.906366+0000","last_active":"2024-09-16T15:38:33.906366+0000","last_peered":"2024-09-16T15:38:33.906366+0000","last_clean":"2024-09-16T15:38:33.906366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:33.906366+0000","last_undegraded":"2024-09-16T15:38:33.906366+0000","last_fullsized":"2024-09-16T15:38:33.906366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-16T15:38:32.909818+0000","last_change":"2024-09-16T15:38:32.909818+0000","last_active":"2024-09-16T15:38:32.900130+0000","last_peered":"2024-09-16T15:38:32.900130+0000","last_clean":"2024-09-16T15:38:32.900130+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:38:32.909818+0000","last_undegraded":"2024-09-16T15:38:32.909818+0000","last_fullsized":"2024-09-16T15:38:32.909818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":62477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":184320,"data_stored":56660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":778240,"data_stored":647028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":75687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:38:39.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:38:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.798+0000 7fb508a00640 1 -- 172.21.15.60:0/2972268561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 msgr2=0x7fb4e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.798+0000 7fb508a00640 1 --2- 172.21.15.60:0/2972268561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 0x7fb4e80051c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009ed0 tx=0x7fb4e8005830 comp rx=0 tx=0).stop 2024-09-16T15:38:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.798+0000 7fb503e00640 1 -- 172.21.15.60:0/2972268561 reap_dead start 2024-09-16T15:38:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.798+0000 7fb508a00640 1 -- 172.21.15.60:0/2972268561 shutdown_connections 2024-09-16T15:38:39.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.798+0000 7fb508a00640 1 -- 172.21.15.60:0/2972268561 >> 172.21.15.60:0/2972268561 conn(0x7fb5040d3df0 msgr2=0x7fb5040d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.799+0000 7fb508a00640 1 -- 172.21.15.60:0/2972268561 shutdown_connections 2024-09-16T15:38:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.799+0000 7fb508a00640 1 -- 172.21.15.60:0/2972268561 wait complete. 2024-09-16T15:38:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.800+0000 7fb508a00640 1 Processor -- start 2024-09-16T15:38:39.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.800+0000 7fb508a00640 1 -- start start 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.801+0000 7fb508a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 0x7fb5041f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.801+0000 7fb508a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5041f7e40 0x7fb5040ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.801+0000 7fb508a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb504155070 0x7fb5040d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.801+0000 7fb508a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb5040b3d40 con 0x7fb4e8004d40 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.801+0000 7fb508a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5040b3bc0 con 0x7fb504155070 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.801+0000 7fb508a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb5040b3ec0 con 0x7fb5041f7e40 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 0x7fb5041f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 0x7fb5041f4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35132/0 (socket says 172.21.15.60:35132) 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 -- 172.21.15.60:0/800827210 learned_addr learned my addr 172.21.15.60:0/800827210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503e00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb504155070 0x7fb5040d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb4faa00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5041f7e40 0x7fb5040ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 -- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5041f7e40 msgr2=0x7fb5040ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5041f7e40 0x7fb5040ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 -- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb504155070 msgr2=0x7fb5040d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb504155070 0x7fb5040d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 -- 172.21.15.60:0/800827210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5040b80f0 con 0x7fb4e8004d40 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb503400640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 0x7fb5041f4850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40065e0 tx=0x7fb4f40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb4fbe00640 1 -- 172.21.15.60:0/800827210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4015070 con 0x7fb4e8004d40 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb4fbe00640 1 -- 172.21.15.60:0/800827210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4f400f980 con 0x7fb4e8004d40 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.802+0000 7fb4fbe00640 1 -- 172.21.15.60:0/800827210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400fdd0 con 0x7fb4e8004d40 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.803+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041f5130 con 0x7fb4e8004d40 2024-09-16T15:38:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.803+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041f56f0 con 0x7fb4e8004d40 2024-09-16T15:38:39.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.804+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5040699f0 con 0x7fb4e8004d40 2024-09-16T15:38:39.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.805+0000 7fb4fbe00640 1 -- 172.21.15.60:0/800827210 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb4f4011430 con 0x7fb4e8004d40 2024-09-16T15:38:39.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.806+0000 7fb4fbe00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb4e0078850 0x7fb4e007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:39.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.806+0000 7fb4fbe00640 1 -- 172.21.15.60:0/800827210 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb4f409b6d0 con 0x7fb4e8004d40 2024-09-16T15:38:39.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.806+0000 7fb4faa00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb4e0078850 0x7fb4e007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:39.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.807+0000 7fb4faa00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb4e0078850 0x7fb4e007ad10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5041f7da0 tx=0x7fb4ec005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:39.810+0000 7fb4fbe00640 1 -- 172.21.15.60:0/800827210 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5040699f0 con 0x7fb4e8004d40 2024-09-16T15:38:40.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.001+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5040b1770 con 0x7fb4e0078850 2024-09-16T15:38:40.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.007+0000 7fb4fbe00640 1 -- 172.21.15.60:0/800827210 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7fb5040b1770 con 0x7fb4e0078850 2024-09-16T15:38:40.011 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.010+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb4e0078850 msgr2=0x7fb4e007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.010+0000 7fb508a00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb4e0078850 0x7fb4e007ad10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5041f7da0 tx=0x7fb4ec005f90 comp rx=0 tx=0).stop 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.010+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 msgr2=0x7fb5041f4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.010+0000 7fb508a00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 0x7fb5041f4850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40065e0 tx=0x7fb4f40039c0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.011+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 shutdown_connections 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.011+0000 7fb508a00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb4e0078850 0x7fb4e007ad10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.011+0000 7fb508a00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb504155070 0x7fb5040d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.011+0000 7fb508a00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5041f7e40 0x7fb5040ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.011+0000 7fb508a00640 1 --2- 172.21.15.60:0/800827210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4e8004d40 0x7fb5041f4850 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.011+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 >> 172.21.15.60:0/800827210 conn(0x7fb5040d3df0 msgr2=0x7fb50415cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.011+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 shutdown_connections 2024-09-16T15:38:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.011+0000 7fb508a00640 1 -- 172.21.15.60:0/800827210 wait complete. 2024-09-16T15:38:40.028 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-16T15:38:39.709152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235928,"kb_used_data":3036,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738792,"statfs":{"total":773094113280,"available":772852523008,"internally_reserved":0,"allocated":3108864,"data_stored":1833872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001467"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328961+0000","last_change":"2024-09-16T15:38:34.948046+0000","last_active":"2024-09-16T15:38:36.328961+0000","last_peered":"2024-09-16T15:38:36.328961+0000","last_clean":"2024-09-16T15:38:36.328961+0000","last_became_active":"2024-09-16T15:38:34.947778+0000","last_became_peered":"2024-09-16T15:38:34.947778+0000","last_unstale":"2024-09-16T15:38:36.328961+0000","last_undegraded":"2024-09-16T15:38:36.328961+0000","last_fullsized":"2024-09-16T15:38:36.328961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329296+0000","last_change":"2024-09-16T15:38:34.934287+0000","last_active":"2024-09-16T15:38:36.329296+0000","last_peered":"2024-09-16T15:38:36.329296+0000","last_clean":"2024-09-16T15:38:36.329296+0000","last_became_active":"2024-09-16T15:38:34.934004+0000","last_became_peered":"2024-09-16T15:38:34.934004+0000","last_unstale":"2024-09-16T15:38:36.329296+0000","last_undegraded":"2024-09-16T15:38:36.329296+0000","last_fullsized":"2024-09-16T15:38:36.329296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:43.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330192+0000","last_change":"2024-09-16T15:38:34.937280+0000","last_active":"2024-09-16T15:38:36.330192+0000","last_peered":"2024-09-16T15:38:36.330192+0000","last_clean":"2024-09-16T15:38:36.330192+0000","last_became_active":"2024-09-16T15:38:34.936979+0000","last_became_peered":"2024-09-16T15:38:34.936979+0000","last_unstale":"2024-09-16T15:38:36.330192+0000","last_undegraded":"2024-09-16T15:38:36.330192+0000","last_fullsized":"2024-09-16T15:38:36.330192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:16.363921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328077+0000","last_change":"2024-09-16T15:38:34.956532+0000","last_active":"2024-09-16T15:38:36.328077+0000","last_peered":"2024-09-16T15:38:36.328077+0000","last_clean":"2024-09-16T15:38:36.328077+0000","last_became_active":"2024-09-16T15:38:34.956261+0000","last_became_peered":"2024-09-16T15:38:34.956261+0000","last_unstale":"2024-09-16T15:38:36.328077+0000","last_undegraded":"2024-09-16T15:38:36.328077+0000","last_fullsized":"2024-09-16T15:38:36.328077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328317+0000","last_change":"2024-09-16T15:38:34.946850+0000","last_active":"2024-09-16T15:38:36.328317+0000","last_peered":"2024-09-16T15:38:36.328317+0000","last_clean":"2024-09-16T15:38:36.328317+0000","last_became_active":"2024-09-16T15:38:34.946584+0000","last_became_peered":"2024-09-16T15:38:34.946584+0000","last_unstale":"2024-09-16T15:38:36.328317+0000","last_undegraded":"2024-09-16T15:38:36.328317+0000","last_fullsized":"2024-09-16T15:38:36.328317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328172+0000","last_change":"2024-09-16T15:38:34.946360+0000","last_active":"2024-09-16T15:38:36.328172+0000","last_peered":"2024-09-16T15:38:36.328172+0000","last_clean":"2024-09-16T15:38:36.328172+0000","last_became_active":"2024-09-16T15:38:34.946022+0000","last_became_peered":"2024-09-16T15:38:34.946022+0000","last_unstale":"2024-09-16T15:38:36.328172+0000","last_undegraded":"2024-09-16T15:38:36.328172+0000","last_fullsized":"2024-09-16T15:38:36.328172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328807+0000","last_change":"2024-09-16T15:38:34.948730+0000","last_active":"2024-09-16T15:38:36.328807+0000","last_peered":"2024-09-16T15:38:36.328807+0000","last_clean":"2024-09-16T15:38:36.328807+0000","last_became_active":"2024-09-16T15:38:34.948488+0000","last_became_peered":"2024-09-16T15:38:34.948488+0000","last_unstale":"2024-09-16T15:38:36.328807+0000","last_undegraded":"2024-09-16T15:38:36.328807+0000","last_fullsized":"2024-09-16T15:38:36.328807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327904+0000","last_change":"2024-09-16T15:38:34.939960+0000","last_active":"2024-09-16T15:38:36.327904+0000","last_peered":"2024-09-16T15:38:36.327904+0000","last_clean":"2024-09-16T15:38:36.327904+0000","last_became_active":"2024-09-16T15:38:34.939730+0000","last_became_peered":"2024-09-16T15:38:34.939730+0000","last_unstale":"2024-09-16T15:38:36.327904+0000","last_undegraded":"2024-09-16T15:38:36.327904+0000","last_fullsized":"2024-09-16T15:38:36.327904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:35.445142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329103+0000","last_change":"2024-09-16T15:38:34.963612+0000","last_active":"2024-09-16T15:38:36.329103+0000","last_peered":"2024-09-16T15:38:36.329103+0000","last_clean":"2024-09-16T15:38:36.329103+0000","last_became_active":"2024-09-16T15:38:34.963287+0000","last_became_peered":"2024-09-16T15:38:34.963287+0000","last_unstale":"2024-09-16T15:38:36.329103+0000","last_undegraded":"2024-09-16T15:38:36.329103+0000","last_fullsized":"2024-09-16T15:38:36.329103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:53.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935830+0000","last_change":"2024-09-16T15:38:34.959111+0000","last_active":"2024-09-16T15:38:35.935830+0000","last_peered":"2024-09-16T15:38:35.935830+0000","last_clean":"2024-09-16T15:38:35.935830+0000","last_became_active":"2024-09-16T15:38:34.958733+0000","last_became_peered":"2024-09-16T15:38:34.958733+0000","last_unstale":"2024-09-16T15:38:35.935830+0000","last_undegraded":"2024-09-16T15:38:35.935830+0000","last_fullsized":"2024-09-16T15:38:35.935830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:21.940460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934331+0000","last_change":"2024-09-16T15:38:34.963928+0000","last_active":"2024-09-16T15:38:35.934331+0000","last_peered":"2024-09-16T15:38:35.934331+0000","last_clean":"2024-09-16T15:38:35.934331+0000","last_became_active":"2024-09-16T15:38:34.963418+0000","last_became_peered":"2024-09-16T15:38:34.963418+0000","last_unstale":"2024-09-16T15:38:35.934331+0000","last_undegraded":"2024-09-16T15:38:35.934331+0000","last_fullsized":"2024-09-16T15:38:35.934331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:54.016578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935041+0000","last_change":"2024-09-16T15:38:34.951463+0000","last_active":"2024-09-16T15:38:35.935041+0000","last_peered":"2024-09-16T15:38:35.935041+0000","last_clean":"2024-09-16T15:38:35.935041+0000","last_became_active":"2024-09-16T15:38:34.949972+0000","last_became_peered":"2024-09-16T15:38:34.949972+0000","last_unstale":"2024-09-16T15:38:35.935041+0000","last_undegraded":"2024-09-16T15:38:35.935041+0000","last_fullsized":"2024-09-16T15:38:35.935041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:36.356482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328377+0000","last_change":"2024-09-16T15:38:34.936161+0000","last_active":"2024-09-16T15:38:36.328377+0000","last_peered":"2024-09-16T15:38:36.328377+0000","last_clean":"2024-09-16T15:38:36.328377+0000","last_became_active":"2024-09-16T15:38:34.935881+0000","last_became_peered":"2024-09-16T15:38:34.935881+0000","last_unstale":"2024-09-16T15:38:36.328377+0000","last_undegraded":"2024-09-16T15:38:36.328377+0000","last_fullsized":"2024-09-16T15:38:36.328377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329953+0000","last_change":"2024-09-16T15:38:34.934084+0000","last_active":"2024-09-16T15:38:36.329953+0000","last_peered":"2024-09-16T15:38:36.329953+0000","last_clean":"2024-09-16T15:38:36.329953+0000","last_became_active":"2024-09-16T15:38:34.933716+0000","last_became_peered":"2024-09-16T15:38:34.933716+0000","last_unstale":"2024-09-16T15:38:36.329953+0000","last_undegraded":"2024-09-16T15:38:36.329953+0000","last_fullsized":"2024-09-16T15:38:36.329953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:51.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328795+0000","last_change":"2024-09-16T15:38:34.951427+0000","last_active":"2024-09-16T15:38:36.328795+0000","last_peered":"2024-09-16T15:38:36.328795+0000","last_clean":"2024-09-16T15:38:36.328795+0000","last_became_active":"2024-09-16T15:38:34.949484+0000","last_became_peered":"2024-09-16T15:38:34.949484+0000","last_unstale":"2024-09-16T15:38:36.328795+0000","last_undegraded":"2024-09-16T15:38:36.328795+0000","last_fullsized":"2024-09-16T15:38:36.328795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:54.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327906+0000","last_change":"2024-09-16T15:38:34.948833+0000","last_active":"2024-09-16T15:38:36.327906+0000","last_peered":"2024-09-16T15:38:36.327906+0000","last_clean":"2024-09-16T15:38:36.327906+0000","last_became_active":"2024-09-16T15:38:34.948127+0000","last_became_peered":"2024-09-16T15:38:34.948127+0000","last_unstale":"2024-09-16T15:38:36.327906+0000","last_undegraded":"2024-09-16T15:38:36.327906+0000","last_fullsized":"2024-09-16T15:38:36.327906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329716+0000","last_change":"2024-09-16T15:38:34.948165+0000","last_active":"2024-09-16T15:38:36.329716+0000","last_peered":"2024-09-16T15:38:36.329716+0000","last_clean":"2024-09-16T15:38:36.329716+0000","last_became_active":"2024-09-16T15:38:34.947862+0000","last_became_peered":"2024-09-16T15:38:34.947862+0000","last_unstale":"2024-09-16T15:38:36.329716+0000","last_undegraded":"2024-09-16T15:38:36.329716+0000","last_fullsized":"2024-09-16T15:38:36.329716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:07.065693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932521+0000","last_change":"2024-09-16T15:38:34.936088+0000","last_active":"2024-09-16T15:38:35.932521+0000","last_peered":"2024-09-16T15:38:35.932521+0000","last_clean":"2024-09-16T15:38:35.932521+0000","last_became_active":"2024-09-16T15:38:34.935649+0000","last_became_peered":"2024-09-16T15:38:34.935649+0000","last_unstale":"2024-09-16T15:38:35.932521+0000","last_undegraded":"2024-09-16T15:38:35.932521+0000","last_fullsized":"2024-09-16T15:38:35.932521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:56.948937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328715+0000","last_change":"2024-09-16T15:38:34.951511+0000","last_active":"2024-09-16T15:38:36.328715+0000","last_peered":"2024-09-16T15:38:36.328715+0000","last_clean":"2024-09-16T15:38:36.328715+0000","last_became_active":"2024-09-16T15:38:34.951241+0000","last_became_peered":"2024-09-16T15:38:34.951241+0000","last_unstale":"2024-09-16T15:38:36.328715+0000","last_undegraded":"2024-09-16T15:38:36.328715+0000","last_fullsized":"2024-09-16T15:38:36.328715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:52.005508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934261+0000","last_change":"2024-09-16T15:38:34.950265+0000","last_active":"2024-09-16T15:38:35.934261+0000","last_peered":"2024-09-16T15:38:35.934261+0000","last_clean":"2024-09-16T15:38:35.934261+0000","last_became_active":"2024-09-16T15:38:34.949457+0000","last_became_peered":"2024-09-16T15:38:34.949457+0000","last_unstale":"2024-09-16T15:38:35.934261+0000","last_undegraded":"2024-09-16T15:38:35.934261+0000","last_fullsized":"2024-09-16T15:38:35.934261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:33.694559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330049+0000","last_change":"2024-09-16T15:38:34.942753+0000","last_active":"2024-09-16T15:38:36.330049+0000","last_peered":"2024-09-16T15:38:36.330049+0000","last_clean":"2024-09-16T15:38:36.330049+0000","last_became_active":"2024-09-16T15:38:34.940841+0000","last_became_peered":"2024-09-16T15:38:34.940841+0000","last_unstale":"2024-09-16T15:38:36.330049+0000","last_undegraded":"2024-09-16T15:38:36.330049+0000","last_fullsized":"2024-09-16T15:38:36.330049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:22.740035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328374+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328374+0000","last_peered":"2024-09-16T15:38:36.328374+0000","last_clean":"2024-09-16T15:38:36.328374+0000","last_became_active":"2024-09-16T15:38:34.958240+0000","last_became_peered":"2024-09-16T15:38:34.958240+0000","last_unstale":"2024-09-16T15:38:36.328374+0000","last_undegraded":"2024-09-16T15:38:36.328374+0000","last_fullsized":"2024-09-16T15:38:36.328374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328083+0000","last_change":"2024-09-16T15:38:34.952703+0000","last_active":"2024-09-16T15:38:36.328083+0000","last_peered":"2024-09-16T15:38:36.328083+0000","last_clean":"2024-09-16T15:38:36.328083+0000","last_became_active":"2024-09-16T15:38:34.949406+0000","last_became_peered":"2024-09-16T15:38:34.949406+0000","last_unstale":"2024-09-16T15:38:36.328083+0000","last_undegraded":"2024-09-16T15:38:36.328083+0000","last_fullsized":"2024-09-16T15:38:36.328083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:20.072155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328173+0000","last_change":"2024-09-16T15:38:34.948442+0000","last_active":"2024-09-16T15:38:36.328173+0000","last_peered":"2024-09-16T15:38:36.328173+0000","last_clean":"2024-09-16T15:38:36.328173+0000","last_became_active":"2024-09-16T15:38:34.947564+0000","last_became_peered":"2024-09-16T15:38:34.947564+0000","last_unstale":"2024-09-16T15:38:36.328173+0000","last_undegraded":"2024-09-16T15:38:36.328173+0000","last_fullsized":"2024-09-16T15:38:36.328173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328339+0000","last_change":"2024-09-16T15:38:34.940042+0000","last_active":"2024-09-16T15:38:36.328339+0000","last_peered":"2024-09-16T15:38:36.328339+0000","last_clean":"2024-09-16T15:38:36.328339+0000","last_became_active":"2024-09-16T15:38:34.939792+0000","last_became_peered":"2024-09-16T15:38:34.939792+0000","last_unstale":"2024-09-16T15:38:36.328339+0000","last_undegraded":"2024-09-16T15:38:36.328339+0000","last_fullsized":"2024-09-16T15:38:36.328339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.386845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935854+0000","last_change":"2024-09-16T15:38:34.938919+0000","last_active":"2024-09-16T15:38:35.935854+0000","last_peered":"2024-09-16T15:38:35.935854+0000","last_clean":"2024-09-16T15:38:35.935854+0000","last_became_active":"2024-09-16T15:38:34.938608+0000","last_became_peered":"2024-09-16T15:38:34.938608+0000","last_unstale":"2024-09-16T15:38:35.935854+0000","last_undegraded":"2024-09-16T15:38:35.935854+0000","last_fullsized":"2024-09-16T15:38:35.935854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:44.137157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329086+0000","last_change":"2024-09-16T15:38:34.934519+0000","last_active":"2024-09-16T15:38:36.329086+0000","last_peered":"2024-09-16T15:38:36.329086+0000","last_clean":"2024-09-16T15:38:36.329086+0000","last_became_active":"2024-09-16T15:38:34.932223+0000","last_became_peered":"2024-09-16T15:38:34.932223+0000","last_unstale":"2024-09-16T15:38:36.329086+0000","last_undegraded":"2024-09-16T15:38:36.329086+0000","last_fullsized":"2024-09-16T15:38:36.329086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.673101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329499+0000","last_change":"2024-09-16T15:38:34.941297+0000","last_active":"2024-09-16T15:38:36.329499+0000","last_peered":"2024-09-16T15:38:36.329499+0000","last_clean":"2024-09-16T15:38:36.329499+0000","last_became_active":"2024-09-16T15:38:34.941007+0000","last_became_peered":"2024-09-16T15:38:34.941007+0000","last_unstale":"2024-09-16T15:38:36.329499+0000","last_undegraded":"2024-09-16T15:38:36.329499+0000","last_fullsized":"2024-09-16T15:38:36.329499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330077+0000","last_change":"2024-09-16T15:38:34.941110+0000","last_active":"2024-09-16T15:38:36.330077+0000","last_peered":"2024-09-16T15:38:36.330077+0000","last_clean":"2024-09-16T15:38:36.330077+0000","last_became_active":"2024-09-16T15:38:34.940650+0000","last_became_peered":"2024-09-16T15:38:34.940650+0000","last_unstale":"2024-09-16T15:38:36.330077+0000","last_undegraded":"2024-09-16T15:38:36.330077+0000","last_fullsized":"2024-09-16T15:38:36.330077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:26.093004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328522+0000","last_change":"2024-09-16T15:38:34.952930+0000","last_active":"2024-09-16T15:38:36.328522+0000","last_peered":"2024-09-16T15:38:36.328522+0000","last_clean":"2024-09-16T15:38:36.328522+0000","last_became_active":"2024-09-16T15:38:34.952652+0000","last_became_peered":"2024-09-16T15:38:34.952652+0000","last_unstale":"2024-09-16T15:38:36.328522+0000","last_undegraded":"2024-09-16T15:38:36.328522+0000","last_fullsized":"2024-09-16T15:38:36.328522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932490+0000","last_change":"2024-09-16T15:38:34.961835+0000","last_active":"2024-09-16T15:38:35.932490+0000","last_peered":"2024-09-16T15:38:35.932490+0000","last_clean":"2024-09-16T15:38:35.932490+0000","last_became_active":"2024-09-16T15:38:34.961529+0000","last_became_peered":"2024-09-16T15:38:34.961529+0000","last_unstale":"2024-09-16T15:38:35.932490+0000","last_undegraded":"2024-09-16T15:38:35.932490+0000","last_fullsized":"2024-09-16T15:38:35.932490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:11.053973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933841+0000","last_change":"2024-09-16T15:38:34.942442+0000","last_active":"2024-09-16T15:38:35.933841+0000","last_peered":"2024-09-16T15:38:35.933841+0000","last_clean":"2024-09-16T15:38:35.933841+0000","last_became_active":"2024-09-16T15:38:34.942144+0000","last_became_peered":"2024-09-16T15:38:34.942144+0000","last_unstale":"2024-09-16T15:38:35.933841+0000","last_undegraded":"2024-09-16T15:38:35.933841+0000","last_fullsized":"2024-09-16T15:38:35.933841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:43.399475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934635+0000","last_change":"2024-09-16T15:38:34.941230+0000","last_active":"2024-09-16T15:38:35.934635+0000","last_peered":"2024-09-16T15:38:35.934635+0000","last_clean":"2024-09-16T15:38:35.934635+0000","last_became_active":"2024-09-16T15:38:34.940613+0000","last_became_peered":"2024-09-16T15:38:34.940613+0000","last_unstale":"2024-09-16T15:38:35.934635+0000","last_undegraded":"2024-09-16T15:38:35.934635+0000","last_fullsized":"2024-09-16T15:38:35.934635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:15.007186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328673+0000","last_change":"2024-09-16T15:38:33.949962+0000","last_active":"2024-09-16T15:38:36.328673+0000","last_peered":"2024-09-16T15:38:36.328673+0000","last_clean":"2024-09-16T15:38:36.328673+0000","last_became_active":"2024-09-16T15:38:33.949620+0000","last_became_peered":"2024-09-16T15:38:33.949620+0000","last_unstale":"2024-09-16T15:38:36.328673+0000","last_undegraded":"2024-09-16T15:38:36.328673+0000","last_fullsized":"2024-09-16T15:38:36.328673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330281+0000","last_change":"2024-09-16T15:38:34.947478+0000","last_active":"2024-09-16T15:38:36.330281+0000","last_peered":"2024-09-16T15:38:36.330281+0000","last_clean":"2024-09-16T15:38:36.330281+0000","last_became_active":"2024-09-16T15:38:34.947158+0000","last_became_peered":"2024-09-16T15:38:34.947158+0000","last_unstale":"2024-09-16T15:38:36.330281+0000","last_undegraded":"2024-09-16T15:38:36.330281+0000","last_fullsized":"2024-09-16T15:38:36.330281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:24.497838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933892+0000","last_change":"2024-09-16T15:38:33.931830+0000","last_active":"2024-09-16T15:38:35.933892+0000","last_peered":"2024-09-16T15:38:35.933892+0000","last_clean":"2024-09-16T15:38:35.933892+0000","last_became_active":"2024-09-16T15:38:33.931129+0000","last_became_peered":"2024-09-16T15:38:33.931129+0000","last_unstale":"2024-09-16T15:38:35.933892+0000","last_undegraded":"2024-09-16T15:38:35.933892+0000","last_fullsized":"2024-09-16T15:38:35.933892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:54.747927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327884+0000","last_change":"2024-09-16T15:38:33.943597+0000","last_active":"2024-09-16T15:38:36.327884+0000","last_peered":"2024-09-16T15:38:36.327884+0000","last_clean":"2024-09-16T15:38:36.327884+0000","last_became_active":"2024-09-16T15:38:33.940904+0000","last_became_peered":"2024-09-16T15:38:33.940904+0000","last_unstale":"2024-09-16T15:38:36.327884+0000","last_undegraded":"2024-09-16T15:38:36.327884+0000","last_fullsized":"2024-09-16T15:38:36.327884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:13.818889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328165+0000","last_change":"2024-09-16T15:38:34.942424+0000","last_active":"2024-09-16T15:38:36.328165+0000","last_peered":"2024-09-16T15:38:36.328165+0000","last_clean":"2024-09-16T15:38:36.328165+0000","last_became_active":"2024-09-16T15:38:34.942111+0000","last_became_peered":"2024-09-16T15:38:34.942111+0000","last_unstale":"2024-09-16T15:38:36.328165+0000","last_undegraded":"2024-09-16T15:38:36.328165+0000","last_fullsized":"2024-09-16T15:38:36.328165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:52.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328244+0000","last_change":"2024-09-16T15:38:33.945014+0000","last_active":"2024-09-16T15:38:36.328244+0000","last_peered":"2024-09-16T15:38:36.328244+0000","last_clean":"2024-09-16T15:38:36.328244+0000","last_became_active":"2024-09-16T15:38:33.944742+0000","last_became_peered":"2024-09-16T15:38:33.944742+0000","last_unstale":"2024-09-16T15:38:36.328244+0000","last_undegraded":"2024-09-16T15:38:36.328244+0000","last_fullsized":"2024-09-16T15:38:36.328244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328641+0000","last_change":"2024-09-16T15:38:34.933709+0000","last_active":"2024-09-16T15:38:36.328641+0000","last_peered":"2024-09-16T15:38:36.328641+0000","last_clean":"2024-09-16T15:38:36.328641+0000","last_became_active":"2024-09-16T15:38:34.932126+0000","last_became_peered":"2024-09-16T15:38:34.932126+0000","last_unstale":"2024-09-16T15:38:36.328641+0000","last_undegraded":"2024-09-16T15:38:36.328641+0000","last_fullsized":"2024-09-16T15:38:36.328641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:44.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935745+0000","last_change":"2024-09-16T15:38:33.945379+0000","last_active":"2024-09-16T15:38:35.935745+0000","last_peered":"2024-09-16T15:38:35.935745+0000","last_clean":"2024-09-16T15:38:35.935745+0000","last_became_active":"2024-09-16T15:38:33.944531+0000","last_became_peered":"2024-09-16T15:38:33.944531+0000","last_unstale":"2024-09-16T15:38:35.935745+0000","last_undegraded":"2024-09-16T15:38:35.935745+0000","last_fullsized":"2024-09-16T15:38:35.935745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:09.939574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329480+0000","last_change":"2024-09-16T15:38:34.961412+0000","last_active":"2024-09-16T15:38:36.329480+0000","last_peered":"2024-09-16T15:38:36.329480+0000","last_clean":"2024-09-16T15:38:36.329480+0000","last_became_active":"2024-09-16T15:38:34.961004+0000","last_became_peered":"2024-09-16T15:38:34.961004+0000","last_unstale":"2024-09-16T15:38:36.329480+0000","last_undegraded":"2024-09-16T15:38:36.329480+0000","last_fullsized":"2024-09-16T15:38:36.329480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:39.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329349+0000","last_change":"2024-09-16T15:38:33.942028+0000","last_active":"2024-09-16T15:38:36.329349+0000","last_peered":"2024-09-16T15:38:36.329349+0000","last_clean":"2024-09-16T15:38:36.329349+0000","last_became_active":"2024-09-16T15:38:33.940695+0000","last_became_peered":"2024-09-16T15:38:33.940695+0000","last_unstale":"2024-09-16T15:38:36.329349+0000","last_undegraded":"2024-09-16T15:38:36.329349+0000","last_fullsized":"2024-09-16T15:38:36.329349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330396+0000","last_change":"2024-09-16T15:38:34.937478+0000","last_active":"2024-09-16T15:38:36.330396+0000","last_peered":"2024-09-16T15:38:36.330396+0000","last_clean":"2024-09-16T15:38:36.330396+0000","last_became_active":"2024-09-16T15:38:34.936844+0000","last_became_peered":"2024-09-16T15:38:34.936844+0000","last_unstale":"2024-09-16T15:38:36.330396+0000","last_undegraded":"2024-09-16T15:38:36.330396+0000","last_fullsized":"2024-09-16T15:38:36.330396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:42.401407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328973+0000","last_change":"2024-09-16T15:38:33.942385+0000","last_active":"2024-09-16T15:38:36.328973+0000","last_peered":"2024-09-16T15:38:36.328973+0000","last_clean":"2024-09-16T15:38:36.328973+0000","last_became_active":"2024-09-16T15:38:33.941960+0000","last_became_peered":"2024-09-16T15:38:33.941960+0000","last_unstale":"2024-09-16T15:38:36.328973+0000","last_undegraded":"2024-09-16T15:38:36.328973+0000","last_fullsized":"2024-09-16T15:38:36.328973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:08.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935156+0000","last_change":"2024-09-16T15:38:34.955492+0000","last_active":"2024-09-16T15:38:35.935156+0000","last_peered":"2024-09-16T15:38:35.935156+0000","last_clean":"2024-09-16T15:38:35.935156+0000","last_became_active":"2024-09-16T15:38:34.955230+0000","last_became_peered":"2024-09-16T15:38:34.955230+0000","last_unstale":"2024-09-16T15:38:35.935156+0000","last_undegraded":"2024-09-16T15:38:35.935156+0000","last_fullsized":"2024-09-16T15:38:35.935156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:13.656183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934327+0000","last_change":"2024-09-16T15:38:33.939711+0000","last_active":"2024-09-16T15:38:35.934327+0000","last_peered":"2024-09-16T15:38:35.934327+0000","last_clean":"2024-09-16T15:38:35.934327+0000","last_became_active":"2024-09-16T15:38:33.937858+0000","last_became_peered":"2024-09-16T15:38:33.937858+0000","last_unstale":"2024-09-16T15:38:35.934327+0000","last_undegraded":"2024-09-16T15:38:35.934327+0000","last_fullsized":"2024-09-16T15:38:35.934327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:42.581730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328596+0000","last_change":"2024-09-16T15:38:34.947613+0000","last_active":"2024-09-16T15:38:36.328596+0000","last_peered":"2024-09-16T15:38:36.328596+0000","last_clean":"2024-09-16T15:38:36.328596+0000","last_became_active":"2024-09-16T15:38:34.947258+0000","last_became_peered":"2024-09-16T15:38:34.947258+0000","last_unstale":"2024-09-16T15:38:36.328596+0000","last_undegraded":"2024-09-16T15:38:36.328596+0000","last_fullsized":"2024-09-16T15:38:36.328596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.135119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330167+0000","last_change":"2024-09-16T15:38:33.936316+0000","last_active":"2024-09-16T15:38:36.330167+0000","last_peered":"2024-09-16T15:38:36.330167+0000","last_clean":"2024-09-16T15:38:36.330167+0000","last_became_active":"2024-09-16T15:38:33.936002+0000","last_became_peered":"2024-09-16T15:38:33.936002+0000","last_unstale":"2024-09-16T15:38:36.330167+0000","last_undegraded":"2024-09-16T15:38:36.330167+0000","last_fullsized":"2024-09-16T15:38:36.330167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.796157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329907+0000","last_change":"2024-09-16T15:38:34.942140+0000","last_active":"2024-09-16T15:38:36.329907+0000","last_peered":"2024-09-16T15:38:36.329907+0000","last_clean":"2024-09-16T15:38:36.329907+0000","last_became_active":"2024-09-16T15:38:34.940716+0000","last_became_peered":"2024-09-16T15:38:34.940716+0000","last_unstale":"2024-09-16T15:38:36.329907+0000","last_undegraded":"2024-09-16T15:38:36.329907+0000","last_fullsized":"2024-09-16T15:38:36.329907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:55.566537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328440+0000","last_change":"2024-09-16T15:38:33.938693+0000","last_active":"2024-09-16T15:38:36.328440+0000","last_peered":"2024-09-16T15:38:36.328440+0000","last_clean":"2024-09-16T15:38:36.328440+0000","last_became_active":"2024-09-16T15:38:33.938207+0000","last_became_peered":"2024-09-16T15:38:33.938207+0000","last_unstale":"2024-09-16T15:38:36.328440+0000","last_undegraded":"2024-09-16T15:38:36.328440+0000","last_fullsized":"2024-09-16T15:38:36.328440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328857+0000","last_change":"2024-09-16T15:38:34.956559+0000","last_active":"2024-09-16T15:38:36.328857+0000","last_peered":"2024-09-16T15:38:36.328857+0000","last_clean":"2024-09-16T15:38:36.328857+0000","last_became_active":"2024-09-16T15:38:34.956290+0000","last_became_peered":"2024-09-16T15:38:34.956290+0000","last_unstale":"2024-09-16T15:38:36.328857+0000","last_undegraded":"2024-09-16T15:38:36.328857+0000","last_fullsized":"2024-09-16T15:38:36.328857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:34.525651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933781+0000","last_change":"2024-09-16T15:38:33.937022+0000","last_active":"2024-09-16T15:38:35.933781+0000","last_peered":"2024-09-16T15:38:35.933781+0000","last_clean":"2024-09-16T15:38:35.933781+0000","last_became_active":"2024-09-16T15:38:33.936692+0000","last_became_peered":"2024-09-16T15:38:33.936692+0000","last_unstale":"2024-09-16T15:38:35.933781+0000","last_undegraded":"2024-09-16T15:38:35.933781+0000","last_fullsized":"2024-09-16T15:38:35.933781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:23.195213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933838+0000","last_change":"2024-09-16T15:38:34.933429+0000","last_active":"2024-09-16T15:38:35.933838+0000","last_peered":"2024-09-16T15:38:35.933838+0000","last_clean":"2024-09-16T15:38:35.933838+0000","last_became_active":"2024-09-16T15:38:34.933116+0000","last_became_peered":"2024-09-16T15:38:34.933116+0000","last_unstale":"2024-09-16T15:38:35.933838+0000","last_undegraded":"2024-09-16T15:38:35.933838+0000","last_fullsized":"2024-09-16T15:38:35.933838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:16.415478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932158+0000","last_change":"2024-09-16T15:38:33.929097+0000","last_active":"2024-09-16T15:38:35.932158+0000","last_peered":"2024-09-16T15:38:35.932158+0000","last_clean":"2024-09-16T15:38:35.932158+0000","last_became_active":"2024-09-16T15:38:33.928783+0000","last_became_peered":"2024-09-16T15:38:33.928783+0000","last_unstale":"2024-09-16T15:38:35.932158+0000","last_undegraded":"2024-09-16T15:38:35.932158+0000","last_fullsized":"2024-09-16T15:38:35.932158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327843+0000","last_change":"2024-09-16T15:38:34.955962+0000","last_active":"2024-09-16T15:38:36.327843+0000","last_peered":"2024-09-16T15:38:36.327843+0000","last_clean":"2024-09-16T15:38:36.327843+0000","last_became_active":"2024-09-16T15:38:34.955617+0000","last_became_peered":"2024-09-16T15:38:34.955617+0000","last_unstale":"2024-09-16T15:38:36.327843+0000","last_undegraded":"2024-09-16T15:38:36.327843+0000","last_fullsized":"2024-09-16T15:38:36.327843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:35.342731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935301+0000","last_change":"2024-09-16T15:38:33.933311+0000","last_active":"2024-09-16T15:38:35.935301+0000","last_peered":"2024-09-16T15:38:35.935301+0000","last_clean":"2024-09-16T15:38:35.935301+0000","last_became_active":"2024-09-16T15:38:33.932882+0000","last_became_peered":"2024-09-16T15:38:33.932882+0000","last_unstale":"2024-09-16T15:38:35.935301+0000","last_undegraded":"2024-09-16T15:38:35.935301+0000","last_fullsized":"2024-09-16T15:38:35.935301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:43.856688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328955+0000","last_change":"2024-09-16T15:38:34.959213+0000","last_active":"2024-09-16T15:38:36.328955+0000","last_peered":"2024-09-16T15:38:36.328955+0000","last_clean":"2024-09-16T15:38:36.328955+0000","last_became_active":"2024-09-16T15:38:34.958928+0000","last_became_peered":"2024-09-16T15:38:34.958928+0000","last_unstale":"2024-09-16T15:38:36.328955+0000","last_undegraded":"2024-09-16T15:38:36.328955+0000","last_fullsized":"2024-09-16T15:38:36.328955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:51.093521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932531+0000","last_change":"2024-09-16T15:38:33.939655+0000","last_active":"2024-09-16T15:38:35.932531+0000","last_peered":"2024-09-16T15:38:35.932531+0000","last_clean":"2024-09-16T15:38:35.932531+0000","last_became_active":"2024-09-16T15:38:33.939329+0000","last_became_peered":"2024-09-16T15:38:33.939329+0000","last_unstale":"2024-09-16T15:38:35.932531+0000","last_undegraded":"2024-09-16T15:38:35.932531+0000","last_fullsized":"2024-09-16T15:38:35.932531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329218+0000","last_change":"2024-09-16T15:38:34.956364+0000","last_active":"2024-09-16T15:38:36.329218+0000","last_peered":"2024-09-16T15:38:36.329218+0000","last_clean":"2024-09-16T15:38:36.329218+0000","last_became_active":"2024-09-16T15:38:34.956041+0000","last_became_peered":"2024-09-16T15:38:34.956041+0000","last_unstale":"2024-09-16T15:38:36.329218+0000","last_undegraded":"2024-09-16T15:38:36.329218+0000","last_fullsized":"2024-09-16T15:38:36.329218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:26.092978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329039+0000","last_change":"2024-09-16T15:38:33.940469+0000","last_active":"2024-09-16T15:38:36.329039+0000","last_peered":"2024-09-16T15:38:36.329039+0000","last_clean":"2024-09-16T15:38:36.329039+0000","last_became_active":"2024-09-16T15:38:33.939752+0000","last_became_peered":"2024-09-16T15:38:33.939752+0000","last_unstale":"2024-09-16T15:38:36.329039+0000","last_undegraded":"2024-09-16T15:38:36.329039+0000","last_fullsized":"2024-09-16T15:38:36.329039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:31.202600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328853+0000","last_change":"2024-09-16T15:38:34.946270+0000","last_active":"2024-09-16T15:38:36.328853+0000","last_peered":"2024-09-16T15:38:36.328853+0000","last_clean":"2024-09-16T15:38:36.328853+0000","last_became_active":"2024-09-16T15:38:34.945965+0000","last_became_peered":"2024-09-16T15:38:34.945965+0000","last_unstale":"2024-09-16T15:38:36.328853+0000","last_undegraded":"2024-09-16T15:38:36.328853+0000","last_fullsized":"2024-09-16T15:38:36.328853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932369+0000","last_change":"2024-09-16T15:38:33.937925+0000","last_active":"2024-09-16T15:38:35.932369+0000","last_peered":"2024-09-16T15:38:35.932369+0000","last_clean":"2024-09-16T15:38:35.932369+0000","last_became_active":"2024-09-16T15:38:33.937648+0000","last_became_peered":"2024-09-16T15:38:33.937648+0000","last_unstale":"2024-09-16T15:38:35.932369+0000","last_undegraded":"2024-09-16T15:38:35.932369+0000","last_fullsized":"2024-09-16T15:38:35.932369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328407+0000","last_change":"2024-09-16T15:38:34.964388+0000","last_active":"2024-09-16T15:38:36.328407+0000","last_peered":"2024-09-16T15:38:36.328407+0000","last_clean":"2024-09-16T15:38:36.328407+0000","last_became_active":"2024-09-16T15:38:34.964136+0000","last_became_peered":"2024-09-16T15:38:34.964136+0000","last_unstale":"2024-09-16T15:38:36.328407+0000","last_undegraded":"2024-09-16T15:38:36.328407+0000","last_fullsized":"2024-09-16T15:38:36.328407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:21.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327747+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:38:36.327747+0000","last_peered":"2024-09-16T15:38:36.327747+0000","last_clean":"2024-09-16T15:38:36.327747+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:38:36.327747+0000","last_undegraded":"2024-09-16T15:38:36.327747+0000","last_fullsized":"2024-09-16T15:38:36.327747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327930+0000","last_change":"2024-09-16T15:38:33.941937+0000","last_active":"2024-09-16T15:38:36.327930+0000","last_peered":"2024-09-16T15:38:36.327930+0000","last_clean":"2024-09-16T15:38:36.327930+0000","last_became_active":"2024-09-16T15:38:33.941212+0000","last_became_peered":"2024-09-16T15:38:33.941212+0000","last_unstale":"2024-09-16T15:38:36.327930+0000","last_undegraded":"2024-09-16T15:38:36.327930+0000","last_fullsized":"2024-09-16T15:38:36.327930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328444+0000","last_change":"2024-09-16T15:38:34.962489+0000","last_active":"2024-09-16T15:38:36.328444+0000","last_peered":"2024-09-16T15:38:36.328444+0000","last_clean":"2024-09-16T15:38:36.328444+0000","last_became_active":"2024-09-16T15:38:34.962245+0000","last_became_peered":"2024-09-16T15:38:34.962245+0000","last_unstale":"2024-09-16T15:38:36.328444+0000","last_undegraded":"2024-09-16T15:38:36.328444+0000","last_fullsized":"2024-09-16T15:38:36.328444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.955896+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.955595+0000","last_became_peered":"2024-09-16T15:38:33.955595+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329773+0000","last_change":"2024-09-16T15:38:34.956268+0000","last_active":"2024-09-16T15:38:36.329773+0000","last_peered":"2024-09-16T15:38:36.329773+0000","last_clean":"2024-09-16T15:38:36.329773+0000","last_became_active":"2024-09-16T15:38:34.956010+0000","last_became_peered":"2024-09-16T15:38:34.956010+0000","last_unstale":"2024-09-16T15:38:36.329773+0000","last_undegraded":"2024-09-16T15:38:36.329773+0000","last_fullsized":"2024-09-16T15:38:36.329773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:02.100232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932349+0000","last_change":"2024-09-16T15:38:33.941933+0000","last_active":"2024-09-16T15:38:35.932349+0000","last_peered":"2024-09-16T15:38:35.932349+0000","last_clean":"2024-09-16T15:38:35.932349+0000","last_became_active":"2024-09-16T15:38:33.941648+0000","last_became_peered":"2024-09-16T15:38:33.941648+0000","last_unstale":"2024-09-16T15:38:35.932349+0000","last_undegraded":"2024-09-16T15:38:35.932349+0000","last_fullsized":"2024-09-16T15:38:35.932349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328622+0000","last_change":"2024-09-16T15:38:34.932401+0000","last_active":"2024-09-16T15:38:36.328622+0000","last_peered":"2024-09-16T15:38:36.328622+0000","last_clean":"2024-09-16T15:38:36.328622+0000","last_became_active":"2024-09-16T15:38:34.931203+0000","last_became_peered":"2024-09-16T15:38:34.931203+0000","last_unstale":"2024-09-16T15:38:36.328622+0000","last_undegraded":"2024-09-16T15:38:36.328622+0000","last_fullsized":"2024-09-16T15:38:36.328622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:24.483689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328535+0000","last_change":"2024-09-16T15:38:33.947515+0000","last_active":"2024-09-16T15:38:36.328535+0000","last_peered":"2024-09-16T15:38:36.328535+0000","last_clean":"2024-09-16T15:38:36.328535+0000","last_became_active":"2024-09-16T15:38:33.946118+0000","last_became_peered":"2024-09-16T15:38:33.946118+0000","last_unstale":"2024-09-16T15:38:36.328535+0000","last_undegraded":"2024-09-16T15:38:36.328535+0000","last_fullsized":"2024-09-16T15:38:36.328535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935354+0000","last_change":"2024-09-16T15:38:34.951051+0000","last_active":"2024-09-16T15:38:35.935354+0000","last_peered":"2024-09-16T15:38:35.935354+0000","last_clean":"2024-09-16T15:38:35.935354+0000","last_became_active":"2024-09-16T15:38:34.950115+0000","last_became_peered":"2024-09-16T15:38:34.950115+0000","last_unstale":"2024-09-16T15:38:35.935354+0000","last_undegraded":"2024-09-16T15:38:35.935354+0000","last_fullsized":"2024-09-16T15:38:35.935354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:29.584103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328940+0000","last_change":"2024-09-16T15:38:33.936405+0000","last_active":"2024-09-16T15:38:36.328940+0000","last_peered":"2024-09-16T15:38:36.328940+0000","last_clean":"2024-09-16T15:38:36.328940+0000","last_became_active":"2024-09-16T15:38:33.935318+0000","last_became_peered":"2024-09-16T15:38:33.935318+0000","last_unstale":"2024-09-16T15:38:36.328940+0000","last_undegraded":"2024-09-16T15:38:36.328940+0000","last_fullsized":"2024-09-16T15:38:36.328940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:27.366083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329408+0000","last_change":"2024-09-16T15:38:34.961561+0000","last_active":"2024-09-16T15:38:36.329408+0000","last_peered":"2024-09-16T15:38:36.329408+0000","last_clean":"2024-09-16T15:38:36.329408+0000","last_became_active":"2024-09-16T15:38:34.961174+0000","last_became_peered":"2024-09-16T15:38:34.961174+0000","last_unstale":"2024-09-16T15:38:36.329408+0000","last_undegraded":"2024-09-16T15:38:36.329408+0000","last_fullsized":"2024-09-16T15:38:36.329408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328656+0000","last_change":"2024-09-16T15:38:33.955332+0000","last_active":"2024-09-16T15:38:36.328656+0000","last_peered":"2024-09-16T15:38:36.328656+0000","last_clean":"2024-09-16T15:38:36.328656+0000","last_became_active":"2024-09-16T15:38:33.954340+0000","last_became_peered":"2024-09-16T15:38:33.954340+0000","last_unstale":"2024-09-16T15:38:36.328656+0000","last_undegraded":"2024-09-16T15:38:36.328656+0000","last_fullsized":"2024-09-16T15:38:36.328656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:45.798957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328724+0000","last_change":"2024-09-16T15:38:34.928319+0000","last_active":"2024-09-16T15:38:36.328724+0000","last_peered":"2024-09-16T15:38:36.328724+0000","last_clean":"2024-09-16T15:38:36.328724+0000","last_became_active":"2024-09-16T15:38:34.928058+0000","last_became_peered":"2024-09-16T15:38:34.928058+0000","last_unstale":"2024-09-16T15:38:36.328724+0000","last_undegraded":"2024-09-16T15:38:36.328724+0000","last_fullsized":"2024-09-16T15:38:36.328724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:00.238312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.929048+0000","last_change":"2024-09-16T15:38:33.931369+0000","last_active":"2024-09-16T15:38:36.929048+0000","last_peered":"2024-09-16T15:38:36.929048+0000","last_clean":"2024-09-16T15:38:36.929048+0000","last_became_active":"2024-09-16T15:38:33.928339+0000","last_became_peered":"2024-09-16T15:38:33.928339+0000","last_unstale":"2024-09-16T15:38:36.929048+0000","last_undegraded":"2024-09-16T15:38:36.929048+0000","last_fullsized":"2024-09-16T15:38:36.929048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:44.073989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329649+0000","last_change":"2024-09-16T15:38:34.942463+0000","last_active":"2024-09-16T15:38:36.329649+0000","last_peered":"2024-09-16T15:38:36.329649+0000","last_clean":"2024-09-16T15:38:36.329649+0000","last_became_active":"2024-09-16T15:38:34.940343+0000","last_became_peered":"2024-09-16T15:38:34.940343+0000","last_unstale":"2024-09-16T15:38:36.329649+0000","last_undegraded":"2024-09-16T15:38:36.329649+0000","last_fullsized":"2024-09-16T15:38:36.329649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935853+0000","last_change":"2024-09-16T15:38:33.947537+0000","last_active":"2024-09-16T15:38:35.935853+0000","last_peered":"2024-09-16T15:38:35.935853+0000","last_clean":"2024-09-16T15:38:35.935853+0000","last_became_active":"2024-09-16T15:38:33.946909+0000","last_became_peered":"2024-09-16T15:38:33.946909+0000","last_unstale":"2024-09-16T15:38:35.935853+0000","last_undegraded":"2024-09-16T15:38:35.935853+0000","last_fullsized":"2024-09-16T15:38:35.935853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:03.179729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330207+0000","last_change":"2024-09-16T15:38:34.941844+0000","last_active":"2024-09-16T15:38:36.330207+0000","last_peered":"2024-09-16T15:38:36.330207+0000","last_clean":"2024-09-16T15:38:36.330207+0000","last_became_active":"2024-09-16T15:38:34.940573+0000","last_became_peered":"2024-09-16T15:38:34.940573+0000","last_unstale":"2024-09-16T15:38:36.330207+0000","last_undegraded":"2024-09-16T15:38:36.330207+0000","last_fullsized":"2024-09-16T15:38:36.330207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:17.824920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.954548+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.954266+0000","last_became_peered":"2024-09-16T15:38:33.954266+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:14.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934862+0000","last_change":"2024-09-16T15:38:34.951875+0000","last_active":"2024-09-16T15:38:35.934862+0000","last_peered":"2024-09-16T15:38:35.934862+0000","last_clean":"2024-09-16T15:38:35.934862+0000","last_became_active":"2024-09-16T15:38:34.946733+0000","last_became_peered":"2024-09-16T15:38:34.946733+0000","last_unstale":"2024-09-16T15:38:35.934862+0000","last_undegraded":"2024-09-16T15:38:35.934862+0000","last_fullsized":"2024-09-16T15:38:35.934862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:35.351445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.947052+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.946593+0000","last_became_peered":"2024-09-16T15:38:33.946593+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:37.618001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328906+0000","last_change":"2024-09-16T15:38:34.933312+0000","last_active":"2024-09-16T15:38:36.328906+0000","last_peered":"2024-09-16T15:38:36.328906+0000","last_clean":"2024-09-16T15:38:36.328906+0000","last_became_active":"2024-09-16T15:38:34.933037+0000","last_became_peered":"2024-09-16T15:38:34.933037+0000","last_unstale":"2024-09-16T15:38:36.328906+0000","last_undegraded":"2024-09-16T15:38:36.328906+0000","last_fullsized":"2024-09-16T15:38:36.328906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:15.092528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.943983+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.943683+0000","last_became_peered":"2024-09-16T15:38:33.943683+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.572327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328590+0000","last_change":"2024-09-16T15:38:34.943957+0000","last_active":"2024-09-16T15:38:36.328590+0000","last_peered":"2024-09-16T15:38:36.328590+0000","last_clean":"2024-09-16T15:38:36.328590+0000","last_became_active":"2024-09-16T15:38:34.941591+0000","last_became_peered":"2024-09-16T15:38:34.941591+0000","last_unstale":"2024-09-16T15:38:36.328590+0000","last_undegraded":"2024-09-16T15:38:36.328590+0000","last_fullsized":"2024-09-16T15:38:36.328590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328378+0000","last_change":"2024-09-16T15:38:33.954584+0000","last_active":"2024-09-16T15:38:36.328378+0000","last_peered":"2024-09-16T15:38:36.328378+0000","last_clean":"2024-09-16T15:38:36.328378+0000","last_became_active":"2024-09-16T15:38:33.954279+0000","last_became_peered":"2024-09-16T15:38:33.954279+0000","last_unstale":"2024-09-16T15:38:36.328378+0000","last_undegraded":"2024-09-16T15:38:36.328378+0000","last_fullsized":"2024-09-16T15:38:36.328378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933817+0000","last_change":"2024-09-16T15:38:34.949723+0000","last_active":"2024-09-16T15:38:35.933817+0000","last_peered":"2024-09-16T15:38:35.933817+0000","last_clean":"2024-09-16T15:38:35.933817+0000","last_became_active":"2024-09-16T15:38:34.949433+0000","last_became_peered":"2024-09-16T15:38:34.949433+0000","last_unstale":"2024-09-16T15:38:35.933817+0000","last_undegraded":"2024-09-16T15:38:35.933817+0000","last_fullsized":"2024-09-16T15:38:35.933817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:33.866690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329345+0000","last_change":"2024-09-16T15:38:34.961924+0000","last_active":"2024-09-16T15:38:36.329345+0000","last_peered":"2024-09-16T15:38:36.329345+0000","last_clean":"2024-09-16T15:38:36.329345+0000","last_became_active":"2024-09-16T15:38:34.961607+0000","last_became_peered":"2024-09-16T15:38:34.961607+0000","last_unstale":"2024-09-16T15:38:36.329345+0000","last_undegraded":"2024-09-16T15:38:36.329345+0000","last_fullsized":"2024-09-16T15:38:36.329345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:58.529110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328347+0000","last_change":"2024-09-16T15:38:33.955384+0000","last_active":"2024-09-16T15:38:36.328347+0000","last_peered":"2024-09-16T15:38:36.328347+0000","last_clean":"2024-09-16T15:38:36.328347+0000","last_became_active":"2024-09-16T15:38:33.954532+0000","last_became_peered":"2024-09-16T15:38:33.954532+0000","last_unstale":"2024-09-16T15:38:36.328347+0000","last_undegraded":"2024-09-16T15:38:36.328347+0000","last_fullsized":"2024-09-16T15:38:36.328347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:28.577867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328852+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328852+0000","last_peered":"2024-09-16T15:38:36.328852+0000","last_clean":"2024-09-16T15:38:36.328852+0000","last_became_active":"2024-09-16T15:38:34.958217+0000","last_became_peered":"2024-09-16T15:38:34.958217+0000","last_unstale":"2024-09-16T15:38:36.328852+0000","last_undegraded":"2024-09-16T15:38:36.328852+0000","last_fullsized":"2024-09-16T15:38:36.328852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:27.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328469+0000","last_change":"2024-09-16T15:38:33.956064+0000","last_active":"2024-09-16T15:38:36.328469+0000","last_peered":"2024-09-16T15:38:36.328469+0000","last_clean":"2024-09-16T15:38:36.328469+0000","last_became_active":"2024-09-16T15:38:33.955717+0000","last_became_peered":"2024-09-16T15:38:33.955717+0000","last_unstale":"2024-09-16T15:38:36.328469+0000","last_undegraded":"2024-09-16T15:38:36.328469+0000","last_fullsized":"2024-09-16T15:38:36.328469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:12.649947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330233+0000","last_change":"2024-09-16T15:38:34.937110+0000","last_active":"2024-09-16T15:38:36.330233+0000","last_peered":"2024-09-16T15:38:36.330233+0000","last_clean":"2024-09-16T15:38:36.330233+0000","last_became_active":"2024-09-16T15:38:34.936439+0000","last_became_peered":"2024-09-16T15:38:34.936439+0000","last_unstale":"2024-09-16T15:38:36.330233+0000","last_undegraded":"2024-09-16T15:38:36.330233+0000","last_fullsized":"2024-09-16T15:38:36.330233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:49.927493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328261+0000","last_change":"2024-09-16T15:38:33.944467+0000","last_active":"2024-09-16T15:38:36.328261+0000","last_peered":"2024-09-16T15:38:36.328261+0000","last_clean":"2024-09-16T15:38:36.328261+0000","last_became_active":"2024-09-16T15:38:33.944211+0000","last_became_peered":"2024-09-16T15:38:33.944211+0000","last_unstale":"2024-09-16T15:38:36.328261+0000","last_undegraded":"2024-09-16T15:38:36.328261+0000","last_fullsized":"2024-09-16T15:38:36.328261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328305+0000","last_change":"2024-09-16T15:38:34.932456+0000","last_active":"2024-09-16T15:38:36.328305+0000","last_peered":"2024-09-16T15:38:36.328305+0000","last_clean":"2024-09-16T15:38:36.328305+0000","last_became_active":"2024-09-16T15:38:34.931374+0000","last_became_peered":"2024-09-16T15:38:34.931374+0000","last_unstale":"2024-09-16T15:38:36.328305+0000","last_undegraded":"2024-09-16T15:38:36.328305+0000","last_fullsized":"2024-09-16T15:38:36.328305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:26.787621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330346+0000","last_change":"2024-09-16T15:38:33.945170+0000","last_active":"2024-09-16T15:38:36.330346+0000","last_peered":"2024-09-16T15:38:36.330346+0000","last_clean":"2024-09-16T15:38:36.330346+0000","last_became_active":"2024-09-16T15:38:33.944681+0000","last_became_peered":"2024-09-16T15:38:33.944681+0000","last_unstale":"2024-09-16T15:38:36.330346+0000","last_undegraded":"2024-09-16T15:38:36.330346+0000","last_fullsized":"2024-09-16T15:38:36.330346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:30.232249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":245760,"data_stored":81953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":672833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":75687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:38:40.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:38:40.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.227+0000 7fe727200640 1 -- 172.21.15.60:0/2126259971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe72006e6f0 msgr2=0x7fe72006eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.227+0000 7fe727200640 1 --2- 172.21.15.60:0/2126259971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe72006e6f0 0x7fe72006eba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe718005fd0 tx=0x7fe71802a820 comp rx=0 tx=0).stop 2024-09-16T15:38:40.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.227+0000 7fe727200640 1 -- 172.21.15.60:0/2126259971 shutdown_connections 2024-09-16T15:38:40.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.227+0000 7fe727200640 1 --2- 172.21.15.60:0/2126259971 >> v2:172.21.15.100:3300/0 conn(0x7fe720073de0 0x7fe720074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.227+0000 7fe727200640 1 --2- 172.21.15.60:0/2126259971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe72006e6f0 0x7fe72006eba0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.227+0000 7fe727200640 1 -- 172.21.15.60:0/2126259971 >> 172.21.15.60:0/2126259971 conn(0x7fe7200b18e0 msgr2=0x7fe7200b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:40.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.228+0000 7fe727200640 1 -- 172.21.15.60:0/2126259971 shutdown_connections 2024-09-16T15:38:40.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.228+0000 7fe727200640 1 -- 172.21.15.60:0/2126259971 wait complete. 2024-09-16T15:38:40.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.229+0000 7fe727200640 1 Processor -- start 2024-09-16T15:38:40.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.229+0000 7fe727200640 1 -- start start 2024-09-16T15:38:40.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.230+0000 7fe727200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe72006e6f0 0x7fe720155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:40.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.230+0000 7fe727200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe720073de0 0x7fe720154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:40.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.230+0000 7fe727200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe72015f9a0 0x7fe72015e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:40.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.230+0000 7fe727200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe7200cd5e0 con 0x7fe720073de0 2024-09-16T15:38:40.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.230+0000 7fe727200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe72006f0e0 con 0x7fe72015f9a0 2024-09-16T15:38:40.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.230+0000 7fe727200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe72006f260 con 0x7fe72006e6f0 2024-09-16T15:38:40.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.230+0000 7fe717400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe720073de0 0x7fe720154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe717400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe720073de0 0x7fe720154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35148/0 (socket says 172.21.15.60:35148) 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe717400640 1 -- 172.21.15.60:0/313958815 learned_addr learned my addr 172.21.15.60:0/313958815 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe71fe00640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe72006e6f0 0x7fe720155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe725200640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe72015f9a0 0x7fe72015e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe717400640 1 -- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe72006e6f0 msgr2=0x7fe720155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe717400640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe72006e6f0 0x7fe720155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe717400640 1 -- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe72015f9a0 msgr2=0x7fe72015e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe717400640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe72015f9a0 0x7fe72015e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe717400640 1 -- 172.21.15.60:0/313958815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7201f5d80 con 0x7fe720073de0 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe725200640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe72015f9a0 0x7fe72015e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe717400640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe720073de0 0x7fe720154470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e9c0 tx=0x7fe70800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:40.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe71cc00640 1 -- 172.21.15.60:0/313958815 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70800cde0 con 0x7fe720073de0 2024-09-16T15:38:40.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe71cc00640 1 -- 172.21.15.60:0/313958815 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe708010520 con 0x7fe720073de0 2024-09-16T15:38:40.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe71cc00640 1 -- 172.21.15.60:0/313958815 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708010970 con 0x7fe720073de0 2024-09-16T15:38:40.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.231+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7200ad990 con 0x7fe720073de0 2024-09-16T15:38:40.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.232+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7200adf50 con 0x7fe720073de0 2024-09-16T15:38:40.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.233+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe720069360 con 0x7fe720073de0 2024-09-16T15:38:40.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.234+0000 7fe71cc00640 1 -- 172.21.15.60:0/313958815 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe70801f020 con 0x7fe720073de0 2024-09-16T15:38:40.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.235+0000 7fe71cc00640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe6fc078850 0x7fe6fc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:40.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.235+0000 7fe71cc00640 1 -- 172.21.15.60:0/313958815 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe70809b2a0 con 0x7fe720073de0 2024-09-16T15:38:40.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.235+0000 7fe71fe00640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe6fc078850 0x7fe6fc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:40.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.236+0000 7fe71fe00640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe6fc078850 0x7fe6fc07ad10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe710001610 tx=0x7fe710009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:40.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.239+0000 7fe71cc00640 1 -- 172.21.15.60:0/313958815 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe708063880 con 0x7fe720073de0 2024-09-16T15:38:40.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.507+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe72006eba0 con 0x7fe720073de0 2024-09-16T15:38:40.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.507+0000 7fe71cc00640 1 -- 172.21.15.60:0/313958815 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1053 (secure 0 0 0) 0x7fe708063180 con 0x7fe720073de0 2024-09-16T15:38:40.509 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:40.509 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501102,"num_in_osds":8,"osd_in_since":1726501092,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":241590272,"bytes_avail":772852523008,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4402}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-16T15:38:08.529603+0000","services":{}},"progress_events":{}} 2024-09-16T15:38:40.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe6fc078850 msgr2=0x7fe6fc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe6fc078850 0x7fe6fc07ad10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe710001610 tx=0x7fe710009040 comp rx=0 tx=0).stop 2024-09-16T15:38:40.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe720073de0 msgr2=0x7fe720154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe720073de0 0x7fe720154470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e9c0 tx=0x7fe70800ee80 comp rx=0 tx=0).stop 2024-09-16T15:38:40.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 shutdown_connections 2024-09-16T15:38:40.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe6fc078850 0x7fe6fc07ad10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe72015f9a0 0x7fe72015e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe720073de0 0x7fe720154470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 --2- 172.21.15.60:0/313958815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe72006e6f0 0x7fe720155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 >> 172.21.15.60:0/313958815 conn(0x7fe7200b18e0 msgr2=0x7fe7200b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 shutdown_connections 2024-09-16T15:38:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.510+0000 7fe727200640 1 -- 172.21.15.60:0/313958815 wait complete. 2024-09-16T15:38:40.527 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:38:40.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:38:40.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.686+0000 7f2983400640 1 -- 172.21.15.60:0/3298821475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f297c15f5a0 msgr2=0x7f297c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.686+0000 7f2983400640 1 --2- 172.21.15.60:0/3298821475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f297c15f5a0 0x7f297c0b2440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f296c005fd0 tx=0x7f296c02a540 comp rx=0 tx=0).stop 2024-09-16T15:38:40.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.686+0000 7f2983400640 1 -- 172.21.15.60:0/3298821475 shutdown_connections 2024-09-16T15:38:40.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.686+0000 7f2983400640 1 --2- 172.21.15.60:0/3298821475 >> v2:172.21.15.100:3300/0 conn(0x7f297c073fb0 0x7f297c074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.686+0000 7f2983400640 1 --2- 172.21.15.60:0/3298821475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f297c15f5a0 0x7f297c0b2440 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.686+0000 7f2983400640 1 -- 172.21.15.60:0/3298821475 >> 172.21.15.60:0/3298821475 conn(0x7f297c0b1950 msgr2=0x7f297c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:40.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.686+0000 7f2983400640 1 -- 172.21.15.60:0/3298821475 shutdown_connections 2024-09-16T15:38:40.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.686+0000 7f2983400640 1 -- 172.21.15.60:0/3298821475 wait complete. 2024-09-16T15:38:40.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.688+0000 7f2983400640 1 Processor -- start 2024-09-16T15:38:40.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.688+0000 7f2983400640 1 -- start start 2024-09-16T15:38:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2983400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f297c073fb0 0x7f297c150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2983400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f297c0ccf00 0x7f297c14bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2983400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f297c15f5a0 0x7f297c1f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2983400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f297c06f2e0 con 0x7f297c0ccf00 2024-09-16T15:38:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2983400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f297c06f160 con 0x7f297c15f5a0 2024-09-16T15:38:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2983400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f297c06f460 con 0x7f297c073fb0 2024-09-16T15:38:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2973e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f297c0ccf00 0x7f297c14bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2973e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f297c0ccf00 0x7f297c14bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35150/0 (socket says 172.21.15.60:35150) 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2973e00640 1 -- 172.21.15.60:0/1938738015 learned_addr learned my addr 172.21.15.60:0/1938738015 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2981400640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f297c15f5a0 0x7f297c1f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.689+0000 7f2980a00640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f297c073fb0 0x7f297c150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2973e00640 1 -- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f297c073fb0 msgr2=0x7f297c150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2973e00640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f297c073fb0 0x7f297c150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2973e00640 1 -- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f297c15f5a0 msgr2=0x7f297c1f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2973e00640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f297c15f5a0 0x7f297c1f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2973e00640 1 -- 172.21.15.60:0/1938738015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c14c240 con 0x7f297c0ccf00 2024-09-16T15:38:40.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2973e00640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f297c0ccf00 0x7f297c14bd00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f296400c420 tx=0x7f296400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2979600640 1 -- 172.21.15.60:0/1938738015 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964017070 con 0x7f297c0ccf00 2024-09-16T15:38:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2979600640 1 -- 172.21.15.60:0/1938738015 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29640053c0 con 0x7f297c0ccf00 2024-09-16T15:38:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c1f9010 con 0x7f297c0ccf00 2024-09-16T15:38:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2979600640 1 -- 172.21.15.60:0/1938738015 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964003850 con 0x7f297c0ccf00 2024-09-16T15:38:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.690+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c0d08c0 con 0x7f297c0ccf00 2024-09-16T15:38:40.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.692+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c06f9f0 con 0x7f297c0ccf00 2024-09-16T15:38:40.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.693+0000 7f2979600640 1 -- 172.21.15.60:0/1938738015 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f296401a070 con 0x7f297c0ccf00 2024-09-16T15:38:40.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.694+0000 7f2979600640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2958078850 0x7f295807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:40.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.694+0000 7f2979600640 1 -- 172.21.15.60:0/1938738015 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2964099bd0 con 0x7f297c0ccf00 2024-09-16T15:38:40.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.694+0000 7f2980a00640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2958078850 0x7f295807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:40.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.695+0000 7f2980a00640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2958078850 0x7f295807ad10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f296c033000 tx=0x7f296c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:40.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.698+0000 7f2979600640 1 -- 172.21.15.60:0/1938738015 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2964061c00 con 0x7f297c0ccf00 2024-09-16T15:38:40.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.891+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f297c0afc30 con 0x7f2958078850 2024-09-16T15:38:40.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.896+0000 7f2979600640 1 -- 172.21.15.60:0/1938738015 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7f297c0afc30 con 0x7f2958078850 2024-09-16T15:38:40.901 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:40.902 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:38:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2958078850 msgr2=0x7f295807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2958078850 0x7f295807ad10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f296c033000 tx=0x7f296c004730 comp rx=0 tx=0).stop 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f297c0ccf00 msgr2=0x7f297c14bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f297c0ccf00 0x7f297c14bd00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f296400c420 tx=0x7f296400c8e0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 shutdown_connections 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2958078850 0x7f295807ad10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f297c15f5a0 0x7f297c1f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f297c0ccf00 0x7f297c14bd00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 --2- 172.21.15.60:0/1938738015 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f297c073fb0 0x7f297c150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 >> 172.21.15.60:0/1938738015 conn(0x7f297c0b1950 msgr2=0x7f297c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 shutdown_connections 2024-09-16T15:38:40.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:40.900+0000 7f2983400640 1 -- 172.21.15.60:0/1938738015 wait complete. 2024-09-16T15:38:40.917 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-16T15:38:39.709152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235928,"kb_used_data":3036,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738792,"statfs":{"total":773094113280,"available":772852523008,"internally_reserved":0,"allocated":3108864,"data_stored":1833872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001467"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328961+0000","last_change":"2024-09-16T15:38:34.948046+0000","last_active":"2024-09-16T15:38:36.328961+0000","last_peered":"2024-09-16T15:38:36.328961+0000","last_clean":"2024-09-16T15:38:36.328961+0000","last_became_active":"2024-09-16T15:38:34.947778+0000","last_became_peered":"2024-09-16T15:38:34.947778+0000","last_unstale":"2024-09-16T15:38:36.328961+0000","last_undegraded":"2024-09-16T15:38:36.328961+0000","last_fullsized":"2024-09-16T15:38:36.328961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329296+0000","last_change":"2024-09-16T15:38:34.934287+0000","last_active":"2024-09-16T15:38:36.329296+0000","last_peered":"2024-09-16T15:38:36.329296+0000","last_clean":"2024-09-16T15:38:36.329296+0000","last_became_active":"2024-09-16T15:38:34.934004+0000","last_became_peered":"2024-09-16T15:38:34.934004+0000","last_unstale":"2024-09-16T15:38:36.329296+0000","last_undegraded":"2024-09-16T15:38:36.329296+0000","last_fullsized":"2024-09-16T15:38:36.329296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:43.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330192+0000","last_change":"2024-09-16T15:38:34.937280+0000","last_active":"2024-09-16T15:38:36.330192+0000","last_peered":"2024-09-16T15:38:36.330192+0000","last_clean":"2024-09-16T15:38:36.330192+0000","last_became_active":"2024-09-16T15:38:34.936979+0000","last_became_peered":"2024-09-16T15:38:34.936979+0000","last_unstale":"2024-09-16T15:38:36.330192+0000","last_undegraded":"2024-09-16T15:38:36.330192+0000","last_fullsized":"2024-09-16T15:38:36.330192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:16.363921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328077+0000","last_change":"2024-09-16T15:38:34.956532+0000","last_active":"2024-09-16T15:38:36.328077+0000","last_peered":"2024-09-16T15:38:36.328077+0000","last_clean":"2024-09-16T15:38:36.328077+0000","last_became_active":"2024-09-16T15:38:34.956261+0000","last_became_peered":"2024-09-16T15:38:34.956261+0000","last_unstale":"2024-09-16T15:38:36.328077+0000","last_undegraded":"2024-09-16T15:38:36.328077+0000","last_fullsized":"2024-09-16T15:38:36.328077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328317+0000","last_change":"2024-09-16T15:38:34.946850+0000","last_active":"2024-09-16T15:38:36.328317+0000","last_peered":"2024-09-16T15:38:36.328317+0000","last_clean":"2024-09-16T15:38:36.328317+0000","last_became_active":"2024-09-16T15:38:34.946584+0000","last_became_peered":"2024-09-16T15:38:34.946584+0000","last_unstale":"2024-09-16T15:38:36.328317+0000","last_undegraded":"2024-09-16T15:38:36.328317+0000","last_fullsized":"2024-09-16T15:38:36.328317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328172+0000","last_change":"2024-09-16T15:38:34.946360+0000","last_active":"2024-09-16T15:38:36.328172+0000","last_peered":"2024-09-16T15:38:36.328172+0000","last_clean":"2024-09-16T15:38:36.328172+0000","last_became_active":"2024-09-16T15:38:34.946022+0000","last_became_peered":"2024-09-16T15:38:34.946022+0000","last_unstale":"2024-09-16T15:38:36.328172+0000","last_undegraded":"2024-09-16T15:38:36.328172+0000","last_fullsized":"2024-09-16T15:38:36.328172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328807+0000","last_change":"2024-09-16T15:38:34.948730+0000","last_active":"2024-09-16T15:38:36.328807+0000","last_peered":"2024-09-16T15:38:36.328807+0000","last_clean":"2024-09-16T15:38:36.328807+0000","last_became_active":"2024-09-16T15:38:34.948488+0000","last_became_peered":"2024-09-16T15:38:34.948488+0000","last_unstale":"2024-09-16T15:38:36.328807+0000","last_undegraded":"2024-09-16T15:38:36.328807+0000","last_fullsized":"2024-09-16T15:38:36.328807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327904+0000","last_change":"2024-09-16T15:38:34.939960+0000","last_active":"2024-09-16T15:38:36.327904+0000","last_peered":"2024-09-16T15:38:36.327904+0000","last_clean":"2024-09-16T15:38:36.327904+0000","last_became_active":"2024-09-16T15:38:34.939730+0000","last_became_peered":"2024-09-16T15:38:34.939730+0000","last_unstale":"2024-09-16T15:38:36.327904+0000","last_undegraded":"2024-09-16T15:38:36.327904+0000","last_fullsized":"2024-09-16T15:38:36.327904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:35.445142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329103+0000","last_change":"2024-09-16T15:38:34.963612+0000","last_active":"2024-09-16T15:38:36.329103+0000","last_peered":"2024-09-16T15:38:36.329103+0000","last_clean":"2024-09-16T15:38:36.329103+0000","last_became_active":"2024-09-16T15:38:34.963287+0000","last_became_peered":"2024-09-16T15:38:34.963287+0000","last_unstale":"2024-09-16T15:38:36.329103+0000","last_undegraded":"2024-09-16T15:38:36.329103+0000","last_fullsized":"2024-09-16T15:38:36.329103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:53.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935830+0000","last_change":"2024-09-16T15:38:34.959111+0000","last_active":"2024-09-16T15:38:35.935830+0000","last_peered":"2024-09-16T15:38:35.935830+0000","last_clean":"2024-09-16T15:38:35.935830+0000","last_became_active":"2024-09-16T15:38:34.958733+0000","last_became_peered":"2024-09-16T15:38:34.958733+0000","last_unstale":"2024-09-16T15:38:35.935830+0000","last_undegraded":"2024-09-16T15:38:35.935830+0000","last_fullsized":"2024-09-16T15:38:35.935830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:21.940460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934331+0000","last_change":"2024-09-16T15:38:34.963928+0000","last_active":"2024-09-16T15:38:35.934331+0000","last_peered":"2024-09-16T15:38:35.934331+0000","last_clean":"2024-09-16T15:38:35.934331+0000","last_became_active":"2024-09-16T15:38:34.963418+0000","last_became_peered":"2024-09-16T15:38:34.963418+0000","last_unstale":"2024-09-16T15:38:35.934331+0000","last_undegraded":"2024-09-16T15:38:35.934331+0000","last_fullsized":"2024-09-16T15:38:35.934331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:54.016578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935041+0000","last_change":"2024-09-16T15:38:34.951463+0000","last_active":"2024-09-16T15:38:35.935041+0000","last_peered":"2024-09-16T15:38:35.935041+0000","last_clean":"2024-09-16T15:38:35.935041+0000","last_became_active":"2024-09-16T15:38:34.949972+0000","last_became_peered":"2024-09-16T15:38:34.949972+0000","last_unstale":"2024-09-16T15:38:35.935041+0000","last_undegraded":"2024-09-16T15:38:35.935041+0000","last_fullsized":"2024-09-16T15:38:35.935041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:36.356482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328377+0000","last_change":"2024-09-16T15:38:34.936161+0000","last_active":"2024-09-16T15:38:36.328377+0000","last_peered":"2024-09-16T15:38:36.328377+0000","last_clean":"2024-09-16T15:38:36.328377+0000","last_became_active":"2024-09-16T15:38:34.935881+0000","last_became_peered":"2024-09-16T15:38:34.935881+0000","last_unstale":"2024-09-16T15:38:36.328377+0000","last_undegraded":"2024-09-16T15:38:36.328377+0000","last_fullsized":"2024-09-16T15:38:36.328377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329953+0000","last_change":"2024-09-16T15:38:34.934084+0000","last_active":"2024-09-16T15:38:36.329953+0000","last_peered":"2024-09-16T15:38:36.329953+0000","last_clean":"2024-09-16T15:38:36.329953+0000","last_became_active":"2024-09-16T15:38:34.933716+0000","last_became_peered":"2024-09-16T15:38:34.933716+0000","last_unstale":"2024-09-16T15:38:36.329953+0000","last_undegraded":"2024-09-16T15:38:36.329953+0000","last_fullsized":"2024-09-16T15:38:36.329953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:51.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328795+0000","last_change":"2024-09-16T15:38:34.951427+0000","last_active":"2024-09-16T15:38:36.328795+0000","last_peered":"2024-09-16T15:38:36.328795+0000","last_clean":"2024-09-16T15:38:36.328795+0000","last_became_active":"2024-09-16T15:38:34.949484+0000","last_became_peered":"2024-09-16T15:38:34.949484+0000","last_unstale":"2024-09-16T15:38:36.328795+0000","last_undegraded":"2024-09-16T15:38:36.328795+0000","last_fullsized":"2024-09-16T15:38:36.328795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:54.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327906+0000","last_change":"2024-09-16T15:38:34.948833+0000","last_active":"2024-09-16T15:38:36.327906+0000","last_peered":"2024-09-16T15:38:36.327906+0000","last_clean":"2024-09-16T15:38:36.327906+0000","last_became_active":"2024-09-16T15:38:34.948127+0000","last_became_peered":"2024-09-16T15:38:34.948127+0000","last_unstale":"2024-09-16T15:38:36.327906+0000","last_undegraded":"2024-09-16T15:38:36.327906+0000","last_fullsized":"2024-09-16T15:38:36.327906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329716+0000","last_change":"2024-09-16T15:38:34.948165+0000","last_active":"2024-09-16T15:38:36.329716+0000","last_peered":"2024-09-16T15:38:36.329716+0000","last_clean":"2024-09-16T15:38:36.329716+0000","last_became_active":"2024-09-16T15:38:34.947862+0000","last_became_peered":"2024-09-16T15:38:34.947862+0000","last_unstale":"2024-09-16T15:38:36.329716+0000","last_undegraded":"2024-09-16T15:38:36.329716+0000","last_fullsized":"2024-09-16T15:38:36.329716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:07.065693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932521+0000","last_change":"2024-09-16T15:38:34.936088+0000","last_active":"2024-09-16T15:38:35.932521+0000","last_peered":"2024-09-16T15:38:35.932521+0000","last_clean":"2024-09-16T15:38:35.932521+0000","last_became_active":"2024-09-16T15:38:34.935649+0000","last_became_peered":"2024-09-16T15:38:34.935649+0000","last_unstale":"2024-09-16T15:38:35.932521+0000","last_undegraded":"2024-09-16T15:38:35.932521+0000","last_fullsized":"2024-09-16T15:38:35.932521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:56.948937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328715+0000","last_change":"2024-09-16T15:38:34.951511+0000","last_active":"2024-09-16T15:38:36.328715+0000","last_peered":"2024-09-16T15:38:36.328715+0000","last_clean":"2024-09-16T15:38:36.328715+0000","last_became_active":"2024-09-16T15:38:34.951241+0000","last_became_peered":"2024-09-16T15:38:34.951241+0000","last_unstale":"2024-09-16T15:38:36.328715+0000","last_undegraded":"2024-09-16T15:38:36.328715+0000","last_fullsized":"2024-09-16T15:38:36.328715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:52.005508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934261+0000","last_change":"2024-09-16T15:38:34.950265+0000","last_active":"2024-09-16T15:38:35.934261+0000","last_peered":"2024-09-16T15:38:35.934261+0000","last_clean":"2024-09-16T15:38:35.934261+0000","last_became_active":"2024-09-16T15:38:34.949457+0000","last_became_peered":"2024-09-16T15:38:34.949457+0000","last_unstale":"2024-09-16T15:38:35.934261+0000","last_undegraded":"2024-09-16T15:38:35.934261+0000","last_fullsized":"2024-09-16T15:38:35.934261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:33.694559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330049+0000","last_change":"2024-09-16T15:38:34.942753+0000","last_active":"2024-09-16T15:38:36.330049+0000","last_peered":"2024-09-16T15:38:36.330049+0000","last_clean":"2024-09-16T15:38:36.330049+0000","last_became_active":"2024-09-16T15:38:34.940841+0000","last_became_peered":"2024-09-16T15:38:34.940841+0000","last_unstale":"2024-09-16T15:38:36.330049+0000","last_undegraded":"2024-09-16T15:38:36.330049+0000","last_fullsized":"2024-09-16T15:38:36.330049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:22.740035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328374+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328374+0000","last_peered":"2024-09-16T15:38:36.328374+0000","last_clean":"2024-09-16T15:38:36.328374+0000","last_became_active":"2024-09-16T15:38:34.958240+0000","last_became_peered":"2024-09-16T15:38:34.958240+0000","last_unstale":"2024-09-16T15:38:36.328374+0000","last_undegraded":"2024-09-16T15:38:36.328374+0000","last_fullsized":"2024-09-16T15:38:36.328374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328083+0000","last_change":"2024-09-16T15:38:34.952703+0000","last_active":"2024-09-16T15:38:36.328083+0000","last_peered":"2024-09-16T15:38:36.328083+0000","last_clean":"2024-09-16T15:38:36.328083+0000","last_became_active":"2024-09-16T15:38:34.949406+0000","last_became_peered":"2024-09-16T15:38:34.949406+0000","last_unstale":"2024-09-16T15:38:36.328083+0000","last_undegraded":"2024-09-16T15:38:36.328083+0000","last_fullsized":"2024-09-16T15:38:36.328083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:20.072155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328173+0000","last_change":"2024-09-16T15:38:34.948442+0000","last_active":"2024-09-16T15:38:36.328173+0000","last_peered":"2024-09-16T15:38:36.328173+0000","last_clean":"2024-09-16T15:38:36.328173+0000","last_became_active":"2024-09-16T15:38:34.947564+0000","last_became_peered":"2024-09-16T15:38:34.947564+0000","last_unstale":"2024-09-16T15:38:36.328173+0000","last_undegraded":"2024-09-16T15:38:36.328173+0000","last_fullsized":"2024-09-16T15:38:36.328173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328339+0000","last_change":"2024-09-16T15:38:34.940042+0000","last_active":"2024-09-16T15:38:36.328339+0000","last_peered":"2024-09-16T15:38:36.328339+0000","last_clean":"2024-09-16T15:38:36.328339+0000","last_became_active":"2024-09-16T15:38:34.939792+0000","last_became_peered":"2024-09-16T15:38:34.939792+0000","last_unstale":"2024-09-16T15:38:36.328339+0000","last_undegraded":"2024-09-16T15:38:36.328339+0000","last_fullsized":"2024-09-16T15:38:36.328339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.386845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935854+0000","last_change":"2024-09-16T15:38:34.938919+0000","last_active":"2024-09-16T15:38:35.935854+0000","last_peered":"2024-09-16T15:38:35.935854+0000","last_clean":"2024-09-16T15:38:35.935854+0000","last_became_active":"2024-09-16T15:38:34.938608+0000","last_became_peered":"2024-09-16T15:38:34.938608+0000","last_unstale":"2024-09-16T15:38:35.935854+0000","last_undegraded":"2024-09-16T15:38:35.935854+0000","last_fullsized":"2024-09-16T15:38:35.935854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:44.137157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329086+0000","last_change":"2024-09-16T15:38:34.934519+0000","last_active":"2024-09-16T15:38:36.329086+0000","last_peered":"2024-09-16T15:38:36.329086+0000","last_clean":"2024-09-16T15:38:36.329086+0000","last_became_active":"2024-09-16T15:38:34.932223+0000","last_became_peered":"2024-09-16T15:38:34.932223+0000","last_unstale":"2024-09-16T15:38:36.329086+0000","last_undegraded":"2024-09-16T15:38:36.329086+0000","last_fullsized":"2024-09-16T15:38:36.329086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.673101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329499+0000","last_change":"2024-09-16T15:38:34.941297+0000","last_active":"2024-09-16T15:38:36.329499+0000","last_peered":"2024-09-16T15:38:36.329499+0000","last_clean":"2024-09-16T15:38:36.329499+0000","last_became_active":"2024-09-16T15:38:34.941007+0000","last_became_peered":"2024-09-16T15:38:34.941007+0000","last_unstale":"2024-09-16T15:38:36.329499+0000","last_undegraded":"2024-09-16T15:38:36.329499+0000","last_fullsized":"2024-09-16T15:38:36.329499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330077+0000","last_change":"2024-09-16T15:38:34.941110+0000","last_active":"2024-09-16T15:38:36.330077+0000","last_peered":"2024-09-16T15:38:36.330077+0000","last_clean":"2024-09-16T15:38:36.330077+0000","last_became_active":"2024-09-16T15:38:34.940650+0000","last_became_peered":"2024-09-16T15:38:34.940650+0000","last_unstale":"2024-09-16T15:38:36.330077+0000","last_undegraded":"2024-09-16T15:38:36.330077+0000","last_fullsized":"2024-09-16T15:38:36.330077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:26.093004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328522+0000","last_change":"2024-09-16T15:38:34.952930+0000","last_active":"2024-09-16T15:38:36.328522+0000","last_peered":"2024-09-16T15:38:36.328522+0000","last_clean":"2024-09-16T15:38:36.328522+0000","last_became_active":"2024-09-16T15:38:34.952652+0000","last_became_peered":"2024-09-16T15:38:34.952652+0000","last_unstale":"2024-09-16T15:38:36.328522+0000","last_undegraded":"2024-09-16T15:38:36.328522+0000","last_fullsized":"2024-09-16T15:38:36.328522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932490+0000","last_change":"2024-09-16T15:38:34.961835+0000","last_active":"2024-09-16T15:38:35.932490+0000","last_peered":"2024-09-16T15:38:35.932490+0000","last_clean":"2024-09-16T15:38:35.932490+0000","last_became_active":"2024-09-16T15:38:34.961529+0000","last_became_peered":"2024-09-16T15:38:34.961529+0000","last_unstale":"2024-09-16T15:38:35.932490+0000","last_undegraded":"2024-09-16T15:38:35.932490+0000","last_fullsized":"2024-09-16T15:38:35.932490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:11.053973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933841+0000","last_change":"2024-09-16T15:38:34.942442+0000","last_active":"2024-09-16T15:38:35.933841+0000","last_peered":"2024-09-16T15:38:35.933841+0000","last_clean":"2024-09-16T15:38:35.933841+0000","last_became_active":"2024-09-16T15:38:34.942144+0000","last_became_peered":"2024-09-16T15:38:34.942144+0000","last_unstale":"2024-09-16T15:38:35.933841+0000","last_undegraded":"2024-09-16T15:38:35.933841+0000","last_fullsized":"2024-09-16T15:38:35.933841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:43.399475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934635+0000","last_change":"2024-09-16T15:38:34.941230+0000","last_active":"2024-09-16T15:38:35.934635+0000","last_peered":"2024-09-16T15:38:35.934635+0000","last_clean":"2024-09-16T15:38:35.934635+0000","last_became_active":"2024-09-16T15:38:34.940613+0000","last_became_peered":"2024-09-16T15:38:34.940613+0000","last_unstale":"2024-09-16T15:38:35.934635+0000","last_undegraded":"2024-09-16T15:38:35.934635+0000","last_fullsized":"2024-09-16T15:38:35.934635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:15.007186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328673+0000","last_change":"2024-09-16T15:38:33.949962+0000","last_active":"2024-09-16T15:38:36.328673+0000","last_peered":"2024-09-16T15:38:36.328673+0000","last_clean":"2024-09-16T15:38:36.328673+0000","last_became_active":"2024-09-16T15:38:33.949620+0000","last_became_peered":"2024-09-16T15:38:33.949620+0000","last_unstale":"2024-09-16T15:38:36.328673+0000","last_undegraded":"2024-09-16T15:38:36.328673+0000","last_fullsized":"2024-09-16T15:38:36.328673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330281+0000","last_change":"2024-09-16T15:38:34.947478+0000","last_active":"2024-09-16T15:38:36.330281+0000","last_peered":"2024-09-16T15:38:36.330281+0000","last_clean":"2024-09-16T15:38:36.330281+0000","last_became_active":"2024-09-16T15:38:34.947158+0000","last_became_peered":"2024-09-16T15:38:34.947158+0000","last_unstale":"2024-09-16T15:38:36.330281+0000","last_undegraded":"2024-09-16T15:38:36.330281+0000","last_fullsized":"2024-09-16T15:38:36.330281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:24.497838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933892+0000","last_change":"2024-09-16T15:38:33.931830+0000","last_active":"2024-09-16T15:38:35.933892+0000","last_peered":"2024-09-16T15:38:35.933892+0000","last_clean":"2024-09-16T15:38:35.933892+0000","last_became_active":"2024-09-16T15:38:33.931129+0000","last_became_peered":"2024-09-16T15:38:33.931129+0000","last_unstale":"2024-09-16T15:38:35.933892+0000","last_undegraded":"2024-09-16T15:38:35.933892+0000","last_fullsized":"2024-09-16T15:38:35.933892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:54.747927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327884+0000","last_change":"2024-09-16T15:38:33.943597+0000","last_active":"2024-09-16T15:38:36.327884+0000","last_peered":"2024-09-16T15:38:36.327884+0000","last_clean":"2024-09-16T15:38:36.327884+0000","last_became_active":"2024-09-16T15:38:33.940904+0000","last_became_peered":"2024-09-16T15:38:33.940904+0000","last_unstale":"2024-09-16T15:38:36.327884+0000","last_undegraded":"2024-09-16T15:38:36.327884+0000","last_fullsized":"2024-09-16T15:38:36.327884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:13.818889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328165+0000","last_change":"2024-09-16T15:38:34.942424+0000","last_active":"2024-09-16T15:38:36.328165+0000","last_peered":"2024-09-16T15:38:36.328165+0000","last_clean":"2024-09-16T15:38:36.328165+0000","last_became_active":"2024-09-16T15:38:34.942111+0000","last_became_peered":"2024-09-16T15:38:34.942111+0000","last_unstale":"2024-09-16T15:38:36.328165+0000","last_undegraded":"2024-09-16T15:38:36.328165+0000","last_fullsized":"2024-09-16T15:38:36.328165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:52.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328244+0000","last_change":"2024-09-16T15:38:33.945014+0000","last_active":"2024-09-16T15:38:36.328244+0000","last_peered":"2024-09-16T15:38:36.328244+0000","last_clean":"2024-09-16T15:38:36.328244+0000","last_became_active":"2024-09-16T15:38:33.944742+0000","last_became_peered":"2024-09-16T15:38:33.944742+0000","last_unstale":"2024-09-16T15:38:36.328244+0000","last_undegraded":"2024-09-16T15:38:36.328244+0000","last_fullsized":"2024-09-16T15:38:36.328244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328641+0000","last_change":"2024-09-16T15:38:34.933709+0000","last_active":"2024-09-16T15:38:36.328641+0000","last_peered":"2024-09-16T15:38:36.328641+0000","last_clean":"2024-09-16T15:38:36.328641+0000","last_became_active":"2024-09-16T15:38:34.932126+0000","last_became_peered":"2024-09-16T15:38:34.932126+0000","last_unstale":"2024-09-16T15:38:36.328641+0000","last_undegraded":"2024-09-16T15:38:36.328641+0000","last_fullsized":"2024-09-16T15:38:36.328641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:44.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935745+0000","last_change":"2024-09-16T15:38:33.945379+0000","last_active":"2024-09-16T15:38:35.935745+0000","last_peered":"2024-09-16T15:38:35.935745+0000","last_clean":"2024-09-16T15:38:35.935745+0000","last_became_active":"2024-09-16T15:38:33.944531+0000","last_became_peered":"2024-09-16T15:38:33.944531+0000","last_unstale":"2024-09-16T15:38:35.935745+0000","last_undegraded":"2024-09-16T15:38:35.935745+0000","last_fullsized":"2024-09-16T15:38:35.935745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:09.939574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329480+0000","last_change":"2024-09-16T15:38:34.961412+0000","last_active":"2024-09-16T15:38:36.329480+0000","last_peered":"2024-09-16T15:38:36.329480+0000","last_clean":"2024-09-16T15:38:36.329480+0000","last_became_active":"2024-09-16T15:38:34.961004+0000","last_became_peered":"2024-09-16T15:38:34.961004+0000","last_unstale":"2024-09-16T15:38:36.329480+0000","last_undegraded":"2024-09-16T15:38:36.329480+0000","last_fullsized":"2024-09-16T15:38:36.329480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:39.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329349+0000","last_change":"2024-09-16T15:38:33.942028+0000","last_active":"2024-09-16T15:38:36.329349+0000","last_peered":"2024-09-16T15:38:36.329349+0000","last_clean":"2024-09-16T15:38:36.329349+0000","last_became_active":"2024-09-16T15:38:33.940695+0000","last_became_peered":"2024-09-16T15:38:33.940695+0000","last_unstale":"2024-09-16T15:38:36.329349+0000","last_undegraded":"2024-09-16T15:38:36.329349+0000","last_fullsized":"2024-09-16T15:38:36.329349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330396+0000","last_change":"2024-09-16T15:38:34.937478+0000","last_active":"2024-09-16T15:38:36.330396+0000","last_peered":"2024-09-16T15:38:36.330396+0000","last_clean":"2024-09-16T15:38:36.330396+0000","last_became_active":"2024-09-16T15:38:34.936844+0000","last_became_peered":"2024-09-16T15:38:34.936844+0000","last_unstale":"2024-09-16T15:38:36.330396+0000","last_undegraded":"2024-09-16T15:38:36.330396+0000","last_fullsized":"2024-09-16T15:38:36.330396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:42.401407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328973+0000","last_change":"2024-09-16T15:38:33.942385+0000","last_active":"2024-09-16T15:38:36.328973+0000","last_peered":"2024-09-16T15:38:36.328973+0000","last_clean":"2024-09-16T15:38:36.328973+0000","last_became_active":"2024-09-16T15:38:33.941960+0000","last_became_peered":"2024-09-16T15:38:33.941960+0000","last_unstale":"2024-09-16T15:38:36.328973+0000","last_undegraded":"2024-09-16T15:38:36.328973+0000","last_fullsized":"2024-09-16T15:38:36.328973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:08.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935156+0000","last_change":"2024-09-16T15:38:34.955492+0000","last_active":"2024-09-16T15:38:35.935156+0000","last_peered":"2024-09-16T15:38:35.935156+0000","last_clean":"2024-09-16T15:38:35.935156+0000","last_became_active":"2024-09-16T15:38:34.955230+0000","last_became_peered":"2024-09-16T15:38:34.955230+0000","last_unstale":"2024-09-16T15:38:35.935156+0000","last_undegraded":"2024-09-16T15:38:35.935156+0000","last_fullsized":"2024-09-16T15:38:35.935156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:13.656183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934327+0000","last_change":"2024-09-16T15:38:33.939711+0000","last_active":"2024-09-16T15:38:35.934327+0000","last_peered":"2024-09-16T15:38:35.934327+0000","last_clean":"2024-09-16T15:38:35.934327+0000","last_became_active":"2024-09-16T15:38:33.937858+0000","last_became_peered":"2024-09-16T15:38:33.937858+0000","last_unstale":"2024-09-16T15:38:35.934327+0000","last_undegraded":"2024-09-16T15:38:35.934327+0000","last_fullsized":"2024-09-16T15:38:35.934327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:42.581730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328596+0000","last_change":"2024-09-16T15:38:34.947613+0000","last_active":"2024-09-16T15:38:36.328596+0000","last_peered":"2024-09-16T15:38:36.328596+0000","last_clean":"2024-09-16T15:38:36.328596+0000","last_became_active":"2024-09-16T15:38:34.947258+0000","last_became_peered":"2024-09-16T15:38:34.947258+0000","last_unstale":"2024-09-16T15:38:36.328596+0000","last_undegraded":"2024-09-16T15:38:36.328596+0000","last_fullsized":"2024-09-16T15:38:36.328596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.135119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330167+0000","last_change":"2024-09-16T15:38:33.936316+0000","last_active":"2024-09-16T15:38:36.330167+0000","last_peered":"2024-09-16T15:38:36.330167+0000","last_clean":"2024-09-16T15:38:36.330167+0000","last_became_active":"2024-09-16T15:38:33.936002+0000","last_became_peered":"2024-09-16T15:38:33.936002+0000","last_unstale":"2024-09-16T15:38:36.330167+0000","last_undegraded":"2024-09-16T15:38:36.330167+0000","last_fullsized":"2024-09-16T15:38:36.330167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.796157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329907+0000","last_change":"2024-09-16T15:38:34.942140+0000","last_active":"2024-09-16T15:38:36.329907+0000","last_peered":"2024-09-16T15:38:36.329907+0000","last_clean":"2024-09-16T15:38:36.329907+0000","last_became_active":"2024-09-16T15:38:34.940716+0000","last_became_peered":"2024-09-16T15:38:34.940716+0000","last_unstale":"2024-09-16T15:38:36.329907+0000","last_undegraded":"2024-09-16T15:38:36.329907+0000","last_fullsized":"2024-09-16T15:38:36.329907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:55.566537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328440+0000","last_change":"2024-09-16T15:38:33.938693+0000","last_active":"2024-09-16T15:38:36.328440+0000","last_peered":"2024-09-16T15:38:36.328440+0000","last_clean":"2024-09-16T15:38:36.328440+0000","last_became_active":"2024-09-16T15:38:33.938207+0000","last_became_peered":"2024-09-16T15:38:33.938207+0000","last_unstale":"2024-09-16T15:38:36.328440+0000","last_undegraded":"2024-09-16T15:38:36.328440+0000","last_fullsized":"2024-09-16T15:38:36.328440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328857+0000","last_change":"2024-09-16T15:38:34.956559+0000","last_active":"2024-09-16T15:38:36.328857+0000","last_peered":"2024-09-16T15:38:36.328857+0000","last_clean":"2024-09-16T15:38:36.328857+0000","last_became_active":"2024-09-16T15:38:34.956290+0000","last_became_peered":"2024-09-16T15:38:34.956290+0000","last_unstale":"2024-09-16T15:38:36.328857+0000","last_undegraded":"2024-09-16T15:38:36.328857+0000","last_fullsized":"2024-09-16T15:38:36.328857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:34.525651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933781+0000","last_change":"2024-09-16T15:38:33.937022+0000","last_active":"2024-09-16T15:38:35.933781+0000","last_peered":"2024-09-16T15:38:35.933781+0000","last_clean":"2024-09-16T15:38:35.933781+0000","last_became_active":"2024-09-16T15:38:33.936692+0000","last_became_peered":"2024-09-16T15:38:33.936692+0000","last_unstale":"2024-09-16T15:38:35.933781+0000","last_undegraded":"2024-09-16T15:38:35.933781+0000","last_fullsized":"2024-09-16T15:38:35.933781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:23.195213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933838+0000","last_change":"2024-09-16T15:38:34.933429+0000","last_active":"2024-09-16T15:38:35.933838+0000","last_peered":"2024-09-16T15:38:35.933838+0000","last_clean":"2024-09-16T15:38:35.933838+0000","last_became_active":"2024-09-16T15:38:34.933116+0000","last_became_peered":"2024-09-16T15:38:34.933116+0000","last_unstale":"2024-09-16T15:38:35.933838+0000","last_undegraded":"2024-09-16T15:38:35.933838+0000","last_fullsized":"2024-09-16T15:38:35.933838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:16.415478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932158+0000","last_change":"2024-09-16T15:38:33.929097+0000","last_active":"2024-09-16T15:38:35.932158+0000","last_peered":"2024-09-16T15:38:35.932158+0000","last_clean":"2024-09-16T15:38:35.932158+0000","last_became_active":"2024-09-16T15:38:33.928783+0000","last_became_peered":"2024-09-16T15:38:33.928783+0000","last_unstale":"2024-09-16T15:38:35.932158+0000","last_undegraded":"2024-09-16T15:38:35.932158+0000","last_fullsized":"2024-09-16T15:38:35.932158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327843+0000","last_change":"2024-09-16T15:38:34.955962+0000","last_active":"2024-09-16T15:38:36.327843+0000","last_peered":"2024-09-16T15:38:36.327843+0000","last_clean":"2024-09-16T15:38:36.327843+0000","last_became_active":"2024-09-16T15:38:34.955617+0000","last_became_peered":"2024-09-16T15:38:34.955617+0000","last_unstale":"2024-09-16T15:38:36.327843+0000","last_undegraded":"2024-09-16T15:38:36.327843+0000","last_fullsized":"2024-09-16T15:38:36.327843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:35.342731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935301+0000","last_change":"2024-09-16T15:38:33.933311+0000","last_active":"2024-09-16T15:38:35.935301+0000","last_peered":"2024-09-16T15:38:35.935301+0000","last_clean":"2024-09-16T15:38:35.935301+0000","last_became_active":"2024-09-16T15:38:33.932882+0000","last_became_peered":"2024-09-16T15:38:33.932882+0000","last_unstale":"2024-09-16T15:38:35.935301+0000","last_undegraded":"2024-09-16T15:38:35.935301+0000","last_fullsized":"2024-09-16T15:38:35.935301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:43.856688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328955+0000","last_change":"2024-09-16T15:38:34.959213+0000","last_active":"2024-09-16T15:38:36.328955+0000","last_peered":"2024-09-16T15:38:36.328955+0000","last_clean":"2024-09-16T15:38:36.328955+0000","last_became_active":"2024-09-16T15:38:34.958928+0000","last_became_peered":"2024-09-16T15:38:34.958928+0000","last_unstale":"2024-09-16T15:38:36.328955+0000","last_undegraded":"2024-09-16T15:38:36.328955+0000","last_fullsized":"2024-09-16T15:38:36.328955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:51.093521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932531+0000","last_change":"2024-09-16T15:38:33.939655+0000","last_active":"2024-09-16T15:38:35.932531+0000","last_peered":"2024-09-16T15:38:35.932531+0000","last_clean":"2024-09-16T15:38:35.932531+0000","last_became_active":"2024-09-16T15:38:33.939329+0000","last_became_peered":"2024-09-16T15:38:33.939329+0000","last_unstale":"2024-09-16T15:38:35.932531+0000","last_undegraded":"2024-09-16T15:38:35.932531+0000","last_fullsized":"2024-09-16T15:38:35.932531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329218+0000","last_change":"2024-09-16T15:38:34.956364+0000","last_active":"2024-09-16T15:38:36.329218+0000","last_peered":"2024-09-16T15:38:36.329218+0000","last_clean":"2024-09-16T15:38:36.329218+0000","last_became_active":"2024-09-16T15:38:34.956041+0000","last_became_peered":"2024-09-16T15:38:34.956041+0000","last_unstale":"2024-09-16T15:38:36.329218+0000","last_undegraded":"2024-09-16T15:38:36.329218+0000","last_fullsized":"2024-09-16T15:38:36.329218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:26.092978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329039+0000","last_change":"2024-09-16T15:38:33.940469+0000","last_active":"2024-09-16T15:38:36.329039+0000","last_peered":"2024-09-16T15:38:36.329039+0000","last_clean":"2024-09-16T15:38:36.329039+0000","last_became_active":"2024-09-16T15:38:33.939752+0000","last_became_peered":"2024-09-16T15:38:33.939752+0000","last_unstale":"2024-09-16T15:38:36.329039+0000","last_undegraded":"2024-09-16T15:38:36.329039+0000","last_fullsized":"2024-09-16T15:38:36.329039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:31.202600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328853+0000","last_change":"2024-09-16T15:38:34.946270+0000","last_active":"2024-09-16T15:38:36.328853+0000","last_peered":"2024-09-16T15:38:36.328853+0000","last_clean":"2024-09-16T15:38:36.328853+0000","last_became_active":"2024-09-16T15:38:34.945965+0000","last_became_peered":"2024-09-16T15:38:34.945965+0000","last_unstale":"2024-09-16T15:38:36.328853+0000","last_undegraded":"2024-09-16T15:38:36.328853+0000","last_fullsized":"2024-09-16T15:38:36.328853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932369+0000","last_change":"2024-09-16T15:38:33.937925+0000","last_active":"2024-09-16T15:38:35.932369+0000","last_peered":"2024-09-16T15:38:35.932369+0000","last_clean":"2024-09-16T15:38:35.932369+0000","last_became_active":"2024-09-16T15:38:33.937648+0000","last_became_peered":"2024-09-16T15:38:33.937648+0000","last_unstale":"2024-09-16T15:38:35.932369+0000","last_undegraded":"2024-09-16T15:38:35.932369+0000","last_fullsized":"2024-09-16T15:38:35.932369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328407+0000","last_change":"2024-09-16T15:38:34.964388+0000","last_active":"2024-09-16T15:38:36.328407+0000","last_peered":"2024-09-16T15:38:36.328407+0000","last_clean":"2024-09-16T15:38:36.328407+0000","last_became_active":"2024-09-16T15:38:34.964136+0000","last_became_peered":"2024-09-16T15:38:34.964136+0000","last_unstale":"2024-09-16T15:38:36.328407+0000","last_undegraded":"2024-09-16T15:38:36.328407+0000","last_fullsized":"2024-09-16T15:38:36.328407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:21.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327747+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:38:36.327747+0000","last_peered":"2024-09-16T15:38:36.327747+0000","last_clean":"2024-09-16T15:38:36.327747+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:38:36.327747+0000","last_undegraded":"2024-09-16T15:38:36.327747+0000","last_fullsized":"2024-09-16T15:38:36.327747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327930+0000","last_change":"2024-09-16T15:38:33.941937+0000","last_active":"2024-09-16T15:38:36.327930+0000","last_peered":"2024-09-16T15:38:36.327930+0000","last_clean":"2024-09-16T15:38:36.327930+0000","last_became_active":"2024-09-16T15:38:33.941212+0000","last_became_peered":"2024-09-16T15:38:33.941212+0000","last_unstale":"2024-09-16T15:38:36.327930+0000","last_undegraded":"2024-09-16T15:38:36.327930+0000","last_fullsized":"2024-09-16T15:38:36.327930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328444+0000","last_change":"2024-09-16T15:38:34.962489+0000","last_active":"2024-09-16T15:38:36.328444+0000","last_peered":"2024-09-16T15:38:36.328444+0000","last_clean":"2024-09-16T15:38:36.328444+0000","last_became_active":"2024-09-16T15:38:34.962245+0000","last_became_peered":"2024-09-16T15:38:34.962245+0000","last_unstale":"2024-09-16T15:38:36.328444+0000","last_undegraded":"2024-09-16T15:38:36.328444+0000","last_fullsized":"2024-09-16T15:38:36.328444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.955896+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.955595+0000","last_became_peered":"2024-09-16T15:38:33.955595+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329773+0000","last_change":"2024-09-16T15:38:34.956268+0000","last_active":"2024-09-16T15:38:36.329773+0000","last_peered":"2024-09-16T15:38:36.329773+0000","last_clean":"2024-09-16T15:38:36.329773+0000","last_became_active":"2024-09-16T15:38:34.956010+0000","last_became_peered":"2024-09-16T15:38:34.956010+0000","last_unstale":"2024-09-16T15:38:36.329773+0000","last_undegraded":"2024-09-16T15:38:36.329773+0000","last_fullsized":"2024-09-16T15:38:36.329773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:02.100232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932349+0000","last_change":"2024-09-16T15:38:33.941933+0000","last_active":"2024-09-16T15:38:35.932349+0000","last_peered":"2024-09-16T15:38:35.932349+0000","last_clean":"2024-09-16T15:38:35.932349+0000","last_became_active":"2024-09-16T15:38:33.941648+0000","last_became_peered":"2024-09-16T15:38:33.941648+0000","last_unstale":"2024-09-16T15:38:35.932349+0000","last_undegraded":"2024-09-16T15:38:35.932349+0000","last_fullsized":"2024-09-16T15:38:35.932349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328622+0000","last_change":"2024-09-16T15:38:34.932401+0000","last_active":"2024-09-16T15:38:36.328622+0000","last_peered":"2024-09-16T15:38:36.328622+0000","last_clean":"2024-09-16T15:38:36.328622+0000","last_became_active":"2024-09-16T15:38:34.931203+0000","last_became_peered":"2024-09-16T15:38:34.931203+0000","last_unstale":"2024-09-16T15:38:36.328622+0000","last_undegraded":"2024-09-16T15:38:36.328622+0000","last_fullsized":"2024-09-16T15:38:36.328622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:24.483689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328535+0000","last_change":"2024-09-16T15:38:33.947515+0000","last_active":"2024-09-16T15:38:36.328535+0000","last_peered":"2024-09-16T15:38:36.328535+0000","last_clean":"2024-09-16T15:38:36.328535+0000","last_became_active":"2024-09-16T15:38:33.946118+0000","last_became_peered":"2024-09-16T15:38:33.946118+0000","last_unstale":"2024-09-16T15:38:36.328535+0000","last_undegraded":"2024-09-16T15:38:36.328535+0000","last_fullsized":"2024-09-16T15:38:36.328535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935354+0000","last_change":"2024-09-16T15:38:34.951051+0000","last_active":"2024-09-16T15:38:35.935354+0000","last_peered":"2024-09-16T15:38:35.935354+0000","last_clean":"2024-09-16T15:38:35.935354+0000","last_became_active":"2024-09-16T15:38:34.950115+0000","last_became_peered":"2024-09-16T15:38:34.950115+0000","last_unstale":"2024-09-16T15:38:35.935354+0000","last_undegraded":"2024-09-16T15:38:35.935354+0000","last_fullsized":"2024-09-16T15:38:35.935354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:29.584103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328940+0000","last_change":"2024-09-16T15:38:33.936405+0000","last_active":"2024-09-16T15:38:36.328940+0000","last_peered":"2024-09-16T15:38:36.328940+0000","last_clean":"2024-09-16T15:38:36.328940+0000","last_became_active":"2024-09-16T15:38:33.935318+0000","last_became_peered":"2024-09-16T15:38:33.935318+0000","last_unstale":"2024-09-16T15:38:36.328940+0000","last_undegraded":"2024-09-16T15:38:36.328940+0000","last_fullsized":"2024-09-16T15:38:36.328940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:27.366083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329408+0000","last_change":"2024-09-16T15:38:34.961561+0000","last_active":"2024-09-16T15:38:36.329408+0000","last_peered":"2024-09-16T15:38:36.329408+0000","last_clean":"2024-09-16T15:38:36.329408+0000","last_became_active":"2024-09-16T15:38:34.961174+0000","last_became_peered":"2024-09-16T15:38:34.961174+0000","last_unstale":"2024-09-16T15:38:36.329408+0000","last_undegraded":"2024-09-16T15:38:36.329408+0000","last_fullsized":"2024-09-16T15:38:36.329408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328656+0000","last_change":"2024-09-16T15:38:33.955332+0000","last_active":"2024-09-16T15:38:36.328656+0000","last_peered":"2024-09-16T15:38:36.328656+0000","last_clean":"2024-09-16T15:38:36.328656+0000","last_became_active":"2024-09-16T15:38:33.954340+0000","last_became_peered":"2024-09-16T15:38:33.954340+0000","last_unstale":"2024-09-16T15:38:36.328656+0000","last_undegraded":"2024-09-16T15:38:36.328656+0000","last_fullsized":"2024-09-16T15:38:36.328656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:45.798957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328724+0000","last_change":"2024-09-16T15:38:34.928319+0000","last_active":"2024-09-16T15:38:36.328724+0000","last_peered":"2024-09-16T15:38:36.328724+0000","last_clean":"2024-09-16T15:38:36.328724+0000","last_became_active":"2024-09-16T15:38:34.928058+0000","last_became_peered":"2024-09-16T15:38:34.928058+0000","last_unstale":"2024-09-16T15:38:36.328724+0000","last_undegraded":"2024-09-16T15:38:36.328724+0000","last_fullsized":"2024-09-16T15:38:36.328724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:00.238312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.929048+0000","last_change":"2024-09-16T15:38:33.931369+0000","last_active":"2024-09-16T15:38:36.929048+0000","last_peered":"2024-09-16T15:38:36.929048+0000","last_clean":"2024-09-16T15:38:36.929048+0000","last_became_active":"2024-09-16T15:38:33.928339+0000","last_became_peered":"2024-09-16T15:38:33.928339+0000","last_unstale":"2024-09-16T15:38:36.929048+0000","last_undegraded":"2024-09-16T15:38:36.929048+0000","last_fullsized":"2024-09-16T15:38:36.929048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:44.073989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329649+0000","last_change":"2024-09-16T15:38:34.942463+0000","last_active":"2024-09-16T15:38:36.329649+0000","last_peered":"2024-09-16T15:38:36.329649+0000","last_clean":"2024-09-16T15:38:36.329649+0000","last_became_active":"2024-09-16T15:38:34.940343+0000","last_became_peered":"2024-09-16T15:38:34.940343+0000","last_unstale":"2024-09-16T15:38:36.329649+0000","last_undegraded":"2024-09-16T15:38:36.329649+0000","last_fullsized":"2024-09-16T15:38:36.329649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935853+0000","last_change":"2024-09-16T15:38:33.947537+0000","last_active":"2024-09-16T15:38:35.935853+0000","last_peered":"2024-09-16T15:38:35.935853+0000","last_clean":"2024-09-16T15:38:35.935853+0000","last_became_active":"2024-09-16T15:38:33.946909+0000","last_became_peered":"2024-09-16T15:38:33.946909+0000","last_unstale":"2024-09-16T15:38:35.935853+0000","last_undegraded":"2024-09-16T15:38:35.935853+0000","last_fullsized":"2024-09-16T15:38:35.935853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:03.179729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330207+0000","last_change":"2024-09-16T15:38:34.941844+0000","last_active":"2024-09-16T15:38:36.330207+0000","last_peered":"2024-09-16T15:38:36.330207+0000","last_clean":"2024-09-16T15:38:36.330207+0000","last_became_active":"2024-09-16T15:38:34.940573+0000","last_became_peered":"2024-09-16T15:38:34.940573+0000","last_unstale":"2024-09-16T15:38:36.330207+0000","last_undegraded":"2024-09-16T15:38:36.330207+0000","last_fullsized":"2024-09-16T15:38:36.330207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:17.824920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.954548+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.954266+0000","last_became_peered":"2024-09-16T15:38:33.954266+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:14.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934862+0000","last_change":"2024-09-16T15:38:34.951875+0000","last_active":"2024-09-16T15:38:35.934862+0000","last_peered":"2024-09-16T15:38:35.934862+0000","last_clean":"2024-09-16T15:38:35.934862+0000","last_became_active":"2024-09-16T15:38:34.946733+0000","last_became_peered":"2024-09-16T15:38:34.946733+0000","last_unstale":"2024-09-16T15:38:35.934862+0000","last_undegraded":"2024-09-16T15:38:35.934862+0000","last_fullsized":"2024-09-16T15:38:35.934862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:35.351445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.947052+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.946593+0000","last_became_peered":"2024-09-16T15:38:33.946593+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:37.618001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328906+0000","last_change":"2024-09-16T15:38:34.933312+0000","last_active":"2024-09-16T15:38:36.328906+0000","last_peered":"2024-09-16T15:38:36.328906+0000","last_clean":"2024-09-16T15:38:36.328906+0000","last_became_active":"2024-09-16T15:38:34.933037+0000","last_became_peered":"2024-09-16T15:38:34.933037+0000","last_unstale":"2024-09-16T15:38:36.328906+0000","last_undegraded":"2024-09-16T15:38:36.328906+0000","last_fullsized":"2024-09-16T15:38:36.328906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:15.092528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.943983+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.943683+0000","last_became_peered":"2024-09-16T15:38:33.943683+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.572327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328590+0000","last_change":"2024-09-16T15:38:34.943957+0000","last_active":"2024-09-16T15:38:36.328590+0000","last_peered":"2024-09-16T15:38:36.328590+0000","last_clean":"2024-09-16T15:38:36.328590+0000","last_became_active":"2024-09-16T15:38:34.941591+0000","last_became_peered":"2024-09-16T15:38:34.941591+0000","last_unstale":"2024-09-16T15:38:36.328590+0000","last_undegraded":"2024-09-16T15:38:36.328590+0000","last_fullsized":"2024-09-16T15:38:36.328590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328378+0000","last_change":"2024-09-16T15:38:33.954584+0000","last_active":"2024-09-16T15:38:36.328378+0000","last_peered":"2024-09-16T15:38:36.328378+0000","last_clean":"2024-09-16T15:38:36.328378+0000","last_became_active":"2024-09-16T15:38:33.954279+0000","last_became_peered":"2024-09-16T15:38:33.954279+0000","last_unstale":"2024-09-16T15:38:36.328378+0000","last_undegraded":"2024-09-16T15:38:36.328378+0000","last_fullsized":"2024-09-16T15:38:36.328378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933817+0000","last_change":"2024-09-16T15:38:34.949723+0000","last_active":"2024-09-16T15:38:35.933817+0000","last_peered":"2024-09-16T15:38:35.933817+0000","last_clean":"2024-09-16T15:38:35.933817+0000","last_became_active":"2024-09-16T15:38:34.949433+0000","last_became_peered":"2024-09-16T15:38:34.949433+0000","last_unstale":"2024-09-16T15:38:35.933817+0000","last_undegraded":"2024-09-16T15:38:35.933817+0000","last_fullsized":"2024-09-16T15:38:35.933817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:33.866690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329345+0000","last_change":"2024-09-16T15:38:34.961924+0000","last_active":"2024-09-16T15:38:36.329345+0000","last_peered":"2024-09-16T15:38:36.329345+0000","last_clean":"2024-09-16T15:38:36.329345+0000","last_became_active":"2024-09-16T15:38:34.961607+0000","last_became_peered":"2024-09-16T15:38:34.961607+0000","last_unstale":"2024-09-16T15:38:36.329345+0000","last_undegraded":"2024-09-16T15:38:36.329345+0000","last_fullsized":"2024-09-16T15:38:36.329345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:58.529110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328347+0000","last_change":"2024-09-16T15:38:33.955384+0000","last_active":"2024-09-16T15:38:36.328347+0000","last_peered":"2024-09-16T15:38:36.328347+0000","last_clean":"2024-09-16T15:38:36.328347+0000","last_became_active":"2024-09-16T15:38:33.954532+0000","last_became_peered":"2024-09-16T15:38:33.954532+0000","last_unstale":"2024-09-16T15:38:36.328347+0000","last_undegraded":"2024-09-16T15:38:36.328347+0000","last_fullsized":"2024-09-16T15:38:36.328347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:28.577867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328852+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328852+0000","last_peered":"2024-09-16T15:38:36.328852+0000","last_clean":"2024-09-16T15:38:36.328852+0000","last_became_active":"2024-09-16T15:38:34.958217+0000","last_became_peered":"2024-09-16T15:38:34.958217+0000","last_unstale":"2024-09-16T15:38:36.328852+0000","last_undegraded":"2024-09-16T15:38:36.328852+0000","last_fullsized":"2024-09-16T15:38:36.328852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:27.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328469+0000","last_change":"2024-09-16T15:38:33.956064+0000","last_active":"2024-09-16T15:38:36.328469+0000","last_peered":"2024-09-16T15:38:36.328469+0000","last_clean":"2024-09-16T15:38:36.328469+0000","last_became_active":"2024-09-16T15:38:33.955717+0000","last_became_peered":"2024-09-16T15:38:33.955717+0000","last_unstale":"2024-09-16T15:38:36.328469+0000","last_undegraded":"2024-09-16T15:38:36.328469+0000","last_fullsized":"2024-09-16T15:38:36.328469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:12.649947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330233+0000","last_change":"2024-09-16T15:38:34.937110+0000","last_active":"2024-09-16T15:38:36.330233+0000","last_peered":"2024-09-16T15:38:36.330233+0000","last_clean":"2024-09-16T15:38:36.330233+0000","last_became_active":"2024-09-16T15:38:34.936439+0000","last_became_peered":"2024-09-16T15:38:34.936439+0000","last_unstale":"2024-09-16T15:38:36.330233+0000","last_undegraded":"2024-09-16T15:38:36.330233+0000","last_fullsized":"2024-09-16T15:38:36.330233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:49.927493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328261+0000","last_change":"2024-09-16T15:38:33.944467+0000","last_active":"2024-09-16T15:38:36.328261+0000","last_peered":"2024-09-16T15:38:36.328261+0000","last_clean":"2024-09-16T15:38:36.328261+0000","last_became_active":"2024-09-16T15:38:33.944211+0000","last_became_peered":"2024-09-16T15:38:33.944211+0000","last_unstale":"2024-09-16T15:38:36.328261+0000","last_undegraded":"2024-09-16T15:38:36.328261+0000","last_fullsized":"2024-09-16T15:38:36.328261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328305+0000","last_change":"2024-09-16T15:38:34.932456+0000","last_active":"2024-09-16T15:38:36.328305+0000","last_peered":"2024-09-16T15:38:36.328305+0000","last_clean":"2024-09-16T15:38:36.328305+0000","last_became_active":"2024-09-16T15:38:34.931374+0000","last_became_peered":"2024-09-16T15:38:34.931374+0000","last_unstale":"2024-09-16T15:38:36.328305+0000","last_undegraded":"2024-09-16T15:38:36.328305+0000","last_fullsized":"2024-09-16T15:38:36.328305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:26.787621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330346+0000","last_change":"2024-09-16T15:38:33.945170+0000","last_active":"2024-09-16T15:38:36.330346+0000","last_peered":"2024-09-16T15:38:36.330346+0000","last_clean":"2024-09-16T15:38:36.330346+0000","last_became_active":"2024-09-16T15:38:33.944681+0000","last_became_peered":"2024-09-16T15:38:33.944681+0000","last_unstale":"2024-09-16T15:38:36.330346+0000","last_undegraded":"2024-09-16T15:38:36.330346+0000","last_fullsized":"2024-09-16T15:38:36.330346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:30.232249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":245760,"data_stored":81953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":672833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":75687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:38:43.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:38:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.093+0000 7f1571c00640 1 -- 172.21.15.60:0/1886088719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1550004800 msgr2=0x7f1550004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.093+0000 7f1571c00640 1 --2- 172.21.15.60:0/1886088719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1550004800 0x7f1550004c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f155400bbb0 tx=0x7f1550005830 comp rx=0 tx=0).stop 2024-09-16T15:38:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.093+0000 7f156b400640 1 -- 172.21.15.60:0/1886088719 reap_dead start 2024-09-16T15:38:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.093+0000 7f1571c00640 1 -- 172.21.15.60:0/1886088719 shutdown_connections 2024-09-16T15:38:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.093+0000 7f1571c00640 1 -- 172.21.15.60:0/1886088719 >> 172.21.15.60:0/1886088719 conn(0x7f156c0b1950 msgr2=0x7f156c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.093+0000 7f1571c00640 1 -- 172.21.15.60:0/1886088719 shutdown_connections 2024-09-16T15:38:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.093+0000 7f1571c00640 1 -- 172.21.15.60:0/1886088719 wait complete. 2024-09-16T15:38:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.095+0000 7f1571c00640 1 Processor -- start 2024-09-16T15:38:44.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.095+0000 7f1571c00640 1 -- start start 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1571c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1550004800 0x7f156c0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1571c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c09d030 0x7f156c0d0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1571c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c0d05c0 0x7f156c1f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1571c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f156c161a70 con 0x7f156c09d030 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1571c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f156c06fdb0 con 0x7f1550004800 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1571c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f156c073280 con 0x7f156c0d05c0 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1562000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c09d030 0x7f156c0d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f156aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1550004800 0x7f156c0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1562000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c09d030 0x7f156c0d0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35164/0 (socket says 172.21.15.60:35164) 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1562000640 1 -- 172.21.15.60:0/1302484187 learned_addr learned my addr 172.21.15.60:0/1302484187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f156b400640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c0d05c0 0x7f156c1f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1562000640 1 -- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c0d05c0 msgr2=0x7f156c1f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1562000640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c0d05c0 0x7f156c1f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1562000640 1 -- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1550004800 msgr2=0x7f156c0cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1562000640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1550004800 0x7f156c0cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f1562000640 1 -- 172.21.15.60:0/1302484187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c1f5860 con 0x7f156c09d030 2024-09-16T15:38:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.096+0000 7f156b400640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c0d05c0 0x7f156c1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.097+0000 7f1562000640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c09d030 0x7f156c0d0080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f155400bcd0 tx=0x7f155400b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.097+0000 7f1563400640 1 -- 172.21.15.60:0/1302484187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554034050 con 0x7f156c09d030 2024-09-16T15:38:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.097+0000 7f1563400640 1 -- 172.21.15.60:0/1302484187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1554005310 con 0x7f156c09d030 2024-09-16T15:38:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.097+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156c09c550 con 0x7f156c09d030 2024-09-16T15:38:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.097+0000 7f1563400640 1 -- 172.21.15.60:0/1302484187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15540047a0 con 0x7f156c09d030 2024-09-16T15:38:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.097+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156c0cf5e0 con 0x7f156c09d030 2024-09-16T15:38:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.098+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f156c0737e0 con 0x7f156c09d030 2024-09-16T15:38:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.099+0000 7f1563400640 1 -- 172.21.15.60:0/1302484187 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1554041030 con 0x7f156c09d030 2024-09-16T15:38:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.100+0000 7f1563400640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1548078880 0x7f154807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.100+0000 7f1563400640 1 -- 172.21.15.60:0/1302484187 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f15540bb3d0 con 0x7f156c09d030 2024-09-16T15:38:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.101+0000 7f156aa00640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1548078880 0x7f154807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.101+0000 7f156aa00640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1548078880 0x7f154807ad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f155c000fb0 tx=0x7f155c006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:44.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.105+0000 7f1563400640 1 -- 172.21.15.60:0/1302484187 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f15540833e0 con 0x7f156c09d030 2024-09-16T15:38:44.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.296+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f156c0d09a0 con 0x7f1548078880 2024-09-16T15:38:44.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.300+0000 7f1563400640 1 -- 172.21.15.60:0/1302484187 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f156c0d09a0 con 0x7f1548078880 2024-09-16T15:38:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:44.306 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:38:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1548078880 msgr2=0x7f154807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1548078880 0x7f154807ad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f155c000fb0 tx=0x7f155c006720 comp rx=0 tx=0).stop 2024-09-16T15:38:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c09d030 msgr2=0x7f156c0d0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c09d030 0x7f156c0d0080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f155400bcd0 tx=0x7f155400b830 comp rx=0 tx=0).stop 2024-09-16T15:38:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 shutdown_connections 2024-09-16T15:38:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1548078880 0x7f154807ad40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c0d05c0 0x7f156c1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c09d030 0x7f156c0d0080 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 --2- 172.21.15.60:0/1302484187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1550004800 0x7f156c0cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 >> 172.21.15.60:0/1302484187 conn(0x7f156c0b1950 msgr2=0x7f156c0ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 shutdown_connections 2024-09-16T15:38:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.305+0000 7f1571c00640 1 -- 172.21.15.60:0/1302484187 wait complete. 2024-09-16T15:38:44.322 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-16T15:38:43.710400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235932,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738788,"statfs":{"total":773094113280,"available":772852518912,"internally_reserved":0,"allocated":3121152,"data_stored":1840172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.793176"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328961+0000","last_change":"2024-09-16T15:38:34.948046+0000","last_active":"2024-09-16T15:38:36.328961+0000","last_peered":"2024-09-16T15:38:36.328961+0000","last_clean":"2024-09-16T15:38:36.328961+0000","last_became_active":"2024-09-16T15:38:34.947778+0000","last_became_peered":"2024-09-16T15:38:34.947778+0000","last_unstale":"2024-09-16T15:38:36.328961+0000","last_undegraded":"2024-09-16T15:38:36.328961+0000","last_fullsized":"2024-09-16T15:38:36.328961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329296+0000","last_change":"2024-09-16T15:38:34.934287+0000","last_active":"2024-09-16T15:38:36.329296+0000","last_peered":"2024-09-16T15:38:36.329296+0000","last_clean":"2024-09-16T15:38:36.329296+0000","last_became_active":"2024-09-16T15:38:34.934004+0000","last_became_peered":"2024-09-16T15:38:34.934004+0000","last_unstale":"2024-09-16T15:38:36.329296+0000","last_undegraded":"2024-09-16T15:38:36.329296+0000","last_fullsized":"2024-09-16T15:38:36.329296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:43.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330192+0000","last_change":"2024-09-16T15:38:34.937280+0000","last_active":"2024-09-16T15:38:36.330192+0000","last_peered":"2024-09-16T15:38:36.330192+0000","last_clean":"2024-09-16T15:38:36.330192+0000","last_became_active":"2024-09-16T15:38:34.936979+0000","last_became_peered":"2024-09-16T15:38:34.936979+0000","last_unstale":"2024-09-16T15:38:36.330192+0000","last_undegraded":"2024-09-16T15:38:36.330192+0000","last_fullsized":"2024-09-16T15:38:36.330192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:16.363921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328077+0000","last_change":"2024-09-16T15:38:34.956532+0000","last_active":"2024-09-16T15:38:36.328077+0000","last_peered":"2024-09-16T15:38:36.328077+0000","last_clean":"2024-09-16T15:38:36.328077+0000","last_became_active":"2024-09-16T15:38:34.956261+0000","last_became_peered":"2024-09-16T15:38:34.956261+0000","last_unstale":"2024-09-16T15:38:36.328077+0000","last_undegraded":"2024-09-16T15:38:36.328077+0000","last_fullsized":"2024-09-16T15:38:36.328077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328317+0000","last_change":"2024-09-16T15:38:34.946850+0000","last_active":"2024-09-16T15:38:36.328317+0000","last_peered":"2024-09-16T15:38:36.328317+0000","last_clean":"2024-09-16T15:38:36.328317+0000","last_became_active":"2024-09-16T15:38:34.946584+0000","last_became_peered":"2024-09-16T15:38:34.946584+0000","last_unstale":"2024-09-16T15:38:36.328317+0000","last_undegraded":"2024-09-16T15:38:36.328317+0000","last_fullsized":"2024-09-16T15:38:36.328317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328172+0000","last_change":"2024-09-16T15:38:34.946360+0000","last_active":"2024-09-16T15:38:36.328172+0000","last_peered":"2024-09-16T15:38:36.328172+0000","last_clean":"2024-09-16T15:38:36.328172+0000","last_became_active":"2024-09-16T15:38:34.946022+0000","last_became_peered":"2024-09-16T15:38:34.946022+0000","last_unstale":"2024-09-16T15:38:36.328172+0000","last_undegraded":"2024-09-16T15:38:36.328172+0000","last_fullsized":"2024-09-16T15:38:36.328172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328807+0000","last_change":"2024-09-16T15:38:34.948730+0000","last_active":"2024-09-16T15:38:36.328807+0000","last_peered":"2024-09-16T15:38:36.328807+0000","last_clean":"2024-09-16T15:38:36.328807+0000","last_became_active":"2024-09-16T15:38:34.948488+0000","last_became_peered":"2024-09-16T15:38:34.948488+0000","last_unstale":"2024-09-16T15:38:36.328807+0000","last_undegraded":"2024-09-16T15:38:36.328807+0000","last_fullsized":"2024-09-16T15:38:36.328807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327904+0000","last_change":"2024-09-16T15:38:34.939960+0000","last_active":"2024-09-16T15:38:36.327904+0000","last_peered":"2024-09-16T15:38:36.327904+0000","last_clean":"2024-09-16T15:38:36.327904+0000","last_became_active":"2024-09-16T15:38:34.939730+0000","last_became_peered":"2024-09-16T15:38:34.939730+0000","last_unstale":"2024-09-16T15:38:36.327904+0000","last_undegraded":"2024-09-16T15:38:36.327904+0000","last_fullsized":"2024-09-16T15:38:36.327904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:35.445142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329103+0000","last_change":"2024-09-16T15:38:34.963612+0000","last_active":"2024-09-16T15:38:36.329103+0000","last_peered":"2024-09-16T15:38:36.329103+0000","last_clean":"2024-09-16T15:38:36.329103+0000","last_became_active":"2024-09-16T15:38:34.963287+0000","last_became_peered":"2024-09-16T15:38:34.963287+0000","last_unstale":"2024-09-16T15:38:36.329103+0000","last_undegraded":"2024-09-16T15:38:36.329103+0000","last_fullsized":"2024-09-16T15:38:36.329103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:53.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935830+0000","last_change":"2024-09-16T15:38:34.959111+0000","last_active":"2024-09-16T15:38:35.935830+0000","last_peered":"2024-09-16T15:38:35.935830+0000","last_clean":"2024-09-16T15:38:35.935830+0000","last_became_active":"2024-09-16T15:38:34.958733+0000","last_became_peered":"2024-09-16T15:38:34.958733+0000","last_unstale":"2024-09-16T15:38:35.935830+0000","last_undegraded":"2024-09-16T15:38:35.935830+0000","last_fullsized":"2024-09-16T15:38:35.935830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:21.940460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934331+0000","last_change":"2024-09-16T15:38:34.963928+0000","last_active":"2024-09-16T15:38:35.934331+0000","last_peered":"2024-09-16T15:38:35.934331+0000","last_clean":"2024-09-16T15:38:35.934331+0000","last_became_active":"2024-09-16T15:38:34.963418+0000","last_became_peered":"2024-09-16T15:38:34.963418+0000","last_unstale":"2024-09-16T15:38:35.934331+0000","last_undegraded":"2024-09-16T15:38:35.934331+0000","last_fullsized":"2024-09-16T15:38:35.934331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:54.016578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935041+0000","last_change":"2024-09-16T15:38:34.951463+0000","last_active":"2024-09-16T15:38:35.935041+0000","last_peered":"2024-09-16T15:38:35.935041+0000","last_clean":"2024-09-16T15:38:35.935041+0000","last_became_active":"2024-09-16T15:38:34.949972+0000","last_became_peered":"2024-09-16T15:38:34.949972+0000","last_unstale":"2024-09-16T15:38:35.935041+0000","last_undegraded":"2024-09-16T15:38:35.935041+0000","last_fullsized":"2024-09-16T15:38:35.935041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:36.356482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328377+0000","last_change":"2024-09-16T15:38:34.936161+0000","last_active":"2024-09-16T15:38:36.328377+0000","last_peered":"2024-09-16T15:38:36.328377+0000","last_clean":"2024-09-16T15:38:36.328377+0000","last_became_active":"2024-09-16T15:38:34.935881+0000","last_became_peered":"2024-09-16T15:38:34.935881+0000","last_unstale":"2024-09-16T15:38:36.328377+0000","last_undegraded":"2024-09-16T15:38:36.328377+0000","last_fullsized":"2024-09-16T15:38:36.328377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329953+0000","last_change":"2024-09-16T15:38:34.934084+0000","last_active":"2024-09-16T15:38:36.329953+0000","last_peered":"2024-09-16T15:38:36.329953+0000","last_clean":"2024-09-16T15:38:36.329953+0000","last_became_active":"2024-09-16T15:38:34.933716+0000","last_became_peered":"2024-09-16T15:38:34.933716+0000","last_unstale":"2024-09-16T15:38:36.329953+0000","last_undegraded":"2024-09-16T15:38:36.329953+0000","last_fullsized":"2024-09-16T15:38:36.329953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:51.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328795+0000","last_change":"2024-09-16T15:38:34.951427+0000","last_active":"2024-09-16T15:38:36.328795+0000","last_peered":"2024-09-16T15:38:36.328795+0000","last_clean":"2024-09-16T15:38:36.328795+0000","last_became_active":"2024-09-16T15:38:34.949484+0000","last_became_peered":"2024-09-16T15:38:34.949484+0000","last_unstale":"2024-09-16T15:38:36.328795+0000","last_undegraded":"2024-09-16T15:38:36.328795+0000","last_fullsized":"2024-09-16T15:38:36.328795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:54.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327906+0000","last_change":"2024-09-16T15:38:34.948833+0000","last_active":"2024-09-16T15:38:36.327906+0000","last_peered":"2024-09-16T15:38:36.327906+0000","last_clean":"2024-09-16T15:38:36.327906+0000","last_became_active":"2024-09-16T15:38:34.948127+0000","last_became_peered":"2024-09-16T15:38:34.948127+0000","last_unstale":"2024-09-16T15:38:36.327906+0000","last_undegraded":"2024-09-16T15:38:36.327906+0000","last_fullsized":"2024-09-16T15:38:36.327906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329716+0000","last_change":"2024-09-16T15:38:34.948165+0000","last_active":"2024-09-16T15:38:36.329716+0000","last_peered":"2024-09-16T15:38:36.329716+0000","last_clean":"2024-09-16T15:38:36.329716+0000","last_became_active":"2024-09-16T15:38:34.947862+0000","last_became_peered":"2024-09-16T15:38:34.947862+0000","last_unstale":"2024-09-16T15:38:36.329716+0000","last_undegraded":"2024-09-16T15:38:36.329716+0000","last_fullsized":"2024-09-16T15:38:36.329716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:07.065693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932521+0000","last_change":"2024-09-16T15:38:34.936088+0000","last_active":"2024-09-16T15:38:35.932521+0000","last_peered":"2024-09-16T15:38:35.932521+0000","last_clean":"2024-09-16T15:38:35.932521+0000","last_became_active":"2024-09-16T15:38:34.935649+0000","last_became_peered":"2024-09-16T15:38:34.935649+0000","last_unstale":"2024-09-16T15:38:35.932521+0000","last_undegraded":"2024-09-16T15:38:35.932521+0000","last_fullsized":"2024-09-16T15:38:35.932521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:56.948937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328715+0000","last_change":"2024-09-16T15:38:34.951511+0000","last_active":"2024-09-16T15:38:36.328715+0000","last_peered":"2024-09-16T15:38:36.328715+0000","last_clean":"2024-09-16T15:38:36.328715+0000","last_became_active":"2024-09-16T15:38:34.951241+0000","last_became_peered":"2024-09-16T15:38:34.951241+0000","last_unstale":"2024-09-16T15:38:36.328715+0000","last_undegraded":"2024-09-16T15:38:36.328715+0000","last_fullsized":"2024-09-16T15:38:36.328715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:52.005508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934261+0000","last_change":"2024-09-16T15:38:34.950265+0000","last_active":"2024-09-16T15:38:35.934261+0000","last_peered":"2024-09-16T15:38:35.934261+0000","last_clean":"2024-09-16T15:38:35.934261+0000","last_became_active":"2024-09-16T15:38:34.949457+0000","last_became_peered":"2024-09-16T15:38:34.949457+0000","last_unstale":"2024-09-16T15:38:35.934261+0000","last_undegraded":"2024-09-16T15:38:35.934261+0000","last_fullsized":"2024-09-16T15:38:35.934261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:33.694559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330049+0000","last_change":"2024-09-16T15:38:34.942753+0000","last_active":"2024-09-16T15:38:36.330049+0000","last_peered":"2024-09-16T15:38:36.330049+0000","last_clean":"2024-09-16T15:38:36.330049+0000","last_became_active":"2024-09-16T15:38:34.940841+0000","last_became_peered":"2024-09-16T15:38:34.940841+0000","last_unstale":"2024-09-16T15:38:36.330049+0000","last_undegraded":"2024-09-16T15:38:36.330049+0000","last_fullsized":"2024-09-16T15:38:36.330049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:22.740035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328374+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328374+0000","last_peered":"2024-09-16T15:38:36.328374+0000","last_clean":"2024-09-16T15:38:36.328374+0000","last_became_active":"2024-09-16T15:38:34.958240+0000","last_became_peered":"2024-09-16T15:38:34.958240+0000","last_unstale":"2024-09-16T15:38:36.328374+0000","last_undegraded":"2024-09-16T15:38:36.328374+0000","last_fullsized":"2024-09-16T15:38:36.328374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328083+0000","last_change":"2024-09-16T15:38:34.952703+0000","last_active":"2024-09-16T15:38:36.328083+0000","last_peered":"2024-09-16T15:38:36.328083+0000","last_clean":"2024-09-16T15:38:36.328083+0000","last_became_active":"2024-09-16T15:38:34.949406+0000","last_became_peered":"2024-09-16T15:38:34.949406+0000","last_unstale":"2024-09-16T15:38:36.328083+0000","last_undegraded":"2024-09-16T15:38:36.328083+0000","last_fullsized":"2024-09-16T15:38:36.328083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:20.072155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328173+0000","last_change":"2024-09-16T15:38:34.948442+0000","last_active":"2024-09-16T15:38:36.328173+0000","last_peered":"2024-09-16T15:38:36.328173+0000","last_clean":"2024-09-16T15:38:36.328173+0000","last_became_active":"2024-09-16T15:38:34.947564+0000","last_became_peered":"2024-09-16T15:38:34.947564+0000","last_unstale":"2024-09-16T15:38:36.328173+0000","last_undegraded":"2024-09-16T15:38:36.328173+0000","last_fullsized":"2024-09-16T15:38:36.328173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328339+0000","last_change":"2024-09-16T15:38:34.940042+0000","last_active":"2024-09-16T15:38:36.328339+0000","last_peered":"2024-09-16T15:38:36.328339+0000","last_clean":"2024-09-16T15:38:36.328339+0000","last_became_active":"2024-09-16T15:38:34.939792+0000","last_became_peered":"2024-09-16T15:38:34.939792+0000","last_unstale":"2024-09-16T15:38:36.328339+0000","last_undegraded":"2024-09-16T15:38:36.328339+0000","last_fullsized":"2024-09-16T15:38:36.328339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.386845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935854+0000","last_change":"2024-09-16T15:38:34.938919+0000","last_active":"2024-09-16T15:38:35.935854+0000","last_peered":"2024-09-16T15:38:35.935854+0000","last_clean":"2024-09-16T15:38:35.935854+0000","last_became_active":"2024-09-16T15:38:34.938608+0000","last_became_peered":"2024-09-16T15:38:34.938608+0000","last_unstale":"2024-09-16T15:38:35.935854+0000","last_undegraded":"2024-09-16T15:38:35.935854+0000","last_fullsized":"2024-09-16T15:38:35.935854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:44.137157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329086+0000","last_change":"2024-09-16T15:38:34.934519+0000","last_active":"2024-09-16T15:38:36.329086+0000","last_peered":"2024-09-16T15:38:36.329086+0000","last_clean":"2024-09-16T15:38:36.329086+0000","last_became_active":"2024-09-16T15:38:34.932223+0000","last_became_peered":"2024-09-16T15:38:34.932223+0000","last_unstale":"2024-09-16T15:38:36.329086+0000","last_undegraded":"2024-09-16T15:38:36.329086+0000","last_fullsized":"2024-09-16T15:38:36.329086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.673101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329499+0000","last_change":"2024-09-16T15:38:34.941297+0000","last_active":"2024-09-16T15:38:36.329499+0000","last_peered":"2024-09-16T15:38:36.329499+0000","last_clean":"2024-09-16T15:38:36.329499+0000","last_became_active":"2024-09-16T15:38:34.941007+0000","last_became_peered":"2024-09-16T15:38:34.941007+0000","last_unstale":"2024-09-16T15:38:36.329499+0000","last_undegraded":"2024-09-16T15:38:36.329499+0000","last_fullsized":"2024-09-16T15:38:36.329499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330077+0000","last_change":"2024-09-16T15:38:34.941110+0000","last_active":"2024-09-16T15:38:36.330077+0000","last_peered":"2024-09-16T15:38:36.330077+0000","last_clean":"2024-09-16T15:38:36.330077+0000","last_became_active":"2024-09-16T15:38:34.940650+0000","last_became_peered":"2024-09-16T15:38:34.940650+0000","last_unstale":"2024-09-16T15:38:36.330077+0000","last_undegraded":"2024-09-16T15:38:36.330077+0000","last_fullsized":"2024-09-16T15:38:36.330077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:26.093004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328522+0000","last_change":"2024-09-16T15:38:34.952930+0000","last_active":"2024-09-16T15:38:36.328522+0000","last_peered":"2024-09-16T15:38:36.328522+0000","last_clean":"2024-09-16T15:38:36.328522+0000","last_became_active":"2024-09-16T15:38:34.952652+0000","last_became_peered":"2024-09-16T15:38:34.952652+0000","last_unstale":"2024-09-16T15:38:36.328522+0000","last_undegraded":"2024-09-16T15:38:36.328522+0000","last_fullsized":"2024-09-16T15:38:36.328522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932490+0000","last_change":"2024-09-16T15:38:34.961835+0000","last_active":"2024-09-16T15:38:35.932490+0000","last_peered":"2024-09-16T15:38:35.932490+0000","last_clean":"2024-09-16T15:38:35.932490+0000","last_became_active":"2024-09-16T15:38:34.961529+0000","last_became_peered":"2024-09-16T15:38:34.961529+0000","last_unstale":"2024-09-16T15:38:35.932490+0000","last_undegraded":"2024-09-16T15:38:35.932490+0000","last_fullsized":"2024-09-16T15:38:35.932490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:11.053973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933841+0000","last_change":"2024-09-16T15:38:34.942442+0000","last_active":"2024-09-16T15:38:35.933841+0000","last_peered":"2024-09-16T15:38:35.933841+0000","last_clean":"2024-09-16T15:38:35.933841+0000","last_became_active":"2024-09-16T15:38:34.942144+0000","last_became_peered":"2024-09-16T15:38:34.942144+0000","last_unstale":"2024-09-16T15:38:35.933841+0000","last_undegraded":"2024-09-16T15:38:35.933841+0000","last_fullsized":"2024-09-16T15:38:35.933841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:43.399475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934635+0000","last_change":"2024-09-16T15:38:34.941230+0000","last_active":"2024-09-16T15:38:35.934635+0000","last_peered":"2024-09-16T15:38:35.934635+0000","last_clean":"2024-09-16T15:38:35.934635+0000","last_became_active":"2024-09-16T15:38:34.940613+0000","last_became_peered":"2024-09-16T15:38:34.940613+0000","last_unstale":"2024-09-16T15:38:35.934635+0000","last_undegraded":"2024-09-16T15:38:35.934635+0000","last_fullsized":"2024-09-16T15:38:35.934635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:15.007186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328673+0000","last_change":"2024-09-16T15:38:33.949962+0000","last_active":"2024-09-16T15:38:36.328673+0000","last_peered":"2024-09-16T15:38:36.328673+0000","last_clean":"2024-09-16T15:38:36.328673+0000","last_became_active":"2024-09-16T15:38:33.949620+0000","last_became_peered":"2024-09-16T15:38:33.949620+0000","last_unstale":"2024-09-16T15:38:36.328673+0000","last_undegraded":"2024-09-16T15:38:36.328673+0000","last_fullsized":"2024-09-16T15:38:36.328673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330281+0000","last_change":"2024-09-16T15:38:34.947478+0000","last_active":"2024-09-16T15:38:36.330281+0000","last_peered":"2024-09-16T15:38:36.330281+0000","last_clean":"2024-09-16T15:38:36.330281+0000","last_became_active":"2024-09-16T15:38:34.947158+0000","last_became_peered":"2024-09-16T15:38:34.947158+0000","last_unstale":"2024-09-16T15:38:36.330281+0000","last_undegraded":"2024-09-16T15:38:36.330281+0000","last_fullsized":"2024-09-16T15:38:36.330281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:24.497838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933892+0000","last_change":"2024-09-16T15:38:33.931830+0000","last_active":"2024-09-16T15:38:35.933892+0000","last_peered":"2024-09-16T15:38:35.933892+0000","last_clean":"2024-09-16T15:38:35.933892+0000","last_became_active":"2024-09-16T15:38:33.931129+0000","last_became_peered":"2024-09-16T15:38:33.931129+0000","last_unstale":"2024-09-16T15:38:35.933892+0000","last_undegraded":"2024-09-16T15:38:35.933892+0000","last_fullsized":"2024-09-16T15:38:35.933892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:54.747927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327884+0000","last_change":"2024-09-16T15:38:33.943597+0000","last_active":"2024-09-16T15:38:36.327884+0000","last_peered":"2024-09-16T15:38:36.327884+0000","last_clean":"2024-09-16T15:38:36.327884+0000","last_became_active":"2024-09-16T15:38:33.940904+0000","last_became_peered":"2024-09-16T15:38:33.940904+0000","last_unstale":"2024-09-16T15:38:36.327884+0000","last_undegraded":"2024-09-16T15:38:36.327884+0000","last_fullsized":"2024-09-16T15:38:36.327884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:13.818889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328165+0000","last_change":"2024-09-16T15:38:34.942424+0000","last_active":"2024-09-16T15:38:36.328165+0000","last_peered":"2024-09-16T15:38:36.328165+0000","last_clean":"2024-09-16T15:38:36.328165+0000","last_became_active":"2024-09-16T15:38:34.942111+0000","last_became_peered":"2024-09-16T15:38:34.942111+0000","last_unstale":"2024-09-16T15:38:36.328165+0000","last_undegraded":"2024-09-16T15:38:36.328165+0000","last_fullsized":"2024-09-16T15:38:36.328165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:52.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328244+0000","last_change":"2024-09-16T15:38:33.945014+0000","last_active":"2024-09-16T15:38:36.328244+0000","last_peered":"2024-09-16T15:38:36.328244+0000","last_clean":"2024-09-16T15:38:36.328244+0000","last_became_active":"2024-09-16T15:38:33.944742+0000","last_became_peered":"2024-09-16T15:38:33.944742+0000","last_unstale":"2024-09-16T15:38:36.328244+0000","last_undegraded":"2024-09-16T15:38:36.328244+0000","last_fullsized":"2024-09-16T15:38:36.328244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328641+0000","last_change":"2024-09-16T15:38:34.933709+0000","last_active":"2024-09-16T15:38:36.328641+0000","last_peered":"2024-09-16T15:38:36.328641+0000","last_clean":"2024-09-16T15:38:36.328641+0000","last_became_active":"2024-09-16T15:38:34.932126+0000","last_became_peered":"2024-09-16T15:38:34.932126+0000","last_unstale":"2024-09-16T15:38:36.328641+0000","last_undegraded":"2024-09-16T15:38:36.328641+0000","last_fullsized":"2024-09-16T15:38:36.328641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:44.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935745+0000","last_change":"2024-09-16T15:38:33.945379+0000","last_active":"2024-09-16T15:38:35.935745+0000","last_peered":"2024-09-16T15:38:35.935745+0000","last_clean":"2024-09-16T15:38:35.935745+0000","last_became_active":"2024-09-16T15:38:33.944531+0000","last_became_peered":"2024-09-16T15:38:33.944531+0000","last_unstale":"2024-09-16T15:38:35.935745+0000","last_undegraded":"2024-09-16T15:38:35.935745+0000","last_fullsized":"2024-09-16T15:38:35.935745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:09.939574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329480+0000","last_change":"2024-09-16T15:38:34.961412+0000","last_active":"2024-09-16T15:38:36.329480+0000","last_peered":"2024-09-16T15:38:36.329480+0000","last_clean":"2024-09-16T15:38:36.329480+0000","last_became_active":"2024-09-16T15:38:34.961004+0000","last_became_peered":"2024-09-16T15:38:34.961004+0000","last_unstale":"2024-09-16T15:38:36.329480+0000","last_undegraded":"2024-09-16T15:38:36.329480+0000","last_fullsized":"2024-09-16T15:38:36.329480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:39.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329349+0000","last_change":"2024-09-16T15:38:33.942028+0000","last_active":"2024-09-16T15:38:36.329349+0000","last_peered":"2024-09-16T15:38:36.329349+0000","last_clean":"2024-09-16T15:38:36.329349+0000","last_became_active":"2024-09-16T15:38:33.940695+0000","last_became_peered":"2024-09-16T15:38:33.940695+0000","last_unstale":"2024-09-16T15:38:36.329349+0000","last_undegraded":"2024-09-16T15:38:36.329349+0000","last_fullsized":"2024-09-16T15:38:36.329349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330396+0000","last_change":"2024-09-16T15:38:34.937478+0000","last_active":"2024-09-16T15:38:36.330396+0000","last_peered":"2024-09-16T15:38:36.330396+0000","last_clean":"2024-09-16T15:38:36.330396+0000","last_became_active":"2024-09-16T15:38:34.936844+0000","last_became_peered":"2024-09-16T15:38:34.936844+0000","last_unstale":"2024-09-16T15:38:36.330396+0000","last_undegraded":"2024-09-16T15:38:36.330396+0000","last_fullsized":"2024-09-16T15:38:36.330396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:42.401407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328973+0000","last_change":"2024-09-16T15:38:33.942385+0000","last_active":"2024-09-16T15:38:36.328973+0000","last_peered":"2024-09-16T15:38:36.328973+0000","last_clean":"2024-09-16T15:38:36.328973+0000","last_became_active":"2024-09-16T15:38:33.941960+0000","last_became_peered":"2024-09-16T15:38:33.941960+0000","last_unstale":"2024-09-16T15:38:36.328973+0000","last_undegraded":"2024-09-16T15:38:36.328973+0000","last_fullsized":"2024-09-16T15:38:36.328973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:08.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935156+0000","last_change":"2024-09-16T15:38:34.955492+0000","last_active":"2024-09-16T15:38:35.935156+0000","last_peered":"2024-09-16T15:38:35.935156+0000","last_clean":"2024-09-16T15:38:35.935156+0000","last_became_active":"2024-09-16T15:38:34.955230+0000","last_became_peered":"2024-09-16T15:38:34.955230+0000","last_unstale":"2024-09-16T15:38:35.935156+0000","last_undegraded":"2024-09-16T15:38:35.935156+0000","last_fullsized":"2024-09-16T15:38:35.935156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:13.656183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934327+0000","last_change":"2024-09-16T15:38:33.939711+0000","last_active":"2024-09-16T15:38:35.934327+0000","last_peered":"2024-09-16T15:38:35.934327+0000","last_clean":"2024-09-16T15:38:35.934327+0000","last_became_active":"2024-09-16T15:38:33.937858+0000","last_became_peered":"2024-09-16T15:38:33.937858+0000","last_unstale":"2024-09-16T15:38:35.934327+0000","last_undegraded":"2024-09-16T15:38:35.934327+0000","last_fullsized":"2024-09-16T15:38:35.934327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:42.581730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328596+0000","last_change":"2024-09-16T15:38:34.947613+0000","last_active":"2024-09-16T15:38:36.328596+0000","last_peered":"2024-09-16T15:38:36.328596+0000","last_clean":"2024-09-16T15:38:36.328596+0000","last_became_active":"2024-09-16T15:38:34.947258+0000","last_became_peered":"2024-09-16T15:38:34.947258+0000","last_unstale":"2024-09-16T15:38:36.328596+0000","last_undegraded":"2024-09-16T15:38:36.328596+0000","last_fullsized":"2024-09-16T15:38:36.328596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.135119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330167+0000","last_change":"2024-09-16T15:38:33.936316+0000","last_active":"2024-09-16T15:38:36.330167+0000","last_peered":"2024-09-16T15:38:36.330167+0000","last_clean":"2024-09-16T15:38:36.330167+0000","last_became_active":"2024-09-16T15:38:33.936002+0000","last_became_peered":"2024-09-16T15:38:33.936002+0000","last_unstale":"2024-09-16T15:38:36.330167+0000","last_undegraded":"2024-09-16T15:38:36.330167+0000","last_fullsized":"2024-09-16T15:38:36.330167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.796157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329907+0000","last_change":"2024-09-16T15:38:34.942140+0000","last_active":"2024-09-16T15:38:36.329907+0000","last_peered":"2024-09-16T15:38:36.329907+0000","last_clean":"2024-09-16T15:38:36.329907+0000","last_became_active":"2024-09-16T15:38:34.940716+0000","last_became_peered":"2024-09-16T15:38:34.940716+0000","last_unstale":"2024-09-16T15:38:36.329907+0000","last_undegraded":"2024-09-16T15:38:36.329907+0000","last_fullsized":"2024-09-16T15:38:36.329907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:55.566537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328440+0000","last_change":"2024-09-16T15:38:33.938693+0000","last_active":"2024-09-16T15:38:36.328440+0000","last_peered":"2024-09-16T15:38:36.328440+0000","last_clean":"2024-09-16T15:38:36.328440+0000","last_became_active":"2024-09-16T15:38:33.938207+0000","last_became_peered":"2024-09-16T15:38:33.938207+0000","last_unstale":"2024-09-16T15:38:36.328440+0000","last_undegraded":"2024-09-16T15:38:36.328440+0000","last_fullsized":"2024-09-16T15:38:36.328440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328857+0000","last_change":"2024-09-16T15:38:34.956559+0000","last_active":"2024-09-16T15:38:36.328857+0000","last_peered":"2024-09-16T15:38:36.328857+0000","last_clean":"2024-09-16T15:38:36.328857+0000","last_became_active":"2024-09-16T15:38:34.956290+0000","last_became_peered":"2024-09-16T15:38:34.956290+0000","last_unstale":"2024-09-16T15:38:36.328857+0000","last_undegraded":"2024-09-16T15:38:36.328857+0000","last_fullsized":"2024-09-16T15:38:36.328857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:34.525651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933781+0000","last_change":"2024-09-16T15:38:33.937022+0000","last_active":"2024-09-16T15:38:35.933781+0000","last_peered":"2024-09-16T15:38:35.933781+0000","last_clean":"2024-09-16T15:38:35.933781+0000","last_became_active":"2024-09-16T15:38:33.936692+0000","last_became_peered":"2024-09-16T15:38:33.936692+0000","last_unstale":"2024-09-16T15:38:35.933781+0000","last_undegraded":"2024-09-16T15:38:35.933781+0000","last_fullsized":"2024-09-16T15:38:35.933781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:23.195213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933838+0000","last_change":"2024-09-16T15:38:34.933429+0000","last_active":"2024-09-16T15:38:35.933838+0000","last_peered":"2024-09-16T15:38:35.933838+0000","last_clean":"2024-09-16T15:38:35.933838+0000","last_became_active":"2024-09-16T15:38:34.933116+0000","last_became_peered":"2024-09-16T15:38:34.933116+0000","last_unstale":"2024-09-16T15:38:35.933838+0000","last_undegraded":"2024-09-16T15:38:35.933838+0000","last_fullsized":"2024-09-16T15:38:35.933838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:16.415478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932158+0000","last_change":"2024-09-16T15:38:33.929097+0000","last_active":"2024-09-16T15:38:35.932158+0000","last_peered":"2024-09-16T15:38:35.932158+0000","last_clean":"2024-09-16T15:38:35.932158+0000","last_became_active":"2024-09-16T15:38:33.928783+0000","last_became_peered":"2024-09-16T15:38:33.928783+0000","last_unstale":"2024-09-16T15:38:35.932158+0000","last_undegraded":"2024-09-16T15:38:35.932158+0000","last_fullsized":"2024-09-16T15:38:35.932158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327843+0000","last_change":"2024-09-16T15:38:34.955962+0000","last_active":"2024-09-16T15:38:36.327843+0000","last_peered":"2024-09-16T15:38:36.327843+0000","last_clean":"2024-09-16T15:38:36.327843+0000","last_became_active":"2024-09-16T15:38:34.955617+0000","last_became_peered":"2024-09-16T15:38:34.955617+0000","last_unstale":"2024-09-16T15:38:36.327843+0000","last_undegraded":"2024-09-16T15:38:36.327843+0000","last_fullsized":"2024-09-16T15:38:36.327843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:35.342731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935301+0000","last_change":"2024-09-16T15:38:33.933311+0000","last_active":"2024-09-16T15:38:35.935301+0000","last_peered":"2024-09-16T15:38:35.935301+0000","last_clean":"2024-09-16T15:38:35.935301+0000","last_became_active":"2024-09-16T15:38:33.932882+0000","last_became_peered":"2024-09-16T15:38:33.932882+0000","last_unstale":"2024-09-16T15:38:35.935301+0000","last_undegraded":"2024-09-16T15:38:35.935301+0000","last_fullsized":"2024-09-16T15:38:35.935301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:43.856688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328955+0000","last_change":"2024-09-16T15:38:34.959213+0000","last_active":"2024-09-16T15:38:36.328955+0000","last_peered":"2024-09-16T15:38:36.328955+0000","last_clean":"2024-09-16T15:38:36.328955+0000","last_became_active":"2024-09-16T15:38:34.958928+0000","last_became_peered":"2024-09-16T15:38:34.958928+0000","last_unstale":"2024-09-16T15:38:36.328955+0000","last_undegraded":"2024-09-16T15:38:36.328955+0000","last_fullsized":"2024-09-16T15:38:36.328955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:51.093521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932531+0000","last_change":"2024-09-16T15:38:33.939655+0000","last_active":"2024-09-16T15:38:35.932531+0000","last_peered":"2024-09-16T15:38:35.932531+0000","last_clean":"2024-09-16T15:38:35.932531+0000","last_became_active":"2024-09-16T15:38:33.939329+0000","last_became_peered":"2024-09-16T15:38:33.939329+0000","last_unstale":"2024-09-16T15:38:35.932531+0000","last_undegraded":"2024-09-16T15:38:35.932531+0000","last_fullsized":"2024-09-16T15:38:35.932531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329218+0000","last_change":"2024-09-16T15:38:34.956364+0000","last_active":"2024-09-16T15:38:36.329218+0000","last_peered":"2024-09-16T15:38:36.329218+0000","last_clean":"2024-09-16T15:38:36.329218+0000","last_became_active":"2024-09-16T15:38:34.956041+0000","last_became_peered":"2024-09-16T15:38:34.956041+0000","last_unstale":"2024-09-16T15:38:36.329218+0000","last_undegraded":"2024-09-16T15:38:36.329218+0000","last_fullsized":"2024-09-16T15:38:36.329218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:26.092978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329039+0000","last_change":"2024-09-16T15:38:33.940469+0000","last_active":"2024-09-16T15:38:36.329039+0000","last_peered":"2024-09-16T15:38:36.329039+0000","last_clean":"2024-09-16T15:38:36.329039+0000","last_became_active":"2024-09-16T15:38:33.939752+0000","last_became_peered":"2024-09-16T15:38:33.939752+0000","last_unstale":"2024-09-16T15:38:36.329039+0000","last_undegraded":"2024-09-16T15:38:36.329039+0000","last_fullsized":"2024-09-16T15:38:36.329039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:31.202600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328853+0000","last_change":"2024-09-16T15:38:34.946270+0000","last_active":"2024-09-16T15:38:36.328853+0000","last_peered":"2024-09-16T15:38:36.328853+0000","last_clean":"2024-09-16T15:38:36.328853+0000","last_became_active":"2024-09-16T15:38:34.945965+0000","last_became_peered":"2024-09-16T15:38:34.945965+0000","last_unstale":"2024-09-16T15:38:36.328853+0000","last_undegraded":"2024-09-16T15:38:36.328853+0000","last_fullsized":"2024-09-16T15:38:36.328853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932369+0000","last_change":"2024-09-16T15:38:33.937925+0000","last_active":"2024-09-16T15:38:35.932369+0000","last_peered":"2024-09-16T15:38:35.932369+0000","last_clean":"2024-09-16T15:38:35.932369+0000","last_became_active":"2024-09-16T15:38:33.937648+0000","last_became_peered":"2024-09-16T15:38:33.937648+0000","last_unstale":"2024-09-16T15:38:35.932369+0000","last_undegraded":"2024-09-16T15:38:35.932369+0000","last_fullsized":"2024-09-16T15:38:35.932369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328407+0000","last_change":"2024-09-16T15:38:34.964388+0000","last_active":"2024-09-16T15:38:36.328407+0000","last_peered":"2024-09-16T15:38:36.328407+0000","last_clean":"2024-09-16T15:38:36.328407+0000","last_became_active":"2024-09-16T15:38:34.964136+0000","last_became_peered":"2024-09-16T15:38:34.964136+0000","last_unstale":"2024-09-16T15:38:36.328407+0000","last_undegraded":"2024-09-16T15:38:36.328407+0000","last_fullsized":"2024-09-16T15:38:36.328407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:21.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327747+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:38:36.327747+0000","last_peered":"2024-09-16T15:38:36.327747+0000","last_clean":"2024-09-16T15:38:36.327747+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:38:36.327747+0000","last_undegraded":"2024-09-16T15:38:36.327747+0000","last_fullsized":"2024-09-16T15:38:36.327747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327930+0000","last_change":"2024-09-16T15:38:33.941937+0000","last_active":"2024-09-16T15:38:36.327930+0000","last_peered":"2024-09-16T15:38:36.327930+0000","last_clean":"2024-09-16T15:38:36.327930+0000","last_became_active":"2024-09-16T15:38:33.941212+0000","last_became_peered":"2024-09-16T15:38:33.941212+0000","last_unstale":"2024-09-16T15:38:36.327930+0000","last_undegraded":"2024-09-16T15:38:36.327930+0000","last_fullsized":"2024-09-16T15:38:36.327930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328444+0000","last_change":"2024-09-16T15:38:34.962489+0000","last_active":"2024-09-16T15:38:36.328444+0000","last_peered":"2024-09-16T15:38:36.328444+0000","last_clean":"2024-09-16T15:38:36.328444+0000","last_became_active":"2024-09-16T15:38:34.962245+0000","last_became_peered":"2024-09-16T15:38:34.962245+0000","last_unstale":"2024-09-16T15:38:36.328444+0000","last_undegraded":"2024-09-16T15:38:36.328444+0000","last_fullsized":"2024-09-16T15:38:36.328444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.955896+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.955595+0000","last_became_peered":"2024-09-16T15:38:33.955595+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329773+0000","last_change":"2024-09-16T15:38:34.956268+0000","last_active":"2024-09-16T15:38:36.329773+0000","last_peered":"2024-09-16T15:38:36.329773+0000","last_clean":"2024-09-16T15:38:36.329773+0000","last_became_active":"2024-09-16T15:38:34.956010+0000","last_became_peered":"2024-09-16T15:38:34.956010+0000","last_unstale":"2024-09-16T15:38:36.329773+0000","last_undegraded":"2024-09-16T15:38:36.329773+0000","last_fullsized":"2024-09-16T15:38:36.329773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:02.100232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932349+0000","last_change":"2024-09-16T15:38:33.941933+0000","last_active":"2024-09-16T15:38:35.932349+0000","last_peered":"2024-09-16T15:38:35.932349+0000","last_clean":"2024-09-16T15:38:35.932349+0000","last_became_active":"2024-09-16T15:38:33.941648+0000","last_became_peered":"2024-09-16T15:38:33.941648+0000","last_unstale":"2024-09-16T15:38:35.932349+0000","last_undegraded":"2024-09-16T15:38:35.932349+0000","last_fullsized":"2024-09-16T15:38:35.932349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328622+0000","last_change":"2024-09-16T15:38:34.932401+0000","last_active":"2024-09-16T15:38:36.328622+0000","last_peered":"2024-09-16T15:38:36.328622+0000","last_clean":"2024-09-16T15:38:36.328622+0000","last_became_active":"2024-09-16T15:38:34.931203+0000","last_became_peered":"2024-09-16T15:38:34.931203+0000","last_unstale":"2024-09-16T15:38:36.328622+0000","last_undegraded":"2024-09-16T15:38:36.328622+0000","last_fullsized":"2024-09-16T15:38:36.328622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:24.483689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328535+0000","last_change":"2024-09-16T15:38:33.947515+0000","last_active":"2024-09-16T15:38:36.328535+0000","last_peered":"2024-09-16T15:38:36.328535+0000","last_clean":"2024-09-16T15:38:36.328535+0000","last_became_active":"2024-09-16T15:38:33.946118+0000","last_became_peered":"2024-09-16T15:38:33.946118+0000","last_unstale":"2024-09-16T15:38:36.328535+0000","last_undegraded":"2024-09-16T15:38:36.328535+0000","last_fullsized":"2024-09-16T15:38:36.328535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935354+0000","last_change":"2024-09-16T15:38:34.951051+0000","last_active":"2024-09-16T15:38:35.935354+0000","last_peered":"2024-09-16T15:38:35.935354+0000","last_clean":"2024-09-16T15:38:35.935354+0000","last_became_active":"2024-09-16T15:38:34.950115+0000","last_became_peered":"2024-09-16T15:38:34.950115+0000","last_unstale":"2024-09-16T15:38:35.935354+0000","last_undegraded":"2024-09-16T15:38:35.935354+0000","last_fullsized":"2024-09-16T15:38:35.935354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:29.584103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328940+0000","last_change":"2024-09-16T15:38:33.936405+0000","last_active":"2024-09-16T15:38:36.328940+0000","last_peered":"2024-09-16T15:38:36.328940+0000","last_clean":"2024-09-16T15:38:36.328940+0000","last_became_active":"2024-09-16T15:38:33.935318+0000","last_became_peered":"2024-09-16T15:38:33.935318+0000","last_unstale":"2024-09-16T15:38:36.328940+0000","last_undegraded":"2024-09-16T15:38:36.328940+0000","last_fullsized":"2024-09-16T15:38:36.328940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:27.366083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329408+0000","last_change":"2024-09-16T15:38:34.961561+0000","last_active":"2024-09-16T15:38:36.329408+0000","last_peered":"2024-09-16T15:38:36.329408+0000","last_clean":"2024-09-16T15:38:36.329408+0000","last_became_active":"2024-09-16T15:38:34.961174+0000","last_became_peered":"2024-09-16T15:38:34.961174+0000","last_unstale":"2024-09-16T15:38:36.329408+0000","last_undegraded":"2024-09-16T15:38:36.329408+0000","last_fullsized":"2024-09-16T15:38:36.329408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328656+0000","last_change":"2024-09-16T15:38:33.955332+0000","last_active":"2024-09-16T15:38:36.328656+0000","last_peered":"2024-09-16T15:38:36.328656+0000","last_clean":"2024-09-16T15:38:36.328656+0000","last_became_active":"2024-09-16T15:38:33.954340+0000","last_became_peered":"2024-09-16T15:38:33.954340+0000","last_unstale":"2024-09-16T15:38:36.328656+0000","last_undegraded":"2024-09-16T15:38:36.328656+0000","last_fullsized":"2024-09-16T15:38:36.328656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:45.798957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328724+0000","last_change":"2024-09-16T15:38:34.928319+0000","last_active":"2024-09-16T15:38:36.328724+0000","last_peered":"2024-09-16T15:38:36.328724+0000","last_clean":"2024-09-16T15:38:36.328724+0000","last_became_active":"2024-09-16T15:38:34.928058+0000","last_became_peered":"2024-09-16T15:38:34.928058+0000","last_unstale":"2024-09-16T15:38:36.328724+0000","last_undegraded":"2024-09-16T15:38:36.328724+0000","last_fullsized":"2024-09-16T15:38:36.328724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:00.238312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.929048+0000","last_change":"2024-09-16T15:38:33.931369+0000","last_active":"2024-09-16T15:38:36.929048+0000","last_peered":"2024-09-16T15:38:36.929048+0000","last_clean":"2024-09-16T15:38:36.929048+0000","last_became_active":"2024-09-16T15:38:33.928339+0000","last_became_peered":"2024-09-16T15:38:33.928339+0000","last_unstale":"2024-09-16T15:38:36.929048+0000","last_undegraded":"2024-09-16T15:38:36.929048+0000","last_fullsized":"2024-09-16T15:38:36.929048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:44.073989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329649+0000","last_change":"2024-09-16T15:38:34.942463+0000","last_active":"2024-09-16T15:38:36.329649+0000","last_peered":"2024-09-16T15:38:36.329649+0000","last_clean":"2024-09-16T15:38:36.329649+0000","last_became_active":"2024-09-16T15:38:34.940343+0000","last_became_peered":"2024-09-16T15:38:34.940343+0000","last_unstale":"2024-09-16T15:38:36.329649+0000","last_undegraded":"2024-09-16T15:38:36.329649+0000","last_fullsized":"2024-09-16T15:38:36.329649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935853+0000","last_change":"2024-09-16T15:38:33.947537+0000","last_active":"2024-09-16T15:38:35.935853+0000","last_peered":"2024-09-16T15:38:35.935853+0000","last_clean":"2024-09-16T15:38:35.935853+0000","last_became_active":"2024-09-16T15:38:33.946909+0000","last_became_peered":"2024-09-16T15:38:33.946909+0000","last_unstale":"2024-09-16T15:38:35.935853+0000","last_undegraded":"2024-09-16T15:38:35.935853+0000","last_fullsized":"2024-09-16T15:38:35.935853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:03.179729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330207+0000","last_change":"2024-09-16T15:38:34.941844+0000","last_active":"2024-09-16T15:38:36.330207+0000","last_peered":"2024-09-16T15:38:36.330207+0000","last_clean":"2024-09-16T15:38:36.330207+0000","last_became_active":"2024-09-16T15:38:34.940573+0000","last_became_peered":"2024-09-16T15:38:34.940573+0000","last_unstale":"2024-09-16T15:38:36.330207+0000","last_undegraded":"2024-09-16T15:38:36.330207+0000","last_fullsized":"2024-09-16T15:38:36.330207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:17.824920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.954548+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.954266+0000","last_became_peered":"2024-09-16T15:38:33.954266+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:14.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934862+0000","last_change":"2024-09-16T15:38:34.951875+0000","last_active":"2024-09-16T15:38:35.934862+0000","last_peered":"2024-09-16T15:38:35.934862+0000","last_clean":"2024-09-16T15:38:35.934862+0000","last_became_active":"2024-09-16T15:38:34.946733+0000","last_became_peered":"2024-09-16T15:38:34.946733+0000","last_unstale":"2024-09-16T15:38:35.934862+0000","last_undegraded":"2024-09-16T15:38:35.934862+0000","last_fullsized":"2024-09-16T15:38:35.934862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:35.351445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.947052+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.946593+0000","last_became_peered":"2024-09-16T15:38:33.946593+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:37.618001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328906+0000","last_change":"2024-09-16T15:38:34.933312+0000","last_active":"2024-09-16T15:38:36.328906+0000","last_peered":"2024-09-16T15:38:36.328906+0000","last_clean":"2024-09-16T15:38:36.328906+0000","last_became_active":"2024-09-16T15:38:34.933037+0000","last_became_peered":"2024-09-16T15:38:34.933037+0000","last_unstale":"2024-09-16T15:38:36.328906+0000","last_undegraded":"2024-09-16T15:38:36.328906+0000","last_fullsized":"2024-09-16T15:38:36.328906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:15.092528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.943983+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.943683+0000","last_became_peered":"2024-09-16T15:38:33.943683+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.572327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328590+0000","last_change":"2024-09-16T15:38:34.943957+0000","last_active":"2024-09-16T15:38:36.328590+0000","last_peered":"2024-09-16T15:38:36.328590+0000","last_clean":"2024-09-16T15:38:36.328590+0000","last_became_active":"2024-09-16T15:38:34.941591+0000","last_became_peered":"2024-09-16T15:38:34.941591+0000","last_unstale":"2024-09-16T15:38:36.328590+0000","last_undegraded":"2024-09-16T15:38:36.328590+0000","last_fullsized":"2024-09-16T15:38:36.328590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328378+0000","last_change":"2024-09-16T15:38:33.954584+0000","last_active":"2024-09-16T15:38:36.328378+0000","last_peered":"2024-09-16T15:38:36.328378+0000","last_clean":"2024-09-16T15:38:36.328378+0000","last_became_active":"2024-09-16T15:38:33.954279+0000","last_became_peered":"2024-09-16T15:38:33.954279+0000","last_unstale":"2024-09-16T15:38:36.328378+0000","last_undegraded":"2024-09-16T15:38:36.328378+0000","last_fullsized":"2024-09-16T15:38:36.328378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933817+0000","last_change":"2024-09-16T15:38:34.949723+0000","last_active":"2024-09-16T15:38:35.933817+0000","last_peered":"2024-09-16T15:38:35.933817+0000","last_clean":"2024-09-16T15:38:35.933817+0000","last_became_active":"2024-09-16T15:38:34.949433+0000","last_became_peered":"2024-09-16T15:38:34.949433+0000","last_unstale":"2024-09-16T15:38:35.933817+0000","last_undegraded":"2024-09-16T15:38:35.933817+0000","last_fullsized":"2024-09-16T15:38:35.933817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:33.866690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329345+0000","last_change":"2024-09-16T15:38:34.961924+0000","last_active":"2024-09-16T15:38:36.329345+0000","last_peered":"2024-09-16T15:38:36.329345+0000","last_clean":"2024-09-16T15:38:36.329345+0000","last_became_active":"2024-09-16T15:38:34.961607+0000","last_became_peered":"2024-09-16T15:38:34.961607+0000","last_unstale":"2024-09-16T15:38:36.329345+0000","last_undegraded":"2024-09-16T15:38:36.329345+0000","last_fullsized":"2024-09-16T15:38:36.329345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:58.529110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328347+0000","last_change":"2024-09-16T15:38:33.955384+0000","last_active":"2024-09-16T15:38:36.328347+0000","last_peered":"2024-09-16T15:38:36.328347+0000","last_clean":"2024-09-16T15:38:36.328347+0000","last_became_active":"2024-09-16T15:38:33.954532+0000","last_became_peered":"2024-09-16T15:38:33.954532+0000","last_unstale":"2024-09-16T15:38:36.328347+0000","last_undegraded":"2024-09-16T15:38:36.328347+0000","last_fullsized":"2024-09-16T15:38:36.328347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:28.577867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328852+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328852+0000","last_peered":"2024-09-16T15:38:36.328852+0000","last_clean":"2024-09-16T15:38:36.328852+0000","last_became_active":"2024-09-16T15:38:34.958217+0000","last_became_peered":"2024-09-16T15:38:34.958217+0000","last_unstale":"2024-09-16T15:38:36.328852+0000","last_undegraded":"2024-09-16T15:38:36.328852+0000","last_fullsized":"2024-09-16T15:38:36.328852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:27.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328469+0000","last_change":"2024-09-16T15:38:33.956064+0000","last_active":"2024-09-16T15:38:36.328469+0000","last_peered":"2024-09-16T15:38:36.328469+0000","last_clean":"2024-09-16T15:38:36.328469+0000","last_became_active":"2024-09-16T15:38:33.955717+0000","last_became_peered":"2024-09-16T15:38:33.955717+0000","last_unstale":"2024-09-16T15:38:36.328469+0000","last_undegraded":"2024-09-16T15:38:36.328469+0000","last_fullsized":"2024-09-16T15:38:36.328469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:12.649947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330233+0000","last_change":"2024-09-16T15:38:34.937110+0000","last_active":"2024-09-16T15:38:36.330233+0000","last_peered":"2024-09-16T15:38:36.330233+0000","last_clean":"2024-09-16T15:38:36.330233+0000","last_became_active":"2024-09-16T15:38:34.936439+0000","last_became_peered":"2024-09-16T15:38:34.936439+0000","last_unstale":"2024-09-16T15:38:36.330233+0000","last_undegraded":"2024-09-16T15:38:36.330233+0000","last_fullsized":"2024-09-16T15:38:36.330233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:49.927493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328261+0000","last_change":"2024-09-16T15:38:33.944467+0000","last_active":"2024-09-16T15:38:36.328261+0000","last_peered":"2024-09-16T15:38:36.328261+0000","last_clean":"2024-09-16T15:38:36.328261+0000","last_became_active":"2024-09-16T15:38:33.944211+0000","last_became_peered":"2024-09-16T15:38:33.944211+0000","last_unstale":"2024-09-16T15:38:36.328261+0000","last_undegraded":"2024-09-16T15:38:36.328261+0000","last_fullsized":"2024-09-16T15:38:36.328261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328305+0000","last_change":"2024-09-16T15:38:34.932456+0000","last_active":"2024-09-16T15:38:36.328305+0000","last_peered":"2024-09-16T15:38:36.328305+0000","last_clean":"2024-09-16T15:38:36.328305+0000","last_became_active":"2024-09-16T15:38:34.931374+0000","last_became_peered":"2024-09-16T15:38:34.931374+0000","last_unstale":"2024-09-16T15:38:36.328305+0000","last_undegraded":"2024-09-16T15:38:36.328305+0000","last_fullsized":"2024-09-16T15:38:36.328305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:26.787621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330346+0000","last_change":"2024-09-16T15:38:33.945170+0000","last_active":"2024-09-16T15:38:36.330346+0000","last_peered":"2024-09-16T15:38:36.330346+0000","last_clean":"2024-09-16T15:38:36.330346+0000","last_became_active":"2024-09-16T15:38:33.944681+0000","last_became_peered":"2024-09-16T15:38:33.944681+0000","last_unstale":"2024-09-16T15:38:36.330346+0000","last_undegraded":"2024-09-16T15:38:36.330346+0000","last_fullsized":"2024-09-16T15:38:36.330346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:30.232249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":245760,"data_stored":81953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":672833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":241664,"data_stored":81987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:38:44.326 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:38:44.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-16T15:38:44.327 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T15:38:44.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.523+0000 7f45bfe00640 1 -- 172.21.15.60:0/2221712820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0161440 msgr2=0x7f45c00b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.523+0000 7f45bfe00640 1 --2- 172.21.15.60:0/2221712820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0161440 0x7f45c00b5370 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f45c00d4ea0 tx=0x7f45b002aaa0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.523+0000 7f45bfe00640 1 -- 172.21.15.60:0/2221712820 shutdown_connections 2024-09-16T15:38:44.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.523+0000 7f45bfe00640 1 --2- 172.21.15.60:0/2221712820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c0161440 0x7f45c00b5370 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.523+0000 7f45bfe00640 1 -- 172.21.15.60:0/2221712820 >> 172.21.15.60:0/2221712820 conn(0x7f45c00d3df0 msgr2=0x7f45c00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:44.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.523+0000 7f45bfe00640 1 -- 172.21.15.60:0/2221712820 shutdown_connections 2024-09-16T15:38:44.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.523+0000 7f45bfe00640 1 -- 172.21.15.60:0/2221712820 wait complete. 2024-09-16T15:38:44.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.525+0000 7f45bfe00640 1 Processor -- start 2024-09-16T15:38:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.525+0000 7f45bfe00640 1 -- start start 2024-09-16T15:38:44.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bfe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c00b34b0 0x7f45c01f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c00b4050 0x7f45c01f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bfe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0161440 0x7f45c00d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bfe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b2210 con 0x7f45c00b4050 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bfe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b2090 con 0x7f45c00b34b0 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bfe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f45c00b2390 con 0x7f45c0161440 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45b6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c00b4050 0x7f45c01f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45b6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c00b4050 0x7f45c01f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35180/0 (socket says 172.21.15.60:35180) 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45b6000640 1 -- 172.21.15.60:0/1866758825 learned_addr learned my addr 172.21.15.60:0/1866758825 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bea00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c00b34b0 0x7f45c01f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bf400640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0161440 0x7f45c00d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45b6000640 1 -- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0161440 msgr2=0x7f45c00d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45b6000640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0161440 0x7f45c00d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45b6000640 1 -- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c00b34b0 msgr2=0x7f45c01f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45b6000640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c00b34b0 0x7f45c01f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45b6000640 1 -- 172.21.15.60:0/1866758825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c00b8e80 con 0x7f45c00b4050 2024-09-16T15:38:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.526+0000 7f45bf400640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0161440 0x7f45c00d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:44.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.527+0000 7f45b6000640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c00b4050 0x7f45c01f4d20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c450 tx=0x7f45a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:44.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.527+0000 7f45b7400640 1 -- 172.21.15.60:0/1866758825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a80052d0 con 0x7f45c00b4050 2024-09-16T15:38:44.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.527+0000 7f45b7400640 1 -- 172.21.15.60:0/1866758825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45a8005470 con 0x7f45c00b4050 2024-09-16T15:38:44.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.527+0000 7f45b7400640 1 -- 172.21.15.60:0/1866758825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8012870 con 0x7f45c00b4050 2024-09-16T15:38:44.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.527+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c00b9160 con 0x7f45c00b4050 2024-09-16T15:38:44.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.527+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c00b9720 con 0x7f45c00b4050 2024-09-16T15:38:44.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.528+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c00b56d0 con 0x7f45c00b4050 2024-09-16T15:38:44.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.529+0000 7f45b7400640 1 -- 172.21.15.60:0/1866758825 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f45a80035f0 con 0x7f45c00b4050 2024-09-16T15:38:44.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.530+0000 7f45b7400640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f459c078850 0x7f459c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.531+0000 7f45b7400640 1 -- 172.21.15.60:0/1866758825 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f45a809a350 con 0x7f45c00b4050 2024-09-16T15:38:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.531+0000 7f45bea00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f459c078850 0x7f459c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.532+0000 7f45bea00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f459c078850 0x7f459c07ad10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45b0035000 tx=0x7f45b0038040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:44.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.534+0000 7f45b7400640 1 -- 172.21.15.60:0/1866758825 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f45c00b56d0 con 0x7f45c00b4050 2024-09-16T15:38:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.728+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f45c00b2810 con 0x7f45c00b4050 2024-09-16T15:38:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.729+0000 7f45b7400640 1 -- 172.21.15.60:0/1866758825 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7f45a8062a50 con 0x7f45c00b4050 2024-09-16T15:38:44.730 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f459c078850 msgr2=0x7f459c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f459c078850 0x7f459c07ad10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f45b0035000 tx=0x7f45b0038040 comp rx=0 tx=0).stop 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c00b4050 msgr2=0x7f45c01f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c00b4050 0x7f45c01f4d20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c450 tx=0x7f45a800c910 comp rx=0 tx=0).stop 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 shutdown_connections 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f459c078850 0x7f459c07ad10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45c0161440 0x7f45c00d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45c00b4050 0x7f45c01f4d20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 --2- 172.21.15.60:0/1866758825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45c00b34b0 0x7f45c01f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 >> 172.21.15.60:0/1866758825 conn(0x7f45c00d3df0 msgr2=0x7f45c015fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 shutdown_connections 2024-09-16T15:38:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.731+0000 7f45bfe00640 1 -- 172.21.15.60:0/1866758825 wait complete. 2024-09-16T15:38:44.748 INFO:teuthology.orchestra.run.smithi060.stdout:[{"version":27,"timestamp":"2024-09-16T15:38:22.359346+0000","name":"","changes":[{"name":"osd.6/osd_mclock_max_capacity_iops_ssd","new_value":"6028.587574"}]}] 2024-09-16T15:38:44.748 INFO:tasks.ceph_manager:config epoch is 27 2024-09-16T15:38:44.748 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T15:38:44.748 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T15:38:44.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T15:38:44.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.908+0000 7fb0d3600640 1 -- 172.21.15.60:0/1264618149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc15e1b0 msgr2=0x7fb0cc0d2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.908+0000 7fb0d3600640 1 --2- 172.21.15.60:0/1264618149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc15e1b0 0x7fb0cc0d2020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0b5a40 tx=0x7fb0c002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.908+0000 7fb0d3600640 1 -- 172.21.15.60:0/1264618149 shutdown_connections 2024-09-16T15:38:44.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.908+0000 7fb0d3600640 1 --2- 172.21.15.60:0/1264618149 >> v2:172.21.15.100:3300/0 conn(0x7fb0cc0b7720 0x7fb0cc0b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.908+0000 7fb0d3600640 1 --2- 172.21.15.60:0/1264618149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc15e1b0 0x7fb0cc0d2020 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.908+0000 7fb0d3600640 1 -- 172.21.15.60:0/1264618149 >> 172.21.15.60:0/1264618149 conn(0x7fb0cc0d3fa0 msgr2=0x7fb0cc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:44.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.908+0000 7fb0d3600640 1 -- 172.21.15.60:0/1264618149 shutdown_connections 2024-09-16T15:38:44.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.908+0000 7fb0d3600640 1 -- 172.21.15.60:0/1264618149 wait complete. 2024-09-16T15:38:44.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.909+0000 7fb0d3600640 1 Processor -- start 2024-09-16T15:38:44.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.910+0000 7fb0d3600640 1 -- start start 2024-09-16T15:38:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.910+0000 7fb0d3600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0cc0b7720 0x7fb0cc1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.910+0000 7fb0d3600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc0d1b70 0x7fb0cc154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.910+0000 7fb0d3600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0cc15e1b0 0x7fb0cc0b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.910+0000 7fb0d3600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc0b8070 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.910+0000 7fb0d3600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc0d2e80 con 0x7fb0cc15e1b0 2024-09-16T15:38:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.910+0000 7fb0d3600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb0cc0b81f0 con 0x7fb0cc0b7720 2024-09-16T15:38:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc0d1b70 0x7fb0cc154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc0d1b70 0x7fb0cc154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35194/0 (socket says 172.21.15.60:35194) 2024-09-16T15:38:44.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 -- 172.21.15.60:0/3521001910 learned_addr learned my addr 172.21.15.60:0/3521001910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:44.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0d0c00640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0cc0b7720 0x7fb0cc1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0d1600640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0cc15e1b0 0x7fb0cc0b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 -- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0cc0b7720 msgr2=0x7fb0cc1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0cc0b7720 0x7fb0cc1611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 -- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0cc15e1b0 msgr2=0x7fb0cc0b8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:44.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0cc15e1b0 0x7fb0cc0b8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 -- 172.21.15.60:0/3521001910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc1f8020 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.911+0000 7fb0cbe00640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc0d1b70 0x7fb0cc154440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c460 tx=0x7fb0b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.912+0000 7fb0c8c00640 1 -- 172.21.15.60:0/3521001910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80052d0 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.912+0000 7fb0c8c00640 1 -- 172.21.15.60:0/3521001910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0b8012630 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.912+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc1f8300 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.912+0000 7fb0c8c00640 1 -- 172.21.15.60:0/3521001910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80054b0 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.912+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc1f88c0 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.913+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0cc0cae70 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.914+0000 7fb0c8c00640 1 -- 172.21.15.60:0/3521001910 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb0b8003610 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.915+0000 7fb0c8c00640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb0ac078850 0x7fb0ac07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:44.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.915+0000 7fb0c8c00640 1 -- 172.21.15.60:0/3521001910 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb0b809a3d0 con 0x7fb0cc0d1b70 2024-09-16T15:38:44.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.915+0000 7fb0d0c00640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb0ac078850 0x7fb0ac07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:44.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.916+0000 7fb0d0c00640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb0ac078850 0x7fb0ac07ad10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0033000 tx=0x7fb0c0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:44.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:44.919+0000 7fb0c8c00640 1 -- 172.21.15.60:0/3521001910 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb0cc0cae70 con 0x7fb0cc0d1b70 2024-09-16T15:38:45.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.167+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb0cc0b7b30 con 0x7fb0cc0d1b70 2024-09-16T15:38:45.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.170+0000 7fb0c8c00640 1 -- 172.21.15.60:0/3521001910 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194348 (secure 0 0 0) 0x7fb0b8062a30 con 0x7fb0cc0d1b70 2024-09-16T15:38:45.173 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:45.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb0ac078850 msgr2=0x7fb0ac07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb0ac078850 0x7fb0ac07ad10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0033000 tx=0x7fb0c0004730 comp rx=0 tx=0).stop 2024-09-16T15:38:45.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc0d1b70 msgr2=0x7fb0cc154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc0d1b70 0x7fb0cc154440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c460 tx=0x7fb0b800c920 comp rx=0 tx=0).stop 2024-09-16T15:38:45.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 shutdown_connections 2024-09-16T15:38:45.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb0ac078850 0x7fb0ac07ad10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0cc15e1b0 0x7fb0cc0b8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0cc0d1b70 0x7fb0cc154440 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 --2- 172.21.15.60:0/3521001910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0cc0b7720 0x7fb0cc1611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 >> 172.21.15.60:0/3521001910 conn(0x7fb0cc0d3fa0 msgr2=0x7fb0cc0d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 shutdown_connections 2024-09-16T15:38:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.174+0000 7fb0d3600640 1 -- 172.21.15.60:0/3521001910 wait complete. 2024-09-16T15:38:45.190 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"flags":0,"active_gid":4115,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2198567224},{"type":"v1","addr":"172.21.15.100:6801","nonce":2198567224}]},"active_addr":"172.21.15.100:6801/2198567224","active_change":"2024-09-16T15:38:13.695929+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4129,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":4072261497}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":4216665037}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1852740290}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1587892015}]}]} 2024-09-16T15:38:45.194 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T15:38:45.194 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T15:38:45.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.353+0000 7fc8a5600640 1 -- 172.21.15.60:0/1609540333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc884004d40 msgr2=0x7fc8840051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.353+0000 7fc8a5600640 1 --2- 172.21.15.60:0/1609540333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc884004d40 0x7fc8840051a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc898001d50 tx=0x7fc884005830 comp rx=0 tx=0).stop 2024-09-16T15:38:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.353+0000 7fc89f400640 1 -- 172.21.15.60:0/1609540333 reap_dead start 2024-09-16T15:38:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.353+0000 7fc8a5600640 1 -- 172.21.15.60:0/1609540333 shutdown_connections 2024-09-16T15:38:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.353+0000 7fc8a5600640 1 -- 172.21.15.60:0/1609540333 >> 172.21.15.60:0/1609540333 conn(0x7fc8a00b1950 msgr2=0x7fc8a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.353+0000 7fc8a5600640 1 -- 172.21.15.60:0/1609540333 shutdown_connections 2024-09-16T15:38:45.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.353+0000 7fc8a5600640 1 -- 172.21.15.60:0/1609540333 wait complete. 2024-09-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.355+0000 7fc8a5600640 1 Processor -- start 2024-09-16T15:38:45.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.355+0000 7fc8a5600640 1 -- start start 2024-09-16T15:38:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc8a5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884004d40 0x7fc8a0159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:45.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc8a5600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00cf7f0 0x7fc8a01f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc8a5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a009aa00 0x7fc8a0073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc8a5600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a006ef50 con 0x7fc8a009aa00 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc8a5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a006fdb0 con 0x7fc884004d40 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc8a5600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a006f0d0 con 0x7fc8a00cf7f0 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc89f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a009aa00 0x7fc8a0073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc89f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a009aa00 0x7fc8a0073ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35212/0 (socket says 172.21.15.60:35212) 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc89f400640 1 -- 172.21.15.60:0/1364833468 learned_addr learned my addr 172.21.15.60:0/1364833468 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc896000640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00cf7f0 0x7fc8a01f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.356+0000 7fc89ea00640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884004d40 0x7fc8a0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc89f400640 1 -- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00cf7f0 msgr2=0x7fc8a01f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc89f400640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00cf7f0 0x7fc8a01f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc89f400640 1 -- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884004d40 msgr2=0x7fc8a0159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc89f400640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884004d40 0x7fc8a0159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc89f400640 1 -- 172.21.15.60:0/1364833468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a01f5960 con 0x7fc8a009aa00 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc89f400640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a009aa00 0x7fc8a0073ad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc898001e70 tx=0x7fc898001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc897400640 1 -- 172.21.15.60:0/1364833468 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8980060b0 con 0x7fc8a009aa00 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc897400640 1 -- 172.21.15.60:0/1364833468 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc898006250 con 0x7fc8a009aa00 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc897400640 1 -- 172.21.15.60:0/1364833468 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8980066a0 con 0x7fc8a009aa00 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.357+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a00cfc70 con 0x7fc8a009aa00 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.358+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a01f53a0 con 0x7fc8a009aa00 2024-09-16T15:38:45.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.359+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a00732a0 con 0x7fc8a009aa00 2024-09-16T15:38:45.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.360+0000 7fc897400640 1 -- 172.21.15.60:0/1364833468 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc898033480 con 0x7fc8a009aa00 2024-09-16T15:38:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.361+0000 7fc897400640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc87c078850 0x7fc87c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.361+0000 7fc897400640 1 -- 172.21.15.60:0/1364833468 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc8980ba0f0 con 0x7fc8a009aa00 2024-09-16T15:38:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.361+0000 7fc89ea00640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc87c078850 0x7fc87c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.362+0000 7fc89ea00640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc87c078850 0x7fc87c07ad10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc8a009dc50 tx=0x7fc890005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.365+0000 7fc897400640 1 -- 172.21.15.60:0/1364833468 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc898046030 con 0x7fc8a009aa00 2024-09-16T15:38:45.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.564+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8a00ae0a0 con 0x7fc8a009aa00 2024-09-16T15:38:45.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.565+0000 7fc897400640 1 -- 172.21.15.60:0/1364833468 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7fc898089e20 con 0x7fc8a009aa00 2024-09-16T15:38:45.566 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:45.566 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:35.924959+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc87c078850 msgr2=0x7fc87c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc87c078850 0x7fc87c07ad10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc8a009dc50 tx=0x7fc890005f90 comp rx=0 tx=0).stop 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a009aa00 msgr2=0x7fc8a0073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a009aa00 0x7fc8a0073ad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc898001e70 tx=0x7fc898001a30 comp rx=0 tx=0).stop 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 shutdown_connections 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc87c078850 0x7fc87c07ad10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a009aa00 0x7fc8a0073ad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a00cf7f0 0x7fc8a01f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 --2- 172.21.15.60:0/1364833468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc884004d40 0x7fc8a0159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.567+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 >> 172.21.15.60:0/1364833468 conn(0x7fc8a00b1950 msgr2=0x7fc8a006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.568+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 shutdown_connections 2024-09-16T15:38:45.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.568+0000 7fc8a5600640 1 -- 172.21.15.60:0/1364833468 wait complete. 2024-09-16T15:38:45.585 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T15:38:45.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.743+0000 7f9426000640 1 -- 172.21.15.60:0/2762156265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 msgr2=0x7f9420074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.743+0000 7f9426000640 1 --2- 172.21.15.60:0/2762156265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 0x7f9420074420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9420075130 tx=0x7f941802aa30 comp rx=0 tx=0).stop 2024-09-16T15:38:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.743+0000 7f9426000640 1 -- 172.21.15.60:0/2762156265 shutdown_connections 2024-09-16T15:38:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.743+0000 7f9426000640 1 --2- 172.21.15.60:0/2762156265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 0x7f9420074420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.743+0000 7f9426000640 1 --2- 172.21.15.60:0/2762156265 >> v2:172.21.15.100:3300/0 conn(0x7f942015f5a0 0x7f94200b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.743+0000 7f9426000640 1 -- 172.21.15.60:0/2762156265 >> 172.21.15.60:0/2762156265 conn(0x7f94200b1950 msgr2=0x7f94200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.743+0000 7f9426000640 1 -- 172.21.15.60:0/2762156265 shutdown_connections 2024-09-16T15:38:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.743+0000 7f9426000640 1 -- 172.21.15.60:0/2762156265 wait complete. 2024-09-16T15:38:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.745+0000 7f9426000640 1 Processor -- start 2024-09-16T15:38:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.745+0000 7f9426000640 1 -- start start 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f9426000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 0x7f94201534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f9426000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94200ccf00 0x7f9420156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f9426000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942015f5a0 0x7f9420157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f9426000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f942006f2e0 con 0x7f9420073fb0 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f9426000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f942006f160 con 0x7f942015f5a0 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f9426000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f942006f460 con 0x7f94200ccf00 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f941f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 0x7f94201534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f941f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 0x7f94201534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35230/0 (socket says 172.21.15.60:35230) 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.746+0000 7f941f400640 1 -- 172.21.15.60:0/551250549 learned_addr learned my addr 172.21.15.60:0/551250549 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f9416a00640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94200ccf00 0x7f9420156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f941fe00640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942015f5a0 0x7f9420157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:45.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f941f400640 1 -- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94200ccf00 msgr2=0x7f9420156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f941f400640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94200ccf00 0x7f9420156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f941f400640 1 -- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942015f5a0 msgr2=0x7f9420157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f941f400640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942015f5a0 0x7f9420157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f941f400640 1 -- 172.21.15.60:0/551250549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9420153ff0 con 0x7f9420073fb0 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f9416a00640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94200ccf00 0x7f9420156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f941f400640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 0x7f94201534e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f941000e790 tx=0x7f941000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f9417e00640 1 -- 172.21.15.60:0/551250549 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941000eea0 con 0x7f9420073fb0 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.747+0000 7f9417e00640 1 -- 172.21.15.60:0/551250549 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f941000cdb0 con 0x7f9420073fb0 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.748+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9420153d10 con 0x7f9420073fb0 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.748+0000 7f9417e00640 1 -- 172.21.15.60:0/551250549 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410010690 con 0x7f9420073fb0 2024-09-16T15:38:45.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.748+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9420158f20 con 0x7f9420073fb0 2024-09-16T15:38:45.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.749+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942006f9f0 con 0x7f9420073fb0 2024-09-16T15:38:45.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.750+0000 7f9417e00640 1 -- 172.21.15.60:0/551250549 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9410010830 con 0x7f9420073fb0 2024-09-16T15:38:45.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.751+0000 7f9417e00640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f93fc078850 0x7f93fc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:45.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.751+0000 7f9417e00640 1 -- 172.21.15.60:0/551250549 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f941009b300 con 0x7f9420073fb0 2024-09-16T15:38:45.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.751+0000 7f9416a00640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f93fc078850 0x7f93fc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:45.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.752+0000 7f9416a00640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f93fc078850 0x7f93fc07ad10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9420070ba0 tx=0x7f9408009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:45.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.755+0000 7f9417e00640 1 -- 172.21.15.60:0/551250549 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9410063280 con 0x7f9420073fb0 2024-09-16T15:38:45.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.950+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94200b26e0 con 0x7f9420073fb0 2024-09-16T15:38:45.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.952+0000 7f9417e00640 1 -- 172.21.15.60:0/551250549 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7f9410067cb0 con 0x7f9420073fb0 2024-09-16T15:38:45.953 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:45.953 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:35.924959+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f93fc078850 msgr2=0x7f93fc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f93fc078850 0x7f93fc07ad10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9420070ba0 tx=0x7f9408009040 comp rx=0 tx=0).stop 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 msgr2=0x7f94201534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 0x7f94201534e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f941000e790 tx=0x7f941000ec50 comp rx=0 tx=0).stop 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 shutdown_connections 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f93fc078850 0x7f93fc07ad10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f942015f5a0 0x7f9420157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94200ccf00 0x7f9420156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 --2- 172.21.15.60:0/551250549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9420073fb0 0x7f94201534e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 >> 172.21.15.60:0/551250549 conn(0x7f94200b1950 msgr2=0x7f94200ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 shutdown_connections 2024-09-16T15:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:45.954+0000 7f9426000640 1 -- 172.21.15.60:0/551250549 wait complete. 2024-09-16T15:38:45.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T15:38:45.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T15:38:45.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T15:38:45.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T15:38:45.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T15:38:45.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T15:38:45.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T15:38:45.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T15:38:46.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.262+0000 7fbe03e00640 1 -- 172.21.15.60:0/2538301029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbde80039b0 msgr2=0x7fbde8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.262+0000 7fbe03e00640 1 --2- 172.21.15.60:0/2538301029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbde80039b0 0x7fbde8003db0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009d10 tx=0x7fbde8005830 comp rx=0 tx=0).stop 2024-09-16T15:38:46.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.262+0000 7fbe03e00640 1 -- 172.21.15.60:0/2538301029 shutdown_connections 2024-09-16T15:38:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.262+0000 7fbe03e00640 1 --2- 172.21.15.60:0/2538301029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbde80039b0 0x7fbde8003db0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.262+0000 7fbe03e00640 1 -- 172.21.15.60:0/2538301029 >> 172.21.15.60:0/2538301029 conn(0x7fbdfc01d770 msgr2=0x7fbdfc054560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.262+0000 7fbe03e00640 1 -- 172.21.15.60:0/2538301029 shutdown_connections 2024-09-16T15:38:46.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.263+0000 7fbe03e00640 1 -- 172.21.15.60:0/2538301029 wait complete. 2024-09-16T15:38:46.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.265+0000 7fe0c8a00640 1 -- 172.21.15.60:0/2699856958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc03ab10 msgr2=0x7fe0bc037940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.266+0000 7fbe03e00640 1 Processor -- start 2024-09-16T15:38:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.265+0000 7fe0c8a00640 1 --2- 172.21.15.60:0/2699856958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc03ab10 0x7fe0bc037940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009860 tx=0x7fe0b802a460 comp rx=0 tx=0).stop 2024-09-16T15:38:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.266+0000 7fe0c8a00640 1 -- 172.21.15.60:0/2699856958 shutdown_connections 2024-09-16T15:38:46.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.266+0000 7fe0c8a00640 1 --2- 172.21.15.60:0/2699856958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc03ab10 0x7fe0bc037940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.266+0000 7fe0c8a00640 1 -- 172.21.15.60:0/2699856958 >> 172.21.15.60:0/2699856958 conn(0x7fe0bc055540 msgr2=0x7fe0bc055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.266+0000 7fbe03e00640 1 -- start start 2024-09-16T15:38:46.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.266+0000 7fe0c8a00640 1 -- 172.21.15.60:0/2699856958 shutdown_connections 2024-09-16T15:38:46.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.268+0000 7fbe03e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbde80039b0 0x7fbdfc178460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.268+0000 7fbe03e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdfc176600 0x7fbdfc04da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.268+0000 7fbe03e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdfc04df90 0x7fbdfc178a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.268+0000 7fbe03e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc055290 con 0x7fbde80039b0 2024-09-16T15:38:46.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.268+0000 7fbe03e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc0e6740 con 0x7fbdfc04df90 2024-09-16T15:38:46.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.268+0000 7fbe03e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbdfc0cb230 con 0x7fbdfc176600 2024-09-16T15:38:46.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.267+0000 7fe0c8a00640 1 -- 172.21.15.60:0/2699856958 wait complete. 2024-09-16T15:38:46.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.269+0000 7fbe03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdfc04df90 0x7fbdfc178a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.269+0000 7fbe03400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdfc04df90 0x7fbdfc178a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56854/0 (socket says 172.21.15.60:56854) 2024-09-16T15:38:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.269+0000 7fbe03400640 1 -- 172.21.15.60:0/2865427669 learned_addr learned my addr 172.21.15.60:0/2865427669 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.269+0000 7fbe02a00640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbde80039b0 0x7fbdfc178460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.269+0000 7fbe02000640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdfc176600 0x7fbdfc04da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fe0c8a00640 1 Processor -- start 2024-09-16T15:38:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fd527400640 1 -- 172.21.15.60:0/3404215271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd504004de0 msgr2=0x7fd5040071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fd527400640 1 --2- 172.21.15.60:0/3404215271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd504004de0 0x7fd5040071d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c610 tx=0x7fd504003810 comp rx=0 tx=0).stop 2024-09-16T15:38:46.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fbe02000640 1 -- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdfc04df90 msgr2=0x7fbdfc178a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fbe02000640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdfc04df90 0x7fbdfc178a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fbe02000640 1 -- 172.21.15.60:0/2865427669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbde80039b0 msgr2=0x7fbdfc178460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fbe02000640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbde80039b0 0x7fbdfc178460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fbe02000640 1 -- 172.21.15.60:0/2865427669 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc04e420 con 0x7fbdfc176600 2024-09-16T15:38:46.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fd527400640 1 -- 172.21.15.60:0/3404215271 shutdown_connections 2024-09-16T15:38:46.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fd527400640 1 --2- 172.21.15.60:0/3404215271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd504004de0 0x7fd5040071d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fd527400640 1 -- 172.21.15.60:0/3404215271 >> 172.21.15.60:0/3404215271 conn(0x7fd52006cd50 msgr2=0x7fd52006d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fd527400640 1 -- 172.21.15.60:0/3404215271 shutdown_connections 2024-09-16T15:38:46.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.270+0000 7fe0c8a00640 1 -- start start 2024-09-16T15:38:46.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fe0c8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0bc03ab10 0x7fe0bc009fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fe0c8a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc0e2a60 0x7fe0bc051d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fe0c8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0bc0e39e0 0x7fe0bc187210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fe0c8a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc0539f0 con 0x7fe0bc03ab10 2024-09-16T15:38:46.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fbe02000640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdfc176600 0x7fbdfc04da50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc004180 tx=0x7fbdf400c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fe0c8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc053870 con 0x7fe0bc0e39e0 2024-09-16T15:38:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.271+0000 7fe0c8a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc053b70 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fe0c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0bc0e39e0 0x7fe0bc187210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fe0c2a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc0e2a60 0x7fe0bc051d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fe0c2a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc0e2a60 0x7fe0bc051d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:53036/0 (socket says 172.21.15.60:53036) 2024-09-16T15:38:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fe0c3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0bc03ab10 0x7fe0bc009fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fe0c2a00640 1 -- 172.21.15.60:0/780129974 learned_addr learned my addr 172.21.15.60:0/780129974 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fd527400640 1 -- 172.21.15.60:0/3404215271 wait complete. 2024-09-16T15:38:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4017070 con 0x7fbdfc176600 2024-09-16T15:38:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdf40035f0 con 0x7fbdfc176600 2024-09-16T15:38:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4003a40 con 0x7fbdfc176600 2024-09-16T15:38:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.272+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdfc0099b0 con 0x7fbdfc176600 2024-09-16T15:38:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdfc009f70 con 0x7fbdfc176600 2024-09-16T15:38:46.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7eff15a00640 1 -- 172.21.15.60:0/1518721977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff10162270 msgr2=0x7eff1006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7fe0c2a00640 1 -- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0bc0e39e0 msgr2=0x7fe0bc187210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7fe0c2a00640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0bc0e39e0 0x7fe0bc187210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7fe0c2a00640 1 -- 172.21.15.60:0/780129974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0bc03ab10 msgr2=0x7fe0bc009fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7fe0c2a00640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0bc03ab10 0x7fe0bc009fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7fe0c2a00640 1 -- 172.21.15.60:0/780129974 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc00cb30 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7eff15a00640 1 --2- 172.21.15.60:0/1518721977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff10162270 0x7eff1006e970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff08005fd0 tx=0x7eff0802a830 comp rx=0 tx=0).stop 2024-09-16T15:38:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7eff15a00640 1 -- 172.21.15.60:0/1518721977 shutdown_connections 2024-09-16T15:38:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7eff15a00640 1 --2- 172.21.15.60:0/1518721977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff10162270 0x7eff1006e970 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7eff15a00640 1 --2- 172.21.15.60:0/1518721977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1015bbd0 0x7eff100cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.273+0000 7eff15a00640 1 -- 172.21.15.60:0/1518721977 >> 172.21.15.60:0/1518721977 conn(0x7eff100b1950 msgr2=0x7eff100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.274+0000 7fe0c2a00640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc0e2a60 0x7fe0bc051d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0b802aa50 tx=0x7fe0b8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.274+0000 7eff15a00640 1 -- 172.21.15.60:0/1518721977 shutdown_connections 2024-09-16T15:38:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.274+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbddc001030 con 0x7fbdfc176600 2024-09-16T15:38:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.274+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8037040 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.274+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0b8040070 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.274+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8005c90 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.274+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc00ce10 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.274+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc00d3d0 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.275+0000 7eff15a00640 1 -- 172.21.15.60:0/1518721977 wait complete. 2024-09-16T15:38:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.275+0000 7fd663a00640 1 -- 172.21.15.60:0/3988556107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0cea10 msgr2=0x7fd65c0ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.275+0000 7fd663a00640 1 --2- 172.21.15.60:0/3988556107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0cea10 0x7fd65c0ceec0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd65c08c380 tx=0x7fd65402aa10 comp rx=0 tx=0).stop 2024-09-16T15:38:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.275+0000 7fd527400640 1 Processor -- start 2024-09-16T15:38:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.275+0000 7fd663a00640 1 -- 172.21.15.60:0/3988556107 shutdown_connections 2024-09-16T15:38:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.275+0000 7fd663a00640 1 --2- 172.21.15.60:0/3988556107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0cea10 0x7fd65c0ceec0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.276+0000 7fd663a00640 1 --2- 172.21.15.60:0/3988556107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d2050 0x7fd65c0d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.276+0000 7fd663a00640 1 --2- 172.21.15.60:0/3988556107 >> v2:172.21.15.100:3300/0 conn(0x7fd65c0d09c0 0x7fd65c0d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.276+0000 7fd663a00640 1 -- 172.21.15.60:0/3988556107 >> 172.21.15.60:0/3988556107 conn(0x7fd65c0d3fe0 msgr2=0x7fd65c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.276+0000 7fd527400640 1 -- start start 2024-09-16T15:38:46.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.276+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0bc186e30 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.276+0000 7fd663a00640 1 -- 172.21.15.60:0/3988556107 shutdown_connections 2024-09-16T15:38:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.276+0000 7fd663a00640 1 -- 172.21.15.60:0/3988556107 wait complete. 2024-09-16T15:38:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.277+0000 7fd527400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd504004de0 0x7fd52015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.277+0000 7fd527400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5201558d0 0x7fd520152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.277+0000 7fd527400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd520154220 0x7fd5201f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.277+0000 7fd527400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd520070870 con 0x7fd5201558d0 2024-09-16T15:38:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.277+0000 7fd527400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5200706f0 con 0x7fd504004de0 2024-09-16T15:38:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.277+0000 7fd527400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd5200cfd50 con 0x7fd520154220 2024-09-16T15:38:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fa5a9800640 1 -- 172.21.15.60:0/3256782747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5a406ff60 msgr2=0x7fa5a40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7f10c5400640 1 -- 172.21.15.60:0/731680060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 msgr2=0x7f10a400bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7f10c5400640 1 --2- 172.21.15.60:0/731680060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 0x7f10a400bda0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f10b800adc0 tx=0x7f10a4005830 comp rx=0 tx=0).stop 2024-09-16T15:38:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fa5a9800640 1 --2- 172.21.15.60:0/3256782747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5a406ff60 0x7fa5a40b4610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa59400b9f0 tx=0x7fa59402c230 comp rx=0 tx=0).stop 2024-09-16T15:38:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7f10c5400640 1 -- 172.21.15.60:0/731680060 shutdown_connections 2024-09-16T15:38:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7f10c5400640 1 --2- 172.21.15.60:0/731680060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 0x7f10a400bda0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7f10c5400640 1 -- 172.21.15.60:0/731680060 >> 172.21.15.60:0/731680060 conn(0x7f10c00b1950 msgr2=0x7f10c00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7f10c5400640 1 -- 172.21.15.60:0/731680060 shutdown_connections 2024-09-16T15:38:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fa5a9800640 1 -- 172.21.15.60:0/3256782747 shutdown_connections 2024-09-16T15:38:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fa5a9800640 1 --2- 172.21.15.60:0/3256782747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5a406ff60 0x7fa5a40b4610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fa5a9800640 1 -- 172.21.15.60:0/3256782747 >> 172.21.15.60:0/3256782747 conn(0x7fa5a40b1950 msgr2=0x7fa5a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7eff15a00640 1 Processor -- start 2024-09-16T15:38:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fa5a9800640 1 -- 172.21.15.60:0/3256782747 shutdown_connections 2024-09-16T15:38:46.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fd663a00640 1 Processor -- start 2024-09-16T15:38:46.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7eff15a00640 1 -- start start 2024-09-16T15:38:46.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fd663a00640 1 -- start start 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fd524a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd504004de0 0x7fd52015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fd524a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd504004de0 0x7fd52015c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56890/0 (socket says 172.21.15.60:56890) 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fd524a00640 1 -- 172.21.15.60:0/2954873284 learned_addr learned my addr 172.21.15.60:0/2954873284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fd524a00640 1 -- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd520154220 msgr2=0x7fd5201f3a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fd524a00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd520154220 0x7fd5201f3a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7fd524a00640 1 -- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5201558d0 msgr2=0x7fd520152d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fd51fe00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5201558d0 0x7fd520152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fd524a00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5201558d0 0x7fd520152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fd524a00640 1 -- 172.21.15.60:0/2954873284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd520153280 con 0x7fd504004de0 2024-09-16T15:38:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fd51fe00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5201558d0 0x7fd520152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:46.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fa5a9800640 1 -- 172.21.15.60:0/3256782747 wait complete. 2024-09-16T15:38:46.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7fd524a00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd504004de0 0x7fd52015c840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd51802b4e0 tx=0x7fd51802b5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7eff15a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff100cd550 0x7eff10161360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7eff15a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff1015bbd0 0x7eff101fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7eff15a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff10162270 0x7eff1009cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7eff15a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7eff1006f240 con 0x7eff1015bbd0 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7eff15a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff1006f0c0 con 0x7eff100cd550 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7eff15a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff1006f3c0 con 0x7eff10162270 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518048070 con 0x7fd504004de0 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd51802ba40 con 0x7fd504004de0 2024-09-16T15:38:46.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518043850 con 0x7fd504004de0 2024-09-16T15:38:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fbdf4003be0 con 0x7fbdfc176600 2024-09-16T15:38:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fbdfaa00640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbde4078880 0x7fbde407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7eff0f400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff10162270 0x7eff1009cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7eff0f400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff10162270 0x7eff1009cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:53066/0 (socket says 172.21.15.60:53066) 2024-09-16T15:38:46.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7eff0f400640 1 -- 172.21.15.60:0/525171848 learned_addr learned my addr 172.21.15.60:0/525171848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7eff0e000640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff1015bbd0 0x7eff101fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.278+0000 7f10c5400640 1 -- 172.21.15.60:0/731680060 wait complete. 2024-09-16T15:38:46.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7f10c5400640 1 Processor -- start 2024-09-16T15:38:46.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbdf4099ac0 con 0x7fbdfc176600 2024-09-16T15:38:46.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fd663a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0cea10 0x7fd65c0a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fd663a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d09c0 0x7fd65c1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.279+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe0b80326a0 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7f10c5400640 1 -- start start 2024-09-16T15:38:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fbe02a00640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbde4078880 0x7fbde407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fbdfaa00640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7fbde4080aa0 0x7fbde4082ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 --> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbde4083570 con 0x7fbde4080aa0 2024-09-16T15:38:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbdf4099eb0 con 0x7fbdfc176600 2024-09-16T15:38:46.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fa5a9800640 1 Processor -- start 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fd663a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0d2050 0x7fd65c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fd663a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0cf790 con 0x7fd65c0d2050 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fd663a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0cf610 con 0x7fd65c0cea10 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7fd663a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c0cf910 con 0x7fd65c0d09c0 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fd661000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0cea10 0x7fd65c0a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fd65be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d09c0 0x7fd65c1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fd661a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0d2050 0x7fd65c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fd661000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0cea10 0x7fd65c0a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56906/0 (socket says 172.21.15.60:56906) 2024-09-16T15:38:46.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fd661000640 1 -- 172.21.15.60:0/1136849526 learned_addr learned my addr 172.21.15.60:0/1136849526 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fa5a9800640 1 -- start start 2024-09-16T15:38:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fd661000640 1 -- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d09c0 msgr2=0x7fd65c1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fd661000640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d09c0 0x7fd65c1523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.282+0000 7fd661000640 1 -- 172.21.15.60:0/1136849526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0d2050 msgr2=0x7fd65c0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7eff0f400640 1 -- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff100cd550 msgr2=0x7eff10161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.281+0000 7eff0ea00640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff100cd550 0x7eff10161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.280+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52015cec0 con 0x7fd504004de0 2024-09-16T15:38:46.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7fd661000640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0d2050 0x7fd65c0cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7fd661000640 1 -- 172.21.15.60:0/1136849526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c09ee50 con 0x7fd65c0cea10 2024-09-16T15:38:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5201550d0 con 0x7fd504004de0 2024-09-16T15:38:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7eff0f400640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff100cd550 0x7eff10161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7eff0f400640 1 -- 172.21.15.60:0/525171848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff1015bbd0 msgr2=0x7eff101fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7eff0f400640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff1015bbd0 0x7eff101fc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7eff0f400640 1 -- 172.21.15.60:0/525171848 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff101fd250 con 0x7eff10162270 2024-09-16T15:38:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7eff0e000640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff1015bbd0 0x7eff101fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:46.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7fbe03400640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7fbde4080aa0 0x7fbde4082ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7fd661a00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0d2050 0x7fd65c0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:46.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7eff0f400640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff10162270 0x7eff1009cd40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7eff08035040 tx=0x7eff08032a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7fd661000640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0cea10 0x7fd65c0a1750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd65000e9f0 tx=0x7fd65000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff08037050 con 0x7eff10162270 2024-09-16T15:38:46.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff08043070 con 0x7eff10162270 2024-09-16T15:38:46.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff080042a0 con 0x7eff10162270 2024-09-16T15:38:46.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7eff15a00640 1 -- 172.21.15.60:0/525171848 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff101fd530 con 0x7eff10162270 2024-09-16T15:38:46.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7fa5a9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5a406ff60 0x7fa5a41f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7eff15a00640 1 -- 172.21.15.60:0/525171848 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7eff101618a0 con 0x7eff10162270 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.283+0000 7f10c5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 0x7f10c01f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10c5400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10c01fcbb0 0x7f10c0140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10c5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c01424d0 0x7f10c0142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10c5400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f10c00b27d0 con 0x7f10c01424d0 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10c5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10c00b2650 con 0x7f10a40099b0 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10c5400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f10c0144a40 con 0x7f10c01fcbb0 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10be800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 0x7f10c01f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10bf200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c01424d0 0x7f10c0142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10be800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 0x7f10c01f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56918/0 (socket says 172.21.15.60:56918) 2024-09-16T15:38:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7f10be800640 1 -- 172.21.15.60:0/3606770394 learned_addr learned my addr 172.21.15.60:0/3606770394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7fa5a9800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5a4070920 0x7fa5a40b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7fa5a9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a41438b0 0x7fa5a40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7fa5a9800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40b4e40 con 0x7fa5a406ff60 2024-09-16T15:38:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7fa5a9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40cc5f0 con 0x7fa5a41438b0 2024-09-16T15:38:46.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.284+0000 7fa5a9800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40af7f0 con 0x7fa5a4070920 2024-09-16T15:38:46.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fe0b7400640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0a4078920 0x7fe0a407ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd520155d30 con 0x7fd504004de0 2024-09-16T15:38:46.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a41438b0 0x7fa5a40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fa5a3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a41438b0 0x7fa5a40b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56920/0 (socket says 172.21.15.60:56920) 2024-09-16T15:38:46.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fa5a3400640 1 -- 172.21.15.60:0/3950804476 learned_addr learned my addr 172.21.15.60:0/3950804476 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fa5a2000640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5a4070920 0x7fa5a40b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7f10bde00640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10c01fcbb0 0x7f10c0140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7f10be800640 1 -- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10c01fcbb0 msgr2=0x7f10c0140600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7f10be800640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10c01fcbb0 0x7f10c0140600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7f10be800640 1 -- 172.21.15.60:0/3606770394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c01424d0 msgr2=0x7f10c0142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7f10be800640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10c01424d0 0x7f10c0142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7f10be800640 1 -- 172.21.15.60:0/3606770394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c009acf0 con 0x7f10a40099b0 2024-09-16T15:38:46.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7f10bde00640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10c01fcbb0 0x7f10c0140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:46.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65000ccc0 con 0x7fd65c0cea10 2024-09-16T15:38:46.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd650010430 con 0x7fd65c0cea10 2024-09-16T15:38:46.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650010840 con 0x7fd65c0cea10 2024-09-16T15:38:46.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fbe02a00640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbde4078880 0x7fbde407ad40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0034000 tx=0x7fbdf00046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65c09e090 con 0x7fd65c0cea10 2024-09-16T15:38:46.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65c09e650 con 0x7fd65c0cea10 2024-09-16T15:38:46.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe0b80ba780 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fa5a3400640 1 -- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5a4070920 msgr2=0x7fa5a40b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fa5a3400640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5a4070920 0x7fa5a40b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fa5a3400640 1 -- 172.21.15.60:0/3950804476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5a406ff60 msgr2=0x7fa5a41f9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.285+0000 7fa5a3400640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5a406ff60 0x7fa5a41f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7fa5a3400640 1 -- 172.21.15.60:0/3950804476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40afad0 con 0x7fa5a41438b0 2024-09-16T15:38:46.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7f10be800640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 0x7f10c01f8c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f10ac004d90 tx=0x7f10ac001630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7fe0c3400640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0a4078920 0x7fe0a407ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7fe0b7400640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7fe0a4080b40 0x7fe0a4082f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 --> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0b8007e80 con 0x7fe0a4080b40 2024-09-16T15:38:46.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe0b80c1050 con 0x7fe0bc0e2a60 2024-09-16T15:38:46.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7fe0c3e00640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7fe0a4080b40 0x7fe0a4082f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7fa5a3400640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a41438b0 0x7fa5a40b12e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa59400b990 tx=0x7fa594003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7fbe03400640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7fbde4080aa0 0x7fbde4082ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.286+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac001d40 con 0x7f10a40099b0 2024-09-16T15:38:46.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10ac0108c0 con 0x7f10a40099b0 2024-09-16T15:38:46.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac010cd0 con 0x7f10a40099b0 2024-09-16T15:38:46.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c009afd0 con 0x7f10a40099b0 2024-09-16T15:38:46.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594032040 con 0x7fa5a41438b0 2024-09-16T15:38:46.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5940368a0 con 0x7fa5a41438b0 2024-09-16T15:38:46.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594032040 con 0x7fa5a41438b0 2024-09-16T15:38:46.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd518006070 con 0x7fd504004de0 2024-09-16T15:38:46.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c009b590 con 0x7f10a40099b0 2024-09-16T15:38:46.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.288+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a40afdb0 con 0x7fa5a41438b0 2024-09-16T15:38:46.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.288+0000 7fe0c3400640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0a4078920 0x7fe0a407ade0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4051c90 tx=0x7fe0c408bf90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.288+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a40b0370 con 0x7fa5a41438b0 2024-09-16T15:38:46.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.288+0000 7eff15a00640 1 -- 172.21.15.60:0/525171848 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff1006f890 con 0x7eff10162270 2024-09-16T15:38:46.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.287+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd65c09de40 con 0x7fd65c0cea10 2024-09-16T15:38:46.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.288+0000 7fe0c3e00640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7fe0a4080b40 0x7fe0a4082f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.289+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd6500109e0 con 0x7fd65c0cea10 2024-09-16T15:38:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.289+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5a40ccaa0 con 0x7fa5a41438b0 2024-09-16T15:38:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.289+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 <== osd.0 v2:172.21.15.60:6816/841472238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbde4083570 con 0x7fbde4080aa0 2024-09-16T15:38:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.289+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10c00b3030 con 0x7f10a40099b0 2024-09-16T15:38:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.290+0000 7fd658c00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd63c078920 0x7fd63c07ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.290+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 <== osd.1 v2:172.21.15.60:6800/910651751 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0b8007e80 con 0x7fe0a4080b40 2024-09-16T15:38:46.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.290+0000 7fd51cc00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd500078880 0x7fd50007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.290+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd65009b2b0 con 0x7fd65c0cea10 2024-09-16T15:38:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.290+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd5180c2840 con 0x7fd504004de0 2024-09-16T15:38:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.289+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7eff08004440 con 0x7eff10162270 2024-09-16T15:38:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.290+0000 7fd658c00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7fd63c080ac0 0x7fd63c082ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.291+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 --> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd65001ee50 con 0x7fd63c080ac0 2024-09-16T15:38:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.291+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd6500a2050 con 0x7fd65c0cea10 2024-09-16T15:38:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.291+0000 7fd51cc00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fd500080aa0 0x7fd500082ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.291+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 --> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd518008e80 con 0x7fd500080aa0 2024-09-16T15:38:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.291+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd5180c72a0 con 0x7fd504004de0 2024-09-16T15:38:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.291+0000 7fd525400640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fd500080aa0 0x7fd500082ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.291+0000 7fd51fe00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd500078880 0x7fd50007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.291+0000 7eff06a00640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efef0078920 0x7efef007ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7fd65be00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd63c078920 0x7fd63c07ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa594046060 con 0x7fa5a41438b0 2024-09-16T15:38:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7eff080bb690 con 0x7eff10162270 2024-09-16T15:38:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7eff0ea00640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efef0078920 0x7efef007ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7eff06a00640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7efef0080b40 0x7efef0082f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 --> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff08045e80 con 0x7efef0080b40 2024-09-16T15:38:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7eff080c2050 con 0x7eff10162270 2024-09-16T15:38:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7fd525400640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fd500080aa0 0x7fd500082ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.292+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f10ac01c470 con 0x7f10a40099b0 2024-09-16T15:38:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.293+0000 7fd661a00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7fd63c080ac0 0x7fd63c082ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7fd661a00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7fd63c080ac0 0x7fd63c082ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 <== osd.2 v2:172.21.15.60:6808/3414864047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd518008e80 con 0x7fd500080aa0 2024-09-16T15:38:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.293+0000 7fa59aa00640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa584078c50 0x7fa58407b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa5940365b0 con 0x7fa5a41438b0 2024-09-16T15:38:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7fa59aa00640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7fa584080df0 0x7fa584083210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 --> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa594001d30 con 0x7fa584080df0 2024-09-16T15:38:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa5940bae40 con 0x7fa5a41438b0 2024-09-16T15:38:46.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7eff0e000640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7efef0080b40 0x7efef0082f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7fd65be00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd63c078920 0x7fd63c07ade0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0cd510 tx=0x7fd64800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7fd51fe00640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd500078880 0x7fd50007ad40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5200a9510 tx=0x7fd50c010040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7f10b6a00640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10a0078920 0x7f10a007ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.294+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f10ac09b1f0 con 0x7f10a40099b0 2024-09-16T15:38:46.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7f10b6a00640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f10a0080b40 0x7f10a0082f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 --> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10ac00de50 con 0x7f10a0080b40 2024-09-16T15:38:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f10ac0a2050 con 0x7f10a40099b0 2024-09-16T15:38:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7fa5a2000640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7fa584080df0 0x7fa584083210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7fa5a2a00640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa584078c50 0x7fa58407b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7eff0ea00640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efef0078920 0x7efef007ade0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff00009aa0 tx=0x7eff00005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7f10bde00640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10a0078920 0x7f10a007ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7eff0e000640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7efef0080b40 0x7efef0082f60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 <== osd.3 v2:172.21.15.60:6824/3085378704 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd65001ee50 con 0x7fd63c080ac0 2024-09-16T15:38:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7f10bf200640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f10a0080b40 0x7f10a0082f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.296+0000 7f10bde00640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10a0078920 0x7f10a007ade0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10c01fc100 tx=0x7f10b0001600 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.295+0000 7fa5a2a00640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa584078c50 0x7fa58407b110 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0012e0 tx=0x7fa59c007d30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.296+0000 7fa5a2000640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7fa584080df0 0x7fa584083210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.296+0000 7f10bf200640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f10a0080b40 0x7f10a0082f60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.296+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 <== osd.5 v2:172.21.15.100:6818/1966336275 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff08045e80 con 0x7efef0080b40 2024-09-16T15:38:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.297+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 <== osd.4 v2:172.21.15.100:6810/3116664612 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa594001d30 con 0x7fa584080df0 2024-09-16T15:38:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.297+0000 7fc6c1c00640 1 -- 172.21.15.60:0/1082917021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc15f5a0 msgr2=0x7fc6bc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.297+0000 7fc6c1c00640 1 --2- 172.21.15.60:0/1082917021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc15f5a0 0x7fc6bc0b2440 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc076720 tx=0x7fc6ac02a510 comp rx=0 tx=0).stop 2024-09-16T15:38:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.298+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 <== osd.6 v2:172.21.15.100:6826/3702271988 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10ac00de50 con 0x7f10a0080b40 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.298+0000 7fc6c1c00640 1 -- 172.21.15.60:0/1082917021 shutdown_connections 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.298+0000 7fc6c1c00640 1 --2- 172.21.15.60:0/1082917021 >> v2:172.21.15.100:3300/0 conn(0x7fc6bc0ad270 0x7fc6bc073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.298+0000 7fc6c1c00640 1 --2- 172.21.15.60:0/1082917021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc0ccf00 0x7fc6bc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.298+0000 7fc6c1c00640 1 --2- 172.21.15.60:0/1082917021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc15f5a0 0x7fc6bc0b2440 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.298+0000 7fc6c1c00640 1 -- 172.21.15.60:0/1082917021 >> 172.21.15.60:0/1082917021 conn(0x7fc6bc0b1950 msgr2=0x7fc6bc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.298+0000 7fc6c1c00640 1 -- 172.21.15.60:0/1082917021 shutdown_connections 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.298+0000 7fc6c1c00640 1 -- 172.21.15.60:0/1082917021 wait complete. 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.301+0000 7fc6c1c00640 1 Processor -- start 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.301+0000 7fc6c1c00640 1 -- start start 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.303+0000 7fc6c1c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc0ad270 0x7fc6bc0c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.303+0000 7fc6c1c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc0ccf00 0x7fc6bc1f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.303+0000 7fc6c1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc15f5a0 0x7fc6bc144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.303+0000 7fc6c1c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc06f160 con 0x7fc6bc0ccf00 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.303+0000 7fc6c1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc074780 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.303+0000 7fc6c1c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc6bc06f2e0 con 0x7fc6bc0ad270 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc15f5a0 0x7fc6bc144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6bb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc15f5a0 0x7fc6bc144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56924/0 (socket says 172.21.15.60:56924) 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6bb400640 1 -- 172.21.15.60:0/4265879604 learned_addr learned my addr 172.21.15.60:0/4265879604 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6b2a00640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc0ad270 0x7fc6bc0c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6bb400640 1 -- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc0ad270 msgr2=0x7fc6bc0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6bb400640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc0ad270 0x7fc6bc0c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6bb400640 1 -- 172.21.15.60:0/4265879604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc0ccf00 msgr2=0x7fc6bc1f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6bb400640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc0ccf00 0x7fc6bc1f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.304+0000 7fc6bb400640 1 -- 172.21.15.60:0/4265879604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc145510 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.305+0000 7fc6bb400640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc15f5a0 0x7fc6bc144fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400b970 tx=0x7fc6b400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.305+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4019070 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.305+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6b400cc50 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.305+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40146a0 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.306+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc0c5c10 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.306+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc0c61d0 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.307+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6bc0c6890 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.311+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc6b401c070 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.312+0000 7fc6b3400640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc69807cc60 0x7fc69807f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.313+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc6b409ada0 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.313+0000 7fc6b3400640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7fc698084e60 0x7fc698087280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.313+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 --> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc698087930 con 0x7fc698084e60 2024-09-16T15:38:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.313+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc6b409b150 con 0x7fc6bc15f5a0 2024-09-16T15:38:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.313+0000 7fc6b2a00640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc69807cc60 0x7fc69807f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.314+0000 7fc6baa00640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7fc698084e60 0x7fc698087280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.314+0000 7fc6b2a00640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc69807cc60 0x7fc69807f120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac033000 tx=0x7fc6ac004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.316+0000 7fc6baa00640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7fc698084e60 0x7fc698087280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.317+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 <== osd.7 v2:172.21.15.100:6802/1334072453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc698087930 con 0x7fc698084e60 2024-09-16T15:38:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.319+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 --> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd65c0ceec0 con 0x7fd63c080ac0 2024-09-16T15:38:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.321+0000 7fd658c00640 1 -- 172.21.15.60:0/1136849526 <== osd.3 v2:172.21.15.60:6824/3085378704 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd65c0ceec0 con 0x7fd63c080ac0 2024-09-16T15:38:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.323+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 --> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbddc002dd0 con 0x7fbde4080aa0 2024-09-16T15:38:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.323+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7fd63c080ac0 msgr2=0x7fd63c082ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.323+0000 7fd663a00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7fd63c080ac0 0x7fd63c082ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.324+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd63c078920 msgr2=0x7fd63c07ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.324+0000 7fd663a00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd63c078920 0x7fd63c07ade0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0cd510 tx=0x7fd64800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.324+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0cea10 msgr2=0x7fd65c0a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.324+0000 7fd663a00640 1 --2- 172.21.15.60:0/1136849526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0cea10 0x7fd65c0a1750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd65000e9f0 tx=0x7fd65000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.324+0000 7fd661a00640 1 -- 172.21.15.60:0/1136849526 reap_dead start 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.324+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 shutdown_connections 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.324+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 >> 172.21.15.60:0/1136849526 conn(0x7fd65c0d3fe0 msgr2=0x7fd65c06d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.324+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 --> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe0bc037940 con 0x7fe0a4080b40 2024-09-16T15:38:46.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.325+0000 7fbdfaa00640 1 -- 172.21.15.60:0/2865427669 <== osd.0 v2:172.21.15.60:6816/841472238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbddc002dd0 con 0x7fbde4080aa0 2024-09-16T15:38:46.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.325+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 shutdown_connections 2024-09-16T15:38:46.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.325+0000 7fd663a00640 1 -- 172.21.15.60:0/1136849526 wait complete. 2024-09-16T15:38:46.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.325+0000 7fe0b7400640 1 -- 172.21.15.60:0/780129974 <== osd.1 v2:172.21.15.60:6800/910651751 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe0bc037940 con 0x7fe0a4080b40 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7fe0a4080b40 msgr2=0x7fe0a4082f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c8a00640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7fe0a4080b40 0x7fe0a4082f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0a4078920 msgr2=0x7fe0a407ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c8a00640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0a4078920 0x7fe0a407ade0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4051c90 tx=0x7fe0c408bf90 comp rx=0 tx=0).stop 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc0e2a60 msgr2=0x7fe0bc051d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c8a00640 1 --2- 172.21.15.60:0/780129974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0bc0e2a60 0x7fe0bc051d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0b802aa50 tx=0x7fe0b8003b90 comp rx=0 tx=0).stop 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c3e00640 1 -- 172.21.15.60:0/780129974 reap_dead start 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 shutdown_connections 2024-09-16T15:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.326+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 >> 172.21.15.60:0/780129974 conn(0x7fe0bc055540 msgr2=0x7fe0bc0e6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7fbde4080aa0 msgr2=0x7fbde4082ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03e00640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7fbde4080aa0 0x7fbde4082ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 shutdown_connections 2024-09-16T15:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fe0c8a00640 1 -- 172.21.15.60:0/780129974 wait complete. 2024-09-16T15:38:46.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbde4078880 msgr2=0x7fbde407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 --> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5200b3040 con 0x7fd500080aa0 2024-09-16T15:38:46.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03e00640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbde4078880 0x7fbde407ad40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0034000 tx=0x7fbdf00046c0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdfc176600 msgr2=0x7fbdfc04da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03e00640 1 --2- 172.21.15.60:0/2865427669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdfc176600 0x7fbdfc04da50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc004180 tx=0x7fbdf400c820 comp rx=0 tx=0).stop 2024-09-16T15:38:46.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03400640 1 -- 172.21.15.60:0/2865427669 reap_dead start 2024-09-16T15:38:46.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 shutdown_connections 2024-09-16T15:38:46.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.327+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 >> 172.21.15.60:0/2865427669 conn(0x7fbdfc01d770 msgr2=0x7fbdfc0dc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.328+0000 7fd51cc00640 1 -- 172.21.15.60:0/2954873284 <== osd.2 v2:172.21.15.60:6808/3414864047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5200b3040 con 0x7fd500080aa0 2024-09-16T15:38:46.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.328+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 shutdown_connections 2024-09-16T15:38:46.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.328+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fd500080aa0 msgr2=0x7fd500082ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.328+0000 7fd527400640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fd500080aa0 0x7fd500082ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.328+0000 7fbe03e00640 1 -- 172.21.15.60:0/2865427669 wait complete. 2024-09-16T15:38:46.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd500078880 msgr2=0x7fd50007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd527400640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd500078880 0x7fd50007ad40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5200a9510 tx=0x7fd50c010040 comp rx=0 tx=0).stop 2024-09-16T15:38:46.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd504004de0 msgr2=0x7fd52015c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd527400640 1 --2- 172.21.15.60:0/2954873284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd504004de0 0x7fd52015c840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd51802b4e0 tx=0x7fd51802b5f0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd525400640 1 -- 172.21.15.60:0/2954873284 reap_dead start 2024-09-16T15:38:46.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 shutdown_connections 2024-09-16T15:38:46.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 >> 172.21.15.60:0/2954873284 conn(0x7fd52006cd50 msgr2=0x7fd5200b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.330+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 --> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5a4073c20 con 0x7fa584080df0 2024-09-16T15:38:46.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 shutdown_connections 2024-09-16T15:38:46.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.329+0000 7fd527400640 1 -- 172.21.15.60:0/2954873284 wait complete. 2024-09-16T15:38:46.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.330+0000 7fa59aa00640 1 -- 172.21.15.60:0/3950804476 <== osd.4 v2:172.21.15.100:6810/3116664612 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa5a4073c20 con 0x7fa584080df0 2024-09-16T15:38:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.331+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7fa584080df0 msgr2=0x7fa584083210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.331+0000 7fa5a9800640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7fa584080df0 0x7fa584083210 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.331+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa584078c50 msgr2=0x7fa58407b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.331+0000 7fa5a9800640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa584078c50 0x7fa58407b110 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0012e0 tx=0x7fa59c007d30 comp rx=0 tx=0).stop 2024-09-16T15:38:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.331+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a41438b0 msgr2=0x7fa5a40b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.331+0000 7fa5a9800640 1 --2- 172.21.15.60:0/3950804476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5a41438b0 0x7fa5a40b12e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa59400b990 tx=0x7fa594003b90 comp rx=0 tx=0).stop 2024-09-16T15:38:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.331+0000 7fa5a3400640 1 -- 172.21.15.60:0/3950804476 reap_dead start 2024-09-16T15:38:46.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.332+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 shutdown_connections 2024-09-16T15:38:46.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.332+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 >> 172.21.15.60:0/3950804476 conn(0x7fa5a40b1950 msgr2=0x7fa5a40b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.332+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 shutdown_connections 2024-09-16T15:38:46.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.332+0000 7fa5a9800640 1 -- 172.21.15.60:0/3950804476 wait complete. 2024-09-16T15:38:46.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.333+0000 7eff15a00640 1 -- 172.21.15.60:0/525171848 --> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff10157e00 con 0x7efef0080b40 2024-09-16T15:38:46.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.333+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 --> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f10c006eec0 con 0x7f10a0080b40 2024-09-16T15:38:46.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.334+0000 7f10b6a00640 1 -- 172.21.15.60:0/3606770394 <== osd.6 v2:172.21.15.100:6826/3702271988 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f10c006eec0 con 0x7f10a0080b40 2024-09-16T15:38:46.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.334+0000 7eff06a00640 1 -- 172.21.15.60:0/525171848 <== osd.5 v2:172.21.15.100:6818/1966336275 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7eff10157e00 con 0x7efef0080b40 2024-09-16T15:38:46.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f10a0080b40 msgr2=0x7f10a0082f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7f10c5400640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f10a0080b40 0x7f10a0082f60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7efefbe00640 1 -- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7efef0080b40 msgr2=0x7efef0082f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7efefbe00640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7efef0080b40 0x7efef0082f60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7efefbe00640 1 -- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efef0078920 msgr2=0x7efef007ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7efefbe00640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efef0078920 0x7efef007ade0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff00009aa0 tx=0x7eff00005f90 comp rx=0 tx=0).stop 2024-09-16T15:38:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7efefbe00640 1 -- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff10162270 msgr2=0x7eff1009cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7efefbe00640 1 --2- 172.21.15.60:0/525171848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff10162270 0x7eff1009cd40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7eff08035040 tx=0x7eff08032a20 comp rx=0 tx=0).stop 2024-09-16T15:38:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.335+0000 7eff0f400640 1 -- 172.21.15.60:0/525171848 reap_dead start 2024-09-16T15:38:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10a0078920 msgr2=0x7f10a007ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7f10c5400640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10a0078920 0x7f10a007ade0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10c01fc100 tx=0x7f10b0001600 comp rx=0 tx=0).stop 2024-09-16T15:38:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 msgr2=0x7f10c01f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7f10c5400640 1 --2- 172.21.15.60:0/3606770394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10a40099b0 0x7f10c01f8c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f10ac004d90 tx=0x7f10ac001630 comp rx=0 tx=0).stop 2024-09-16T15:38:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7f10bf200640 1 -- 172.21.15.60:0/3606770394 reap_dead start 2024-09-16T15:38:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 shutdown_connections 2024-09-16T15:38:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 >> 172.21.15.60:0/3606770394 conn(0x7f10c00b1950 msgr2=0x7f10c006f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7efefbe00640 1 -- 172.21.15.60:0/525171848 shutdown_connections 2024-09-16T15:38:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.336+0000 7efefbe00640 1 -- 172.21.15.60:0/525171848 >> 172.21.15.60:0/525171848 conn(0x7eff100b1950 msgr2=0x7eff100735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.337+0000 7efefbe00640 1 -- 172.21.15.60:0/525171848 shutdown_connections 2024-09-16T15:38:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.337+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 shutdown_connections 2024-09-16T15:38:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.337+0000 7efefbe00640 1 -- 172.21.15.60:0/525171848 wait complete. 2024-09-16T15:38:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.337+0000 7f10c5400640 1 -- 172.21.15.60:0/3606770394 wait complete. 2024-09-16T15:38:46.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.347+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 --> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6bc06f870 con 0x7fc698084e60 2024-09-16T15:38:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.348+0000 7fc6b3400640 1 -- 172.21.15.60:0/4265879604 <== osd.7 v2:172.21.15.100:6802/1334072453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6bc06f870 con 0x7fc698084e60 2024-09-16T15:38:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.351+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7fc698084e60 msgr2=0x7fc698087280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.351+0000 7fc6c1c00640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7fc698084e60 0x7fc698087280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.355 INFO:teuthology.orchestra.run.smithi060.stdout:60129542152 2024-09-16T15:38:46.356 INFO:teuthology.orchestra.run.smithi060.stdout:64424509447 2024-09-16T15:38:46.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T15:38:46.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T15:38:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc69807cc60 msgr2=0x7fc69807f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6c1c00640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc69807cc60 0x7fc69807f120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac033000 tx=0x7fc6ac004730 comp rx=0 tx=0).stop 2024-09-16T15:38:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc15f5a0 msgr2=0x7fc6bc144fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6c1c00640 1 --2- 172.21.15.60:0/4265879604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc15f5a0 0x7fc6bc144fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400b970 tx=0x7fc6b400be30 comp rx=0 tx=0).stop 2024-09-16T15:38:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6bb400640 1 -- 172.21.15.60:0/4265879604 reap_dead start 2024-09-16T15:38:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 shutdown_connections 2024-09-16T15:38:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 >> 172.21.15.60:0/4265879604 conn(0x7fc6bc0b1950 msgr2=0x7fc6bc09bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 shutdown_connections 2024-09-16T15:38:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.355+0000 7fc6c1c00640 1 -- 172.21.15.60:0/4265879604 wait complete. 2024-09-16T15:38:46.364 INFO:teuthology.orchestra.run.smithi060.stdout:64424509448 2024-09-16T15:38:46.365 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-09-16T15:38:46.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T15:38:46.366 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-09-16T15:38:46.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T15:38:46.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T15:38:46.367 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-09-16T15:38:46.368 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T15:38:46.369 INFO:teuthology.orchestra.run.smithi060.stdout:64424509448 2024-09-16T15:38:46.369 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T15:38:46.376 INFO:teuthology.orchestra.run.smithi060.stdout:64424509448 2024-09-16T15:38:46.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T15:38:46.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.639+0000 7fdb25a00640 1 -- 172.21.15.60:0/2198914113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb04003a50 msgr2=0x7fdb04003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.639+0000 7fdb25a00640 1 --2- 172.21.15.60:0/2198914113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb04003a50 0x7fdb04003e70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009c10 tx=0x7fdb04005830 comp rx=0 tx=0).stop 2024-09-16T15:38:46.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.640+0000 7fdb25a00640 1 -- 172.21.15.60:0/2198914113 shutdown_connections 2024-09-16T15:38:46.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.640+0000 7fdb25a00640 1 --2- 172.21.15.60:0/2198914113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb04003a50 0x7fdb04003e70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.640+0000 7fdb25a00640 1 -- 172.21.15.60:0/2198914113 >> 172.21.15.60:0/2198914113 conn(0x7fdb200d3df0 msgr2=0x7fdb200d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.642+0000 7fdb25a00640 1 -- 172.21.15.60:0/2198914113 shutdown_connections 2024-09-16T15:38:46.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.642+0000 7fdb25a00640 1 -- 172.21.15.60:0/2198914113 wait complete. 2024-09-16T15:38:46.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.644+0000 7fd652200640 1 -- 172.21.15.60:0/2527090631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440e2a60 msgr2=0x7fd644037940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.644+0000 7fdb25a00640 1 Processor -- start 2024-09-16T15:38:46.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.645+0000 7fdb25a00640 1 -- start start 2024-09-16T15:38:46.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.645+0000 7fd652200640 1 --2- 172.21.15.60:0/2527090631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440e2a60 0x7fd644037940 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd64403e060 tx=0x7fd63802a640 comp rx=0 tx=0).stop 2024-09-16T15:38:46.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.645+0000 7fd652200640 1 -- 172.21.15.60:0/2527090631 shutdown_connections 2024-09-16T15:38:46.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.645+0000 7fd652200640 1 --2- 172.21.15.60:0/2527090631 >> v2:172.21.15.100:3300/0 conn(0x7fd64403ab10 0x7fd644052ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.645+0000 7fd652200640 1 --2- 172.21.15.60:0/2527090631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440e2a60 0x7fd644037940 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd64403e060 tx=0x7fd63802a640 comp rx=0 tx=0).stop 2024-09-16T15:38:46.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.645+0000 7fd652200640 1 -- 172.21.15.60:0/2527090631 >> 172.21.15.60:0/2527090631 conn(0x7fd644055540 msgr2=0x7fd644055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.646+0000 7fd652200640 1 -- 172.21.15.60:0/2527090631 shutdown_connections 2024-09-16T15:38:46.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.646+0000 7fd652200640 1 -- 172.21.15.60:0/2527090631 wait complete. 2024-09-16T15:38:46.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.647+0000 7fdb25a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb04003a50 0x7fdb20155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.647+0000 7fdb25a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb20154620 0x7fdb200b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.647+0000 7fdb25a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb200b1ec0 0x7fdb200b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.647+0000 7fdb25a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdb200b3ac0 con 0x7fdb200b1ec0 2024-09-16T15:38:46.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.647+0000 7fdb25a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb200b3940 con 0x7fdb20154620 2024-09-16T15:38:46.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.647+0000 7fdb25a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdb200b3c40 con 0x7fdb04003a50 2024-09-16T15:38:46.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.648+0000 7fdb1f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb20154620 0x7fdb200b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.648+0000 7fdb1f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb20154620 0x7fdb200b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56950/0 (socket says 172.21.15.60:56950) 2024-09-16T15:38:46.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.648+0000 7fdb1f400640 1 -- 172.21.15.60:0/2741300424 learned_addr learned my addr 172.21.15.60:0/2741300424 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.649+0000 7fdb1fe00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb04003a50 0x7fdb20155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.649+0000 7fdb25000640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb200b1ec0 0x7fdb200b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.650+0000 7fdb1fe00640 1 -- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb20154620 msgr2=0x7fdb200b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.650+0000 7fdb1fe00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb20154620 0x7fdb200b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.650+0000 7fdb1fe00640 1 -- 172.21.15.60:0/2741300424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb200b1ec0 msgr2=0x7fdb200b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.650+0000 7fdb1fe00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb200b1ec0 0x7fdb200b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.650+0000 7fdb1fe00640 1 -- 172.21.15.60:0/2741300424 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb200c7920 con 0x7fdb04003a50 2024-09-16T15:38:46.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.649+0000 7fd652200640 1 Processor -- start 2024-09-16T15:38:46.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.650+0000 7fdb1fe00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb04003a50 0x7fdb20155ae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c110 tx=0x7fdb0c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.651+0000 7fdb17e00640 1 -- 172.21.15.60:0/2741300424 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c038070 con 0x7fdb04003a50 2024-09-16T15:38:46.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.651+0000 7fdb17e00640 1 -- 172.21.15.60:0/2741300424 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb0c039040 con 0x7fdb04003a50 2024-09-16T15:38:46.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.652+0000 7fdb17e00640 1 -- 172.21.15.60:0/2741300424 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c003e00 con 0x7fdb04003a50 2024-09-16T15:38:46.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.652+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb200c7b90 con 0x7fdb04003a50 2024-09-16T15:38:46.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.652+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb200c8150 con 0x7fdb04003a50 2024-09-16T15:38:46.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.651+0000 7fd652200640 1 -- start start 2024-09-16T15:38:46.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.652+0000 7fd652200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64403ab10 0x7fd64416cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.652+0000 7fd652200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440e39e0 0x7fd64417d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.652+0000 7fd652200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440503a0 0x7fd644180380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.652+0000 7fd652200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd644053770 con 0x7fd6440503a0 2024-09-16T15:38:46.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.653+0000 7fd652200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6440535f0 con 0x7fd6440e39e0 2024-09-16T15:38:46.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.653+0000 7fd652200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd6440538f0 con 0x7fd64403ab10 2024-09-16T15:38:46.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.654+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb200d1d60 con 0x7fdb04003a50 2024-09-16T15:38:46.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.655+0000 7f375b200640 1 -- 172.21.15.60:0/3099293622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c0b4860 msgr2=0x7f375c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.655+0000 7f375b200640 1 --2- 172.21.15.60:0/3099293622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c0b4860 0x7f375c0b4ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3748009830 tx=0x7f374802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.655+0000 7f375b200640 1 -- 172.21.15.60:0/3099293622 shutdown_connections 2024-09-16T15:38:46.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.655+0000 7f375b200640 1 --2- 172.21.15.60:0/3099293622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c0b4860 0x7f375c0b4ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.655+0000 7f375b200640 1 -- 172.21.15.60:0/3099293622 >> 172.21.15.60:0/3099293622 conn(0x7f375c0d3df0 msgr2=0x7f375c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.656+0000 7f375b200640 1 -- 172.21.15.60:0/3099293622 shutdown_connections 2024-09-16T15:38:46.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.656+0000 7fdb17e00640 1 -- 172.21.15.60:0/2741300424 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fdb0c044070 con 0x7fdb04003a50 2024-09-16T15:38:46.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.654+0000 7fd650e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64403ab10 0x7fd64416cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.656+0000 7fd64be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440e39e0 0x7fd64417d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.657+0000 7fd64be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440e39e0 0x7fd64417d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56958/0 (socket says 172.21.15.60:56958) 2024-09-16T15:38:46.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.657+0000 7fd64be00640 1 -- 172.21.15.60:0/2498742334 learned_addr learned my addr 172.21.15.60:0/2498742334 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.656+0000 7f375b200640 1 -- 172.21.15.60:0/3099293622 wait complete. 2024-09-16T15:38:46.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.657+0000 7fd64be00640 1 -- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64403ab10 msgr2=0x7fd64416cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.657+0000 7fd64be00640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64403ab10 0x7fd64416cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.657+0000 7fd64be00640 1 -- 172.21.15.60:0/2498742334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440503a0 msgr2=0x7fd644180380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.657+0000 7fdb17e00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdb000788a0 0x7fdb0007ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.657+0000 7fdb17e00640 1 -- 172.21.15.60:0/2741300424 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdb0c0ba6a0 con 0x7fdb04003a50 2024-09-16T15:38:46.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.656+0000 7fd651800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440503a0 0x7fd644180380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.658+0000 7fd64be00640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440503a0 0x7fd644180380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.658+0000 7fd64be00640 1 -- 172.21.15.60:0/2498742334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd644050850 con 0x7fd6440e39e0 2024-09-16T15:38:46.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.658+0000 7f375b200640 1 Processor -- start 2024-09-16T15:38:46.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.659+0000 7fdb1f400640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdb000788a0 0x7fdb0007ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.659+0000 7fd651800640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440503a0 0x7fd644180380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.659+0000 7f375b200640 1 -- start start 2024-09-16T15:38:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.659+0000 7fd64be00640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440e39e0 0x7fd64417d620 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd64000b3f0 tx=0x7fd64000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.660+0000 7fd648c00640 1 -- 172.21.15.60:0/2498742334 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640015020 con 0x7fd6440e39e0 2024-09-16T15:38:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.660+0000 7fdb1f400640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdb000788a0 0x7fdb0007ad60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdb201f7e20 tx=0x7fdb1000b260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.660+0000 7fd648c00640 1 -- 172.21.15.60:0/2498742334 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd640011920 con 0x7fd6440e39e0 2024-09-16T15:38:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.660+0000 7fd648c00640 1 -- 172.21.15.60:0/2498742334 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640011d70 con 0x7fd6440e39e0 2024-09-16T15:38:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.661+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64416d3a0 con 0x7fd6440e39e0 2024-09-16T15:38:46.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.661+0000 7f375b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c0b4860 0x7f375c154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.661+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd644051760 con 0x7fd6440e39e0 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.661+0000 7f375b200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0b5410 0x7f375c1618b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.661+0000 7f375b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c0d1fd0 0x7f375c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.661+0000 7f375b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f375c0b16f0 con 0x7f375c0d1fd0 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.661+0000 7f375b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f375c0b1570 con 0x7f375c0b4860 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.661+0000 7f375b200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f375c15fc80 con 0x7f375c0b5410 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c0b4860 0x7f375c154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c0b4860 0x7f375c154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:56974/0 (socket says 172.21.15.60:56974) 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759e00640 1 -- 172.21.15.60:0/898634385 learned_addr learned my addr 172.21.15.60:0/898634385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0b5410 0x7f375c1618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f375a800640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c0d1fd0 0x7f375c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759e00640 1 -- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0b5410 msgr2=0x7f375c1618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759e00640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0b5410 0x7f375c1618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759e00640 1 -- 172.21.15.60:0/898634385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c0d1fd0 msgr2=0x7f375c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759e00640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c0d1fd0 0x7f375c0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.662+0000 7f3759e00640 1 -- 172.21.15.60:0/898634385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c0d16f0 con 0x7f375c0b4860 2024-09-16T15:38:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.663+0000 7f3759e00640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c0b4860 0x7f375c154f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f374c00c460 tx=0x7f374c00c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.663+0000 7f3752000640 1 -- 172.21.15.60:0/898634385 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c016020 con 0x7f375c0b4860 2024-09-16T15:38:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.664+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64416d8c0 con 0x7fd6440e39e0 2024-09-16T15:38:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.664+0000 7fdb17e00640 1 -- 172.21.15.60:0/2741300424 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdb0c0828c0 con 0x7fdb04003a50 2024-09-16T15:38:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.663+0000 7f3752000640 1 -- 172.21.15.60:0/898634385 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f374c012940 con 0x7f375c0b4860 2024-09-16T15:38:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.663+0000 7f3752000640 1 -- 172.21.15.60:0/898634385 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c012d90 con 0x7f375c0b4860 2024-09-16T15:38:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.664+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c0d1410 con 0x7f375c0b4860 2024-09-16T15:38:46.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.664+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c1540c0 con 0x7f375c0b4860 2024-09-16T15:38:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.666+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375c0b4ed0 con 0x7f375c0b4860 2024-09-16T15:38:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.666+0000 7f606de00640 1 -- 172.21.15.60:0/3170212144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815dff0 msgr2=0x7f60680b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.666+0000 7f606de00640 1 --2- 172.21.15.60:0/3170212144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815dff0 0x7f60680b0970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6058005fd0 tx=0x7f605802a580 comp rx=0 tx=0).stop 2024-09-16T15:38:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.667+0000 7fd648c00640 1 -- 172.21.15.60:0/2498742334 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd640018020 con 0x7fd6440e39e0 2024-09-16T15:38:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.667+0000 7f606de00640 1 -- 172.21.15.60:0/3170212144 shutdown_connections 2024-09-16T15:38:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.667+0000 7f606de00640 1 --2- 172.21.15.60:0/3170212144 >> v2:172.21.15.60:3300/0 conn(0x7f60680ceda0 0x7f60680d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.667+0000 7f606de00640 1 --2- 172.21.15.60:0/3170212144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60680b0eb0 0x7f60680ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.667+0000 7f606de00640 1 --2- 172.21.15.60:0/3170212144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f606815dff0 0x7f60680b0970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.667+0000 7f606de00640 1 -- 172.21.15.60:0/3170212144 >> 172.21.15.60:0/3170212144 conn(0x7f60680d3de0 msgr2=0x7f60680d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.667+0000 7f606de00640 1 -- 172.21.15.60:0/3170212144 shutdown_connections 2024-09-16T15:38:46.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.668+0000 7fd648c00640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd62c0788a0 0x7fd62c07ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.667+0000 7f606de00640 1 -- 172.21.15.60:0/3170212144 wait complete. 2024-09-16T15:38:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.668+0000 7fd648c00640 1 -- 172.21.15.60:0/2498742334 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd64009a340 con 0x7fd6440e39e0 2024-09-16T15:38:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.668+0000 7fd650e00640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd62c0788a0 0x7fd62c07ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.669+0000 7fd650e00640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd62c0788a0 0x7fd62c07ad60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd638033000 tx=0x7fd638004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.670+0000 7f7131c00640 1 -- 172.21.15.60:0/2390130805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f712c0d2050 msgr2=0x7f712c0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.670+0000 7f7131c00640 1 --2- 172.21.15.60:0/2390130805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f712c0d2050 0x7f712c0d10d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f711c009860 tx=0x7f711c02a5a0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.671+0000 7fd648c00640 1 -- 172.21.15.60:0/2498742334 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd64001f070 con 0x7fd6440e39e0 2024-09-16T15:38:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.671+0000 7f606de00640 1 Processor -- start 2024-09-16T15:38:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.671+0000 7f7131c00640 1 -- 172.21.15.60:0/2390130805 shutdown_connections 2024-09-16T15:38:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.672+0000 7f606de00640 1 -- start start 2024-09-16T15:38:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.672+0000 7f7131c00640 1 --2- 172.21.15.60:0/2390130805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f712c0d2050 0x7f712c0d10d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.672+0000 7f7131c00640 1 -- 172.21.15.60:0/2390130805 >> 172.21.15.60:0/2390130805 conn(0x7f712c0d3fe0 msgr2=0x7f712c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.672+0000 7f7131c00640 1 -- 172.21.15.60:0/2390130805 shutdown_connections 2024-09-16T15:38:46.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.672+0000 7f7131c00640 1 -- 172.21.15.60:0/2390130805 wait complete. 2024-09-16T15:38:46.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.673+0000 7f606de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60680b0eb0 0x7f60680c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.673+0000 7f3c9ce00640 1 -- 172.21.15.60:0/1475974575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c981622d0 msgr2=0x7f3c980d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.673+0000 7f606de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60680ceda0 0x7f606815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.673+0000 7f606de00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815dff0 0x7f606815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.673+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/1475974575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c981622d0 0x7f3c980d1b70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3c98069630 tx=0x7f3c9002aa50 comp rx=0 tx=0).stop 2024-09-16T15:38:46.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f606de00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f60680d1fe0 con 0x7f60680ceda0 2024-09-16T15:38:46.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f606de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60680d1e60 con 0x7f60680b0eb0 2024-09-16T15:38:46.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f3c9ce00640 1 -- 172.21.15.60:0/1475974575 shutdown_connections 2024-09-16T15:38:46.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/1475974575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c981622d0 0x7f3c980d1b70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/1475974575 >> v2:172.21.15.100:3300/0 conn(0x7f3c980b4a50 0x7f3c980d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f3c9ce00640 1 -- 172.21.15.60:0/1475974575 >> 172.21.15.60:0/1475974575 conn(0x7f3c980d3dc0 msgr2=0x7f3c980d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f606de00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f60680d2160 con 0x7f606815dff0 2024-09-16T15:38:46.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f6066a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60680b0eb0 0x7f60680c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.675+0000 7f6067400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815dff0 0x7f606815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.675+0000 7f6066000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60680ceda0 0x7f606815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.675+0000 7f3c9ce00640 1 -- 172.21.15.60:0/1475974575 shutdown_connections 2024-09-16T15:38:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.675+0000 7f6067400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815dff0 0x7f606815b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:53178/0 (socket says 172.21.15.60:53178) 2024-09-16T15:38:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.675+0000 7f6067400640 1 -- 172.21.15.60:0/963972129 learned_addr learned my addr 172.21.15.60:0/963972129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.676+0000 7f3c9ce00640 1 -- 172.21.15.60:0/1475974575 wait complete. 2024-09-16T15:38:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.676+0000 7f6067400640 1 -- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60680b0eb0 msgr2=0x7f60680c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.676+0000 7f6067400640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60680b0eb0 0x7f60680c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.676+0000 7f6067400640 1 -- 172.21.15.60:0/963972129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60680ceda0 msgr2=0x7f606815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.674+0000 7f3752000640 1 -- 172.21.15.60:0/898634385 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f374c019020 con 0x7f375c0b4860 2024-09-16T15:38:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.676+0000 7f6067400640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60680ceda0 0x7f606815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.676+0000 7f6067400640 1 -- 172.21.15.60:0/963972129 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60680c7cb0 con 0x7f606815dff0 2024-09-16T15:38:46.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.677+0000 7f3752000640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f373c078850 0x7f373c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.677+0000 7f3752000640 1 -- 172.21.15.60:0/898634385 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f374c09a780 con 0x7f375c0b4860 2024-09-16T15:38:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.676+0000 7f7131c00640 1 Processor -- start 2024-09-16T15:38:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.676+0000 7f7131c00640 1 -- start start 2024-09-16T15:38:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.678+0000 7f6067400640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815dff0 0x7f606815b300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f606000cce0 tx=0x7f606000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.678+0000 7f3759400640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f373c078850 0x7f373c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.678+0000 7f3752000640 1 -- 172.21.15.60:0/898634385 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f374c020030 con 0x7f375c0b4860 2024-09-16T15:38:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.678+0000 7f7131c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f712c0d1610 0x7f712c0c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f7131c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712c0d2050 0x7f712c0b9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f7131c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f712c15e0d0 0x7f712c1fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f7131c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f712c0cf840 con 0x7f712c0d2050 2024-09-16T15:38:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f7131c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f712c0cf6c0 con 0x7f712c0d1610 2024-09-16T15:38:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f7131c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f712c0cf9c0 con 0x7f712c15e0d0 2024-09-16T15:38:46.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f712aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f712c0d1610 0x7f712c0c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f712aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f712c0d1610 0x7f712c0c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:57010/0 (socket says 172.21.15.60:57010) 2024-09-16T15:38:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f712aa00640 1 -- 172.21.15.60:0/954010833 learned_addr learned my addr 172.21.15.60:0/954010833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f605f400640 1 -- 172.21.15.60:0/963972129 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606000f450 con 0x7f606815dff0 2024-09-16T15:38:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f605f400640 1 -- 172.21.15.60:0/963972129 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f606000f700 con 0x7f606815dff0 2024-09-16T15:38:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f605f400640 1 -- 172.21.15.60:0/963972129 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606000fb50 con 0x7f606815dff0 2024-09-16T15:38:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60680c7f90 con 0x7f606815dff0 2024-09-16T15:38:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.679+0000 7f712b400640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f712c15e0d0 0x7f712c1fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f60680c8550 con 0x7f606815dff0 2024-09-16T15:38:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f3759400640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f373c078850 0x7f373c07ad10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3748034000 tx=0x7f3748004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f7123e00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712c0d2050 0x7f712c0b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f712aa00640 1 -- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f712c15e0d0 msgr2=0x7f712c1fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f712aa00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f712c15e0d0 0x7f712c1fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f712aa00640 1 -- 172.21.15.60:0/954010833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712c0d2050 msgr2=0x7f712c0b9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f712aa00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712c0d2050 0x7f712c0b9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.680+0000 7f712aa00640 1 -- 172.21.15.60:0/954010833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c1f9810 con 0x7f712c0d1610 2024-09-16T15:38:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.681+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60680ca690 con 0x7f606815dff0 2024-09-16T15:38:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.681+0000 7f712aa00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f712c0d1610 0x7f712c0c7240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f711400c3a0 tx=0x7f711400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f7122a00640 1 -- 172.21.15.60:0/954010833 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114011790 con 0x7f712c0d1610 2024-09-16T15:38:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f7122a00640 1 -- 172.21.15.60:0/954010833 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7114017070 con 0x7f712c0d1610 2024-09-16T15:38:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f7122a00640 1 -- 172.21.15.60:0/954010833 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114011ca0 con 0x7f712c0d1610 2024-09-16T15:38:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c155550 con 0x7f712c0d1610 2024-09-16T15:38:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f8114e00640 1 -- 172.21.15.60:0/1820830477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f811015bbd0 msgr2=0x7f81100cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f8114e00640 1 --2- 172.21.15.60:0/1820830477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f811015bbd0 0x7f81100cd010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f81100698e0 tx=0x7f810802a830 comp rx=0 tx=0).stop 2024-09-16T15:38:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c155ad0 con 0x7f712c0d1610 2024-09-16T15:38:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.683+0000 7f8114e00640 1 -- 172.21.15.60:0/1820830477 shutdown_connections 2024-09-16T15:38:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.683+0000 7f8114e00640 1 --2- 172.21.15.60:0/1820830477 >> v2:172.21.15.60:3300/0 conn(0x7f8110162270 0x7f811006e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.683+0000 7f8114e00640 1 --2- 172.21.15.60:0/1820830477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f811015bbd0 0x7f81100cd010 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.683+0000 7f8114e00640 1 -- 172.21.15.60:0/1820830477 >> 172.21.15.60:0/1820830477 conn(0x7f81100b1950 msgr2=0x7f81100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.684+0000 7f8114e00640 1 -- 172.21.15.60:0/1820830477 shutdown_connections 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f3c9ce00640 1 Processor -- start 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.682+0000 7f3c9ce00640 1 -- start start 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.684+0000 7f3c9ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b4a50 0x7f3c9815b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.684+0000 7f3c9ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c981613a0 0x7f3c9815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.684+0000 7f3c9ce00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c981622d0 0x7f3c980ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.684+0000 7f3c9ce00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3c980d20b0 con 0x7f3c980b4a50 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.684+0000 7f3c9ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98161990 con 0x7f3c981613a0 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.684+0000 7f3c9ce00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3c980d2230 con 0x7f3c981622d0 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.685+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c981622d0 0x7f3c980ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.685+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c981622d0 0x7f3c980ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:53196/0 (socket says 172.21.15.60:53196) 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.685+0000 7f3c97e00640 1 -- 172.21.15.60:0/2158897480 learned_addr learned my addr 172.21.15.60:0/2158897480 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.685+0000 7f3c97400640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b4a50 0x7f3c9815b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.685+0000 7fd7d7e00640 1 -- 172.21.15.60:0/3144563033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d8161440 msgr2=0x7fd7d80b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.685+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/3144563033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d8161440 0x7fd7d80b5370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80b6960 tx=0x7fd7c402a560 comp rx=0 tx=0).stop 2024-09-16T15:38:46.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.685+0000 7f3c8fe00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c981613a0 0x7f3c9815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.684+0000 7f8114e00640 1 -- 172.21.15.60:0/1820830477 wait complete. 2024-09-16T15:38:46.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7f3c97e00640 1 -- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c981613a0 msgr2=0x7f3c9815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7f3c97e00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c981613a0 0x7f3c9815bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7fd7d7e00640 1 -- 172.21.15.60:0/3144563033 shutdown_connections 2024-09-16T15:38:46.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7f3c97e00640 1 -- 172.21.15.60:0/2158897480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b4a50 msgr2=0x7f3c9815b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7f3c97e00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b4a50 0x7f3c9815b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7f3c97e00640 1 -- 172.21.15.60:0/2158897480 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c980cae00 con 0x7f3c981622d0 2024-09-16T15:38:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/3144563033 >> v2:172.21.15.100:3300/0 conn(0x7fd7d80b4050 0x7fd7d80b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/3144563033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d8161440 0x7fd7d80b5370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7fd7d7e00640 1 -- 172.21.15.60:0/3144563033 >> 172.21.15.60:0/3144563033 conn(0x7fd7d80d3df0 msgr2=0x7fd7d80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.683+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c0008d0 con 0x7f712c0d1610 2024-09-16T15:38:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.685+0000 7f7122a00640 1 -- 172.21.15.60:0/954010833 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f711401c460 con 0x7f712c0d1610 2024-09-16T15:38:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7fd7d7e00640 1 -- 172.21.15.60:0/3144563033 shutdown_connections 2024-09-16T15:38:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7fd7d7e00640 1 -- 172.21.15.60:0/3144563033 wait complete. 2024-09-16T15:38:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.686+0000 7f605f400640 1 -- 172.21.15.60:0/963972129 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f606000fcf0 con 0x7f606815dff0 2024-09-16T15:38:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.687+0000 7f8114e00640 1 Processor -- start 2024-09-16T15:38:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.687+0000 7f8114e00640 1 -- start start 2024-09-16T15:38:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.687+0000 7f7122a00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f71080788a0 0x7f710807ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.687+0000 7f7122a00640 1 -- 172.21.15.60:0/954010833 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f711409a6d0 con 0x7f712c0d1610 2024-09-16T15:38:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.688+0000 7f7123e00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f71080788a0 0x7f710807ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.687+0000 7f3c97e00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c981622d0 0x7f3c980ccfa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009d60 tx=0x7f3c90003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.688+0000 7f8114e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100cd550 0x7f81101516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.688+0000 7f8114e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015bbd0 0x7f81101fc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.688+0000 7f8114e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8110162270 0x7f81101fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.688+0000 7f8114e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f811006f240 con 0x7f811015bbd0 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.688+0000 7f8114e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f811006f0c0 con 0x7f81100cd550 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f8114e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f811006f3c0 con 0x7f8110162270 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f810de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100cd550 0x7f81101516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f810de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100cd550 0x7f81101516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:57038/0 (socket says 172.21.15.60:57038) 2024-09-16T15:38:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f810de00640 1 -- 172.21.15.60:0/2033417454 learned_addr learned my addr 172.21.15.60:0/2033417454 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f7123e00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f71080788a0 0x7f710807ad60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f711c034000 tx=0x7f711c00b040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7fd7d7e00640 1 Processor -- start 2024-09-16T15:38:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f605f400640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6048078920 0x7f604807ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f810de00640 1 -- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8110162270 msgr2=0x7f81101fcb50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:38:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f810de00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8110162270 0x7f81101fcb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f810de00640 1 -- 172.21.15.60:0/2033417454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015bbd0 msgr2=0x7f81101fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f810de00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015bbd0 0x7f81101fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f810de00640 1 -- 172.21.15.60:0/2033417454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81101608f0 con 0x7f81100cd550 2024-09-16T15:38:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.690+0000 7fd7d7e00640 1 -- start start 2024-09-16T15:38:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.690+0000 7f810de00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100cd550 0x7f81101516a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8108009ca0 tx=0x7f8108033a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f3c8f400640 1 -- 172.21.15.60:0/2158897480 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90044070 con 0x7f3c981622d0 2024-09-16T15:38:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.690+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9815bc60 con 0x7f3c981622d0 2024-09-16T15:38:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.690+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9815cb20 con 0x7f3c981622d0 2024-09-16T15:38:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.690+0000 7f3c8f400640 1 -- 172.21.15.60:0/2158897480 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c90009870 con 0x7f3c981622d0 2024-09-16T15:38:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.690+0000 7f3c8f400640 1 -- 172.21.15.60:0/2158897480 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90044070 con 0x7f3c981622d0 2024-09-16T15:38:46.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.691+0000 7f8106000640 1 -- 172.21.15.60:0/2033417454 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108042020 con 0x7f81100cd550 2024-09-16T15:38:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.691+0000 7f8106000640 1 -- 172.21.15.60:0/2033417454 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8108038040 con 0x7f81100cd550 2024-09-16T15:38:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.691+0000 7f8106000640 1 -- 172.21.15.60:0/2033417454 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108037070 con 0x7f81100cd550 2024-09-16T15:38:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.692+0000 7fd7d7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d80b34b0 0x7fd7d80b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.692+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70005290 con 0x7f3c981622d0 2024-09-16T15:38:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.692+0000 7fd7d7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d80b4050 0x7fd7d80c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f605f400640 1 -- 172.21.15.60:0/963972129 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f606009b510 con 0x7f606815dff0 2024-09-16T15:38:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.692+0000 7f7122a00640 1 -- 172.21.15.60:0/954010833 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7114062d20 con 0x7f712c0d1610 2024-09-16T15:38:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.692+0000 7fd7d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d80b34b0 0x7fd7d80b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.692+0000 7fd7d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d80b34b0 0x7fd7d80b90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35504/0 (socket says 172.21.15.60:35504) 2024-09-16T15:38:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.692+0000 7fd7d6a00640 1 -- 172.21.15.60:0/1385754105 learned_addr learned my addr 172.21.15.60:0/1385754105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.689+0000 7f6066a00640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6048078920 0x7f604807ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.691+0000 7f605f400640 1 -- 172.21.15.60:0/963972129 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6060014030 con 0x7f606815dff0 2024-09-16T15:38:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.692+0000 7f6066a00640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6048078920 0x7f604807ade0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6058033000 tx=0x7f6058004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.693+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d8161440 0x7fd7d81fbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.693+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d80b2040 con 0x7fd7d80b34b0 2024-09-16T15:38:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.693+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d80b1ec0 con 0x7fd7d80b4050 2024-09-16T15:38:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.693+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8110160bd0 con 0x7f81100cd550 2024-09-16T15:38:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.693+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8110161190 con 0x7f81100cd550 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.693+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d80b21c0 con 0x7fd7d8161440 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.693+0000 7fd7d7400640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d8161440 0x7fd7d81fbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.693+0000 7fd7d6000640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d80b4050 0x7fd7d80c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.694+0000 7fd7d6a00640 1 -- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d8161440 msgr2=0x7fd7d81fbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.694+0000 7fd7d6a00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d8161440 0x7fd7d81fbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.694+0000 7fd7d6a00640 1 -- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d80b4050 msgr2=0x7fd7d80c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.694+0000 7fd7d6a00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d80b4050 0x7fd7d80c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.694+0000 7fd7d6a00640 1 -- 172.21.15.60:0/1385754105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d80c83a0 con 0x7fd7d80b34b0 2024-09-16T15:38:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.694+0000 7fd7d6a00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d80b34b0 0x7fd7d80b90e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4034040 tx=0x7fd7c4003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.695+0000 7f3c8f400640 1 -- 172.21.15.60:0/2158897480 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f3c90047070 con 0x7f3c981622d0 2024-09-16T15:38:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.695+0000 7fd7cea00640 1 -- 172.21.15.60:0/1385754105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c403e030 con 0x7fd7d80b34b0 2024-09-16T15:38:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.695+0000 7fd7cea00640 1 -- 172.21.15.60:0/1385754105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7c4003e50 con 0x7fd7d80b34b0 2024-09-16T15:38:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.695+0000 7fd7cea00640 1 -- 172.21.15.60:0/1385754105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c403e030 con 0x7fd7d80b34b0 2024-09-16T15:38:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.695+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d80c8680 con 0x7fd7d80b34b0 2024-09-16T15:38:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.696+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d80c8c40 con 0x7fd7d80b34b0 2024-09-16T15:38:46.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.697+0000 7f3c8f400640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3c74078c50 0x7f3c7407b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.697+0000 7f3c8f400640 1 -- 172.21.15.60:0/2158897480 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3c9003c070 con 0x7f3c981622d0 2024-09-16T15:38:46.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.697+0000 7f3c97400640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3c74078c50 0x7f3c7407b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.696+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f811006f8e0 con 0x7f81100cd550 2024-09-16T15:38:46.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.699+0000 7fd7cea00640 1 -- 172.21.15.60:0/1385754105 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd7c4041030 con 0x7fd7d80b34b0 2024-09-16T15:38:46.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.699+0000 7f8106000640 1 -- 172.21.15.60:0/2033417454 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8108045020 con 0x7f81100cd550 2024-09-16T15:38:46.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.700+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b0005290 con 0x7fd7d80b34b0 2024-09-16T15:38:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.701+0000 7f3c97400640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3c74078c50 0x7f3c7407b110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c88000f60 tx=0x7f3c88009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.700+0000 7fd7cea00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd7b8078a50 0x7fd7b807af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.701+0000 7f8106000640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f80f0078920 0x7f80f007ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.701+0000 7f8106000640 1 -- 172.21.15.60:0/2033417454 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f81080bb570 con 0x7f81100cd550 2024-09-16T15:38:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.700+0000 7fd7cea00640 1 -- 172.21.15.60:0/1385754105 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd7c40ba970 con 0x7fd7d80b34b0 2024-09-16T15:38:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.702+0000 7f3c8f400640 1 -- 172.21.15.60:0/2158897480 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3c90032030 con 0x7f3c981622d0 2024-09-16T15:38:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.702+0000 7fd7d6000640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd7b8078a50 0x7fd7b807af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.702+0000 7f810d400640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f80f0078920 0x7f80f007ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:46.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.703+0000 7fd7d6000640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd7b8078a50 0x7fd7b807af10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80c6ce0 tx=0x7fd7c8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.704+0000 7f810d400640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f80f0078920 0x7f80f007ade0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0046b0 tx=0x7f80fc0012b0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:46.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.704+0000 7f8106000640 1 -- 172.21.15.60:0/2033417454 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f810804c020 con 0x7f81100cd550 2024-09-16T15:38:46.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:46.709+0000 7fd7cea00640 1 -- 172.21.15.60:0/1385754105 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd7c4082a20 con 0x7fd7d80b34b0 2024-09-16T15:38:47.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.118+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdb200b22a0 con 0x7fdb04003a50 2024-09-16T15:38:47.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.119+0000 7fdb17e00640 1 -- 172.21.15.60:0/2741300424 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb0c0872f0 con 0x7fdb04003a50 2024-09-16T15:38:47.120 INFO:teuthology.orchestra.run.smithi060.stdout:60129542151 2024-09-16T15:38:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.124+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdb000788a0 msgr2=0x7fdb0007ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.124+0000 7fdb25a00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdb000788a0 0x7fdb0007ad60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdb201f7e20 tx=0x7fdb1000b260 comp rx=0 tx=0).stop 2024-09-16T15:38:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.124+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb04003a50 msgr2=0x7fdb20155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.124+0000 7fdb25a00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb04003a50 0x7fdb20155ae0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c110 tx=0x7fdb0c003820 comp rx=0 tx=0).stop 2024-09-16T15:38:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.124+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 shutdown_connections 2024-09-16T15:38:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.125+0000 7fdb25a00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdb000788a0 0x7fdb0007ad60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.125+0000 7fdb25a00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb200b1ec0 0x7fdb200b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.125+0000 7fdb25a00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb20154620 0x7fdb200b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.125+0000 7fdb25a00640 1 --2- 172.21.15.60:0/2741300424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb04003a50 0x7fdb20155ae0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.125+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 >> 172.21.15.60:0/2741300424 conn(0x7fdb200d3df0 msgr2=0x7fdb20155360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.125+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 shutdown_connections 2024-09-16T15:38:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.126+0000 7fdb25a00640 1 -- 172.21.15.60:0/2741300424 wait complete. 2024-09-16T15:38:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.132+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd644053eb0 con 0x7fd6440e39e0 2024-09-16T15:38:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.133+0000 7fd648c00640 1 -- 172.21.15.60:0/2498742334 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd640062c30 con 0x7fd6440e39e0 2024-09-16T15:38:47.137 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-09-16T15:38:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.137+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f375c06d120 con 0x7f375c0b4860 2024-09-16T15:38:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd62c0788a0 msgr2=0x7fd62c07ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd62c0788a0 0x7fd62c07ad60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd638033000 tx=0x7fd638004730 comp rx=0 tx=0).stop 2024-09-16T15:38:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440e39e0 msgr2=0x7fd64417d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440e39e0 0x7fd64417d620 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd64000b3f0 tx=0x7fd64000b8b0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 shutdown_connections 2024-09-16T15:38:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd62c0788a0 0x7fd62c07ad60 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd638033000 tx=0x7fd638004730 comp rx=0 tx=0).stop 2024-09-16T15:38:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440503a0 0x7fd644180380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440e39e0 0x7fd64417d620 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 --2- 172.21.15.60:0/2498742334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd64403ab10 0x7fd64416cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.138+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 >> 172.21.15.60:0/2498742334 conn(0x7fd644055540 msgr2=0x7fd6440e5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:47.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.141+0000 7f3752000640 1 -- 172.21.15.60:0/898634385 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f374c062ed0 con 0x7f375c0b4860 2024-09-16T15:38:47.142 INFO:teuthology.orchestra.run.smithi060.stdout:64424509447 2024-09-16T15:38:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.141+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 shutdown_connections 2024-09-16T15:38:47.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.141+0000 7fd652200640 1 -- 172.21.15.60:0/2498742334 wait complete. 2024-09-16T15:38:47.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.144+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f712c1fc4f0 con 0x7f712c0d1610 2024-09-16T15:38:47.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.145+0000 7f7122a00640 1 -- 172.21.15.60:0/954010833 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7114062620 con 0x7f712c0d1610 2024-09-16T15:38:47.147 INFO:teuthology.orchestra.run.smithi060.stdout:64424509445 2024-09-16T15:38:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.148+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f373c078850 msgr2=0x7f373c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.148+0000 7f375b200640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f373c078850 0x7f373c07ad10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3748034000 tx=0x7f3748004730 comp rx=0 tx=0).stop 2024-09-16T15:38:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.148+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c0b4860 msgr2=0x7f375c154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.148+0000 7f375b200640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c0b4860 0x7f375c154f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f374c00c460 tx=0x7f374c00c920 comp rx=0 tx=0).stop 2024-09-16T15:38:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f71080788a0 msgr2=0x7f710807ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f71080788a0 0x7f710807ad60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f711c034000 tx=0x7f711c00b040 comp rx=0 tx=0).stop 2024-09-16T15:38:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f712c0d1610 msgr2=0x7f712c0c7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f712c0d1610 0x7f712c0c7240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f711400c3a0 tx=0x7f711400c860 comp rx=0 tx=0).stop 2024-09-16T15:38:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 shutdown_connections 2024-09-16T15:38:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f71080788a0 0x7f710807ad60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f712c15e0d0 0x7f712c1fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f712c0d2050 0x7f712c0b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 --2- 172.21.15.60:0/954010833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f712c0d1610 0x7f712c0c7240 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.149+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 >> 172.21.15.60:0/954010833 conn(0x7f712c0d3fe0 msgr2=0x7f712c160a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.150+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 shutdown_connections 2024-09-16T15:38:47.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.150+0000 7f7131c00640 1 -- 172.21.15.60:0/954010833 wait complete. 2024-09-16T15:38:47.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.1 2024-09-16T15:38:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.153+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 shutdown_connections 2024-09-16T15:38:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.153+0000 7f375b200640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f373c078850 0x7f373c07ad10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.153+0000 7f375b200640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c0d1fd0 0x7f375c0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.153+0000 7f375b200640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0b5410 0x7f375c1618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.153+0000 7f375b200640 1 --2- 172.21.15.60:0/898634385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c0b4860 0x7f375c154f90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.153+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 >> 172.21.15.60:0/898634385 conn(0x7f375c0d3df0 msgr2=0x7f375c0d23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:47.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.154+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 shutdown_connections 2024-09-16T15:38:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.154+0000 7f375b200640 1 -- 172.21.15.60:0/898634385 wait complete. 2024-09-16T15:38:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.155+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f60680d2610 con 0x7f606815dff0 2024-09-16T15:38:47.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.156+0000 7f605f400640 1 -- 172.21.15.60:0/963972129 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6060063b60 con 0x7f606815dff0 2024-09-16T15:38:47.159 INFO:teuthology.orchestra.run.smithi060.stdout:64424509445 2024-09-16T15:38:47.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.158+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f811015a9f0 con 0x7f81100cd550 2024-09-16T15:38:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6048078920 msgr2=0x7f604807ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6048078920 0x7f604807ade0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6058033000 tx=0x7f6058004730 comp rx=0 tx=0).stop 2024-09-16T15:38:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815dff0 msgr2=0x7f606815b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815dff0 0x7f606815b300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f606000cce0 tx=0x7f606000a620 comp rx=0 tx=0).stop 2024-09-16T15:38:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 shutdown_connections 2024-09-16T15:38:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6048078920 0x7f604807ade0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f606815dff0 0x7f606815b300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f8106000640 1 -- 172.21.15.60:0/2033417454 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8108083bc0 con 0x7f81100cd550 2024-09-16T15:38:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60680ceda0 0x7f606815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 --2- 172.21.15.60:0/963972129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60680b0eb0 0x7f60680c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.159+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 >> 172.21.15.60:0/963972129 conn(0x7f60680d3de0 msgr2=0x7f60680cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.160+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 shutdown_connections 2024-09-16T15:38:47.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.160+0000 7f606de00640 1 -- 172.21.15.60:0/963972129 wait complete. 2024-09-16T15:38:47.164 INFO:teuthology.orchestra.run.smithi060.stdout:64424509447 2024-09-16T15:38:47.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2024-09-16T15:38:47.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f80f0078920 msgr2=0x7f80f007ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f80f0078920 0x7f80f007ade0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0046b0 tx=0x7f80fc0012b0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100cd550 msgr2=0x7f81101516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100cd550 0x7f81101516a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8108009ca0 tx=0x7f8108033a50 comp rx=0 tx=0).stop 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 shutdown_connections 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f80f0078920 0x7f80f007ade0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0046b0 tx=0x7f80fc0012b0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8110162270 0x7f81101fcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015bbd0 0x7f81101fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 --2- 172.21.15.60:0/2033417454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100cd550 0x7f81101516a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 >> 172.21.15.60:0/2033417454 conn(0x7f81100b1950 msgr2=0x7f81100cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:47.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 shutdown_connections 2024-09-16T15:38:47.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.163+0000 7f8114e00640 1 -- 172.21.15.60:0/2033417454 wait complete. 2024-09-16T15:38:47.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.164+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3c70005850 con 0x7f3c981622d0 2024-09-16T15:38:47.167 INFO:teuthology.orchestra.run.smithi060.stdout:64424509445 2024-09-16T15:38:47.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.165+0000 7f3c8f400640 1 -- 172.21.15.60:0/2158897480 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c90083520 con 0x7f3c981622d0 2024-09-16T15:38:47.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.167+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd7b0005850 con 0x7fd7d80b34b0 2024-09-16T15:38:47.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.168+0000 7fd7cea00640 1 -- 172.21.15.60:0/1385754105 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd7c4087450 con 0x7fd7d80b34b0 2024-09-16T15:38:47.169 INFO:teuthology.orchestra.run.smithi060.stdout:64424509447 2024-09-16T15:38:47.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.168+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3c74078c50 msgr2=0x7f3c7407b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.168+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3c74078c50 0x7f3c7407b110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3c88000f60 tx=0x7f3c88009040 comp rx=0 tx=0).stop 2024-09-16T15:38:47.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.168+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c981622d0 msgr2=0x7f3c980ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.168+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c981622d0 0x7f3c980ccfa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009d60 tx=0x7f3c90003ba0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.169+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 shutdown_connections 2024-09-16T15:38:47.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.169+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3c74078c50 0x7f3c7407b110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.169+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c981622d0 0x7f3c980ccfa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.169+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c981613a0 0x7f3c9815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.169+0000 7f3c9ce00640 1 --2- 172.21.15.60:0/2158897480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b4a50 0x7f3c9815b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.169+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 >> 172.21.15.60:0/2158897480 conn(0x7f3c980d3dc0 msgr2=0x7f3c980d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:47.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.169+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 shutdown_connections 2024-09-16T15:38:47.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.169+0000 7f3c9ce00640 1 -- 172.21.15.60:0/2158897480 wait complete. 2024-09-16T15:38:47.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd7b8078a50 msgr2=0x7fd7b807af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd7b8078a50 0x7fd7b807af10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80c6ce0 tx=0x7fd7c8009040 comp rx=0 tx=0).stop 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d80b34b0 msgr2=0x7fd7d80b90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d80b34b0 0x7fd7d80b90e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4034040 tx=0x7fd7c4003960 comp rx=0 tx=0).stop 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 shutdown_connections 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd7b8078a50 0x7fd7b807af10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d8161440 0x7fd7d81fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d80b4050 0x7fd7d80c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 --2- 172.21.15.60:0/1385754105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d80b34b0 0x7fd7d80b90e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 >> 172.21.15.60:0/1385754105 conn(0x7fd7d80d3df0 msgr2=0x7fd7d815e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:47.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.171+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 shutdown_connections 2024-09-16T15:38:47.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:47.172+0000 7fd7d7e00640 1 -- 172.21.15.60:0/1385754105 wait complete. 2024-09-16T15:38:47.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-09-16T15:38:47.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.0 2024-09-16T15:38:47.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-16T15:38:47.188 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.6 2024-09-16T15:38:47.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-09-16T15:38:47.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.7 2024-09-16T15:38:48.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T15:38:48.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T15:38:48.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T15:38:48.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T15:38:48.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T15:38:48.188 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T15:38:48.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T15:38:48.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T15:38:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.414+0000 7f7860a00640 1 -- 172.21.15.60:0/377513956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854005a60 msgr2=0x7f7854035fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.414+0000 7f7860a00640 1 --2- 172.21.15.60:0/377513956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854005a60 0x7f7854035fe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f785c052430 tx=0x7f785c0a6f30 comp rx=0 tx=0).stop 2024-09-16T15:38:48.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.416+0000 7f7860a00640 1 -- 172.21.15.60:0/377513956 shutdown_connections 2024-09-16T15:38:48.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.416+0000 7f7860a00640 1 --2- 172.21.15.60:0/377513956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854005a60 0x7f7854035fe0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.416+0000 7f7860a00640 1 --2- 172.21.15.60:0/377513956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f785401c890 0x7f785401cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.416+0000 7f7860a00640 1 -- 172.21.15.60:0/377513956 >> 172.21.15.60:0/377513956 conn(0x7f7854054560 msgr2=0x7f7854054940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.416+0000 7f7860a00640 1 -- 172.21.15.60:0/377513956 shutdown_connections 2024-09-16T15:38:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.416+0000 7f7860a00640 1 -- 172.21.15.60:0/377513956 wait complete. 2024-09-16T15:38:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.419+0000 7f7860a00640 1 Processor -- start 2024-09-16T15:38:48.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.419+0000 7f7860a00640 1 -- start start 2024-09-16T15:38:48.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.421+0000 7f7860a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854005a60 0x7f7854048a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.422+0000 7f7860a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f785401c890 0x7f785404a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.422+0000 7f7860a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7854036520 0x7f7854050d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.423+0000 7f7860a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f785400bb80 con 0x7f7854005a60 2024-09-16T15:38:48.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.423+0000 7f7860a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f785400ba00 con 0x7f7854036520 2024-09-16T15:38:48.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.423+0000 7f7860a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f785400aa60 con 0x7f785401c890 2024-09-16T15:38:48.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.423+0000 7f785be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7854036520 0x7f7854050d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.423+0000 7f785be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7854036520 0x7f7854050d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:57042/0 (socket says 172.21.15.60:57042) 2024-09-16T15:38:48.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.423+0000 7f785be00640 1 -- 172.21.15.60:0/48150385 learned_addr learned my addr 172.21.15.60:0/48150385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:48.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.423+0000 7f785aa00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f785401c890 0x7f785404a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.424+0000 7f785be00640 1 -- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f785401c890 msgr2=0x7f785404a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.424+0000 7f785be00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f785401c890 0x7f785404a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.424+0000 7f785be00640 1 -- 172.21.15.60:0/48150385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854005a60 msgr2=0x7f7854048a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.424+0000 7f785be00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854005a60 0x7f7854048a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.424+0000 7f785be00640 1 -- 172.21.15.60:0/48150385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7854052c20 con 0x7f7854036520 2024-09-16T15:38:48.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.424+0000 7f785be00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7854036520 0x7f7854050d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f785000c3c0 tx=0x7f785000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.425+0000 7f784f400640 1 -- 172.21.15.60:0/48150385 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850017070 con 0x7f7854036520 2024-09-16T15:38:48.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.425+0000 7f784f400640 1 -- 172.21.15.60:0/48150385 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78500053c0 con 0x7f7854036520 2024-09-16T15:38:48.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.425+0000 7f784f400640 1 -- 172.21.15.60:0/48150385 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850003810 con 0x7f7854036520 2024-09-16T15:38:48.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.425+0000 7f7860a00640 1 -- 172.21.15.60:0/48150385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78540e1180 con 0x7f7854036520 2024-09-16T15:38:48.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.426+0000 7f7860a00640 1 -- 172.21.15.60:0/48150385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7854048f50 con 0x7f7854036520 2024-09-16T15:38:48.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.429+0000 7f784cc00640 1 -- 172.21.15.60:0/48150385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7854035fe0 con 0x7f7854036520 2024-09-16T15:38:48.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.430+0000 7f784f400640 1 -- 172.21.15.60:0/48150385 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f785001a070 con 0x7f7854036520 2024-09-16T15:38:48.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.432+0000 7f784f400640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f783c078880 0x7f783c07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.433+0000 7f784f400640 1 -- 172.21.15.60:0/48150385 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7850099b20 con 0x7f7854036520 2024-09-16T15:38:48.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.433+0000 7f785b400640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f783c078880 0x7f783c07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.434+0000 7f785b400640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f783c078880 0x7f783c07ad40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f784800b9f0 tx=0x7f784800b3d0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.436+0000 7f784f400640 1 -- 172.21.15.60:0/48150385 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7850061b30 con 0x7f7854036520 2024-09-16T15:38:48.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.498+0000 7fd1a5000640 1 -- 172.21.15.60:0/1899859352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00b4ca0 msgr2=0x7fd1a00b5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.498+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1899859352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00b4ca0 0x7fd1a00b5840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd190005fd0 tx=0x7fd19002a8a0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.499+0000 7fd1a5000640 1 -- 172.21.15.60:0/1899859352 shutdown_connections 2024-09-16T15:38:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.499+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1899859352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00b4ca0 0x7fd1a00b5840 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.499+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1899859352 >> v2:172.21.15.60:3300/0 conn(0x7fd1a00d2580 0x7fd1a00d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.499+0000 7fd1a5000640 1 -- 172.21.15.60:0/1899859352 >> 172.21.15.60:0/1899859352 conn(0x7fd1a00d3dc0 msgr2=0x7fd1a00d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.499+0000 7f3510a00640 1 -- 172.21.15.60:0/1358837105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3504037940 msgr2=0x7f35040e39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.499+0000 7fd1a5000640 1 -- 172.21.15.60:0/1899859352 shutdown_connections 2024-09-16T15:38:48.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.499+0000 7f3510a00640 1 --2- 172.21.15.60:0/1358837105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3504037940 0x7f35040e39e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f350c052430 tx=0x7f350c0a5e30 comp rx=0 tx=0).stop 2024-09-16T15:38:48.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.500+0000 7fd1a5000640 1 -- 172.21.15.60:0/1899859352 wait complete. 2024-09-16T15:38:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.500+0000 7f1b90400640 1 -- 172.21.15.60:0/4264514568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b880d0290 msgr2=0x7f1b8806e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.501+0000 7f1b90400640 1 --2- 172.21.15.60:0/4264514568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b880d0290 0x7f1b8806e7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b880d0a10 tx=0x7f1b7c02a490 comp rx=0 tx=0).stop 2024-09-16T15:38:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.501+0000 7f3510a00640 1 -- 172.21.15.60:0/1358837105 shutdown_connections 2024-09-16T15:38:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.501+0000 7f3510a00640 1 --2- 172.21.15.60:0/1358837105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3504037940 0x7f35040e39e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.501+0000 7f3510a00640 1 -- 172.21.15.60:0/1358837105 >> 172.21.15.60:0/1358837105 conn(0x7f3504055540 msgr2=0x7f3504055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.501+0000 7f3510a00640 1 -- 172.21.15.60:0/1358837105 shutdown_connections 2024-09-16T15:38:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.501+0000 7f3510a00640 1 -- 172.21.15.60:0/1358837105 wait complete. 2024-09-16T15:38:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.501+0000 7f8649400640 1 -- 172.21.15.60:0/3754841694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 msgr2=0x7f863c056c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.501+0000 7f8649400640 1 --2- 172.21.15.60:0/3754841694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 0x7f863c056c30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f864406bb30 tx=0x7f86440a5e20 comp rx=0 tx=0).stop 2024-09-16T15:38:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.502+0000 7f1b90400640 1 -- 172.21.15.60:0/4264514568 shutdown_connections 2024-09-16T15:38:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.503+0000 7fd1a5000640 1 Processor -- start 2024-09-16T15:38:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.503+0000 7f8649400640 1 -- 172.21.15.60:0/3754841694 shutdown_connections 2024-09-16T15:38:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.503+0000 7f8649400640 1 --2- 172.21.15.60:0/3754841694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 0x7f863c056c30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.503+0000 7f8649400640 1 --2- 172.21.15.60:0/3754841694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f863c0e2790 0x7f863c0e6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.503+0000 7f8649400640 1 -- 172.21.15.60:0/3754841694 >> 172.21.15.60:0/3754841694 conn(0x7f863c03a3c0 msgr2=0x7f863c03a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.503+0000 7fd1a5000640 1 -- start start 2024-09-16T15:38:48.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.503+0000 7f8649400640 1 -- 172.21.15.60:0/3754841694 shutdown_connections 2024-09-16T15:38:48.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7f3510a00640 1 Processor -- start 2024-09-16T15:38:48.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.503+0000 7f8649400640 1 -- 172.21.15.60:0/3754841694 wait complete. 2024-09-16T15:38:48.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.502+0000 7f1b90400640 1 --2- 172.21.15.60:0/4264514568 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b880d0290 0x7f1b8806e7d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7f1b90400640 1 --2- 172.21.15.60:0/4264514568 >> v2:172.21.15.100:3300/0 conn(0x7f1b8813d200 0x7f1b880cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7f1b90400640 1 --2- 172.21.15.60:0/4264514568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b880cf080 0x7f1b880b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7f1b90400640 1 -- 172.21.15.60:0/4264514568 >> 172.21.15.60:0/4264514568 conn(0x7f1b8806cd50 msgr2=0x7f1b8806d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7f3510a00640 1 -- start start 2024-09-16T15:38:48.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7f1b90400640 1 -- 172.21.15.60:0/4264514568 shutdown_connections 2024-09-16T15:38:48.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7fd1a5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1a00b4ca0 0x7fd1a00b9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7fd1a5000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00d2580 0x7fd1a00b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7f1b90400640 1 -- 172.21.15.60:0/4264514568 wait complete. 2024-09-16T15:38:48.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7fd1a5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a015bf70 0x7fd1a00d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7fd1a5000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a00cec20 con 0x7fd1a00b4ca0 2024-09-16T15:38:48.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7fd1a5000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a00d2b70 con 0x7fd1a015bf70 2024-09-16T15:38:48.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.504+0000 7fd1a5000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a00ceda0 con 0x7fd1a00d2580 2024-09-16T15:38:48.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7efcebc00640 1 -- 172.21.15.60:0/435598285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a4d0 msgr2=0x7efcdc035fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7efcebc00640 1 --2- 172.21.15.60:0/435598285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a4d0 0x7efcdc035fd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00d310 tx=0x7efcd002a650 comp rx=0 tx=0).stop 2024-09-16T15:38:48.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7fd19e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a015bf70 0x7fd1a00d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7efcebc00640 1 -- 172.21.15.60:0/435598285 shutdown_connections 2024-09-16T15:38:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7fd19e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a015bf70 0x7fd1a00d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:57110/0 (socket says 172.21.15.60:57110) 2024-09-16T15:38:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7fd19e800640 1 -- 172.21.15.60:0/1130416235 learned_addr learned my addr 172.21.15.60:0/1130416235 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7fd19d400640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00d2580 0x7fd1a00b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7efcebc00640 1 --2- 172.21.15.60:0/435598285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a4d0 0x7efcdc035fd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7efcebc00640 1 --2- 172.21.15.60:0/435598285 >> v2:172.21.15.100:3300/0 conn(0x7efcdc00a0f0 0x7efcdc006c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.505+0000 7efcebc00640 1 -- 172.21.15.60:0/435598285 >> 172.21.15.60:0/435598285 conn(0x7efcdc054550 msgr2=0x7efcdc054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7fd19de00640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1a00b4ca0 0x7fd1a00b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7fd19e800640 1 -- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00d2580 msgr2=0x7fd1a00b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7fd8d4000640 1 -- 172.21.15.60:0/83750365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c4037680 msgr2=0x7fd8c4035fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7fd8d4000640 1 --2- 172.21.15.60:0/83750365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c4037680 0x7fd8c4035fd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8005fd0 tx=0x7fd8b802a670 comp rx=0 tx=0).stop 2024-09-16T15:38:48.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7efcebc00640 1 -- 172.21.15.60:0/435598285 shutdown_connections 2024-09-16T15:38:48.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7f8649400640 1 Processor -- start 2024-09-16T15:38:48.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd19e800640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00d2580 0x7fd1a00b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd19e800640 1 -- 172.21.15.60:0/1130416235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1a00b4ca0 msgr2=0x7fd1a00b9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd19e800640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1a00b4ca0 0x7fd1a00b9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd8d4000640 1 -- 172.21.15.60:0/83750365 shutdown_connections 2024-09-16T15:38:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd8d4000640 1 --2- 172.21.15.60:0/83750365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c4037680 0x7fd8c4035fd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd8d4000640 1 -- 172.21.15.60:0/83750365 >> 172.21.15.60:0/83750365 conn(0x7fd8c4054550 msgr2=0x7fd8c4054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd19e800640 1 -- 172.21.15.60:0/1130416235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a00b43b0 con 0x7fd1a015bf70 2024-09-16T15:38:48.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd8d4000640 1 -- 172.21.15.60:0/83750365 shutdown_connections 2024-09-16T15:38:48.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7f3510a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3504037940 0x7f35040494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7f3510a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350403ab10 0x7f3504051d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7f3510a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35040e2a60 0x7f35040476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7f3510a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f35040539f0 con 0x7f35040e2a60 2024-09-16T15:38:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd8d4000640 1 -- 172.21.15.60:0/83750365 wait complete. 2024-09-16T15:38:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7f3510a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3504053870 con 0x7f350403ab10 2024-09-16T15:38:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.506+0000 7f3510a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3504053b70 con 0x7f3504037940 2024-09-16T15:38:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd19de00640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1a00b4ca0 0x7fd1a00b9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7fd19d400640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00d2580 0x7fd1a00b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7efcebc00640 1 -- 172.21.15.60:0/435598285 wait complete. 2024-09-16T15:38:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7f8649400640 1 -- start start 2024-09-16T15:38:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.507+0000 7f1b90400640 1 Processor -- start 2024-09-16T15:38:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.508+0000 7fd19e800640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a015bf70 0x7fd1a00d03f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c390 tx=0x7fd18800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.508+0000 7f350b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35040e2a60 0x7f35040476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.508+0000 7f350b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35040e2a60 0x7f35040476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35580/0 (socket says 172.21.15.60:35580) 2024-09-16T15:38:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.508+0000 7f1b90400640 1 -- start start 2024-09-16T15:38:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.508+0000 7fd196000640 1 -- 172.21.15.60:0/1130416235 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188017070 con 0x7fd1a015bf70 2024-09-16T15:38:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.508+0000 7fd196000640 1 -- 172.21.15.60:0/1130416235 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1880053e0 con 0x7fd1a015bf70 2024-09-16T15:38:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.508+0000 7fd196000640 1 -- 172.21.15.60:0/1130416235 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188003850 con 0x7fd1a015bf70 2024-09-16T15:38:48.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a00cb8b0 con 0x7fd1a015bf70 2024-09-16T15:38:48.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a00cbe70 con 0x7fd1a015bf70 2024-09-16T15:38:48.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7f350b200640 1 -- 172.21.15.60:0/3572468134 learned_addr learned my addr 172.21.15.60:0/3572468134 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:48.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7f3509e00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350403ab10 0x7f3504051d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.508+0000 7f350a800640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3504037940 0x7f35040494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7f1b90400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b880cf080 0x7f1b881f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7f1b90400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b880d0290 0x7f1b88152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7f1b90400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8813d200 0x7f1b880ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7f1b90400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1b880cf5d0 con 0x7f1b880d0290 2024-09-16T15:38:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7f1b90400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b880b3790 con 0x7f1b880cf080 2024-09-16T15:38:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7f1b90400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1b880706f0 con 0x7f1b8813d200 2024-09-16T15:38:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f6f0ca00640 1 -- 172.21.15.60:0/284697921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f000e27a0 msgr2=0x7f6f000e6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f6f0ca00640 1 --2- 172.21.15.60:0/284697921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f000e27a0 0x7f6f000e6430 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005fd0 tx=0x7f6efc02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7fd8d4000640 1 Processor -- start 2024-09-16T15:38:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f8649400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c037f00 0x7f863c04f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f3509e00640 1 -- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3504037940 msgr2=0x7f35040494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f8649400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 0x7f863c050fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f6f0ca00640 1 -- 172.21.15.60:0/284697921 shutdown_connections 2024-09-16T15:38:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f6f0ca00640 1 --2- 172.21.15.60:0/284697921 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f000e27a0 0x7f6f000e6430 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f6f0ca00640 1 -- 172.21.15.60:0/284697921 >> 172.21.15.60:0/284697921 conn(0x7f6f0003a3d0 msgr2=0x7f6f0003a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f8649400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f863c0e2790 0x7f863c0e0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f1b8e400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8813d200 0x7f1b880ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f1b8e400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8813d200 0x7f1b880ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:53292/0 (socket says 172.21.15.60:53292) 2024-09-16T15:38:48.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f1b8e400640 1 -- 172.21.15.60:0/4063547949 learned_addr learned my addr 172.21.15.60:0/4063547949 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f8649400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f863c057540 con 0x7f863c037f00 2024-09-16T15:38:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7f8649400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f863c0573c0 con 0x7f863c03b930 2024-09-16T15:38:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f8643200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f863c0e2790 0x7f863c0e0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f8643200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f863c0e2790 0x7f863c0e0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:53300/0 (socket says 172.21.15.60:53300) 2024-09-16T15:38:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f8641e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 0x7f863c050fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.509+0000 7efcebc00640 1 Processor -- start 2024-09-16T15:38:48.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f8643200640 1 -- 172.21.15.60:0/517331168 learned_addr learned my addr 172.21.15.60:0/517331168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:48.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f863c0576c0 con 0x7f863c0e2790 2024-09-16T15:38:48.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.510+0000 7fd8d4000640 1 -- start start 2024-09-16T15:38:48.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f1b8da00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b880cf080 0x7f1b881f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f3509e00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3504037940 0x7f35040494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7efcebc00640 1 -- start start 2024-09-16T15:38:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f6f0ca00640 1 -- 172.21.15.60:0/284697921 shutdown_connections 2024-09-16T15:38:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f1b8d000640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b880d0290 0x7f1b88152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f8642800640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c037f00 0x7f863c04f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f1b8e400640 1 -- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b880cf080 msgr2=0x7f1b881f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f1b8e400640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b880cf080 0x7f1b881f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f1b8e400640 1 -- 172.21.15.60:0/4063547949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b880d0290 msgr2=0x7f1b88152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f1b8e400640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b880d0290 0x7f1b88152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f1b8e400640 1 -- 172.21.15.60:0/4063547949 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b88153760 con 0x7f1b8813d200 2024-09-16T15:38:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d4000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c4009e00 0x7fd8c404a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d4000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c4036510 0x7fd8c404a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d4000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c4037680 0x7fd8c4180d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f3509e00640 1 -- 172.21.15.60:0/3572468134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35040e2a60 msgr2=0x7f35040476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f3509e00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35040e2a60 0x7f35040476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d4000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c40064f0 con 0x7fd8c4009e00 2024-09-16T15:38:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d4000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c4006370 con 0x7fd8c4037680 2024-09-16T15:38:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d4000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c4006670 con 0x7fd8c4036510 2024-09-16T15:38:48.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f3509e00640 1 -- 172.21.15.60:0/3572468134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35040dd540 con 0x7f350403ab10 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f350b200640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35040e2a60 0x7f35040476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f350a800640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3504037940 0x7f35040494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f8641e00640 1 -- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f863c0e2790 msgr2=0x7f863c0e0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f8641e00640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f863c0e2790 0x7f863c0e0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f8641e00640 1 -- 172.21.15.60:0/517331168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c037f00 msgr2=0x7f863c04f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f8641e00640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c037f00 0x7f863c04f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f8641e00640 1 -- 172.21.15.60:0/517331168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c0de400 con 0x7f863c03b930 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f8642800640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c037f00 0x7f863c04f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:48.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f3509e00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350403ab10 0x7f3504051d60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34f800c3e0 tx=0x7f34f800c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c4009e00 0x7fd8c404a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c4009e00 0x7fd8c404a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35614/0 (socket says 172.21.15.60:35614) 2024-09-16T15:38:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7fd8d1600640 1 -- 172.21.15.60:0/2187353821 learned_addr learned my addr 172.21.15.60:0/2187353821 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.512+0000 7f1b8e400640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8813d200 0x7f1b880ae770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c02aa80 tx=0x7f1b7c032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f3502a00640 1 -- 172.21.15.60:0/3572468134 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8017070 con 0x7f350403ab10 2024-09-16T15:38:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f3502a00640 1 -- 172.21.15.60:0/3572468134 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f80035f0 con 0x7f350403ab10 2024-09-16T15:38:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f3502a00640 1 -- 172.21.15.60:0/3572468134 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8003a40 con 0x7f350403ab10 2024-09-16T15:38:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3504049c60 con 0x7f350403ab10 2024-09-16T15:38:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a00d48e0 con 0x7fd1a015bf70 2024-09-16T15:38:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f8641e00640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 0x7f863c050fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f863000c3c0 tx=0x7f863000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350404a220 con 0x7f350403ab10 2024-09-16T15:38:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f863aa00640 1 -- 172.21.15.60:0/517331168 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630017070 con 0x7f863c03b930 2024-09-16T15:38:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f1b85600640 1 -- 172.21.15.60:0/4063547949 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c005020 con 0x7f1b8813d200 2024-09-16T15:38:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7fd196000640 1 -- 172.21.15.60:0/1130416235 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd18801a070 con 0x7fd1a015bf70 2024-09-16T15:38:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c0de740 con 0x7f863c03b930 2024-09-16T15:38:48.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f863aa00640 1 -- 172.21.15.60:0/517331168 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86300053c0 con 0x7f863c03b930 2024-09-16T15:38:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f863aa00640 1 -- 172.21.15.60:0/517331168 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630017070 con 0x7f863c03b930 2024-09-16T15:38:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.514+0000 7fd8d0c00640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c4036510 0x7fd8c404a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.514+0000 7fd8d2000640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c4037680 0x7fd8c4180d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f1b85600640 1 -- 172.21.15.60:0/4063547949 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b7c0051e0 con 0x7f1b8813d200 2024-09-16T15:38:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f1b85600640 1 -- 172.21.15.60:0/4063547949 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c005630 con 0x7f1b8813d200 2024-09-16T15:38:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f1b90400640 1 -- 172.21.15.60:0/4063547949 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b88153a40 con 0x7f1b8813d200 2024-09-16T15:38:48.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.514+0000 7f1b90400640 1 -- 172.21.15.60:0/4063547949 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b881f4610 con 0x7f1b8813d200 2024-09-16T15:38:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.513+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c04caa0 con 0x7f863c03b930 2024-09-16T15:38:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7fd8d1600640 1 -- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c4036510 msgr2=0x7fd8c404a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.511+0000 7f6f0ca00640 1 -- 172.21.15.60:0/284697921 wait complete. 2024-09-16T15:38:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.514+0000 7f6f0ca00640 1 Processor -- start 2024-09-16T15:38:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c04f660 con 0x7f863c03b930 2024-09-16T15:38:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.516+0000 7f6f0ca00640 1 -- start start 2024-09-16T15:38:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7efcebc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a0f0 0x7efcdc17cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7efcebc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc00a4d0 0x7efcdc17cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.516+0000 7fd196000640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd17c078950 0x7fd17c07ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7efcebc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc036510 0x7efcdc0d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.516+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3504053fc0 con 0x7f350403ab10 2024-09-16T15:38:48.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.516+0000 7fd196000640 1 -- 172.21.15.60:0/1130416235 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd188099cb0 con 0x7fd1a015bf70 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7fd8d1600640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c4036510 0x7fd8c404a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7fd8d1600640 1 -- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c4037680 msgr2=0x7fd8c4180d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7fd8d1600640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c4037680 0x7fd8c4180d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7fd8d1600640 1 -- 172.21.15.60:0/2187353821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c417eb40 con 0x7fd8c4009e00 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7fd8d1600640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c4009e00 0x7fd8c404a1c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000c430 tx=0x7fd8c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7fd8c9600640 1 -- 172.21.15.60:0/2187353821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0017070 con 0x7fd8c4009e00 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.515+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c417ee20 con 0x7fd8c4009e00 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.516+0000 7fd8c9600640 1 -- 172.21.15.60:0/2187353821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8c00052d0 con 0x7fd8c4009e00 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.516+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c417f3e0 con 0x7fd8c4009e00 2024-09-16T15:38:48.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.516+0000 7fd8c9600640 1 -- 172.21.15.60:0/2187353821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0017070 con 0x7fd8c4009e00 2024-09-16T15:38:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7f1b7aa00640 1 -- 172.21.15.60:0/4063547949 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b881f14e0 con 0x7f1b8813d200 2024-09-16T15:38:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7f1b85600640 1 -- 172.21.15.60:0/4063547949 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1b7c0047a0 con 0x7f1b8813d200 2024-09-16T15:38:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7fd19de00640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd17c078950 0x7fd17c07ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7efce9200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a0f0 0x7efcdc17cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7efce9200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a0f0 0x7efcdc17cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35624/0 (socket says 172.21.15.60:35624) 2024-09-16T15:38:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7efce9200640 1 -- 172.21.15.60:0/3665759536 learned_addr learned my addr 172.21.15.60:0/3665759536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7efcebc00640 1 -- 172.21.15.60:0/3665759536 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc038a30 con 0x7efcdc00a0f0 2024-09-16T15:38:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7efce9c00640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc036510 0x7efcdc0d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7efce3e00640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc00a4d0 0x7efcdc17cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7efcebc00640 1 -- 172.21.15.60:0/3665759536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc0388b0 con 0x7efcdc00a4d0 2024-09-16T15:38:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.517+0000 7efcebc00640 1 -- 172.21.15.60:0/3665759536 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7efcdc006160 con 0x7efcdc036510 2024-09-16T15:38:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c4006b20 con 0x7fd8c4009e00 2024-09-16T15:38:48.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7f6f0ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f00037f10 0x7f6f000d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7f6f0ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f0003b940 0x7f6f00031a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7f6f0ca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f000e27a0 0x7f6f00038ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7efce9c00640 1 -- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc00a4d0 msgr2=0x7efcdc17cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7efce9c00640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc00a4d0 0x7efcdc17cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7efce9c00640 1 -- 172.21.15.60:0/3665759536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a0f0 msgr2=0x7efcdc17cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7efce9c00640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a0f0 0x7efcdc17cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7efce9c00640 1 -- 172.21.15.60:0/3665759536 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc17d880 con 0x7efcdc036510 2024-09-16T15:38:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7f6f0ca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6f00057820 con 0x7f6f0003b940 2024-09-16T15:38:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7f6f0ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f000576a0 con 0x7f6f00037f10 2024-09-16T15:38:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.518+0000 7f6f0ca00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6f000579a0 con 0x7f6f000e27a0 2024-09-16T15:38:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.519+0000 7f6f07e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f000e27a0 0x7f6f00038ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.519+0000 7f6f07e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f000e27a0 0x7f6f00038ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:53324/0 (socket says 172.21.15.60:53324) 2024-09-16T15:38:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.519+0000 7f6f07e00640 1 -- 172.21.15.60:0/1442152465 learned_addr learned my addr 172.21.15.60:0/1442152465 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.519+0000 7f6f07400640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f00037f10 0x7f6f000d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.519+0000 7efce9c00640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc036510 0x7efcdc0d7ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efce4069f40 tx=0x7efce406aed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.519+0000 7f1b85600640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b68078920 0x7f1b6807ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.519+0000 7f1b85600640 1 -- 172.21.15.60:0/4063547949 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1b7c0bcd50 con 0x7f1b8813d200 2024-09-16T15:38:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7f1b8da00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b68078920 0x7f1b6807ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7f6f07e00640 1 -- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f00037f10 msgr2=0x7f6f000d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7f6f07e00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f00037f10 0x7f6f000d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7f6f07e00640 1 -- 172.21.15.60:0/1442152465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f0003b940 msgr2=0x7f6f00031a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7f6f06a00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f0003b940 0x7f6f00031a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7f6f07e00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f0003b940 0x7f6f00031a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7f6f07e00640 1 -- 172.21.15.60:0/1442152465 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f00179920 con 0x7f6f000e27a0 2024-09-16T15:38:48.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7fd196000640 1 -- 172.21.15.60:0/1130416235 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd188062270 con 0x7fd1a015bf70 2024-09-16T15:38:48.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7fd8c9600640 1 -- 172.21.15.60:0/2187353821 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd8c001a070 con 0x7fd8c4009e00 2024-09-16T15:38:48.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.521+0000 7f6f06a00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f0003b940 0x7f6f00031a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:48.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.521+0000 7f6f07e00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f000e27a0 0x7f6f00038ad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f0802eb50 tx=0x7f6f0806a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.521+0000 7f1b8da00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b68078920 0x7f1b6807ade0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009f20 tx=0x7f1b800094f0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.520+0000 7fd19de00640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd17c078950 0x7fd17c07ae10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd198005fd0 tx=0x7fd198009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.522+0000 7fd8c9600640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8ac078a50 0x7fd8ac07af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.522+0000 7fd8c9600640 1 -- 172.21.15.60:0/2187353821 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd8c0099ca0 con 0x7fd8c4009e00 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.522+0000 7fd8d0c00640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8ac078a50 0x7fd8ac07af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.523+0000 7fd8d0c00640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8ac078a50 0x7fd8ac07af10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8033000 tx=0x7fd8b8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7f6efb400640 1 -- 172.21.15.60:0/1442152465 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0808f070 con 0x7f6f000e27a0 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7f6efb400640 1 -- 172.21.15.60:0/1442152465 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f0806de50 con 0x7f6f000e27a0 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7f6efb400640 1 -- 172.21.15.60:0/1442152465 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08072890 con 0x7f6f000e27a0 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f00179ee0 con 0x7f6f000e27a0 2024-09-16T15:38:48.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f000d2190 con 0x7f6f000e27a0 2024-09-16T15:38:48.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7f3502a00640 1 -- 172.21.15.60:0/3572468134 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f34f801a070 con 0x7f350403ab10 2024-09-16T15:38:48.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7f1b85600640 1 -- 172.21.15.60:0/4063547949 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1b7c085430 con 0x7f1b8813d200 2024-09-16T15:38:48.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7efce0c00640 1 -- 172.21.15.60:0/3665759536 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce408f070 con 0x7efcdc036510 2024-09-16T15:38:48.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.525+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f00057ea0 con 0x7f6f000e27a0 2024-09-16T15:38:48.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.524+0000 7f863aa00640 1 -- 172.21.15.60:0/517331168 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f863001a070 con 0x7f863c03b930 2024-09-16T15:38:48.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.526+0000 7f863aa00640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8624078a80 0x7f862407af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.526+0000 7f863aa00640 1 -- 172.21.15.60:0/517331168 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8630099a70 con 0x7f863c03b930 2024-09-16T15:38:48.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.525+0000 7efce0c00640 1 -- 172.21.15.60:0/3665759536 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efce4074720 con 0x7efcdc036510 2024-09-16T15:38:48.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.525+0000 7efce0c00640 1 -- 172.21.15.60:0/3665759536 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4074bb0 con 0x7efcdc036510 2024-09-16T15:38:48.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.525+0000 7efcebc00640 1 -- 172.21.15.60:0/3665759536 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc0e48e0 con 0x7efcdc036510 2024-09-16T15:38:48.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.525+0000 7efcebc00640 1 -- 172.21.15.60:0/3665759536 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc17a790 con 0x7efcdc036510 2024-09-16T15:38:48.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.526+0000 7f3502a00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34ec078920 0x7f34ec07ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.527+0000 7f3502a00640 1 -- 172.21.15.60:0/3572468134 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f34f809a0f0 con 0x7f350403ab10 2024-09-16T15:38:48.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.527+0000 7f3502a00640 1 -- 172.21.15.60:0/3572468134 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f34f8021020 con 0x7f350403ab10 2024-09-16T15:38:48.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.527+0000 7f350a800640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34ec078920 0x7f34ec07ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.528+0000 7f350a800640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34ec078920 0x7f34ec07ade0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f34fc001670 tx=0x7f34fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.529+0000 7f6efb400640 1 -- 172.21.15.60:0/1442152465 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6f08072a30 con 0x7f6f000e27a0 2024-09-16T15:38:48.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.529+0000 7f8642800640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8624078a80 0x7f862407af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.529+0000 7f8642800640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8624078a80 0x7f862407af40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8634005fd0 tx=0x7f8634009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.528+0000 7efcd6000640 1 -- 172.21.15.60:0/3665759536 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb8005290 con 0x7efcdc036510 2024-09-16T15:38:48.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.530+0000 7fd8c9600640 1 -- 172.21.15.60:0/2187353821 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8c0021020 con 0x7fd8c4009e00 2024-09-16T15:38:48.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.530+0000 7f6efb400640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6ee8078850 0x7f6ee807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.531+0000 7f6efb400640 1 -- 172.21.15.60:0/1442152465 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6f08115720 con 0x7f6f000e27a0 2024-09-16T15:38:48.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.531+0000 7f6f07400640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6ee8078850 0x7f6ee807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.531+0000 7f863aa00640 1 -- 172.21.15.60:0/517331168 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f863c04f660 con 0x7f863c03b930 2024-09-16T15:38:48.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.532+0000 7f6efb400640 1 -- 172.21.15.60:0/1442152465 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6f080ddf10 con 0x7f6f000e27a0 2024-09-16T15:38:48.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.533+0000 7f6f07400640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6ee8078850 0x7f6ee807ad10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6efc033000 tx=0x7f6efc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.535+0000 7efce0c00640 1 -- 172.21.15.60:0/3665759536 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7efce4092070 con 0x7efcdc036510 2024-09-16T15:38:48.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.537+0000 7efce0c00640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efcc4078850 0x7efcc407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:48.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.538+0000 7efce0c00640 1 -- 172.21.15.60:0/3665759536 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efce4115bb0 con 0x7efcdc036510 2024-09-16T15:38:48.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.538+0000 7efce9200640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efcc4078850 0x7efcc407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:48.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.538+0000 7efce0c00640 1 -- 172.21.15.60:0/3665759536 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efce40d9eb0 con 0x7efcdc036510 2024-09-16T15:38:48.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.545+0000 7efce9200640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efcc4078850 0x7efcc407ad10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009cc0 tx=0x7efcd8005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:48.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.874+0000 7f784cc00640 1 -- 172.21.15.60:0/48150385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f785401d9a0 con 0x7f7854036520 2024-09-16T15:38:48.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.875+0000 7f784f400640 1 -- 172.21.15.60:0/48150385 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7850066560 con 0x7f7854036520 2024-09-16T15:38:48.877 INFO:teuthology.orchestra.run.smithi060.stdout:60129542153 2024-09-16T15:38:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.879+0000 7f784cc00640 1 -- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f783c078880 msgr2=0x7f783c07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.880+0000 7f784cc00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f783c078880 0x7f783c07ad40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f784800b9f0 tx=0x7f784800b3d0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.880+0000 7f784cc00640 1 -- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7854036520 msgr2=0x7f7854050d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.880+0000 7f784cc00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7854036520 0x7f7854050d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f785000c3c0 tx=0x7f785000c880 comp rx=0 tx=0).stop 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.880+0000 7f784cc00640 1 -- 172.21.15.60:0/48150385 shutdown_connections 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.881+0000 7f784cc00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f783c078880 0x7f783c07ad40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.881+0000 7f784cc00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7854036520 0x7f7854050d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.881+0000 7f784cc00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f785401c890 0x7f785404a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.881+0000 7f784cc00640 1 --2- 172.21.15.60:0/48150385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7854005a60 0x7f7854048a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.881+0000 7f784cc00640 1 -- 172.21.15.60:0/48150385 >> 172.21.15.60:0/48150385 conn(0x7f7854054560 msgr2=0x7f7854038a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.881+0000 7f784cc00640 1 -- 172.21.15.60:0/48150385 shutdown_connections 2024-09-16T15:38:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.882+0000 7f784cc00640 1 -- 172.21.15.60:0/48150385 wait complete. 2024-09-16T15:38:48.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542153 for osd.1 2024-09-16T15:38:48.911 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:48.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.937+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f863c04cca0 con 0x7f863c03b930 2024-09-16T15:38:48.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.937+0000 7f863aa00640 1 -- 172.21.15.60:0/517331168 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f863c04cca0 con 0x7f863c03b930 2024-09-16T15:38:48.939 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-09-16T15:38:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.940+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8624078a80 msgr2=0x7f862407af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.940+0000 7f8649400640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8624078a80 0x7f862407af40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8634005fd0 tx=0x7f8634009260 comp rx=0 tx=0).stop 2024-09-16T15:38:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.940+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 msgr2=0x7f863c050fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.940+0000 7f8649400640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 0x7f863c050fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f863000c3c0 tx=0x7f863000c880 comp rx=0 tx=0).stop 2024-09-16T15:38:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.941+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 shutdown_connections 2024-09-16T15:38:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.941+0000 7f8649400640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8624078a80 0x7f862407af40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.941+0000 7f8649400640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f863c0e2790 0x7f863c0e0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.941+0000 7f8649400640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f863c03b930 0x7f863c050fe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.941+0000 7f8649400640 1 --2- 172.21.15.60:0/517331168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c037f00 0x7f863c04f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.941+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 >> 172.21.15.60:0/517331168 conn(0x7f863c03a3c0 msgr2=0x7f863c004f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.941+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 shutdown_connections 2024-09-16T15:38:48.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.941+0000 7f8649400640 1 -- 172.21.15.60:0/517331168 wait complete. 2024-09-16T15:38:48.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-16T15:38:48.958 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:48.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.968+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd1a0050fa0 con 0x7fd1a015bf70 2024-09-16T15:38:48.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.968+0000 7fd196000640 1 -- 172.21.15.60:0/1130416235 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd188007e20 con 0x7fd1a015bf70 2024-09-16T15:38:48.970 INFO:teuthology.orchestra.run.smithi060.stdout:64424509448 2024-09-16T15:38:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.972+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd17c078950 msgr2=0x7fd17c07ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.972+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd17c078950 0x7fd17c07ae10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd198005fd0 tx=0x7fd198009260 comp rx=0 tx=0).stop 2024-09-16T15:38:48.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a015bf70 msgr2=0x7fd1a00d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a015bf70 0x7fd1a00d03f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c390 tx=0x7fd18800c850 comp rx=0 tx=0).stop 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 shutdown_connections 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd17c078950 0x7fd17c07ae10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1a015bf70 0x7fd1a00d03f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1a00d2580 0x7fd1a00b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 --2- 172.21.15.60:0/1130416235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1a00b4ca0 0x7fd1a00b9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 >> 172.21.15.60:0/1130416235 conn(0x7fd1a00d3dc0 msgr2=0x7fd1a006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 shutdown_connections 2024-09-16T15:38:48.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.973+0000 7fd1a5000640 1 -- 172.21.15.60:0/1130416235 wait complete. 2024-09-16T15:38:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.976+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd8c40392c0 con 0x7fd8c4009e00 2024-09-16T15:38:48.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.977+0000 7fd8c9600640 1 -- 172.21.15.60:0/2187353821 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd8c0061cc0 con 0x7fd8c4009e00 2024-09-16T15:38:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:64424509448 2024-09-16T15:38:48.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8ac078a50 msgr2=0x7fd8ac07af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8ac078a50 0x7fd8ac07af10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8033000 tx=0x7fd8b8004730 comp rx=0 tx=0).stop 2024-09-16T15:38:48.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c4009e00 msgr2=0x7fd8c404a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c4009e00 0x7fd8c404a1c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000c430 tx=0x7fd8c000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 shutdown_connections 2024-09-16T15:38:48.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8ac078a50 0x7fd8ac07af10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c4037680 0x7fd8c4180d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c4036510 0x7fd8c404a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 --2- 172.21.15.60:0/2187353821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c4009e00 0x7fd8c404a1c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.980+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 >> 172.21.15.60:0/2187353821 conn(0x7fd8c4054550 msgr2=0x7fd8c4050ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.981+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 shutdown_connections 2024-09-16T15:38:48.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.981+0000 7fd8d4000640 1 -- 172.21.15.60:0/2187353821 wait complete. 2024-09-16T15:38:48.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.981+0000 7efcd6000640 1 -- 172.21.15.60:0/3665759536 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efcb8005850 con 0x7efcdc036510 2024-09-16T15:38:48.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.982+0000 7efce0c00640 1 -- 172.21.15.60:0/3665759536 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efce40ddc40 con 0x7efcdc036510 2024-09-16T15:38:48.984 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-09-16T15:38:48.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7f1b7aa00640 1 -- 172.21.15.60:0/4063547949 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1b880cf7c0 con 0x7f1b8813d200 2024-09-16T15:38:48.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 -- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efcc4078850 msgr2=0x7efcc407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efcc4078850 0x7efcc407ad10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009cc0 tx=0x7efcd8005f30 comp rx=0 tx=0).stop 2024-09-16T15:38:48.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 -- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc036510 msgr2=0x7efcdc0d7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc036510 0x7efcdc0d7ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efce4069f40 tx=0x7efce406aed0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 -- 172.21.15.60:0/3665759536 shutdown_connections 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efcc4078850 0x7efcc407ad10 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009cc0 tx=0x7efcd8005f30 comp rx=0 tx=0).stop 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc036510 0x7efcdc0d7ac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc00a4d0 0x7efcdc17cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 --2- 172.21.15.60:0/3665759536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc00a0f0 0x7efcdc17cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 -- 172.21.15.60:0/3665759536 >> 172.21.15.60:0/3665759536 conn(0x7efcdc054550 msgr2=0x7efcdc033fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7f1b85600640 1 -- 172.21.15.60:0/4063547949 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b7c084d30 con 0x7f1b8813d200 2024-09-16T15:38:48.987 INFO:teuthology.orchestra.run.smithi060.stdout:64424509446 2024-09-16T15:38:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7efcd6000640 1 -- 172.21.15.60:0/3665759536 shutdown_connections 2024-09-16T15:38:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.985+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f350404f6b0 con 0x7f350403ab10 2024-09-16T15:38:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.986+0000 7efcd6000640 1 -- 172.21.15.60:0/3665759536 wait complete. 2024-09-16T15:38:48.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.986+0000 7f3502a00640 1 -- 172.21.15.60:0/3572468134 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f34f80626b0 con 0x7f350403ab10 2024-09-16T15:38:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:64424509447 2024-09-16T15:38:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.987+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6f00007250 con 0x7f6f000e27a0 2024-09-16T15:38:48.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.987+0000 7f6efb400640 1 -- 172.21.15.60:0/1442152465 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6f080dd810 con 0x7f6f000e27a0 2024-09-16T15:38:48.989 INFO:teuthology.orchestra.run.smithi060.stdout:64424509448 2024-09-16T15:38:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.989+0000 7f1b7aa00640 1 -- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b68078920 msgr2=0x7f1b6807ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.989+0000 7f1b7aa00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b68078920 0x7f1b6807ade0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009f20 tx=0x7f1b800094f0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.989+0000 7f1b7aa00640 1 -- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8813d200 msgr2=0x7f1b880ae770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.989+0000 7f1b7aa00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8813d200 0x7f1b880ae770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c02aa80 tx=0x7f1b7c032b90 comp rx=0 tx=0).stop 2024-09-16T15:38:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.989+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34ec078920 msgr2=0x7f34ec07ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.989+0000 7f3510a00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34ec078920 0x7f34ec07ade0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f34fc001670 tx=0x7f34fc009040 comp rx=0 tx=0).stop 2024-09-16T15:38:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350403ab10 msgr2=0x7f3504051d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350403ab10 0x7f3504051d60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f34f800c3e0 tx=0x7f34f800c8a0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f1b7aa00640 1 -- 172.21.15.60:0/4063547949 shutdown_connections 2024-09-16T15:38:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f1b7aa00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b68078920 0x7f1b6807ade0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009f20 tx=0x7f1b800094f0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f1b7aa00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8813d200 0x7f1b880ae770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 shutdown_connections 2024-09-16T15:38:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34ec078920 0x7f34ec07ade0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f1b7aa00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b880d0290 0x7f1b88152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f1b7aa00640 1 --2- 172.21.15.60:0/4063547949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b880cf080 0x7f1b881f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f1b7aa00640 1 -- 172.21.15.60:0/4063547949 >> 172.21.15.60:0/4063547949 conn(0x7f1b8806cd50 msgr2=0x7f1b8806d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35040e2a60 0x7f35040476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f350403ab10 0x7f3504051d60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 --2- 172.21.15.60:0/3572468134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3504037940 0x7f35040494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 >> 172.21.15.60:0/3572468134 conn(0x7f3504055540 msgr2=0x7f3504055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 shutdown_connections 2024-09-16T15:38:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f3510a00640 1 -- 172.21.15.60:0/3572468134 wait complete. 2024-09-16T15:38:48.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.990+0000 7f1b7aa00640 1 -- 172.21.15.60:0/4063547949 shutdown_connections 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f1b7aa00640 1 -- 172.21.15.60:0/4063547949 wait complete. 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6ee8078850 msgr2=0x7f6ee807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6ee8078850 0x7f6ee807ad10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6efc033000 tx=0x7f6efc004730 comp rx=0 tx=0).stop 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f000e27a0 msgr2=0x7f6f00038ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f000e27a0 0x7f6f00038ad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f0802eb50 tx=0x7f6f0806a370 comp rx=0 tx=0).stop 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 shutdown_connections 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6ee8078850 0x7f6ee807ad10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f000e27a0 0x7f6f00038ad0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f0003b940 0x7f6f00031a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 --2- 172.21.15.60:0/1442152465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f00037f10 0x7f6f000d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 >> 172.21.15.60:0/1442152465 conn(0x7f6f0003a3d0 msgr2=0x7f6f00039af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 shutdown_connections 2024-09-16T15:38:48.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:48.992+0000 7f6f0ca00640 1 -- 172.21.15.60:0/1442152465 wait complete. 2024-09-16T15:38:48.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.0 2024-09-16T15:38:48.996 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:49.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-09-16T15:38:49.010 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:49.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2024-09-16T15:38:49.012 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:49.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-09-16T15:38:49.017 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:49.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-09-16T15:38:49.018 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:49.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-16T15:38:49.019 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:49.019 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:38:49.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:38:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.220+0000 7f35f2c00640 1 -- 172.21.15.60:0/794531580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 msgr2=0x7f35ec0d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.220+0000 7f35f2c00640 1 --2- 172.21.15.60:0/794531580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 0x7f35ec0d22a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009830 tx=0x7f35d802a510 comp rx=0 tx=0).stop 2024-09-16T15:38:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.220+0000 7f35f2c00640 1 -- 172.21.15.60:0/794531580 shutdown_connections 2024-09-16T15:38:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.220+0000 7f35f2c00640 1 --2- 172.21.15.60:0/794531580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 0x7f35ec0d22a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.220+0000 7f35f2c00640 1 --2- 172.21.15.60:0/794531580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35ec0b4d10 0x7f35ec162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.220+0000 7f35f2c00640 1 -- 172.21.15.60:0/794531580 >> 172.21.15.60:0/794531580 conn(0x7f35ec0d3de0 msgr2=0x7f35ec0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.220+0000 7f35f2c00640 1 -- 172.21.15.60:0/794531580 shutdown_connections 2024-09-16T15:38:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.220+0000 7f35f2c00640 1 -- 172.21.15.60:0/794531580 wait complete. 2024-09-16T15:38:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.222+0000 7f35f2c00640 1 Processor -- start 2024-09-16T15:38:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.222+0000 7f35f2c00640 1 -- start start 2024-09-16T15:38:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 0x7f35ec1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f2c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35ec0b4d10 0x7f35ec0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f2c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35ec0d1ec0 0x7f35ec0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f2c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0b3a80 con 0x7f35ec0b4d10 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0b3900 con 0x7f35ec0b28d0 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f2c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f35ec0b3c00 con 0x7f35ec0d1ec0 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35ec0b4d10 0x7f35ec0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35ec0b4d10 0x7f35ec0caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43352/0 (socket says 172.21.15.60:43352) 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f0e00640 1 -- 172.21.15.60:0/4174655819 learned_addr learned my addr 172.21.15.60:0/4174655819 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 0x7f35ec1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.223+0000 7f35f2200640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35ec0d1ec0 0x7f35ec0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f0e00640 1 -- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35ec0d1ec0 msgr2=0x7f35ec0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f0e00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35ec0d1ec0 0x7f35ec0cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f0e00640 1 -- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 msgr2=0x7f35ec1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f0e00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 0x7f35ec1523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f0e00640 1 -- 172.21.15.60:0/4174655819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec1602e0 con 0x7f35ec0b4d10 2024-09-16T15:38:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f1800640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 0x7f35ec1523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f0e00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35ec0b4d10 0x7f35ec0caeb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009950 tx=0x7f35d80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35e9600640 1 -- 172.21.15.60:0/4174655819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8040070 con 0x7f35ec0b4d10 2024-09-16T15:38:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35e9600640 1 -- 172.21.15.60:0/4174655819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35d8005310 con 0x7f35ec0b4d10 2024-09-16T15:38:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec15eb50 con 0x7f35ec0b4d10 2024-09-16T15:38:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35e9600640 1 -- 172.21.15.60:0/4174655819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d80044c0 con 0x7f35ec0b4d10 2024-09-16T15:38:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.224+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec15f110 con 0x7f35ec0b4d10 2024-09-16T15:38:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.226+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec0d22a0 con 0x7f35ec0b4d10 2024-09-16T15:38:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.227+0000 7f35e9600640 1 -- 172.21.15.60:0/4174655819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f35d8039040 con 0x7f35ec0b4d10 2024-09-16T15:38:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.228+0000 7f35e9600640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f35cc078880 0x7f35cc07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.228+0000 7f35e9600640 1 -- 172.21.15.60:0/4174655819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f35d80bb780 con 0x7f35ec0b4d10 2024-09-16T15:38:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.228+0000 7f35f1800640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f35cc078880 0x7f35cc07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:49.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.229+0000 7f35f1800640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f35cc078880 0x7f35cc07ad40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f35e0007fd0 tx=0x7f35e0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:49.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.232+0000 7f35e9600640 1 -- 172.21.15.60:0/4174655819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f35d8031770 con 0x7f35ec0b4d10 2024-09-16T15:38:49.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.422+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35ec076160 con 0x7f35cc078880 2024-09-16T15:38:49.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.427+0000 7f35e9600640 1 -- 172.21.15.60:0/4174655819 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f35ec076160 con 0x7f35cc078880 2024-09-16T15:38:49.431 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:49.432 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:38:49.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.430+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f35cc078880 msgr2=0x7f35cc07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.430+0000 7f35f2c00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f35cc078880 0x7f35cc07ad40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f35e0007fd0 tx=0x7f35e0009040 comp rx=0 tx=0).stop 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.430+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35ec0b4d10 msgr2=0x7f35ec0caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.430+0000 7f35f2c00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35ec0b4d10 0x7f35ec0caeb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009950 tx=0x7f35d80040c0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.430+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 shutdown_connections 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.430+0000 7f35f2c00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f35cc078880 0x7f35cc07ad40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.430+0000 7f35f2c00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35ec0d1ec0 0x7f35ec0cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.430+0000 7f35f2c00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35ec0b4d10 0x7f35ec0caeb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.431+0000 7f35f2c00640 1 --2- 172.21.15.60:0/4174655819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ec0b28d0 0x7f35ec1523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.431+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 >> 172.21.15.60:0/4174655819 conn(0x7f35ec0d3de0 msgr2=0x7f35ec15e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.431+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 shutdown_connections 2024-09-16T15:38:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.431+0000 7f35f2c00640 1 -- 172.21.15.60:0/4174655819 wait complete. 2024-09-16T15:38:49.448 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-16T15:38:47.711714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235932,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738788,"statfs":{"total":773094113280,"available":772852518912,"internally_reserved":0,"allocated":3121152,"data_stored":1840172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.780517"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328961+0000","last_change":"2024-09-16T15:38:34.948046+0000","last_active":"2024-09-16T15:38:36.328961+0000","last_peered":"2024-09-16T15:38:36.328961+0000","last_clean":"2024-09-16T15:38:36.328961+0000","last_became_active":"2024-09-16T15:38:34.947778+0000","last_became_peered":"2024-09-16T15:38:34.947778+0000","last_unstale":"2024-09-16T15:38:36.328961+0000","last_undegraded":"2024-09-16T15:38:36.328961+0000","last_fullsized":"2024-09-16T15:38:36.328961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329296+0000","last_change":"2024-09-16T15:38:34.934287+0000","last_active":"2024-09-16T15:38:36.329296+0000","last_peered":"2024-09-16T15:38:36.329296+0000","last_clean":"2024-09-16T15:38:36.329296+0000","last_became_active":"2024-09-16T15:38:34.934004+0000","last_became_peered":"2024-09-16T15:38:34.934004+0000","last_unstale":"2024-09-16T15:38:36.329296+0000","last_undegraded":"2024-09-16T15:38:36.329296+0000","last_fullsized":"2024-09-16T15:38:36.329296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:43.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330192+0000","last_change":"2024-09-16T15:38:34.937280+0000","last_active":"2024-09-16T15:38:36.330192+0000","last_peered":"2024-09-16T15:38:36.330192+0000","last_clean":"2024-09-16T15:38:36.330192+0000","last_became_active":"2024-09-16T15:38:34.936979+0000","last_became_peered":"2024-09-16T15:38:34.936979+0000","last_unstale":"2024-09-16T15:38:36.330192+0000","last_undegraded":"2024-09-16T15:38:36.330192+0000","last_fullsized":"2024-09-16T15:38:36.330192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:16.363921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328077+0000","last_change":"2024-09-16T15:38:34.956532+0000","last_active":"2024-09-16T15:38:36.328077+0000","last_peered":"2024-09-16T15:38:36.328077+0000","last_clean":"2024-09-16T15:38:36.328077+0000","last_became_active":"2024-09-16T15:38:34.956261+0000","last_became_peered":"2024-09-16T15:38:34.956261+0000","last_unstale":"2024-09-16T15:38:36.328077+0000","last_undegraded":"2024-09-16T15:38:36.328077+0000","last_fullsized":"2024-09-16T15:38:36.328077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328317+0000","last_change":"2024-09-16T15:38:34.946850+0000","last_active":"2024-09-16T15:38:36.328317+0000","last_peered":"2024-09-16T15:38:36.328317+0000","last_clean":"2024-09-16T15:38:36.328317+0000","last_became_active":"2024-09-16T15:38:34.946584+0000","last_became_peered":"2024-09-16T15:38:34.946584+0000","last_unstale":"2024-09-16T15:38:36.328317+0000","last_undegraded":"2024-09-16T15:38:36.328317+0000","last_fullsized":"2024-09-16T15:38:36.328317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328172+0000","last_change":"2024-09-16T15:38:34.946360+0000","last_active":"2024-09-16T15:38:36.328172+0000","last_peered":"2024-09-16T15:38:36.328172+0000","last_clean":"2024-09-16T15:38:36.328172+0000","last_became_active":"2024-09-16T15:38:34.946022+0000","last_became_peered":"2024-09-16T15:38:34.946022+0000","last_unstale":"2024-09-16T15:38:36.328172+0000","last_undegraded":"2024-09-16T15:38:36.328172+0000","last_fullsized":"2024-09-16T15:38:36.328172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328807+0000","last_change":"2024-09-16T15:38:34.948730+0000","last_active":"2024-09-16T15:38:36.328807+0000","last_peered":"2024-09-16T15:38:36.328807+0000","last_clean":"2024-09-16T15:38:36.328807+0000","last_became_active":"2024-09-16T15:38:34.948488+0000","last_became_peered":"2024-09-16T15:38:34.948488+0000","last_unstale":"2024-09-16T15:38:36.328807+0000","last_undegraded":"2024-09-16T15:38:36.328807+0000","last_fullsized":"2024-09-16T15:38:36.328807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327904+0000","last_change":"2024-09-16T15:38:34.939960+0000","last_active":"2024-09-16T15:38:36.327904+0000","last_peered":"2024-09-16T15:38:36.327904+0000","last_clean":"2024-09-16T15:38:36.327904+0000","last_became_active":"2024-09-16T15:38:34.939730+0000","last_became_peered":"2024-09-16T15:38:34.939730+0000","last_unstale":"2024-09-16T15:38:36.327904+0000","last_undegraded":"2024-09-16T15:38:36.327904+0000","last_fullsized":"2024-09-16T15:38:36.327904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:35.445142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329103+0000","last_change":"2024-09-16T15:38:34.963612+0000","last_active":"2024-09-16T15:38:36.329103+0000","last_peered":"2024-09-16T15:38:36.329103+0000","last_clean":"2024-09-16T15:38:36.329103+0000","last_became_active":"2024-09-16T15:38:34.963287+0000","last_became_peered":"2024-09-16T15:38:34.963287+0000","last_unstale":"2024-09-16T15:38:36.329103+0000","last_undegraded":"2024-09-16T15:38:36.329103+0000","last_fullsized":"2024-09-16T15:38:36.329103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:53.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935830+0000","last_change":"2024-09-16T15:38:34.959111+0000","last_active":"2024-09-16T15:38:35.935830+0000","last_peered":"2024-09-16T15:38:35.935830+0000","last_clean":"2024-09-16T15:38:35.935830+0000","last_became_active":"2024-09-16T15:38:34.958733+0000","last_became_peered":"2024-09-16T15:38:34.958733+0000","last_unstale":"2024-09-16T15:38:35.935830+0000","last_undegraded":"2024-09-16T15:38:35.935830+0000","last_fullsized":"2024-09-16T15:38:35.935830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:21.940460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934331+0000","last_change":"2024-09-16T15:38:34.963928+0000","last_active":"2024-09-16T15:38:35.934331+0000","last_peered":"2024-09-16T15:38:35.934331+0000","last_clean":"2024-09-16T15:38:35.934331+0000","last_became_active":"2024-09-16T15:38:34.963418+0000","last_became_peered":"2024-09-16T15:38:34.963418+0000","last_unstale":"2024-09-16T15:38:35.934331+0000","last_undegraded":"2024-09-16T15:38:35.934331+0000","last_fullsized":"2024-09-16T15:38:35.934331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:54.016578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935041+0000","last_change":"2024-09-16T15:38:34.951463+0000","last_active":"2024-09-16T15:38:35.935041+0000","last_peered":"2024-09-16T15:38:35.935041+0000","last_clean":"2024-09-16T15:38:35.935041+0000","last_became_active":"2024-09-16T15:38:34.949972+0000","last_became_peered":"2024-09-16T15:38:34.949972+0000","last_unstale":"2024-09-16T15:38:35.935041+0000","last_undegraded":"2024-09-16T15:38:35.935041+0000","last_fullsized":"2024-09-16T15:38:35.935041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:36.356482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328377+0000","last_change":"2024-09-16T15:38:34.936161+0000","last_active":"2024-09-16T15:38:36.328377+0000","last_peered":"2024-09-16T15:38:36.328377+0000","last_clean":"2024-09-16T15:38:36.328377+0000","last_became_active":"2024-09-16T15:38:34.935881+0000","last_became_peered":"2024-09-16T15:38:34.935881+0000","last_unstale":"2024-09-16T15:38:36.328377+0000","last_undegraded":"2024-09-16T15:38:36.328377+0000","last_fullsized":"2024-09-16T15:38:36.328377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329953+0000","last_change":"2024-09-16T15:38:34.934084+0000","last_active":"2024-09-16T15:38:36.329953+0000","last_peered":"2024-09-16T15:38:36.329953+0000","last_clean":"2024-09-16T15:38:36.329953+0000","last_became_active":"2024-09-16T15:38:34.933716+0000","last_became_peered":"2024-09-16T15:38:34.933716+0000","last_unstale":"2024-09-16T15:38:36.329953+0000","last_undegraded":"2024-09-16T15:38:36.329953+0000","last_fullsized":"2024-09-16T15:38:36.329953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:51.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328795+0000","last_change":"2024-09-16T15:38:34.951427+0000","last_active":"2024-09-16T15:38:36.328795+0000","last_peered":"2024-09-16T15:38:36.328795+0000","last_clean":"2024-09-16T15:38:36.328795+0000","last_became_active":"2024-09-16T15:38:34.949484+0000","last_became_peered":"2024-09-16T15:38:34.949484+0000","last_unstale":"2024-09-16T15:38:36.328795+0000","last_undegraded":"2024-09-16T15:38:36.328795+0000","last_fullsized":"2024-09-16T15:38:36.328795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:54.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327906+0000","last_change":"2024-09-16T15:38:34.948833+0000","last_active":"2024-09-16T15:38:36.327906+0000","last_peered":"2024-09-16T15:38:36.327906+0000","last_clean":"2024-09-16T15:38:36.327906+0000","last_became_active":"2024-09-16T15:38:34.948127+0000","last_became_peered":"2024-09-16T15:38:34.948127+0000","last_unstale":"2024-09-16T15:38:36.327906+0000","last_undegraded":"2024-09-16T15:38:36.327906+0000","last_fullsized":"2024-09-16T15:38:36.327906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329716+0000","last_change":"2024-09-16T15:38:34.948165+0000","last_active":"2024-09-16T15:38:36.329716+0000","last_peered":"2024-09-16T15:38:36.329716+0000","last_clean":"2024-09-16T15:38:36.329716+0000","last_became_active":"2024-09-16T15:38:34.947862+0000","last_became_peered":"2024-09-16T15:38:34.947862+0000","last_unstale":"2024-09-16T15:38:36.329716+0000","last_undegraded":"2024-09-16T15:38:36.329716+0000","last_fullsized":"2024-09-16T15:38:36.329716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:07.065693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932521+0000","last_change":"2024-09-16T15:38:34.936088+0000","last_active":"2024-09-16T15:38:35.932521+0000","last_peered":"2024-09-16T15:38:35.932521+0000","last_clean":"2024-09-16T15:38:35.932521+0000","last_became_active":"2024-09-16T15:38:34.935649+0000","last_became_peered":"2024-09-16T15:38:34.935649+0000","last_unstale":"2024-09-16T15:38:35.932521+0000","last_undegraded":"2024-09-16T15:38:35.932521+0000","last_fullsized":"2024-09-16T15:38:35.932521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:56.948937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328715+0000","last_change":"2024-09-16T15:38:34.951511+0000","last_active":"2024-09-16T15:38:36.328715+0000","last_peered":"2024-09-16T15:38:36.328715+0000","last_clean":"2024-09-16T15:38:36.328715+0000","last_became_active":"2024-09-16T15:38:34.951241+0000","last_became_peered":"2024-09-16T15:38:34.951241+0000","last_unstale":"2024-09-16T15:38:36.328715+0000","last_undegraded":"2024-09-16T15:38:36.328715+0000","last_fullsized":"2024-09-16T15:38:36.328715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:52.005508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934261+0000","last_change":"2024-09-16T15:38:34.950265+0000","last_active":"2024-09-16T15:38:35.934261+0000","last_peered":"2024-09-16T15:38:35.934261+0000","last_clean":"2024-09-16T15:38:35.934261+0000","last_became_active":"2024-09-16T15:38:34.949457+0000","last_became_peered":"2024-09-16T15:38:34.949457+0000","last_unstale":"2024-09-16T15:38:35.934261+0000","last_undegraded":"2024-09-16T15:38:35.934261+0000","last_fullsized":"2024-09-16T15:38:35.934261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:33.694559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330049+0000","last_change":"2024-09-16T15:38:34.942753+0000","last_active":"2024-09-16T15:38:36.330049+0000","last_peered":"2024-09-16T15:38:36.330049+0000","last_clean":"2024-09-16T15:38:36.330049+0000","last_became_active":"2024-09-16T15:38:34.940841+0000","last_became_peered":"2024-09-16T15:38:34.940841+0000","last_unstale":"2024-09-16T15:38:36.330049+0000","last_undegraded":"2024-09-16T15:38:36.330049+0000","last_fullsized":"2024-09-16T15:38:36.330049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:22.740035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328374+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328374+0000","last_peered":"2024-09-16T15:38:36.328374+0000","last_clean":"2024-09-16T15:38:36.328374+0000","last_became_active":"2024-09-16T15:38:34.958240+0000","last_became_peered":"2024-09-16T15:38:34.958240+0000","last_unstale":"2024-09-16T15:38:36.328374+0000","last_undegraded":"2024-09-16T15:38:36.328374+0000","last_fullsized":"2024-09-16T15:38:36.328374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328083+0000","last_change":"2024-09-16T15:38:34.952703+0000","last_active":"2024-09-16T15:38:36.328083+0000","last_peered":"2024-09-16T15:38:36.328083+0000","last_clean":"2024-09-16T15:38:36.328083+0000","last_became_active":"2024-09-16T15:38:34.949406+0000","last_became_peered":"2024-09-16T15:38:34.949406+0000","last_unstale":"2024-09-16T15:38:36.328083+0000","last_undegraded":"2024-09-16T15:38:36.328083+0000","last_fullsized":"2024-09-16T15:38:36.328083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:20.072155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328173+0000","last_change":"2024-09-16T15:38:34.948442+0000","last_active":"2024-09-16T15:38:36.328173+0000","last_peered":"2024-09-16T15:38:36.328173+0000","last_clean":"2024-09-16T15:38:36.328173+0000","last_became_active":"2024-09-16T15:38:34.947564+0000","last_became_peered":"2024-09-16T15:38:34.947564+0000","last_unstale":"2024-09-16T15:38:36.328173+0000","last_undegraded":"2024-09-16T15:38:36.328173+0000","last_fullsized":"2024-09-16T15:38:36.328173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328339+0000","last_change":"2024-09-16T15:38:34.940042+0000","last_active":"2024-09-16T15:38:36.328339+0000","last_peered":"2024-09-16T15:38:36.328339+0000","last_clean":"2024-09-16T15:38:36.328339+0000","last_became_active":"2024-09-16T15:38:34.939792+0000","last_became_peered":"2024-09-16T15:38:34.939792+0000","last_unstale":"2024-09-16T15:38:36.328339+0000","last_undegraded":"2024-09-16T15:38:36.328339+0000","last_fullsized":"2024-09-16T15:38:36.328339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.386845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935854+0000","last_change":"2024-09-16T15:38:34.938919+0000","last_active":"2024-09-16T15:38:35.935854+0000","last_peered":"2024-09-16T15:38:35.935854+0000","last_clean":"2024-09-16T15:38:35.935854+0000","last_became_active":"2024-09-16T15:38:34.938608+0000","last_became_peered":"2024-09-16T15:38:34.938608+0000","last_unstale":"2024-09-16T15:38:35.935854+0000","last_undegraded":"2024-09-16T15:38:35.935854+0000","last_fullsized":"2024-09-16T15:38:35.935854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:44.137157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329086+0000","last_change":"2024-09-16T15:38:34.934519+0000","last_active":"2024-09-16T15:38:36.329086+0000","last_peered":"2024-09-16T15:38:36.329086+0000","last_clean":"2024-09-16T15:38:36.329086+0000","last_became_active":"2024-09-16T15:38:34.932223+0000","last_became_peered":"2024-09-16T15:38:34.932223+0000","last_unstale":"2024-09-16T15:38:36.329086+0000","last_undegraded":"2024-09-16T15:38:36.329086+0000","last_fullsized":"2024-09-16T15:38:36.329086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.673101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329499+0000","last_change":"2024-09-16T15:38:34.941297+0000","last_active":"2024-09-16T15:38:36.329499+0000","last_peered":"2024-09-16T15:38:36.329499+0000","last_clean":"2024-09-16T15:38:36.329499+0000","last_became_active":"2024-09-16T15:38:34.941007+0000","last_became_peered":"2024-09-16T15:38:34.941007+0000","last_unstale":"2024-09-16T15:38:36.329499+0000","last_undegraded":"2024-09-16T15:38:36.329499+0000","last_fullsized":"2024-09-16T15:38:36.329499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330077+0000","last_change":"2024-09-16T15:38:34.941110+0000","last_active":"2024-09-16T15:38:36.330077+0000","last_peered":"2024-09-16T15:38:36.330077+0000","last_clean":"2024-09-16T15:38:36.330077+0000","last_became_active":"2024-09-16T15:38:34.940650+0000","last_became_peered":"2024-09-16T15:38:34.940650+0000","last_unstale":"2024-09-16T15:38:36.330077+0000","last_undegraded":"2024-09-16T15:38:36.330077+0000","last_fullsized":"2024-09-16T15:38:36.330077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:26.093004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328522+0000","last_change":"2024-09-16T15:38:34.952930+0000","last_active":"2024-09-16T15:38:36.328522+0000","last_peered":"2024-09-16T15:38:36.328522+0000","last_clean":"2024-09-16T15:38:36.328522+0000","last_became_active":"2024-09-16T15:38:34.952652+0000","last_became_peered":"2024-09-16T15:38:34.952652+0000","last_unstale":"2024-09-16T15:38:36.328522+0000","last_undegraded":"2024-09-16T15:38:36.328522+0000","last_fullsized":"2024-09-16T15:38:36.328522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932490+0000","last_change":"2024-09-16T15:38:34.961835+0000","last_active":"2024-09-16T15:38:35.932490+0000","last_peered":"2024-09-16T15:38:35.932490+0000","last_clean":"2024-09-16T15:38:35.932490+0000","last_became_active":"2024-09-16T15:38:34.961529+0000","last_became_peered":"2024-09-16T15:38:34.961529+0000","last_unstale":"2024-09-16T15:38:35.932490+0000","last_undegraded":"2024-09-16T15:38:35.932490+0000","last_fullsized":"2024-09-16T15:38:35.932490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:11.053973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933841+0000","last_change":"2024-09-16T15:38:34.942442+0000","last_active":"2024-09-16T15:38:35.933841+0000","last_peered":"2024-09-16T15:38:35.933841+0000","last_clean":"2024-09-16T15:38:35.933841+0000","last_became_active":"2024-09-16T15:38:34.942144+0000","last_became_peered":"2024-09-16T15:38:34.942144+0000","last_unstale":"2024-09-16T15:38:35.933841+0000","last_undegraded":"2024-09-16T15:38:35.933841+0000","last_fullsized":"2024-09-16T15:38:35.933841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:43.399475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934635+0000","last_change":"2024-09-16T15:38:34.941230+0000","last_active":"2024-09-16T15:38:35.934635+0000","last_peered":"2024-09-16T15:38:35.934635+0000","last_clean":"2024-09-16T15:38:35.934635+0000","last_became_active":"2024-09-16T15:38:34.940613+0000","last_became_peered":"2024-09-16T15:38:34.940613+0000","last_unstale":"2024-09-16T15:38:35.934635+0000","last_undegraded":"2024-09-16T15:38:35.934635+0000","last_fullsized":"2024-09-16T15:38:35.934635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:15.007186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328673+0000","last_change":"2024-09-16T15:38:33.949962+0000","last_active":"2024-09-16T15:38:36.328673+0000","last_peered":"2024-09-16T15:38:36.328673+0000","last_clean":"2024-09-16T15:38:36.328673+0000","last_became_active":"2024-09-16T15:38:33.949620+0000","last_became_peered":"2024-09-16T15:38:33.949620+0000","last_unstale":"2024-09-16T15:38:36.328673+0000","last_undegraded":"2024-09-16T15:38:36.328673+0000","last_fullsized":"2024-09-16T15:38:36.328673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330281+0000","last_change":"2024-09-16T15:38:34.947478+0000","last_active":"2024-09-16T15:38:36.330281+0000","last_peered":"2024-09-16T15:38:36.330281+0000","last_clean":"2024-09-16T15:38:36.330281+0000","last_became_active":"2024-09-16T15:38:34.947158+0000","last_became_peered":"2024-09-16T15:38:34.947158+0000","last_unstale":"2024-09-16T15:38:36.330281+0000","last_undegraded":"2024-09-16T15:38:36.330281+0000","last_fullsized":"2024-09-16T15:38:36.330281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:24.497838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933892+0000","last_change":"2024-09-16T15:38:33.931830+0000","last_active":"2024-09-16T15:38:35.933892+0000","last_peered":"2024-09-16T15:38:35.933892+0000","last_clean":"2024-09-16T15:38:35.933892+0000","last_became_active":"2024-09-16T15:38:33.931129+0000","last_became_peered":"2024-09-16T15:38:33.931129+0000","last_unstale":"2024-09-16T15:38:35.933892+0000","last_undegraded":"2024-09-16T15:38:35.933892+0000","last_fullsized":"2024-09-16T15:38:35.933892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:54.747927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327884+0000","last_change":"2024-09-16T15:38:33.943597+0000","last_active":"2024-09-16T15:38:36.327884+0000","last_peered":"2024-09-16T15:38:36.327884+0000","last_clean":"2024-09-16T15:38:36.327884+0000","last_became_active":"2024-09-16T15:38:33.940904+0000","last_became_peered":"2024-09-16T15:38:33.940904+0000","last_unstale":"2024-09-16T15:38:36.327884+0000","last_undegraded":"2024-09-16T15:38:36.327884+0000","last_fullsized":"2024-09-16T15:38:36.327884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:13.818889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328165+0000","last_change":"2024-09-16T15:38:34.942424+0000","last_active":"2024-09-16T15:38:36.328165+0000","last_peered":"2024-09-16T15:38:36.328165+0000","last_clean":"2024-09-16T15:38:36.328165+0000","last_became_active":"2024-09-16T15:38:34.942111+0000","last_became_peered":"2024-09-16T15:38:34.942111+0000","last_unstale":"2024-09-16T15:38:36.328165+0000","last_undegraded":"2024-09-16T15:38:36.328165+0000","last_fullsized":"2024-09-16T15:38:36.328165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:52.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328244+0000","last_change":"2024-09-16T15:38:33.945014+0000","last_active":"2024-09-16T15:38:36.328244+0000","last_peered":"2024-09-16T15:38:36.328244+0000","last_clean":"2024-09-16T15:38:36.328244+0000","last_became_active":"2024-09-16T15:38:33.944742+0000","last_became_peered":"2024-09-16T15:38:33.944742+0000","last_unstale":"2024-09-16T15:38:36.328244+0000","last_undegraded":"2024-09-16T15:38:36.328244+0000","last_fullsized":"2024-09-16T15:38:36.328244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328641+0000","last_change":"2024-09-16T15:38:34.933709+0000","last_active":"2024-09-16T15:38:36.328641+0000","last_peered":"2024-09-16T15:38:36.328641+0000","last_clean":"2024-09-16T15:38:36.328641+0000","last_became_active":"2024-09-16T15:38:34.932126+0000","last_became_peered":"2024-09-16T15:38:34.932126+0000","last_unstale":"2024-09-16T15:38:36.328641+0000","last_undegraded":"2024-09-16T15:38:36.328641+0000","last_fullsized":"2024-09-16T15:38:36.328641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:44.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935745+0000","last_change":"2024-09-16T15:38:33.945379+0000","last_active":"2024-09-16T15:38:35.935745+0000","last_peered":"2024-09-16T15:38:35.935745+0000","last_clean":"2024-09-16T15:38:35.935745+0000","last_became_active":"2024-09-16T15:38:33.944531+0000","last_became_peered":"2024-09-16T15:38:33.944531+0000","last_unstale":"2024-09-16T15:38:35.935745+0000","last_undegraded":"2024-09-16T15:38:35.935745+0000","last_fullsized":"2024-09-16T15:38:35.935745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:09.939574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329480+0000","last_change":"2024-09-16T15:38:34.961412+0000","last_active":"2024-09-16T15:38:36.329480+0000","last_peered":"2024-09-16T15:38:36.329480+0000","last_clean":"2024-09-16T15:38:36.329480+0000","last_became_active":"2024-09-16T15:38:34.961004+0000","last_became_peered":"2024-09-16T15:38:34.961004+0000","last_unstale":"2024-09-16T15:38:36.329480+0000","last_undegraded":"2024-09-16T15:38:36.329480+0000","last_fullsized":"2024-09-16T15:38:36.329480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:39.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329349+0000","last_change":"2024-09-16T15:38:33.942028+0000","last_active":"2024-09-16T15:38:36.329349+0000","last_peered":"2024-09-16T15:38:36.329349+0000","last_clean":"2024-09-16T15:38:36.329349+0000","last_became_active":"2024-09-16T15:38:33.940695+0000","last_became_peered":"2024-09-16T15:38:33.940695+0000","last_unstale":"2024-09-16T15:38:36.329349+0000","last_undegraded":"2024-09-16T15:38:36.329349+0000","last_fullsized":"2024-09-16T15:38:36.329349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330396+0000","last_change":"2024-09-16T15:38:34.937478+0000","last_active":"2024-09-16T15:38:36.330396+0000","last_peered":"2024-09-16T15:38:36.330396+0000","last_clean":"2024-09-16T15:38:36.330396+0000","last_became_active":"2024-09-16T15:38:34.936844+0000","last_became_peered":"2024-09-16T15:38:34.936844+0000","last_unstale":"2024-09-16T15:38:36.330396+0000","last_undegraded":"2024-09-16T15:38:36.330396+0000","last_fullsized":"2024-09-16T15:38:36.330396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:42.401407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328973+0000","last_change":"2024-09-16T15:38:33.942385+0000","last_active":"2024-09-16T15:38:36.328973+0000","last_peered":"2024-09-16T15:38:36.328973+0000","last_clean":"2024-09-16T15:38:36.328973+0000","last_became_active":"2024-09-16T15:38:33.941960+0000","last_became_peered":"2024-09-16T15:38:33.941960+0000","last_unstale":"2024-09-16T15:38:36.328973+0000","last_undegraded":"2024-09-16T15:38:36.328973+0000","last_fullsized":"2024-09-16T15:38:36.328973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:08.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935156+0000","last_change":"2024-09-16T15:38:34.955492+0000","last_active":"2024-09-16T15:38:35.935156+0000","last_peered":"2024-09-16T15:38:35.935156+0000","last_clean":"2024-09-16T15:38:35.935156+0000","last_became_active":"2024-09-16T15:38:34.955230+0000","last_became_peered":"2024-09-16T15:38:34.955230+0000","last_unstale":"2024-09-16T15:38:35.935156+0000","last_undegraded":"2024-09-16T15:38:35.935156+0000","last_fullsized":"2024-09-16T15:38:35.935156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:13.656183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934327+0000","last_change":"2024-09-16T15:38:33.939711+0000","last_active":"2024-09-16T15:38:35.934327+0000","last_peered":"2024-09-16T15:38:35.934327+0000","last_clean":"2024-09-16T15:38:35.934327+0000","last_became_active":"2024-09-16T15:38:33.937858+0000","last_became_peered":"2024-09-16T15:38:33.937858+0000","last_unstale":"2024-09-16T15:38:35.934327+0000","last_undegraded":"2024-09-16T15:38:35.934327+0000","last_fullsized":"2024-09-16T15:38:35.934327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:42.581730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328596+0000","last_change":"2024-09-16T15:38:34.947613+0000","last_active":"2024-09-16T15:38:36.328596+0000","last_peered":"2024-09-16T15:38:36.328596+0000","last_clean":"2024-09-16T15:38:36.328596+0000","last_became_active":"2024-09-16T15:38:34.947258+0000","last_became_peered":"2024-09-16T15:38:34.947258+0000","last_unstale":"2024-09-16T15:38:36.328596+0000","last_undegraded":"2024-09-16T15:38:36.328596+0000","last_fullsized":"2024-09-16T15:38:36.328596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.135119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330167+0000","last_change":"2024-09-16T15:38:33.936316+0000","last_active":"2024-09-16T15:38:36.330167+0000","last_peered":"2024-09-16T15:38:36.330167+0000","last_clean":"2024-09-16T15:38:36.330167+0000","last_became_active":"2024-09-16T15:38:33.936002+0000","last_became_peered":"2024-09-16T15:38:33.936002+0000","last_unstale":"2024-09-16T15:38:36.330167+0000","last_undegraded":"2024-09-16T15:38:36.330167+0000","last_fullsized":"2024-09-16T15:38:36.330167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.796157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329907+0000","last_change":"2024-09-16T15:38:34.942140+0000","last_active":"2024-09-16T15:38:36.329907+0000","last_peered":"2024-09-16T15:38:36.329907+0000","last_clean":"2024-09-16T15:38:36.329907+0000","last_became_active":"2024-09-16T15:38:34.940716+0000","last_became_peered":"2024-09-16T15:38:34.940716+0000","last_unstale":"2024-09-16T15:38:36.329907+0000","last_undegraded":"2024-09-16T15:38:36.329907+0000","last_fullsized":"2024-09-16T15:38:36.329907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:55.566537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328440+0000","last_change":"2024-09-16T15:38:33.938693+0000","last_active":"2024-09-16T15:38:36.328440+0000","last_peered":"2024-09-16T15:38:36.328440+0000","last_clean":"2024-09-16T15:38:36.328440+0000","last_became_active":"2024-09-16T15:38:33.938207+0000","last_became_peered":"2024-09-16T15:38:33.938207+0000","last_unstale":"2024-09-16T15:38:36.328440+0000","last_undegraded":"2024-09-16T15:38:36.328440+0000","last_fullsized":"2024-09-16T15:38:36.328440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328857+0000","last_change":"2024-09-16T15:38:34.956559+0000","last_active":"2024-09-16T15:38:36.328857+0000","last_peered":"2024-09-16T15:38:36.328857+0000","last_clean":"2024-09-16T15:38:36.328857+0000","last_became_active":"2024-09-16T15:38:34.956290+0000","last_became_peered":"2024-09-16T15:38:34.956290+0000","last_unstale":"2024-09-16T15:38:36.328857+0000","last_undegraded":"2024-09-16T15:38:36.328857+0000","last_fullsized":"2024-09-16T15:38:36.328857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:34.525651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933781+0000","last_change":"2024-09-16T15:38:33.937022+0000","last_active":"2024-09-16T15:38:35.933781+0000","last_peered":"2024-09-16T15:38:35.933781+0000","last_clean":"2024-09-16T15:38:35.933781+0000","last_became_active":"2024-09-16T15:38:33.936692+0000","last_became_peered":"2024-09-16T15:38:33.936692+0000","last_unstale":"2024-09-16T15:38:35.933781+0000","last_undegraded":"2024-09-16T15:38:35.933781+0000","last_fullsized":"2024-09-16T15:38:35.933781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:23.195213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933838+0000","last_change":"2024-09-16T15:38:34.933429+0000","last_active":"2024-09-16T15:38:35.933838+0000","last_peered":"2024-09-16T15:38:35.933838+0000","last_clean":"2024-09-16T15:38:35.933838+0000","last_became_active":"2024-09-16T15:38:34.933116+0000","last_became_peered":"2024-09-16T15:38:34.933116+0000","last_unstale":"2024-09-16T15:38:35.933838+0000","last_undegraded":"2024-09-16T15:38:35.933838+0000","last_fullsized":"2024-09-16T15:38:35.933838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:16.415478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932158+0000","last_change":"2024-09-16T15:38:33.929097+0000","last_active":"2024-09-16T15:38:35.932158+0000","last_peered":"2024-09-16T15:38:35.932158+0000","last_clean":"2024-09-16T15:38:35.932158+0000","last_became_active":"2024-09-16T15:38:33.928783+0000","last_became_peered":"2024-09-16T15:38:33.928783+0000","last_unstale":"2024-09-16T15:38:35.932158+0000","last_undegraded":"2024-09-16T15:38:35.932158+0000","last_fullsized":"2024-09-16T15:38:35.932158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327843+0000","last_change":"2024-09-16T15:38:34.955962+0000","last_active":"2024-09-16T15:38:36.327843+0000","last_peered":"2024-09-16T15:38:36.327843+0000","last_clean":"2024-09-16T15:38:36.327843+0000","last_became_active":"2024-09-16T15:38:34.955617+0000","last_became_peered":"2024-09-16T15:38:34.955617+0000","last_unstale":"2024-09-16T15:38:36.327843+0000","last_undegraded":"2024-09-16T15:38:36.327843+0000","last_fullsized":"2024-09-16T15:38:36.327843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:35.342731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935301+0000","last_change":"2024-09-16T15:38:33.933311+0000","last_active":"2024-09-16T15:38:35.935301+0000","last_peered":"2024-09-16T15:38:35.935301+0000","last_clean":"2024-09-16T15:38:35.935301+0000","last_became_active":"2024-09-16T15:38:33.932882+0000","last_became_peered":"2024-09-16T15:38:33.932882+0000","last_unstale":"2024-09-16T15:38:35.935301+0000","last_undegraded":"2024-09-16T15:38:35.935301+0000","last_fullsized":"2024-09-16T15:38:35.935301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:43.856688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328955+0000","last_change":"2024-09-16T15:38:34.959213+0000","last_active":"2024-09-16T15:38:36.328955+0000","last_peered":"2024-09-16T15:38:36.328955+0000","last_clean":"2024-09-16T15:38:36.328955+0000","last_became_active":"2024-09-16T15:38:34.958928+0000","last_became_peered":"2024-09-16T15:38:34.958928+0000","last_unstale":"2024-09-16T15:38:36.328955+0000","last_undegraded":"2024-09-16T15:38:36.328955+0000","last_fullsized":"2024-09-16T15:38:36.328955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:51.093521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932531+0000","last_change":"2024-09-16T15:38:33.939655+0000","last_active":"2024-09-16T15:38:35.932531+0000","last_peered":"2024-09-16T15:38:35.932531+0000","last_clean":"2024-09-16T15:38:35.932531+0000","last_became_active":"2024-09-16T15:38:33.939329+0000","last_became_peered":"2024-09-16T15:38:33.939329+0000","last_unstale":"2024-09-16T15:38:35.932531+0000","last_undegraded":"2024-09-16T15:38:35.932531+0000","last_fullsized":"2024-09-16T15:38:35.932531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329218+0000","last_change":"2024-09-16T15:38:34.956364+0000","last_active":"2024-09-16T15:38:36.329218+0000","last_peered":"2024-09-16T15:38:36.329218+0000","last_clean":"2024-09-16T15:38:36.329218+0000","last_became_active":"2024-09-16T15:38:34.956041+0000","last_became_peered":"2024-09-16T15:38:34.956041+0000","last_unstale":"2024-09-16T15:38:36.329218+0000","last_undegraded":"2024-09-16T15:38:36.329218+0000","last_fullsized":"2024-09-16T15:38:36.329218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:26.092978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329039+0000","last_change":"2024-09-16T15:38:33.940469+0000","last_active":"2024-09-16T15:38:36.329039+0000","last_peered":"2024-09-16T15:38:36.329039+0000","last_clean":"2024-09-16T15:38:36.329039+0000","last_became_active":"2024-09-16T15:38:33.939752+0000","last_became_peered":"2024-09-16T15:38:33.939752+0000","last_unstale":"2024-09-16T15:38:36.329039+0000","last_undegraded":"2024-09-16T15:38:36.329039+0000","last_fullsized":"2024-09-16T15:38:36.329039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:31.202600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328853+0000","last_change":"2024-09-16T15:38:34.946270+0000","last_active":"2024-09-16T15:38:36.328853+0000","last_peered":"2024-09-16T15:38:36.328853+0000","last_clean":"2024-09-16T15:38:36.328853+0000","last_became_active":"2024-09-16T15:38:34.945965+0000","last_became_peered":"2024-09-16T15:38:34.945965+0000","last_unstale":"2024-09-16T15:38:36.328853+0000","last_undegraded":"2024-09-16T15:38:36.328853+0000","last_fullsized":"2024-09-16T15:38:36.328853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932369+0000","last_change":"2024-09-16T15:38:33.937925+0000","last_active":"2024-09-16T15:38:35.932369+0000","last_peered":"2024-09-16T15:38:35.932369+0000","last_clean":"2024-09-16T15:38:35.932369+0000","last_became_active":"2024-09-16T15:38:33.937648+0000","last_became_peered":"2024-09-16T15:38:33.937648+0000","last_unstale":"2024-09-16T15:38:35.932369+0000","last_undegraded":"2024-09-16T15:38:35.932369+0000","last_fullsized":"2024-09-16T15:38:35.932369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328407+0000","last_change":"2024-09-16T15:38:34.964388+0000","last_active":"2024-09-16T15:38:36.328407+0000","last_peered":"2024-09-16T15:38:36.328407+0000","last_clean":"2024-09-16T15:38:36.328407+0000","last_became_active":"2024-09-16T15:38:34.964136+0000","last_became_peered":"2024-09-16T15:38:34.964136+0000","last_unstale":"2024-09-16T15:38:36.328407+0000","last_undegraded":"2024-09-16T15:38:36.328407+0000","last_fullsized":"2024-09-16T15:38:36.328407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:21.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327747+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:38:36.327747+0000","last_peered":"2024-09-16T15:38:36.327747+0000","last_clean":"2024-09-16T15:38:36.327747+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:38:36.327747+0000","last_undegraded":"2024-09-16T15:38:36.327747+0000","last_fullsized":"2024-09-16T15:38:36.327747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327930+0000","last_change":"2024-09-16T15:38:33.941937+0000","last_active":"2024-09-16T15:38:36.327930+0000","last_peered":"2024-09-16T15:38:36.327930+0000","last_clean":"2024-09-16T15:38:36.327930+0000","last_became_active":"2024-09-16T15:38:33.941212+0000","last_became_peered":"2024-09-16T15:38:33.941212+0000","last_unstale":"2024-09-16T15:38:36.327930+0000","last_undegraded":"2024-09-16T15:38:36.327930+0000","last_fullsized":"2024-09-16T15:38:36.327930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328444+0000","last_change":"2024-09-16T15:38:34.962489+0000","last_active":"2024-09-16T15:38:36.328444+0000","last_peered":"2024-09-16T15:38:36.328444+0000","last_clean":"2024-09-16T15:38:36.328444+0000","last_became_active":"2024-09-16T15:38:34.962245+0000","last_became_peered":"2024-09-16T15:38:34.962245+0000","last_unstale":"2024-09-16T15:38:36.328444+0000","last_undegraded":"2024-09-16T15:38:36.328444+0000","last_fullsized":"2024-09-16T15:38:36.328444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.955896+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.955595+0000","last_became_peered":"2024-09-16T15:38:33.955595+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329773+0000","last_change":"2024-09-16T15:38:34.956268+0000","last_active":"2024-09-16T15:38:36.329773+0000","last_peered":"2024-09-16T15:38:36.329773+0000","last_clean":"2024-09-16T15:38:36.329773+0000","last_became_active":"2024-09-16T15:38:34.956010+0000","last_became_peered":"2024-09-16T15:38:34.956010+0000","last_unstale":"2024-09-16T15:38:36.329773+0000","last_undegraded":"2024-09-16T15:38:36.329773+0000","last_fullsized":"2024-09-16T15:38:36.329773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:02.100232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932349+0000","last_change":"2024-09-16T15:38:33.941933+0000","last_active":"2024-09-16T15:38:35.932349+0000","last_peered":"2024-09-16T15:38:35.932349+0000","last_clean":"2024-09-16T15:38:35.932349+0000","last_became_active":"2024-09-16T15:38:33.941648+0000","last_became_peered":"2024-09-16T15:38:33.941648+0000","last_unstale":"2024-09-16T15:38:35.932349+0000","last_undegraded":"2024-09-16T15:38:35.932349+0000","last_fullsized":"2024-09-16T15:38:35.932349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328622+0000","last_change":"2024-09-16T15:38:34.932401+0000","last_active":"2024-09-16T15:38:36.328622+0000","last_peered":"2024-09-16T15:38:36.328622+0000","last_clean":"2024-09-16T15:38:36.328622+0000","last_became_active":"2024-09-16T15:38:34.931203+0000","last_became_peered":"2024-09-16T15:38:34.931203+0000","last_unstale":"2024-09-16T15:38:36.328622+0000","last_undegraded":"2024-09-16T15:38:36.328622+0000","last_fullsized":"2024-09-16T15:38:36.328622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:24.483689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328535+0000","last_change":"2024-09-16T15:38:33.947515+0000","last_active":"2024-09-16T15:38:36.328535+0000","last_peered":"2024-09-16T15:38:36.328535+0000","last_clean":"2024-09-16T15:38:36.328535+0000","last_became_active":"2024-09-16T15:38:33.946118+0000","last_became_peered":"2024-09-16T15:38:33.946118+0000","last_unstale":"2024-09-16T15:38:36.328535+0000","last_undegraded":"2024-09-16T15:38:36.328535+0000","last_fullsized":"2024-09-16T15:38:36.328535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935354+0000","last_change":"2024-09-16T15:38:34.951051+0000","last_active":"2024-09-16T15:38:35.935354+0000","last_peered":"2024-09-16T15:38:35.935354+0000","last_clean":"2024-09-16T15:38:35.935354+0000","last_became_active":"2024-09-16T15:38:34.950115+0000","last_became_peered":"2024-09-16T15:38:34.950115+0000","last_unstale":"2024-09-16T15:38:35.935354+0000","last_undegraded":"2024-09-16T15:38:35.935354+0000","last_fullsized":"2024-09-16T15:38:35.935354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:29.584103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328940+0000","last_change":"2024-09-16T15:38:33.936405+0000","last_active":"2024-09-16T15:38:36.328940+0000","last_peered":"2024-09-16T15:38:36.328940+0000","last_clean":"2024-09-16T15:38:36.328940+0000","last_became_active":"2024-09-16T15:38:33.935318+0000","last_became_peered":"2024-09-16T15:38:33.935318+0000","last_unstale":"2024-09-16T15:38:36.328940+0000","last_undegraded":"2024-09-16T15:38:36.328940+0000","last_fullsized":"2024-09-16T15:38:36.328940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:27.366083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329408+0000","last_change":"2024-09-16T15:38:34.961561+0000","last_active":"2024-09-16T15:38:36.329408+0000","last_peered":"2024-09-16T15:38:36.329408+0000","last_clean":"2024-09-16T15:38:36.329408+0000","last_became_active":"2024-09-16T15:38:34.961174+0000","last_became_peered":"2024-09-16T15:38:34.961174+0000","last_unstale":"2024-09-16T15:38:36.329408+0000","last_undegraded":"2024-09-16T15:38:36.329408+0000","last_fullsized":"2024-09-16T15:38:36.329408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328656+0000","last_change":"2024-09-16T15:38:33.955332+0000","last_active":"2024-09-16T15:38:36.328656+0000","last_peered":"2024-09-16T15:38:36.328656+0000","last_clean":"2024-09-16T15:38:36.328656+0000","last_became_active":"2024-09-16T15:38:33.954340+0000","last_became_peered":"2024-09-16T15:38:33.954340+0000","last_unstale":"2024-09-16T15:38:36.328656+0000","last_undegraded":"2024-09-16T15:38:36.328656+0000","last_fullsized":"2024-09-16T15:38:36.328656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:45.798957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328724+0000","last_change":"2024-09-16T15:38:34.928319+0000","last_active":"2024-09-16T15:38:36.328724+0000","last_peered":"2024-09-16T15:38:36.328724+0000","last_clean":"2024-09-16T15:38:36.328724+0000","last_became_active":"2024-09-16T15:38:34.928058+0000","last_became_peered":"2024-09-16T15:38:34.928058+0000","last_unstale":"2024-09-16T15:38:36.328724+0000","last_undegraded":"2024-09-16T15:38:36.328724+0000","last_fullsized":"2024-09-16T15:38:36.328724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:00.238312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.929048+0000","last_change":"2024-09-16T15:38:33.931369+0000","last_active":"2024-09-16T15:38:36.929048+0000","last_peered":"2024-09-16T15:38:36.929048+0000","last_clean":"2024-09-16T15:38:36.929048+0000","last_became_active":"2024-09-16T15:38:33.928339+0000","last_became_peered":"2024-09-16T15:38:33.928339+0000","last_unstale":"2024-09-16T15:38:36.929048+0000","last_undegraded":"2024-09-16T15:38:36.929048+0000","last_fullsized":"2024-09-16T15:38:36.929048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:44.073989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329649+0000","last_change":"2024-09-16T15:38:34.942463+0000","last_active":"2024-09-16T15:38:36.329649+0000","last_peered":"2024-09-16T15:38:36.329649+0000","last_clean":"2024-09-16T15:38:36.329649+0000","last_became_active":"2024-09-16T15:38:34.940343+0000","last_became_peered":"2024-09-16T15:38:34.940343+0000","last_unstale":"2024-09-16T15:38:36.329649+0000","last_undegraded":"2024-09-16T15:38:36.329649+0000","last_fullsized":"2024-09-16T15:38:36.329649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935853+0000","last_change":"2024-09-16T15:38:33.947537+0000","last_active":"2024-09-16T15:38:35.935853+0000","last_peered":"2024-09-16T15:38:35.935853+0000","last_clean":"2024-09-16T15:38:35.935853+0000","last_became_active":"2024-09-16T15:38:33.946909+0000","last_became_peered":"2024-09-16T15:38:33.946909+0000","last_unstale":"2024-09-16T15:38:35.935853+0000","last_undegraded":"2024-09-16T15:38:35.935853+0000","last_fullsized":"2024-09-16T15:38:35.935853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:03.179729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330207+0000","last_change":"2024-09-16T15:38:34.941844+0000","last_active":"2024-09-16T15:38:36.330207+0000","last_peered":"2024-09-16T15:38:36.330207+0000","last_clean":"2024-09-16T15:38:36.330207+0000","last_became_active":"2024-09-16T15:38:34.940573+0000","last_became_peered":"2024-09-16T15:38:34.940573+0000","last_unstale":"2024-09-16T15:38:36.330207+0000","last_undegraded":"2024-09-16T15:38:36.330207+0000","last_fullsized":"2024-09-16T15:38:36.330207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:17.824920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.954548+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.954266+0000","last_became_peered":"2024-09-16T15:38:33.954266+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:14.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934862+0000","last_change":"2024-09-16T15:38:34.951875+0000","last_active":"2024-09-16T15:38:35.934862+0000","last_peered":"2024-09-16T15:38:35.934862+0000","last_clean":"2024-09-16T15:38:35.934862+0000","last_became_active":"2024-09-16T15:38:34.946733+0000","last_became_peered":"2024-09-16T15:38:34.946733+0000","last_unstale":"2024-09-16T15:38:35.934862+0000","last_undegraded":"2024-09-16T15:38:35.934862+0000","last_fullsized":"2024-09-16T15:38:35.934862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:35.351445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.947052+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.946593+0000","last_became_peered":"2024-09-16T15:38:33.946593+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:37.618001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328906+0000","last_change":"2024-09-16T15:38:34.933312+0000","last_active":"2024-09-16T15:38:36.328906+0000","last_peered":"2024-09-16T15:38:36.328906+0000","last_clean":"2024-09-16T15:38:36.328906+0000","last_became_active":"2024-09-16T15:38:34.933037+0000","last_became_peered":"2024-09-16T15:38:34.933037+0000","last_unstale":"2024-09-16T15:38:36.328906+0000","last_undegraded":"2024-09-16T15:38:36.328906+0000","last_fullsized":"2024-09-16T15:38:36.328906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:15.092528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.943983+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.943683+0000","last_became_peered":"2024-09-16T15:38:33.943683+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.572327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328590+0000","last_change":"2024-09-16T15:38:34.943957+0000","last_active":"2024-09-16T15:38:36.328590+0000","last_peered":"2024-09-16T15:38:36.328590+0000","last_clean":"2024-09-16T15:38:36.328590+0000","last_became_active":"2024-09-16T15:38:34.941591+0000","last_became_peered":"2024-09-16T15:38:34.941591+0000","last_unstale":"2024-09-16T15:38:36.328590+0000","last_undegraded":"2024-09-16T15:38:36.328590+0000","last_fullsized":"2024-09-16T15:38:36.328590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328378+0000","last_change":"2024-09-16T15:38:33.954584+0000","last_active":"2024-09-16T15:38:36.328378+0000","last_peered":"2024-09-16T15:38:36.328378+0000","last_clean":"2024-09-16T15:38:36.328378+0000","last_became_active":"2024-09-16T15:38:33.954279+0000","last_became_peered":"2024-09-16T15:38:33.954279+0000","last_unstale":"2024-09-16T15:38:36.328378+0000","last_undegraded":"2024-09-16T15:38:36.328378+0000","last_fullsized":"2024-09-16T15:38:36.328378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933817+0000","last_change":"2024-09-16T15:38:34.949723+0000","last_active":"2024-09-16T15:38:35.933817+0000","last_peered":"2024-09-16T15:38:35.933817+0000","last_clean":"2024-09-16T15:38:35.933817+0000","last_became_active":"2024-09-16T15:38:34.949433+0000","last_became_peered":"2024-09-16T15:38:34.949433+0000","last_unstale":"2024-09-16T15:38:35.933817+0000","last_undegraded":"2024-09-16T15:38:35.933817+0000","last_fullsized":"2024-09-16T15:38:35.933817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:33.866690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329345+0000","last_change":"2024-09-16T15:38:34.961924+0000","last_active":"2024-09-16T15:38:36.329345+0000","last_peered":"2024-09-16T15:38:36.329345+0000","last_clean":"2024-09-16T15:38:36.329345+0000","last_became_active":"2024-09-16T15:38:34.961607+0000","last_became_peered":"2024-09-16T15:38:34.961607+0000","last_unstale":"2024-09-16T15:38:36.329345+0000","last_undegraded":"2024-09-16T15:38:36.329345+0000","last_fullsized":"2024-09-16T15:38:36.329345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:58.529110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328347+0000","last_change":"2024-09-16T15:38:33.955384+0000","last_active":"2024-09-16T15:38:36.328347+0000","last_peered":"2024-09-16T15:38:36.328347+0000","last_clean":"2024-09-16T15:38:36.328347+0000","last_became_active":"2024-09-16T15:38:33.954532+0000","last_became_peered":"2024-09-16T15:38:33.954532+0000","last_unstale":"2024-09-16T15:38:36.328347+0000","last_undegraded":"2024-09-16T15:38:36.328347+0000","last_fullsized":"2024-09-16T15:38:36.328347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:28.577867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328852+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328852+0000","last_peered":"2024-09-16T15:38:36.328852+0000","last_clean":"2024-09-16T15:38:36.328852+0000","last_became_active":"2024-09-16T15:38:34.958217+0000","last_became_peered":"2024-09-16T15:38:34.958217+0000","last_unstale":"2024-09-16T15:38:36.328852+0000","last_undegraded":"2024-09-16T15:38:36.328852+0000","last_fullsized":"2024-09-16T15:38:36.328852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:27.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328469+0000","last_change":"2024-09-16T15:38:33.956064+0000","last_active":"2024-09-16T15:38:36.328469+0000","last_peered":"2024-09-16T15:38:36.328469+0000","last_clean":"2024-09-16T15:38:36.328469+0000","last_became_active":"2024-09-16T15:38:33.955717+0000","last_became_peered":"2024-09-16T15:38:33.955717+0000","last_unstale":"2024-09-16T15:38:36.328469+0000","last_undegraded":"2024-09-16T15:38:36.328469+0000","last_fullsized":"2024-09-16T15:38:36.328469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:12.649947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330233+0000","last_change":"2024-09-16T15:38:34.937110+0000","last_active":"2024-09-16T15:38:36.330233+0000","last_peered":"2024-09-16T15:38:36.330233+0000","last_clean":"2024-09-16T15:38:36.330233+0000","last_became_active":"2024-09-16T15:38:34.936439+0000","last_became_peered":"2024-09-16T15:38:34.936439+0000","last_unstale":"2024-09-16T15:38:36.330233+0000","last_undegraded":"2024-09-16T15:38:36.330233+0000","last_fullsized":"2024-09-16T15:38:36.330233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:49.927493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328261+0000","last_change":"2024-09-16T15:38:33.944467+0000","last_active":"2024-09-16T15:38:36.328261+0000","last_peered":"2024-09-16T15:38:36.328261+0000","last_clean":"2024-09-16T15:38:36.328261+0000","last_became_active":"2024-09-16T15:38:33.944211+0000","last_became_peered":"2024-09-16T15:38:33.944211+0000","last_unstale":"2024-09-16T15:38:36.328261+0000","last_undegraded":"2024-09-16T15:38:36.328261+0000","last_fullsized":"2024-09-16T15:38:36.328261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328305+0000","last_change":"2024-09-16T15:38:34.932456+0000","last_active":"2024-09-16T15:38:36.328305+0000","last_peered":"2024-09-16T15:38:36.328305+0000","last_clean":"2024-09-16T15:38:36.328305+0000","last_became_active":"2024-09-16T15:38:34.931374+0000","last_became_peered":"2024-09-16T15:38:34.931374+0000","last_unstale":"2024-09-16T15:38:36.328305+0000","last_undegraded":"2024-09-16T15:38:36.328305+0000","last_fullsized":"2024-09-16T15:38:36.328305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:26.787621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330346+0000","last_change":"2024-09-16T15:38:33.945170+0000","last_active":"2024-09-16T15:38:36.330346+0000","last_peered":"2024-09-16T15:38:36.330346+0000","last_clean":"2024-09-16T15:38:36.330346+0000","last_became_active":"2024-09-16T15:38:33.944681+0000","last_became_peered":"2024-09-16T15:38:33.944681+0000","last_unstale":"2024-09-16T15:38:36.330346+0000","last_undegraded":"2024-09-16T15:38:36.330346+0000","last_fullsized":"2024-09-16T15:38:36.330346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:30.232249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":245760,"data_stored":81953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":672833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":241664,"data_stored":81987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:38:49.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:38:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.649+0000 7f6923e00640 1 -- 172.21.15.60:0/2422046033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f691c0ccf00 msgr2=0x7f691c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.649+0000 7f6923e00640 1 --2- 172.21.15.60:0/2422046033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f691c0ccf00 0x7f691c0acd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6908005fd0 tx=0x7f690802a540 comp rx=0 tx=0).stop 2024-09-16T15:38:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.649+0000 7f6923e00640 1 -- 172.21.15.60:0/2422046033 shutdown_connections 2024-09-16T15:38:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.649+0000 7f6923e00640 1 --2- 172.21.15.60:0/2422046033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f691c0ccf00 0x7f691c0acd30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.649+0000 7f6923e00640 1 --2- 172.21.15.60:0/2422046033 >> v2:172.21.15.100:3300/0 conn(0x7f691c15f5a0 0x7f691c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.649+0000 7f6923e00640 1 -- 172.21.15.60:0/2422046033 >> 172.21.15.60:0/2422046033 conn(0x7f691c0b1950 msgr2=0x7f691c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.649+0000 7f6923e00640 1 -- 172.21.15.60:0/2422046033 shutdown_connections 2024-09-16T15:38:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.650+0000 7f6923e00640 1 -- 172.21.15.60:0/2422046033 wait complete. 2024-09-16T15:38:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.651+0000 7f6923e00640 1 Processor -- start 2024-09-16T15:38:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.651+0000 7f6923e00640 1 -- start start 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6923e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f691c0ad270 0x7f691c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6923e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f691c0ccf00 0x7f691c154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6923e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f691c15f5a0 0x7f691c1f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6923e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f691c06f2e0 con 0x7f691c0ccf00 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6923e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f691c06f160 con 0x7f691c15f5a0 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6923e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f691c06f460 con 0x7f691c0ad270 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6920a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f691c0ccf00 0x7f691c154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6920a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f691c0ccf00 0x7f691c154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43360/0 (socket says 172.21.15.60:43360) 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.652+0000 7f6920a00640 1 -- 172.21.15.60:0/3918102252 learned_addr learned my addr 172.21.15.60:0/3918102252 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6921400640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f691c0ad270 0x7f691c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6921e00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f691c15f5a0 0x7f691c1f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6920a00640 1 -- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f691c0ad270 msgr2=0x7f691c14daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6920a00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f691c0ad270 0x7f691c14daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6920a00640 1 -- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f691c15f5a0 msgr2=0x7f691c1f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6920a00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f691c15f5a0 0x7f691c1f65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6920a00640 1 -- 172.21.15.60:0/3918102252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c157eb0 con 0x7f691c0ccf00 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6920a00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f691c0ccf00 0x7f691c154450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6908009850 tx=0x7f69080038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6919600640 1 -- 172.21.15.60:0/3918102252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690803f070 con 0x7f691c0ccf00 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.653+0000 7f6919600640 1 -- 172.21.15.60:0/3918102252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69080047a0 con 0x7f691c0ccf00 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.654+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c158190 con 0x7f691c0ccf00 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.654+0000 7f6919600640 1 -- 172.21.15.60:0/3918102252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908035030 con 0x7f691c0ccf00 2024-09-16T15:38:49.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.654+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c14e030 con 0x7f691c0ccf00 2024-09-16T15:38:49.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.655+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c06f9f0 con 0x7f691c0ccf00 2024-09-16T15:38:49.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.656+0000 7f6919600640 1 -- 172.21.15.60:0/3918102252 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6908042070 con 0x7f691c0ccf00 2024-09-16T15:38:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.657+0000 7f6919600640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68fc078850 0x7f68fc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.657+0000 7f6919600640 1 -- 172.21.15.60:0/3918102252 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f69080bb660 con 0x7f691c0ccf00 2024-09-16T15:38:49.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.657+0000 7f6921400640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68fc078850 0x7f68fc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.658+0000 7f6921400640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68fc078850 0x7f68fc07ad10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f69100095f0 tx=0x7f6910005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:49.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.661+0000 7f6919600640 1 -- 172.21.15.60:0/3918102252 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6908083670 con 0x7f691c0ccf00 2024-09-16T15:38:49.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.852+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f691c15d860 con 0x7f68fc078850 2024-09-16T15:38:49.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.857+0000 7f6919600640 1 -- 172.21.15.60:0/3918102252 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f691c15d860 con 0x7f68fc078850 2024-09-16T15:38:49.861 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:49.862 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:38:49.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68fc078850 msgr2=0x7f68fc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68fc078850 0x7f68fc07ad10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f69100095f0 tx=0x7f6910005f90 comp rx=0 tx=0).stop 2024-09-16T15:38:49.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f691c0ccf00 msgr2=0x7f691c154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:49.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f691c0ccf00 0x7f691c154450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6908009850 tx=0x7f69080038a0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 shutdown_connections 2024-09-16T15:38:49.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68fc078850 0x7f68fc07ad10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f691c15f5a0 0x7f691c1f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f691c0ccf00 0x7f691c154450 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 --2- 172.21.15.60:0/3918102252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f691c0ad270 0x7f691c14daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:49.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 >> 172.21.15.60:0/3918102252 conn(0x7f691c0b1950 msgr2=0x7f691c15e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:49.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 shutdown_connections 2024-09-16T15:38:49.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:49.861+0000 7f6923e00640 1 -- 172.21.15.60:0/3918102252 wait complete. 2024-09-16T15:38:49.877 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-16T15:38:49.712303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235932,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738788,"statfs":{"total":773094113280,"available":772852518912,"internally_reserved":0,"allocated":3121152,"data_stored":1840172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003610"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328961+0000","last_change":"2024-09-16T15:38:34.948046+0000","last_active":"2024-09-16T15:38:36.328961+0000","last_peered":"2024-09-16T15:38:36.328961+0000","last_clean":"2024-09-16T15:38:36.328961+0000","last_became_active":"2024-09-16T15:38:34.947778+0000","last_became_peered":"2024-09-16T15:38:34.947778+0000","last_unstale":"2024-09-16T15:38:36.328961+0000","last_undegraded":"2024-09-16T15:38:36.328961+0000","last_fullsized":"2024-09-16T15:38:36.328961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329296+0000","last_change":"2024-09-16T15:38:34.934287+0000","last_active":"2024-09-16T15:38:36.329296+0000","last_peered":"2024-09-16T15:38:36.329296+0000","last_clean":"2024-09-16T15:38:36.329296+0000","last_became_active":"2024-09-16T15:38:34.934004+0000","last_became_peered":"2024-09-16T15:38:34.934004+0000","last_unstale":"2024-09-16T15:38:36.329296+0000","last_undegraded":"2024-09-16T15:38:36.329296+0000","last_fullsized":"2024-09-16T15:38:36.329296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:43.843291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330192+0000","last_change":"2024-09-16T15:38:34.937280+0000","last_active":"2024-09-16T15:38:36.330192+0000","last_peered":"2024-09-16T15:38:36.330192+0000","last_clean":"2024-09-16T15:38:36.330192+0000","last_became_active":"2024-09-16T15:38:34.936979+0000","last_became_peered":"2024-09-16T15:38:34.936979+0000","last_unstale":"2024-09-16T15:38:36.330192+0000","last_undegraded":"2024-09-16T15:38:36.330192+0000","last_fullsized":"2024-09-16T15:38:36.330192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:16.363921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328077+0000","last_change":"2024-09-16T15:38:34.956532+0000","last_active":"2024-09-16T15:38:36.328077+0000","last_peered":"2024-09-16T15:38:36.328077+0000","last_clean":"2024-09-16T15:38:36.328077+0000","last_became_active":"2024-09-16T15:38:34.956261+0000","last_became_peered":"2024-09-16T15:38:34.956261+0000","last_unstale":"2024-09-16T15:38:36.328077+0000","last_undegraded":"2024-09-16T15:38:36.328077+0000","last_fullsized":"2024-09-16T15:38:36.328077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328317+0000","last_change":"2024-09-16T15:38:34.946850+0000","last_active":"2024-09-16T15:38:36.328317+0000","last_peered":"2024-09-16T15:38:36.328317+0000","last_clean":"2024-09-16T15:38:36.328317+0000","last_became_active":"2024-09-16T15:38:34.946584+0000","last_became_peered":"2024-09-16T15:38:34.946584+0000","last_unstale":"2024-09-16T15:38:36.328317+0000","last_undegraded":"2024-09-16T15:38:36.328317+0000","last_fullsized":"2024-09-16T15:38:36.328317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328172+0000","last_change":"2024-09-16T15:38:34.946360+0000","last_active":"2024-09-16T15:38:36.328172+0000","last_peered":"2024-09-16T15:38:36.328172+0000","last_clean":"2024-09-16T15:38:36.328172+0000","last_became_active":"2024-09-16T15:38:34.946022+0000","last_became_peered":"2024-09-16T15:38:34.946022+0000","last_unstale":"2024-09-16T15:38:36.328172+0000","last_undegraded":"2024-09-16T15:38:36.328172+0000","last_fullsized":"2024-09-16T15:38:36.328172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:52.783126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328807+0000","last_change":"2024-09-16T15:38:34.948730+0000","last_active":"2024-09-16T15:38:36.328807+0000","last_peered":"2024-09-16T15:38:36.328807+0000","last_clean":"2024-09-16T15:38:36.328807+0000","last_became_active":"2024-09-16T15:38:34.948488+0000","last_became_peered":"2024-09-16T15:38:34.948488+0000","last_unstale":"2024-09-16T15:38:36.328807+0000","last_undegraded":"2024-09-16T15:38:36.328807+0000","last_fullsized":"2024-09-16T15:38:36.328807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327904+0000","last_change":"2024-09-16T15:38:34.939960+0000","last_active":"2024-09-16T15:38:36.327904+0000","last_peered":"2024-09-16T15:38:36.327904+0000","last_clean":"2024-09-16T15:38:36.327904+0000","last_became_active":"2024-09-16T15:38:34.939730+0000","last_became_peered":"2024-09-16T15:38:34.939730+0000","last_unstale":"2024-09-16T15:38:36.327904+0000","last_undegraded":"2024-09-16T15:38:36.327904+0000","last_fullsized":"2024-09-16T15:38:36.327904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:35.445142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329103+0000","last_change":"2024-09-16T15:38:34.963612+0000","last_active":"2024-09-16T15:38:36.329103+0000","last_peered":"2024-09-16T15:38:36.329103+0000","last_clean":"2024-09-16T15:38:36.329103+0000","last_became_active":"2024-09-16T15:38:34.963287+0000","last_became_peered":"2024-09-16T15:38:34.963287+0000","last_unstale":"2024-09-16T15:38:36.329103+0000","last_undegraded":"2024-09-16T15:38:36.329103+0000","last_fullsized":"2024-09-16T15:38:36.329103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:53.707175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935830+0000","last_change":"2024-09-16T15:38:34.959111+0000","last_active":"2024-09-16T15:38:35.935830+0000","last_peered":"2024-09-16T15:38:35.935830+0000","last_clean":"2024-09-16T15:38:35.935830+0000","last_became_active":"2024-09-16T15:38:34.958733+0000","last_became_peered":"2024-09-16T15:38:34.958733+0000","last_unstale":"2024-09-16T15:38:35.935830+0000","last_undegraded":"2024-09-16T15:38:35.935830+0000","last_fullsized":"2024-09-16T15:38:35.935830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:21.940460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934331+0000","last_change":"2024-09-16T15:38:34.963928+0000","last_active":"2024-09-16T15:38:35.934331+0000","last_peered":"2024-09-16T15:38:35.934331+0000","last_clean":"2024-09-16T15:38:35.934331+0000","last_became_active":"2024-09-16T15:38:34.963418+0000","last_became_peered":"2024-09-16T15:38:34.963418+0000","last_unstale":"2024-09-16T15:38:35.934331+0000","last_undegraded":"2024-09-16T15:38:35.934331+0000","last_fullsized":"2024-09-16T15:38:35.934331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:54.016578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935041+0000","last_change":"2024-09-16T15:38:34.951463+0000","last_active":"2024-09-16T15:38:35.935041+0000","last_peered":"2024-09-16T15:38:35.935041+0000","last_clean":"2024-09-16T15:38:35.935041+0000","last_became_active":"2024-09-16T15:38:34.949972+0000","last_became_peered":"2024-09-16T15:38:34.949972+0000","last_unstale":"2024-09-16T15:38:35.935041+0000","last_undegraded":"2024-09-16T15:38:35.935041+0000","last_fullsized":"2024-09-16T15:38:35.935041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:36.356482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328377+0000","last_change":"2024-09-16T15:38:34.936161+0000","last_active":"2024-09-16T15:38:36.328377+0000","last_peered":"2024-09-16T15:38:36.328377+0000","last_clean":"2024-09-16T15:38:36.328377+0000","last_became_active":"2024-09-16T15:38:34.935881+0000","last_became_peered":"2024-09-16T15:38:34.935881+0000","last_unstale":"2024-09-16T15:38:36.328377+0000","last_undegraded":"2024-09-16T15:38:36.328377+0000","last_fullsized":"2024-09-16T15:38:36.328377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329953+0000","last_change":"2024-09-16T15:38:34.934084+0000","last_active":"2024-09-16T15:38:36.329953+0000","last_peered":"2024-09-16T15:38:36.329953+0000","last_clean":"2024-09-16T15:38:36.329953+0000","last_became_active":"2024-09-16T15:38:34.933716+0000","last_became_peered":"2024-09-16T15:38:34.933716+0000","last_unstale":"2024-09-16T15:38:36.329953+0000","last_undegraded":"2024-09-16T15:38:36.329953+0000","last_fullsized":"2024-09-16T15:38:36.329953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:51.549114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328795+0000","last_change":"2024-09-16T15:38:34.951427+0000","last_active":"2024-09-16T15:38:36.328795+0000","last_peered":"2024-09-16T15:38:36.328795+0000","last_clean":"2024-09-16T15:38:36.328795+0000","last_became_active":"2024-09-16T15:38:34.949484+0000","last_became_peered":"2024-09-16T15:38:34.949484+0000","last_unstale":"2024-09-16T15:38:36.328795+0000","last_undegraded":"2024-09-16T15:38:36.328795+0000","last_fullsized":"2024-09-16T15:38:36.328795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:54.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327906+0000","last_change":"2024-09-16T15:38:34.948833+0000","last_active":"2024-09-16T15:38:36.327906+0000","last_peered":"2024-09-16T15:38:36.327906+0000","last_clean":"2024-09-16T15:38:36.327906+0000","last_became_active":"2024-09-16T15:38:34.948127+0000","last_became_peered":"2024-09-16T15:38:34.948127+0000","last_unstale":"2024-09-16T15:38:36.327906+0000","last_undegraded":"2024-09-16T15:38:36.327906+0000","last_fullsized":"2024-09-16T15:38:36.327906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:31.028333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329716+0000","last_change":"2024-09-16T15:38:34.948165+0000","last_active":"2024-09-16T15:38:36.329716+0000","last_peered":"2024-09-16T15:38:36.329716+0000","last_clean":"2024-09-16T15:38:36.329716+0000","last_became_active":"2024-09-16T15:38:34.947862+0000","last_became_peered":"2024-09-16T15:38:34.947862+0000","last_unstale":"2024-09-16T15:38:36.329716+0000","last_undegraded":"2024-09-16T15:38:36.329716+0000","last_fullsized":"2024-09-16T15:38:36.329716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:07.065693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932521+0000","last_change":"2024-09-16T15:38:34.936088+0000","last_active":"2024-09-16T15:38:35.932521+0000","last_peered":"2024-09-16T15:38:35.932521+0000","last_clean":"2024-09-16T15:38:35.932521+0000","last_became_active":"2024-09-16T15:38:34.935649+0000","last_became_peered":"2024-09-16T15:38:34.935649+0000","last_unstale":"2024-09-16T15:38:35.932521+0000","last_undegraded":"2024-09-16T15:38:35.932521+0000","last_fullsized":"2024-09-16T15:38:35.932521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:56.948937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328715+0000","last_change":"2024-09-16T15:38:34.951511+0000","last_active":"2024-09-16T15:38:36.328715+0000","last_peered":"2024-09-16T15:38:36.328715+0000","last_clean":"2024-09-16T15:38:36.328715+0000","last_became_active":"2024-09-16T15:38:34.951241+0000","last_became_peered":"2024-09-16T15:38:34.951241+0000","last_unstale":"2024-09-16T15:38:36.328715+0000","last_undegraded":"2024-09-16T15:38:36.328715+0000","last_fullsized":"2024-09-16T15:38:36.328715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:52.005508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934261+0000","last_change":"2024-09-16T15:38:34.950265+0000","last_active":"2024-09-16T15:38:35.934261+0000","last_peered":"2024-09-16T15:38:35.934261+0000","last_clean":"2024-09-16T15:38:35.934261+0000","last_became_active":"2024-09-16T15:38:34.949457+0000","last_became_peered":"2024-09-16T15:38:34.949457+0000","last_unstale":"2024-09-16T15:38:35.934261+0000","last_undegraded":"2024-09-16T15:38:35.934261+0000","last_fullsized":"2024-09-16T15:38:35.934261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:33.694559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330049+0000","last_change":"2024-09-16T15:38:34.942753+0000","last_active":"2024-09-16T15:38:36.330049+0000","last_peered":"2024-09-16T15:38:36.330049+0000","last_clean":"2024-09-16T15:38:36.330049+0000","last_became_active":"2024-09-16T15:38:34.940841+0000","last_became_peered":"2024-09-16T15:38:34.940841+0000","last_unstale":"2024-09-16T15:38:36.330049+0000","last_undegraded":"2024-09-16T15:38:36.330049+0000","last_fullsized":"2024-09-16T15:38:36.330049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:22.740035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328374+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328374+0000","last_peered":"2024-09-16T15:38:36.328374+0000","last_clean":"2024-09-16T15:38:36.328374+0000","last_became_active":"2024-09-16T15:38:34.958240+0000","last_became_peered":"2024-09-16T15:38:34.958240+0000","last_unstale":"2024-09-16T15:38:36.328374+0000","last_undegraded":"2024-09-16T15:38:36.328374+0000","last_fullsized":"2024-09-16T15:38:36.328374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328083+0000","last_change":"2024-09-16T15:38:34.952703+0000","last_active":"2024-09-16T15:38:36.328083+0000","last_peered":"2024-09-16T15:38:36.328083+0000","last_clean":"2024-09-16T15:38:36.328083+0000","last_became_active":"2024-09-16T15:38:34.949406+0000","last_became_peered":"2024-09-16T15:38:34.949406+0000","last_unstale":"2024-09-16T15:38:36.328083+0000","last_undegraded":"2024-09-16T15:38:36.328083+0000","last_fullsized":"2024-09-16T15:38:36.328083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:20.072155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328173+0000","last_change":"2024-09-16T15:38:34.948442+0000","last_active":"2024-09-16T15:38:36.328173+0000","last_peered":"2024-09-16T15:38:36.328173+0000","last_clean":"2024-09-16T15:38:36.328173+0000","last_became_active":"2024-09-16T15:38:34.947564+0000","last_became_peered":"2024-09-16T15:38:34.947564+0000","last_unstale":"2024-09-16T15:38:36.328173+0000","last_undegraded":"2024-09-16T15:38:36.328173+0000","last_fullsized":"2024-09-16T15:38:36.328173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328339+0000","last_change":"2024-09-16T15:38:34.940042+0000","last_active":"2024-09-16T15:38:36.328339+0000","last_peered":"2024-09-16T15:38:36.328339+0000","last_clean":"2024-09-16T15:38:36.328339+0000","last_became_active":"2024-09-16T15:38:34.939792+0000","last_became_peered":"2024-09-16T15:38:34.939792+0000","last_unstale":"2024-09-16T15:38:36.328339+0000","last_undegraded":"2024-09-16T15:38:36.328339+0000","last_fullsized":"2024-09-16T15:38:36.328339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:21.386845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935854+0000","last_change":"2024-09-16T15:38:34.938919+0000","last_active":"2024-09-16T15:38:35.935854+0000","last_peered":"2024-09-16T15:38:35.935854+0000","last_clean":"2024-09-16T15:38:35.935854+0000","last_became_active":"2024-09-16T15:38:34.938608+0000","last_became_peered":"2024-09-16T15:38:34.938608+0000","last_unstale":"2024-09-16T15:38:35.935854+0000","last_undegraded":"2024-09-16T15:38:35.935854+0000","last_fullsized":"2024-09-16T15:38:35.935854+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:44.137157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329086+0000","last_change":"2024-09-16T15:38:34.934519+0000","last_active":"2024-09-16T15:38:36.329086+0000","last_peered":"2024-09-16T15:38:36.329086+0000","last_clean":"2024-09-16T15:38:36.329086+0000","last_became_active":"2024-09-16T15:38:34.932223+0000","last_became_peered":"2024-09-16T15:38:34.932223+0000","last_unstale":"2024-09-16T15:38:36.329086+0000","last_undegraded":"2024-09-16T15:38:36.329086+0000","last_fullsized":"2024-09-16T15:38:36.329086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.673101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329499+0000","last_change":"2024-09-16T15:38:34.941297+0000","last_active":"2024-09-16T15:38:36.329499+0000","last_peered":"2024-09-16T15:38:36.329499+0000","last_clean":"2024-09-16T15:38:36.329499+0000","last_became_active":"2024-09-16T15:38:34.941007+0000","last_became_peered":"2024-09-16T15:38:34.941007+0000","last_unstale":"2024-09-16T15:38:36.329499+0000","last_undegraded":"2024-09-16T15:38:36.329499+0000","last_fullsized":"2024-09-16T15:38:36.329499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:48.460862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330077+0000","last_change":"2024-09-16T15:38:34.941110+0000","last_active":"2024-09-16T15:38:36.330077+0000","last_peered":"2024-09-16T15:38:36.330077+0000","last_clean":"2024-09-16T15:38:36.330077+0000","last_became_active":"2024-09-16T15:38:34.940650+0000","last_became_peered":"2024-09-16T15:38:34.940650+0000","last_unstale":"2024-09-16T15:38:36.330077+0000","last_undegraded":"2024-09-16T15:38:36.330077+0000","last_fullsized":"2024-09-16T15:38:36.330077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:26.093004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328522+0000","last_change":"2024-09-16T15:38:34.952930+0000","last_active":"2024-09-16T15:38:36.328522+0000","last_peered":"2024-09-16T15:38:36.328522+0000","last_clean":"2024-09-16T15:38:36.328522+0000","last_became_active":"2024-09-16T15:38:34.952652+0000","last_became_peered":"2024-09-16T15:38:34.952652+0000","last_unstale":"2024-09-16T15:38:36.328522+0000","last_undegraded":"2024-09-16T15:38:36.328522+0000","last_fullsized":"2024-09-16T15:38:36.328522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932490+0000","last_change":"2024-09-16T15:38:34.961835+0000","last_active":"2024-09-16T15:38:35.932490+0000","last_peered":"2024-09-16T15:38:35.932490+0000","last_clean":"2024-09-16T15:38:35.932490+0000","last_became_active":"2024-09-16T15:38:34.961529+0000","last_became_peered":"2024-09-16T15:38:34.961529+0000","last_unstale":"2024-09-16T15:38:35.932490+0000","last_undegraded":"2024-09-16T15:38:35.932490+0000","last_fullsized":"2024-09-16T15:38:35.932490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:11.053973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933841+0000","last_change":"2024-09-16T15:38:34.942442+0000","last_active":"2024-09-16T15:38:35.933841+0000","last_peered":"2024-09-16T15:38:35.933841+0000","last_clean":"2024-09-16T15:38:35.933841+0000","last_became_active":"2024-09-16T15:38:34.942144+0000","last_became_peered":"2024-09-16T15:38:34.942144+0000","last_unstale":"2024-09-16T15:38:35.933841+0000","last_undegraded":"2024-09-16T15:38:35.933841+0000","last_fullsized":"2024-09-16T15:38:35.933841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:43.399475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934635+0000","last_change":"2024-09-16T15:38:34.941230+0000","last_active":"2024-09-16T15:38:35.934635+0000","last_peered":"2024-09-16T15:38:35.934635+0000","last_clean":"2024-09-16T15:38:35.934635+0000","last_became_active":"2024-09-16T15:38:34.940613+0000","last_became_peered":"2024-09-16T15:38:34.940613+0000","last_unstale":"2024-09-16T15:38:35.934635+0000","last_undegraded":"2024-09-16T15:38:35.934635+0000","last_fullsized":"2024-09-16T15:38:35.934635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:15.007186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328673+0000","last_change":"2024-09-16T15:38:33.949962+0000","last_active":"2024-09-16T15:38:36.328673+0000","last_peered":"2024-09-16T15:38:36.328673+0000","last_clean":"2024-09-16T15:38:36.328673+0000","last_became_active":"2024-09-16T15:38:33.949620+0000","last_became_peered":"2024-09-16T15:38:33.949620+0000","last_unstale":"2024-09-16T15:38:36.328673+0000","last_undegraded":"2024-09-16T15:38:36.328673+0000","last_fullsized":"2024-09-16T15:38:36.328673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330281+0000","last_change":"2024-09-16T15:38:34.947478+0000","last_active":"2024-09-16T15:38:36.330281+0000","last_peered":"2024-09-16T15:38:36.330281+0000","last_clean":"2024-09-16T15:38:36.330281+0000","last_became_active":"2024-09-16T15:38:34.947158+0000","last_became_peered":"2024-09-16T15:38:34.947158+0000","last_unstale":"2024-09-16T15:38:36.330281+0000","last_undegraded":"2024-09-16T15:38:36.330281+0000","last_fullsized":"2024-09-16T15:38:36.330281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:24.497838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933892+0000","last_change":"2024-09-16T15:38:33.931830+0000","last_active":"2024-09-16T15:38:35.933892+0000","last_peered":"2024-09-16T15:38:35.933892+0000","last_clean":"2024-09-16T15:38:35.933892+0000","last_became_active":"2024-09-16T15:38:33.931129+0000","last_became_peered":"2024-09-16T15:38:33.931129+0000","last_unstale":"2024-09-16T15:38:35.933892+0000","last_undegraded":"2024-09-16T15:38:35.933892+0000","last_fullsized":"2024-09-16T15:38:35.933892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:54.747927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327884+0000","last_change":"2024-09-16T15:38:33.943597+0000","last_active":"2024-09-16T15:38:36.327884+0000","last_peered":"2024-09-16T15:38:36.327884+0000","last_clean":"2024-09-16T15:38:36.327884+0000","last_became_active":"2024-09-16T15:38:33.940904+0000","last_became_peered":"2024-09-16T15:38:33.940904+0000","last_unstale":"2024-09-16T15:38:36.327884+0000","last_undegraded":"2024-09-16T15:38:36.327884+0000","last_fullsized":"2024-09-16T15:38:36.327884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:13.818889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328165+0000","last_change":"2024-09-16T15:38:34.942424+0000","last_active":"2024-09-16T15:38:36.328165+0000","last_peered":"2024-09-16T15:38:36.328165+0000","last_clean":"2024-09-16T15:38:36.328165+0000","last_became_active":"2024-09-16T15:38:34.942111+0000","last_became_peered":"2024-09-16T15:38:34.942111+0000","last_unstale":"2024-09-16T15:38:36.328165+0000","last_undegraded":"2024-09-16T15:38:36.328165+0000","last_fullsized":"2024-09-16T15:38:36.328165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:52.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328244+0000","last_change":"2024-09-16T15:38:33.945014+0000","last_active":"2024-09-16T15:38:36.328244+0000","last_peered":"2024-09-16T15:38:36.328244+0000","last_clean":"2024-09-16T15:38:36.328244+0000","last_became_active":"2024-09-16T15:38:33.944742+0000","last_became_peered":"2024-09-16T15:38:33.944742+0000","last_unstale":"2024-09-16T15:38:36.328244+0000","last_undegraded":"2024-09-16T15:38:36.328244+0000","last_fullsized":"2024-09-16T15:38:36.328244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328641+0000","last_change":"2024-09-16T15:38:34.933709+0000","last_active":"2024-09-16T15:38:36.328641+0000","last_peered":"2024-09-16T15:38:36.328641+0000","last_clean":"2024-09-16T15:38:36.328641+0000","last_became_active":"2024-09-16T15:38:34.932126+0000","last_became_peered":"2024-09-16T15:38:34.932126+0000","last_unstale":"2024-09-16T15:38:36.328641+0000","last_undegraded":"2024-09-16T15:38:36.328641+0000","last_fullsized":"2024-09-16T15:38:36.328641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:44.738695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935745+0000","last_change":"2024-09-16T15:38:33.945379+0000","last_active":"2024-09-16T15:38:35.935745+0000","last_peered":"2024-09-16T15:38:35.935745+0000","last_clean":"2024-09-16T15:38:35.935745+0000","last_became_active":"2024-09-16T15:38:33.944531+0000","last_became_peered":"2024-09-16T15:38:33.944531+0000","last_unstale":"2024-09-16T15:38:35.935745+0000","last_undegraded":"2024-09-16T15:38:35.935745+0000","last_fullsized":"2024-09-16T15:38:35.935745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:09.939574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329480+0000","last_change":"2024-09-16T15:38:34.961412+0000","last_active":"2024-09-16T15:38:36.329480+0000","last_peered":"2024-09-16T15:38:36.329480+0000","last_clean":"2024-09-16T15:38:36.329480+0000","last_became_active":"2024-09-16T15:38:34.961004+0000","last_became_peered":"2024-09-16T15:38:34.961004+0000","last_unstale":"2024-09-16T15:38:36.329480+0000","last_undegraded":"2024-09-16T15:38:36.329480+0000","last_fullsized":"2024-09-16T15:38:36.329480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:39.927190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329349+0000","last_change":"2024-09-16T15:38:33.942028+0000","last_active":"2024-09-16T15:38:36.329349+0000","last_peered":"2024-09-16T15:38:36.329349+0000","last_clean":"2024-09-16T15:38:36.329349+0000","last_became_active":"2024-09-16T15:38:33.940695+0000","last_became_peered":"2024-09-16T15:38:33.940695+0000","last_unstale":"2024-09-16T15:38:36.329349+0000","last_undegraded":"2024-09-16T15:38:36.329349+0000","last_fullsized":"2024-09-16T15:38:36.329349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330396+0000","last_change":"2024-09-16T15:38:34.937478+0000","last_active":"2024-09-16T15:38:36.330396+0000","last_peered":"2024-09-16T15:38:36.330396+0000","last_clean":"2024-09-16T15:38:36.330396+0000","last_became_active":"2024-09-16T15:38:34.936844+0000","last_became_peered":"2024-09-16T15:38:34.936844+0000","last_unstale":"2024-09-16T15:38:36.330396+0000","last_undegraded":"2024-09-16T15:38:36.330396+0000","last_fullsized":"2024-09-16T15:38:36.330396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:42.401407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328973+0000","last_change":"2024-09-16T15:38:33.942385+0000","last_active":"2024-09-16T15:38:36.328973+0000","last_peered":"2024-09-16T15:38:36.328973+0000","last_clean":"2024-09-16T15:38:36.328973+0000","last_became_active":"2024-09-16T15:38:33.941960+0000","last_became_peered":"2024-09-16T15:38:33.941960+0000","last_unstale":"2024-09-16T15:38:36.328973+0000","last_undegraded":"2024-09-16T15:38:36.328973+0000","last_fullsized":"2024-09-16T15:38:36.328973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:08.339886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935156+0000","last_change":"2024-09-16T15:38:34.955492+0000","last_active":"2024-09-16T15:38:35.935156+0000","last_peered":"2024-09-16T15:38:35.935156+0000","last_clean":"2024-09-16T15:38:35.935156+0000","last_became_active":"2024-09-16T15:38:34.955230+0000","last_became_peered":"2024-09-16T15:38:34.955230+0000","last_unstale":"2024-09-16T15:38:35.935156+0000","last_undegraded":"2024-09-16T15:38:35.935156+0000","last_fullsized":"2024-09-16T15:38:35.935156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:13.656183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934327+0000","last_change":"2024-09-16T15:38:33.939711+0000","last_active":"2024-09-16T15:38:35.934327+0000","last_peered":"2024-09-16T15:38:35.934327+0000","last_clean":"2024-09-16T15:38:35.934327+0000","last_became_active":"2024-09-16T15:38:33.937858+0000","last_became_peered":"2024-09-16T15:38:33.937858+0000","last_unstale":"2024-09-16T15:38:35.934327+0000","last_undegraded":"2024-09-16T15:38:35.934327+0000","last_fullsized":"2024-09-16T15:38:35.934327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:42.581730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328596+0000","last_change":"2024-09-16T15:38:34.947613+0000","last_active":"2024-09-16T15:38:36.328596+0000","last_peered":"2024-09-16T15:38:36.328596+0000","last_clean":"2024-09-16T15:38:36.328596+0000","last_became_active":"2024-09-16T15:38:34.947258+0000","last_became_peered":"2024-09-16T15:38:34.947258+0000","last_unstale":"2024-09-16T15:38:36.328596+0000","last_undegraded":"2024-09-16T15:38:36.328596+0000","last_fullsized":"2024-09-16T15:38:36.328596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.135119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330167+0000","last_change":"2024-09-16T15:38:33.936316+0000","last_active":"2024-09-16T15:38:36.330167+0000","last_peered":"2024-09-16T15:38:36.330167+0000","last_clean":"2024-09-16T15:38:36.330167+0000","last_became_active":"2024-09-16T15:38:33.936002+0000","last_became_peered":"2024-09-16T15:38:33.936002+0000","last_unstale":"2024-09-16T15:38:36.330167+0000","last_undegraded":"2024-09-16T15:38:36.330167+0000","last_fullsized":"2024-09-16T15:38:36.330167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.796157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329907+0000","last_change":"2024-09-16T15:38:34.942140+0000","last_active":"2024-09-16T15:38:36.329907+0000","last_peered":"2024-09-16T15:38:36.329907+0000","last_clean":"2024-09-16T15:38:36.329907+0000","last_became_active":"2024-09-16T15:38:34.940716+0000","last_became_peered":"2024-09-16T15:38:34.940716+0000","last_unstale":"2024-09-16T15:38:36.329907+0000","last_undegraded":"2024-09-16T15:38:36.329907+0000","last_fullsized":"2024-09-16T15:38:36.329907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:55.566537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328440+0000","last_change":"2024-09-16T15:38:33.938693+0000","last_active":"2024-09-16T15:38:36.328440+0000","last_peered":"2024-09-16T15:38:36.328440+0000","last_clean":"2024-09-16T15:38:36.328440+0000","last_became_active":"2024-09-16T15:38:33.938207+0000","last_became_peered":"2024-09-16T15:38:33.938207+0000","last_unstale":"2024-09-16T15:38:36.328440+0000","last_undegraded":"2024-09-16T15:38:36.328440+0000","last_fullsized":"2024-09-16T15:38:36.328440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328857+0000","last_change":"2024-09-16T15:38:34.956559+0000","last_active":"2024-09-16T15:38:36.328857+0000","last_peered":"2024-09-16T15:38:36.328857+0000","last_clean":"2024-09-16T15:38:36.328857+0000","last_became_active":"2024-09-16T15:38:34.956290+0000","last_became_peered":"2024-09-16T15:38:34.956290+0000","last_unstale":"2024-09-16T15:38:36.328857+0000","last_undegraded":"2024-09-16T15:38:36.328857+0000","last_fullsized":"2024-09-16T15:38:36.328857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:34.525651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933781+0000","last_change":"2024-09-16T15:38:33.937022+0000","last_active":"2024-09-16T15:38:35.933781+0000","last_peered":"2024-09-16T15:38:35.933781+0000","last_clean":"2024-09-16T15:38:35.933781+0000","last_became_active":"2024-09-16T15:38:33.936692+0000","last_became_peered":"2024-09-16T15:38:33.936692+0000","last_unstale":"2024-09-16T15:38:35.933781+0000","last_undegraded":"2024-09-16T15:38:35.933781+0000","last_fullsized":"2024-09-16T15:38:35.933781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:23.195213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933838+0000","last_change":"2024-09-16T15:38:34.933429+0000","last_active":"2024-09-16T15:38:35.933838+0000","last_peered":"2024-09-16T15:38:35.933838+0000","last_clean":"2024-09-16T15:38:35.933838+0000","last_became_active":"2024-09-16T15:38:34.933116+0000","last_became_peered":"2024-09-16T15:38:34.933116+0000","last_unstale":"2024-09-16T15:38:35.933838+0000","last_undegraded":"2024-09-16T15:38:35.933838+0000","last_fullsized":"2024-09-16T15:38:35.933838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:16.415478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932158+0000","last_change":"2024-09-16T15:38:33.929097+0000","last_active":"2024-09-16T15:38:35.932158+0000","last_peered":"2024-09-16T15:38:35.932158+0000","last_clean":"2024-09-16T15:38:35.932158+0000","last_became_active":"2024-09-16T15:38:33.928783+0000","last_became_peered":"2024-09-16T15:38:33.928783+0000","last_unstale":"2024-09-16T15:38:35.932158+0000","last_undegraded":"2024-09-16T15:38:35.932158+0000","last_fullsized":"2024-09-16T15:38:35.932158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327843+0000","last_change":"2024-09-16T15:38:34.955962+0000","last_active":"2024-09-16T15:38:36.327843+0000","last_peered":"2024-09-16T15:38:36.327843+0000","last_clean":"2024-09-16T15:38:36.327843+0000","last_became_active":"2024-09-16T15:38:34.955617+0000","last_became_peered":"2024-09-16T15:38:34.955617+0000","last_unstale":"2024-09-16T15:38:36.327843+0000","last_undegraded":"2024-09-16T15:38:36.327843+0000","last_fullsized":"2024-09-16T15:38:36.327843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:35.342731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935301+0000","last_change":"2024-09-16T15:38:33.933311+0000","last_active":"2024-09-16T15:38:35.935301+0000","last_peered":"2024-09-16T15:38:35.935301+0000","last_clean":"2024-09-16T15:38:35.935301+0000","last_became_active":"2024-09-16T15:38:33.932882+0000","last_became_peered":"2024-09-16T15:38:33.932882+0000","last_unstale":"2024-09-16T15:38:35.935301+0000","last_undegraded":"2024-09-16T15:38:35.935301+0000","last_fullsized":"2024-09-16T15:38:35.935301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:43.856688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328955+0000","last_change":"2024-09-16T15:38:34.959213+0000","last_active":"2024-09-16T15:38:36.328955+0000","last_peered":"2024-09-16T15:38:36.328955+0000","last_clean":"2024-09-16T15:38:36.328955+0000","last_became_active":"2024-09-16T15:38:34.958928+0000","last_became_peered":"2024-09-16T15:38:34.958928+0000","last_unstale":"2024-09-16T15:38:36.328955+0000","last_undegraded":"2024-09-16T15:38:36.328955+0000","last_fullsized":"2024-09-16T15:38:36.328955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:51.093521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932531+0000","last_change":"2024-09-16T15:38:33.939655+0000","last_active":"2024-09-16T15:38:35.932531+0000","last_peered":"2024-09-16T15:38:35.932531+0000","last_clean":"2024-09-16T15:38:35.932531+0000","last_became_active":"2024-09-16T15:38:33.939329+0000","last_became_peered":"2024-09-16T15:38:33.939329+0000","last_unstale":"2024-09-16T15:38:35.932531+0000","last_undegraded":"2024-09-16T15:38:35.932531+0000","last_fullsized":"2024-09-16T15:38:35.932531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:30.610326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329218+0000","last_change":"2024-09-16T15:38:34.956364+0000","last_active":"2024-09-16T15:38:36.329218+0000","last_peered":"2024-09-16T15:38:36.329218+0000","last_clean":"2024-09-16T15:38:36.329218+0000","last_became_active":"2024-09-16T15:38:34.956041+0000","last_became_peered":"2024-09-16T15:38:34.956041+0000","last_unstale":"2024-09-16T15:38:36.329218+0000","last_undegraded":"2024-09-16T15:38:36.329218+0000","last_fullsized":"2024-09-16T15:38:36.329218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:26.092978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329039+0000","last_change":"2024-09-16T15:38:33.940469+0000","last_active":"2024-09-16T15:38:36.329039+0000","last_peered":"2024-09-16T15:38:36.329039+0000","last_clean":"2024-09-16T15:38:36.329039+0000","last_became_active":"2024-09-16T15:38:33.939752+0000","last_became_peered":"2024-09-16T15:38:33.939752+0000","last_unstale":"2024-09-16T15:38:36.329039+0000","last_undegraded":"2024-09-16T15:38:36.329039+0000","last_fullsized":"2024-09-16T15:38:36.329039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:31.202600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328853+0000","last_change":"2024-09-16T15:38:34.946270+0000","last_active":"2024-09-16T15:38:36.328853+0000","last_peered":"2024-09-16T15:38:36.328853+0000","last_clean":"2024-09-16T15:38:36.328853+0000","last_became_active":"2024-09-16T15:38:34.945965+0000","last_became_peered":"2024-09-16T15:38:34.945965+0000","last_unstale":"2024-09-16T15:38:36.328853+0000","last_undegraded":"2024-09-16T15:38:36.328853+0000","last_fullsized":"2024-09-16T15:38:36.328853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:36.868939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932369+0000","last_change":"2024-09-16T15:38:33.937925+0000","last_active":"2024-09-16T15:38:35.932369+0000","last_peered":"2024-09-16T15:38:35.932369+0000","last_clean":"2024-09-16T15:38:35.932369+0000","last_became_active":"2024-09-16T15:38:33.937648+0000","last_became_peered":"2024-09-16T15:38:33.937648+0000","last_unstale":"2024-09-16T15:38:35.932369+0000","last_undegraded":"2024-09-16T15:38:35.932369+0000","last_fullsized":"2024-09-16T15:38:35.932369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:53.914692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328407+0000","last_change":"2024-09-16T15:38:34.964388+0000","last_active":"2024-09-16T15:38:36.328407+0000","last_peered":"2024-09-16T15:38:36.328407+0000","last_clean":"2024-09-16T15:38:36.328407+0000","last_became_active":"2024-09-16T15:38:34.964136+0000","last_became_peered":"2024-09-16T15:38:34.964136+0000","last_unstale":"2024-09-16T15:38:36.328407+0000","last_undegraded":"2024-09-16T15:38:36.328407+0000","last_fullsized":"2024-09-16T15:38:36.328407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:21.999129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327747+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:38:36.327747+0000","last_peered":"2024-09-16T15:38:36.327747+0000","last_clean":"2024-09-16T15:38:36.327747+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:38:36.327747+0000","last_undegraded":"2024-09-16T15:38:36.327747+0000","last_fullsized":"2024-09-16T15:38:36.327747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327930+0000","last_change":"2024-09-16T15:38:33.941937+0000","last_active":"2024-09-16T15:38:36.327930+0000","last_peered":"2024-09-16T15:38:36.327930+0000","last_clean":"2024-09-16T15:38:36.327930+0000","last_became_active":"2024-09-16T15:38:33.941212+0000","last_became_peered":"2024-09-16T15:38:33.941212+0000","last_unstale":"2024-09-16T15:38:36.327930+0000","last_undegraded":"2024-09-16T15:38:36.327930+0000","last_fullsized":"2024-09-16T15:38:36.327930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328444+0000","last_change":"2024-09-16T15:38:34.962489+0000","last_active":"2024-09-16T15:38:36.328444+0000","last_peered":"2024-09-16T15:38:36.328444+0000","last_clean":"2024-09-16T15:38:36.328444+0000","last_became_active":"2024-09-16T15:38:34.962245+0000","last_became_peered":"2024-09-16T15:38:34.962245+0000","last_unstale":"2024-09-16T15:38:36.328444+0000","last_undegraded":"2024-09-16T15:38:36.328444+0000","last_fullsized":"2024-09-16T15:38:36.328444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:25.942896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.955896+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.955595+0000","last_became_peered":"2024-09-16T15:38:33.955595+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329773+0000","last_change":"2024-09-16T15:38:34.956268+0000","last_active":"2024-09-16T15:38:36.329773+0000","last_peered":"2024-09-16T15:38:36.329773+0000","last_clean":"2024-09-16T15:38:36.329773+0000","last_became_active":"2024-09-16T15:38:34.956010+0000","last_became_peered":"2024-09-16T15:38:34.956010+0000","last_unstale":"2024-09-16T15:38:36.329773+0000","last_undegraded":"2024-09-16T15:38:36.329773+0000","last_fullsized":"2024-09-16T15:38:36.329773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:02.100232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.932349+0000","last_change":"2024-09-16T15:38:33.941933+0000","last_active":"2024-09-16T15:38:35.932349+0000","last_peered":"2024-09-16T15:38:35.932349+0000","last_clean":"2024-09-16T15:38:35.932349+0000","last_became_active":"2024-09-16T15:38:33.941648+0000","last_became_peered":"2024-09-16T15:38:33.941648+0000","last_unstale":"2024-09-16T15:38:35.932349+0000","last_undegraded":"2024-09-16T15:38:35.932349+0000","last_fullsized":"2024-09-16T15:38:35.932349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:30.652915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328622+0000","last_change":"2024-09-16T15:38:34.932401+0000","last_active":"2024-09-16T15:38:36.328622+0000","last_peered":"2024-09-16T15:38:36.328622+0000","last_clean":"2024-09-16T15:38:36.328622+0000","last_became_active":"2024-09-16T15:38:34.931203+0000","last_became_peered":"2024-09-16T15:38:34.931203+0000","last_unstale":"2024-09-16T15:38:36.328622+0000","last_undegraded":"2024-09-16T15:38:36.328622+0000","last_fullsized":"2024-09-16T15:38:36.328622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:24.483689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328535+0000","last_change":"2024-09-16T15:38:33.947515+0000","last_active":"2024-09-16T15:38:36.328535+0000","last_peered":"2024-09-16T15:38:36.328535+0000","last_clean":"2024-09-16T15:38:36.328535+0000","last_became_active":"2024-09-16T15:38:33.946118+0000","last_became_peered":"2024-09-16T15:38:33.946118+0000","last_unstale":"2024-09-16T15:38:36.328535+0000","last_undegraded":"2024-09-16T15:38:36.328535+0000","last_fullsized":"2024-09-16T15:38:36.328535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:34.345209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935354+0000","last_change":"2024-09-16T15:38:34.951051+0000","last_active":"2024-09-16T15:38:35.935354+0000","last_peered":"2024-09-16T15:38:35.935354+0000","last_clean":"2024-09-16T15:38:35.935354+0000","last_became_active":"2024-09-16T15:38:34.950115+0000","last_became_peered":"2024-09-16T15:38:34.950115+0000","last_unstale":"2024-09-16T15:38:35.935354+0000","last_undegraded":"2024-09-16T15:38:35.935354+0000","last_fullsized":"2024-09-16T15:38:35.935354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:29.584103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328940+0000","last_change":"2024-09-16T15:38:33.936405+0000","last_active":"2024-09-16T15:38:36.328940+0000","last_peered":"2024-09-16T15:38:36.328940+0000","last_clean":"2024-09-16T15:38:36.328940+0000","last_became_active":"2024-09-16T15:38:33.935318+0000","last_became_peered":"2024-09-16T15:38:33.935318+0000","last_unstale":"2024-09-16T15:38:36.328940+0000","last_undegraded":"2024-09-16T15:38:36.328940+0000","last_fullsized":"2024-09-16T15:38:36.328940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:27.366083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329408+0000","last_change":"2024-09-16T15:38:34.961561+0000","last_active":"2024-09-16T15:38:36.329408+0000","last_peered":"2024-09-16T15:38:36.329408+0000","last_clean":"2024-09-16T15:38:36.329408+0000","last_became_active":"2024-09-16T15:38:34.961174+0000","last_became_peered":"2024-09-16T15:38:34.961174+0000","last_unstale":"2024-09-16T15:38:36.329408+0000","last_undegraded":"2024-09-16T15:38:36.329408+0000","last_fullsized":"2024-09-16T15:38:36.329408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:52.168399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328656+0000","last_change":"2024-09-16T15:38:33.955332+0000","last_active":"2024-09-16T15:38:36.328656+0000","last_peered":"2024-09-16T15:38:36.328656+0000","last_clean":"2024-09-16T15:38:36.328656+0000","last_became_active":"2024-09-16T15:38:33.954340+0000","last_became_peered":"2024-09-16T15:38:33.954340+0000","last_unstale":"2024-09-16T15:38:36.328656+0000","last_undegraded":"2024-09-16T15:38:36.328656+0000","last_fullsized":"2024-09-16T15:38:36.328656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:45.798957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328724+0000","last_change":"2024-09-16T15:38:34.928319+0000","last_active":"2024-09-16T15:38:36.328724+0000","last_peered":"2024-09-16T15:38:36.328724+0000","last_clean":"2024-09-16T15:38:36.328724+0000","last_became_active":"2024-09-16T15:38:34.928058+0000","last_became_peered":"2024-09-16T15:38:34.928058+0000","last_unstale":"2024-09-16T15:38:36.328724+0000","last_undegraded":"2024-09-16T15:38:36.328724+0000","last_fullsized":"2024-09-16T15:38:36.328724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:00.238312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.929048+0000","last_change":"2024-09-16T15:38:33.931369+0000","last_active":"2024-09-16T15:38:36.929048+0000","last_peered":"2024-09-16T15:38:36.929048+0000","last_clean":"2024-09-16T15:38:36.929048+0000","last_became_active":"2024-09-16T15:38:33.928339+0000","last_became_peered":"2024-09-16T15:38:33.928339+0000","last_unstale":"2024-09-16T15:38:36.929048+0000","last_undegraded":"2024-09-16T15:38:36.929048+0000","last_fullsized":"2024-09-16T15:38:36.929048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:44.073989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329649+0000","last_change":"2024-09-16T15:38:34.942463+0000","last_active":"2024-09-16T15:38:36.329649+0000","last_peered":"2024-09-16T15:38:36.329649+0000","last_clean":"2024-09-16T15:38:36.329649+0000","last_became_active":"2024-09-16T15:38:34.940343+0000","last_became_peered":"2024-09-16T15:38:34.940343+0000","last_unstale":"2024-09-16T15:38:36.329649+0000","last_undegraded":"2024-09-16T15:38:36.329649+0000","last_fullsized":"2024-09-16T15:38:36.329649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.897937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.935853+0000","last_change":"2024-09-16T15:38:33.947537+0000","last_active":"2024-09-16T15:38:35.935853+0000","last_peered":"2024-09-16T15:38:35.935853+0000","last_clean":"2024-09-16T15:38:35.935853+0000","last_became_active":"2024-09-16T15:38:33.946909+0000","last_became_peered":"2024-09-16T15:38:33.946909+0000","last_unstale":"2024-09-16T15:38:35.935853+0000","last_undegraded":"2024-09-16T15:38:35.935853+0000","last_fullsized":"2024-09-16T15:38:35.935853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:03.179729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330207+0000","last_change":"2024-09-16T15:38:34.941844+0000","last_active":"2024-09-16T15:38:36.330207+0000","last_peered":"2024-09-16T15:38:36.330207+0000","last_clean":"2024-09-16T15:38:36.330207+0000","last_became_active":"2024-09-16T15:38:34.940573+0000","last_became_peered":"2024-09-16T15:38:34.940573+0000","last_unstale":"2024-09-16T15:38:36.330207+0000","last_undegraded":"2024-09-16T15:38:36.330207+0000","last_fullsized":"2024-09-16T15:38:36.330207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:17.824920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.327596+0000","last_change":"2024-09-16T15:38:33.954548+0000","last_active":"2024-09-16T15:38:36.327596+0000","last_peered":"2024-09-16T15:38:36.327596+0000","last_clean":"2024-09-16T15:38:36.327596+0000","last_became_active":"2024-09-16T15:38:33.954266+0000","last_became_peered":"2024-09-16T15:38:33.954266+0000","last_unstale":"2024-09-16T15:38:36.327596+0000","last_undegraded":"2024-09-16T15:38:36.327596+0000","last_fullsized":"2024-09-16T15:38:36.327596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:14.000950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.934862+0000","last_change":"2024-09-16T15:38:34.951875+0000","last_active":"2024-09-16T15:38:35.934862+0000","last_peered":"2024-09-16T15:38:35.934862+0000","last_clean":"2024-09-16T15:38:35.934862+0000","last_became_active":"2024-09-16T15:38:34.946733+0000","last_became_peered":"2024-09-16T15:38:34.946733+0000","last_unstale":"2024-09-16T15:38:35.934862+0000","last_undegraded":"2024-09-16T15:38:35.934862+0000","last_fullsized":"2024-09-16T15:38:35.934862+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:35.351445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.947052+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.946593+0000","last_became_peered":"2024-09-16T15:38:33.946593+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:37.618001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328906+0000","last_change":"2024-09-16T15:38:34.933312+0000","last_active":"2024-09-16T15:38:36.328906+0000","last_peered":"2024-09-16T15:38:36.328906+0000","last_clean":"2024-09-16T15:38:36.328906+0000","last_became_active":"2024-09-16T15:38:34.933037+0000","last_became_peered":"2024-09-16T15:38:34.933037+0000","last_unstale":"2024-09-16T15:38:36.328906+0000","last_undegraded":"2024-09-16T15:38:36.328906+0000","last_fullsized":"2024-09-16T15:38:36.328906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:15.092528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328790+0000","last_change":"2024-09-16T15:38:33.943983+0000","last_active":"2024-09-16T15:38:36.328790+0000","last_peered":"2024-09-16T15:38:36.328790+0000","last_clean":"2024-09-16T15:38:36.328790+0000","last_became_active":"2024-09-16T15:38:33.943683+0000","last_became_peered":"2024-09-16T15:38:33.943683+0000","last_unstale":"2024-09-16T15:38:36.328790+0000","last_undegraded":"2024-09-16T15:38:36.328790+0000","last_fullsized":"2024-09-16T15:38:36.328790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.572327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328590+0000","last_change":"2024-09-16T15:38:34.943957+0000","last_active":"2024-09-16T15:38:36.328590+0000","last_peered":"2024-09-16T15:38:36.328590+0000","last_clean":"2024-09-16T15:38:36.328590+0000","last_became_active":"2024-09-16T15:38:34.941591+0000","last_became_peered":"2024-09-16T15:38:34.941591+0000","last_unstale":"2024-09-16T15:38:36.328590+0000","last_undegraded":"2024-09-16T15:38:36.328590+0000","last_fullsized":"2024-09-16T15:38:36.328590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:44.851503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328378+0000","last_change":"2024-09-16T15:38:33.954584+0000","last_active":"2024-09-16T15:38:36.328378+0000","last_peered":"2024-09-16T15:38:36.328378+0000","last_clean":"2024-09-16T15:38:36.328378+0000","last_became_active":"2024-09-16T15:38:33.954279+0000","last_became_peered":"2024-09-16T15:38:33.954279+0000","last_unstale":"2024-09-16T15:38:36.328378+0000","last_undegraded":"2024-09-16T15:38:36.328378+0000","last_fullsized":"2024-09-16T15:38:36.328378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.350246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:35.933817+0000","last_change":"2024-09-16T15:38:34.949723+0000","last_active":"2024-09-16T15:38:35.933817+0000","last_peered":"2024-09-16T15:38:35.933817+0000","last_clean":"2024-09-16T15:38:35.933817+0000","last_became_active":"2024-09-16T15:38:34.949433+0000","last_became_peered":"2024-09-16T15:38:34.949433+0000","last_unstale":"2024-09-16T15:38:35.933817+0000","last_undegraded":"2024-09-16T15:38:35.933817+0000","last_fullsized":"2024-09-16T15:38:35.933817+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:33.866690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.329345+0000","last_change":"2024-09-16T15:38:34.961924+0000","last_active":"2024-09-16T15:38:36.329345+0000","last_peered":"2024-09-16T15:38:36.329345+0000","last_clean":"2024-09-16T15:38:36.329345+0000","last_became_active":"2024-09-16T15:38:34.961607+0000","last_became_peered":"2024-09-16T15:38:34.961607+0000","last_unstale":"2024-09-16T15:38:36.329345+0000","last_undegraded":"2024-09-16T15:38:36.329345+0000","last_fullsized":"2024-09-16T15:38:36.329345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:58.529110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328347+0000","last_change":"2024-09-16T15:38:33.955384+0000","last_active":"2024-09-16T15:38:36.328347+0000","last_peered":"2024-09-16T15:38:36.328347+0000","last_clean":"2024-09-16T15:38:36.328347+0000","last_became_active":"2024-09-16T15:38:33.954532+0000","last_became_peered":"2024-09-16T15:38:33.954532+0000","last_unstale":"2024-09-16T15:38:36.328347+0000","last_undegraded":"2024-09-16T15:38:36.328347+0000","last_fullsized":"2024-09-16T15:38:36.328347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:28.577867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328852+0000","last_change":"2024-09-16T15:38:34.958471+0000","last_active":"2024-09-16T15:38:36.328852+0000","last_peered":"2024-09-16T15:38:36.328852+0000","last_clean":"2024-09-16T15:38:36.328852+0000","last_became_active":"2024-09-16T15:38:34.958217+0000","last_became_peered":"2024-09-16T15:38:34.958217+0000","last_unstale":"2024-09-16T15:38:36.328852+0000","last_undegraded":"2024-09-16T15:38:36.328852+0000","last_fullsized":"2024-09-16T15:38:36.328852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:27.839440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328469+0000","last_change":"2024-09-16T15:38:33.956064+0000","last_active":"2024-09-16T15:38:36.328469+0000","last_peered":"2024-09-16T15:38:36.328469+0000","last_clean":"2024-09-16T15:38:36.328469+0000","last_became_active":"2024-09-16T15:38:33.955717+0000","last_became_peered":"2024-09-16T15:38:33.955717+0000","last_unstale":"2024-09-16T15:38:36.328469+0000","last_undegraded":"2024-09-16T15:38:36.328469+0000","last_fullsized":"2024-09-16T15:38:36.328469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:12.649947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330233+0000","last_change":"2024-09-16T15:38:34.937110+0000","last_active":"2024-09-16T15:38:36.330233+0000","last_peered":"2024-09-16T15:38:36.330233+0000","last_clean":"2024-09-16T15:38:36.330233+0000","last_became_active":"2024-09-16T15:38:34.936439+0000","last_became_peered":"2024-09-16T15:38:34.936439+0000","last_unstale":"2024-09-16T15:38:36.330233+0000","last_undegraded":"2024-09-16T15:38:36.330233+0000","last_fullsized":"2024-09-16T15:38:36.330233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:49.927493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328261+0000","last_change":"2024-09-16T15:38:33.944467+0000","last_active":"2024-09-16T15:38:36.328261+0000","last_peered":"2024-09-16T15:38:36.328261+0000","last_clean":"2024-09-16T15:38:36.328261+0000","last_became_active":"2024-09-16T15:38:33.944211+0000","last_became_peered":"2024-09-16T15:38:33.944211+0000","last_unstale":"2024-09-16T15:38:36.328261+0000","last_undegraded":"2024-09-16T15:38:36.328261+0000","last_fullsized":"2024-09-16T15:38:36.328261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:30.042385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.328305+0000","last_change":"2024-09-16T15:38:34.932456+0000","last_active":"2024-09-16T15:38:36.328305+0000","last_peered":"2024-09-16T15:38:36.328305+0000","last_clean":"2024-09-16T15:38:36.328305+0000","last_became_active":"2024-09-16T15:38:34.931374+0000","last_became_peered":"2024-09-16T15:38:34.931374+0000","last_unstale":"2024-09-16T15:38:36.328305+0000","last_undegraded":"2024-09-16T15:38:36.328305+0000","last_fullsized":"2024-09-16T15:38:36.328305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:33.906366+0000","last_clean_scrub_stamp":"2024-09-16T15:38:33.906366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:26.787621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T15:38:36.330346+0000","last_change":"2024-09-16T15:38:33.945170+0000","last_active":"2024-09-16T15:38:36.330346+0000","last_peered":"2024-09-16T15:38:36.330346+0000","last_clean":"2024-09-16T15:38:36.330346+0000","last_became_active":"2024-09-16T15:38:33.944681+0000","last_became_peered":"2024-09-16T15:38:33.944681+0000","last_unstale":"2024-09-16T15:38:36.330346+0000","last_undegraded":"2024-09-16T15:38:36.330346+0000","last_fullsized":"2024-09-16T15:38:36.330346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:32.900130+0000","last_clean_scrub_stamp":"2024-09-16T15:38:32.900130+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:30.232249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":245760,"data_stored":81953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":245760,"data_stored":83439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":672833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":241664,"data_stored":81987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:38:49.880 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:38:49.881 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T15:38:49.881 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T15:38:49.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T15:38:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.077+0000 7f8915800640 1 -- 172.21.15.60:0/548451195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891006e6f0 msgr2=0x7f891006eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.077+0000 7f8915800640 1 --2- 172.21.15.60:0/548451195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891006e6f0 0x7f891006eba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8908005fd0 tx=0x7f890802a670 comp rx=0 tx=0).stop 2024-09-16T15:38:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.077+0000 7f8915800640 1 -- 172.21.15.60:0/548451195 shutdown_connections 2024-09-16T15:38:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.077+0000 7f8915800640 1 --2- 172.21.15.60:0/548451195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891006e6f0 0x7f891006eba0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.077+0000 7f8915800640 1 --2- 172.21.15.60:0/548451195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89100733e0 0x7f89100cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.077+0000 7f8915800640 1 -- 172.21.15.60:0/548451195 >> 172.21.15.60:0/548451195 conn(0x7f89100b18e0 msgr2=0x7f89100b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.077+0000 7f8915800640 1 -- 172.21.15.60:0/548451195 shutdown_connections 2024-09-16T15:38:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.077+0000 7f8915800640 1 -- 172.21.15.60:0/548451195 wait complete. 2024-09-16T15:38:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.079+0000 7f8915800640 1 Processor -- start 2024-09-16T15:38:50.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.079+0000 7f8915800640 1 -- start start 2024-09-16T15:38:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f8915800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891006e6f0 0x7f89100b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f8915800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891015f910 0x7f89100aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f8915800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89100cb420 0x7f89100b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f8915800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f891006f0e0 con 0x7f891015f910 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f8915800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f89100cd5a0 con 0x7f89100cb420 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f8915800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f891006f260 con 0x7f891006e6f0 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f890e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891015f910 0x7f89100aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f890e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891015f910 0x7f89100aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43376/0 (socket says 172.21.15.60:43376) 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f890e000640 1 -- 172.21.15.60:0/848289019 learned_addr learned my addr 172.21.15.60:0/848289019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f890ea00640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891006e6f0 0x7f89100b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.080+0000 7f890f400640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89100cb420 0x7f89100b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f890e000640 1 -- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891006e6f0 msgr2=0x7f89100b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f890e000640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891006e6f0 0x7f89100b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f890e000640 1 -- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89100cb420 msgr2=0x7f89100b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f890e000640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89100cb420 0x7f89100b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f890e000640 1 -- 172.21.15.60:0/848289019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89100aefe0 con 0x7f891015f910 2024-09-16T15:38:50.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f890e000640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891015f910 0x7f89100aeaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00d7a0 tx=0x7f88fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:50.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f8906a00640 1 -- 172.21.15.60:0/848289019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc017070 con 0x7f891015f910 2024-09-16T15:38:50.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f8906a00640 1 -- 172.21.15.60:0/848289019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88fc009cc0 con 0x7f891015f910 2024-09-16T15:38:50.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f8906a00640 1 -- 172.21.15.60:0/848289019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc005280 con 0x7f891015f910 2024-09-16T15:38:50.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89100af280 con 0x7f891015f910 2024-09-16T15:38:50.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.081+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891014a190 con 0x7f891015f910 2024-09-16T15:38:50.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.082+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8910069360 con 0x7f891015f910 2024-09-16T15:38:50.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.084+0000 7f8906a00640 1 -- 172.21.15.60:0/848289019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f88fc005420 con 0x7f891015f910 2024-09-16T15:38:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.085+0000 7f8906a00640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88f0078850 0x7f88f007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.085+0000 7f8906a00640 1 -- 172.21.15.60:0/848289019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f88fc09aa30 con 0x7f891015f910 2024-09-16T15:38:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.086+0000 7f890ea00640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88f0078850 0x7f88f007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.086+0000 7f890ea00640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88f0078850 0x7f88f007ad10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8900001e20 tx=0x7f8900009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:50.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.089+0000 7f8906a00640 1 -- 172.21.15.60:0/848289019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f88fc0629b0 con 0x7f891015f910 2024-09-16T15:38:50.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.324+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f891015c020 con 0x7f891015f910 2024-09-16T15:38:50.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.325+0000 7f8906a00640 1 -- 172.21.15.60:0/848289019 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f891015c020 con 0x7f891015f910 2024-09-16T15:38:50.326 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:50.326 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T15:38:50.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88f0078850 msgr2=0x7f88f007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88f0078850 0x7f88f007ad10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8900001e20 tx=0x7f8900009040 comp rx=0 tx=0).stop 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891015f910 msgr2=0x7f89100aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891015f910 0x7f89100aeaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00d7a0 tx=0x7f88fc00dc60 comp rx=0 tx=0).stop 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 shutdown_connections 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88f0078850 0x7f88f007ad10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89100cb420 0x7f89100b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f891015f910 0x7f89100aeaa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 --2- 172.21.15.60:0/848289019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f891006e6f0 0x7f89100b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 >> 172.21.15.60:0/848289019 conn(0x7f89100b18e0 msgr2=0x7f891015a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 shutdown_connections 2024-09-16T15:38:50.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.327+0000 7f8915800640 1 -- 172.21.15.60:0/848289019 wait complete. 2024-09-16T15:38:50.343 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T15:38:50.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:38:50.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.502+0000 7f193de00640 1 -- 172.21.15.60:0/3200907084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f191c004800 msgr2=0x7f191c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.502+0000 7f193de00640 1 --2- 172.21.15.60:0/3200907084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f191c004800 0x7f191c004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1920009b20 tx=0x7f191c005830 comp rx=0 tx=0).stop 2024-09-16T15:38:50.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.502+0000 7f1937400640 1 -- 172.21.15.60:0/3200907084 reap_dead start 2024-09-16T15:38:50.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.502+0000 7f193de00640 1 -- 172.21.15.60:0/3200907084 shutdown_connections 2024-09-16T15:38:50.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.502+0000 7f193de00640 1 -- 172.21.15.60:0/3200907084 >> 172.21.15.60:0/3200907084 conn(0x7f19380b1950 msgr2=0x7f19380b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:50.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.502+0000 7f193de00640 1 -- 172.21.15.60:0/3200907084 shutdown_connections 2024-09-16T15:38:50.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.502+0000 7f193de00640 1 -- 172.21.15.60:0/3200907084 wait complete. 2024-09-16T15:38:50.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.504+0000 7f193de00640 1 Processor -- start 2024-09-16T15:38:50.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.504+0000 7f193de00640 1 -- start start 2024-09-16T15:38:50.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f193de00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19381f2010 0x7f19381f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f193de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19381f23f0 0x7f1938154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f193de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19381568b0 0x7f1938156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f193de00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f193806f9f0 con 0x7f19381568b0 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f193de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f193806f870 con 0x7f19381f23f0 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f193de00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f193806fb70 con 0x7f19381f2010 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f1937400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19381568b0 0x7f1938156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f1937400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19381568b0 0x7f1938156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43392/0 (socket says 172.21.15.60:43392) 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f1937400640 1 -- 172.21.15.60:0/2599584246 learned_addr learned my addr 172.21.15.60:0/2599584246 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f192e000640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19381f23f0 0x7f1938154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.505+0000 7f1936a00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19381f2010 0x7f19381f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f1937400640 1 -- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19381f2010 msgr2=0x7f19381f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f1937400640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19381f2010 0x7f19381f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f1937400640 1 -- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19381f23f0 msgr2=0x7f1938154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f1937400640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19381f23f0 0x7f1938154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f1937400640 1 -- 172.21.15.60:0/2599584246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19381f91b0 con 0x7f19381568b0 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f1937400640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19381568b0 0x7f1938156d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1938070d40 tx=0x7f1930001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f192fe00640 1 -- 172.21.15.60:0/2599584246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930011840 con 0x7f19381568b0 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f192fe00640 1 -- 172.21.15.60:0/2599584246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1930018070 con 0x7f19381568b0 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1938155000 con 0x7f19381568b0 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f192fe00640 1 -- 172.21.15.60:0/2599584246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930011c80 con 0x7f19381568b0 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.506+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381459c0 con 0x7f19381568b0 2024-09-16T15:38:50.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.508+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19380732a0 con 0x7f19381568b0 2024-09-16T15:38:50.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.509+0000 7f192fe00640 1 -- 172.21.15.60:0/2599584246 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f193001d460 con 0x7f19381568b0 2024-09-16T15:38:50.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.510+0000 7f192fe00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1914078880 0x7f191407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.510+0000 7f192fe00640 1 -- 172.21.15.60:0/2599584246 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f193009ae90 con 0x7f19381568b0 2024-09-16T15:38:50.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.510+0000 7f1936a00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1914078880 0x7f191407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.511+0000 7f1936a00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1914078880 0x7f191407ad40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1928000fb0 tx=0x7f19280067a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:50.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.514+0000 7f192fe00640 1 -- 172.21.15.60:0/2599584246 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1938155000 con 0x7f19381568b0 2024-09-16T15:38:50.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.748+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f193806f2b0 con 0x7f19381568b0 2024-09-16T15:38:50.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.749+0000 7f192fe00640 1 -- 172.21.15.60:0/2599584246 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f19381f91b0 con 0x7f19381568b0 2024-09-16T15:38:50.750 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:50.750 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:38:34.368784+0000","modified":"2024-09-16T15:38:37.370114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-16T15:38:50.750 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1914078880 msgr2=0x7f191407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1914078880 0x7f191407ad40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1928000fb0 tx=0x7f19280067a0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19381568b0 msgr2=0x7f1938156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19381568b0 0x7f1938156d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1938070d40 tx=0x7f1930001bb0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 shutdown_connections 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1914078880 0x7f191407ad40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19381568b0 0x7f1938156d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19381f23f0 0x7f1938154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 --2- 172.21.15.60:0/2599584246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19381f2010 0x7f19381f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 >> 172.21.15.60:0/2599584246 conn(0x7f19380b1950 msgr2=0x7f193815e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 shutdown_connections 2024-09-16T15:38:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.751+0000 7f193de00640 1 -- 172.21.15.60:0/2599584246 wait complete. 2024-09-16T15:38:50.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:38:50.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.927+0000 7f8353e00640 1 -- 172.21.15.60:0/3641313324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83540725d0 msgr2=0x7f8354073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.927+0000 7f8353e00640 1 --2- 172.21.15.60:0/3641313324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83540725d0 0x7f8354073d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f83540cd570 tx=0x7f833c02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.927+0000 7f8353e00640 1 -- 172.21.15.60:0/3641313324 shutdown_connections 2024-09-16T15:38:50.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.927+0000 7f8353e00640 1 --2- 172.21.15.60:0/3641313324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83540725d0 0x7f8354073d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.927+0000 7f8353e00640 1 --2- 172.21.15.60:0/3641313324 >> v2:172.21.15.100:3300/0 conn(0x7f835415bc30 0x7f83540b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.927+0000 7f8353e00640 1 -- 172.21.15.60:0/3641313324 >> 172.21.15.60:0/3641313324 conn(0x7f83540cfa20 msgr2=0x7f83540cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:50.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.927+0000 7f8353e00640 1 -- 172.21.15.60:0/3641313324 shutdown_connections 2024-09-16T15:38:50.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.927+0000 7f8353e00640 1 -- 172.21.15.60:0/3641313324 wait complete. 2024-09-16T15:38:50.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.929+0000 7f8353e00640 1 Processor -- start 2024-09-16T15:38:50.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.929+0000 7f8353e00640 1 -- start start 2024-09-16T15:38:50.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f8353e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83540d0540 0x7f83540b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f8353e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f835415bc30 0x7f835414ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f8353e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83540c9740 0x7f83541580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f8353e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8354074610 con 0x7f835415bc30 2024-09-16T15:38:50.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f8353e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8354074490 con 0x7f83540c9740 2024-09-16T15:38:50.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f8353e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8354074790 con 0x7f83540d0540 2024-09-16T15:38:50.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f834a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f835415bc30 0x7f835414ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f834a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f835415bc30 0x7f835414ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43414/0 (socket says 172.21.15.60:43414) 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f834a000640 1 -- 172.21.15.60:0/2372647665 learned_addr learned my addr 172.21.15.60:0/2372647665 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f8352a00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83540d0540 0x7f83540b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.930+0000 7f8353400640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83540c9740 0x7f83541580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834a000640 1 -- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83540d0540 msgr2=0x7f83540b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834a000640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83540d0540 0x7f83540b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834a000640 1 -- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83540c9740 msgr2=0x7f83541580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834a000640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83540c9740 0x7f83541580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834a000640 1 -- 172.21.15.60:0/2372647665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83541f8190 con 0x7f835415bc30 2024-09-16T15:38:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834a000640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f835415bc30 0x7f835414ff30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f833c000c00 tx=0x7f833c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:50.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834b400640 1 -- 172.21.15.60:0/2372647665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c03f070 con 0x7f835415bc30 2024-09-16T15:38:50.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834b400640 1 -- 172.21.15.60:0/2372647665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f833c0047a0 con 0x7f835415bc30 2024-09-16T15:38:50.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f834b400640 1 -- 172.21.15.60:0/2372647665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c035030 con 0x7f835415bc30 2024-09-16T15:38:50.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.931+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83541f8470 con 0x7f835415bc30 2024-09-16T15:38:50.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.932+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83540b1120 con 0x7f835415bc30 2024-09-16T15:38:50.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.933+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835406fd20 con 0x7f835415bc30 2024-09-16T15:38:50.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.934+0000 7f834b400640 1 -- 172.21.15.60:0/2372647665 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f833c042070 con 0x7f835415bc30 2024-09-16T15:38:50.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.935+0000 7f834b400640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8330078880 0x7f833007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:50.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.935+0000 7f834b400640 1 -- 172.21.15.60:0/2372647665 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f833c0bb690 con 0x7f835415bc30 2024-09-16T15:38:50.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.935+0000 7f8352a00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8330078880 0x7f833007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:50.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.936+0000 7f8352a00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8330078880 0x7f833007ad40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f834400bb50 tx=0x7f834400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:50.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:50.940+0000 7f834b400640 1 -- 172.21.15.60:0/2372647665 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f833c0836a0 con 0x7f835415bc30 2024-09-16T15:38:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.175+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8354157bb0 con 0x7f835415bc30 2024-09-16T15:38:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.175+0000 7f834b400640 1 -- 172.21.15.60:0/2372647665 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f833c0880d0 con 0x7f835415bc30 2024-09-16T15:38:51.177 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:51.177 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:38:34.368784+0000","modified":"2024-09-16T15:38:37.370114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-16T15:38:51.177 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.177+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8330078880 msgr2=0x7f833007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.177+0000 7f8353e00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8330078880 0x7f833007ad40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f834400bb50 tx=0x7f834400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f835415bc30 msgr2=0x7f835414ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f835415bc30 0x7f835414ff30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f833c000c00 tx=0x7f833c003910 comp rx=0 tx=0).stop 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 shutdown_connections 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8330078880 0x7f833007ad40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83540c9740 0x7f83541580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f835415bc30 0x7f835414ff30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 --2- 172.21.15.60:0/2372647665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83540d0540 0x7f83540b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 >> 172.21.15.60:0/2372647665 conn(0x7f83540cfa20 msgr2=0x7f83540b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:51.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 shutdown_connections 2024-09-16T15:38:51.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.178+0000 7f8353e00640 1 -- 172.21.15.60:0/2372647665 wait complete. 2024-09-16T15:38:51.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:38:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.354+0000 7faf1fe00640 1 -- 172.21.15.60:0/2474707729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf200b4020 msgr2=0x7faf200b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.354+0000 7faf1fe00640 1 --2- 172.21.15.60:0/2474707729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf200b4020 0x7faf200b1770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faf200d4ea0 tx=0x7faf1802a9f0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.354+0000 7faf1fe00640 1 -- 172.21.15.60:0/2474707729 shutdown_connections 2024-09-16T15:38:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.354+0000 7faf1fe00640 1 --2- 172.21.15.60:0/2474707729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf200b4020 0x7faf200b1770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.354+0000 7faf1fe00640 1 --2- 172.21.15.60:0/2474707729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf20161440 0x7faf200b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.354+0000 7faf1fe00640 1 -- 172.21.15.60:0/2474707729 >> 172.21.15.60:0/2474707729 conn(0x7faf200d3df0 msgr2=0x7faf200d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.354+0000 7faf1fe00640 1 -- 172.21.15.60:0/2474707729 shutdown_connections 2024-09-16T15:38:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.354+0000 7faf1fe00640 1 -- 172.21.15.60:0/2474707729 wait complete. 2024-09-16T15:38:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.355+0000 7faf1fe00640 1 Processor -- start 2024-09-16T15:38:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.355+0000 7faf1fe00640 1 -- start start 2024-09-16T15:38:51.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.356+0000 7faf1fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf200b34b0 0x7faf200cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:51.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.356+0000 7faf1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf200b4020 0x7faf201fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:51.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.356+0000 7faf1fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf20161440 0x7faf200cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:51.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.356+0000 7faf1fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faf200b2040 con 0x7faf20161440 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.356+0000 7faf1fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf200b1ec0 con 0x7faf200b4020 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.356+0000 7faf1fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7faf200b21c0 con 0x7faf200b34b0 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf20161440 0x7faf200cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf20161440 0x7faf200cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43444/0 (socket says 172.21.15.60:43444) 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 -- 172.21.15.60:0/249835371 learned_addr learned my addr 172.21.15.60:0/249835371 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1ea00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf200b34b0 0x7faf200cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1e000640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf200b4020 0x7faf201fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 -- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf200b34b0 msgr2=0x7faf200cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf200b34b0 0x7faf200cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 -- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf200b4020 msgr2=0x7faf201fbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf200b4020 0x7faf201fbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 -- 172.21.15.60:0/249835371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf201fc3c0 con 0x7faf20161440 2024-09-16T15:38:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.357+0000 7faf1f400640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf20161440 0x7faf200cbee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf1802afe0 tx=0x7faf18003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.358+0000 7faf16a00640 1 -- 172.21.15.60:0/249835371 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18047070 con 0x7faf20161440 2024-09-16T15:38:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.358+0000 7faf16a00640 1 -- 172.21.15.60:0/249835371 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf1802f650 con 0x7faf20161440 2024-09-16T15:38:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.358+0000 7faf16a00640 1 -- 172.21.15.60:0/249835371 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18033050 con 0x7faf20161440 2024-09-16T15:38:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.358+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf201fc660 con 0x7faf20161440 2024-09-16T15:38:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.358+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf201fcc20 con 0x7faf20161440 2024-09-16T15:38:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.359+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf200b1770 con 0x7faf20161440 2024-09-16T15:38:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.360+0000 7faf16a00640 1 -- 172.21.15.60:0/249835371 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7faf1804a070 con 0x7faf20161440 2024-09-16T15:38:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.361+0000 7faf16a00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faf00078850 0x7faf0007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.362+0000 7faf16a00640 1 -- 172.21.15.60:0/249835371 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7faf180bbbb0 con 0x7faf20161440 2024-09-16T15:38:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.362+0000 7faf1ea00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faf00078850 0x7faf0007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.362+0000 7faf1ea00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faf00078850 0x7faf0007ad10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faf10009ac0 tx=0x7faf10005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.366+0000 7faf16a00640 1 -- 172.21.15.60:0/249835371 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faf18084180 con 0x7faf20161440 2024-09-16T15:38:51.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.559+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf2015ff70 con 0x7faf20161440 2024-09-16T15:38:51.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.560+0000 7faf16a00640 1 -- 172.21.15.60:0/249835371 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7faf18083a80 con 0x7faf20161440 2024-09-16T15:38:51.562 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:51.562 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:35.924959+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:38:51.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faf00078850 msgr2=0x7faf0007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faf00078850 0x7faf0007ad10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faf10009ac0 tx=0x7faf10005f90 comp rx=0 tx=0).stop 2024-09-16T15:38:51.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf20161440 msgr2=0x7faf200cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf20161440 0x7faf200cbee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf1802afe0 tx=0x7faf18003f10 comp rx=0 tx=0).stop 2024-09-16T15:38:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 shutdown_connections 2024-09-16T15:38:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faf00078850 0x7faf0007ad10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf20161440 0x7faf200cbee0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf200b4020 0x7faf201fbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 --2- 172.21.15.60:0/249835371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf200b34b0 0x7faf200cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 >> 172.21.15.60:0/249835371 conn(0x7faf200d3df0 msgr2=0x7faf200d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 shutdown_connections 2024-09-16T15:38:51.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.563+0000 7faf1fe00640 1 -- 172.21.15.60:0/249835371 wait complete. 2024-09-16T15:38:51.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:38:51.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.742+0000 7f4fe5a00640 1 -- 172.21.15.60:0/211969713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc4005230 msgr2=0x7f4fc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.742+0000 7f4fe5a00640 1 --2- 172.21.15.60:0/211969713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc4005230 0x7f4fc400b9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009ed0 tx=0x7f4fc4005830 comp rx=0 tx=0).stop 2024-09-16T15:38:51.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.742+0000 7f4fe5a00640 1 -- 172.21.15.60:0/211969713 shutdown_connections 2024-09-16T15:38:51.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.742+0000 7f4fe5a00640 1 --2- 172.21.15.60:0/211969713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc4005230 0x7f4fc400b9c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.742+0000 7f4fe5a00640 1 -- 172.21.15.60:0/211969713 >> 172.21.15.60:0/211969713 conn(0x7f4fe00d1dc0 msgr2=0x7f4fe00d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:51.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.742+0000 7f4fe5a00640 1 -- 172.21.15.60:0/211969713 shutdown_connections 2024-09-16T15:38:51.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.742+0000 7f4fe5a00640 1 -- 172.21.15.60:0/211969713 wait complete. 2024-09-16T15:38:51.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.744+0000 7f4fe5a00640 1 Processor -- start 2024-09-16T15:38:51.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.744+0000 7f4fe5a00640 1 -- start start 2024-09-16T15:38:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fe5a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc4005230 0x7f4fe015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fe5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe01e5150 0x7f4fe01e3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fe5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe01e3a80 0x7f4fe00b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fe5a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe006e810 con 0x7f4fe01e3a80 2024-09-16T15:38:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fe5a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe006e690 con 0x7f4fe01e5150 2024-09-16T15:38:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fe5a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe006e990 con 0x7f4fc4005230 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe01e3a80 0x7f4fe00b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe01e3a80 0x7f4fe00b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43460/0 (socket says 172.21.15.60:43460) 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 -- 172.21.15.60:0/3765229875 learned_addr learned my addr 172.21.15.60:0/3765229875 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdea00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc4005230 0x7f4fe015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fd7e00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe01e5150 0x7f4fe01e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 -- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc4005230 msgr2=0x7f4fe015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc4005230 0x7f4fe015a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 -- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe01e5150 msgr2=0x7f4fe01e3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe01e5150 0x7f4fe01e3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 -- 172.21.15.60:0/3765229875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe015a820 con 0x7f4fe01e3a80 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.745+0000 7f4fdf400640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe01e3a80 0x7f4fe00b3b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009e20 tx=0x7f4fd8004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.746+0000 7f4fd6a00640 1 -- 172.21.15.60:0/3765229875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8004650 con 0x7f4fe01e3a80 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.746+0000 7f4fd6a00640 1 -- 172.21.15.60:0/3765229875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fd803c480 con 0x7f4fe01e3a80 2024-09-16T15:38:51.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.746+0000 7f4fd6a00640 1 -- 172.21.15.60:0/3765229875 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd803c890 con 0x7f4fe01e3a80 2024-09-16T15:38:51.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.746+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe00cbff0 con 0x7f4fe01e3a80 2024-09-16T15:38:51.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.746+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe01e4390 con 0x7f4fe01e3a80 2024-09-16T15:38:51.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.747+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe00d0c40 con 0x7f4fe01e3a80 2024-09-16T15:38:51.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.748+0000 7f4fd6a00640 1 -- 172.21.15.60:0/3765229875 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4fd8044020 con 0x7f4fe01e3a80 2024-09-16T15:38:51.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.750+0000 7f4fd6a00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4fbc078880 0x7f4fbc07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:51.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.750+0000 7f4fd6a00640 1 -- 172.21.15.60:0/3765229875 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4fd80bb590 con 0x7f4fe01e3a80 2024-09-16T15:38:51.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.750+0000 7f4fdea00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4fbc078880 0x7f4fbc07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:51.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.751+0000 7f4fdea00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4fbc078880 0x7f4fbc07ad40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0001eb0 tx=0x7f4fd0001d80 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:51.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.754+0000 7f4fd6a00640 1 -- 172.21.15.60:0/3765229875 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4fd804b020 con 0x7f4fe01e3a80 2024-09-16T15:38:51.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.986+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fe00b4e30 con 0x7f4fe01e3a80 2024-09-16T15:38:51.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.987+0000 7f4fd6a00640 1 -- 172.21.15.60:0/3765229875 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f4fd80835a0 con 0x7f4fe01e3a80 2024-09-16T15:38:51.988 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:38:51.988 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:38:34.368784+0000","modified":"2024-09-16T15:38:37.370114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-16T15:38:51.989 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-09-16T15:38:51.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4fbc078880 msgr2=0x7f4fbc07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4fbc078880 0x7f4fbc07ad40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0001eb0 tx=0x7f4fd0001d80 comp rx=0 tx=0).stop 2024-09-16T15:38:51.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe01e3a80 msgr2=0x7f4fe00b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:51.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe01e3a80 0x7f4fe00b3b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009e20 tx=0x7f4fd8004250 comp rx=0 tx=0).stop 2024-09-16T15:38:51.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 shutdown_connections 2024-09-16T15:38:51.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4fbc078880 0x7f4fbc07ad40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fe01e3a80 0x7f4fe00b3b40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fe01e5150 0x7f4fe01e3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 --2- 172.21.15.60:0/3765229875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc4005230 0x7f4fe015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:51.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 >> 172.21.15.60:0/3765229875 conn(0x7f4fe00d1dc0 msgr2=0x7f4fe01ea870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:51.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 shutdown_connections 2024-09-16T15:38:51.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:51.990+0000 7f4fe5a00640 1 -- 172.21.15.60:0/3765229875 wait complete. 2024-09-16T15:38:52.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:38:34.368784+0000', 'modified': '2024-09-16T15:38:37.370114+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4402': {'gid': 4402, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6837/614854687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6835', 'nonce': 614854687}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 614854687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4402, 'qdb_cluster': [4402]} 2024-09-16T15:38:52.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:38:52.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-16T15:38:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.574+0000 7f29ab000640 1 -- 172.21.15.60:0/1186097794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a40cff70 msgr2=0x7f29a406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.574+0000 7f29ab000640 1 --2- 172.21.15.60:0/1186097794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a40cff70 0x7f29a406d000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f29a40763d0 tx=0x7f299c02aac0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.574+0000 7f29ab000640 1 -- 172.21.15.60:0/1186097794 shutdown_connections 2024-09-16T15:38:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.574+0000 7f29ab000640 1 --2- 172.21.15.60:0/1186097794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a40cff70 0x7f29a406d000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.574+0000 7f29ab000640 1 --2- 172.21.15.60:0/1186097794 >> v2:172.21.15.100:3300/0 conn(0x7f29a413efa0 0x7f29a40cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.574+0000 7f29ab000640 1 -- 172.21.15.60:0/1186097794 >> 172.21.15.60:0/1186097794 conn(0x7f29a40b2230 msgr2=0x7f29a40b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.574+0000 7f29ab000640 1 -- 172.21.15.60:0/1186097794 shutdown_connections 2024-09-16T15:38:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.574+0000 7f29ab000640 1 -- 172.21.15.60:0/1186097794 wait complete. 2024-09-16T15:38:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.575+0000 7f29ab000640 1 Processor -- start 2024-09-16T15:38:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.576+0000 7f29ab000640 1 -- start start 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.576+0000 7f29ab000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a406cb70 0x7f29a41f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.576+0000 7f29ab000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29a40cff70 0x7f29a40a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.576+0000 7f29ab000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29a413efa0 0x7f29a415b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.576+0000 7f29ab000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f29a40cd890 con 0x7f29a406cb70 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.576+0000 7f29ab000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f29a40cd710 con 0x7f29a413efa0 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.576+0000 7f29ab000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f29a40737d0 con 0x7f29a40cff70 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a406cb70 0x7f29a41f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a406cb70 0x7f29a41f3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43464/0 (socket says 172.21.15.60:43464) 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 -- 172.21.15.60:0/3271416856 learned_addr learned my addr 172.21.15.60:0/3271416856 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 -- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29a40cff70 msgr2=0x7f29a40a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3400640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29a40cff70 0x7f29a40a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a9000640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29a413efa0 0x7f29a415b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29a40cff70 0x7f29a40a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 -- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29a413efa0 msgr2=0x7f29a415b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29a413efa0 0x7f29a415b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 -- 172.21.15.60:0/3271416856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a413c6a0 con 0x7f29a406cb70 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3400640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29a40cff70 0x7f29a40a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.577+0000 7f29a3e00640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a406cb70 0x7f29a41f3ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f299400d740 tx=0x7f299400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.578+0000 7f299be00640 1 -- 172.21.15.60:0/3271416856 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994017070 con 0x7f29a406cb70 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.578+0000 7f299be00640 1 -- 172.21.15.60:0/3271416856 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2994009d20 con 0x7f29a406cb70 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.578+0000 7f299be00640 1 -- 172.21.15.60:0/3271416856 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29940106c0 con 0x7f29a406cb70 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.578+0000 7f29ab000640 1 -- 172.21.15.60:0/3271416856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a413b920 con 0x7f29a406cb70 2024-09-16T15:38:52.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.578+0000 7f29ab000640 1 -- 172.21.15.60:0/3271416856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a413bbc0 con 0x7f29a406cb70 2024-09-16T15:38:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.580+0000 7f299be00640 1 -- 172.21.15.60:0/3271416856 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f299401a070 con 0x7f29a406cb70 2024-09-16T15:38:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.581+0000 7f299be00640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2984078850 0x7f298407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.581+0000 7f299be00640 1 -- 172.21.15.60:0/3271416856 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f299409ab90 con 0x7f29a406cb70 2024-09-16T15:38:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.581+0000 7f29ade80740 1 Processor -- start 2024-09-16T15:38:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.581+0000 7f29a3400640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2984078850 0x7f298407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.582+0000 7f29ade80740 1 -- start start 2024-09-16T15:38:52.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.582+0000 7f29a3400640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2984078850 0x7f298407ad10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29a413cf80 tx=0x7f2990009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.582+0000 7f29ade80740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.582+0000 7f29ade80740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039da20 0x5654e03e1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29ade80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29ade80740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5654e0320430 con 0x5654e03e28b0 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29ade80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5654e03105d0 con 0x5654e039dfa0 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29ade80740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5654e02ffa00 con 0x5654e039da20 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03e2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43478/0 (socket says 172.21.15.60:43478) 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 -- 172.21.15.60:0/1608121622 learned_addr learned my addr 172.21.15.60:0/1608121622 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3400640 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039da20 0x5654e03e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a9000640 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 -- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039da20 msgr2=0x5654e03e1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039da20 0x5654e03e1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 -- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 msgr2=0x5654e03e18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03e18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 -- 172.21.15.60:0/1608121622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e02c8520 con 0x5654e03e28b0 2024-09-16T15:38:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3400640 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039da20 0x5654e03e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a9000640 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03e18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.583+0000 7f29a3e00640 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03e2c90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29940629f0 tx=0x7f2994062630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb5b1b6d079da47 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f2998c00640 1 -- 172.21.15.60:0/1608121622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994021040 con 0x5654e03e28b0 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f2998c00640 1 -- 172.21.15.60:0/1608121622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2994067540 con 0x5654e03e28b0 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f2998c00640 1 -- 172.21.15.60:0/1608121622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29940672f0 con 0x5654e03e28b0 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 -- 172.21.15.60:0/1608121622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 msgr2=0x5654e03e2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03e2c90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29940629f0 tx=0x7f2994062630 comp rx=0 tx=0).stop 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 -- 172.21.15.60:0/1608121622 shutdown_connections 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03e18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039da20 0x5654e03e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 --2- 172.21.15.60:0/1608121622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03e2c90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 -- 172.21.15.60:0/1608121622 >> 172.21.15.60:0/1608121622 conn(0x5654e0398ff0 msgr2=0x5654e039ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:52.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 -- 172.21.15.60:0/1608121622 shutdown_connections 2024-09-16T15:38:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.584+0000 7f29ade80740 1 -- 172.21.15.60:0/1608121622 wait complete. 2024-09-16T15:38:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.585+0000 7f29ade80740 1 -- start start 2024-09-16T15:38:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.585+0000 7f29ade80740 1 Processor -- start 2024-09-16T15:38:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.586+0000 7f29ade80740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039dfa0 0x5654e03e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.586+0000 7f29ade80740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.586+0000 7f29ade80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e03ded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.586+0000 7f29ade80740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5654e02a6330 con 0x5654e03e28b0 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.586+0000 7f29ade80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5654e0303810 con 0x5654e03b4550 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.586+0000 7f29ade80740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5654e0070620 con 0x5654e039dfa0 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.586+0000 7f29ade80740 10 client.0 _ll_register_callbacks cb 0x5654e02ab420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43488/0 (socket says 172.21.15.60:43488) 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 -- 172.21.15.60:0/2213781148 learned_addr learned my addr 172.21.15.60:0/2213781148 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3e00640 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039dfa0 0x5654e03e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a9000640 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e03ded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 -- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039dfa0 msgr2=0x5654e03e0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039dfa0 0x5654e03e0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 -- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 msgr2=0x5654e03ded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:38:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e03ded00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 -- 172.21.15.60:0/2213781148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e0061850 con 0x5654e03e28b0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a9000640 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e03ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f29a3400640 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03b1ab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f299000dd30 tx=0x7f2990000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853e17c2ca1db9f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.587+0000 7f298d600640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f298d600640 1 -- 172.21.15.60:0/2213781148 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299001f610 con 0x5654e03e28b0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f298d600640 1 -- 172.21.15.60:0/2213781148 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f299001f7b0 con 0x5654e03e28b0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 1 -- 172.21.15.60:0/2213781148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2994067540 con 0x5654e03e28b0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f298d600640 1 -- 172.21.15.60:0/2213781148 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f299002c050 con 0x5654e03e28b0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 10 client.4674 fetch_fsmap learned FSMap version 8 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 1 -- 172.21.15.60:0/2213781148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5654e00d86b0 con 0x5654e03e28b0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f298d600640 1 -- 172.21.15.60:0/2213781148 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2994067540 con 0x5654e03e28b0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f298d600640 1 -- 172.21.15.60:0/2213781148 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f299001f9b0 con 0x5654e03e28b0 2024-09-16T15:38:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2024-09-16T15:38:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 10 client.4674 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:38:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 20 client.4674 populate_metadata read hostname 'smithi060' 2024-09-16T15:38:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] conn(0x5654e03c5980 0x5654e03c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 4 client.4674 mds_command: new command op to 4402 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:38:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.588+0000 7f29ade80740 1 -- 172.21.15.60:0/2213781148 --> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654e0312520 con 0x5654e03c5980 2024-09-16T15:38:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.589+0000 7f29a3e00640 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] conn(0x5654e03c5980 0x5654e03c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.589+0000 7f29a3e00640 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] conn(0x5654e03c5980 0x5654e03c5d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71220d6520b48c3 server_cookie=a9da40d1c844ac79 in_seq=0 out_seq=0 2024-09-16T15:38:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.589+0000 7f298d600640 10 client.4674 ms_handle_connect on v2:172.21.15.100:6835/614854687 2024-09-16T15:38:52.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.590+0000 7f298d600640 1 -- 172.21.15.60:0/2213781148 <== mds.0 v2:172.21.15.100:6835/614854687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654e0312520 con 0x5654e03c5980 2024-09-16T15:38:52.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.590+0000 7f298d600640 10 client.4674 handle_command_reply: tid=0 2024-09-16T15:38:52.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 client.4674 shutdown 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 -- 172.21.15.60:0/2213781148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 msgr2=0x5654e03b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03b1ab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f299000dd30 tx=0x7f2990000bc0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 -- 172.21.15.60:0/2213781148 shutdown_connections 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] conn(0x5654e03c5980 0x5654e03c5d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e03ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f298d600640 0 client.4674 ms_handle_reset on v2:172.21.15.100:6835/614854687 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03e28b0 0x5654e03b1ab0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 --2- 172.21.15.60:0/2213781148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e039dfa0 0x5654e03e0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 -- 172.21.15.60:0/2213781148 >> 172.21.15.60:0/2213781148 conn(0x5654e039da20 msgr2=0x5654e0397bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 -- 172.21.15.60:0/2213781148 shutdown_connections 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 1 -- 172.21.15.60:0/2213781148 wait complete. 2024-09-16T15:38:52.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.591+0000 7f29ade80740 20 client.4674 trim_cache size 0 max 16384 2024-09-16T15:38:52.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.611+0000 7f29ade80740 1 Processor -- start 2024-09-16T15:38:52.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.611+0000 7f29ade80740 1 -- start start 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29ade80740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e039dfa0 0x5654e039b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29ade80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e039c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29ade80740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 0x5654e03ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29ade80740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4200410 con 0x5654e039dfa0 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29ade80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f29a40b4010 con 0x5654e03b4550 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29ade80740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f29a40b4660 con 0x5654e03e28b0 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e039dfa0 0x5654e039b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e039dfa0 0x5654e039b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43502/0 (socket says 172.21.15.60:43502) 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 -- 172.21.15.60:0/723733286 learned_addr learned my addr 172.21.15.60:0/723733286 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a9000640 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 0x5654e03ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3400640 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e039c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 -- 172.21.15.60:0/723733286 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 msgr2=0x5654e03ddd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 0x5654e03ddd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 -- 172.21.15.60:0/723733286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 msgr2=0x5654e039c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e039c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 -- 172.21.15.60:0/723733286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e0392f80 con 0x5654e039dfa0 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.612+0000 7f29a3e00640 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e039dfa0 0x5654e039b480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2994009610 tx=0x7f299405f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b3a8af17420d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f298fe00640 1 -- 172.21.15.60:0/723733286 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994021040 con 0x5654e039dfa0 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f298fe00640 1 -- 172.21.15.60:0/723733286 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29940670b0 con 0x5654e039dfa0 2024-09-16T15:38:52.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f298fe00640 1 -- 172.21.15.60:0/723733286 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994061cc0 con 0x5654e039dfa0 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 -- 172.21.15.60:0/723733286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e039dfa0 msgr2=0x5654e039b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e039dfa0 0x5654e039b480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2994009610 tx=0x7f299405f1e0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 -- 172.21.15.60:0/723733286 shutdown_connections 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 0x5654e03ddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e03b4550 0x5654e039c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 --2- 172.21.15.60:0/723733286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e039dfa0 0x5654e039b480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 -- 172.21.15.60:0/723733286 >> 172.21.15.60:0/723733286 conn(0x5654e039da20 msgr2=0x5654e03f3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 -- 172.21.15.60:0/723733286 shutdown_connections 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.613+0000 7f29ade80740 1 -- 172.21.15.60:0/723733286 wait complete. 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.614+0000 7f29ade80740 1 -- start start 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.614+0000 7f29ade80740 1 Processor -- start 2024-09-16T15:38:52.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.615+0000 7f29ade80740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.615+0000 7f29ade80740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03b4550 0x5654e0405190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.615+0000 7f29ade80740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 0x5654e04056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.615+0000 7f29ade80740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5654e03b1ff0 con 0x5654e03b4550 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.615+0000 7f29ade80740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5654e03e2100 con 0x5654e039dfa0 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.615+0000 7f29ade80740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5654e03f52f0 con 0x5654e03e28b0 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.615+0000 7f29ade80740 10 client.0 _ll_register_callbacks cb 0x7f299000d980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03b4550 0x5654e0405190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a9000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 0x5654e04056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44328/0 (socket says 172.21.15.60:44328) 2024-09-16T15:38:52.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3e00640 1 -- 172.21.15.60:0/467438895 learned_addr learned my addr 172.21.15.60:0/467438895 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3400640 1 -- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 msgr2=0x5654e04056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:38:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3400640 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 0x5654e04056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3400640 1 -- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 msgr2=0x5654e03c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:38:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3400640 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3400640 1 -- 172.21.15.60:0/467438895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e0466590 con 0x5654e03b4550 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3e00640 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f29a3400640 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03b4550 0x5654e0405190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x5654e03c5790 tx=0x7f29900072f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04da3f01157e4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.616+0000 7f298d600640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f298d600640 1 -- 172.21.15.60:0/467438895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990034070 con 0x5654e03b4550 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f298d600640 1 -- 172.21.15.60:0/467438895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f29900069d0 con 0x5654e03b4550 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 1 -- 172.21.15.60:0/467438895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29940670b0 con 0x5654e03b4550 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f298d600640 1 -- 172.21.15.60:0/467438895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f2990006bd0 con 0x5654e03b4550 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 10 client.4680 fetch_fsmap learned FSMap version 8 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 1 -- 172.21.15.60:0/467438895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5654e03c6ba0 con 0x5654e03b4550 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f298d600640 1 -- 172.21.15.60:0/467438895 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29940670b0 con 0x5654e03b4550 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f298d600640 1 -- 172.21.15.60:0/467438895 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f299001f3e0 con 0x5654e03b4550 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 10 client.4680 fetch_fsmap finished waiting for FSMap version 8 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 10 client.4680 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:38:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 20 client.4680 populate_metadata read hostname 'smithi060' 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] conn(0x5654e03f3550 0x5654e0461df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 4 client.4680 mds_command: new command op to 4402 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.617+0000 7f29ade80740 1 -- 172.21.15.60:0/467438895 --> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] -- command(tid 0: {"prefix": "status"}) -- 0x5654e030dfd0 con 0x5654e03f3550 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.618+0000 7f29a3e00640 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] conn(0x5654e03f3550 0x5654e0461df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.618+0000 7f29a3e00640 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] conn(0x5654e03f3550 0x5654e0461df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914963078ce5d73d server_cookie=be88f666a9b0bf68 in_seq=0 out_seq=0 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.618+0000 7f298d600640 10 client.4680 ms_handle_connect on v2:172.21.15.100:6835/614854687 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f298d600640 1 -- 172.21.15.60:0/467438895 <== mds.0 v2:172.21.15.100:6835/614854687 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5654e030dfd0 con 0x5654e03f3550 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f298d600640 10 client.4680 handle_command_reply: tid=0 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 client.4680 shutdown 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 -- 172.21.15.60:0/467438895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03b4550 msgr2=0x5654e0405190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03b4550 0x5654e0405190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x5654e03c5790 tx=0x7f29900072f0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 -- 172.21.15.60:0/467438895 shutdown_connections 2024-09-16T15:38:52.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:6835/614854687,v1:172.21.15.100:6837/614854687] conn(0x5654e03f3550 0x5654e0461df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5654e03e28b0 0x5654e04056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5654e03b4550 0x5654e0405190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f298d600640 0 client.4680 ms_handle_reset on v2:172.21.15.100:6835/614854687 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 --2- 172.21.15.60:0/467438895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5654e039dfa0 0x5654e03c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 -- 172.21.15.60:0/467438895 >> 172.21.15.60:0/467438895 conn(0x5654e039da20 msgr2=0x5654e039c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 -- 172.21.15.60:0/467438895 shutdown_connections 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 1 -- 172.21.15.60:0/467438895 wait complete. 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.619+0000 7f29ade80740 20 client.4680 trim_cache size 0 max 16384 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:38:52.624 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4402, 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 17.702234971999999, 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 8, 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 23, 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 23, 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 23.716171001999999 2024-09-16T15:38:52.625 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:38:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 -- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2984078850 msgr2=0x7f298407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2984078850 0x7f298407ad10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29a413cf80 tx=0x7f2990009040 comp rx=0 tx=0).stop 2024-09-16T15:38:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 -- 172.21.15.60:0/3271416856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a406cb70 msgr2=0x7f29a41f3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a406cb70 0x7f29a41f3ee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f299400d740 tx=0x7f299400dc00 comp rx=0 tx=0).stop 2024-09-16T15:38:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 -- 172.21.15.60:0/3271416856 shutdown_connections 2024-09-16T15:38:52.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2984078850 0x7f298407ad10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29a413efa0 0x7f29a415b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29a40cff70 0x7f29a40a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 --2- 172.21.15.60:0/3271416856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29a406cb70 0x7f29a41f3ee0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:52.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 -- 172.21.15.60:0/3271416856 >> 172.21.15.60:0/3271416856 conn(0x7f29a40b2230 msgr2=0x7f29a40b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:52.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 -- 172.21.15.60:0/3271416856 shutdown_connections 2024-09-16T15:38:52.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:52.620+0000 7f29ab000640 1 -- 172.21.15.60:0/3271416856 wait complete. 2024-09-16T15:38:52.636 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T15:38:52.660 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T15:38:52.660 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-16T15:38:52.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:52.661 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:38:52.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:52.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-09-16T15:38:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2024-09-16T15:38:52.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:38:52.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:52.798 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:38:52.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:52.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-09-16T15:38:52.932 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2024-09-16T15:38:52.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:38:52.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:52.935 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:38:52.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:52.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-16T15:38:53.078 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-09-16T15:38:53.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:38:53.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:53.082 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:38:53.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:53.117 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-16T15:38:53.215 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-09-16T15:38:53.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:38:53.218 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-16T15:38:53.218 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T15:38:53.218 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:38:53.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:38:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.378+0000 7fb3e9800640 1 -- 172.21.15.60:0/831537021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e40b43f0 msgr2=0x7fb3e40b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.378+0000 7fb3e9800640 1 --2- 172.21.15.60:0/831537021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e40b43f0 0x7fb3e40b47d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4005fd0 tx=0x7fb3d402a4b0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.378+0000 7fb3e9800640 1 -- 172.21.15.60:0/831537021 shutdown_connections 2024-09-16T15:38:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.378+0000 7fb3e9800640 1 --2- 172.21.15.60:0/831537021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3e4073bb0 0x7fb3e406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.378+0000 7fb3e9800640 1 --2- 172.21.15.60:0/831537021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e40b43f0 0x7fb3e40b47d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.378+0000 7fb3e9800640 1 -- 172.21.15.60:0/831537021 >> 172.21.15.60:0/831537021 conn(0x7fb3e40b1950 msgr2=0x7fb3e40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.378+0000 7fb3e9800640 1 -- 172.21.15.60:0/831537021 shutdown_connections 2024-09-16T15:38:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.378+0000 7fb3e9800640 1 -- 172.21.15.60:0/831537021 wait complete. 2024-09-16T15:38:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.380+0000 7fb3e9800640 1 Processor -- start 2024-09-16T15:38:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.380+0000 7fb3e9800640 1 -- start start 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3e4073bb0 0x7fb3e4153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e4074150 0x7fb3e40cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e9800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3e40b43f0 0x7fb3e40ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e9800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e406f3b0 con 0x7fb3e4074150 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e40704c0 con 0x7fb3e4073bb0 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e9800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e406f530 con 0x7fb3e40b43f0 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e4074150 0x7fb3e40cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e4074150 0x7fb3e40cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43544/0 (socket says 172.21.15.60:43544) 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e2000640 1 -- 172.21.15.60:0/2343610325 learned_addr learned my addr 172.21.15.60:0/2343610325 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e2a00640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3e4073bb0 0x7fb3e4153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:53.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.381+0000 7fb3e3400640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3e40b43f0 0x7fb3e40ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:53.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3e2000640 1 -- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3e40b43f0 msgr2=0x7fb3e40ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:53.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3e2000640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3e40b43f0 0x7fb3e40ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3e2000640 1 -- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3e4073bb0 msgr2=0x7fb3e4153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:53.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3e2000640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3e4073bb0 0x7fb3e4153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3e2000640 1 -- 172.21.15.60:0/2343610325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e4159f80 con 0x7fb3e4074150 2024-09-16T15:38:53.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3e2000640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e4074150 0x7fb3e40cdc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c730 tx=0x7fb3d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:53.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3daa00640 1 -- 172.21.15.60:0/2343610325 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0017070 con 0x7fb3e4074150 2024-09-16T15:38:53.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3daa00640 1 -- 172.21.15.60:0/2343610325 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3d0005180 con 0x7fb3e4074150 2024-09-16T15:38:53.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3daa00640 1 -- 172.21.15.60:0/2343610325 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000a640 con 0x7fb3e4074150 2024-09-16T15:38:53.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e415a260 con 0x7fb3e4074150 2024-09-16T15:38:53.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.382+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e415a820 con 0x7fb3e4074150 2024-09-16T15:38:53.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.383+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e415ac50 con 0x7fb3e4074150 2024-09-16T15:38:53.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.385+0000 7fb3daa00640 1 -- 172.21.15.60:0/2343610325 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb3d001a070 con 0x7fb3e4074150 2024-09-16T15:38:53.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.385+0000 7fb3daa00640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb3c4078880 0x7fb3c407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.386+0000 7fb3daa00640 1 -- 172.21.15.60:0/2343610325 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb3d0099c90 con 0x7fb3e4074150 2024-09-16T15:38:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.386+0000 7fb3e2a00640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb3c4078880 0x7fb3c407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.387+0000 7fb3e2a00640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb3c4078880 0x7fb3c407ad40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4033000 tx=0x7fb3d4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.390+0000 7fb3daa00640 1 -- 172.21.15.60:0/2343610325 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb3e40cf9d0 con 0x7fb3e4074150 2024-09-16T15:38:53.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.625+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb3e40760b0 con 0x7fb3e4074150 2024-09-16T15:38:53.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.626+0000 7fb3daa00640 1 -- 172.21.15.60:0/2343610325 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fb3d0061c10 con 0x7fb3e4074150 2024-09-16T15:38:53.627 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:38:53.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb3c4078880 msgr2=0x7fb3c407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:53.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb3c4078880 0x7fb3c407ad40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4033000 tx=0x7fb3d4004730 comp rx=0 tx=0).stop 2024-09-16T15:38:53.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e4074150 msgr2=0x7fb3e40cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:53.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e4074150 0x7fb3e40cdc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c730 tx=0x7fb3d000cbf0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 shutdown_connections 2024-09-16T15:38:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb3c4078880 0x7fb3c407ad40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3e40b43f0 0x7fb3e40ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3e4074150 0x7fb3e40cdc00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 --2- 172.21.15.60:0/2343610325 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3e4073bb0 0x7fb3e4153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 >> 172.21.15.60:0/2343610325 conn(0x7fb3e40b1950 msgr2=0x7fb3e415ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 shutdown_connections 2024-09-16T15:38:53.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53.628+0000 7fb3e9800640 1 -- 172.21.15.60:0/2343610325 wait complete. 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:38:53.645 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:38:53.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:53.646 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:38:53.680 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:38:53.681 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42716sec preferred_lft 42716sec 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:38:53.683 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T15:38:53.683 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:53.683 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:53.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-09-16T15:38:53.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-09-16T15:38:53.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-09-16T15:38:53.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T15:38:53.684 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:54.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:54.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:54.111 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:38:54.169 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:38:54.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:54.173 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:38:54.220 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:38:54.220 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:38:54.221 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:38:54.221 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:38:54.221 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:38:54.221 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:54.221 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:54.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:38:54.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:38:54.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:38:54.222 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:54.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:54.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:54.855 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:38:54.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:54.892 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-09-16T15:38:54.968 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:54.968 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:54.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:38:54.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T15:38:54.969 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:55.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:55.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:55.150 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T15:38:55.150 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:55.150 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:55.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T15:38:55.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T15:38:55.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T15:38:55.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T15:38:55.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T15:38:55.151 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:55.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:55.391 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:55.391 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:55.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:38:55.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T15:38:55.391 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:55.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:38:55.561 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:38:55.596 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:38:55.596 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:38:55.596 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:38:55.670 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:38:55.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:55.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:38:55.814 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:38:55.814 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:38:56.446 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:38:56.446 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:38:56.446 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:38:56.446 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:38:56.446 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:38:56.449 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:38:56.449 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:38:56.449 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:38:56.449 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:38:56.449 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:38:56.452 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:38:56.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:56.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:38:56.510 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:38:56.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:38:56.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:56.610 INFO:teuthology.orchestra.run.smithi060.stdout:client4692 2024-09-16T15:38:56.610 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:56.610 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:38:56.646 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:38:56.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:38:56.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:56.784 INFO:teuthology.orchestra.run.smithi060.stdout:client4692 2024-09-16T15:38:56.785 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:38:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:38:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:56.885 INFO:teuthology.orchestra.run.smithi060.stdout:client4692 2024-09-16T15:38:56.885 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b577f92e-7c09-4d32-a34c-b870818f4d85.client4692/status') 2024-09-16T15:38:56.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:56.991 INFO:teuthology.orchestra.run.smithi060.stdout:instance: client.4692 (3)192.168.0.1:0/2866811965 2024-09-16T15:38:56.991 INFO:teuthology.orchestra.run.smithi060.stdout:blocklisted: false 2024-09-16T15:38:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records in 2024-09-16T15:38:56.991 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records out 2024-09-16T15:38:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:69 bytes copied, 5.9539e-05 s, 1.2 MB/s 2024-09-16T15:38:56.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:57.002 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-16T15:38:57.002 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T15:38:57.002 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-16T15:38:57.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.162+0000 7ffba4c00640 1 -- 172.21.15.60:0/3742761432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00cf300 msgr2=0x7ffba00729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.162+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3742761432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00cf300 0x7ffba00729a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009890 tx=0x7ffb8802c7d0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.162+0000 7ffba4c00640 1 -- 172.21.15.60:0/3742761432 shutdown_connections 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.162+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3742761432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffba00ccfd0 0x7ffba0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.162+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3742761432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00cf300 0x7ffba00729a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.162+0000 7ffba4c00640 1 -- 172.21.15.60:0/3742761432 >> 172.21.15.60:0/3742761432 conn(0x7ffba00b1950 msgr2=0x7ffba00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.163+0000 7ffba4c00640 1 -- 172.21.15.60:0/3742761432 shutdown_connections 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.163+0000 7ffba4c00640 1 -- 172.21.15.60:0/3742761432 wait complete. 2024-09-16T15:38:57.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.164+0000 7ffba4c00640 1 Processor -- start 2024-09-16T15:38:57.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.164+0000 7ffba4c00640 1 -- start start 2024-09-16T15:38:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffba4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00ccfd0 0x7ffba009d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffba4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba00cf300 0x7ffba01f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffba4c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffba01445d0 0x7ffba006f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffba4c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0073420 con 0x7ffba00ccfd0 2024-09-16T15:38:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffba4c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffba00b4f60 con 0x7ffba00cf300 2024-09-16T15:38:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffba4c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00735a0 con 0x7ffba01445d0 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffb9de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00ccfd0 0x7ffba009d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffb9de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00ccfd0 0x7ffba009d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43554/0 (socket says 172.21.15.60:43554) 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.165+0000 7ffb9de00640 1 -- 172.21.15.60:0/3629427292 learned_addr learned my addr 172.21.15.60:0/3629427292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb9e800640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffba01445d0 0x7ffba006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb95400640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba00cf300 0x7ffba01f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb9de00640 1 -- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffba01445d0 msgr2=0x7ffba006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb9de00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffba01445d0 0x7ffba006f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb9de00640 1 -- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba00cf300 msgr2=0x7ffba01f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb9de00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba00cf300 0x7ffba01f84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb9de00640 1 -- 172.21.15.60:0/3629427292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba00c9170 con 0x7ffba00ccfd0 2024-09-16T15:38:57.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb9de00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00ccfd0 0x7ffba009d250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c430 tx=0x7ffb9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb96a00640 1 -- 172.21.15.60:0/3629427292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90011790 con 0x7ffba00ccfd0 2024-09-16T15:38:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb96a00640 1 -- 172.21.15.60:0/3629427292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb90017070 con 0x7ffba00ccfd0 2024-09-16T15:38:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffb96a00640 1 -- 172.21.15.60:0/3629427292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90011ce0 con 0x7ffba00ccfd0 2024-09-16T15:38:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.166+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba01f8eb0 con 0x7ffba00ccfd0 2024-09-16T15:38:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.167+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba009d790 con 0x7ffba00ccfd0 2024-09-16T15:38:57.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.168+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba0073a50 con 0x7ffba00ccfd0 2024-09-16T15:38:57.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.169+0000 7ffb96a00640 1 -- 172.21.15.60:0/3629427292 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ffb9001c460 con 0x7ffba00ccfd0 2024-09-16T15:38:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.170+0000 7ffb96a00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ffb7c078850 0x7ffb7c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.170+0000 7ffb96a00640 1 -- 172.21.15.60:0/3629427292 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ffb9009a620 con 0x7ffba00ccfd0 2024-09-16T15:38:57.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.171+0000 7ffb95400640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ffb7c078850 0x7ffb7c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.171+0000 7ffb95400640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ffb7c078850 0x7ffb7c07ad10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffb88037000 tx=0x7ffb88004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.174+0000 7ffb96a00640 1 -- 172.21.15.60:0/3629427292 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffb900625a0 con 0x7ffba00ccfd0 2024-09-16T15:38:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.410+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffba01f93d0 con 0x7ffba00ccfd0 2024-09-16T15:38:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.410+0000 7ffb96a00640 1 -- 172.21.15.60:0/3629427292 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7ffb90066fd0 con 0x7ffba00ccfd0 2024-09-16T15:38:57.412 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.412+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ffb7c078850 msgr2=0x7ffb7c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.412+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ffb7c078850 0x7ffb7c07ad10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffb88037000 tx=0x7ffb88004730 comp rx=0 tx=0).stop 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.412+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00ccfd0 msgr2=0x7ffba009d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.412+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00ccfd0 0x7ffba009d250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c430 tx=0x7ffb9000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.413+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 shutdown_connections 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.413+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ffb7c078850 0x7ffb7c07ad10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.413+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffba01445d0 0x7ffba006f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.413+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffba00cf300 0x7ffba01f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.413+0000 7ffba4c00640 1 --2- 172.21.15.60:0/3629427292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffba00ccfd0 0x7ffba009d250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.413+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 >> 172.21.15.60:0/3629427292 conn(0x7ffba00b1950 msgr2=0x7ffba00aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.413+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 shutdown_connections 2024-09-16T15:38:57.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57.413+0000 7ffba4c00640 1 -- 172.21.15.60:0/3629427292 wait complete. 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:38:57.430 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-16T15:38:57.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:57.430 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:38:57.467 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42712sec preferred_lft 42712sec 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:38:57.468 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:38:57.469 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:38:57.469 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:57.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:57.470 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:38:57.564 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:38:57.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:57.567 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:38:57.603 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:38:57.603 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:38:57.603 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:38:57.603 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:38:57.603 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:38:57.603 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:57.604 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:57.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:38:57.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:38:57.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:38:57.604 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:57.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:57.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:57.802 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:38:57.837 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:38:57.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:57.837 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-09-16T15:38:57.912 INFO:teuthology.orchestra.run.smithi060.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-16T15:38:57.912 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:57.912 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:57.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:38:57.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-16T15:38:57.912 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:58.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:58.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:58.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:58.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 8e:8e:e1:1a:72:1d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:58.148 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c8e:e1ff:fe1a:721d/64 scope link 2024-09-16T15:38:58.149 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:58.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:58.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-16T15:38:58.206 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:38:58.206 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:38:58.206 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:38:58.206 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:58.206 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:38:58.206 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:58.207 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:38:58.207 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 8e:8e:e1:1a:72:1d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T15:38:58.207 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T15:38:58.207 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:58.207 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::8c8e:e1ff:fe1a:721d/64 scope link 2024-09-16T15:38:58.207 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:58.209 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-16T15:38:58.209 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:58.210 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:58.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-16T15:38:58.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-16T15:38:58.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-16T15:38:58.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-16T15:38:58.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-16T15:38:58.210 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:58.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:58.426 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:58.427 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:38:58.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.1 up 2024-09-16T15:38:58.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.1 master ceph-brx 2024-09-16T15:38:58.427 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:38:58.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:58.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:58.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:38:58.593 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-16T15:38:58.628 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-16T15:38:58.629 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:38:58.629 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-16T15:38:58.704 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-16T15:38:58.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:58.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:38:58.834 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:38:58.834 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:38:58.862 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:38:58.863 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:38:58.863 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:38:58.863 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 2@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:38:58.863 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:38:58.864 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-16T15:38:58.864 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:38:58.864 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:38:58.864 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:38:58.864 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:38:58.867 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:38:58.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:58.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-16T15:38:58.923 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:38:59.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:38:59.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:59.016 INFO:teuthology.orchestra.run.smithi060.stdout:client4768 2024-09-16T15:38:59.016 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:38:59.016 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:38:59.052 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:38:59.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:38:59.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:59.181 INFO:teuthology.orchestra.run.smithi060.stdout:client4768 2024-09-16T15:38:59.181 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:38:59.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:38:59.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:59.271 INFO:teuthology.orchestra.run.smithi060.stdout:client4768 2024-09-16T15:38:59.272 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b577f92e-7c09-4d32-a34c-b870818f4d85.client4768/status') 2024-09-16T15:38:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:38:59.371 INFO:teuthology.orchestra.run.smithi060.stdout:instance: client.4768 (3)172.21.15.60:0/3612305200 2024-09-16T15:38:59.372 INFO:teuthology.orchestra.run.smithi060.stdout:blocklisted: false 2024-09-16T15:38:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records in 2024-09-16T15:38:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records out 2024-09-16T15:38:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:70 bytes copied, 6.991e-05 s, 1.0 MB/s 2024-09-16T15:38:59.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:38:59.382 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-16T15:38:59.382 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T15:38:59.382 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:38:59.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:38:59.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.543+0000 7f8729c00640 1 -- 172.21.15.60:0/3438747049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 msgr2=0x7f87240729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.543+0000 7f8729c00640 1 --2- 172.21.15.60:0/3438747049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 0x7f87240729a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f870c009860 tx=0x7f870c02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.543+0000 7f8729c00640 1 -- 172.21.15.60:0/3438747049 shutdown_connections 2024-09-16T15:38:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.543+0000 7f8729c00640 1 --2- 172.21.15.60:0/3438747049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 0x7f87240729a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.543+0000 7f8729c00640 1 -- 172.21.15.60:0/3438747049 >> 172.21.15.60:0/3438747049 conn(0x7f87240b1950 msgr2=0x7f87240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.543+0000 7f8729c00640 1 -- 172.21.15.60:0/3438747049 shutdown_connections 2024-09-16T15:38:59.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.543+0000 7f8729c00640 1 -- 172.21.15.60:0/3438747049 wait complete. 2024-09-16T15:38:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.545+0000 7f8729c00640 1 Processor -- start 2024-09-16T15:38:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.545+0000 7f8729c00640 1 -- start start 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8729c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240ccfd0 0x7f872413cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8729c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 0x7f872415dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8729c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87241445d0 0x7f87241e8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8729c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f87240735f0 con 0x7f87240ccfd0 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8729c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87240b4f60 con 0x7f87240cf300 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8729c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8724073770 con 0x7f87241445d0 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8722a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240ccfd0 0x7f872413cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8722a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240ccfd0 0x7f872413cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41486/0 (socket says 172.21.15.60:41486) 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8722a00640 1 -- 172.21.15.60:0/3950512738 learned_addr learned my addr 172.21.15.60:0/3950512738 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f871a000640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 0x7f872415dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.546+0000 7f8723400640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87241445d0 0x7f87241e8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f8722a00640 1 -- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87241445d0 msgr2=0x7f87241e8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f8722a00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87241445d0 0x7f87241e8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f8722a00640 1 -- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 msgr2=0x7f872415dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f8722a00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 0x7f872415dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f8722a00640 1 -- 172.21.15.60:0/3950512738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872413bea0 con 0x7f87240ccfd0 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f871a000640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 0x7f872415dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f8722a00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240ccfd0 0x7f872413cf40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f871400c400 tx=0x7f871400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f871b400640 1 -- 172.21.15.60:0/3950512738 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714017070 con 0x7f87240ccfd0 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f871b400640 1 -- 172.21.15.60:0/3950512738 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87140053c0 con 0x7f87240ccfd0 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f871b400640 1 -- 172.21.15.60:0/3950512738 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714003850 con 0x7f87240ccfd0 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872413c180 con 0x7f87240ccfd0 2024-09-16T15:38:59.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.547+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872413c740 con 0x7f87240ccfd0 2024-09-16T15:38:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.548+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872413cb80 con 0x7f87240ccfd0 2024-09-16T15:38:59.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.550+0000 7f871b400640 1 -- 172.21.15.60:0/3950512738 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f871401a070 con 0x7f87240ccfd0 2024-09-16T15:38:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.551+0000 7f871b400640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8700078850 0x7f870007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.551+0000 7f871b400640 1 -- 172.21.15.60:0/3950512738 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8714099d70 con 0x7f87240ccfd0 2024-09-16T15:38:59.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.551+0000 7f871a000640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8700078850 0x7f870007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.552+0000 7f871a000640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8700078850 0x7f870007ad10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f870c034000 tx=0x7f870c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:59.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.555+0000 7f871b400640 1 -- 172.21.15.60:0/3950512738 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8714062340 con 0x7f87240ccfd0 2024-09-16T15:38:59.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.796+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8724073c20 con 0x7f87240ccfd0 2024-09-16T15:38:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.796+0000 7f871b400640 1 -- 172.21.15.60:0/3950512738 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f8714061c40 con 0x7f87240ccfd0 2024-09-16T15:38:59.799 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:38:59.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8700078850 msgr2=0x7f870007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8700078850 0x7f870007ad10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f870c034000 tx=0x7f870c004730 comp rx=0 tx=0).stop 2024-09-16T15:38:59.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240ccfd0 msgr2=0x7f872413cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240ccfd0 0x7f872413cf40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f871400c400 tx=0x7f871400c8c0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 shutdown_connections 2024-09-16T15:38:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8700078850 0x7f870007ad10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87241445d0 0x7f87241e8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87240cf300 0x7f872415dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 --2- 172.21.15.60:0/3950512738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87240ccfd0 0x7f872413cf40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 >> 172.21.15.60:0/3950512738 conn(0x7f87240b1950 msgr2=0x7f87240ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 shutdown_connections 2024-09-16T15:38:59.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:38:59.799+0000 7f8729c00640 1 -- 172.21.15.60:0/3950512738 wait complete. 2024-09-16T15:38:59.817 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T15:38:59.817 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:38:59.817 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:38:59.817 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:38:59.817 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:38:59.817 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T15:38:59.817 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T15:38:59.818 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:38:59.818 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:38:59.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:38:59.818 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:38:59.857 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:38:59.857 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:38:59.857 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:38:59.857 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:38:59.858 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42709sec preferred_lft 42709sec 2024-09-16T15:38:59.859 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:38:59.859 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:38:59.859 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T15:38:59.859 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:38:59.859 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:38:59.859 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-09-16T15:38:59.859 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-09-16T15:38:59.859 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-09-16T15:38:59.859 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T15:38:59.859 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:38:59.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:38:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:00.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:39:00.030 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:39:00.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:00.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:00.326 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:39:00.381 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:39:00.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:00.384 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:39:00.438 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:39:00.438 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:39:00.438 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:39:00.438 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:39:00.438 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:39:00.438 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:00.438 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:00.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:39:00.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:39:00.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:39:00.439 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:00.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:01.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:01.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:01.058 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:39:01.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:01.094 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-09-16T15:39:01.169 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:01.169 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:01.169 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:39:01.169 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-16T15:39:01.169 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:01.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:01.345 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-16T15:39:01.346 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:01.346 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:01.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-16T15:39:01.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T15:39:01.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-16T15:39:01.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-16T15:39:01.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-16T15:39:01.346 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:01.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:01.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:01.584 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:01.584 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:01.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:39:01.585 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T15:39:01.585 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:01.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:01.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:01.751 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:01.751 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:01.785 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:39:01.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:01.786 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:01.861 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:39:01.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:01.861 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:01.995 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:01.995 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:39:02.595 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:02.596 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:39:02.596 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:39:02.596 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:39:02.596 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:39:02.598 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:39:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:39:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:39:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:39:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:39:02.602 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:39:02.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:02.602 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:02.658 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:02.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:02.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:02.747 INFO:teuthology.orchestra.run.smithi100.stdout:client4710 2024-09-16T15:39:02.747 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:02.747 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:02.782 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:02.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:02.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:02.910 INFO:teuthology.orchestra.run.smithi100.stdout:client4710 2024-09-16T15:39:02.910 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:02.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:02.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:02.999 INFO:teuthology.orchestra.run.smithi100.stdout:client4710 2024-09-16T15:39:02.999 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b577f92e-7c09-4d32-a34c-b870818f4d85.client4710/status') 2024-09-16T15:39:03.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:03.098 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.4710 (3)172.21.15.100:0/3066256859 2024-09-16T15:39:03.098 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-09-16T15:39:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-09-16T15:39:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-09-16T15:39:03.098 INFO:teuthology.orchestra.run.smithi100.stderr:71 bytes copied, 5.967e-05 s, 1.2 MB/s 2024-09-16T15:39:03.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:03.109 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-16T15:39:03.109 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T15:39:03.109 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:03.109 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:39:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.279+0000 7f2db9000640 1 -- 172.21.15.60:0/352666754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db40ccf00 msgr2=0x7f2db40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.279+0000 7f2db9000640 1 --2- 172.21.15.60:0/352666754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db40ccf00 0x7f2db40b2440 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2db406dac0 tx=0x7f2d9c02a560 comp rx=0 tx=0).stop 2024-09-16T15:39:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.279+0000 7f2db9000640 1 -- 172.21.15.60:0/352666754 shutdown_connections 2024-09-16T15:39:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.279+0000 7f2db9000640 1 --2- 172.21.15.60:0/352666754 >> v2:172.21.15.100:3300/0 conn(0x7f2db4073fb0 0x7f2db4074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.279+0000 7f2db9000640 1 --2- 172.21.15.60:0/352666754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db40ccf00 0x7f2db40b2440 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.279+0000 7f2db9000640 1 -- 172.21.15.60:0/352666754 >> 172.21.15.60:0/352666754 conn(0x7f2db40b1950 msgr2=0x7f2db40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.279+0000 7f2db9000640 1 -- 172.21.15.60:0/352666754 shutdown_connections 2024-09-16T15:39:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.279+0000 7f2db9000640 1 -- 172.21.15.60:0/352666754 wait complete. 2024-09-16T15:39:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.280+0000 7f2db9000640 1 Processor -- start 2024-09-16T15:39:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.280+0000 7f2db9000640 1 -- start start 2024-09-16T15:39:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.281+0000 7f2db9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db4073fb0 0x7f2db41f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.281+0000 7f2db9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db40ccf00 0x7f2db415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.281+0000 7f2db9000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2db415f5a0 0x7f2db4156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.281+0000 7f2db9000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2db406f2e0 con 0x7f2db4073fb0 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.281+0000 7f2db9000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2db406f160 con 0x7f2db40ccf00 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.281+0000 7f2db9000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2db406f460 con 0x7f2db415f5a0 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db4073fb0 0x7f2db41f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db4073fb0 0x7f2db41f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41510/0 (socket says 172.21.15.60:41510) 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 -- 172.21.15.60:0/989511800 learned_addr learned my addr 172.21.15.60:0/989511800 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2da9400640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db40ccf00 0x7f2db415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db2800640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2db415f5a0 0x7f2db4156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 -- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2db415f5a0 msgr2=0x7f2db4156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2db415f5a0 0x7f2db4156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 -- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db40ccf00 msgr2=0x7f2db415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db40ccf00 0x7f2db415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 -- 172.21.15.60:0/989511800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db40cebd0 con 0x7f2db4073fb0 2024-09-16T15:39:03.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.282+0000 7f2db1e00640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db4073fb0 0x7f2db41f58b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2da400c3c0 tx=0x7f2da400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:03.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.283+0000 7f2daaa00640 1 -- 172.21.15.60:0/989511800 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4017070 con 0x7f2db4073fb0 2024-09-16T15:39:03.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.283+0000 7f2daaa00640 1 -- 172.21.15.60:0/989511800 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2da40035f0 con 0x7f2db4073fb0 2024-09-16T15:39:03.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.283+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db415c670 con 0x7f2db4073fb0 2024-09-16T15:39:03.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.283+0000 7f2daaa00640 1 -- 172.21.15.60:0/989511800 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4003a80 con 0x7f2db4073fb0 2024-09-16T15:39:03.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.283+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db40cee70 con 0x7f2db4073fb0 2024-09-16T15:39:03.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.284+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db406f9f0 con 0x7f2db4073fb0 2024-09-16T15:39:03.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.285+0000 7f2daaa00640 1 -- 172.21.15.60:0/989511800 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2da401a070 con 0x7f2db4073fb0 2024-09-16T15:39:03.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.286+0000 7f2daaa00640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2d90078850 0x7f2d9007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:03.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.286+0000 7f2daaa00640 1 -- 172.21.15.60:0/989511800 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2da4099c40 con 0x7f2db4073fb0 2024-09-16T15:39:03.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.286+0000 7f2da9400640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2d90078850 0x7f2d9007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:03.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.287+0000 7f2da9400640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2d90078850 0x7f2d9007ad10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c033000 tx=0x7f2d9c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:03.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.290+0000 7f2daaa00640 1 -- 172.21.15.60:0/989511800 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2da4061bc0 con 0x7f2db4073fb0 2024-09-16T15:39:03.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.523+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2db40ad040 con 0x7f2db4073fb0 2024-09-16T15:39:03.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.524+0000 7f2daaa00640 1 -- 172.21.15.60:0/989511800 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f2da40665f0 con 0x7f2db4073fb0 2024-09-16T15:39:03.526 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2d90078850 msgr2=0x7f2d9007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2d90078850 0x7f2d9007ad10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c033000 tx=0x7f2d9c004730 comp rx=0 tx=0).stop 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db4073fb0 msgr2=0x7f2db41f58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db4073fb0 0x7f2db41f58b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2da400c3c0 tx=0x7f2da400c880 comp rx=0 tx=0).stop 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 shutdown_connections 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2d90078850 0x7f2d9007ad10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2db415f5a0 0x7f2db4156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2db40ccf00 0x7f2db415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 --2- 172.21.15.60:0/989511800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2db4073fb0 0x7f2db41f58b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 >> 172.21.15.60:0/989511800 conn(0x7f2db40b1950 msgr2=0x7f2db40b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 shutdown_connections 2024-09-16T15:39:03.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:03.526+0000 7f2db9000640 1 -- 172.21.15.60:0/989511800 wait complete. 2024-09-16T15:39:03.544 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T15:39:03.544 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:39:03.544 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:39:03.544 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-16T15:39:03.545 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:03.545 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T15:39:03.545 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T15:39:03.545 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:39:03.545 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-16T15:39:03.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:03.545 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:39:03.581 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:03.582 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42705sec preferred_lft 42705sec 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:39:03.583 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:39:03.584 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:39:03.584 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:03.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:03.584 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:39:03.676 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:39:03.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:03.681 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:39:03.716 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:39:03.716 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:39:03.716 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:39:03.716 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:39:03.716 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:39:03.717 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:03.717 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:03.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:39:03.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:39:03.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:39:03.717 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:03.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:03.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:03.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:03.912 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:39:03.946 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:39:03.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:03.947 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-09-16T15:39:04.022 INFO:teuthology.orchestra.run.smithi100.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-16T15:39:04.023 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:04.023 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:04.023 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:39:04.023 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-16T15:39:04.023 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:04.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:04.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:04.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:04.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-16T15:39:04.253 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:04.253 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:04.254 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 62:e0:df:9c:3b:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T15:39:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T15:39:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::60e0:dfff:fe9c:3b50/64 scope link 2024-09-16T15:39:04.254 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:04.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:04.256 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-16T15:39:04.310 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 62:e0:df:9c:3b:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::60e0:dfff:fe9c:3b50/64 scope link 2024-09-16T15:39:04.311 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:04.313 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-16T15:39:04.314 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:04.314 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:04.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-16T15:39:04.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-16T15:39:04.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-16T15:39:04.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-16T15:39:04.314 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-16T15:39:04.314 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:04.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:04.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:04.528 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:04.528 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:04.528 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.1 up 2024-09-16T15:39:04.528 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.1 master ceph-brx 2024-09-16T15:39:04.528 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:04.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:04.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:04.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:04.693 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:04.727 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-16T15:39:04.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:04.728 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:04.802 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-16T15:39:04.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:04.802 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:04.930 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:04.930 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:39:04.958 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:04.958 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:39:04.958 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:39:04.958 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 3@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:39:04.958 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:39:04.960 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-16T15:39:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:39:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:39:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:39:04.961 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:39:04.963 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:39:04.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:04.964 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:05.018 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:05.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:05.107 INFO:teuthology.orchestra.run.smithi100.stdout:client4589 2024-09-16T15:39:05.107 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:05.107 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:05.142 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:05.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:05.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:05.269 INFO:teuthology.orchestra.run.smithi100.stdout:client4589 2024-09-16T15:39:05.270 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:05.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:05.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:05.356 INFO:teuthology.orchestra.run.smithi100.stdout:client4589 2024-09-16T15:39:05.357 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b577f92e-7c09-4d32-a34c-b870818f4d85.client4589/status') 2024-09-16T15:39:05.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:05.454 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.4589 (3)192.168.0.2:0/475854335 2024-09-16T15:39:05.454 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-09-16T15:39:05.455 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-09-16T15:39:05.455 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-09-16T15:39:05.455 INFO:teuthology.orchestra.run.smithi100.stderr:68 bytes copied, 7.3681e-05 s, 923 kB/s 2024-09-16T15:39:05.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:05.465 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-16T15:39:05.548 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-16T15:39:05.548 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:39:05.548 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:05.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.719+0000 7f269ea00640 1 -- 172.21.15.60:0/246624005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f269815f5a0 msgr2=0x7f26980b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:05.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.719+0000 7f269ea00640 1 --2- 172.21.15.60:0/246624005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f269815f5a0 0x7f26980b2440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2688009830 tx=0x7f268802a480 comp rx=0 tx=0).stop 2024-09-16T15:39:05.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.719+0000 7f269ea00640 1 -- 172.21.15.60:0/246624005 shutdown_connections 2024-09-16T15:39:05.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.719+0000 7f269ea00640 1 --2- 172.21.15.60:0/246624005 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26980ccf00 0x7f26980acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:05.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.719+0000 7f269ea00640 1 --2- 172.21.15.60:0/246624005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f269815f5a0 0x7f26980b2440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:05.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.719+0000 7f269ea00640 1 -- 172.21.15.60:0/246624005 >> 172.21.15.60:0/246624005 conn(0x7f26980b1950 msgr2=0x7f26980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:05.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.719+0000 7f269ea00640 1 -- 172.21.15.60:0/246624005 shutdown_connections 2024-09-16T15:39:05.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.719+0000 7f269ea00640 1 -- 172.21.15.60:0/246624005 wait complete. 2024-09-16T15:39:05.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.720+0000 7f269ea00640 1 Processor -- start 2024-09-16T15:39:05.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.721+0000 7f269ea00640 1 -- start start 2024-09-16T15:39:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.721+0000 7f269ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26980ad270 0x7f269814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.721+0000 7f269ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26980ccf00 0x7f26980cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.721+0000 7f269ea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269815f5a0 0x7f26980b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.721+0000 7f269ea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f269806f2e0 con 0x7f26980ccf00 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.721+0000 7f269ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f269806f160 con 0x7f26980ad270 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.721+0000 7f269ea00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f269806f460 con 0x7f269815f5a0 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f268fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26980ccf00 0x7f26980cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f268fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26980ccf00 0x7f26980cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41512/0 (socket says 172.21.15.60:41512) 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f268fe00640 1 -- 172.21.15.60:0/2106741324 learned_addr learned my addr 172.21.15.60:0/2106741324 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f2697e00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26980ad270 0x7f269814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f269ca00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269815f5a0 0x7f26980b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f268fe00640 1 -- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269815f5a0 msgr2=0x7f26980b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f268fe00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269815f5a0 0x7f26980b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f268fe00640 1 -- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26980ad270 msgr2=0x7f269814c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:05.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f268fe00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26980ad270 0x7f269814c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:05.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f268fe00640 1 -- 172.21.15.60:0/2106741324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269814cc70 con 0x7f26980ccf00 2024-09-16T15:39:05.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.722+0000 7f269ca00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269815f5a0 0x7f26980b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:05.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.723+0000 7f268fe00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26980ccf00 0x7f26980cdae0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f268000ea60 tx=0x7f268000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:05.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.723+0000 7f2694c00640 1 -- 172.21.15.60:0/2106741324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268000cde0 con 0x7f26980ccf00 2024-09-16T15:39:05.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.723+0000 7f2694c00640 1 -- 172.21.15.60:0/2106741324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2680004020 con 0x7f26980ccf00 2024-09-16T15:39:05.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.723+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26980ce3e0 con 0x7f26980ccf00 2024-09-16T15:39:05.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.723+0000 7f2694c00640 1 -- 172.21.15.60:0/2106741324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680004450 con 0x7f26980ccf00 2024-09-16T15:39:05.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.723+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26980ce9a0 con 0x7f26980ccf00 2024-09-16T15:39:05.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.724+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26981509b0 con 0x7f26980ccf00 2024-09-16T15:39:05.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.725+0000 7f2694c00640 1 -- 172.21.15.60:0/2106741324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f26800035f0 con 0x7f26980ccf00 2024-09-16T15:39:05.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.726+0000 7f2694c00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2674078880 0x7f267407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.727+0000 7f2694c00640 1 -- 172.21.15.60:0/2106741324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f268009b290 con 0x7f26980ccf00 2024-09-16T15:39:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.727+0000 7f2697e00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2674078880 0x7f267407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.727+0000 7f2697e00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2674078880 0x7f267407ad40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2688034000 tx=0x7f2688004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:05.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.731+0000 7f2694c00640 1 -- 172.21.15.60:0/2106741324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f26981509b0 con 0x7f26980ccf00 2024-09-16T15:39:05.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.930+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26980afdf0 con 0x7f26980ccf00 2024-09-16T15:39:05.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.931+0000 7f2694c00640 1 -- 172.21.15.60:0/2106741324 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7f269814cc70 con 0x7f26980ccf00 2024-09-16T15:39:05.934 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:05.934 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:35.924959+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2674078880 msgr2=0x7f267407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2674078880 0x7f267407ad40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2688034000 tx=0x7f2688004730 comp rx=0 tx=0).stop 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26980ccf00 msgr2=0x7f26980cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26980ccf00 0x7f26980cdae0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f268000ea60 tx=0x7f268000ef20 comp rx=0 tx=0).stop 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 shutdown_connections 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2674078880 0x7f267407ad40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f269815f5a0 0x7f26980b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26980ccf00 0x7f26980cdae0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 --2- 172.21.15.60:0/2106741324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26980ad270 0x7f269814c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:05.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 >> 172.21.15.60:0/2106741324 conn(0x7f26980b1950 msgr2=0x7f269815e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:05.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.934+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 shutdown_connections 2024-09-16T15:39:05.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:05.935+0000 7f269ea00640 1 -- 172.21.15.60:0/2106741324 wait complete. 2024-09-16T15:39:05.955 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T15:38:32.430442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T15:38:33.311774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:39:05.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:39:06.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.119+0000 7fd365600640 1 -- 172.21.15.60:0/4075868498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600b4e70 msgr2=0x7fd3600b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.119+0000 7fd365600640 1 --2- 172.21.15.60:0/4075868498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600b4e70 0x7fd3600b34c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd348009860 tx=0x7fd34802a620 comp rx=0 tx=0).stop 2024-09-16T15:39:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.119+0000 7fd365600640 1 -- 172.21.15.60:0/4075868498 shutdown_connections 2024-09-16T15:39:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.119+0000 7fd365600640 1 --2- 172.21.15.60:0/4075868498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600b4e70 0x7fd3600b34c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.119+0000 7fd365600640 1 -- 172.21.15.60:0/4075868498 >> 172.21.15.60:0/4075868498 conn(0x7fd3600d3e20 msgr2=0x7fd3600d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.119+0000 7fd365600640 1 -- 172.21.15.60:0/4075868498 shutdown_connections 2024-09-16T15:39:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.119+0000 7fd365600640 1 -- 172.21.15.60:0/4075868498 wait complete. 2024-09-16T15:39:06.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.121+0000 7fd365600640 1 Processor -- start 2024-09-16T15:39:06.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.121+0000 7fd365600640 1 -- start start 2024-09-16T15:39:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd365600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3600b3a00 0x7fd36015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd365600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3600b4e70 0x7fd36014f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd365600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600d2000 0x7fd3600ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd365600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600b43c0 con 0x7fd3600b4e70 2024-09-16T15:39:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd365600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600b4240 con 0x7fd3600b3a00 2024-09-16T15:39:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd365600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd3600b1d90 con 0x7fd3600d2000 2024-09-16T15:39:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd357e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3600b4e70 0x7fd36014f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd357e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3600b4e70 0x7fd36014f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41526/0 (socket says 172.21.15.60:41526) 2024-09-16T15:39:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd357e00640 1 -- 172.21.15.60:0/2886685322 learned_addr learned my addr 172.21.15.60:0/2886685322 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd35fe00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3600b3a00 0x7fd36015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.122+0000 7fd364c00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600d2000 0x7fd3600ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd357e00640 1 -- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600d2000 msgr2=0x7fd3600ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd357e00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600d2000 0x7fd3600ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd357e00640 1 -- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3600b3a00 msgr2=0x7fd36015c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd357e00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3600b3a00 0x7fd36015c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd357e00640 1 -- 172.21.15.60:0/2886685322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3601f4670 con 0x7fd3600b4e70 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd35fe00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3600b3a00 0x7fd36015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd364c00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600d2000 0x7fd3600ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd357e00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3600b4e70 0x7fd36014f1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd348009980 tx=0x7fd348032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd35cc00640 1 -- 172.21.15.60:0/2886685322 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348040070 con 0x7fd3600b4e70 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd35cc00640 1 -- 172.21.15.60:0/2886685322 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3480038e0 con 0x7fd3600b4e70 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3601f4950 con 0x7fd3600b4e70 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd35cc00640 1 -- 172.21.15.60:0/2886685322 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348003d30 con 0x7fd3600b4e70 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.123+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36014f700 con 0x7fd3600b4e70 2024-09-16T15:39:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.124+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3600b34c0 con 0x7fd3600b4e70 2024-09-16T15:39:06.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.125+0000 7fd35cc00640 1 -- 172.21.15.60:0/2886685322 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd348039040 con 0x7fd3600b4e70 2024-09-16T15:39:06.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.126+0000 7fd35cc00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd33c078850 0x7fd33c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.126+0000 7fd35cc00640 1 -- 172.21.15.60:0/2886685322 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd3480bbd20 con 0x7fd3600b4e70 2024-09-16T15:39:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.127+0000 7fd35fe00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd33c078850 0x7fd33c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.127+0000 7fd35fe00640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd33c078850 0x7fd33c07ad10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd350001120 tx=0x7fd350009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:06.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.131+0000 7fd35cc00640 1 -- 172.21.15.60:0/2886685322 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd348084380 con 0x7fd3600b4e70 2024-09-16T15:39:06.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.328+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd36006e6b0 con 0x7fd3600b4e70 2024-09-16T15:39:06.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.329+0000 7fd35cc00640 1 -- 172.21.15.60:0/2886685322 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fd348083c80 con 0x7fd3600b4e70 2024-09-16T15:39:06.331 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:39:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd33c078850 msgr2=0x7fd33c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd33c078850 0x7fd33c07ad10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd350001120 tx=0x7fd350009040 comp rx=0 tx=0).stop 2024-09-16T15:39:06.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3600b4e70 msgr2=0x7fd36014f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3600b4e70 0x7fd36014f1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd348009980 tx=0x7fd348032b90 comp rx=0 tx=0).stop 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 shutdown_connections 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd33c078850 0x7fd33c07ad10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3600d2000 0x7fd3600ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3600b4e70 0x7fd36014f1c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 --2- 172.21.15.60:0/2886685322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3600b3a00 0x7fd36015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 >> 172.21.15.60:0/2886685322 conn(0x7fd3600d3e20 msgr2=0x7fd360161660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 shutdown_connections 2024-09-16T15:39:06.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.331+0000 7fd365600640 1 -- 172.21.15.60:0/2886685322 wait complete. 2024-09-16T15:39:06.349 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T15:39:06.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.511+0000 7f203ca00640 1 -- 172.21.15.60:0/571497273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 msgr2=0x7f201c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.511+0000 7f203ca00640 1 --2- 172.21.15.60:0/571497273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 0x7f201c004c80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2024001830 tx=0x7f201c005830 comp rx=0 tx=0).stop 2024-09-16T15:39:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.511+0000 7f2036800640 1 -- 172.21.15.60:0/571497273 reap_dead start 2024-09-16T15:39:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.511+0000 7f203ca00640 1 -- 172.21.15.60:0/571497273 shutdown_connections 2024-09-16T15:39:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.511+0000 7f203ca00640 1 -- 172.21.15.60:0/571497273 >> 172.21.15.60:0/571497273 conn(0x7f20380b1840 msgr2=0x7f20380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.511+0000 7f203ca00640 1 -- 172.21.15.60:0/571497273 shutdown_connections 2024-09-16T15:39:06.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.511+0000 7f203ca00640 1 -- 172.21.15.60:0/571497273 wait complete. 2024-09-16T15:39:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.513+0000 7f203ca00640 1 Processor -- start 2024-09-16T15:39:06.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.513+0000 7f203ca00640 1 -- start start 2024-09-16T15:39:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f203ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 0x7f203813e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f203ca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203813fbe0 0x7f20381f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f203ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20381f1f60 0x7f20381f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f203ca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20380ce2c0 con 0x7f201c004800 2024-09-16T15:39:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f203ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f20380736a0 con 0x7f20381f1f60 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f203ca00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f20380ce440 con 0x7f203813fbe0 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 0x7f203813e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 0x7f203813e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41552/0 (socket says 172.21.15.60:41552) 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035e00640 1 -- 172.21.15.60:0/3409210219 learned_addr learned my addr 172.21.15.60:0/3409210219 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035400640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203813fbe0 0x7f20381f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2036800640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20381f1f60 0x7f20381f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035e00640 1 -- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203813fbe0 msgr2=0x7f20381f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035e00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203813fbe0 0x7f20381f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035e00640 1 -- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20381f1f60 msgr2=0x7f20381f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035e00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20381f1f60 0x7f20381f8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2035e00640 1 -- 172.21.15.60:0/3409210219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20381f23e0 con 0x7f201c004800 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.514+0000 7f2036800640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20381f1f60 0x7f20381f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:06.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.515+0000 7f2035e00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 0x7f203813e460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2028000fd0 tx=0x7f20280039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.515+0000 7f202e000640 1 -- 172.21.15.60:0/3409210219 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028015070 con 0x7f201c004800 2024-09-16T15:39:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.515+0000 7f202e000640 1 -- 172.21.15.60:0/3409210219 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2028011840 con 0x7f201c004800 2024-09-16T15:39:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.515+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203813e9a0 con 0x7f201c004800 2024-09-16T15:39:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.515+0000 7f202e000640 1 -- 172.21.15.60:0/3409210219 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028011c90 con 0x7f201c004800 2024-09-16T15:39:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.515+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20380b10a0 con 0x7f201c004800 2024-09-16T15:39:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.516+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20380b0e90 con 0x7f201c004800 2024-09-16T15:39:06.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.517+0000 7f202e000640 1 -- 172.21.15.60:0/3409210219 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2028018070 con 0x7f201c004800 2024-09-16T15:39:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.519+0000 7f202e000640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2018078850 0x7f201807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.519+0000 7f202e000640 1 -- 172.21.15.60:0/3409210219 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f202809b5c0 con 0x7f201c004800 2024-09-16T15:39:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.519+0000 7f2035400640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2018078850 0x7f201807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.520+0000 7f2035400640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2018078850 0x7f201807ad10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2024032000 tx=0x7f2024001360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:06.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.523+0000 7f202e000640 1 -- 172.21.15.60:0/3409210219 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f20381f23e0 con 0x7f201c004800 2024-09-16T15:39:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.718+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20380ce9d0 con 0x7f201c004800 2024-09-16T15:39:06.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.719+0000 7f202e000640 1 -- 172.21.15.60:0/3409210219 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f2028063550 con 0x7f201c004800 2024-09-16T15:39:06.721 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-09-16T15:39:06.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2018078850 msgr2=0x7f201807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2018078850 0x7f201807ad10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2024032000 tx=0x7f2024001360 comp rx=0 tx=0).stop 2024-09-16T15:39:06.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 msgr2=0x7f203813e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 0x7f203813e460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2028000fd0 tx=0x7f20280039c0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 shutdown_connections 2024-09-16T15:39:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2018078850 0x7f201807ad10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20381f1f60 0x7f20381f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f203813fbe0 0x7f20381f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 --2- 172.21.15.60:0/3409210219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201c004800 0x7f203813e460 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 >> 172.21.15.60:0/3409210219 conn(0x7f20380b1840 msgr2=0x7f20380aa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 shutdown_connections 2024-09-16T15:39:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.721+0000 7f203ca00640 1 -- 172.21.15.60:0/3409210219 wait complete. 2024-09-16T15:39:06.740 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T15:39:06.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 -- 172.21.15.60:0/2639878404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5340b4f40 msgr2=0x7fd5340ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 --2- 172.21.15.60:0/2639878404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5340b4f40 0x7fd5340ce820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5340d1970 tx=0x7fd52c02a980 comp rx=0 tx=0).stop 2024-09-16T15:39:06.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 -- 172.21.15.60:0/2639878404 shutdown_connections 2024-09-16T15:39:06.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 --2- 172.21.15.60:0/2639878404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5340b4f40 0x7fd5340ce820 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 --2- 172.21.15.60:0/2639878404 >> v2:172.21.15.100:3300/0 conn(0x7fd5340b5480 0x7fd5340d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 --2- 172.21.15.60:0/2639878404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd534161460 0x7fd5340d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 -- 172.21.15.60:0/2639878404 >> 172.21.15.60:0/2639878404 conn(0x7fd53409d0b0 msgr2=0x7fd5340d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:06.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 -- 172.21.15.60:0/2639878404 shutdown_connections 2024-09-16T15:39:06.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.902+0000 7fd53a600640 1 -- 172.21.15.60:0/2639878404 wait complete. 2024-09-16T15:39:06.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.904+0000 7fd53a600640 1 Processor -- start 2024-09-16T15:39:06.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.904+0000 7fd53a600640 1 -- start start 2024-09-16T15:39:06.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd53a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5340b4f40 0x7fd53409fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd53a600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5340b5480 0x7fd5341f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd53a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd534161460 0x7fd53409db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd53a600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd5340cf0f0 con 0x7fd534161460 2024-09-16T15:39:06.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd53a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5340cef70 con 0x7fd5340b4f40 2024-09-16T15:39:06.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd53a600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd5340cf270 con 0x7fd5340b5480 2024-09-16T15:39:06.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd534161460 0x7fd53409db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd534161460 0x7fd53409db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41582/0 (socket says 172.21.15.60:41582) 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539c00640 1 -- 172.21.15.60:0/1976640466 learned_addr learned my addr 172.21.15.60:0/1976640466 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539200640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5340b4f40 0x7fd53409fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd533e00640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5340b5480 0x7fd5341f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539c00640 1 -- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5340b5480 msgr2=0x7fd5341f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539c00640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5340b5480 0x7fd5341f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539c00640 1 -- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5340b4f40 msgr2=0x7fd53409fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539c00640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5340b4f40 0x7fd53409fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.905+0000 7fd539c00640 1 -- 172.21.15.60:0/1976640466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd534151cd0 con 0x7fd534161460 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.906+0000 7fd539c00640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd534161460 0x7fd53409db90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009d60 tx=0x7fd52c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.906+0000 7fd530c00640 1 -- 172.21.15.60:0/1976640466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c034050 con 0x7fd534161460 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.906+0000 7fd530c00640 1 -- 172.21.15.60:0/1976640466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd52c004450 con 0x7fd534161460 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.906+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5341519f0 con 0x7fd534161460 2024-09-16T15:39:06.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.906+0000 7fd530c00640 1 -- 172.21.15.60:0/1976640466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c043440 con 0x7fd534161460 2024-09-16T15:39:06.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.906+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5340b16a0 con 0x7fd534161460 2024-09-16T15:39:06.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.907+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5340d1d10 con 0x7fd534161460 2024-09-16T15:39:06.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.908+0000 7fd530c00640 1 -- 172.21.15.60:0/1976640466 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd52c037020 con 0x7fd534161460 2024-09-16T15:39:06.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.909+0000 7fd530c00640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd514078850 0x7fd51407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:06.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.909+0000 7fd530c00640 1 -- 172.21.15.60:0/1976640466 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd52c0bbca0 con 0x7fd534161460 2024-09-16T15:39:06.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.910+0000 7fd539200640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd514078850 0x7fd51407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:06.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.910+0000 7fd539200640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd514078850 0x7fd51407ad10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd5341f7b90 tx=0x7fd528005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:06.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:06.913+0000 7fd530c00640 1 -- 172.21.15.60:0/1976640466 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd5340d1d10 con 0x7fd534161460 2024-09-16T15:39:07.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.112+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd5340d2ee0 con 0x7fd534161460 2024-09-16T15:39:07.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.112+0000 7fd530c00640 1 -- 172.21.15.60:0/1976640466 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fd5340d2ee0 con 0x7fd534161460 2024-09-16T15:39:07.114 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-09-16T15:39:07.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.114+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd514078850 msgr2=0x7fd51407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:07.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.114+0000 7fd53a600640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd514078850 0x7fd51407ad10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd5341f7b90 tx=0x7fd528005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:07.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.114+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd534161460 msgr2=0x7fd53409db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:07.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.114+0000 7fd53a600640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd534161460 0x7fd53409db90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009d60 tx=0x7fd52c003f10 comp rx=0 tx=0).stop 2024-09-16T15:39:07.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.115+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 shutdown_connections 2024-09-16T15:39:07.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.115+0000 7fd53a600640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd514078850 0x7fd51407ad10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:07.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.115+0000 7fd53a600640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd534161460 0x7fd53409db90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:07.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.115+0000 7fd53a600640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5340b5480 0x7fd5341f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:07.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.115+0000 7fd53a600640 1 --2- 172.21.15.60:0/1976640466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5340b4f40 0x7fd53409fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:07.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.115+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 >> 172.21.15.60:0/1976640466 conn(0x7fd53409d0b0 msgr2=0x7fd53406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:07.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.115+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 shutdown_connections 2024-09-16T15:39:07.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.115+0000 7fd53a600640 1 -- 172.21.15.60:0/1976640466 wait complete. 2024-09-16T15:39:07.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-16T15:39:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.292+0000 7f67bb600640 1 -- 172.21.15.60:0/992724608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d1b70 msgr2=0x7f67b40d2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.292+0000 7f67bb600640 1 --2- 172.21.15.60:0/992724608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d1b70 0x7f67b40d2560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67b4069740 tx=0x7f679c02a540 comp rx=0 tx=0).stop 2024-09-16T15:39:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.292+0000 7f67bb600640 1 -- 172.21.15.60:0/992724608 shutdown_connections 2024-09-16T15:39:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.292+0000 7f67bb600640 1 --2- 172.21.15.60:0/992724608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d1b70 0x7f67b40d2560 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.292+0000 7f67bb600640 1 --2- 172.21.15.60:0/992724608 >> v2:172.21.15.100:3300/0 conn(0x7f67b415e1b0 0x7f67b40d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:07.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.292+0000 7f67bb600640 1 -- 172.21.15.60:0/992724608 >> 172.21.15.60:0/992724608 conn(0x7f67b40d3fa0 msgr2=0x7f67b40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.292+0000 7f67bb600640 1 -- 172.21.15.60:0/992724608 shutdown_connections 2024-09-16T15:39:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.292+0000 7f67bb600640 1 -- 172.21.15.60:0/992724608 wait complete. 2024-09-16T15:39:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.293+0000 7f67bb600640 1 Processor -- start 2024-09-16T15:39:07.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.294+0000 7f67bb600640 1 -- start start 2024-09-16T15:39:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.294+0000 7f67bb600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b40d1b70 0x7f67b41572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67bb600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d2aa0 0x7f67b41f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67bb600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b415e1b0 0x7f67b4160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67bb600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f67b40d0e60 con 0x7f67b40d2aa0 2024-09-16T15:39:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67bb600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f67b40b8940 con 0x7f67b415e1b0 2024-09-16T15:39:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67bb600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f67b40d0fe0 con 0x7f67b40d1b70 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d2aa0 0x7f67b41f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d2aa0 0x7f67b41f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41598/0 (socket says 172.21.15.60:41598) 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67abe00640 1 -- 172.21.15.60:0/3795822123 learned_addr learned my addr 172.21.15.60:0/3795822123 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67b8c00640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b40d1b70 0x7f67b41572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67abe00640 1 -- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b40d1b70 msgr2=0x7f67b41572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67b9600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b415e1b0 0x7f67b4160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67abe00640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b40d1b70 0x7f67b41572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67abe00640 1 -- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b415e1b0 msgr2=0x7f67b4160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67abe00640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b415e1b0 0x7f67b4160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67abe00640 1 -- 172.21.15.60:0/3795822123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b41579b0 con 0x7f67b40d2aa0 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67b8c00640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b40d1b70 0x7f67b41572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.295+0000 7f67b9600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b415e1b0 0x7f67b4160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.296+0000 7f67abe00640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d2aa0 0x7f67b41f4e80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f679c009780 tx=0x7f679c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.296+0000 7f67b1600640 1 -- 172.21.15.60:0/3795822123 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c040070 con 0x7f67b40d2aa0 2024-09-16T15:39:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.296+0000 7f67b1600640 1 -- 172.21.15.60:0/3795822123 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f679c005310 con 0x7f67b40d2aa0 2024-09-16T15:39:07.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.296+0000 7f67b1600640 1 -- 172.21.15.60:0/3795822123 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c0047a0 con 0x7f67b40d2aa0 2024-09-16T15:39:07.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.296+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b4157c90 con 0x7f67b40d2aa0 2024-09-16T15:39:07.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.296+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b41f4960 con 0x7f67b40d2aa0 2024-09-16T15:39:07.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.297+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b40d1490 con 0x7f67b40d2aa0 2024-09-16T15:39:07.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.298+0000 7f67b1600640 1 -- 172.21.15.60:0/3795822123 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f679c039040 con 0x7f67b40d2aa0 2024-09-16T15:39:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.299+0000 7f67b1600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6790078850 0x7f679007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.299+0000 7f67b1600640 1 -- 172.21.15.60:0/3795822123 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f679c0bbcf0 con 0x7f67b40d2aa0 2024-09-16T15:39:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.300+0000 7f67b8c00640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6790078850 0x7f679007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.300+0000 7f67b8c00640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6790078850 0x7f679007ad10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009ab0 tx=0x7f67a4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:07.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.303+0000 7f67b1600640 1 -- 172.21.15.60:0/3795822123 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f679c0842c0 con 0x7f67b40d2aa0 2024-09-16T15:39:07.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:07.570+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f67b406f260 con 0x7f67b40d2aa0 2024-09-16T15:39:08.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.043+0000 7f67b1600640 1 -- 172.21.15.60:0/3795822123 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v56) ==== 161+0+0 (secure 0 0 0) 0x7f679c083bc0 con 0x7f67b40d2aa0 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6790078850 msgr2=0x7f679007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6790078850 0x7f679007ad10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009ab0 tx=0x7f67a4005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d2aa0 msgr2=0x7f67b41f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d2aa0 0x7f67b41f4e80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f679c009780 tx=0x7f679c032c10 comp rx=0 tx=0).stop 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 shutdown_connections 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6790078850 0x7f679007ad10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67b415e1b0 0x7f67b4160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67b40d2aa0 0x7f67b41f4e80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.046+0000 7f67bb600640 1 --2- 172.21.15.60:0/3795822123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67b40d1b70 0x7f67b41572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.047+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 >> 172.21.15.60:0/3795822123 conn(0x7f67b40d3fa0 msgr2=0x7f67b415c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.047+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 shutdown_connections 2024-09-16T15:39:08.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.047+0000 7f67bb600640 1 -- 172.21.15.60:0/3795822123 wait complete. 2024-09-16T15:39:08.067 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:39:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.229+0000 7f1684c00640 1 -- 172.21.15.60:0/2002959541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1680073fb0 msgr2=0x7f1680074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.229+0000 7f1684c00640 1 --2- 172.21.15.60:0/2002959541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1680073fb0 0x7f1680074440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1678005fd0 tx=0x7f167802aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.229+0000 7f1684c00640 1 -- 172.21.15.60:0/2002959541 shutdown_connections 2024-09-16T15:39:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.229+0000 7f1684c00640 1 --2- 172.21.15.60:0/2002959541 >> v2:172.21.15.100:3300/0 conn(0x7f16800ccf00 0x7f16800b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.229+0000 7f1684c00640 1 --2- 172.21.15.60:0/2002959541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1680073fb0 0x7f1680074440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.229+0000 7f1684c00640 1 -- 172.21.15.60:0/2002959541 >> 172.21.15.60:0/2002959541 conn(0x7f16800b1950 msgr2=0x7f16800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.229+0000 7f1684c00640 1 -- 172.21.15.60:0/2002959541 shutdown_connections 2024-09-16T15:39:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.229+0000 7f1684c00640 1 -- 172.21.15.60:0/2002959541 wait complete. 2024-09-16T15:39:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.230+0000 7f1684c00640 1 Processor -- start 2024-09-16T15:39:08.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.231+0000 7f1684c00640 1 -- start start 2024-09-16T15:39:08.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.231+0000 7f1684c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680073fb0 0x7f16801f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.231+0000 7f1684c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16800ccf00 0x7f16800cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.231+0000 7f1684c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168015f5a0 0x7f16800d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.231+0000 7f1684c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f168006f2e0 con 0x7f168015f5a0 2024-09-16T15:39:08.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.231+0000 7f1684c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f168006f160 con 0x7f1680073fb0 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.231+0000 7f1684c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f168006f460 con 0x7f16800ccf00 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168015f5a0 0x7f16800d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168015f5a0 0x7f16800d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41614/0 (socket says 172.21.15.60:41614) 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167e800640 1 -- 172.21.15.60:0/2263673038 learned_addr learned my addr 172.21.15.60:0/2263673038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167de00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680073fb0 0x7f16801f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167d400640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16800ccf00 0x7f16800cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167e800640 1 -- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16800ccf00 msgr2=0x7f16800cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167e800640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16800ccf00 0x7f16800cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167e800640 1 -- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680073fb0 msgr2=0x7f16801f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167e800640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680073fb0 0x7f16801f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.232+0000 7f167e800640 1 -- 172.21.15.60:0/2263673038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16801f5910 con 0x7f168015f5a0 2024-09-16T15:39:08.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.233+0000 7f167e800640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168015f5a0 0x7f16800d01c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1678006010 tx=0x7f1678003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.233+0000 7f1676000640 1 -- 172.21.15.60:0/2263673038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678047070 con 0x7f168015f5a0 2024-09-16T15:39:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.233+0000 7f1676000640 1 -- 172.21.15.60:0/2263673038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f167802f670 con 0x7f168015f5a0 2024-09-16T15:39:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.233+0000 7f1676000640 1 -- 172.21.15.60:0/2263673038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1678033050 con 0x7f168015f5a0 2024-09-16T15:39:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.233+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16801f5630 con 0x7f168015f5a0 2024-09-16T15:39:08.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.233+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16800cded0 con 0x7f168015f5a0 2024-09-16T15:39:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.234+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f168006f980 con 0x7f168015f5a0 2024-09-16T15:39:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.235+0000 7f1676000640 1 -- 172.21.15.60:0/2263673038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f167804a070 con 0x7f168015f5a0 2024-09-16T15:39:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.236+0000 7f1676000640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1660078850 0x7f166007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.236+0000 7f1676000640 1 -- 172.21.15.60:0/2263673038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f16780bbb20 con 0x7f168015f5a0 2024-09-16T15:39:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.237+0000 7f167de00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1660078850 0x7f166007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.237+0000 7f167de00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1660078850 0x7f166007ad10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1670001720 tx=0x7f1670009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:08.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.240+0000 7f1676000640 1 -- 172.21.15.60:0/2263673038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1678083aa0 con 0x7f168015f5a0 2024-09-16T15:39:08.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.435+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f168015dac0 con 0x7f168015f5a0 2024-09-16T15:39:08.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.435+0000 7f1676000640 1 -- 172.21.15.60:0/2263673038 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f16780884d0 con 0x7f168015f5a0 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:39:08.438 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:39:08.439 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:39:08.440 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:39:08.440 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1660078850 msgr2=0x7f166007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1660078850 0x7f166007ad10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1670001720 tx=0x7f1670009040 comp rx=0 tx=0).stop 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168015f5a0 msgr2=0x7f16800d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168015f5a0 0x7f16800d01c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1678006010 tx=0x7f1678003ef0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 shutdown_connections 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1660078850 0x7f166007ad10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f168015f5a0 0x7f16800d01c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16800ccf00 0x7f16800cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 --2- 172.21.15.60:0/2263673038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1680073fb0 0x7f16801f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 >> 172.21.15.60:0/2263673038 conn(0x7f16800b1950 msgr2=0x7f16800ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 shutdown_connections 2024-09-16T15:39:08.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.438+0000 7f1684c00640 1 -- 172.21.15.60:0/2263673038 wait complete. 2024-09-16T15:39:08.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:39:08.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.617+0000 7f8bdb200640 1 -- 172.21.15.60:0/1910294857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 msgr2=0x7f8bdc073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.617+0000 7f8bdb200640 1 --2- 172.21.15.60:0/1910294857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 0x7f8bdc073ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0c9ff0 tx=0x7f8bd402aa30 comp rx=0 tx=0).stop 2024-09-16T15:39:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.617+0000 7f8bdb200640 1 -- 172.21.15.60:0/1910294857 shutdown_connections 2024-09-16T15:39:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.617+0000 7f8bdb200640 1 --2- 172.21.15.60:0/1910294857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 0x7f8bdc073ef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.617+0000 7f8bdb200640 1 --2- 172.21.15.60:0/1910294857 >> v2:172.21.15.100:3300/0 conn(0x7f8bdc15bec0 0x7f8bdc06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.617+0000 7f8bdb200640 1 -- 172.21.15.60:0/1910294857 >> 172.21.15.60:0/1910294857 conn(0x7f8bdc0cf9f0 msgr2=0x7f8bdc0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.617+0000 7f8bdb200640 1 -- 172.21.15.60:0/1910294857 shutdown_connections 2024-09-16T15:39:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.617+0000 7f8bdb200640 1 -- 172.21.15.60:0/1910294857 wait complete. 2024-09-16T15:39:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.619+0000 7f8bdb200640 1 Processor -- start 2024-09-16T15:39:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.619+0000 7f8bdb200640 1 -- start start 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bdb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc072560 0x7f8bdc155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bdb200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 0x7f8bdc154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bdb200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bdc15bec0 0x7f8bdc1f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bdb200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc0ae930 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bdb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc0ae7b0 con 0x7f8bdc072560 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bdb200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc0aeab0 con 0x7f8bdc15bec0 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bd9400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 0x7f8bdc154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bd9400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 0x7f8bdc154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41640/0 (socket says 172.21.15.60:41640) 2024-09-16T15:39:08.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bd9400640 1 -- 172.21.15.60:0/2957875249 learned_addr learned my addr 172.21.15.60:0/2957875249 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bd9e00640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc072560 0x7f8bdc155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.620+0000 7f8bda800640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bdc15bec0 0x7f8bdc1f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd9400640 1 -- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bdc15bec0 msgr2=0x7f8bdc1f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd9400640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bdc15bec0 0x7f8bdc1f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd9400640 1 -- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc072560 msgr2=0x7f8bdc155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd9400640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc072560 0x7f8bdc155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd9400640 1 -- 172.21.15.60:0/2957875249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc155770 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd9400640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 0x7f8bdc154160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c440 tx=0x7f8bc800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:08.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd2000640 1 -- 172.21.15.60:0/2957875249 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8017070 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd2000640 1 -- 172.21.15.60:0/2957875249 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bc80035f0 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bd2000640 1 -- 172.21.15.60:0/2957875249 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8003a40 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc155a50 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.621+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc1f7fc0 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.622+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bdc072c70 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.624+0000 7f8bd2000640 1 -- 172.21.15.60:0/2957875249 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8bc8003be0 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.625+0000 7f8bd2000640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8bbc078850 0x7f8bbc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.625+0000 7f8bd2000640 1 -- 172.21.15.60:0/2957875249 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8bc8099b00 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.626+0000 7f8bd9e00640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8bbc078850 0x7f8bbc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.626+0000 7f8bd9e00640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8bbc078850 0x7f8bbc07ad10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0098e0 tx=0x7f8bcc009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:08.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.630+0000 7f8bd2000640 1 -- 172.21.15.60:0/2957875249 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8bdc072c70 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.828+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8bdc0aefe0 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.831+0000 7f8bd2000640 1 -- 172.21.15.60:0/2957875249 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f8bc8099e20 con 0x7f8bdc0cbc00 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8bbc078850 msgr2=0x7f8bbc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8bbc078850 0x7f8bbc07ad10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0098e0 tx=0x7f8bcc009060 comp rx=0 tx=0).stop 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 msgr2=0x7f8bdc154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 0x7f8bdc154160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800c440 tx=0x7f8bc800c900 comp rx=0 tx=0).stop 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 shutdown_connections 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8bbc078850 0x7f8bbc07ad10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bdc15bec0 0x7f8bdc1f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bdc0cbc00 0x7f8bdc154160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 --2- 172.21.15.60:0/2957875249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc072560 0x7f8bdc155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 >> 172.21.15.60:0/2957875249 conn(0x7f8bdc0cf9f0 msgr2=0x7f8bdc06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:08.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 shutdown_connections 2024-09-16T15:39:08.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:08.834+0000 7f8bdb200640 1 -- 172.21.15.60:0/2957875249 wait complete. 2024-09-16T15:39:08.852 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:39:08.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:39:08.936 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:39:09.032 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:39:09.032 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:09.032 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:09.068 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:09.068 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:09.143 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:09.143 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:09.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:09.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.417+0000 7f4a85000640 1 -- 172.21.15.60:0/2705378665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a80072df0 msgr2=0x7f4a80073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:09.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.417+0000 7f4a85000640 1 --2- 172.21.15.60:0/2705378665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a80072df0 0x7f4a80073240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a800b4890 tx=0x7f4a7802a820 comp rx=0 tx=0).stop 2024-09-16T15:39:09.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.418+0000 7f4a85000640 1 -- 172.21.15.60:0/2705378665 shutdown_connections 2024-09-16T15:39:09.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.418+0000 7f4a85000640 1 --2- 172.21.15.60:0/2705378665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a80072df0 0x7f4a80073240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.418+0000 7f4a85000640 1 --2- 172.21.15.60:0/2705378665 >> v2:172.21.15.100:3300/0 conn(0x7f4a800af3d0 0x7f4a8006f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.418+0000 7f4a85000640 1 -- 172.21.15.60:0/2705378665 >> 172.21.15.60:0/2705378665 conn(0x7f4a800b1950 msgr2=0x7f4a800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.418+0000 7f4a85000640 1 -- 172.21.15.60:0/2705378665 shutdown_connections 2024-09-16T15:39:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.418+0000 7f4a85000640 1 -- 172.21.15.60:0/2705378665 wait complete. 2024-09-16T15:39:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.419+0000 7f4a85000640 1 Processor -- start 2024-09-16T15:39:09.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.419+0000 7f4a85000640 1 -- start start 2024-09-16T15:39:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.420+0000 7f4a85000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a80072df0 0x7f4a80154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.420+0000 7f4a85000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a800af3d0 0x7f4a8009bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:09.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.420+0000 7f4a85000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a80161550 0x7f4a800d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.420+0000 7f4a85000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80073780 con 0x7f4a800af3d0 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.420+0000 7f4a85000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8006fdb0 con 0x7f4a80072df0 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.420+0000 7f4a85000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4a80073900 con 0x7f4a80161550 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a75400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a800af3d0 0x7f4a8009bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a75400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a800af3d0 0x7f4a8009bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57044/0 (socket says 172.21.15.60:57044) 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a75400640 1 -- 172.21.15.60:0/533476499 learned_addr learned my addr 172.21.15.60:0/533476499 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a7de00640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a80072df0 0x7f4a80154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a7e800640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a80161550 0x7f4a800d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a75400640 1 -- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a80161550 msgr2=0x7f4a800d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a75400640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a80161550 0x7f4a800d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a75400640 1 -- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a80072df0 msgr2=0x7f4a80154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a75400640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a80072df0 0x7f4a80154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.421+0000 7f4a75400640 1 -- 172.21.15.60:0/533476499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8009c260 con 0x7f4a800af3d0 2024-09-16T15:39:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.422+0000 7f4a75400640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a800af3d0 0x7f4a8009bd20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800e9f0 tx=0x7f4a6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.422+0000 7f4a76a00640 1 -- 172.21.15.60:0/533476499 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6800cd80 con 0x7f4a800af3d0 2024-09-16T15:39:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.422+0000 7f4a76a00640 1 -- 172.21.15.60:0/533476499 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a68010430 con 0x7f4a800af3d0 2024-09-16T15:39:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.422+0000 7f4a76a00640 1 -- 172.21.15.60:0/533476499 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68010840 con 0x7f4a800af3d0 2024-09-16T15:39:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.422+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a801f4880 con 0x7f4a800af3d0 2024-09-16T15:39:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.422+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a801f4e40 con 0x7f4a800af3d0 2024-09-16T15:39:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.423+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a80073df0 con 0x7f4a800af3d0 2024-09-16T15:39:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.424+0000 7f4a76a00640 1 -- 172.21.15.60:0/533476499 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4a680109e0 con 0x7f4a800af3d0 2024-09-16T15:39:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.426+0000 7f4a76a00640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a5c078850 0x7f4a5c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.426+0000 7f4a76a00640 1 -- 172.21.15.60:0/533476499 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4a6809b410 con 0x7f4a800af3d0 2024-09-16T15:39:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.426+0000 7f4a7de00640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a5c078850 0x7f4a5c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.427+0000 7f4a7de00640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a5c078850 0x7f4a5c07ad10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a70007fd0 tx=0x7f4a70001370 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:09.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.430+0000 7f4a76a00640 1 -- 172.21.15.60:0/533476499 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4a680634e0 con 0x7f4a800af3d0 2024-09-16T15:39:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.637+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4a801579a0 con 0x7f4a800af3d0 2024-09-16T15:39:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.638+0000 7f4a76a00640 1 -- 172.21.15.60:0/533476499 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f4a68067f10 con 0x7f4a800af3d0 2024-09-16T15:39:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:39:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a5c078850 msgr2=0x7f4a5c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:09.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a5c078850 0x7f4a5c07ad10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4a70007fd0 tx=0x7f4a70001370 comp rx=0 tx=0).stop 2024-09-16T15:39:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a800af3d0 msgr2=0x7f4a8009bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a800af3d0 0x7f4a8009bd20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800e9f0 tx=0x7f4a6800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 shutdown_connections 2024-09-16T15:39:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a5c078850 0x7f4a5c07ad10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a80161550 0x7f4a800d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a800af3d0 0x7f4a8009bd20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 --2- 172.21.15.60:0/533476499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a80072df0 0x7f4a80154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 >> 172.21.15.60:0/533476499 conn(0x7f4a800b1950 msgr2=0x7f4a800b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:09.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 shutdown_connections 2024-09-16T15:39:09.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:09.640+0000 7f4a85000640 1 -- 172.21.15.60:0/533476499 wait complete. 2024-09-16T15:39:09.658 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:39:09.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:09.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-16T15:39:09.721 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:39:09.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:09.721 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:39:09.763 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:09.763 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:09.803 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:09.803 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:09.881 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:09.881 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:09.957 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.120+0000 7fa902000640 1 -- 172.21.15.60:0/3527762035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8fc06ff60 msgr2=0x7fa8fc0b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.120+0000 7fa902000640 1 --2- 172.21.15.60:0/3527762035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8fc06ff60 0x7fa8fc0b4610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009be0 tx=0x7fa8f402a920 comp rx=0 tx=0).stop 2024-09-16T15:39:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.120+0000 7fa902000640 1 -- 172.21.15.60:0/3527762035 shutdown_connections 2024-09-16T15:39:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.120+0000 7fa902000640 1 --2- 172.21.15.60:0/3527762035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8fc06ff60 0x7fa8fc0b4610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.120+0000 7fa902000640 1 -- 172.21.15.60:0/3527762035 >> 172.21.15.60:0/3527762035 conn(0x7fa8fc0b1950 msgr2=0x7fa8fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.120+0000 7fa902000640 1 -- 172.21.15.60:0/3527762035 shutdown_connections 2024-09-16T15:39:10.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.120+0000 7fa902000640 1 -- 172.21.15.60:0/3527762035 wait complete. 2024-09-16T15:39:10.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.122+0000 7fa902000640 1 Processor -- start 2024-09-16T15:39:10.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.122+0000 7fa902000640 1 -- start start 2024-09-16T15:39:10.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa902000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8fc06ff60 0x7fa8fc1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:10.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa902000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8fc070920 0x7fa8fc09d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:10.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa902000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8fc1438b0 0x7fa8fc1eeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa902000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc0b4e40 con 0x7fa8fc06ff60 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa902000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc0cc5f0 con 0x7fa8fc1438b0 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa902000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa8fc1f6140 con 0x7fa8fc070920 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa8fb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8fc06ff60 0x7fa8fc1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa8fb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8fc06ff60 0x7fa8fc1f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57052/0 (socket says 172.21.15.60:57052) 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa8fb400640 1 -- 172.21.15.60:0/1084708116 learned_addr learned my addr 172.21.15.60:0/1084708116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa8fbe00640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8fc1438b0 0x7fa8fc1eeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.123+0000 7fa8faa00640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8fc070920 0x7fa8fc09d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8fb400640 1 -- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8fc070920 msgr2=0x7fa8fc09d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8fb400640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8fc070920 0x7fa8fc09d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8fb400640 1 -- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8fc1438b0 msgr2=0x7fa8fc1eeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8fb400640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8fc1438b0 0x7fa8fc1eeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8fb400640 1 -- 172.21.15.60:0/1084708116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8fc09b560 con 0x7fa8fc06ff60 2024-09-16T15:39:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8fb400640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8fc06ff60 0x7fa8fc1f5c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c420 tx=0x7fa8ec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8f3400640 1 -- 172.21.15.60:0/1084708116 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec017070 con 0x7fa8fc06ff60 2024-09-16T15:39:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8f3400640 1 -- 172.21.15.60:0/1084708116 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8ec0053c0 con 0x7fa8fc06ff60 2024-09-16T15:39:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa8f3400640 1 -- 172.21.15.60:0/1084708116 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec003850 con 0x7fa8fc06ff60 2024-09-16T15:39:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8fc09b840 con 0x7fa8fc06ff60 2024-09-16T15:39:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.124+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8fc09bb20 con 0x7fa8fc06ff60 2024-09-16T15:39:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.126+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8fc141bf0 con 0x7fa8fc06ff60 2024-09-16T15:39:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.127+0000 7fa8f3400640 1 -- 172.21.15.60:0/1084708116 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa8ec01a070 con 0x7fa8fc06ff60 2024-09-16T15:39:10.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.127+0000 7fa8f3400640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa8dc078850 0x7fa8dc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:10.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.128+0000 7fa8f3400640 1 -- 172.21.15.60:0/1084708116 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa8ec099cc0 con 0x7fa8fc06ff60 2024-09-16T15:39:10.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.128+0000 7fa8faa00640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa8dc078850 0x7fa8dc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:10.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.129+0000 7fa8faa00640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa8dc078850 0x7fa8dc07ad10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8007e60 tx=0x7fa8e8007df0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:10.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.132+0000 7fa8f3400640 1 -- 172.21.15.60:0/1084708116 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8ec0a0080 con 0x7fa8fc06ff60 2024-09-16T15:39:10.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.339+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa8fc0ccaa0 con 0x7fa8fc06ff60 2024-09-16T15:39:10.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.339+0000 7fa8f3400640 1 -- 172.21.15.60:0/1084708116 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fa8ec061c40 con 0x7fa8fc06ff60 2024-09-16T15:39:10.342 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:39:10.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa8dc078850 msgr2=0x7fa8dc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:10.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa8dc078850 0x7fa8dc07ad10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8007e60 tx=0x7fa8e8007df0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8fc06ff60 msgr2=0x7fa8fc1f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8fc06ff60 0x7fa8fc1f5c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c420 tx=0x7fa8ec00c8e0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 shutdown_connections 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa8dc078850 0x7fa8dc07ad10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8fc1438b0 0x7fa8fc1eeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8fc070920 0x7fa8fc09d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 --2- 172.21.15.60:0/1084708116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8fc06ff60 0x7fa8fc1f5c00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 >> 172.21.15.60:0/1084708116 conn(0x7fa8fc0b1950 msgr2=0x7fa8fc075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 shutdown_connections 2024-09-16T15:39:10.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.342+0000 7fa902000640 1 -- 172.21.15.60:0/1084708116 wait complete. 2024-09-16T15:39:10.360 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:39:10.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:10.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-16T15:39:10.437 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:39:10.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:10.437 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:39:10.478 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:10.478 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:10.514 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:10.514 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:10.589 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:10.589 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:10.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:10.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.864+0000 7fe074e00640 1 -- 172.21.15.60:0/312559445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b5370 msgr2=0x7fe0700b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:10.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.864+0000 7fe074e00640 1 --2- 172.21.15.60:0/312559445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b5370 0x7fe0700b3bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe0700d4ea0 tx=0x7fe05c02a510 comp rx=0 tx=0).stop 2024-09-16T15:39:10.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.864+0000 7fe074e00640 1 -- 172.21.15.60:0/312559445 shutdown_connections 2024-09-16T15:39:10.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.864+0000 7fe074e00640 1 --2- 172.21.15.60:0/312559445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b5370 0x7fe0700b3bc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.864+0000 7fe074e00640 1 -- 172.21.15.60:0/312559445 >> 172.21.15.60:0/312559445 conn(0x7fe0700d3df0 msgr2=0x7fe0700d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:10.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.864+0000 7fe074e00640 1 -- 172.21.15.60:0/312559445 shutdown_connections 2024-09-16T15:39:10.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.864+0000 7fe074e00640 1 -- 172.21.15.60:0/312559445 wait complete. 2024-09-16T15:39:10.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.865+0000 7fe074e00640 1 Processor -- start 2024-09-16T15:39:10.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.865+0000 7fe074e00640 1 -- start start 2024-09-16T15:39:10.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.866+0000 7fe074e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b4100 0x7fe0700b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:10.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.866+0000 7fe074e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0700b5370 0x7fe0701f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:10.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.866+0000 7fe074e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161440 0x7fe0700c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:10.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.866+0000 7fe074e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe0700b2210 con 0x7fe0700b4100 2024-09-16T15:39:10.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.866+0000 7fe074e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0700b2090 con 0x7fe0700b5370 2024-09-16T15:39:10.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.866+0000 7fe074e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe0700b2390 con 0x7fe070161440 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.866+0000 7fe06f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b4100 0x7fe0700b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b4100 0x7fe0700b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57072/0 (socket says 172.21.15.60:57072) 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06f400640 1 -- 172.21.15.60:0/807227646 learned_addr learned my addr 172.21.15.60:0/807227646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06ea00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0700b5370 0x7fe0701f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06fe00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161440 0x7fe0700c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06f400640 1 -- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161440 msgr2=0x7fe0700c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06f400640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161440 0x7fe0700c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06f400640 1 -- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0700b5370 msgr2=0x7fe0701f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06f400640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0700b5370 0x7fe0701f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06f400640 1 -- 172.21.15.60:0/807227646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0700b8950 con 0x7fe0700b4100 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06fe00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161440 0x7fe0700c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:10.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe06f400640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b4100 0x7fe0700b8410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c400 tx=0x7fe06000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:10.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.867+0000 7fe067400640 1 -- 172.21.15.60:0/807227646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060011790 con 0x7fe0700b4100 2024-09-16T15:39:10.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.868+0000 7fe067400640 1 -- 172.21.15.60:0/807227646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe060017070 con 0x7fe0700b4100 2024-09-16T15:39:10.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.868+0000 7fe067400640 1 -- 172.21.15.60:0/807227646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060011ce0 con 0x7fe0700b4100 2024-09-16T15:39:10.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.868+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0701f5790 con 0x7fe0700b4100 2024-09-16T15:39:10.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.868+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0700c7e10 con 0x7fe0700b4100 2024-09-16T15:39:10.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.869+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0700b2840 con 0x7fe0700b4100 2024-09-16T15:39:10.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.870+0000 7fe067400640 1 -- 172.21.15.60:0/807227646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe06001c460 con 0x7fe0700b4100 2024-09-16T15:39:10.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.871+0000 7fe067400640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe050078850 0x7fe05007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:10.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.871+0000 7fe067400640 1 -- 172.21.15.60:0/807227646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe06009a700 con 0x7fe0700b4100 2024-09-16T15:39:10.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.871+0000 7fe06ea00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe050078850 0x7fe05007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:10.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.872+0000 7fe06ea00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe050078850 0x7fe05007ad10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe05c034000 tx=0x7fe05c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:10.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:10.875+0000 7fe067400640 1 -- 172.21.15.60:0/807227646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe060062d60 con 0x7fe0700b4100 2024-09-16T15:39:11.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.082+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe07006fb50 con 0x7fe0700b4100 2024-09-16T15:39:11.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.082+0000 7fe067400640 1 -- 172.21.15.60:0/807227646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fe060011e80 con 0x7fe0700b4100 2024-09-16T15:39:11.084 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:39:11.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe050078850 msgr2=0x7fe05007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:11.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe050078850 0x7fe05007ad10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe05c034000 tx=0x7fe05c004730 comp rx=0 tx=0).stop 2024-09-16T15:39:11.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b4100 msgr2=0x7fe0700b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:11.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b4100 0x7fe0700b8410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c400 tx=0x7fe06000c8c0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 shutdown_connections 2024-09-16T15:39:11.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe050078850 0x7fe05007ad10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161440 0x7fe0700c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0700b5370 0x7fe0701f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 --2- 172.21.15.60:0/807227646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b4100 0x7fe0700b8410 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 >> 172.21.15.60:0/807227646 conn(0x7fe0700d3df0 msgr2=0x7fe07006d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 shutdown_connections 2024-09-16T15:39:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.084+0000 7fe074e00640 1 -- 172.21.15.60:0/807227646 wait complete. 2024-09-16T15:39:11.103 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-16T15:39:11.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:11.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-16T15:39:11.163 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:39:11.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:11.163 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-16T15:39:11.199 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:11.199 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:11.234 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:11.234 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:11.310 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:11.310 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:11.385 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:11.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 -- 172.21.15.60:0/4024419963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 msgr2=0x7fbd200d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:11.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 --2- 172.21.15.60:0/4024419963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 0x7fbd200d21a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd200d4ef0 tx=0x7fbd1802aa30 comp rx=0 tx=0).stop 2024-09-16T15:39:11.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 -- 172.21.15.60:0/4024419963 shutdown_connections 2024-09-16T15:39:11.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 --2- 172.21.15.60:0/4024419963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 0x7fbd200d21a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 --2- 172.21.15.60:0/4024419963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd200d4980 0x7fbd200d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 --2- 172.21.15.60:0/4024419963 >> v2:172.21.15.100:3300/0 conn(0x7fbd201612d0 0x7fbd200b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 -- 172.21.15.60:0/4024419963 >> 172.21.15.60:0/4024419963 conn(0x7fbd200d3df0 msgr2=0x7fbd200d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:11.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 -- 172.21.15.60:0/4024419963 shutdown_connections 2024-09-16T15:39:11.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.546+0000 7fbd25e00640 1 -- 172.21.15.60:0/4024419963 wait complete. 2024-09-16T15:39:11.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.548+0000 7fbd25e00640 1 Processor -- start 2024-09-16T15:39:11.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.548+0000 7fbd25e00640 1 -- start start 2024-09-16T15:39:11.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd200d4980 0x7fbd201f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:11.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd201612d0 0x7fbd2014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:11.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 0x7fbd200c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:11.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd25e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbd200b3a10 con 0x7fbd201616b0 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd25e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbd200b3890 con 0x7fbd200d4980 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd25e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbd200b3b90 con 0x7fbd201612d0 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd24a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd200d4980 0x7fbd201f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd25400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 0x7fbd200c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd1fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd201612d0 0x7fbd2014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd24a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd200d4980 0x7fbd201f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:52020/0 (socket says 172.21.15.60:52020) 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd24a00640 1 -- 172.21.15.60:0/3701134034 learned_addr learned my addr 172.21.15.60:0/3701134034 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd24a00640 1 -- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd201612d0 msgr2=0x7fbd2014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd24a00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd201612d0 0x7fbd2014f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd24a00640 1 -- 172.21.15.60:0/3701134034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 msgr2=0x7fbd200c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:11.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd24a00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 0x7fbd200c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd24a00640 1 -- 172.21.15.60:0/3701134034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd201f7df0 con 0x7fbd200d4980 2024-09-16T15:39:11.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.549+0000 7fbd25400640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 0x7fbd200c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:11.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.550+0000 7fbd24a00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd200d4980 0x7fbd201f47d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400e9f0 tx=0x7fbd1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:11.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.550+0000 7fbd1cc00640 1 -- 172.21.15.60:0/3701134034 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400cd80 con 0x7fbd200d4980 2024-09-16T15:39:11.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.550+0000 7fbd1cc00640 1 -- 172.21.15.60:0/3701134034 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd14010430 con 0x7fbd200d4980 2024-09-16T15:39:11.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.550+0000 7fbd1cc00640 1 -- 172.21.15.60:0/3701134034 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14010840 con 0x7fbd200d4980 2024-09-16T15:39:11.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.550+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd201f8090 con 0x7fbd200d4980 2024-09-16T15:39:11.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.550+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd201f8650 con 0x7fbd200d4980 2024-09-16T15:39:11.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.552+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd201f8d10 con 0x7fbd200d4980 2024-09-16T15:39:11.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.553+0000 7fbd1cc00640 1 -- 172.21.15.60:0/3701134034 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fbd140109e0 con 0x7fbd200d4980 2024-09-16T15:39:11.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.554+0000 7fbd1cc00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbd00078850 0x7fbd0007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:11.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.554+0000 7fbd1cc00640 1 -- 172.21.15.60:0/3701134034 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbd1409b4a0 con 0x7fbd200d4980 2024-09-16T15:39:11.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.554+0000 7fbd1fe00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbd00078850 0x7fbd0007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:11.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.554+0000 7fbd1fe00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbd00078850 0x7fbd0007ad10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbd200c92d0 tx=0x7fbd0c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:11.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.559+0000 7fbd1cc00640 1 -- 172.21.15.60:0/3701134034 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbd201f8d10 con 0x7fbd200d4980 2024-09-16T15:39:11.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.767+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd200b4120 con 0x7fbd200d4980 2024-09-16T15:39:11.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.768+0000 7fbd1cc00640 1 -- 172.21.15.60:0/3701134034 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fbd140634e0 con 0x7fbd200d4980 2024-09-16T15:39:11.770 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:39:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbd00078850 msgr2=0x7fbd0007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbd00078850 0x7fbd0007ad10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbd200c92d0 tx=0x7fbd0c005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd200d4980 msgr2=0x7fbd201f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd200d4980 0x7fbd201f47d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400e9f0 tx=0x7fbd1400eeb0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 shutdown_connections 2024-09-16T15:39:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbd00078850 0x7fbd0007ad10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd201616b0 0x7fbd200c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd201612d0 0x7fbd2014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 --2- 172.21.15.60:0/3701134034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd200d4980 0x7fbd201f47d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:11.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.770+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 >> 172.21.15.60:0/3701134034 conn(0x7fbd200d3df0 msgr2=0x7fbd20160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:11.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.771+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 shutdown_connections 2024-09-16T15:39:11.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:11.771+0000 7fbd25e00640 1 -- 172.21.15.60:0/3701134034 wait complete. 2024-09-16T15:39:11.788 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-16T15:39:11.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:11.788 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-16T15:39:11.848 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:39:11.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:11.848 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-16T15:39:11.883 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:39:11.883 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.042+0000 7f46ab600640 1 -- 172.21.15.60:0/3329880950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46a4073fb0 msgr2=0x7f46a4074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.042+0000 7f46ab600640 1 --2- 172.21.15.60:0/3329880950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46a4073fb0 0x7f46a4074440 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f469c005fd0 tx=0x7f469c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.042+0000 7f46ab600640 1 -- 172.21.15.60:0/3329880950 shutdown_connections 2024-09-16T15:39:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.042+0000 7f46ab600640 1 --2- 172.21.15.60:0/3329880950 >> v2:172.21.15.100:3300/0 conn(0x7f46a40ccf00 0x7f46a40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.042+0000 7f46ab600640 1 --2- 172.21.15.60:0/3329880950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46a4073fb0 0x7f46a4074440 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.042+0000 7f46ab600640 1 -- 172.21.15.60:0/3329880950 >> 172.21.15.60:0/3329880950 conn(0x7f46a40b1950 msgr2=0x7f46a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.043+0000 7f46ab600640 1 -- 172.21.15.60:0/3329880950 shutdown_connections 2024-09-16T15:39:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.043+0000 7f46ab600640 1 -- 172.21.15.60:0/3329880950 wait complete. 2024-09-16T15:39:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.044+0000 7f46ab600640 1 Processor -- start 2024-09-16T15:39:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.044+0000 7f46ab600640 1 -- start start 2024-09-16T15:39:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.045+0000 7f46ab600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a4073fb0 0x7f46a40d0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.045+0000 7f46ab600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46a40ccf00 0x7f46a41f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.045+0000 7f46ab600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46a415f5a0 0x7f46a4140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.045+0000 7f46ab600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f46a406f2e0 con 0x7f46a415f5a0 2024-09-16T15:39:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.045+0000 7f46ab600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46a406f160 con 0x7f46a4073fb0 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.045+0000 7f46ab600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f46a406f460 con 0x7f46a40ccf00 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46a415f5a0 0x7f46a4140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46a415f5a0 0x7f46a4140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57098/0 (socket says 172.21.15.60:57098) 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 -- 172.21.15.60:0/1255510613 learned_addr learned my addr 172.21.15.60:0/1255510613 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a8c00640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a4073fb0 0x7f46a40d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f469be00640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46a40ccf00 0x7f46a41f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 -- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46a40ccf00 msgr2=0x7f46a41f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46a40ccf00 0x7f46a41f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 -- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a4073fb0 msgr2=0x7f46a40d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a4073fb0 0x7f46a40d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 -- 172.21.15.60:0/1255510613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a4157240 con 0x7f46a415f5a0 2024-09-16T15:39:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.046+0000 7f46a9600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46a415f5a0 0x7f46a4140150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f469c009d90 tx=0x7f469c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.047+0000 7f46a1600640 1 -- 172.21.15.60:0/1255510613 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c047070 con 0x7f46a415f5a0 2024-09-16T15:39:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.047+0000 7f46a1600640 1 -- 172.21.15.60:0/1255510613 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f469c02f670 con 0x7f46a415f5a0 2024-09-16T15:39:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.047+0000 7f46a1600640 1 -- 172.21.15.60:0/1255510613 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c033050 con 0x7f46a415f5a0 2024-09-16T15:39:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.047+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a4157520 con 0x7f46a415f5a0 2024-09-16T15:39:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.047+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a41f21e0 con 0x7f46a415f5a0 2024-09-16T15:39:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.048+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a406f980 con 0x7f46a415f5a0 2024-09-16T15:39:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.049+0000 7f46a1600640 1 -- 172.21.15.60:0/1255510613 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f469c04a070 con 0x7f46a415f5a0 2024-09-16T15:39:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.050+0000 7f46a1600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4680078850 0x7f468007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.051+0000 7f46a1600640 1 -- 172.21.15.60:0/1255510613 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f469c0bb8f0 con 0x7f46a415f5a0 2024-09-16T15:39:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.051+0000 7f46a8c00640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4680078850 0x7f468007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.052+0000 7f46a8c00640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4680078850 0x7f468007ad10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4694007fd0 tx=0x7f4694009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.055+0000 7f46a1600640 1 -- 172.21.15.60:0/1255510613 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f469c083900 con 0x7f46a415f5a0 2024-09-16T15:39:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.291+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46a415d9a0 con 0x7f46a415f5a0 2024-09-16T15:39:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.292+0000 7f46a1600640 1 -- 172.21.15.60:0/1255510613 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f469c088330 con 0x7f46a415f5a0 2024-09-16T15:39:12.294 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:12.294 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:38:34.368784+0000","modified":"2024-09-16T15:38:37.370114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-16T15:39:12.295 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4680078850 msgr2=0x7f468007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4680078850 0x7f468007ad10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4694007fd0 tx=0x7f4694009040 comp rx=0 tx=0).stop 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46a415f5a0 msgr2=0x7f46a4140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46a415f5a0 0x7f46a4140150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f469c009d90 tx=0x7f469c003f10 comp rx=0 tx=0).stop 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 shutdown_connections 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4680078850 0x7f468007ad10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46a415f5a0 0x7f46a4140150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46a40ccf00 0x7f46a41f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 --2- 172.21.15.60:0/1255510613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a4073fb0 0x7f46a40d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.294+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 >> 172.21.15.60:0/1255510613 conn(0x7f46a40b1950 msgr2=0x7f46a40af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:12.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.295+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 shutdown_connections 2024-09-16T15:39:12.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.295+0000 7f46ab600640 1 -- 172.21.15.60:0/1255510613 wait complete. 2024-09-16T15:39:12.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:12.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.472+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1488995803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ff415f5a0 msgr2=0x7f2ff40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.472+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1488995803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ff415f5a0 0x7f2ff40b2440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009830 tx=0x7f2fe402a480 comp rx=0 tx=0).stop 2024-09-16T15:39:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.472+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1488995803 shutdown_connections 2024-09-16T15:39:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.472+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1488995803 >> v2:172.21.15.100:3300/0 conn(0x7f2ff4073fb0 0x7f2ff4074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.472+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1488995803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ff415f5a0 0x7f2ff40b2440 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.472+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1488995803 >> 172.21.15.60:0/1488995803 conn(0x7f2ff40b1950 msgr2=0x7f2ff40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.472+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1488995803 shutdown_connections 2024-09-16T15:39:12.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.472+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1488995803 wait complete. 2024-09-16T15:39:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.474+0000 7f2ff9a00640 1 Processor -- start 2024-09-16T15:39:12.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.474+0000 7f2ff9a00640 1 -- start start 2024-09-16T15:39:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff9a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ff4073fb0 0x7f2ff414e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff9a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ff40ccf00 0x7f2ff41f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff9a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ff415f5a0 0x7f2ff41f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff9a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff406f2e0 con 0x7f2ff4073fb0 2024-09-16T15:39:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff9a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff406f160 con 0x7f2ff40ccf00 2024-09-16T15:39:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff9a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff406f460 con 0x7f2ff415f5a0 2024-09-16T15:39:12.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ff4073fb0 0x7f2ff414e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ff4073fb0 0x7f2ff414e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57100/0 (socket says 172.21.15.60:57100) 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.475+0000 7f2ff2a00640 1 -- 172.21.15.60:0/1492158854 learned_addr learned my addr 172.21.15.60:0/1492158854 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2fea000640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ff40ccf00 0x7f2ff41f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2ff3400640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ff415f5a0 0x7f2ff41f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2ff2a00640 1 -- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ff415f5a0 msgr2=0x7f2ff41f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2ff2a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ff415f5a0 0x7f2ff41f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2ff2a00640 1 -- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ff40ccf00 msgr2=0x7f2ff41f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2ff2a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ff40ccf00 0x7f2ff41f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2ff2a00640 1 -- 172.21.15.60:0/1492158854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff414ec20 con 0x7f2ff4073fb0 2024-09-16T15:39:12.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2ff2a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ff4073fb0 0x7f2ff414e6e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009950 tx=0x7f2fe4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:12.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.476+0000 7f2feb400640 1 -- 172.21.15.60:0/1492158854 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4005c90 con 0x7f2ff4073fb0 2024-09-16T15:39:12.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.477+0000 7f2feb400640 1 -- 172.21.15.60:0/1492158854 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe4031e50 con 0x7f2ff4073fb0 2024-09-16T15:39:12.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.477+0000 7f2feb400640 1 -- 172.21.15.60:0/1492158854 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4005260 con 0x7f2ff4073fb0 2024-09-16T15:39:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.477+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff41f6cb0 con 0x7f2ff4073fb0 2024-09-16T15:39:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.477+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff409b850 con 0x7f2ff4073fb0 2024-09-16T15:39:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.478+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff409b640 con 0x7f2ff4073fb0 2024-09-16T15:39:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.479+0000 7f2feb400640 1 -- 172.21.15.60:0/1492158854 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2fe4005400 con 0x7f2ff4073fb0 2024-09-16T15:39:12.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.480+0000 7f2feb400640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2fd0078850 0x7f2fd007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.480+0000 7f2feb400640 1 -- 172.21.15.60:0/1492158854 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2fe40bb7f0 con 0x7f2ff4073fb0 2024-09-16T15:39:12.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.481+0000 7f2fea000640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2fd0078850 0x7f2fd007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.481+0000 7f2fea000640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2fd0078850 0x7f2fd007ad10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc007fd0 tx=0x7f2fdc001410 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:12.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.485+0000 7f2feb400640 1 -- 172.21.15.60:0/1492158854 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ff414ec20 con 0x7f2ff4073fb0 2024-09-16T15:39:12.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.721+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ff406f9f0 con 0x7f2ff4073fb0 2024-09-16T15:39:12.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.721+0000 7f2feb400640 1 -- 172.21.15.60:0/1492158854 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f2fe403f030 con 0x7f2ff4073fb0 2024-09-16T15:39:12.723 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:12.724 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:38:34.368784+0000","modified":"2024-09-16T15:38:37.370114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-16T15:39:12.724 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.723+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2fd0078850 msgr2=0x7f2fd007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.723+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2fd0078850 0x7f2fd007ad10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc007fd0 tx=0x7f2fdc001410 comp rx=0 tx=0).stop 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.723+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ff4073fb0 msgr2=0x7f2ff414e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.723+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ff4073fb0 0x7f2ff414e6e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009950 tx=0x7f2fe4033c20 comp rx=0 tx=0).stop 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.723+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 shutdown_connections 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.723+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2fd0078850 0x7f2fd007ad10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.723+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ff415f5a0 0x7f2ff41f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.723+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ff40ccf00 0x7f2ff41f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.724+0000 7f2ff9a00640 1 --2- 172.21.15.60:0/1492158854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ff4073fb0 0x7f2ff414e6e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.724+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 >> 172.21.15.60:0/1492158854 conn(0x7f2ff40b1950 msgr2=0x7f2ff40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.724+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 shutdown_connections 2024-09-16T15:39:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.724+0000 7f2ff9a00640 1 -- 172.21.15.60:0/1492158854 wait complete. 2024-09-16T15:39:12.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.900+0000 7fa3e5000640 1 -- 172.21.15.60:0/765794035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 msgr2=0x7fa3c4003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.900+0000 7fa3e5000640 1 --2- 172.21.15.60:0/765794035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 0x7fa3c4003e90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009b50 tx=0x7fa3c4005830 comp rx=0 tx=0).stop 2024-09-16T15:39:12.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.900+0000 7fa3e5000640 1 -- 172.21.15.60:0/765794035 shutdown_connections 2024-09-16T15:39:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.900+0000 7fa3e5000640 1 --2- 172.21.15.60:0/765794035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 0x7fa3c4003e90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.900+0000 7fa3e5000640 1 -- 172.21.15.60:0/765794035 >> 172.21.15.60:0/765794035 conn(0x7fa3e00d3df0 msgr2=0x7fa3e00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.900+0000 7fa3e5000640 1 -- 172.21.15.60:0/765794035 shutdown_connections 2024-09-16T15:39:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.900+0000 7fa3e5000640 1 -- 172.21.15.60:0/765794035 wait complete. 2024-09-16T15:39:12.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.902+0000 7fa3e5000640 1 Processor -- start 2024-09-16T15:39:12.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.902+0000 7fa3e5000640 1 -- start start 2024-09-16T15:39:12.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.902+0000 7fa3e5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 0x7fa3e0158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3e5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3e01f5500 0x7fa3e00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3e5000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3e0154d50 0x7fa3e01551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3e5000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e00b5700 con 0x7fa3c4003a50 2024-09-16T15:39:12.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3e5000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e00b5580 con 0x7fa3e01f5500 2024-09-16T15:39:12.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3e5000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e00b3940 con 0x7fa3e0154d50 2024-09-16T15:39:12.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3df400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 0x7fa3e0158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3df400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 0x7fa3e0158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57112/0 (socket says 172.21.15.60:57112) 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3df400640 1 -- 172.21.15.60:0/2137081952 learned_addr learned my addr 172.21.15.60:0/2137081952 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3dea00640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3e01f5500 0x7fa3e00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3dfe00640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3e0154d50 0x7fa3e01551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3df400640 1 -- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3e0154d50 msgr2=0x7fa3e01551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3df400640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3e0154d50 0x7fa3e01551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3df400640 1 -- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3e01f5500 msgr2=0x7fa3e00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3df400640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3e01f5500 0x7fa3e00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3df400640 1 -- 172.21.15.60:0/2137081952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e00cbf10 con 0x7fa3c4003a50 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.903+0000 7fa3dfe00640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3e0154d50 0x7fa3e01551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.904+0000 7fa3df400640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 0x7fa3e0158a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0036040 tx=0x7fa3d00336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.904+0000 7fa3d7400640 1 -- 172.21.15.60:0/2137081952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0038040 con 0x7fa3c4003a50 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.904+0000 7fa3d7400640 1 -- 172.21.15.60:0/2137081952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3d0041070 con 0x7fa3c4003a50 2024-09-16T15:39:12.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.904+0000 7fa3d7400640 1 -- 172.21.15.60:0/2137081952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d003c430 con 0x7fa3c4003a50 2024-09-16T15:39:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.904+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01f5960 con 0x7fa3c4003a50 2024-09-16T15:39:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.904+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e00cc490 con 0x7fa3c4003a50 2024-09-16T15:39:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.905+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e00b3e10 con 0x7fa3c4003a50 2024-09-16T15:39:12.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.906+0000 7fa3d7400640 1 -- 172.21.15.60:0/2137081952 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa3d003c5d0 con 0x7fa3c4003a50 2024-09-16T15:39:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.907+0000 7fa3d7400640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa3c0078850 0x7fa3c007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.907+0000 7fa3d7400640 1 -- 172.21.15.60:0/2137081952 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa3d00baf00 con 0x7fa3c4003a50 2024-09-16T15:39:12.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.908+0000 7fa3dea00640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa3c0078850 0x7fa3c007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:12.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.908+0000 7fa3dea00640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa3c0078850 0x7fa3c007ad10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00cd960 tx=0x7fa3cc006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:12.911+0000 7fa3d7400640 1 -- 172.21.15.60:0/2137081952 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa3d004b020 con 0x7fa3c4003a50 2024-09-16T15:39:13.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.111+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3e015d1d0 con 0x7fa3c4003a50 2024-09-16T15:39:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.112+0000 7fa3d7400640 1 -- 172.21.15.60:0/2137081952 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7fa3d0082f20 con 0x7fa3c4003a50 2024-09-16T15:39:13.115 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:13.115 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:38:35.924959+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa3c0078850 msgr2=0x7fa3c007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa3c0078850 0x7fa3c007ad10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00cd960 tx=0x7fa3cc006b70 comp rx=0 tx=0).stop 2024-09-16T15:39:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 msgr2=0x7fa3e0158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 0x7fa3e0158a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0036040 tx=0x7fa3d00336f0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 shutdown_connections 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa3c0078850 0x7fa3c007ad10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3e0154d50 0x7fa3e01551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3e01f5500 0x7fa3e00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 --2- 172.21.15.60:0/2137081952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3c4003a50 0x7fa3e0158a00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 >> 172.21.15.60:0/2137081952 conn(0x7fa3e00d3df0 msgr2=0x7fa3e0160300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 shutdown_connections 2024-09-16T15:39:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.115+0000 7fa3e5000640 1 -- 172.21.15.60:0/2137081952 wait complete. 2024-09-16T15:39:13.134 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:39:13.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.293+0000 7f6db0000640 1 -- 172.21.15.60:0/3798739128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d90004d40 msgr2=0x7f6d900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.293+0000 7f6db0000640 1 --2- 172.21.15.60:0/3798739128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d90004d40 0x7f6d900051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6da0001800 tx=0x7f6d90005830 comp rx=0 tx=0).stop 2024-09-16T15:39:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.293+0000 7f6dae000640 1 -- 172.21.15.60:0/3798739128 reap_dead start 2024-09-16T15:39:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.293+0000 7f6db0000640 1 -- 172.21.15.60:0/3798739128 shutdown_connections 2024-09-16T15:39:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.293+0000 7f6db0000640 1 -- 172.21.15.60:0/3798739128 >> 172.21.15.60:0/3798739128 conn(0x7f6da80b1950 msgr2=0x7f6da80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.294+0000 7f6db0000640 1 -- 172.21.15.60:0/3798739128 shutdown_connections 2024-09-16T15:39:13.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.294+0000 7f6db0000640 1 -- 172.21.15.60:0/3798739128 wait complete. 2024-09-16T15:39:13.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.295+0000 7f6db0000640 1 Processor -- start 2024-09-16T15:39:13.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.295+0000 7f6db0000640 1 -- start start 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.296+0000 7f6db0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90004d40 0x7f6da809b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.296+0000 7f6db0000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6da80c9f20 0x7f6da80af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.296+0000 7f6db0000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6da80afdb0 0x7f6da81f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.296+0000 7f6db0000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6da806f9f0 con 0x7f6da80c9f20 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.296+0000 7f6db0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6da806f870 con 0x7f6d90004d40 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.296+0000 7f6db0000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6da806fb70 con 0x7f6da80afdb0 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.296+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6da80c9f20 0x7f6da80af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6da80c9f20 0x7f6da80af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57128/0 (socket says 172.21.15.60:57128) 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dacc00640 1 -- 172.21.15.60:0/2558224999 learned_addr learned my addr 172.21.15.60:0/2558224999 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:13.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dad600640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90004d40 0x7f6da809b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dae000640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6da80afdb0 0x7f6da81f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dacc00640 1 -- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6da80afdb0 msgr2=0x7f6da81f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dacc00640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6da80afdb0 0x7f6da81f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dacc00640 1 -- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90004d40 msgr2=0x7f6da809b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dacc00640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90004d40 0x7f6da809b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dacc00640 1 -- 172.21.15.60:0/2558224999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da809c2c0 con 0x7f6da80c9f20 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dad600640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90004d40 0x7f6da809b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dae000640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6da80afdb0 0x7f6da81f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.297+0000 7f6dacc00640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6da80c9f20 0x7f6da80af870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6d94011ab0 tx=0x7f6d94011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.298+0000 7f6da5600640 1 -- 172.21.15.60:0/2558224999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9401e070 con 0x7f6da80c9f20 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.298+0000 7f6da5600640 1 -- 172.21.15.60:0/2558224999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d9400fc70 con 0x7f6da80c9f20 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.298+0000 7f6da5600640 1 -- 172.21.15.60:0/2558224999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94019670 con 0x7f6da80c9f20 2024-09-16T15:39:13.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.298+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da81f5aa0 con 0x7f6da80c9f20 2024-09-16T15:39:13.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.298+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da809aaa0 con 0x7f6da80c9f20 2024-09-16T15:39:13.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.299+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da815f630 con 0x7f6da80c9f20 2024-09-16T15:39:13.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.300+0000 7f6da5600640 1 -- 172.21.15.60:0/2558224999 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6d94021070 con 0x7f6da80c9f20 2024-09-16T15:39:13.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.301+0000 7f6da5600640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6d88078880 0x7f6d8807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:13.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.302+0000 7f6da5600640 1 -- 172.21.15.60:0/2558224999 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6d9409b1a0 con 0x7f6da80c9f20 2024-09-16T15:39:13.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.302+0000 7f6dad600640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6d88078880 0x7f6d8807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:13.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.303+0000 7f6dad600640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6d88078880 0x7f6d8807ad40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c006fd0 tx=0x7f6d9c006e60 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:13.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.306+0000 7f6da5600640 1 -- 172.21.15.60:0/2558224999 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6da815f630 con 0x7f6da80c9f20 2024-09-16T15:39:13.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.537+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6da80b1460 con 0x7f6da80c9f20 2024-09-16T15:39:13.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.538+0000 7f6da5600640 1 -- 172.21.15.60:0/2558224999 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f6d9400fe60 con 0x7f6da80c9f20 2024-09-16T15:39:13.540 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:13.540 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:38:34.368784+0000","modified":"2024-09-16T15:38:37.370114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-16T15:39:13.541 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.540+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6d88078880 msgr2=0x7f6d8807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.540+0000 7f6db0000640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6d88078880 0x7f6d8807ad40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c006fd0 tx=0x7f6d9c006e60 comp rx=0 tx=0).stop 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.540+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6da80c9f20 msgr2=0x7f6da80af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.540+0000 7f6db0000640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6da80c9f20 0x7f6da80af870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6d94011ab0 tx=0x7f6d94011f70 comp rx=0 tx=0).stop 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.541+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 shutdown_connections 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.541+0000 7f6db0000640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6d88078880 0x7f6d8807ad40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.541+0000 7f6db0000640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6da80afdb0 0x7f6da81f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.541+0000 7f6db0000640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6da80c9f20 0x7f6da80af870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.541+0000 7f6db0000640 1 --2- 172.21.15.60:0/2558224999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d90004d40 0x7f6da809b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.541+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 >> 172.21.15.60:0/2558224999 conn(0x7f6da80b1950 msgr2=0x7f6da80ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.541+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 shutdown_connections 2024-09-16T15:39:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.541+0000 7f6db0000640 1 -- 172.21.15.60:0/2558224999 wait complete. 2024-09-16T15:39:13.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.716+0000 7f0138e00640 1 -- 172.21.15.60:0/3366169215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 msgr2=0x7f011800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.716+0000 7f0138e00640 1 --2- 172.21.15.60:0/3366169215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 0x7f011800b9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c590 tx=0x7f0118005830 comp rx=0 tx=0).stop 2024-09-16T15:39:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.716+0000 7f0132800640 1 -- 172.21.15.60:0/3366169215 reap_dead start 2024-09-16T15:39:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.716+0000 7f0138e00640 1 -- 172.21.15.60:0/3366169215 shutdown_connections 2024-09-16T15:39:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.716+0000 7f0138e00640 1 -- 172.21.15.60:0/3366169215 >> 172.21.15.60:0/3366169215 conn(0x7f01340b1950 msgr2=0x7f01340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.716+0000 7f0138e00640 1 -- 172.21.15.60:0/3366169215 shutdown_connections 2024-09-16T15:39:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.716+0000 7f0138e00640 1 -- 172.21.15.60:0/3366169215 wait complete. 2024-09-16T15:39:13.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.718+0000 7f0138e00640 1 Processor -- start 2024-09-16T15:39:13.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.718+0000 7f0138e00640 1 -- start start 2024-09-16T15:39:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.719+0000 7f0138e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 0x7f01341467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.719+0000 7f0138e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0134146ce0 0x7f01341f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.719+0000 7f0138e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01341f6920 0x7f0134149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.719+0000 7f0138e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f01340aceb0 con 0x7f01180051c0 2024-09-16T15:39:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.719+0000 7f0138e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f01340acd30 con 0x7f01341f6920 2024-09-16T15:39:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.719+0000 7f0138e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f01340ad030 con 0x7f0134146ce0 2024-09-16T15:39:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 0x7f01341467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 0x7f01341467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57150/0 (socket says 172.21.15.60:57150) 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 -- 172.21.15.60:0/2043155747 learned_addr learned my addr 172.21.15.60:0/2043155747 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131400640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0134146ce0 0x7f01341f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0132800640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01341f6920 0x7f0134149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 -- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0134146ce0 msgr2=0x7f01341f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0134146ce0 0x7f01341f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 -- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01341f6920 msgr2=0x7f0134149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01341f6920 0x7f0134149420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 -- 172.21.15.60:0/2043155747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0134157680 con 0x7f01180051c0 2024-09-16T15:39:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0132800640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01341f6920 0x7f0134149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:13.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.720+0000 7f0131e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 0x7f01341467a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0124011ab0 tx=0x7f0124011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:13.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.721+0000 7f012a000640 1 -- 172.21.15.60:0/2043155747 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012400fe10 con 0x7f01180051c0 2024-09-16T15:39:13.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.721+0000 7f012a000640 1 -- 172.21.15.60:0/2043155747 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0124016070 con 0x7f01180051c0 2024-09-16T15:39:13.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.721+0000 7f012a000640 1 -- 172.21.15.60:0/2043155747 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124012690 con 0x7f01180051c0 2024-09-16T15:39:13.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.721+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0134157920 con 0x7f01180051c0 2024-09-16T15:39:13.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.721+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0134157ee0 con 0x7f01180051c0 2024-09-16T15:39:13.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.722+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01340b2440 con 0x7f01180051c0 2024-09-16T15:39:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.724+0000 7f012a000640 1 -- 172.21.15.60:0/2043155747 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0124018030 con 0x7f01180051c0 2024-09-16T15:39:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.725+0000 7f012a000640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0114078880 0x7f011407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.725+0000 7f012a000640 1 -- 172.21.15.60:0/2043155747 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f012409ba00 con 0x7f01180051c0 2024-09-16T15:39:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.725+0000 7f0131400640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0114078880 0x7f011407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.726+0000 7f0131400640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0114078880 0x7f011407ad40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01341f60d0 tx=0x7f0120005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.728+0000 7f012a000640 1 -- 172.21.15.60:0/2043155747 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0124064150 con 0x7f01180051c0 2024-09-16T15:39:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.962+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01340ada20 con 0x7f01180051c0 2024-09-16T15:39:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.963+0000 7f012a000640 1 -- 172.21.15.60:0/2043155747 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f0124063a50 con 0x7f01180051c0 2024-09-16T15:39:13.965 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:13.965 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"btime":"2024-09-16T15:38:37:938909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3697863306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3697863306},{"type":"v1","addr":"172.21.15.60:6835","nonce":3697863306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4353,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1889406986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1889406986},{"type":"v1","addr":"172.21.15.60:6834","nonce":1889406986}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4399,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1344819984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1344819984},{"type":"v1","addr":"172.21.15.100:6836","nonce":1344819984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:38:34.368784+0000","modified":"2024-09-16T15:38:37.370114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4402":{"gid":4402,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/614854687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":614854687},{"type":"v1","addr":"172.21.15.100:6837","nonce":614854687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4402,"qdb_cluster":[4402]},"id":1}]} 2024-09-16T15:39:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0114078880 msgr2=0x7f011407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0114078880 0x7f011407ad40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01341f60d0 tx=0x7f0120005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 msgr2=0x7f01341467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 0x7f01341467a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0124011ab0 tx=0x7f0124011eb0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 shutdown_connections 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0114078880 0x7f011407ad40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01341f6920 0x7f0134149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0134146ce0 0x7f01341f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 --2- 172.21.15.60:0/2043155747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01180051c0 0x7f01341467a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.965+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 >> 172.21.15.60:0/2043155747 conn(0x7f01340b1950 msgr2=0x7f0134073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:13.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.966+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 shutdown_connections 2024-09-16T15:39:13.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:13.966+0000 7f0138e00640 1 -- 172.21.15.60:0/2043155747 wait complete. 2024-09-16T15:39:13.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.149+0000 7f7a38200640 1 -- 172.21.15.60:0/4059482331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a30073fb0 msgr2=0x7f7a30074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.149+0000 7f7a38200640 1 --2- 172.21.15.60:0/4059482331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a30073fb0 0x7f7a30074440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005fd0 tx=0x7f7a2802a980 comp rx=0 tx=0).stop 2024-09-16T15:39:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.149+0000 7f7a38200640 1 -- 172.21.15.60:0/4059482331 shutdown_connections 2024-09-16T15:39:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.149+0000 7f7a38200640 1 --2- 172.21.15.60:0/4059482331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a30073fb0 0x7f7a30074440 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.149+0000 7f7a38200640 1 --2- 172.21.15.60:0/4059482331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a300ccf00 0x7f7a300b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.149+0000 7f7a38200640 1 -- 172.21.15.60:0/4059482331 >> 172.21.15.60:0/4059482331 conn(0x7f7a300b1950 msgr2=0x7f7a300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.149+0000 7f7a38200640 1 -- 172.21.15.60:0/4059482331 shutdown_connections 2024-09-16T15:39:14.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.149+0000 7f7a38200640 1 -- 172.21.15.60:0/4059482331 wait complete. 2024-09-16T15:39:14.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.151+0000 7f7a38200640 1 Processor -- start 2024-09-16T15:39:14.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.151+0000 7f7a38200640 1 -- start start 2024-09-16T15:39:14.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a38200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a30073fb0 0x7f7a300cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a38200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a300ccf00 0x7f7a30158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a38200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a3015f5a0 0x7f7a301f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a38200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3006f2e0 con 0x7f7a3015f5a0 2024-09-16T15:39:14.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a38200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3006f160 con 0x7f7a300ccf00 2024-09-16T15:39:14.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a38200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7a3006f460 con 0x7f7a30073fb0 2024-09-16T15:39:14.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a36200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a3015f5a0 0x7f7a301f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a36200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a3015f5a0 0x7f7a301f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57166/0 (socket says 172.21.15.60:57166) 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a36200640 1 -- 172.21.15.60:0/3043362867 learned_addr learned my addr 172.21.15.60:0/3043362867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a34e00640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a300ccf00 0x7f7a30158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.152+0000 7f7a35800640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a30073fb0 0x7f7a300cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a36200640 1 -- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a30073fb0 msgr2=0x7f7a300cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a36200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a30073fb0 0x7f7a300cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a36200640 1 -- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a300ccf00 msgr2=0x7f7a30158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a36200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a300ccf00 0x7f7a30158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a36200640 1 -- 172.21.15.60:0/3043362867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a301582f0 con 0x7f7a3015f5a0 2024-09-16T15:39:14.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a36200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a3015f5a0 0x7f7a301f23c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a28006010 tx=0x7f7a28003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:14.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a2d600640 1 -- 172.21.15.60:0/3043362867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28047070 con 0x7f7a3015f5a0 2024-09-16T15:39:14.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a2d600640 1 -- 172.21.15.60:0/3043362867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a280045c0 con 0x7f7a3015f5a0 2024-09-16T15:39:14.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a2d600640 1 -- 172.21.15.60:0/3043362867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28034070 con 0x7f7a3015f5a0 2024-09-16T15:39:14.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a301f2e10 con 0x7f7a3015f5a0 2024-09-16T15:39:14.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.153+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a301585b0 con 0x7f7a3015f5a0 2024-09-16T15:39:14.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.154+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3006f9f0 con 0x7f7a3015f5a0 2024-09-16T15:39:14.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.156+0000 7f7a2d600640 1 -- 172.21.15.60:0/3043362867 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7a2804a070 con 0x7f7a3015f5a0 2024-09-16T15:39:14.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.156+0000 7f7a2d600640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a10078850 0x7f7a1007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.157+0000 7f7a2d600640 1 -- 172.21.15.60:0/3043362867 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f7a280bbd10 con 0x7f7a3015f5a0 2024-09-16T15:39:14.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.157+0000 7f7a35800640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a10078850 0x7f7a1007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.158+0000 7f7a35800640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a10078850 0x7f7a1007ad10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a24006fd0 tx=0x7f7a24008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.161+0000 7f7a2d600640 1 -- 172.21.15.60:0/3043362867 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7a280c1020 con 0x7f7a3015f5a0 2024-09-16T15:39:14.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.355+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a300cc010 con 0x7f7a3015f5a0 2024-09-16T15:39:14.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.357+0000 7f7a2d600640 1 -- 172.21.15.60:0/3043362867 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+17008 (secure 0 0 0) 0x7f7a2803f070 con 0x7f7a3015f5a0 2024-09-16T15:39:14.359 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:14.359 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:39:14.062901+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:38:32.430442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:38:33.311774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":0,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a10078850 msgr2=0x7f7a1007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a10078850 0x7f7a1007ad10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a24006fd0 tx=0x7f7a24008040 comp rx=0 tx=0).stop 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a3015f5a0 msgr2=0x7f7a301f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a3015f5a0 0x7f7a301f23c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a28006010 tx=0x7f7a28003ef0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 shutdown_connections 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a10078850 0x7f7a1007ad10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a3015f5a0 0x7f7a301f23c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a300ccf00 0x7f7a30158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 --2- 172.21.15.60:0/3043362867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a30073fb0 0x7f7a300cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 >> 172.21.15.60:0/3043362867 conn(0x7f7a300b1950 msgr2=0x7f7a3015cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 shutdown_connections 2024-09-16T15:39:14.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.359+0000 7f7a38200640 1 -- 172.21.15.60:0/3043362867 wait complete. 2024-09-16T15:39:14.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:39:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.537+0000 7f929de00640 1 -- 172.21.15.60:0/1513974911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c004d40 msgr2=0x7f927c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.537+0000 7f929de00640 1 --2- 172.21.15.60:0/1513974911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f927c004d40 0x7f927c0051c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9290005fd0 tx=0x7f927c005830 comp rx=0 tx=0).stop 2024-09-16T15:39:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.537+0000 7f9297400640 1 -- 172.21.15.60:0/1513974911 reap_dead start 2024-09-16T15:39:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.538+0000 7f929de00640 1 -- 172.21.15.60:0/1513974911 shutdown_connections 2024-09-16T15:39:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.538+0000 7f929de00640 1 -- 172.21.15.60:0/1513974911 >> 172.21.15.60:0/1513974911 conn(0x7f92980d3fe0 msgr2=0x7f92980d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.538+0000 7f929de00640 1 -- 172.21.15.60:0/1513974911 shutdown_connections 2024-09-16T15:39:14.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.538+0000 7f929de00640 1 -- 172.21.15.60:0/1513974911 wait complete. 2024-09-16T15:39:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.539+0000 7f929de00640 1 Processor -- start 2024-09-16T15:39:14.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.539+0000 7f929de00640 1 -- start start 2024-09-16T15:39:14.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.540+0000 7f929de00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f927c004d40 0x7f92980cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.540+0000 7f929de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92981f56f0 0x7f92981608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.540+0000 7f929de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929815d050 0x7f929815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.540+0000 7f929de00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f92980d0d50 con 0x7f929815d050 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.540+0000 7f929de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92980d0bd0 con 0x7f92981f56f0 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.540+0000 7f929de00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f92980cf200 con 0x7f927c004d40 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.540+0000 7f9297400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929815d050 0x7f929815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9297400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929815d050 0x7f929815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57182/0 (socket says 172.21.15.60:57182) 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9297400640 1 -- 172.21.15.60:0/3957548760 learned_addr learned my addr 172.21.15.60:0/3957548760 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9296000640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92981f56f0 0x7f92981608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9296a00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f927c004d40 0x7f92980cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9297400640 1 -- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f927c004d40 msgr2=0x7f92980cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9297400640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f927c004d40 0x7f92980cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9297400640 1 -- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92981f56f0 msgr2=0x7f92981608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9297400640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92981f56f0 0x7f92981608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9297400640 1 -- 172.21.15.60:0/3957548760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92980ce1c0 con 0x7f929815d050 2024-09-16T15:39:14.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f9297400640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929815d050 0x7f929815d4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9290036040 tx=0x7f9290034bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:14.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.541+0000 7f928f400640 1 -- 172.21.15.60:0/3957548760 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290043070 con 0x7f929815d050 2024-09-16T15:39:14.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.542+0000 7f928f400640 1 -- 172.21.15.60:0/3957548760 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9290034d20 con 0x7f929815d050 2024-09-16T15:39:14.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.542+0000 7f928f400640 1 -- 172.21.15.60:0/3957548760 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290038040 con 0x7f929815d050 2024-09-16T15:39:14.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.542+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92980ce4a0 con 0x7f929815d050 2024-09-16T15:39:14.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.542+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92981f6220 con 0x7f929815d050 2024-09-16T15:39:14.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.543+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929808cf70 con 0x7f929815d050 2024-09-16T15:39:14.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.544+0000 7f928f400640 1 -- 172.21.15.60:0/3957548760 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9290046070 con 0x7f929815d050 2024-09-16T15:39:14.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.545+0000 7f928f400640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9278078880 0x7f927807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.545+0000 7f928f400640 1 -- 172.21.15.60:0/3957548760 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5883+0+0 (secure 0 0 0) 0x7f92900ba5c0 con 0x7f929815d050 2024-09-16T15:39:14.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.545+0000 7f9296a00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9278078880 0x7f927807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.546+0000 7f9296a00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9278078880 0x7f927807ad40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9288009b90 tx=0x7f9288005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:14.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.549+0000 7f928f400640 1 -- 172.21.15.60:0/3957548760 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f929808cf70 con 0x7f929815d050 2024-09-16T15:39:14.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.787+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f92980cecc0 con 0x7f929815d050 2024-09-16T15:39:14.790 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:39:14.788+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:39:14.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.790+0000 7f928f400640 1 -- 172.21.15.60:0/3957548760 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f929008b020 con 0x7f929815d050 2024-09-16T15:39:14.792 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9278078880 msgr2=0x7f927807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9278078880 0x7f927807ad40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9288009b90 tx=0x7f9288005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929815d050 msgr2=0x7f929815d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929815d050 0x7f929815d4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9290036040 tx=0x7f9290034bb0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 shutdown_connections 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9278078880 0x7f927807ad40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929815d050 0x7f929815d4b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92981f56f0 0x7f92981608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.793+0000 7f929de00640 1 --2- 172.21.15.60:0/3957548760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f927c004d40 0x7f92980cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.794+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 >> 172.21.15.60:0/3957548760 conn(0x7f92980d3fe0 msgr2=0x7f92980b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:14.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.794+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 shutdown_connections 2024-09-16T15:39:14.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.794+0000 7f929de00640 1 -- 172.21.15.60:0/3957548760 wait complete. 2024-09-16T15:39:14.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:39:14.865 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:39:14.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.978+0000 7f5271e00640 1 -- 172.21.15.60:0/104569053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0cd6f0 msgr2=0x7f526c0d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.978+0000 7f5271e00640 1 --2- 172.21.15.60:0/104569053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0cd6f0 0x7f526c0d01d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f526c073cd0 tx=0x7f525802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.978+0000 7f5271e00640 1 -- 172.21.15.60:0/104569053 shutdown_connections 2024-09-16T15:39:14.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.978+0000 7f5271e00640 1 --2- 172.21.15.60:0/104569053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0cd6f0 0x7f526c0d01d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.978+0000 7f5271e00640 1 --2- 172.21.15.60:0/104569053 >> v2:172.21.15.100:3300/0 conn(0x7f526c15bf10 0x7f526c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.978+0000 7f5271e00640 1 -- 172.21.15.60:0/104569053 >> 172.21.15.60:0/104569053 conn(0x7f526c0d1e20 msgr2=0x7f526c0d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.978+0000 7f5271e00640 1 -- 172.21.15.60:0/104569053 shutdown_connections 2024-09-16T15:39:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.978+0000 7f5271e00640 1 -- 172.21.15.60:0/104569053 wait complete. 2024-09-16T15:39:14.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.979+0000 7f5271e00640 1 Processor -- start 2024-09-16T15:39:14.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.980+0000 7f5271e00640 1 -- start start 2024-09-16T15:39:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.980+0000 7f5271e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c0cd6f0 0x7f526c1f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f5271e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0d0710 0x7f526c0c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f5271e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f526c15bf10 0x7f526c152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f5271e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f526c0cdce0 con 0x7f526c0d0710 2024-09-16T15:39:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f5271e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f526c0cdb60 con 0x7f526c0cd6f0 2024-09-16T15:39:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f5271e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f526c0cf9f0 con 0x7f526c15bf10 2024-09-16T15:39:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0d0710 0x7f526c0c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0d0710 0x7f526c0c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57198/0 (socket says 172.21.15.60:57198) 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526a000640 1 -- 172.21.15.60:0/2541002751 learned_addr learned my addr 172.21.15.60:0/2541002751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526aa00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c0cd6f0 0x7f526c1f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526b400640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f526c15bf10 0x7f526c152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526a000640 1 -- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f526c15bf10 msgr2=0x7f526c152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526a000640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f526c15bf10 0x7f526c152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526a000640 1 -- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c0cd6f0 msgr2=0x7f526c1f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526a000640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c0cd6f0 0x7f526c1f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526a000640 1 -- 172.21.15.60:0/2541002751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c0cbf20 con 0x7f526c0d0710 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.981+0000 7f526aa00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c0cd6f0 0x7f526c1f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.982+0000 7f526a000640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0d0710 0x7f526c0c8ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5258009850 tx=0x7f52580039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.982+0000 7f5263400640 1 -- 172.21.15.60:0/2541002751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525803f070 con 0x7f526c0d0710 2024-09-16T15:39:14.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.982+0000 7f5263400640 1 -- 172.21.15.60:0/2541002751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5258004890 con 0x7f526c0d0710 2024-09-16T15:39:14.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.982+0000 7f5263400640 1 -- 172.21.15.60:0/2541002751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258035030 con 0x7f526c0d0710 2024-09-16T15:39:14.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.982+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c1f7db0 con 0x7f526c0d0710 2024-09-16T15:39:14.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.982+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c0cc200 con 0x7f526c0d0710 2024-09-16T15:39:14.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.983+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c0d01d0 con 0x7f526c0d0710 2024-09-16T15:39:14.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.984+0000 7f5263400640 1 -- 172.21.15.60:0/2541002751 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5258042070 con 0x7f526c0d0710 2024-09-16T15:39:14.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.985+0000 7f5263400640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f524c078850 0x7f524c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:14.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.985+0000 7f5263400640 1 -- 172.21.15.60:0/2541002751 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5969+0+0 (secure 0 0 0) 0x7f52580bba30 con 0x7f526c0d0710 2024-09-16T15:39:14.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.985+0000 7f526aa00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f524c078850 0x7f524c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:14.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.986+0000 7f526aa00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f524c078850 0x7f524c07ad10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f525c00ab40 tx=0x7f525c00a520 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:14.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:14.989+0000 7f5263400640 1 -- 172.21.15.60:0/2541002751 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5258083ef0 con 0x7f526c0d0710 2024-09-16T15:39:15.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.222+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f526c069630 con 0x7f526c0d0710 2024-09-16T15:39:15.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.792+0000 7f5263400640 1 -- 172.21.15.60:0/2541002751 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f52580837f0 con 0x7f526c0d0710 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f524c078850 msgr2=0x7f524c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f524c078850 0x7f524c07ad10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f525c00ab40 tx=0x7f525c00a520 comp rx=0 tx=0).stop 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0d0710 msgr2=0x7f526c0c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0d0710 0x7f526c0c8ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5258009850 tx=0x7f52580039e0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 shutdown_connections 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f524c078850 0x7f524c07ad10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f526c15bf10 0x7f526c152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c0d0710 0x7f526c0c8ef0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 --2- 172.21.15.60:0/2541002751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c0cd6f0 0x7f526c1f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 >> 172.21.15.60:0/2541002751 conn(0x7f526c0d1e20 msgr2=0x7f526c06eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:15.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.794+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 shutdown_connections 2024-09-16T15:39:15.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.795+0000 7f5271e00640 1 -- 172.21.15.60:0/2541002751 wait complete. 2024-09-16T15:39:15.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 -- 172.21.15.60:0/2469314520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 msgr2=0x7f96840741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 --2- 172.21.15.60:0/2469314520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 0x7f96840741a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9684070850 tx=0x7f967802a560 comp rx=0 tx=0).stop 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 -- 172.21.15.60:0/2469314520 shutdown_connections 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 --2- 172.21.15.60:0/2469314520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968406f220 0x7f968406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 --2- 172.21.15.60:0/2469314520 >> v2:172.21.15.100:3300/0 conn(0x7f968415cff0 0x7f968406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 --2- 172.21.15.60:0/2469314520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 0x7f96840741a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 -- 172.21.15.60:0/2469314520 >> 172.21.15.60:0/2469314520 conn(0x7f96840b18e0 msgr2=0x7f96840b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 -- 172.21.15.60:0/2469314520 shutdown_connections 2024-09-16T15:39:15.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.979+0000 7f968be00640 1 -- 172.21.15.60:0/2469314520 wait complete. 2024-09-16T15:39:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.981+0000 7f968be00640 1 Processor -- start 2024-09-16T15:39:15.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.981+0000 7f968be00640 1 -- start start 2024-09-16T15:39:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f968be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968406f220 0x7f968414b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f968be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 0x7f96840c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f968be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968415cff0 0x7f96840b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f968be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f96840cc310 con 0x7f9684159aa0 2024-09-16T15:39:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f968be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96840cc190 con 0x7f968406f220 2024-09-16T15:39:15.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f968be00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f96840cc490 con 0x7f968415cff0 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f9688a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 0x7f96840c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f9688a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 0x7f96840c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57222/0 (socket says 172.21.15.60:57222) 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.982+0000 7f9688a00640 1 -- 172.21.15.60:0/1846876281 learned_addr learned my addr 172.21.15.60:0/1846876281 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9689e00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968415cff0 0x7f96840b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9689400640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968406f220 0x7f968414b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9688a00640 1 -- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968415cff0 msgr2=0x7f96840b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9688a00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968415cff0 0x7f96840b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9688a00640 1 -- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968406f220 msgr2=0x7f968414b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9688a00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968406f220 0x7f968414b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9688a00640 1 -- 172.21.15.60:0/1846876281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968414bc70 con 0x7f9684159aa0 2024-09-16T15:39:15.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9688a00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 0x7f96840c8f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f967000c970 tx=0x7f967000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:15.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9681600640 1 -- 172.21.15.60:0/1846876281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670007bb0 con 0x7f9684159aa0 2024-09-16T15:39:15.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9681600640 1 -- 172.21.15.60:0/1846876281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9670007e40 con 0x7f9684159aa0 2024-09-16T15:39:15.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f9681600640 1 -- 172.21.15.60:0/1846876281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967000f690 con 0x7f9684159aa0 2024-09-16T15:39:15.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.983+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96840c97b0 con 0x7f9684159aa0 2024-09-16T15:39:15.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.984+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96840c9d70 con 0x7f9684159aa0 2024-09-16T15:39:15.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.985+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9684069360 con 0x7f9684159aa0 2024-09-16T15:39:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.986+0000 7f9681600640 1 -- 172.21.15.60:0/1846876281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f967000f830 con 0x7f9684159aa0 2024-09-16T15:39:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.987+0000 7f9681600640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9664078850 0x7f966407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:15.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.987+0000 7f9681600640 1 -- 172.21.15.60:0/1846876281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5969+0+0 (secure 0 0 0) 0x7f967009b170 con 0x7f9684159aa0 2024-09-16T15:39:15.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.987+0000 7f9689400640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9664078850 0x7f966407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:15.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.988+0000 7f9689400640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9664078850 0x7f966407ad10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9678033000 tx=0x7f9678004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:15.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:15.991+0000 7f9681600640 1 -- 172.21.15.60:0/1846876281 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f968414bc70 con 0x7f9684159aa0 2024-09-16T15:39:16.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:16.186+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f96840741a0 con 0x7f9684159aa0 2024-09-16T15:39:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.083+0000 7f9681600640 1 -- 172.21.15.60:0/1846876281 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7f9670014030 con 0x7f9684159aa0 2024-09-16T15:39:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:39:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9664078850 msgr2=0x7f966407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9664078850 0x7f966407ad10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9678033000 tx=0x7f9678004730 comp rx=0 tx=0).stop 2024-09-16T15:39:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 msgr2=0x7f96840c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 0x7f96840c8f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f967000c970 tx=0x7f967000ce30 comp rx=0 tx=0).stop 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 shutdown_connections 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9664078850 0x7f966407ad10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f968415cff0 0x7f96840b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9684159aa0 0x7f96840c8f60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 --2- 172.21.15.60:0/1846876281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f968406f220 0x7f968414b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 >> 172.21.15.60:0/1846876281 conn(0x7f96840b18e0 msgr2=0x7f96840b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 shutdown_connections 2024-09-16T15:39:17.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.088+0000 7f968be00640 1 -- 172.21.15.60:0/1846876281 wait complete. 2024-09-16T15:39:17.109 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:39:17.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.277+0000 7fe0d9a00640 1 -- 172.21.15.60:0/2043561284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0d4073de0 msgr2=0x7fe0d4074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:17.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.277+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/2043561284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0d4073de0 0x7fe0d4074240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005fd0 tx=0x7fe0bc02a540 comp rx=0 tx=0).stop 2024-09-16T15:39:17.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.277+0000 7fe0d9a00640 1 -- 172.21.15.60:0/2043561284 shutdown_connections 2024-09-16T15:39:17.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.277+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/2043561284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0d4073de0 0x7fe0d4074240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:17.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.277+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/2043561284 >> v2:172.21.15.100:3300/0 conn(0x7fe0d415f9a0 0x7fe0d415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:17.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.277+0000 7fe0d9a00640 1 -- 172.21.15.60:0/2043561284 >> 172.21.15.60:0/2043561284 conn(0x7fe0d40b18e0 msgr2=0x7fe0d40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:17.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.277+0000 7fe0d9a00640 1 -- 172.21.15.60:0/2043561284 shutdown_connections 2024-09-16T15:39:17.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.277+0000 7fe0d9a00640 1 -- 172.21.15.60:0/2043561284 wait complete. 2024-09-16T15:39:17.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.279+0000 7fe0d9a00640 1 Processor -- start 2024-09-16T15:39:17.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.279+0000 7fe0d9a00640 1 -- start start 2024-09-16T15:39:17.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.280+0000 7fe0d9a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d4073de0 0x7fe0d41528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.280+0000 7fe0d9a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d415c3f0 0x7fe0d40c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.280+0000 7fe0d9a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0d415f9a0 0x7fe0d4147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.280+0000 7fe0d9a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d40cd3d0 con 0x7fe0d415c3f0 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.280+0000 7fe0d9a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d40ccbd0 con 0x7fe0d4073de0 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.280+0000 7fe0d9a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d40cd550 con 0x7fe0d415f9a0 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d415c3f0 0x7fe0d40c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0d2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d4073de0 0x7fe0d41528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d415c3f0 0x7fe0d40c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57224/0 (socket says 172.21.15.60:57224) 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 -- 172.21.15.60:0/285354516 learned_addr learned my addr 172.21.15.60:0/285354516 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:17.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0d3400640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0d415f9a0 0x7fe0d4147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 -- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0d415f9a0 msgr2=0x7fe0d4147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0d415f9a0 0x7fe0d4147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 -- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d4073de0 msgr2=0x7fe0d41528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d4073de0 0x7fe0d41528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 -- 172.21.15.60:0/285354516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d4145360 con 0x7fe0d415c3f0 2024-09-16T15:39:17.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.281+0000 7fe0cbe00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d415c3f0 0x7fe0d40c8f60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009850 tx=0x7fe0bc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:17.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.282+0000 7fe0caa00640 1 -- 172.21.15.60:0/285354516 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc03f070 con 0x7fe0d415c3f0 2024-09-16T15:39:17.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.282+0000 7fe0caa00640 1 -- 172.21.15.60:0/285354516 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0bc0047a0 con 0x7fe0d415c3f0 2024-09-16T15:39:17.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.282+0000 7fe0caa00640 1 -- 172.21.15.60:0/285354516 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc035030 con 0x7fe0d415c3f0 2024-09-16T15:39:17.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.282+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d4145640 con 0x7fe0d415c3f0 2024-09-16T15:39:17.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.282+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d4145c00 con 0x7fe0d415c3f0 2024-09-16T15:39:17.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.283+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d4069360 con 0x7fe0d415c3f0 2024-09-16T15:39:17.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.284+0000 7fe0caa00640 1 -- 172.21.15.60:0/285354516 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe0bc042070 con 0x7fe0d415c3f0 2024-09-16T15:39:17.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.286+0000 7fe0caa00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0b0078850 0x7fe0b007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:17.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.286+0000 7fe0caa00640 1 -- 172.21.15.60:0/285354516 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5531+0+0 (secure 0 0 0) 0x7fe0bc0bb950 con 0x7fe0d415c3f0 2024-09-16T15:39:17.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.286+0000 7fe0d2a00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0b0078850 0x7fe0b007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:17.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.287+0000 7fe0d2a00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0b0078850 0x7fe0b007ad10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009b30 tx=0x7fe0c4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:17.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.290+0000 7fe0caa00640 1 -- 172.21.15.60:0/285354516 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe0bc083980 con 0x7fe0d415c3f0 2024-09-16T15:39:17.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:17.493+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe0d4075e70 con 0x7fe0d415c3f0 2024-09-16T15:39:18.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.089+0000 7fe0caa00640 1 -- 172.21.15.60:0/285354516 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7fe0bc0883b0 con 0x7fe0d415c3f0 2024-09-16T15:39:18.091 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:39:18.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0b0078850 msgr2=0x7fe0b007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:18.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0b0078850 0x7fe0b007ad10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009b30 tx=0x7fe0c4005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:18.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d415c3f0 msgr2=0x7fe0d40c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:18.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d415c3f0 0x7fe0d40c8f60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009850 tx=0x7fe0bc0038a0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 shutdown_connections 2024-09-16T15:39:18.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe0b0078850 0x7fe0b007ad10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0d415f9a0 0x7fe0d4147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0d415c3f0 0x7fe0d40c8f60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.093+0000 7fe0d9a00640 1 --2- 172.21.15.60:0/285354516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0d4073de0 0x7fe0d41528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.094+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 >> 172.21.15.60:0/285354516 conn(0x7fe0d40b18e0 msgr2=0x7fe0d40aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:18.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.094+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 shutdown_connections 2024-09-16T15:39:18.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:18.094+0000 7fe0d9a00640 1 -- 172.21.15.60:0/285354516 wait complete. 2024-09-16T15:39:18.116 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:39:18.117 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:39:18.117 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:39:18.117 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:39:18.117 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:39:18.117 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:39:18.117 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:39:18.118 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:39:18.118 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:39:18.118 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:39:18.118 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:39:18.118 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:39:18.118 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:39:18.118 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:39:18.118 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:39:18.119 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:39:18.135 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:39:18.135 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:39:18.136 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:39:18.136 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:39:18.137 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:39:18.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:39:18.181 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:39:18.181 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:18.183 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:39:18.183 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:18.184 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:39:18.184 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:18.313 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:39:18.316 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:39:18.316 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:39:19.577 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-16T15:39:24.137 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:39:24.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:39:24.142 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:39:24.142 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:24.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:39:24.283 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:39:24.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.314+0000 7f6596000640 1 -- 172.21.15.60:0/3989005266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900725d0 msgr2=0x7f6590073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:24.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.314+0000 7f6596000640 1 --2- 172.21.15.60:0/3989005266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900725d0 0x7f6590073d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f657c005fd0 tx=0x7f657c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.314+0000 7f6596000640 1 -- 172.21.15.60:0/3989005266 shutdown_connections 2024-09-16T15:39:24.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.314+0000 7f6596000640 1 --2- 172.21.15.60:0/3989005266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900725d0 0x7f6590073d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.314+0000 7f6596000640 1 --2- 172.21.15.60:0/3989005266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f659015bc30 0x7f65900b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.314+0000 7f6596000640 1 -- 172.21.15.60:0/3989005266 >> 172.21.15.60:0/3989005266 conn(0x7f65900cfa20 msgr2=0x7f65900cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:24.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.314+0000 7f6596000640 1 -- 172.21.15.60:0/3989005266 shutdown_connections 2024-09-16T15:39:24.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.314+0000 7f6596000640 1 -- 172.21.15.60:0/3989005266 wait complete. 2024-09-16T15:39:24.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.316+0000 7f6596000640 1 Processor -- start 2024-09-16T15:39:24.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.316+0000 7f6596000640 1 -- start start 2024-09-16T15:39:24.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6596000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900d0540 0x7f6590156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:24.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6596000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f659015bc30 0x7f6590150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:24.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6596000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6590151810 0x7f6590154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:24.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6596000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6590074610 con 0x7f6590151810 2024-09-16T15:39:24.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6596000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6590074490 con 0x7f65900d0540 2024-09-16T15:39:24.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6596000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6590074790 con 0x7f659015bc30 2024-09-16T15:39:24.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6594c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900d0540 0x7f6590156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:24.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6595600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6590151810 0x7f6590154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6594c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900d0540 0x7f6590156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:46322/0 (socket says 172.21.15.60:46322) 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f6594c00640 1 -- 172.21.15.60:0/2022834429 learned_addr learned my addr 172.21.15.60:0/2022834429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.317+0000 7f658fe00640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f659015bc30 0x7f6590150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.318+0000 7f6594c00640 1 -- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f659015bc30 msgr2=0x7f6590150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.318+0000 7f6594c00640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f659015bc30 0x7f6590150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.318+0000 7f6594c00640 1 -- 172.21.15.60:0/2022834429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6590151810 msgr2=0x7f6590154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.318+0000 7f6594c00640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6590151810 0x7f6590154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.318+0000 7f6594c00640 1 -- 172.21.15.60:0/2022834429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65901f2700 con 0x7f65900d0540 2024-09-16T15:39:24.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.318+0000 7f6595600640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6590151810 0x7f6590154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:24.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.318+0000 7f6594c00640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900d0540 0x7f6590156e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f658400e790 tx=0x7f658400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:24.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.318+0000 7f658cc00640 1 -- 172.21.15.60:0/2022834429 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658400eea0 con 0x7f65900d0540 2024-09-16T15:39:24.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.319+0000 7f658cc00640 1 -- 172.21.15.60:0/2022834429 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f658400cdb0 con 0x7f65900d0540 2024-09-16T15:39:24.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.319+0000 7f658cc00640 1 -- 172.21.15.60:0/2022834429 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584010690 con 0x7f65900d0540 2024-09-16T15:39:24.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.319+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65901f29e0 con 0x7f65900d0540 2024-09-16T15:39:24.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.319+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65900c9970 con 0x7f65900d0540 2024-09-16T15:39:24.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.320+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659006fc60 con 0x7f65900d0540 2024-09-16T15:39:24.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.321+0000 7f658cc00640 1 -- 172.21.15.60:0/2022834429 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6584010830 con 0x7f65900d0540 2024-09-16T15:39:24.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.322+0000 7f658cc00640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6570078850 0x7f657007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:24.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.322+0000 7f658cc00640 1 -- 172.21.15.60:0/2022834429 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4943+0+0 (secure 0 0 0) 0x7f658409b260 con 0x7f65900d0540 2024-09-16T15:39:24.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.323+0000 7f658fe00640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6570078850 0x7f657007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:24.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.323+0000 7f658fe00640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6570078850 0x7f657007ad10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f657c033000 tx=0x7f657c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:24.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.326+0000 7f658cc00640 1 -- 172.21.15.60:0/2022834429 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f65840636f0 con 0x7f65900d0540 2024-09-16T15:39:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:24.535+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f65900ca1d0 con 0x7f65900d0540 2024-09-16T15:39:26.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.122+0000 7f658cc00640 1 -- 172.21.15.60:0/2022834429 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f6584068120 con 0x7f65900d0540 2024-09-16T15:39:26.124 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:39:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6570078850 msgr2=0x7f657007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6570078850 0x7f657007ad10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f657c033000 tx=0x7f657c004730 comp rx=0 tx=0).stop 2024-09-16T15:39:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900d0540 msgr2=0x7f6590156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900d0540 0x7f6590156e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f658400e790 tx=0x7f658400ec50 comp rx=0 tx=0).stop 2024-09-16T15:39:26.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 shutdown_connections 2024-09-16T15:39:26.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6570078850 0x7f657007ad10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6590151810 0x7f6590154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f659015bc30 0x7f6590150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 --2- 172.21.15.60:0/2022834429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65900d0540 0x7f6590156e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 >> 172.21.15.60:0/2022834429 conn(0x7f65900cfa20 msgr2=0x7f65900b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 shutdown_connections 2024-09-16T15:39:26.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.125+0000 7f6596000640 1 -- 172.21.15.60:0/2022834429 wait complete. 2024-09-16T15:39:26.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:39:26.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-16T15:39:26.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.309+0000 7f8269e00640 1 -- 172.21.15.60:0/4046411998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82640ccf00 msgr2=0x7f82640acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.309+0000 7f8269e00640 1 --2- 172.21.15.60:0/4046411998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82640ccf00 0x7f82640acd30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f824c009830 tx=0x7f824c02a480 comp rx=0 tx=0).stop 2024-09-16T15:39:26.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.309+0000 7f8269e00640 1 -- 172.21.15.60:0/4046411998 shutdown_connections 2024-09-16T15:39:26.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.309+0000 7f8269e00640 1 --2- 172.21.15.60:0/4046411998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82640ccf00 0x7f82640acd30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.309+0000 7f8269e00640 1 --2- 172.21.15.60:0/4046411998 >> v2:172.21.15.100:3300/0 conn(0x7f826415f5a0 0x7f82640b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.309+0000 7f8269e00640 1 -- 172.21.15.60:0/4046411998 >> 172.21.15.60:0/4046411998 conn(0x7f82640b1950 msgr2=0x7f82640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.310+0000 7f8269e00640 1 -- 172.21.15.60:0/4046411998 shutdown_connections 2024-09-16T15:39:26.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.310+0000 7f8269e00640 1 -- 172.21.15.60:0/4046411998 wait complete. 2024-09-16T15:39:26.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.311+0000 7f8269e00640 1 Processor -- start 2024-09-16T15:39:26.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.311+0000 7f8269e00640 1 -- start start 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.312+0000 7f8269e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82640ad270 0x7f82640cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.312+0000 7f8269e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82640ccf00 0x7f8264156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.312+0000 7f8269e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826415f5a0 0x7f826415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.312+0000 7f8269e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f826406f2e0 con 0x7f826415f5a0 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.312+0000 7f8269e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f826406f160 con 0x7f82640ad270 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.312+0000 7f8269e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f826406f460 con 0x7f82640ccf00 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.312+0000 7f8263400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826415f5a0 0x7f826415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8263400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826415f5a0 0x7f826415a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48846/0 (socket says 172.21.15.60:48846) 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8263400640 1 -- 172.21.15.60:0/988308186 learned_addr learned my addr 172.21.15.60:0/988308186 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f825a000640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82640ccf00 0x7f8264156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8262a00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82640ad270 0x7f82640cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8263400640 1 -- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82640ccf00 msgr2=0x7f8264156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8263400640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82640ccf00 0x7f8264156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8263400640 1 -- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82640ad270 msgr2=0x7f82640cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8263400640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82640ad270 0x7f82640cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8263400640 1 -- 172.21.15.60:0/988308186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8264156800 con 0x7f826415f5a0 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8262a00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82640ad270 0x7f82640cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f8263400640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826415f5a0 0x7f826415a0f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f825c00cbc0 tx=0x7f825c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.313+0000 7f825be00640 1 -- 172.21.15.60:0/988308186 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c016020 con 0x7f826415f5a0 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.314+0000 7f825be00640 1 -- 172.21.15.60:0/988308186 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c00add0 con 0x7f826415f5a0 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.314+0000 7f825be00640 1 -- 172.21.15.60:0/988308186 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c012680 con 0x7f826415f5a0 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.314+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8264156ae0 con 0x7f826415f5a0 2024-09-16T15:39:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.314+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82641598f0 con 0x7f826415f5a0 2024-09-16T15:39:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.315+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826406f980 con 0x7f826415f5a0 2024-09-16T15:39:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.316+0000 7f825be00640 1 -- 172.21.15.60:0/988308186 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f825c019020 con 0x7f826415f5a0 2024-09-16T15:39:26.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.317+0000 7f825be00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8240078850 0x7f824007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.317+0000 7f825be00640 1 -- 172.21.15.60:0/988308186 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f825c09aa10 con 0x7f826415f5a0 2024-09-16T15:39:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.318+0000 7f8262a00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8240078850 0x7f824007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.318+0000 7f8262a00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8240078850 0x7f824007ad10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8254009b80 tx=0x7f8254005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.322+0000 7f825be00640 1 -- 172.21.15.60:0/988308186 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f825c020030 con 0x7f826415f5a0 2024-09-16T15:39:26.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.592+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f826415da40 con 0x7f826415f5a0 2024-09-16T15:39:26.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.592+0000 7f825be00640 1 -- 172.21.15.60:0/988308186 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f825c0632a0 con 0x7f826415f5a0 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:39:26.596 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:39:26.597 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:39:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:39:26.599 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:39:26.600 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:39:26.601 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:39:26.602 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:39:26.603 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:39:26.605 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mds", 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDgUOhmmsJ+HxAAsk1C8A1U3D20OfEcZB/6BQ==", 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mds" 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mgr", 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDgUOhm0NJ+HxAAgWbq2ruwmE0afqYJqYBbhw==", 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-osd", 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDgUOhmx+V+HxAAXlUO6MZ0iMgL3ej36KWaTA==", 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-osd" 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd", 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDgUOhmqvV+HxAAn2cAvortJ8hHXswkDaqJFg==", 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.608 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDgUOhmtgV/HxAAL2OJ8EDAebbXlMtdGcBPrQ==", 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rgw", 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDgUOhmPhV/HxAAeY0Q5yS5QM2FtoyqnGfXmA==", 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.609 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:39:26.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.611 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:26.611 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:39:26.611 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:39:26.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8240078850 msgr2=0x7f824007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8240078850 0x7f824007ad10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8254009b80 tx=0x7f8254005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826415f5a0 msgr2=0x7f826415a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826415f5a0 0x7f826415a0f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f825c00cbc0 tx=0x7f825c00a430 comp rx=0 tx=0).stop 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 shutdown_connections 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8240078850 0x7f824007ad10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f826415f5a0 0x7f826415a0f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82640ccf00 0x7f8264156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 --2- 172.21.15.60:0/988308186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82640ad270 0x7f82640cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 >> 172.21.15.60:0/988308186 conn(0x7f82640b1950 msgr2=0x7f826409d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 shutdown_connections 2024-09-16T15:39:26.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.595+0000 7f8269e00640 1 -- 172.21.15.60:0/988308186 wait complete. 2024-09-16T15:39:26.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-16T15:39:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.772+0000 7fa64e000640 1 -- 172.21.15.60:0/128933123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648072df0 msgr2=0x7fa6480732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.772+0000 7fa64e000640 1 --2- 172.21.15.60:0/128933123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648072df0 0x7fa6480732a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa640005fd0 tx=0x7fa64002aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.773+0000 7fa64e000640 1 -- 172.21.15.60:0/128933123 shutdown_connections 2024-09-16T15:39:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.773+0000 7fa64e000640 1 --2- 172.21.15.60:0/128933123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648072df0 0x7fa6480732a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.773+0000 7fa64e000640 1 --2- 172.21.15.60:0/128933123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa648161550 0x7fa64806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.773+0000 7fa64e000640 1 -- 172.21.15.60:0/128933123 >> 172.21.15.60:0/128933123 conn(0x7fa6480b1950 msgr2=0x7fa6480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.773+0000 7fa64e000640 1 -- 172.21.15.60:0/128933123 shutdown_connections 2024-09-16T15:39:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.773+0000 7fa64e000640 1 -- 172.21.15.60:0/128933123 wait complete. 2024-09-16T15:39:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.774+0000 7fa64e000640 1 Processor -- start 2024-09-16T15:39:26.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.774+0000 7fa64e000640 1 -- start start 2024-09-16T15:39:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.775+0000 7fa64e000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa648072df0 0x7fa6480c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.775+0000 7fa64e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6480af3d0 0x7fa6480cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.775+0000 7fa64e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648161550 0x7fa648151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.775+0000 7fa64e000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa6480737e0 con 0x7fa648161550 2024-09-16T15:39:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.775+0000 7fa64e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa64806fdb0 con 0x7fa6480af3d0 2024-09-16T15:39:26.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.775+0000 7fa64e000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa648073960 con 0x7fa648072df0 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648161550 0x7fa648151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648161550 0x7fa648151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48860/0 (socket says 172.21.15.60:48860) 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647e00640 1 -- 172.21.15.60:0/837462000 learned_addr learned my addr 172.21.15.60:0/837462000 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647400640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa648072df0 0x7fa6480c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa63ea00640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6480af3d0 0x7fa6480cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647e00640 1 -- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa648072df0 msgr2=0x7fa6480c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647e00640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa648072df0 0x7fa6480c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647e00640 1 -- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6480af3d0 msgr2=0x7fa6480cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647e00640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6480af3d0 0x7fa6480cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.776+0000 7fa647e00640 1 -- 172.21.15.60:0/837462000 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6480cc150 con 0x7fa648161550 2024-09-16T15:39:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.777+0000 7fa647e00640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648161550 0x7fa648151c60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa640006010 tx=0x7fa640003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.777+0000 7fa63fe00640 1 -- 172.21.15.60:0/837462000 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640047070 con 0x7fa648161550 2024-09-16T15:39:26.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.777+0000 7fa63fe00640 1 -- 172.21.15.60:0/837462000 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa64002f670 con 0x7fa648161550 2024-09-16T15:39:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.777+0000 7fa63fe00640 1 -- 172.21.15.60:0/837462000 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640033050 con 0x7fa648161550 2024-09-16T15:39:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.777+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6480cc430 con 0x7fa648161550 2024-09-16T15:39:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.777+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6480cc9f0 con 0x7fa648161550 2024-09-16T15:39:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.778+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6480b3030 con 0x7fa648161550 2024-09-16T15:39:26.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.779+0000 7fa63fe00640 1 -- 172.21.15.60:0/837462000 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa64004a070 con 0x7fa648161550 2024-09-16T15:39:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.780+0000 7fa63fe00640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa624078850 0x7fa62407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.780+0000 7fa63fe00640 1 -- 172.21.15.60:0/837462000 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fa6400bb730 con 0x7fa648161550 2024-09-16T15:39:26.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.780+0000 7fa647400640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa624078850 0x7fa62407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.781+0000 7fa647400640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa624078850 0x7fa62407ad10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa638009b30 tx=0x7fa638005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:26.784+0000 7fa63fe00640 1 -- 172.21.15.60:0/837462000 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa6480b3030 con 0x7fa648161550 2024-09-16T15:39:27.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.052+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fa648074390 con 0x7fa648161550 2024-09-16T15:39:27.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.055+0000 7fa63fe00640 1 -- 172.21.15.60:0/837462000 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fa640083a00 con 0x7fa648161550 2024-09-16T15:39:27.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.057+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa624078850 msgr2=0x7fa62407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.057+0000 7fa64e000640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa624078850 0x7fa62407ad10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa638009b30 tx=0x7fa638005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:27.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.057+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648161550 msgr2=0x7fa648151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.057+0000 7fa64e000640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648161550 0x7fa648151c60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa640006010 tx=0x7fa640003ef0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.058+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 shutdown_connections 2024-09-16T15:39:27.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.058+0000 7fa64e000640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa624078850 0x7fa62407ad10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.058+0000 7fa64e000640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa648161550 0x7fa648151c60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.058+0000 7fa64e000640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6480af3d0 0x7fa6480cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.058+0000 7fa64e000640 1 --2- 172.21.15.60:0/837462000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa648072df0 0x7fa6480c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.058+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 >> 172.21.15.60:0/837462000 conn(0x7fa6480b1950 msgr2=0x7fa64806ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.058+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 shutdown_connections 2024-09-16T15:39:27.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.058+0000 7fa64e000640 1 -- 172.21.15.60:0/837462000 wait complete. 2024-09-16T15:39:27.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-16T15:39:27.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.237+0000 7fe99ae00640 1 -- 172.21.15.60:0/3579008938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe978003fc0 msgr2=0x7fe978004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.237+0000 7fe99ae00640 1 --2- 172.21.15.60:0/3579008938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe978003fc0 0x7fe978004800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe97c006f40 tx=0x7fe978005830 comp rx=0 tx=0).stop 2024-09-16T15:39:27.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.237+0000 7fe998e00640 1 -- 172.21.15.60:0/3579008938 reap_dead start 2024-09-16T15:39:27.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.238+0000 7fe99ae00640 1 -- 172.21.15.60:0/3579008938 shutdown_connections 2024-09-16T15:39:27.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.238+0000 7fe99ae00640 1 -- 172.21.15.60:0/3579008938 >> 172.21.15.60:0/3579008938 conn(0x7fe9940b1950 msgr2=0x7fe9940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.238+0000 7fe99ae00640 1 -- 172.21.15.60:0/3579008938 shutdown_connections 2024-09-16T15:39:27.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.238+0000 7fe99ae00640 1 -- 172.21.15.60:0/3579008938 wait complete. 2024-09-16T15:39:27.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.239+0000 7fe99ae00640 1 Processor -- start 2024-09-16T15:39:27.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.239+0000 7fe99ae00640 1 -- start start 2024-09-16T15:39:27.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.240+0000 7fe99ae00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe978003fc0 0x7fe9940d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.240+0000 7fe99ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe99409aa00 0x7fe99409ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.240+0000 7fe99ae00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9941f7c00 0x7fe9941f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.240+0000 7fe99ae00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe99406ef50 con 0x7fe9941f7c00 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.240+0000 7fe99ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe99406fdb0 con 0x7fe99409aa00 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.240+0000 7fe99ae00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe99406f0d0 con 0x7fe978003fc0 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9941f7c00 0x7fe9941f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9941f7c00 0x7fe9941f8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48882/0 (socket says 172.21.15.60:48882) 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 -- 172.21.15.60:0/1436945383 learned_addr learned my addr 172.21.15.60:0/1436945383 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe993e00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe978003fc0 0x7fe9940d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe98b400640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe99409aa00 0x7fe99409ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 -- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe978003fc0 msgr2=0x7fe9940d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe978003fc0 0x7fe9940d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 -- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe99409aa00 msgr2=0x7fe99409ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe99409aa00 0x7fe99409ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 -- 172.21.15.60:0/1436945383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99409d210 con 0x7fe9941f7c00 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.241+0000 7fe998e00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9941f7c00 0x7fe9941f8080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe994075130 tx=0x7fe98c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.242+0000 7fe990c00640 1 -- 172.21.15.60:0/1436945383 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c015020 con 0x7fe9941f7c00 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.242+0000 7fe990c00640 1 -- 172.21.15.60:0/1436945383 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe98c0119a0 con 0x7fe9941f7c00 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.242+0000 7fe990c00640 1 -- 172.21.15.60:0/1436945383 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c011df0 con 0x7fe9941f7c00 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.242+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99409d4d0 con 0x7fe9941f7c00 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.242+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99409da90 con 0x7fe9941f7c00 2024-09-16T15:39:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.243+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9940732a0 con 0x7fe9941f7c00 2024-09-16T15:39:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.244+0000 7fe990c00640 1 -- 172.21.15.60:0/1436945383 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe98c018020 con 0x7fe9941f7c00 2024-09-16T15:39:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.245+0000 7fe990c00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe970078880 0x7fe97007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.245+0000 7fe990c00640 1 -- 172.21.15.60:0/1436945383 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fe98c09ac10 con 0x7fe9941f7c00 2024-09-16T15:39:27.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.246+0000 7fe993e00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe970078880 0x7fe97007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.246+0000 7fe993e00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe970078880 0x7fe97007ad40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe984009c50 tx=0x7fe984005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.249+0000 7fe990c00640 1 -- 172.21.15.60:0/1436945383 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe98c01f030 con 0x7fe9941f7c00 2024-09-16T15:39:27.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.519+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fe99406f610 con 0x7fe9941f7c00 2024-09-16T15:39:27.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.521+0000 7fe990c00640 1 -- 172.21.15.60:0/1436945383 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fe98c063530 con 0x7fe9941f7c00 2024-09-16T15:39:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe970078880 msgr2=0x7fe97007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe970078880 0x7fe97007ad40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe984009c50 tx=0x7fe984005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9941f7c00 msgr2=0x7fe9941f8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9941f7c00 0x7fe9941f8080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe994075130 tx=0x7fe98c001a80 comp rx=0 tx=0).stop 2024-09-16T15:39:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 shutdown_connections 2024-09-16T15:39:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe970078880 0x7fe97007ad40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9941f7c00 0x7fe9941f8080 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe99409aa00 0x7fe99409ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 --2- 172.21.15.60:0/1436945383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe978003fc0 0x7fe9940d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 >> 172.21.15.60:0/1436945383 conn(0x7fe9940b1950 msgr2=0x7fe9940ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 shutdown_connections 2024-09-16T15:39:27.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.525+0000 7fe99ae00640 1 -- 172.21.15.60:0/1436945383 wait complete. 2024-09-16T15:39:27.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-16T15:39:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.703+0000 7f6915400640 1 -- 172.21.15.60:0/4292459613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 msgr2=0x7f68f40051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.703+0000 7f6915400640 1 --2- 172.21.15.60:0/4292459613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 0x7f68f40051a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6908009ed0 tx=0x7f68f4005830 comp rx=0 tx=0).stop 2024-09-16T15:39:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.703+0000 7f690f200640 1 -- 172.21.15.60:0/4292459613 reap_dead start 2024-09-16T15:39:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.704+0000 7f6915400640 1 -- 172.21.15.60:0/4292459613 shutdown_connections 2024-09-16T15:39:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.704+0000 7f6915400640 1 -- 172.21.15.60:0/4292459613 >> 172.21.15.60:0/4292459613 conn(0x7f69100b1950 msgr2=0x7f69100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.704+0000 7f6915400640 1 -- 172.21.15.60:0/4292459613 shutdown_connections 2024-09-16T15:39:27.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.704+0000 7f6915400640 1 -- 172.21.15.60:0/4292459613 wait complete. 2024-09-16T15:39:27.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.705+0000 7f6915400640 1 Processor -- start 2024-09-16T15:39:27.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.706+0000 7f6915400640 1 -- start start 2024-09-16T15:39:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.706+0000 7f6915400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 0x7f6910148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.706+0000 7f6915400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69101f88a0 0x7f691015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f6915400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6910159a60 0x7f6910159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f6915400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6910074840 con 0x7f68f4004d40 2024-09-16T15:39:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f6915400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69100746c0 con 0x7f6910159a60 2024-09-16T15:39:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f6915400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f691006f160 con 0x7f69101f88a0 2024-09-16T15:39:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 0x7f6910148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 0x7f6910148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48906/0 (socket says 172.21.15.60:48906) 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690e800640 1 -- 172.21.15.60:0/3914366134 learned_addr learned my addr 172.21.15.60:0/3914366134 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690f200640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6910159a60 0x7f6910159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f6905e00640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69101f88a0 0x7f691015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690e800640 1 -- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69101f88a0 msgr2=0x7f691015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690e800640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69101f88a0 0x7f691015ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690e800640 1 -- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6910159a60 msgr2=0x7f6910159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690e800640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6910159a60 0x7f6910159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690e800640 1 -- 172.21.15.60:0/3914366134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691015c2d0 con 0x7f68f4004d40 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f6905e00640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69101f88a0 0x7f691015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.707+0000 7f690f200640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6910159a60 0x7f6910159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.708+0000 7f690e800640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 0x7f6910148710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6900001260 tx=0x7f6900001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.708+0000 7f6907400640 1 -- 172.21.15.60:0/3914366134 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900015020 con 0x7f68f4004d40 2024-09-16T15:39:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.708+0000 7f6907400640 1 -- 172.21.15.60:0/3914366134 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6900011900 con 0x7f68f4004d40 2024-09-16T15:39:27.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.708+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691015c5b0 con 0x7f68f4004d40 2024-09-16T15:39:27.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.708+0000 7f6907400640 1 -- 172.21.15.60:0/3914366134 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900011d50 con 0x7f68f4004d40 2024-09-16T15:39:27.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.708+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691015cb70 con 0x7f68f4004d40 2024-09-16T15:39:27.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.709+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691006f6c0 con 0x7f68f4004d40 2024-09-16T15:39:27.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.710+0000 7f6907400640 1 -- 172.21.15.60:0/3914366134 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6900018020 con 0x7f68f4004d40 2024-09-16T15:39:27.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.711+0000 7f6907400640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68ec078850 0x7f68ec07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.711+0000 7f6907400640 1 -- 172.21.15.60:0/3914366134 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f690009b130 con 0x7f68f4004d40 2024-09-16T15:39:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.712+0000 7f6905e00640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68ec078850 0x7f68ec07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.712+0000 7f6905e00640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68ec078850 0x7f68ec07ad10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68f8000fb0 tx=0x7f68f8006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:27.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.715+0000 7f6907400640 1 -- 172.21.15.60:0/3914366134 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f690001f020 con 0x7f68f4004d40 2024-09-16T15:39:27.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.981+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f69100ceb30 con 0x7f68f4004d40 2024-09-16T15:39:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.983+0000 7f6907400640 1 -- 172.21.15.60:0/3914366134 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f6900063ae0 con 0x7f68f4004d40 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.985+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68ec078850 msgr2=0x7f68ec07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.985+0000 7f6915400640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68ec078850 0x7f68ec07ad10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68f8000fb0 tx=0x7f68f8006720 comp rx=0 tx=0).stop 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.985+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 msgr2=0x7f6910148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.985+0000 7f6915400640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 0x7f6910148710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6900001260 tx=0x7f6900001720 comp rx=0 tx=0).stop 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.986+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 shutdown_connections 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.986+0000 7f6915400640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f68ec078850 0x7f68ec07ad10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.986+0000 7f6915400640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6910159a60 0x7f6910159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.986+0000 7f6915400640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69101f88a0 0x7f691015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.986+0000 7f6915400640 1 --2- 172.21.15.60:0/3914366134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f4004d40 0x7f6910148710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.986+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 >> 172.21.15.60:0/3914366134 conn(0x7f69100b1950 msgr2=0x7f691009c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.986+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 shutdown_connections 2024-09-16T15:39:27.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:27.986+0000 7f6915400640 1 -- 172.21.15.60:0/3914366134 wait complete. 2024-09-16T15:39:28.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-16T15:39:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.164+0000 7f6bc2800640 1 -- 172.21.15.60:0/3498431264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc073fb0 msgr2=0x7f6bbc074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.164+0000 7f6bc2800640 1 --2- 172.21.15.60:0/3498431264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc073fb0 0x7f6bbc074420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0cfbb0 tx=0x7f6bb402aa30 comp rx=0 tx=0).stop 2024-09-16T15:39:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.164+0000 7f6bc2800640 1 -- 172.21.15.60:0/3498431264 shutdown_connections 2024-09-16T15:39:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.164+0000 7f6bc2800640 1 --2- 172.21.15.60:0/3498431264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc073fb0 0x7f6bbc074420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.164+0000 7f6bc2800640 1 --2- 172.21.15.60:0/3498431264 >> v2:172.21.15.100:3300/0 conn(0x7f6bbc0ccf00 0x7f6bbc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.164+0000 7f6bc2800640 1 -- 172.21.15.60:0/3498431264 >> 172.21.15.60:0/3498431264 conn(0x7f6bbc0b1950 msgr2=0x7f6bbc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.164+0000 7f6bc2800640 1 -- 172.21.15.60:0/3498431264 shutdown_connections 2024-09-16T15:39:28.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.165+0000 7f6bc2800640 1 -- 172.21.15.60:0/3498431264 wait complete. 2024-09-16T15:39:28.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.166+0000 7f6bc2800640 1 Processor -- start 2024-09-16T15:39:28.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.166+0000 7f6bc2800640 1 -- start start 2024-09-16T15:39:28.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.167+0000 7f6bc2800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bbc073fb0 0x7f6bbc0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.167+0000 7f6bc2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc0ccf00 0x7f6bbc0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.167+0000 7f6bc2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bbc15f5a0 0x7f6bbc0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.167+0000 7f6bc2800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc06f2e0 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.167+0000 7f6bc2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc06f160 con 0x7f6bbc15f5a0 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.167+0000 7f6bc2800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc06f460 con 0x7f6bbc073fb0 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc0ccf00 0x7f6bbc0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc0ccf00 0x7f6bbc0b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48916/0 (socket says 172.21.15.60:48916) 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 -- 172.21.15.60:0/4033544274 learned_addr learned my addr 172.21.15.60:0/4033544274 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bbb400640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bbc073fb0 0x7f6bbc0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bbbe00640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bbc15f5a0 0x7f6bbc0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 -- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bbc073fb0 msgr2=0x7f6bbc0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bbc073fb0 0x7f6bbc0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 -- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bbc15f5a0 msgr2=0x7f6bbc0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bbc15f5a0 0x7f6bbc0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 -- 172.21.15.60:0/4033544274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc15b9b0 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.168+0000 7f6bb2a00640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc0ccf00 0x7f6bbc0b3c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400d950 tx=0x7f6ba400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.169+0000 7f6bb3e00640 1 -- 172.21.15.60:0/4033544274 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4017070 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.169+0000 7f6bb3e00640 1 -- 172.21.15.60:0/4033544274 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ba4009d20 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.169+0000 7f6bb3e00640 1 -- 172.21.15.60:0/4033544274 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4004280 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.169+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc15bc90 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.169+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc15c250 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.170+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc06f9f0 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.171+0000 7f6bb3e00640 1 -- 172.21.15.60:0/4033544274 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6ba40035f0 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.172+0000 7f6bb3e00640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6b98078850 0x7f6b9807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.173+0000 7f6bb3e00640 1 -- 172.21.15.60:0/4033544274 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f6ba409a6f0 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.173+0000 7f6bbb400640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6b98078850 0x7f6b9807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.174+0000 7f6bbb400640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6b98078850 0x7f6b9807ad10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6bac001720 tx=0x7f6bac009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:28.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.177+0000 7f6bb3e00640 1 -- 172.21.15.60:0/4033544274 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6ba40629c0 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.446+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f6bbc0cbf60 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.448+0000 7f6bb3e00640 1 -- 172.21.15.60:0/4033544274 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f6ba40673f0 con 0x7f6bbc0ccf00 2024-09-16T15:39:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6b98078850 msgr2=0x7f6b9807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6b98078850 0x7f6b9807ad10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6bac001720 tx=0x7f6bac009040 comp rx=0 tx=0).stop 2024-09-16T15:39:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc0ccf00 msgr2=0x7f6bbc0b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc0ccf00 0x7f6bbc0b3c80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400d950 tx=0x7f6ba400de10 comp rx=0 tx=0).stop 2024-09-16T15:39:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 shutdown_connections 2024-09-16T15:39:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6b98078850 0x7f6b9807ad10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bbc15f5a0 0x7f6bbc0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bbc0ccf00 0x7f6bbc0b3c80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 --2- 172.21.15.60:0/4033544274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bbc073fb0 0x7f6bbc0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 >> 172.21.15.60:0/4033544274 conn(0x7f6bbc0b1950 msgr2=0x7f6bbc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 shutdown_connections 2024-09-16T15:39:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.451+0000 7f6bc2800640 1 -- 172.21.15.60:0/4033544274 wait complete. 2024-09-16T15:39:28.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 -- 172.21.15.60:0/2651640630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 msgr2=0x7f20040acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 --2- 172.21.15.60:0/2651640630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 0x7f20040acd30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2004076720 tx=0x7f1fec02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 -- 172.21.15.60:0/2651640630 shutdown_connections 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 --2- 172.21.15.60:0/2651640630 >> v2:172.21.15.100:3300/0 conn(0x7f20040ad270 0x7f2004073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 --2- 172.21.15.60:0/2651640630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 0x7f20040acd30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 --2- 172.21.15.60:0/2651640630 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f200415f5a0 0x7f20040b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 -- 172.21.15.60:0/2651640630 >> 172.21.15.60:0/2651640630 conn(0x7f20040b1950 msgr2=0x7f20040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 -- 172.21.15.60:0/2651640630 shutdown_connections 2024-09-16T15:39:28.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.629+0000 7f200a200640 1 -- 172.21.15.60:0/2651640630 wait complete. 2024-09-16T15:39:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.631+0000 7f200a200640 1 Processor -- start 2024-09-16T15:39:28.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.631+0000 7f200a200640 1 -- start start 2024-09-16T15:39:28.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f200a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20040ad270 0x7f2004159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f200a200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 0x7f200414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f200a200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f200415f5a0 0x7f200409c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f200a200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f200406f160 con 0x7f20040ccf00 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f200a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2004074780 con 0x7f20040ad270 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f200a200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f200406f2e0 con 0x7f200415f5a0 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f1ffaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 0x7f200414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f1ffaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 0x7f200414fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48936/0 (socket says 172.21.15.60:48936) 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f1ffaa00640 1 -- 172.21.15.60:0/2287886664 learned_addr learned my addr 172.21.15.60:0/2287886664 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f2003400640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20040ad270 0x7f2004159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.632+0000 7f2003e00640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f200415f5a0 0x7f200409c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffaa00640 1 -- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f200415f5a0 msgr2=0x7f200409c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffaa00640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f200415f5a0 0x7f200409c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffaa00640 1 -- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20040ad270 msgr2=0x7f2004159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffaa00640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20040ad270 0x7f2004159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffaa00640 1 -- 172.21.15.60:0/2287886664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200415a070 con 0x7f20040ccf00 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f2003400640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20040ad270 0x7f2004159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f2003e00640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f200415f5a0 0x7f200409c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffaa00640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 0x7f200414fe10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009850 tx=0x7f1fec0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffbe00640 1 -- 172.21.15.60:0/2287886664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec03f070 con 0x7f20040ccf00 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffbe00640 1 -- 172.21.15.60:0/2287886664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fec004890 con 0x7f20040ccf00 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f1ffbe00640 1 -- 172.21.15.60:0/2287886664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec035030 con 0x7f20040ccf00 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.633+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200415a350 con 0x7f20040ccf00 2024-09-16T15:39:28.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.634+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200415a910 con 0x7f20040ccf00 2024-09-16T15:39:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.635+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200406f870 con 0x7f20040ccf00 2024-09-16T15:39:28.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.636+0000 7f1ffbe00640 1 -- 172.21.15.60:0/2287886664 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1fec042070 con 0x7f20040ccf00 2024-09-16T15:39:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.637+0000 7f1ffbe00640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1fe0078850 0x7f1fe007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.637+0000 7f1ffbe00640 1 -- 172.21.15.60:0/2287886664 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f1fec0bb4f0 con 0x7f20040ccf00 2024-09-16T15:39:28.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.637+0000 7f2003400640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1fe0078850 0x7f1fe007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.638+0000 7f2003400640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1fe0078850 0x7f1fe007ad10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400ab40 tx=0x7f1ff400a520 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:28.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.641+0000 7f1ffbe00640 1 -- 172.21.15.60:0/2287886664 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1fec083f20 con 0x7f20040ccf00 2024-09-16T15:39:28.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.911+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f20040ae9f0 con 0x7f20040ccf00 2024-09-16T15:39:28.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.913+0000 7f1ffbe00640 1 -- 172.21.15.60:0/2287886664 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f1fec083820 con 0x7f20040ccf00 2024-09-16T15:39:28.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1fe0078850 msgr2=0x7f1fe007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1fe0078850 0x7f1fe007ad10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400ab40 tx=0x7f1ff400a520 comp rx=0 tx=0).stop 2024-09-16T15:39:28.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 msgr2=0x7f200414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 0x7f200414fe10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009850 tx=0x7f1fec0039e0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 shutdown_connections 2024-09-16T15:39:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1fe0078850 0x7f1fe007ad10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f200415f5a0 0x7f200409c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20040ccf00 0x7f200414fe10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 --2- 172.21.15.60:0/2287886664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20040ad270 0x7f2004159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 >> 172.21.15.60:0/2287886664 conn(0x7f20040b1950 msgr2=0x7f20041575b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 shutdown_connections 2024-09-16T15:39:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:28.916+0000 7f200a200640 1 -- 172.21.15.60:0/2287886664 wait complete. 2024-09-16T15:39:28.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-16T15:39:29.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.095+0000 7fa9a6400640 1 -- 172.21.15.60:0/1151834712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9a00d0540 msgr2=0x7fa9a00b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.095+0000 7fa9a6400640 1 --2- 172.21.15.60:0/1151834712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9a00d0540 0x7fa9a00b28f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa998005fd0 tx=0x7fa99802a940 comp rx=0 tx=0).stop 2024-09-16T15:39:29.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.095+0000 7fa9a6400640 1 -- 172.21.15.60:0/1151834712 shutdown_connections 2024-09-16T15:39:29.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.095+0000 7fa9a6400640 1 --2- 172.21.15.60:0/1151834712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9a00d0540 0x7fa9a00b28f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.095+0000 7fa9a6400640 1 -- 172.21.15.60:0/1151834712 >> 172.21.15.60:0/1151834712 conn(0x7fa9a00cfa20 msgr2=0x7fa9a00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:29.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.095+0000 7fa9a6400640 1 -- 172.21.15.60:0/1151834712 shutdown_connections 2024-09-16T15:39:29.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.095+0000 7fa9a6400640 1 -- 172.21.15.60:0/1151834712 wait complete. 2024-09-16T15:39:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.097+0000 7fa9a6400640 1 Processor -- start 2024-09-16T15:39:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.097+0000 7fa9a6400640 1 -- start start 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a6400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a00d0540 0x7fa9a00a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9a015bc30 0x7fa9a0146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a6400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9a01f6100 0x7fa9a01f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a6400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0074660 con 0x7fa9a00d0540 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a006fa30 con 0x7fa9a015bc30 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a6400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a00c7be0 con 0x7fa9a01f6100 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a00d0540 0x7fa9a00a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a00d0540 0x7fa9a00a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41886/0 (socket says 172.21.15.60:41886) 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a5000640 1 -- 172.21.15.60:0/2405461043 learned_addr learned my addr 172.21.15.60:0/2405461043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa997e00640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9a015bc30 0x7fa9a0146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.098+0000 7fa9a5a00640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9a01f6100 0x7fa9a01f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa9a5000640 1 -- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9a01f6100 msgr2=0x7fa9a01f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa9a5000640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9a01f6100 0x7fa9a01f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa9a5000640 1 -- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9a015bc30 msgr2=0x7fa9a0146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa9a5000640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9a015bc30 0x7fa9a0146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa9a5000640 1 -- 172.21.15.60:0/2405461043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a00c7e80 con 0x7fa9a00d0540 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa9a5000640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a00d0540 0x7fa9a00a9470 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa99000c430 tx=0x7fa99000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa99d600640 1 -- 172.21.15.60:0/2405461043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990017070 con 0x7fa9a00d0540 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa99d600640 1 -- 172.21.15.60:0/2405461043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9900053e0 con 0x7fa9a00d0540 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a00c8160 con 0x7fa9a00d0540 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa99d600640 1 -- 172.21.15.60:0/2405461043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990003850 con 0x7fa9a00d0540 2024-09-16T15:39:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.099+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a00a9a50 con 0x7fa9a00d0540 2024-09-16T15:39:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.100+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a00b28f0 con 0x7fa9a00d0540 2024-09-16T15:39:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.102+0000 7fa99d600640 1 -- 172.21.15.60:0/2405461043 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa99001a070 con 0x7fa9a00d0540 2024-09-16T15:39:29.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.103+0000 7fa99d600640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa97c078850 0x7fa97c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.103+0000 7fa99d600640 1 -- 172.21.15.60:0/2405461043 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fa990099840 con 0x7fa9a00d0540 2024-09-16T15:39:29.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.103+0000 7fa997e00640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa97c078850 0x7fa97c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.104+0000 7fa997e00640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa97c078850 0x7fa97c07ad10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa988007f20 tx=0x7fa988007eb0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:29.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.107+0000 7fa99d600640 1 -- 172.21.15.60:0/2405461043 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa990061c40 con 0x7fa9a00d0540 2024-09-16T15:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.375+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fa9a00b5700 con 0x7fa9a00d0540 2024-09-16T15:39:29.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.377+0000 7fa99d600640 1 -- 172.21.15.60:0/2405461043 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fa990066670 con 0x7fa9a00d0540 2024-09-16T15:39:29.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa97c078850 msgr2=0x7fa97c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa97c078850 0x7fa97c07ad10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa988007f20 tx=0x7fa988007eb0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a00d0540 msgr2=0x7fa9a00a9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a00d0540 0x7fa9a00a9470 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa99000c430 tx=0x7fa99000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 shutdown_connections 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa97c078850 0x7fa97c07ad10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9a01f6100 0x7fa9a01f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9a015bc30 0x7fa9a0146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 --2- 172.21.15.60:0/2405461043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a00d0540 0x7fa9a00a9470 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 >> 172.21.15.60:0/2405461043 conn(0x7fa9a00cfa20 msgr2=0x7fa9a00cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 shutdown_connections 2024-09-16T15:39:29.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.380+0000 7fa9a6400640 1 -- 172.21.15.60:0/2405461043 wait complete. 2024-09-16T15:39:29.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:39:29.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.557+0000 7fb13f200640 1 -- 172.21.15.60:0/3525057690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb140161440 msgr2=0x7fb1400b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.557+0000 7fb13f200640 1 --2- 172.21.15.60:0/3525057690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb140161440 0x7fb1400b5370 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb130009830 tx=0x7fb13002a470 comp rx=0 tx=0).stop 2024-09-16T15:39:29.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.557+0000 7fb13f200640 1 -- 172.21.15.60:0/3525057690 shutdown_connections 2024-09-16T15:39:29.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.557+0000 7fb13f200640 1 --2- 172.21.15.60:0/3525057690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1400b4050 0x7fb1400b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.557+0000 7fb13f200640 1 --2- 172.21.15.60:0/3525057690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb140161440 0x7fb1400b5370 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.557+0000 7fb13f200640 1 -- 172.21.15.60:0/3525057690 >> 172.21.15.60:0/3525057690 conn(0x7fb1400d3df0 msgr2=0x7fb1400d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:29.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.557+0000 7fb13f200640 1 -- 172.21.15.60:0/3525057690 shutdown_connections 2024-09-16T15:39:29.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.557+0000 7fb13f200640 1 -- 172.21.15.60:0/3525057690 wait complete. 2024-09-16T15:39:29.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.558+0000 7fb13f200640 1 Processor -- start 2024-09-16T15:39:29.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.559+0000 7fb13f200640 1 -- start start 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.559+0000 7fb13f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1400b34b0 0x7fb1400ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.559+0000 7fb13f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1400b4050 0x7fb14014dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.559+0000 7fb13f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb140161440 0x7fb14014e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.559+0000 7fb13f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb1400b2040 con 0x7fb1400b4050 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.559+0000 7fb13f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1400b1ec0 con 0x7fb1400b34b0 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.559+0000 7fb13f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb1400b21c0 con 0x7fb140161440 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1400b4050 0x7fb14014dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1400b4050 0x7fb14014dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41908/0 (socket says 172.21.15.60:41908) 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 -- 172.21.15.60:0/3261693519 learned_addr learned my addr 172.21.15.60:0/3261693519 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb13de00640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1400b34b0 0x7fb1400ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb13e800640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb140161440 0x7fb14014e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 -- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb140161440 msgr2=0x7fb14014e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb140161440 0x7fb14014e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 -- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1400b34b0 msgr2=0x7fb1400ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1400b34b0 0x7fb1400ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 -- 172.21.15.60:0/3261693519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1400c9370 con 0x7fb1400b4050 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.560+0000 7fb137e00640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1400b4050 0x7fb14014dd10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c480 tx=0x7fb12800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.561+0000 7fb136000640 1 -- 172.21.15.60:0/3261693519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128016020 con 0x7fb1400b4050 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.561+0000 7fb136000640 1 -- 172.21.15.60:0/3261693519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb128012940 con 0x7fb1400b4050 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.561+0000 7fb136000640 1 -- 172.21.15.60:0/3261693519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128012d90 con 0x7fb1400b4050 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.561+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1400c9650 con 0x7fb1400b4050 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.561+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1400c9ef0 con 0x7fb1400b4050 2024-09-16T15:39:29.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.562+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1400b1770 con 0x7fb1400b4050 2024-09-16T15:39:29.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.563+0000 7fb136000640 1 -- 172.21.15.60:0/3261693519 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb128019020 con 0x7fb1400b4050 2024-09-16T15:39:29.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.564+0000 7fb136000640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb11c078850 0x7fb11c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.565+0000 7fb136000640 1 -- 172.21.15.60:0/3261693519 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fb12809a250 con 0x7fb1400b4050 2024-09-16T15:39:29.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.565+0000 7fb13de00640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb11c078850 0x7fb11c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.566+0000 7fb13de00640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb11c078850 0x7fb11c07ad10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb130034000 tx=0x7fb130004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:29.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.569+0000 7fb136000640 1 -- 172.21.15.60:0/3261693519 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb128020030 con 0x7fb1400b4050 2024-09-16T15:39:29.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.765+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb1400c9820 con 0x7fb1400b4050 2024-09-16T15:39:29.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.766+0000 7fb136000640 1 -- 172.21.15.60:0/3261693519 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) ==== 84+0+65 (secure 0 0 0) 0x7fb128062d60 con 0x7fb1400b4050 2024-09-16T15:39:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:29.768 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:39:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:29.768 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:39:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:39:29.769 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:29.769 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb11c078850 msgr2=0x7fb11c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb11c078850 0x7fb11c07ad10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb130034000 tx=0x7fb130004730 comp rx=0 tx=0).stop 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1400b4050 msgr2=0x7fb14014dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1400b4050 0x7fb14014dd10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb12800c480 tx=0x7fb12800c940 comp rx=0 tx=0).stop 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 shutdown_connections 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb11c078850 0x7fb11c07ad10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb140161440 0x7fb14014e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1400b4050 0x7fb14014dd10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 --2- 172.21.15.60:0/3261693519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1400b34b0 0x7fb1400ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 >> 172.21.15.60:0/3261693519 conn(0x7fb1400d3df0 msgr2=0x7fb1400d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:29.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 shutdown_connections 2024-09-16T15:39:29.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.768+0000 7fb13f200640 1 -- 172.21.15.60:0/3261693519 wait complete. 2024-09-16T15:39:29.786 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:39:29.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:39:29.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.950+0000 7fafdb600640 1 -- 172.21.15.60:0/3053306945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd4159aa0 msgr2=0x7fafd40741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.950+0000 7fafdb600640 1 --2- 172.21.15.60:0/3053306945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd4159aa0 0x7fafd40741a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009860 tx=0x7fafc802a540 comp rx=0 tx=0).stop 2024-09-16T15:39:29.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.950+0000 7fafdb600640 1 -- 172.21.15.60:0/3053306945 shutdown_connections 2024-09-16T15:39:29.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.950+0000 7fafdb600640 1 --2- 172.21.15.60:0/3053306945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafd415cff0 0x7fafd406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.950+0000 7fafdb600640 1 --2- 172.21.15.60:0/3053306945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd4159aa0 0x7fafd40741a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.950+0000 7fafdb600640 1 -- 172.21.15.60:0/3053306945 >> 172.21.15.60:0/3053306945 conn(0x7fafd40b18e0 msgr2=0x7fafd40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:29.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.950+0000 7fafdb600640 1 -- 172.21.15.60:0/3053306945 shutdown_connections 2024-09-16T15:39:29.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.950+0000 7fafdb600640 1 -- 172.21.15.60:0/3053306945 wait complete. 2024-09-16T15:39:29.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.951+0000 7fafdb600640 1 Processor -- start 2024-09-16T15:39:29.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.952+0000 7fafdb600640 1 -- start start 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.952+0000 7fafdb600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafd406f220 0x7fafd40cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafdb600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafd4159aa0 0x7fafd40c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafdb600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd415cff0 0x7fafd40b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafdb600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fafd406fa60 con 0x7fafd415cff0 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafdb600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fafd406f8e0 con 0x7fafd406f220 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafdb600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fafd40cc8a0 con 0x7fafd4159aa0 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd415cff0 0x7fafd40b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd415cff0 0x7fafd40b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41936/0 (socket says 172.21.15.60:41936) 2024-09-16T15:39:29.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd9600640 1 -- 172.21.15.60:0/3490517893 learned_addr learned my addr 172.21.15.60:0/3490517893 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:29.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd8c00640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafd406f220 0x7fafd40cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd3e00640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafd4159aa0 0x7fafd40c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd9600640 1 -- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafd4159aa0 msgr2=0x7fafd40c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd9600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafd4159aa0 0x7fafd40c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd9600640 1 -- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafd406f220 msgr2=0x7fafd40cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd9600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafd406f220 0x7fafd40cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd9600640 1 -- 172.21.15.60:0/3490517893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd41f4db0 con 0x7fafd415cff0 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd8c00640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafd406f220 0x7fafd40cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.953+0000 7fafd3e00640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafd4159aa0 0x7fafd40c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.954+0000 7fafd9600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd415cff0 0x7fafd40b4570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00cbc0 tx=0x7fafcc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.954+0000 7fafd0c00640 1 -- 172.21.15.60:0/3490517893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc016020 con 0x7fafd415cff0 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.954+0000 7fafd0c00640 1 -- 172.21.15.60:0/3490517893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafcc00add0 con 0x7fafd415cff0 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.954+0000 7fafd0c00640 1 -- 172.21.15.60:0/3490517893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc012680 con 0x7fafd415cff0 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.954+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd41f5090 con 0x7fafd415cff0 2024-09-16T15:39:29.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.954+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd41f5650 con 0x7fafd415cff0 2024-09-16T15:39:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.955+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd4069360 con 0x7fafd415cff0 2024-09-16T15:39:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.956+0000 7fafd0c00640 1 -- 172.21.15.60:0/3490517893 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fafcc019020 con 0x7fafd415cff0 2024-09-16T15:39:29.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.958+0000 7fafd0c00640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fafb4078880 0x7fafb407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:29.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.958+0000 7fafd0c00640 1 -- 172.21.15.60:0/3490517893 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fafcc09a960 con 0x7fafd415cff0 2024-09-16T15:39:29.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.958+0000 7fafd8c00640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fafb4078880 0x7fafb407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:29.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.959+0000 7fafd8c00640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fafb4078880 0x7fafb407ad40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafc8034000 tx=0x7fafc8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:29.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:29.962+0000 7fafd0c00640 1 -- 172.21.15.60:0/3490517893 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fafcc020030 con 0x7fafd415cff0 2024-09-16T15:39:30.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:30.159+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fafd406e6f0 con 0x7fafd415cff0 2024-09-16T15:39:30.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:39:30.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:39:31.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.136+0000 7fafd0c00640 1 -- 172.21.15.60:0/3490517893 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) ==== 138+0+0 (secure 0 0 0) 0x7fafcc063320 con 0x7fafd415cff0 2024-09-16T15:39:31.138 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:39:31.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fafb4078880 msgr2=0x7fafb407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fafb4078880 0x7fafb407ad40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fafc8034000 tx=0x7fafc8004730 comp rx=0 tx=0).stop 2024-09-16T15:39:31.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd415cff0 msgr2=0x7fafd40b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd415cff0 0x7fafd40b4570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00cbc0 tx=0x7fafcc00a430 comp rx=0 tx=0).stop 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 shutdown_connections 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fafb4078880 0x7fafb407ad40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafd415cff0 0x7fafd40b4570 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafd4159aa0 0x7fafd40c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 --2- 172.21.15.60:0/3490517893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafd406f220 0x7fafd40cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 >> 172.21.15.60:0/3490517893 conn(0x7fafd40b18e0 msgr2=0x7fafd4159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 shutdown_connections 2024-09-16T15:39:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.139+0000 7fafdb600640 1 -- 172.21.15.60:0/3490517893 wait complete. 2024-09-16T15:39:31.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:39:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.328+0000 7f6703e00640 1 -- 172.21.15.60:0/1605236817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc1614e0 msgr2=0x7f66fc0d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.328+0000 7f6703e00640 1 --2- 172.21.15.60:0/1605236817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc1614e0 0x7f66fc0d4bc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0ced60 tx=0x7f66f002a4f0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.328+0000 7f6703e00640 1 -- 172.21.15.60:0/1605236817 shutdown_connections 2024-09-16T15:39:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.328+0000 7f6703e00640 1 --2- 172.21.15.60:0/1605236817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66fc1618c0 0x7f66fc0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.328+0000 7f6703e00640 1 --2- 172.21.15.60:0/1605236817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc1614e0 0x7f66fc0d4bc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.328+0000 7f6703e00640 1 -- 172.21.15.60:0/1605236817 >> 172.21.15.60:0/1605236817 conn(0x7f66fc0d40a0 msgr2=0x7f66fc0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:31.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.328+0000 7f6703e00640 1 -- 172.21.15.60:0/1605236817 shutdown_connections 2024-09-16T15:39:31.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.328+0000 7f6703e00640 1 -- 172.21.15.60:0/1605236817 wait complete. 2024-09-16T15:39:31.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.329+0000 7f6703e00640 1 Processor -- start 2024-09-16T15:39:31.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.330+0000 7f6703e00640 1 -- start start 2024-09-16T15:39:31.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.330+0000 7f6703e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc0b3f20 0x7f66fc15bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:31.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.330+0000 7f6703e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66fc1614e0 0x7f66fc0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:31.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6703e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc1618c0 0x7f66fc0b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:31.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6703e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc0d25c0 con 0x7f66fc1618c0 2024-09-16T15:39:31.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6703e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc0d2440 con 0x7f66fc0b3f20 2024-09-16T15:39:31.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6703e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f66fc0d2740 con 0x7f66fc1614e0 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6700a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66fc1614e0 0x7f66fc0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6700a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66fc1614e0 0x7f66fc0c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56520/0 (socket says 172.21.15.60:56520) 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6700a00640 1 -- 172.21.15.60:0/41612521 learned_addr learned my addr 172.21.15.60:0/41612521 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6701400640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc0b3f20 0x7f66fc15bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.331+0000 7f6701e00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc1618c0 0x7f66fc0b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.332+0000 7f6701400640 1 -- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66fc1614e0 msgr2=0x7f66fc0c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.332+0000 7f6701400640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66fc1614e0 0x7f66fc0c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.332+0000 7f6701400640 1 -- 172.21.15.60:0/41612521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc1618c0 msgr2=0x7f66fc0b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.332+0000 7f6701400640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc1618c0 0x7f66fc0b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.332+0000 7f6701400640 1 -- 172.21.15.60:0/41612521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc0c80c0 con 0x7f66fc0b3f20 2024-09-16T15:39:31.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.332+0000 7f6700a00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66fc1614e0 0x7f66fc0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:31.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.332+0000 7f6701400640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc0b3f20 0x7f66fc15bb90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0d15b0 tx=0x7f66f0004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:31.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.333+0000 7f66f9600640 1 -- 172.21.15.60:0/41612521 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0040070 con 0x7f66fc0b3f20 2024-09-16T15:39:31.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.333+0000 7f66f9600640 1 -- 172.21.15.60:0/41612521 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66f0004e50 con 0x7f66fc0b3f20 2024-09-16T15:39:31.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.333+0000 7f66f9600640 1 -- 172.21.15.60:0/41612521 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0005c30 con 0x7f66fc0b3f20 2024-09-16T15:39:31.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.333+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc0c83a0 con 0x7f66fc0b3f20 2024-09-16T15:39:31.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.333+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc0c8960 con 0x7f66fc0b3f20 2024-09-16T15:39:31.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.334+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66fc0ba020 con 0x7f66fc0b3f20 2024-09-16T15:39:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.336+0000 7f66f9600640 1 -- 172.21.15.60:0/41612521 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f66f0039040 con 0x7f66fc0b3f20 2024-09-16T15:39:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.337+0000 7f66f9600640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66dc078850 0x7f66dc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.337+0000 7f66f9600640 1 -- 172.21.15.60:0/41612521 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5227+0+0 (secure 0 0 0) 0x7f66f00bba20 con 0x7f66fc0b3f20 2024-09-16T15:39:31.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.337+0000 7f6700a00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66dc078850 0x7f66dc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:31.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.338+0000 7f6700a00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66dc078850 0x7f66dc07ad10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f66fc1f7f90 tx=0x7f66e8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:31.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.342+0000 7f66f9600640 1 -- 172.21.15.60:0/41612521 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66f0083c10 con 0x7f66fc0b3f20 2024-09-16T15:39:31.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:31.539+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f66fc160b00 con 0x7f66fc0b3f20 2024-09-16T15:39:32.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.150+0000 7f66f9600640 1 -- 172.21.15.60:0/41612521 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) ==== 170+0+0 (secure 0 0 0) 0x7f66f0088640 con 0x7f66fc0b3f20 2024-09-16T15:39:32.156 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:39:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.156+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66dc078850 msgr2=0x7f66dc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.156+0000 7f6703e00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66dc078850 0x7f66dc07ad10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f66fc1f7f90 tx=0x7f66e8005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.156+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc0b3f20 msgr2=0x7f66fc15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.156+0000 7f6703e00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc0b3f20 0x7f66fc15bb90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0d15b0 tx=0x7f66f0004a50 comp rx=0 tx=0).stop 2024-09-16T15:39:32.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.157+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 shutdown_connections 2024-09-16T15:39:32.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.157+0000 7f6703e00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66dc078850 0x7f66dc07ad10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.157+0000 7f6703e00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc1618c0 0x7f66fc0b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.157+0000 7f6703e00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66fc1614e0 0x7f66fc0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.157+0000 7f6703e00640 1 --2- 172.21.15.60:0/41612521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc0b3f20 0x7f66fc15bb90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.157+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 >> 172.21.15.60:0/41612521 conn(0x7f66fc0d40a0 msgr2=0x7f66fc0d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:32.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.157+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 shutdown_connections 2024-09-16T15:39:32.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.157+0000 7f6703e00640 1 -- 172.21.15.60:0/41612521 wait complete. 2024-09-16T15:39:32.188 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:39:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.351+0000 7f65ee800640 1 -- 172.21.15.60:0/3525953408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 msgr2=0x7f65e8161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.351+0000 7f65ee800640 1 --2- 172.21.15.60:0/3525953408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 0x7f65e8161c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009900 tx=0x7f65d402a610 comp rx=0 tx=0).stop 2024-09-16T15:39:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.351+0000 7f65ee800640 1 -- 172.21.15.60:0/3525953408 shutdown_connections 2024-09-16T15:39:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.351+0000 7f65ee800640 1 --2- 172.21.15.60:0/3525953408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 0x7f65e8161c20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.351+0000 7f65ee800640 1 --2- 172.21.15.60:0/3525953408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e815e250 0x7f65e80d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.351+0000 7f65ee800640 1 -- 172.21.15.60:0/3525953408 >> 172.21.15.60:0/3525953408 conn(0x7f65e80d1e20 msgr2=0x7f65e80d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.351+0000 7f65ee800640 1 -- 172.21.15.60:0/3525953408 shutdown_connections 2024-09-16T15:39:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.351+0000 7f65ee800640 1 -- 172.21.15.60:0/3525953408 wait complete. 2024-09-16T15:39:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.352+0000 7f65ee800640 1 Processor -- start 2024-09-16T15:39:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.352+0000 7f65ee800640 1 -- start start 2024-09-16T15:39:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.353+0000 7f65ee800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 0x7f65e80c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.353+0000 7f65ee800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e80d1000 0x7f65e80c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.353+0000 7f65ee800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65e815e250 0x7f65e8160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.353+0000 7f65ee800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81624f0 con 0x7f65e806ed00 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.353+0000 7f65ee800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65e8162370 con 0x7f65e815e250 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.353+0000 7f65ee800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f65e80cef10 con 0x7f65e80d1000 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 0x7f65e80c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 0x7f65e80c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41978/0 (socket says 172.21.15.60:41978) 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 -- 172.21.15.60:0/3451603646 learned_addr learned my addr 172.21.15.60:0/3451603646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e6a00640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e80d1000 0x7f65e80c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 -- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e80d1000 msgr2=0x7f65e80c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7e00640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65e815e250 0x7f65e8160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e80d1000 0x7f65e80c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 -- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65e815e250 msgr2=0x7f65e8160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65e815e250 0x7f65e8160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 -- 172.21.15.60:0/3451603646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e81f8980 con 0x7f65e806ed00 2024-09-16T15:39:32.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e6a00640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e80d1000 0x7f65e80c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:32.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7e00640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65e815e250 0x7f65e8160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:32.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.354+0000 7f65e7400640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 0x7f65e80c8790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e790 tx=0x7f65d800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:32.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.355+0000 7f65df400640 1 -- 172.21.15.60:0/3451603646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d800eea0 con 0x7f65e806ed00 2024-09-16T15:39:32.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.355+0000 7f65df400640 1 -- 172.21.15.60:0/3451603646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65d800cdb0 con 0x7f65e806ed00 2024-09-16T15:39:32.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.355+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e81f8c60 con 0x7f65e806ed00 2024-09-16T15:39:32.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.355+0000 7f65df400640 1 -- 172.21.15.60:0/3451603646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8010690 con 0x7f65e806ed00 2024-09-16T15:39:32.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.355+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e81f9220 con 0x7f65e806ed00 2024-09-16T15:39:32.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.356+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e80cf410 con 0x7f65e806ed00 2024-09-16T15:39:32.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.357+0000 7f65df400640 1 -- 172.21.15.60:0/3451603646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f65d8010830 con 0x7f65e806ed00 2024-09-16T15:39:32.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.358+0000 7f65df400640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65c8078850 0x7f65c807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.358+0000 7f65df400640 1 -- 172.21.15.60:0/3451603646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5609+0+0 (secure 0 0 0) 0x7f65d809b350 con 0x7f65e806ed00 2024-09-16T15:39:32.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.358+0000 7f65e6a00640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65c8078850 0x7f65c807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.359+0000 7f65e6a00640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65c8078850 0x7f65c807ad10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65d4033000 tx=0x7f65d4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:32.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.363+0000 7f65df400640 1 -- 172.21.15.60:0/3451603646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f65d8063980 con 0x7f65e806ed00 2024-09-16T15:39:32.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:32.602+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f65e8069630 con 0x7f65e806ed00 2024-09-16T15:39:33.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.145+0000 7f65df400640 1 -- 172.21.15.60:0/3451603646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v14) ==== 172+0+0 (secure 0 0 0) 0x7f65d8063280 con 0x7f65e806ed00 2024-09-16T15:39:33.147 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 4 and data pool 5 2024-09-16T15:39:33.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.149+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65c8078850 msgr2=0x7f65c807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.149+0000 7f65ee800640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65c8078850 0x7f65c807ad10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65d4033000 tx=0x7f65d4004730 comp rx=0 tx=0).stop 2024-09-16T15:39:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.149+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 msgr2=0x7f65e80c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.149+0000 7f65ee800640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 0x7f65e80c8790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e790 tx=0x7f65d800ec50 comp rx=0 tx=0).stop 2024-09-16T15:39:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.150+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 shutdown_connections 2024-09-16T15:39:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.150+0000 7f65ee800640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65c8078850 0x7f65c807ad10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.150+0000 7f65ee800640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65e815e250 0x7f65e8160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.150+0000 7f65ee800640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65e80d1000 0x7f65e80c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.150+0000 7f65ee800640 1 --2- 172.21.15.60:0/3451603646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65e806ed00 0x7f65e80c8790 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e790 tx=0x7f65d800ec50 comp rx=0 tx=0).stop 2024-09-16T15:39:33.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.150+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 >> 172.21.15.60:0/3451603646 conn(0x7f65e80d1e20 msgr2=0x7f65e8071330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.153+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 shutdown_connections 2024-09-16T15:39:33.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.153+0000 7f65ee800640 1 -- 172.21.15.60:0/3451603646 wait complete. 2024-09-16T15:39:33.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:33.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.356+0000 7f5a8ea00640 1 -- 172.21.15.60:0/4180935120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a88073fb0 msgr2=0x7f5a88074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.356+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/4180935120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a88073fb0 0x7f5a88074460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005fd0 tx=0x7f5a8002aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:33.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.356+0000 7f5a8ea00640 1 -- 172.21.15.60:0/4180935120 shutdown_connections 2024-09-16T15:39:33.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.356+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/4180935120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a88073fb0 0x7f5a88074460 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.356+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/4180935120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a8815f5a0 0x7f5a880b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.356+0000 7f5a8ea00640 1 -- 172.21.15.60:0/4180935120 >> 172.21.15.60:0/4180935120 conn(0x7f5a880b1950 msgr2=0x7f5a880b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.356+0000 7f5a8ea00640 1 -- 172.21.15.60:0/4180935120 shutdown_connections 2024-09-16T15:39:33.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.356+0000 7f5a8ea00640 1 -- 172.21.15.60:0/4180935120 wait complete. 2024-09-16T15:39:33.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.358+0000 7f5a8ea00640 1 Processor -- start 2024-09-16T15:39:33.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.358+0000 7f5a8ea00640 1 -- start start 2024-09-16T15:39:33.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a8ea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a88073fb0 0x7f5a881404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a8ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a880ccf00 0x7f5a880d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a8ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8815f5a0 0x7f5a8814d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a8ea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8806f2e0 con 0x7f5a880ccf00 2024-09-16T15:39:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a8ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8806f160 con 0x7f5a8815f5a0 2024-09-16T15:39:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a8ea00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5a8806f460 con 0x7f5a88073fb0 2024-09-16T15:39:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a7f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a880ccf00 0x7f5a880d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a7f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a880ccf00 0x7f5a880d06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42000/0 (socket says 172.21.15.60:42000) 2024-09-16T15:39:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a7f400640 1 -- 172.21.15.60:0/2913847646 learned_addr learned my addr 172.21.15.60:0/2913847646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a87e00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a88073fb0 0x7f5a881404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.359+0000 7f5a8ca00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8815f5a0 0x7f5a8814d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a7f400640 1 -- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a88073fb0 msgr2=0x7f5a881404b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a7f400640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a88073fb0 0x7f5a881404b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a7f400640 1 -- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8815f5a0 msgr2=0x7f5a8814d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a7f400640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8815f5a0 0x7f5a8814d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a7f400640 1 -- 172.21.15.60:0/2913847646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a881f62c0 con 0x7f5a880ccf00 2024-09-16T15:39:33.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a7f400640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a880ccf00 0x7f5a880d06e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000c420 tx=0x7f5a7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a84c00640 1 -- 172.21.15.60:0/2913847646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70017070 con 0x7f5a880ccf00 2024-09-16T15:39:33.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a84c00640 1 -- 172.21.15.60:0/2913847646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a700053c0 con 0x7f5a880ccf00 2024-09-16T15:39:33.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a84c00640 1 -- 172.21.15.60:0/2913847646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70003850 con 0x7f5a880ccf00 2024-09-16T15:39:33.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a88140a90 con 0x7f5a880ccf00 2024-09-16T15:39:33.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.360+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a88141050 con 0x7f5a880ccf00 2024-09-16T15:39:33.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.361+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8806f9f0 con 0x7f5a880ccf00 2024-09-16T15:39:33.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.363+0000 7f5a84c00640 1 -- 172.21.15.60:0/2913847646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5a7001a070 con 0x7f5a880ccf00 2024-09-16T15:39:33.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.365+0000 7f5a84c00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5a64078850 0x7f5a6407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.365+0000 7f5a84c00640 1 -- 172.21.15.60:0/2913847646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5a70099c10 con 0x7f5a880ccf00 2024-09-16T15:39:33.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.365+0000 7f5a87e00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5a64078850 0x7f5a6407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.365+0000 7f5a87e00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5a64078850 0x7f5a6407ad10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009b30 tx=0x7f5a78005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.368+0000 7f5a84c00640 1 -- 172.21.15.60:0/2913847646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5a70061c20 con 0x7f5a880ccf00 2024-09-16T15:39:33.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.564+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a8809aa00 con 0x7f5a880ccf00 2024-09-16T15:39:33.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.565+0000 7f5a84c00640 1 -- 172.21.15.60:0/2913847646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16742 (secure 0 0 0) 0x7f5a70066650 con 0x7f5a880ccf00 2024-09-16T15:39:33.566 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:33.566 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:39:32.603975+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:30.169700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:31.550025+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5a64078850 msgr2=0x7f5a6407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5a64078850 0x7f5a6407ad10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009b30 tx=0x7f5a78005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a880ccf00 msgr2=0x7f5a880d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a880ccf00 0x7f5a880d06e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000c420 tx=0x7f5a7000c8e0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 shutdown_connections 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5a64078850 0x7f5a6407ad10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a8815f5a0 0x7f5a8814d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a880ccf00 0x7f5a880d06e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 --2- 172.21.15.60:0/2913847646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a88073fb0 0x7f5a881404b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 >> 172.21.15.60:0/2913847646 conn(0x7f5a880b1950 msgr2=0x7f5a880b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 shutdown_connections 2024-09-16T15:39:33.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.567+0000 7f5a8ea00640 1 -- 172.21.15.60:0/2913847646 wait complete. 2024-09-16T15:39:33.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.744+0000 7fb6eda00640 1 -- 172.21.15.60:0/22956275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e81445d0 msgr2=0x7fb6e80cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.744+0000 7fb6eda00640 1 --2- 172.21.15.60:0/22956275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e81445d0 0x7fb6e80cebf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009890 tx=0x7fb6d802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.744+0000 7fb6eda00640 1 -- 172.21.15.60:0/22956275 shutdown_connections 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.744+0000 7fb6eda00640 1 --2- 172.21.15.60:0/22956275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e81445d0 0x7fb6e80cebf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.744+0000 7fb6eda00640 1 -- 172.21.15.60:0/22956275 >> 172.21.15.60:0/22956275 conn(0x7fb6e80b1950 msgr2=0x7fb6e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.744+0000 7fb6eda00640 1 -- 172.21.15.60:0/22956275 shutdown_connections 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.744+0000 7fb6eda00640 1 -- 172.21.15.60:0/22956275 wait complete. 2024-09-16T15:39:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.746+0000 7fb6eda00640 1 Processor -- start 2024-09-16T15:39:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.746+0000 7fb6eda00640 1 -- start start 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6eda00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e80b4ca0 0x7fb6e813d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6eda00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e80ccfd0 0x7fb6e813ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6eda00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e81445d0 0x7fb6e81f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6eda00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8073b30 con 0x7fb6e81445d0 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6eda00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e80739b0 con 0x7fb6e80b4ca0 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6eda00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e8073cb0 con 0x7fb6e80ccfd0 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6e7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e81445d0 0x7fb6e81f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6e7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e81445d0 0x7fb6e81f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42004/0 (socket says 172.21.15.60:42004) 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.747+0000 7fb6e7400640 1 -- 172.21.15.60:0/121466284 learned_addr learned my addr 172.21.15.60:0/121466284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e6a00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e80b4ca0 0x7fb6e813d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e6000640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e80ccfd0 0x7fb6e813ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e7400640 1 -- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e80ccfd0 msgr2=0x7fb6e813ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e7400640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e80ccfd0 0x7fb6e813ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e7400640 1 -- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e80b4ca0 msgr2=0x7fb6e813d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e7400640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e80b4ca0 0x7fb6e813d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e7400640 1 -- 172.21.15.60:0/121466284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e813dcb0 con 0x7fb6e81445d0 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e6000640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e80ccfd0 0x7fb6e813ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6e7400640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e81445d0 0x7fb6e81f6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c780 tx=0x7fb6e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6dea00640 1 -- 172.21.15.60:0/121466284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0003b70 con 0x7fb6e81445d0 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6dea00640 1 -- 172.21.15.60:0/121466284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6e0003d10 con 0x7fb6e81445d0 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6dea00640 1 -- 172.21.15.60:0/121466284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000aab0 con 0x7fb6e81445d0 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e813df90 con 0x7fb6e81445d0 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.748+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e813e550 con 0x7fb6e81445d0 2024-09-16T15:39:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.749+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e80cf430 con 0x7fb6e81445d0 2024-09-16T15:39:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.751+0000 7fb6dea00640 1 -- 172.21.15.60:0/121466284 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb6e000ac50 con 0x7fb6e81445d0 2024-09-16T15:39:33.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.752+0000 7fb6dea00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb6c8078880 0x7fb6c807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.752+0000 7fb6dea00640 1 -- 172.21.15.60:0/121466284 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb6e009a7d0 con 0x7fb6e81445d0 2024-09-16T15:39:33.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.752+0000 7fb6e6a00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb6c8078880 0x7fb6c807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.753+0000 7fb6e6a00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb6c8078880 0x7fb6c807ad40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8034000 tx=0x7fb6d8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.756+0000 7fb6dea00640 1 -- 172.21.15.60:0/121466284 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb6e0062e30 con 0x7fb6e81445d0 2024-09-16T15:39:33.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.950+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6e813e720 con 0x7fb6e81445d0 2024-09-16T15:39:33.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.952+0000 7fb6dea00640 1 -- 172.21.15.60:0/121466284 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16742 (secure 0 0 0) 0x7fb6e0062730 con 0x7fb6e81445d0 2024-09-16T15:39:33.953 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:33.953 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:39:32.603975+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:30.169700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:31.550025+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:33.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb6c8078880 msgr2=0x7fb6c807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb6c8078880 0x7fb6c807ad40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8034000 tx=0x7fb6d8004730 comp rx=0 tx=0).stop 2024-09-16T15:39:33.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e81445d0 msgr2=0x7fb6e81f6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e81445d0 0x7fb6e81f6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c780 tx=0x7fb6e000cc40 comp rx=0 tx=0).stop 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 shutdown_connections 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb6c8078880 0x7fb6c807ad40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6e81445d0 0x7fb6e81f6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6e80ccfd0 0x7fb6e813ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 --2- 172.21.15.60:0/121466284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6e80b4ca0 0x7fb6e813d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 >> 172.21.15.60:0/121466284 conn(0x7fb6e80b1950 msgr2=0x7fb6e80b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 shutdown_connections 2024-09-16T15:39:33.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:33.954+0000 7fb6eda00640 1 -- 172.21.15.60:0/121466284 wait complete. 2024-09-16T15:39:33.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.134+0000 7f186d800640 1 -- 172.21.15.60:0/1812526589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f186806efa0 msgr2=0x7f18680cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.134+0000 7f186d800640 1 --2- 172.21.15.60:0/1812526589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f186806efa0 0x7f18680cc5c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1850005fd0 tx=0x7f185002a540 comp rx=0 tx=0).stop 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.134+0000 7f186d800640 1 -- 172.21.15.60:0/1812526589 shutdown_connections 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.134+0000 7f186d800640 1 --2- 172.21.15.60:0/1812526589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f186806efa0 0x7f18680cc5c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.134+0000 7f186d800640 1 --2- 172.21.15.60:0/1812526589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1868145750 0x7f1868073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.134+0000 7f186d800640 1 -- 172.21.15.60:0/1812526589 >> 172.21.15.60:0/1812526589 conn(0x7f18680b1840 msgr2=0x7f18680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.134+0000 7f186d800640 1 -- 172.21.15.60:0/1812526589 shutdown_connections 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.134+0000 7f186d800640 1 -- 172.21.15.60:0/1812526589 wait complete. 2024-09-16T15:39:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.135+0000 7f186d800640 1 Processor -- start 2024-09-16T15:39:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.136+0000 7f186d800640 1 -- start start 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.136+0000 7f186d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f186806efa0 0x7f186815ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.136+0000 7f186d800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18680ccb00 0x7f18681f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.136+0000 7f186d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1868145750 0x7f18681e8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.136+0000 7f186d800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f18680b26b0 con 0x7f1868145750 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.136+0000 7f186d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18680cedd0 con 0x7f186806efa0 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.136+0000 7f186d800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1868160100 con 0x7f18680ccb00 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1867400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1868145750 0x7f18681e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1867400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1868145750 0x7f18681e8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42014/0 (socket says 172.21.15.60:42014) 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1867400640 1 -- 172.21.15.60:0/2150905895 learned_addr learned my addr 172.21.15.60:0/2150905895 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1866a00640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f186806efa0 0x7f186815ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f185e000640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18680ccb00 0x7f18681f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1867400640 1 -- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18680ccb00 msgr2=0x7f18681f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1867400640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18680ccb00 0x7f18681f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1867400640 1 -- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f186806efa0 msgr2=0x7f186815ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1867400640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f186806efa0 0x7f186815ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.137+0000 7f1867400640 1 -- 172.21.15.60:0/2150905895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18681e7680 con 0x7f1868145750 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.138+0000 7f1867400640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1868145750 0x7f18681e8720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f186000c7a0 tx=0x7f186000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.138+0000 7f185f400640 1 -- 172.21.15.60:0/2150905895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860003b70 con 0x7f1868145750 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.138+0000 7f185f400640 1 -- 172.21.15.60:0/2150905895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1860003d10 con 0x7f1868145750 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.138+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18681e7960 con 0x7f1868145750 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.138+0000 7f185f400640 1 -- 172.21.15.60:0/2150905895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186000aab0 con 0x7f1868145750 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.138+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681e7f20 con 0x7f1868145750 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.139+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18681605b0 con 0x7f1868145750 2024-09-16T15:39:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.140+0000 7f185f400640 1 -- 172.21.15.60:0/2150905895 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f186000ac50 con 0x7f1868145750 2024-09-16T15:39:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.141+0000 7f185f400640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1844078850 0x7f184407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.141+0000 7f185f400640 1 -- 172.21.15.60:0/2150905895 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f186009a820 con 0x7f1868145750 2024-09-16T15:39:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.142+0000 7f1866a00640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1844078850 0x7f184407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.142+0000 7f1866a00640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1844078850 0x7f184407ad10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f18681f7f50 tx=0x7f1858005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.145+0000 7f185f400640 1 -- 172.21.15.60:0/2150905895 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1860062e00 con 0x7f1868145750 2024-09-16T15:39:34.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:34.385+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f186815f9e0 con 0x7f1868145750 2024-09-16T15:39:35.131 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-09-16T15:39:35.130+0000 7f7964800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:39:35.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.164+0000 7f185f400640 1 -- 172.21.15.60:0/2150905895 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f1860062700 con 0x7f1868145750 2024-09-16T15:39:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1844078850 msgr2=0x7f184407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1844078850 0x7f184407ad10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f18681f7f50 tx=0x7f1858005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1868145750 msgr2=0x7f18681e8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1868145750 0x7f18681e8720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f186000c7a0 tx=0x7f186000cc60 comp rx=0 tx=0).stop 2024-09-16T15:39:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 shutdown_connections 2024-09-16T15:39:35.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1844078850 0x7f184407ad10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1868145750 0x7f18681e8720 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18680ccb00 0x7f18681f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 --2- 172.21.15.60:0/2150905895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f186806efa0 0x7f186815ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.167+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 >> 172.21.15.60:0/2150905895 conn(0x7f18680b1840 msgr2=0x7f18680b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.168+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 shutdown_connections 2024-09-16T15:39:35.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.168+0000 7f186d800640 1 -- 172.21.15.60:0/2150905895 wait complete. 2024-09-16T15:39:35.187 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:39:35.187 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:39:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.354+0000 7fd8cf000640 1 -- 172.21.15.60:0/3152465340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 msgr2=0x7fd8c80cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.354+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3152465340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 0x7fd8c80cf090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0005fd0 tx=0x7fd8c002aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.354+0000 7fd8cf000640 1 -- 172.21.15.60:0/3152465340 shutdown_connections 2024-09-16T15:39:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.354+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3152465340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 0x7fd8c80cf090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.354+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3152465340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c8161760 0x7fd8c80ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.354+0000 7fd8cf000640 1 -- 172.21.15.60:0/3152465340 >> 172.21.15.60:0/3152465340 conn(0x7fd8c80b1840 msgr2=0x7fd8c80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.355+0000 7fd8cf000640 1 -- 172.21.15.60:0/3152465340 shutdown_connections 2024-09-16T15:39:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.355+0000 7fd8cf000640 1 -- 172.21.15.60:0/3152465340 wait complete. 2024-09-16T15:39:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.356+0000 7fd8cf000640 1 Processor -- start 2024-09-16T15:39:35.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.356+0000 7fd8cf000640 1 -- start start 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.357+0000 7fd8cf000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80af190 0x7fd8c815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.357+0000 7fd8cf000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 0x7fd8c8158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.357+0000 7fd8cf000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161760 0x7fd8c81f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.357+0000 7fd8cf000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8162190 con 0x7fd8c80cec60 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.357+0000 7fd8cf000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8162310 con 0x7fd8c8161760 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.357+0000 7fd8cf000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c8162490 con 0x7fd8c80af190 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 0x7fd8c8158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 0x7fd8c8158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42024/0 (socket says 172.21.15.60:42024) 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 -- 172.21.15.60:0/3591793117 learned_addr learned my addr 172.21.15.60:0/3591793117 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:35.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8c7e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80af190 0x7fd8c815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8cd000640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161760 0x7fd8c81f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 -- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80af190 msgr2=0x7fd8c815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80af190 0x7fd8c815ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 -- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161760 msgr2=0x7fd8c81f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161760 0x7fd8c81f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 -- 172.21.15.60:0/3591793117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c815f520 con 0x7fd8c80cec60 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.358+0000 7fd8bf400640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 0x7fd8c8158eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000c450 tx=0x7fd8b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.359+0000 7fd8c4c00640 1 -- 172.21.15.60:0/3591793117 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00052d0 con 0x7fd8c80cec60 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.359+0000 7fd8c4c00640 1 -- 172.21.15.60:0/3591793117 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8b0005470 con 0x7fd8c80cec60 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.359+0000 7fd8c4c00640 1 -- 172.21.15.60:0/3591793117 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0012870 con 0x7fd8c80cec60 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.359+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c815f800 con 0x7fd8c80cec60 2024-09-16T15:39:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.359+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c815fdc0 con 0x7fd8c80cec60 2024-09-16T15:39:35.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.360+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c8162910 con 0x7fd8c80cec60 2024-09-16T15:39:35.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.361+0000 7fd8c4c00640 1 -- 172.21.15.60:0/3591793117 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd8b00035f0 con 0x7fd8c80cec60 2024-09-16T15:39:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.362+0000 7fd8c4c00640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a4078880 0x7fd8a407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.362+0000 7fd8c4c00640 1 -- 172.21.15.60:0/3591793117 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd8b009a3c0 con 0x7fd8c80cec60 2024-09-16T15:39:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.362+0000 7fd8c7e00640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a4078880 0x7fd8a407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.363+0000 7fd8c7e00640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a4078880 0x7fd8a407ad40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8004630 tx=0x7fd8b800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.366+0000 7fd8c4c00640 1 -- 172.21.15.60:0/3591793117 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8b00623d0 con 0x7fd8c80cec60 2024-09-16T15:39:35.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:35.604+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd8c815b410 con 0x7fd8c80cec60 2024-09-16T15:39:36.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.174+0000 7fd8c4c00640 1 -- 172.21.15.60:0/3591793117 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fd8b0066e00 con 0x7fd8c80cec60 2024-09-16T15:39:36.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a4078880 msgr2=0x7fd8a407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a4078880 0x7fd8a407ad40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8004630 tx=0x7fd8b800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 msgr2=0x7fd8c8158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 0x7fd8c8158eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000c450 tx=0x7fd8b000c910 comp rx=0 tx=0).stop 2024-09-16T15:39:36.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 shutdown_connections 2024-09-16T15:39:36.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a4078880 0x7fd8a407ad40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161760 0x7fd8c81f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c80cec60 0x7fd8c8158eb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 --2- 172.21.15.60:0/3591793117 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80af190 0x7fd8c815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 >> 172.21.15.60:0/3591793117 conn(0x7fd8c80b1840 msgr2=0x7fd8c815e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 shutdown_connections 2024-09-16T15:39:36.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.178+0000 7fd8cf000640 1 -- 172.21.15.60:0/3591793117 wait complete. 2024-09-16T15:39:36.197 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:39:36.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:36.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.359+0000 7f9e90a00640 1 -- 172.21.15.60:0/222011357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 msgr2=0x7f9e8c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.359+0000 7f9e90a00640 1 --2- 172.21.15.60:0/222011357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 0x7f9e8c06f160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e84005fd0 tx=0x7f9e8402aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:36.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.359+0000 7f9e90a00640 1 -- 172.21.15.60:0/222011357 shutdown_connections 2024-09-16T15:39:36.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.359+0000 7f9e90a00640 1 --2- 172.21.15.60:0/222011357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 0x7f9e8c06f160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.359+0000 7f9e90a00640 1 -- 172.21.15.60:0/222011357 >> 172.21.15.60:0/222011357 conn(0x7f9e8c0b1950 msgr2=0x7f9e8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.359+0000 7f9e90a00640 1 -- 172.21.15.60:0/222011357 shutdown_connections 2024-09-16T15:39:36.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.359+0000 7f9e90a00640 1 -- 172.21.15.60:0/222011357 wait complete. 2024-09-16T15:39:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.361+0000 7f9e90a00640 1 Processor -- start 2024-09-16T15:39:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.361+0000 7f9e90a00640 1 -- start start 2024-09-16T15:39:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e90a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 0x7f9e8c149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e90a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e8c0ccf00 0x7f9e8c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e90a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e8c15f5a0 0x7f9e8c1e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e90a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c074840 con 0x7f9e8c0b2440 2024-09-16T15:39:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e90a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8c0746c0 con 0x7f9e8c0ccf00 2024-09-16T15:39:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e90a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8c06f6a0 con 0x7f9e8c15f5a0 2024-09-16T15:39:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e89e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 0x7f9e8c149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e89e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 0x7f9e8c149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42038/0 (socket says 172.21.15.60:42038) 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.362+0000 7f9e89e00640 1 -- 172.21.15.60:0/2917285757 learned_addr learned my addr 172.21.15.60:0/2917285757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e81400640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e8c0ccf00 0x7f9e8c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e8a800640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e8c15f5a0 0x7f9e8c1e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e89e00640 1 -- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e8c15f5a0 msgr2=0x7f9e8c1e8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e89e00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e8c15f5a0 0x7f9e8c1e8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e89e00640 1 -- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e8c0ccf00 msgr2=0x7f9e8c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e89e00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e8c0ccf00 0x7f9e8c09aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e89e00640 1 -- 172.21.15.60:0/2917285757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8c149c60 con 0x7f9e8c0b2440 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e8a800640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e8c15f5a0 0x7f9e8c1e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e81400640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e8c0ccf00 0x7f9e8c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e89e00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 0x7f9e8c149720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00b3f0 tx=0x7f9e7c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e82a00640 1 -- 172.21.15.60:0/2917285757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c015020 con 0x7f9e8c0b2440 2024-09-16T15:39:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e82a00640 1 -- 172.21.15.60:0/2917285757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e7c011920 con 0x7f9e8c0b2440 2024-09-16T15:39:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e82a00640 1 -- 172.21.15.60:0/2917285757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c011d70 con 0x7f9e8c0b2440 2024-09-16T15:39:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.363+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8c15b350 con 0x7f9e8c0b2440 2024-09-16T15:39:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.364+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8c15b910 con 0x7f9e8c0b2440 2024-09-16T15:39:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.365+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8c06f160 con 0x7f9e8c0b2440 2024-09-16T15:39:36.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.366+0000 7f9e82a00640 1 -- 172.21.15.60:0/2917285757 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9e7c018020 con 0x7f9e8c0b2440 2024-09-16T15:39:36.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.367+0000 7f9e82a00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9e68078850 0x7f9e6807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.367+0000 7f9e82a00640 1 -- 172.21.15.60:0/2917285757 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9e7c09a510 con 0x7f9e8c0b2440 2024-09-16T15:39:36.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.367+0000 7f9e81400640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9e68078850 0x7f9e6807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.368+0000 7f9e81400640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9e68078850 0x7f9e6807ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e74007fd0 tx=0x7f9e74009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:36.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.371+0000 7f9e82a00640 1 -- 172.21.15.60:0/2917285757 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9e7c01f030 con 0x7f9e8c0b2440 2024-09-16T15:39:36.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.604+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e8c0b0a00 con 0x7f9e8c0b2440 2024-09-16T15:39:36.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.605+0000 7f9e82a00640 1 -- 172.21.15.60:0/2917285757 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f9e7c062b80 con 0x7f9e8c0b2440 2024-09-16T15:39:36.606 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:36.606 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"btime":"2024-09-16T15:39:36:169518+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4866,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2562305658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2562305658},{"type":"v1","addr":"172.21.15.60:6833","nonce":2562305658}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/664909622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":664909622},{"type":"v1","addr":"172.21.15.100:6836","nonce":664909622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/801262237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":801262237},{"type":"v1","addr":"172.21.15.100:6837","nonce":801262237}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:32.603941+0000","modified":"2024-09-16T15:39:35.605148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/846626570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":846626570},{"type":"v1","addr":"172.21.15.60:6835","nonce":846626570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4881,"qdb_cluster":[4881]},"id":2}]} 2024-09-16T15:39:36.606 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 17 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9e68078850 msgr2=0x7f9e6807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9e68078850 0x7f9e6807ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e74007fd0 tx=0x7f9e74009040 comp rx=0 tx=0).stop 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 msgr2=0x7f9e8c149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 0x7f9e8c149720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00b3f0 tx=0x7f9e7c00b8b0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 shutdown_connections 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9e68078850 0x7f9e6807ad10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e8c15f5a0 0x7f9e8c1e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e8c0ccf00 0x7f9e8c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 --2- 172.21.15.60:0/2917285757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e8c0b2440 0x7f9e8c149720 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 >> 172.21.15.60:0/2917285757 conn(0x7f9e8c0b1950 msgr2=0x7f9e8c0cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.607+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 shutdown_connections 2024-09-16T15:39:36.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.608+0000 7f9e90a00640 1 -- 172.21.15.60:0/2917285757 wait complete. 2024-09-16T15:39:36.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:36.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.782+0000 7fa440c00640 1 -- 172.21.15.60:0/573301590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c15df20 msgr2=0x7fa43c15e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.782+0000 7fa440c00640 1 --2- 172.21.15.60:0/573301590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c15df20 0x7fa43c15e300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa42c005fd0 tx=0x7fa42c02a540 comp rx=0 tx=0).stop 2024-09-16T15:39:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.782+0000 7fa440c00640 1 -- 172.21.15.60:0/573301590 shutdown_connections 2024-09-16T15:39:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.782+0000 7fa440c00640 1 --2- 172.21.15.60:0/573301590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa43c0cebf0 0x7fa43c162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.782+0000 7fa440c00640 1 --2- 172.21.15.60:0/573301590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c15df20 0x7fa43c15e300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.782+0000 7fa440c00640 1 -- 172.21.15.60:0/573301590 >> 172.21.15.60:0/573301590 conn(0x7fa43c0d1da0 msgr2=0x7fa43c0d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.782+0000 7fa440c00640 1 -- 172.21.15.60:0/573301590 shutdown_connections 2024-09-16T15:39:36.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.782+0000 7fa440c00640 1 -- 172.21.15.60:0/573301590 wait complete. 2024-09-16T15:39:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.784+0000 7fa440c00640 1 Processor -- start 2024-09-16T15:39:36.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.784+0000 7fa440c00640 1 -- start start 2024-09-16T15:39:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa440c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c0ce760 0x7fa43c1f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa440c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa43c0cebf0 0x7fa43c158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa440c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa43c15df20 0x7fa43c14bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa440c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c06ee10 con 0x7fa43c0ce760 2024-09-16T15:39:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa440c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c06ec90 con 0x7fa43c15df20 2024-09-16T15:39:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa440c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa43c0cf510 con 0x7fa43c0cebf0 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa439e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c0ce760 0x7fa43c1f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa439e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c0ce760 0x7fa43c1f8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42062/0 (socket says 172.21.15.60:42062) 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa439e00640 1 -- 172.21.15.60:0/152088278 learned_addr learned my addr 172.21.15.60:0/152088278 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa439e00640 1 -- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa43c0cebf0 msgr2=0x7fa43c158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa439e00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa43c0cebf0 0x7fa43c158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa439e00640 1 -- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa43c15df20 msgr2=0x7fa43c14bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa439e00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa43c15df20 0x7fa43c14bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.785+0000 7fa439e00640 1 -- 172.21.15.60:0/152088278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c154b50 con 0x7fa43c0ce760 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.786+0000 7fa439e00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c0ce760 0x7fa43c1f8740 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0099f0 tx=0x7fa42c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.786+0000 7fa432a00640 1 -- 172.21.15.60:0/152088278 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c03f070 con 0x7fa43c0ce760 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.786+0000 7fa432a00640 1 -- 172.21.15.60:0/152088278 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa42c0047a0 con 0x7fa43c0ce760 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.786+0000 7fa432a00640 1 -- 172.21.15.60:0/152088278 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c035030 con 0x7fa43c0ce760 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.786+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c154df0 con 0x7fa43c0ce760 2024-09-16T15:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.786+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c155690 con 0x7fa43c0ce760 2024-09-16T15:39:36.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.787+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43c15e300 con 0x7fa43c0ce760 2024-09-16T15:39:36.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.788+0000 7fa432a00640 1 -- 172.21.15.60:0/152088278 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa42c042070 con 0x7fa43c0ce760 2024-09-16T15:39:36.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.790+0000 7fa432a00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa418078850 0x7fa41807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.790+0000 7fa432a00640 1 -- 172.21.15.60:0/152088278 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa42c0bbb80 con 0x7fa43c0ce760 2024-09-16T15:39:36.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.790+0000 7fa431400640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa418078850 0x7fa41807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.791+0000 7fa431400640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa418078850 0x7fa41807ad10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa424000f60 tx=0x7fa4240079b0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:36.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.794+0000 7fa432a00640 1 -- 172.21.15.60:0/152088278 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa42c084150 con 0x7fa43c0ce760 2024-09-16T15:39:36.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.990+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa43c071cb0 con 0x7fa43c0ce760 2024-09-16T15:39:36.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.991+0000 7fa432a00640 1 -- 172.21.15.60:0/152088278 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16729 (secure 0 0 0) 0x7fa42c083a50 con 0x7fa43c0ce760 2024-09-16T15:39:36.993 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:36.993 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":36,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:39:34.155062+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:30.169700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:31.550025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa418078850 msgr2=0x7fa41807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa418078850 0x7fa41807ad10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa424000f60 tx=0x7fa4240079b0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c0ce760 msgr2=0x7fa43c1f8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c0ce760 0x7fa43c1f8740 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0099f0 tx=0x7fa42c003910 comp rx=0 tx=0).stop 2024-09-16T15:39:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 shutdown_connections 2024-09-16T15:39:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa418078850 0x7fa41807ad10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa43c15df20 0x7fa43c14bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa43c0cebf0 0x7fa43c158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 --2- 172.21.15.60:0/152088278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa43c0ce760 0x7fa43c1f8740 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 >> 172.21.15.60:0/152088278 conn(0x7fa43c0d1da0 msgr2=0x7fa43c15d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 shutdown_connections 2024-09-16T15:39:36.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:36.994+0000 7fa440c00640 1 -- 172.21.15.60:0/152088278 wait complete. 2024-09-16T15:39:37.011 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:39:37.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:37.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 -- 172.21.15.60:0/2163795346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5134162560 msgr2=0x7f51341629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 --2- 172.21.15.60:0/2163795346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5134162560 0x7f51341629d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51340d4ee0 tx=0x7f512c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:37.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 -- 172.21.15.60:0/2163795346 shutdown_connections 2024-09-16T15:39:37.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 --2- 172.21.15.60:0/2163795346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5134162560 0x7f51341629d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 --2- 172.21.15.60:0/2163795346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51340b4d10 0x7f5134162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 --2- 172.21.15.60:0/2163795346 >> v2:172.21.15.100:3300/0 conn(0x7f51340d1ec0 0x7f51340d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 -- 172.21.15.60:0/2163795346 >> 172.21.15.60:0/2163795346 conn(0x7f51340d3de0 msgr2=0x7f51340d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 -- 172.21.15.60:0/2163795346 shutdown_connections 2024-09-16T15:39:37.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.170+0000 7f513a800640 1 -- 172.21.15.60:0/2163795346 wait complete. 2024-09-16T15:39:37.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.172+0000 7f513a800640 1 Processor -- start 2024-09-16T15:39:37.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.172+0000 7f513a800640 1 -- start start 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f513a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4d10 0x7f51341f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f513a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340d1ec0 0x7f51340ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f513a800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134162560 0x7f51340cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f513a800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51340b1d10 con 0x7f51340b4d10 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f513a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51340b1b90 con 0x7f51340d1ec0 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f513a800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f51340b1e90 con 0x7f5134162560 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4d10 0x7f51341f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4d10 0x7f51341f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42078/0 (socket says 172.21.15.60:42078) 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139400640 1 -- 172.21.15.60:0/621065478 learned_addr learned my addr 172.21.15.60:0/621065478 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5138a00640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340d1ec0 0x7f51340ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139e00640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134162560 0x7f51340cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139400640 1 -- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134162560 msgr2=0x7f51340cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139400640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134162560 0x7f51340cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139400640 1 -- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340d1ec0 msgr2=0x7f51340ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139400640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340d1ec0 0x7f51340ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139400640 1 -- 172.21.15.60:0/621065478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51341f4f40 con 0x7f51340b4d10 2024-09-16T15:39:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.173+0000 7f5139e00640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134162560 0x7f51340cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.174+0000 7f5139400640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4d10 0x7f51341f86b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f512800e9f0 tx=0x7f512800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.174+0000 7f5131600640 1 -- 172.21.15.60:0/621065478 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512800cd80 con 0x7f51340b4d10 2024-09-16T15:39:37.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.174+0000 7f5131600640 1 -- 172.21.15.60:0/621065478 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5128010430 con 0x7f51340b4d10 2024-09-16T15:39:37.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.174+0000 7f5131600640 1 -- 172.21.15.60:0/621065478 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128010840 con 0x7f51340b4d10 2024-09-16T15:39:37.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.174+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51341f5220 con 0x7f51340b4d10 2024-09-16T15:39:37.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.174+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51341f57e0 con 0x7f51340b4d10 2024-09-16T15:39:37.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.176+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51341f5a20 con 0x7f51340b4d10 2024-09-16T15:39:37.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.177+0000 7f5131600640 1 -- 172.21.15.60:0/621065478 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f51280109e0 con 0x7f51340b4d10 2024-09-16T15:39:37.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.178+0000 7f5131600640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5114078850 0x7f511407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.178+0000 7f5131600640 1 -- 172.21.15.60:0/621065478 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f512809b4a0 con 0x7f51340b4d10 2024-09-16T15:39:37.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.178+0000 7f5138a00640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5114078850 0x7f511407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.179+0000 7f5138a00640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5114078850 0x7f511407ad10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f51340cdad0 tx=0x7f5120005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.183+0000 7f5131600640 1 -- 172.21.15.60:0/621065478 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51341f5a20 con 0x7f51340b4d10 2024-09-16T15:39:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.375+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51340c93b0 con 0x7f5114078850 2024-09-16T15:39:37.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.381+0000 7f5131600640 1 -- 172.21.15.60:0/621065478 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245748 (secure 0 0 0) 0x7f51340c93b0 con 0x7f5114078850 2024-09-16T15:39:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:37.386 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:39:37.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5114078850 msgr2=0x7f511407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5114078850 0x7f511407ad10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f51340cdad0 tx=0x7f5120005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4d10 msgr2=0x7f51341f86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4d10 0x7f51341f86b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f512800e9f0 tx=0x7f512800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 shutdown_connections 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5114078850 0x7f511407ad10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134162560 0x7f51340cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340d1ec0 0x7f51340ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 --2- 172.21.15.60:0/621065478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4d10 0x7f51341f86b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 >> 172.21.15.60:0/621065478 conn(0x7f51340d3de0 msgr2=0x7f513415e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 shutdown_connections 2024-09-16T15:39:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.385+0000 7f513a800640 1 -- 172.21.15.60:0/621065478 wait complete. 2024-09-16T15:39:37.401 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-16T15:39:35.722791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":178,"acting":178,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236964,"kb_used_data":4080,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737756,"statfs":{"total":773094113280,"available":772851462144,"internally_reserved":0,"allocated":4177920,"data_stored":2320186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.581842"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154188+0000","last_change":"2024-09-16T15:39:32.154188+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154188+0000","last_undegraded":"2024-09-16T15:39:32.154188+0000","last_fullsized":"2024-09-16T15:39:32.154188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159798+0000","last_change":"2024-09-16T15:39:32.159798+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159798+0000","last_undegraded":"2024-09-16T15:39:32.159798+0000","last_fullsized":"2024-09-16T15:39:32.159798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156813+0000","last_change":"2024-09-16T15:39:32.156813+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156813+0000","last_undegraded":"2024-09-16T15:39:32.156813+0000","last_fullsized":"2024-09-16T15:39:32.156813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:32.138319+0000","last_change":"2024-09-16T15:39:32.138319+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.138319+0000","last_undegraded":"2024-09-16T15:39:32.138319+0000","last_fullsized":"2024-09-16T15:39:32.138319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156011+0000","last_change":"2024-09-16T15:39:32.156011+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156011+0000","last_undegraded":"2024-09-16T15:39:32.156011+0000","last_fullsized":"2024-09-16T15:39:32.156011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.158243+0000","last_change":"2024-09-16T15:39:32.158243+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.158243+0000","last_undegraded":"2024-09-16T15:39:32.158243+0000","last_fullsized":"2024-09-16T15:39:32.158243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159374+0000","last_change":"2024-09-16T15:39:32.159374+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159374+0000","last_undegraded":"2024-09-16T15:39:32.159374+0000","last_fullsized":"2024-09-16T15:39:32.159374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165592+0000","last_change":"2024-09-16T15:39:32.165592+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165592+0000","last_undegraded":"2024-09-16T15:39:32.165592+0000","last_fullsized":"2024-09-16T15:39:32.165592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150787+0000","last_change":"2024-09-16T15:39:32.150787+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150787+0000","last_undegraded":"2024-09-16T15:39:32.150787+0000","last_fullsized":"2024-09-16T15:39:32.150787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:32.138319+0000","last_change":"2024-09-16T15:39:32.138319+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.138319+0000","last_undegraded":"2024-09-16T15:39:32.138319+0000","last_fullsized":"2024-09-16T15:39:32.138319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165508+0000","last_change":"2024-09-16T15:39:32.165508+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165508+0000","last_undegraded":"2024-09-16T15:39:32.165508+0000","last_fullsized":"2024-09-16T15:39:32.165508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156865+0000","last_change":"2024-09-16T15:39:32.156865+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156865+0000","last_undegraded":"2024-09-16T15:39:32.156865+0000","last_fullsized":"2024-09-16T15:39:32.156865+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151813+0000","last_change":"2024-09-16T15:39:32.151813+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151813+0000","last_undegraded":"2024-09-16T15:39:32.151813+0000","last_fullsized":"2024-09-16T15:39:32.151813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156333+0000","last_change":"2024-09-16T15:39:32.156333+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156333+0000","last_undegraded":"2024-09-16T15:39:32.156333+0000","last_fullsized":"2024-09-16T15:39:32.156333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.161031+0000","last_change":"2024-09-16T15:39:32.161031+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.161031+0000","last_undegraded":"2024-09-16T15:39:32.161031+0000","last_fullsized":"2024-09-16T15:39:32.161031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.158855+0000","last_change":"2024-09-16T15:39:32.158855+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.158855+0000","last_undegraded":"2024-09-16T15:39:32.158855+0000","last_fullsized":"2024-09-16T15:39:32.158855+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.164804+0000","last_change":"2024-09-16T15:39:32.164804+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.164804+0000","last_undegraded":"2024-09-16T15:39:32.164804+0000","last_fullsized":"2024-09-16T15:39:32.164804+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156100+0000","last_change":"2024-09-16T15:39:32.156100+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156100+0000","last_undegraded":"2024-09-16T15:39:32.156100+0000","last_fullsized":"2024-09-16T15:39:32.156100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153593+0000","last_change":"2024-09-16T15:39:32.153593+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153593+0000","last_undegraded":"2024-09-16T15:39:32.153593+0000","last_fullsized":"2024-09-16T15:39:32.153593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152311+0000","last_change":"2024-09-16T15:39:32.152311+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152311+0000","last_undegraded":"2024-09-16T15:39:32.152311+0000","last_fullsized":"2024-09-16T15:39:32.152311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152875+0000","last_change":"2024-09-16T15:39:32.152875+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152875+0000","last_undegraded":"2024-09-16T15:39:32.152875+0000","last_fullsized":"2024-09-16T15:39:32.152875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151657+0000","last_change":"2024-09-16T15:39:32.151657+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151657+0000","last_undegraded":"2024-09-16T15:39:32.151657+0000","last_fullsized":"2024-09-16T15:39:32.151657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151923+0000","last_change":"2024-09-16T15:39:32.151923+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151923+0000","last_undegraded":"2024-09-16T15:39:32.151923+0000","last_fullsized":"2024-09-16T15:39:32.151923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155225+0000","last_change":"2024-09-16T15:39:32.155225+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155225+0000","last_undegraded":"2024-09-16T15:39:32.155225+0000","last_fullsized":"2024-09-16T15:39:32.155225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:32.138319+0000","last_change":"2024-09-16T15:39:32.138319+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.138319+0000","last_undegraded":"2024-09-16T15:39:32.138319+0000","last_fullsized":"2024-09-16T15:39:32.138319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153291+0000","last_change":"2024-09-16T15:39:32.153291+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153291+0000","last_undegraded":"2024-09-16T15:39:32.153291+0000","last_fullsized":"2024-09-16T15:39:32.153291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:32.138319+0000","last_change":"2024-09-16T15:39:32.138319+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.138319+0000","last_undegraded":"2024-09-16T15:39:32.138319+0000","last_fullsized":"2024-09-16T15:39:32.138319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156476+0000","last_change":"2024-09-16T15:39:32.156476+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156476+0000","last_undegraded":"2024-09-16T15:39:32.156476+0000","last_fullsized":"2024-09-16T15:39:32.156476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152413+0000","last_change":"2024-09-16T15:39:32.152413+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152413+0000","last_undegraded":"2024-09-16T15:39:32.152413+0000","last_fullsized":"2024-09-16T15:39:32.152413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165973+0000","last_change":"2024-09-16T15:39:32.165973+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165973+0000","last_undegraded":"2024-09-16T15:39:32.165973+0000","last_fullsized":"2024-09-16T15:39:32.165973+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155795+0000","last_change":"2024-09-16T15:39:32.155795+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155795+0000","last_undegraded":"2024-09-16T15:39:32.155795+0000","last_fullsized":"2024-09-16T15:39:32.155795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165147+0000","last_change":"2024-09-16T15:39:32.165147+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165147+0000","last_undegraded":"2024-09-16T15:39:32.165147+0000","last_fullsized":"2024-09-16T15:39:32.165147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150200+0000","last_change":"2024-09-16T15:39:32.150200+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150200+0000","last_undegraded":"2024-09-16T15:39:32.150200+0000","last_fullsized":"2024-09-16T15:39:32.150200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.168107+0000","last_change":"2024-09-16T15:39:32.164389+0000","last_active":"2024-09-16T15:39:32.168107+0000","last_peered":"2024-09-16T15:39:32.168107+0000","last_clean":"2024-09-16T15:39:32.168107+0000","last_became_active":"2024-09-16T15:39:32.164146+0000","last_became_peered":"2024-09-16T15:39:32.164146+0000","last_unstale":"2024-09-16T15:39:32.168107+0000","last_undegraded":"2024-09-16T15:39:32.168107+0000","last_fullsized":"2024-09-16T15:39:32.168107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:22.804697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153708+0000","last_change":"2024-09-16T15:39:32.153708+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153708+0000","last_undegraded":"2024-09-16T15:39:32.153708+0000","last_fullsized":"2024-09-16T15:39:32.153708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167838+0000","last_change":"2024-09-16T15:39:32.166998+0000","last_active":"2024-09-16T15:39:32.167838+0000","last_peered":"2024-09-16T15:39:32.167838+0000","last_clean":"2024-09-16T15:39:32.167838+0000","last_became_active":"2024-09-16T15:39:32.166503+0000","last_became_peered":"2024-09-16T15:39:32.166503+0000","last_unstale":"2024-09-16T15:39:32.167838+0000","last_undegraded":"2024-09-16T15:39:32.167838+0000","last_fullsized":"2024-09-16T15:39:32.167838+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:39.190074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151859+0000","last_change":"2024-09-16T15:39:32.151859+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151859+0000","last_undegraded":"2024-09-16T15:39:32.151859+0000","last_fullsized":"2024-09-16T15:39:32.151859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.173010+0000","last_change":"2024-09-16T15:39:32.170881+0000","last_active":"2024-09-16T15:39:32.173010+0000","last_peered":"2024-09-16T15:39:32.173010+0000","last_clean":"2024-09-16T15:39:32.173010+0000","last_became_active":"2024-09-16T15:39:32.170639+0000","last_became_peered":"2024-09-16T15:39:32.170639+0000","last_unstale":"2024-09-16T15:39:32.173010+0000","last_undegraded":"2024-09-16T15:39:32.173010+0000","last_fullsized":"2024-09-16T15:39:32.173010+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:03.240823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:32.138319+0000","last_change":"2024-09-16T15:39:32.138319+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.138319+0000","last_undegraded":"2024-09-16T15:39:32.138319+0000","last_fullsized":"2024-09-16T15:39:32.138319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172991+0000","last_change":"2024-09-16T15:39:32.171713+0000","last_active":"2024-09-16T15:39:32.172991+0000","last_peered":"2024-09-16T15:39:32.172991+0000","last_clean":"2024-09-16T15:39:32.172991+0000","last_became_active":"2024-09-16T15:39:32.171452+0000","last_became_peered":"2024-09-16T15:39:32.171452+0000","last_unstale":"2024-09-16T15:39:32.172991+0000","last_undegraded":"2024-09-16T15:39:32.172991+0000","last_fullsized":"2024-09-16T15:39:32.172991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:45.685224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165563+0000","last_change":"2024-09-16T15:39:32.165563+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165563+0000","last_undegraded":"2024-09-16T15:39:32.165563+0000","last_fullsized":"2024-09-16T15:39:32.165563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166016+0000","last_change":"2024-09-16T15:39:32.162895+0000","last_active":"2024-09-16T15:39:32.166016+0000","last_peered":"2024-09-16T15:39:32.166016+0000","last_clean":"2024-09-16T15:39:32.166016+0000","last_became_active":"2024-09-16T15:39:32.162267+0000","last_became_peered":"2024-09-16T15:39:32.162267+0000","last_unstale":"2024-09-16T15:39:32.166016+0000","last_undegraded":"2024-09-16T15:39:32.166016+0000","last_fullsized":"2024-09-16T15:39:32.166016+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:35.359481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155925+0000","last_change":"2024-09-16T15:39:32.155925+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155925+0000","last_undegraded":"2024-09-16T15:39:32.155925+0000","last_fullsized":"2024-09-16T15:39:32.155925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166200+0000","last_change":"2024-09-16T15:39:32.163428+0000","last_active":"2024-09-16T15:39:32.166200+0000","last_peered":"2024-09-16T15:39:32.166200+0000","last_clean":"2024-09-16T15:39:32.166200+0000","last_became_active":"2024-09-16T15:39:32.163148+0000","last_became_peered":"2024-09-16T15:39:32.163148+0000","last_unstale":"2024-09-16T15:39:32.166200+0000","last_undegraded":"2024-09-16T15:39:32.166200+0000","last_fullsized":"2024-09-16T15:39:32.166200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:31.750012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150903+0000","last_change":"2024-09-16T15:39:32.150903+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150903+0000","last_undegraded":"2024-09-16T15:39:32.150903+0000","last_fullsized":"2024-09-16T15:39:32.150903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171837+0000","last_change":"2024-09-16T15:39:32.164943+0000","last_active":"2024-09-16T15:39:32.171837+0000","last_peered":"2024-09-16T15:39:32.171837+0000","last_clean":"2024-09-16T15:39:32.171837+0000","last_became_active":"2024-09-16T15:39:32.164576+0000","last_became_peered":"2024-09-16T15:39:32.164576+0000","last_unstale":"2024-09-16T15:39:32.171837+0000","last_undegraded":"2024-09-16T15:39:32.171837+0000","last_fullsized":"2024-09-16T15:39:32.171837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:36.312022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:32.138319+0000","last_change":"2024-09-16T15:39:32.138319+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.138319+0000","last_undegraded":"2024-09-16T15:39:32.138319+0000","last_fullsized":"2024-09-16T15:39:32.138319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165602+0000","last_change":"2024-09-16T15:39:32.160722+0000","last_active":"2024-09-16T15:39:32.165602+0000","last_peered":"2024-09-16T15:39:32.165602+0000","last_clean":"2024-09-16T15:39:32.165602+0000","last_became_active":"2024-09-16T15:39:32.160298+0000","last_became_peered":"2024-09-16T15:39:32.160298+0000","last_unstale":"2024-09-16T15:39:32.165602+0000","last_undegraded":"2024-09-16T15:39:32.165602+0000","last_fullsized":"2024-09-16T15:39:32.165602+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:32.567093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156442+0000","last_change":"2024-09-16T15:39:32.156442+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156442+0000","last_undegraded":"2024-09-16T15:39:32.156442+0000","last_fullsized":"2024-09-16T15:39:32.156442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172968+0000","last_change":"2024-09-16T15:39:32.170156+0000","last_active":"2024-09-16T15:39:32.172968+0000","last_peered":"2024-09-16T15:39:32.172968+0000","last_clean":"2024-09-16T15:39:32.172968+0000","last_became_active":"2024-09-16T15:39:32.169818+0000","last_became_peered":"2024-09-16T15:39:32.169818+0000","last_unstale":"2024-09-16T15:39:32.172968+0000","last_undegraded":"2024-09-16T15:39:32.172968+0000","last_fullsized":"2024-09-16T15:39:32.172968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:42.329583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153208+0000","last_change":"2024-09-16T15:39:32.153208+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153208+0000","last_undegraded":"2024-09-16T15:39:32.153208+0000","last_fullsized":"2024-09-16T15:39:32.153208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165089+0000","last_change":"2024-09-16T15:39:32.161560+0000","last_active":"2024-09-16T15:39:32.165089+0000","last_peered":"2024-09-16T15:39:32.165089+0000","last_clean":"2024-09-16T15:39:32.165089+0000","last_became_active":"2024-09-16T15:39:32.161287+0000","last_became_peered":"2024-09-16T15:39:32.161287+0000","last_unstale":"2024-09-16T15:39:32.165089+0000","last_undegraded":"2024-09-16T15:39:32.165089+0000","last_fullsized":"2024-09-16T15:39:32.165089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:33.389071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154532+0000","last_change":"2024-09-16T15:39:32.154532+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154532+0000","last_undegraded":"2024-09-16T15:39:32.154532+0000","last_fullsized":"2024-09-16T15:39:32.154532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.175093+0000","last_change":"2024-09-16T15:39:32.170843+0000","last_active":"2024-09-16T15:39:32.175093+0000","last_peered":"2024-09-16T15:39:32.175093+0000","last_clean":"2024-09-16T15:39:32.175093+0000","last_became_active":"2024-09-16T15:39:32.170538+0000","last_became_peered":"2024-09-16T15:39:32.170538+0000","last_unstale":"2024-09-16T15:39:32.175093+0000","last_undegraded":"2024-09-16T15:39:32.175093+0000","last_fullsized":"2024-09-16T15:39:32.175093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:06.203396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.157250+0000","last_change":"2024-09-16T15:39:32.157250+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.157250+0000","last_undegraded":"2024-09-16T15:39:32.157250+0000","last_fullsized":"2024-09-16T15:39:32.157250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171781+0000","last_change":"2024-09-16T15:39:32.165779+0000","last_active":"2024-09-16T15:39:32.171781+0000","last_peered":"2024-09-16T15:39:32.171781+0000","last_clean":"2024-09-16T15:39:32.171781+0000","last_became_active":"2024-09-16T15:39:32.165510+0000","last_became_peered":"2024-09-16T15:39:32.165510+0000","last_unstale":"2024-09-16T15:39:32.171781+0000","last_undegraded":"2024-09-16T15:39:32.171781+0000","last_fullsized":"2024-09-16T15:39:32.171781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:08.898328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:32.138319+0000","last_change":"2024-09-16T15:39:32.138319+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.138319+0000","last_undegraded":"2024-09-16T15:39:32.138319+0000","last_fullsized":"2024-09-16T15:39:32.138319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166107+0000","last_change":"2024-09-16T15:39:32.163172+0000","last_active":"2024-09-16T15:39:32.166107+0000","last_peered":"2024-09-16T15:39:32.166107+0000","last_clean":"2024-09-16T15:39:32.166107+0000","last_became_active":"2024-09-16T15:39:32.162626+0000","last_became_peered":"2024-09-16T15:39:32.162626+0000","last_unstale":"2024-09-16T15:39:32.166107+0000","last_undegraded":"2024-09-16T15:39:32.166107+0000","last_fullsized":"2024-09-16T15:39:32.166107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:49.229870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154053+0000","last_change":"2024-09-16T15:39:32.154053+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154053+0000","last_undegraded":"2024-09-16T15:39:32.154053+0000","last_fullsized":"2024-09-16T15:39:32.154053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T15:39:31.144218+0000","last_change":"2024-09-16T15:39:31.144218+0000","last_active":"2024-09-16T15:39:31.130728+0000","last_peered":"2024-09-16T15:39:31.130728+0000","last_clean":"2024-09-16T15:39:31.130728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:31.144218+0000","last_undegraded":"2024-09-16T15:39:31.144218+0000","last_fullsized":"2024-09-16T15:39:31.144218+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.163921+0000","last_change":"2024-09-16T15:39:32.163921+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.163921+0000","last_undegraded":"2024-09-16T15:39:32.163921+0000","last_fullsized":"2024-09-16T15:39:32.163921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171868+0000","last_change":"2024-09-16T15:39:32.169367+0000","last_active":"2024-09-16T15:39:32.171868+0000","last_peered":"2024-09-16T15:39:32.171868+0000","last_clean":"2024-09-16T15:39:32.171868+0000","last_became_active":"2024-09-16T15:39:32.169188+0000","last_became_peered":"2024-09-16T15:39:32.169188+0000","last_unstale":"2024-09-16T15:39:32.171868+0000","last_undegraded":"2024-09-16T15:39:32.171868+0000","last_fullsized":"2024-09-16T15:39:32.171868+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:14.156509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151486+0000","last_change":"2024-09-16T15:39:32.151486+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151486+0000","last_undegraded":"2024-09-16T15:39:32.151486+0000","last_fullsized":"2024-09-16T15:39:32.151486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.168920+0000","last_change":"2024-09-16T15:39:32.163551+0000","last_active":"2024-09-16T15:39:32.168920+0000","last_peered":"2024-09-16T15:39:32.168920+0000","last_clean":"2024-09-16T15:39:32.168920+0000","last_became_active":"2024-09-16T15:39:32.163283+0000","last_became_peered":"2024-09-16T15:39:32.163283+0000","last_unstale":"2024-09-16T15:39:32.168920+0000","last_undegraded":"2024-09-16T15:39:32.168920+0000","last_fullsized":"2024-09-16T15:39:32.168920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:54.716352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151031+0000","last_change":"2024-09-16T15:39:32.151031+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151031+0000","last_undegraded":"2024-09-16T15:39:32.151031+0000","last_fullsized":"2024-09-16T15:39:32.151031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T15:39:31.144484+0000","last_change":"2024-09-16T15:39:31.144484+0000","last_active":"2024-09-16T15:39:31.130728+0000","last_peered":"2024-09-16T15:39:31.130728+0000","last_clean":"2024-09-16T15:39:31.130728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:31.144484+0000","last_undegraded":"2024-09-16T15:39:31.144484+0000","last_fullsized":"2024-09-16T15:39:31.144484+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.157317+0000","last_change":"2024-09-16T15:39:32.157317+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.157317+0000","last_undegraded":"2024-09-16T15:39:32.157317+0000","last_fullsized":"2024-09-16T15:39:32.157317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166121+0000","last_change":"2024-09-16T15:39:32.163386+0000","last_active":"2024-09-16T15:39:32.166121+0000","last_peered":"2024-09-16T15:39:32.166121+0000","last_clean":"2024-09-16T15:39:32.166121+0000","last_became_active":"2024-09-16T15:39:32.161214+0000","last_became_peered":"2024-09-16T15:39:32.161214+0000","last_unstale":"2024-09-16T15:39:32.166121+0000","last_undegraded":"2024-09-16T15:39:32.166121+0000","last_fullsized":"2024-09-16T15:39:32.166121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:34.093301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:32.138319+0000","last_change":"2024-09-16T15:39:32.138319+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.138319+0000","last_undegraded":"2024-09-16T15:39:32.138319+0000","last_fullsized":"2024-09-16T15:39:32.138319+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.169802+0000","last_change":"2024-09-16T15:39:32.166140+0000","last_active":"2024-09-16T15:39:32.169802+0000","last_peered":"2024-09-16T15:39:32.169802+0000","last_clean":"2024-09-16T15:39:32.169802+0000","last_became_active":"2024-09-16T15:39:32.165774+0000","last_became_peered":"2024-09-16T15:39:32.165774+0000","last_unstale":"2024-09-16T15:39:32.169802+0000","last_undegraded":"2024-09-16T15:39:32.169802+0000","last_fullsized":"2024-09-16T15:39:32.169802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:48.317883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151377+0000","last_change":"2024-09-16T15:39:32.151377+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151377+0000","last_undegraded":"2024-09-16T15:39:32.151377+0000","last_fullsized":"2024-09-16T15:39:32.151377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165944+0000","last_change":"2024-09-16T15:39:32.162492+0000","last_active":"2024-09-16T15:39:32.165944+0000","last_peered":"2024-09-16T15:39:32.165944+0000","last_clean":"2024-09-16T15:39:32.165944+0000","last_became_active":"2024-09-16T15:39:32.162229+0000","last_became_peered":"2024-09-16T15:39:32.162229+0000","last_unstale":"2024-09-16T15:39:32.165944+0000","last_undegraded":"2024-09-16T15:39:32.165944+0000","last_fullsized":"2024-09-16T15:39:32.165944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:19.223491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150686+0000","last_change":"2024-09-16T15:39:32.150686+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150686+0000","last_undegraded":"2024-09-16T15:39:32.150686+0000","last_fullsized":"2024-09-16T15:39:32.150686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.164987+0000","last_change":"2024-09-16T15:39:32.161540+0000","last_active":"2024-09-16T15:39:32.164987+0000","last_peered":"2024-09-16T15:39:32.164987+0000","last_clean":"2024-09-16T15:39:32.164987+0000","last_became_active":"2024-09-16T15:39:32.161225+0000","last_became_peered":"2024-09-16T15:39:32.161225+0000","last_unstale":"2024-09-16T15:39:32.164987+0000","last_undegraded":"2024-09-16T15:39:32.164987+0000","last_fullsized":"2024-09-16T15:39:32.164987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:21:04.290055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151288+0000","last_change":"2024-09-16T15:39:32.151288+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151288+0000","last_undegraded":"2024-09-16T15:39:32.151288+0000","last_fullsized":"2024-09-16T15:39:32.151288+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.160197+0000","last_change":"2024-09-16T15:39:32.157654+0000","last_active":"2024-09-16T15:39:32.160197+0000","last_peered":"2024-09-16T15:39:32.160197+0000","last_clean":"2024-09-16T15:39:32.160197+0000","last_became_active":"2024-09-16T15:39:32.157393+0000","last_became_peered":"2024-09-16T15:39:32.157393+0000","last_unstale":"2024-09-16T15:39:32.160197+0000","last_undegraded":"2024-09-16T15:39:32.160197+0000","last_fullsized":"2024-09-16T15:39:32.160197+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:07.931223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154817+0000","last_change":"2024-09-16T15:39:32.154817+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154817+0000","last_undegraded":"2024-09-16T15:39:32.154817+0000","last_fullsized":"2024-09-16T15:39:32.154817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166535+0000","last_change":"2024-09-16T15:39:32.162100+0000","last_active":"2024-09-16T15:39:32.166535+0000","last_peered":"2024-09-16T15:39:32.166535+0000","last_clean":"2024-09-16T15:39:32.166535+0000","last_became_active":"2024-09-16T15:39:32.161782+0000","last_became_peered":"2024-09-16T15:39:32.161782+0000","last_unstale":"2024-09-16T15:39:32.166535+0000","last_undegraded":"2024-09-16T15:39:32.166535+0000","last_fullsized":"2024-09-16T15:39:32.166535+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:51:50.007487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.161447+0000","last_change":"2024-09-16T15:39:32.161447+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.161447+0000","last_undegraded":"2024-09-16T15:39:32.161447+0000","last_fullsized":"2024-09-16T15:39:32.161447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172551+0000","last_change":"2024-09-16T15:39:32.168491+0000","last_active":"2024-09-16T15:39:32.172551+0000","last_peered":"2024-09-16T15:39:32.172551+0000","last_clean":"2024-09-16T15:39:32.172551+0000","last_became_active":"2024-09-16T15:39:32.168195+0000","last_became_peered":"2024-09-16T15:39:32.168195+0000","last_unstale":"2024-09-16T15:39:32.172551+0000","last_undegraded":"2024-09-16T15:39:32.172551+0000","last_fullsized":"2024-09-16T15:39:32.172551+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:00.428705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150369+0000","last_change":"2024-09-16T15:39:32.150369+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150369+0000","last_undegraded":"2024-09-16T15:39:32.150369+0000","last_fullsized":"2024-09-16T15:39:32.150369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T15:39:31.144074+0000","last_change":"2024-09-16T15:39:31.144074+0000","last_active":"2024-09-16T15:39:31.130728+0000","last_peered":"2024-09-16T15:39:31.130728+0000","last_clean":"2024-09-16T15:39:31.130728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:31.144074+0000","last_undegraded":"2024-09-16T15:39:31.144074+0000","last_fullsized":"2024-09-16T15:39:31.144074+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156915+0000","last_change":"2024-09-16T15:39:32.156915+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156915+0000","last_undegraded":"2024-09-16T15:39:32.156915+0000","last_fullsized":"2024-09-16T15:39:32.156915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.158697+0000","last_change":"2024-09-16T15:39:32.155559+0000","last_active":"2024-09-16T15:39:32.158697+0000","last_peered":"2024-09-16T15:39:32.158697+0000","last_clean":"2024-09-16T15:39:32.158697+0000","last_became_active":"2024-09-16T15:39:32.155286+0000","last_became_peered":"2024-09-16T15:39:32.155286+0000","last_unstale":"2024-09-16T15:39:32.158697+0000","last_undegraded":"2024-09-16T15:39:32.158697+0000","last_fullsized":"2024-09-16T15:39:32.158697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:53:37.151552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150242+0000","last_change":"2024-09-16T15:39:32.150242+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150242+0000","last_undegraded":"2024-09-16T15:39:32.150242+0000","last_fullsized":"2024-09-16T15:39:32.150242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T15:39:31.143699+0000","last_change":"2024-09-16T15:39:31.143699+0000","last_active":"2024-09-16T15:39:31.130728+0000","last_peered":"2024-09-16T15:39:31.130728+0000","last_clean":"2024-09-16T15:39:31.130728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:31.143699+0000","last_undegraded":"2024-09-16T15:39:31.143699+0000","last_fullsized":"2024-09-16T15:39:31.143699+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151198+0000","last_change":"2024-09-16T15:39:32.151198+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151198+0000","last_undegraded":"2024-09-16T15:39:32.151198+0000","last_fullsized":"2024-09-16T15:39:32.151198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165849+0000","last_change":"2024-09-16T15:39:32.164165+0000","last_active":"2024-09-16T15:39:32.165849+0000","last_peered":"2024-09-16T15:39:32.165849+0000","last_clean":"2024-09-16T15:39:32.165849+0000","last_became_active":"2024-09-16T15:39:32.163728+0000","last_became_peered":"2024-09-16T15:39:32.163728+0000","last_unstale":"2024-09-16T15:39:32.165849+0000","last_undegraded":"2024-09-16T15:39:32.165849+0000","last_fullsized":"2024-09-16T15:39:32.165849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:07.235894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150024+0000","last_change":"2024-09-16T15:39:32.150024+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150024+0000","last_undegraded":"2024-09-16T15:39:32.150024+0000","last_fullsized":"2024-09-16T15:39:32.150024+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-16T15:39:31.143760+0000","last_change":"2024-09-16T15:39:31.143760+0000","last_active":"2024-09-16T15:39:31.130728+0000","last_peered":"2024-09-16T15:39:31.130728+0000","last_clean":"2024-09-16T15:39:31.130728+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:31.143760+0000","last_undegraded":"2024-09-16T15:39:31.143760+0000","last_fullsized":"2024-09-16T15:39:31.143760+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152940+0000","last_change":"2024-09-16T15:39:32.152940+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152940+0000","last_undegraded":"2024-09-16T15:39:32.152940+0000","last_fullsized":"2024-09-16T15:39:32.152940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172404+0000","last_change":"2024-09-16T15:39:32.169431+0000","last_active":"2024-09-16T15:39:32.172404+0000","last_peered":"2024-09-16T15:39:32.172404+0000","last_clean":"2024-09-16T15:39:32.172404+0000","last_became_active":"2024-09-16T15:39:32.169166+0000","last_became_peered":"2024-09-16T15:39:32.169166+0000","last_unstale":"2024-09-16T15:39:32.172404+0000","last_undegraded":"2024-09-16T15:39:32.172404+0000","last_fullsized":"2024-09-16T15:39:32.172404+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:12.103417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153357+0000","last_change":"2024-09-16T15:39:32.153357+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153357+0000","last_undegraded":"2024-09-16T15:39:32.153357+0000","last_fullsized":"2024-09-16T15:39:32.153357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171856+0000","last_change":"2024-09-16T15:39:32.170091+0000","last_active":"2024-09-16T15:39:32.171856+0000","last_peered":"2024-09-16T15:39:32.171856+0000","last_clean":"2024-09-16T15:39:32.171856+0000","last_became_active":"2024-09-16T15:39:32.169831+0000","last_became_peered":"2024-09-16T15:39:32.169831+0000","last_unstale":"2024-09-16T15:39:32.171856+0000","last_undegraded":"2024-09-16T15:39:32.171856+0000","last_fullsized":"2024-09-16T15:39:32.171856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:18.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159832+0000","last_change":"2024-09-16T15:39:32.159832+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159832+0000","last_undegraded":"2024-09-16T15:39:32.159832+0000","last_fullsized":"2024-09-16T15:39:32.159832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167962+0000","last_change":"2024-09-16T15:39:32.163553+0000","last_active":"2024-09-16T15:39:32.167962+0000","last_peered":"2024-09-16T15:39:32.167962+0000","last_clean":"2024-09-16T15:39:32.167962+0000","last_became_active":"2024-09-16T15:39:32.162649+0000","last_became_peered":"2024-09-16T15:39:32.162649+0000","last_unstale":"2024-09-16T15:39:32.167962+0000","last_undegraded":"2024-09-16T15:39:32.167962+0000","last_fullsized":"2024-09-16T15:39:32.167962+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:12.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":131,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167970+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:39:32.167970+0000","last_peered":"2024-09-16T15:39:32.167970+0000","last_clean":"2024-09-16T15:39:32.167970+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:39:32.167970+0000","last_undegraded":"2024-09-16T15:39:32.167970+0000","last_fullsized":"2024-09-16T15:39:32.167970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":734366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":970752,"data_stored":734366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":364544,"data_stored":137731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:37.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:37.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.601+0000 7f56d1200640 1 -- 172.21.15.60:0/94211892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0b2440 msgr2=0x7f56cc06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.601+0000 7f56d1200640 1 --2- 172.21.15.60:0/94211892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0b2440 0x7f56cc06f160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009be0 tx=0x7f56c402a820 comp rx=0 tx=0).stop 2024-09-16T15:39:37.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.601+0000 7f56d1200640 1 -- 172.21.15.60:0/94211892 shutdown_connections 2024-09-16T15:39:37.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.601+0000 7f56d1200640 1 --2- 172.21.15.60:0/94211892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0b2440 0x7f56cc06f160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.601+0000 7f56d1200640 1 -- 172.21.15.60:0/94211892 >> 172.21.15.60:0/94211892 conn(0x7f56cc0b1950 msgr2=0x7f56cc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.601+0000 7f56d1200640 1 -- 172.21.15.60:0/94211892 shutdown_connections 2024-09-16T15:39:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.601+0000 7f56d1200640 1 -- 172.21.15.60:0/94211892 wait complete. 2024-09-16T15:39:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.602+0000 7f56d1200640 1 Processor -- start 2024-09-16T15:39:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.602+0000 7f56d1200640 1 -- start start 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.603+0000 7f56d1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc0b2440 0x7f56cc1f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.603+0000 7f56d1200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccf00 0x7f56cc0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.603+0000 7f56d1200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc15f5a0 0x7f56cc1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.603+0000 7f56d1200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc074840 con 0x7f56cc0b2440 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.603+0000 7f56d1200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc0746c0 con 0x7f56cc15f5a0 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.603+0000 7f56d1200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc06f6a0 con 0x7f56cc0ccf00 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc0b2440 0x7f56cc1f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc0b2440 0x7f56cc1f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42092/0 (socket says 172.21.15.60:42092) 2024-09-16T15:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 -- 172.21.15.60:0/4079030109 learned_addr learned my addr 172.21.15.60:0/4079030109 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56ca800640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc15f5a0 0x7f56cc1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c1400640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccf00 0x7f56cc0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 -- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccf00 msgr2=0x7f56cc0b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccf00 0x7f56cc0b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 -- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc15f5a0 msgr2=0x7f56cc1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc15f5a0 0x7f56cc1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 -- 172.21.15.60:0/4079030109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc0aff90 con 0x7f56cc0b2440 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c1400640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccf00 0x7f56cc0b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.604+0000 7f56c9e00640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc0b2440 0x7f56cc1f52b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c4d0 tx=0x7f56bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.605+0000 7f56c2a00640 1 -- 172.21.15.60:0/4079030109 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc016020 con 0x7f56cc0b2440 2024-09-16T15:39:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.605+0000 7f56c2a00640 1 -- 172.21.15.60:0/4079030109 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56bc012920 con 0x7f56cc0b2440 2024-09-16T15:39:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.605+0000 7f56c2a00640 1 -- 172.21.15.60:0/4079030109 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc012d70 con 0x7f56cc0b2440 2024-09-16T15:39:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.605+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc0b0270 con 0x7f56cc0b2440 2024-09-16T15:39:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.605+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc0b0830 con 0x7f56cc0b2440 2024-09-16T15:39:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.606+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc0b1570 con 0x7f56cc0b2440 2024-09-16T15:39:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.607+0000 7f56c2a00640 1 -- 172.21.15.60:0/4079030109 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f56bc019020 con 0x7f56cc0b2440 2024-09-16T15:39:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.608+0000 7f56c2a00640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f56a8078880 0x7f56a807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.608+0000 7f56c2a00640 1 -- 172.21.15.60:0/4079030109 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f56bc09aa10 con 0x7f56cc0b2440 2024-09-16T15:39:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.608+0000 7f56c1400640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f56a8078880 0x7f56a807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.609+0000 7f56c1400640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f56a8078880 0x7f56a807ad40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f56cc1f7d80 tx=0x7f56b4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.612+0000 7f56c2a00640 1 -- 172.21.15.60:0/4079030109 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f56bc020020 con 0x7f56cc0b2440 2024-09-16T15:39:37.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.806+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56cc06f160 con 0x7f56a8078880 2024-09-16T15:39:37.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.811+0000 7f56c2a00640 1 -- 172.21.15.60:0/4079030109 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247320 (secure 0 0 0) 0x7f56cc06f160 con 0x7f56a8078880 2024-09-16T15:39:37.814 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:37.816 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:39:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f56a8078880 msgr2=0x7f56a807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f56a8078880 0x7f56a807ad40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f56cc1f7d80 tx=0x7f56b4009040 comp rx=0 tx=0).stop 2024-09-16T15:39:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc0b2440 msgr2=0x7f56cc1f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc0b2440 0x7f56cc1f52b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00c4d0 tx=0x7f56bc00c990 comp rx=0 tx=0).stop 2024-09-16T15:39:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 shutdown_connections 2024-09-16T15:39:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f56a8078880 0x7f56a807ad40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc15f5a0 0x7f56cc1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccf00 0x7f56cc0b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 --2- 172.21.15.60:0/4079030109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc0b2440 0x7f56cc1f52b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 >> 172.21.15.60:0/4079030109 conn(0x7f56cc0b1950 msgr2=0x7f56cc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 shutdown_connections 2024-09-16T15:39:37.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:37.815+0000 7f56d1200640 1 -- 172.21.15.60:0/4079030109 wait complete. 2024-09-16T15:39:37.832 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-16T15:39:37.723221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237024,"kb_used_data":4132,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737696,"statfs":{"total":773094113280,"available":772851400704,"internally_reserved":0,"allocated":4231168,"data_stored":2341162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154188+0000","last_change":"2024-09-16T15:39:32.154188+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154188+0000","last_undegraded":"2024-09-16T15:39:32.154188+0000","last_fullsized":"2024-09-16T15:39:32.154188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159798+0000","last_change":"2024-09-16T15:39:32.159798+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159798+0000","last_undegraded":"2024-09-16T15:39:32.159798+0000","last_fullsized":"2024-09-16T15:39:32.159798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156813+0000","last_change":"2024-09-16T15:39:32.156813+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156813+0000","last_undegraded":"2024-09-16T15:39:32.156813+0000","last_fullsized":"2024-09-16T15:39:32.156813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165367+0000","last_change":"2024-09-16T15:39:33.175477+0000","last_active":"2024-09-16T15:39:34.165367+0000","last_peered":"2024-09-16T15:39:34.165367+0000","last_clean":"2024-09-16T15:39:34.165367+0000","last_became_active":"2024-09-16T15:39:33.174529+0000","last_became_peered":"2024-09-16T15:39:33.174529+0000","last_unstale":"2024-09-16T15:39:34.165367+0000","last_undegraded":"2024-09-16T15:39:34.165367+0000","last_fullsized":"2024-09-16T15:39:34.165367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:16.836397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156011+0000","last_change":"2024-09-16T15:39:32.156011+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156011+0000","last_undegraded":"2024-09-16T15:39:32.156011+0000","last_fullsized":"2024-09-16T15:39:32.156011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.158243+0000","last_change":"2024-09-16T15:39:32.158243+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.158243+0000","last_undegraded":"2024-09-16T15:39:32.158243+0000","last_fullsized":"2024-09-16T15:39:32.158243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159374+0000","last_change":"2024-09-16T15:39:32.159374+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159374+0000","last_undegraded":"2024-09-16T15:39:32.159374+0000","last_fullsized":"2024-09-16T15:39:32.159374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165592+0000","last_change":"2024-09-16T15:39:32.165592+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165592+0000","last_undegraded":"2024-09-16T15:39:32.165592+0000","last_fullsized":"2024-09-16T15:39:32.165592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150787+0000","last_change":"2024-09-16T15:39:32.150787+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150787+0000","last_undegraded":"2024-09-16T15:39:32.150787+0000","last_fullsized":"2024-09-16T15:39:32.150787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165700+0000","last_change":"2024-09-16T15:39:33.177863+0000","last_active":"2024-09-16T15:39:34.165700+0000","last_peered":"2024-09-16T15:39:34.165700+0000","last_clean":"2024-09-16T15:39:34.165700+0000","last_became_active":"2024-09-16T15:39:33.176375+0000","last_became_peered":"2024-09-16T15:39:33.176375+0000","last_unstale":"2024-09-16T15:39:34.165700+0000","last_undegraded":"2024-09-16T15:39:34.165700+0000","last_fullsized":"2024-09-16T15:39:34.165700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:10.080289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165508+0000","last_change":"2024-09-16T15:39:32.165508+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165508+0000","last_undegraded":"2024-09-16T15:39:32.165508+0000","last_fullsized":"2024-09-16T15:39:32.165508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156865+0000","last_change":"2024-09-16T15:39:32.156865+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156865+0000","last_undegraded":"2024-09-16T15:39:32.156865+0000","last_fullsized":"2024-09-16T15:39:32.156865+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151813+0000","last_change":"2024-09-16T15:39:32.151813+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151813+0000","last_undegraded":"2024-09-16T15:39:32.151813+0000","last_fullsized":"2024-09-16T15:39:32.151813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156333+0000","last_change":"2024-09-16T15:39:32.156333+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156333+0000","last_undegraded":"2024-09-16T15:39:32.156333+0000","last_fullsized":"2024-09-16T15:39:32.156333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.161031+0000","last_change":"2024-09-16T15:39:32.161031+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.161031+0000","last_undegraded":"2024-09-16T15:39:32.161031+0000","last_fullsized":"2024-09-16T15:39:32.161031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.158855+0000","last_change":"2024-09-16T15:39:32.158855+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.158855+0000","last_undegraded":"2024-09-16T15:39:32.158855+0000","last_fullsized":"2024-09-16T15:39:32.158855+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.164804+0000","last_change":"2024-09-16T15:39:32.164804+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.164804+0000","last_undegraded":"2024-09-16T15:39:32.164804+0000","last_fullsized":"2024-09-16T15:39:32.164804+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156100+0000","last_change":"2024-09-16T15:39:32.156100+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156100+0000","last_undegraded":"2024-09-16T15:39:32.156100+0000","last_fullsized":"2024-09-16T15:39:32.156100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153593+0000","last_change":"2024-09-16T15:39:32.153593+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153593+0000","last_undegraded":"2024-09-16T15:39:32.153593+0000","last_fullsized":"2024-09-16T15:39:32.153593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152311+0000","last_change":"2024-09-16T15:39:32.152311+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152311+0000","last_undegraded":"2024-09-16T15:39:32.152311+0000","last_fullsized":"2024-09-16T15:39:32.152311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152875+0000","last_change":"2024-09-16T15:39:32.152875+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152875+0000","last_undegraded":"2024-09-16T15:39:32.152875+0000","last_fullsized":"2024-09-16T15:39:32.152875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151657+0000","last_change":"2024-09-16T15:39:32.151657+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151657+0000","last_undegraded":"2024-09-16T15:39:32.151657+0000","last_fullsized":"2024-09-16T15:39:32.151657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151923+0000","last_change":"2024-09-16T15:39:32.151923+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151923+0000","last_undegraded":"2024-09-16T15:39:32.151923+0000","last_fullsized":"2024-09-16T15:39:32.151923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155225+0000","last_change":"2024-09-16T15:39:32.155225+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155225+0000","last_undegraded":"2024-09-16T15:39:32.155225+0000","last_fullsized":"2024-09-16T15:39:32.155225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165596+0000","last_change":"2024-09-16T15:39:33.178266+0000","last_active":"2024-09-16T15:39:34.165596+0000","last_peered":"2024-09-16T15:39:34.165596+0000","last_clean":"2024-09-16T15:39:34.165596+0000","last_became_active":"2024-09-16T15:39:33.177230+0000","last_became_peered":"2024-09-16T15:39:33.177230+0000","last_unstale":"2024-09-16T15:39:34.165596+0000","last_undegraded":"2024-09-16T15:39:34.165596+0000","last_fullsized":"2024-09-16T15:39:34.165596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:59.832311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153291+0000","last_change":"2024-09-16T15:39:32.153291+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153291+0000","last_undegraded":"2024-09-16T15:39:32.153291+0000","last_fullsized":"2024-09-16T15:39:32.153291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165652+0000","last_change":"2024-09-16T15:39:33.176217+0000","last_active":"2024-09-16T15:39:34.165652+0000","last_peered":"2024-09-16T15:39:34.165652+0000","last_clean":"2024-09-16T15:39:34.165652+0000","last_became_active":"2024-09-16T15:39:33.175855+0000","last_became_peered":"2024-09-16T15:39:33.175855+0000","last_unstale":"2024-09-16T15:39:34.165652+0000","last_undegraded":"2024-09-16T15:39:34.165652+0000","last_fullsized":"2024-09-16T15:39:34.165652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:45.876870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156476+0000","last_change":"2024-09-16T15:39:32.156476+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156476+0000","last_undegraded":"2024-09-16T15:39:32.156476+0000","last_fullsized":"2024-09-16T15:39:32.156476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152413+0000","last_change":"2024-09-16T15:39:32.152413+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152413+0000","last_undegraded":"2024-09-16T15:39:32.152413+0000","last_fullsized":"2024-09-16T15:39:32.152413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165973+0000","last_change":"2024-09-16T15:39:32.165973+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165973+0000","last_undegraded":"2024-09-16T15:39:32.165973+0000","last_fullsized":"2024-09-16T15:39:32.165973+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155795+0000","last_change":"2024-09-16T15:39:32.155795+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155795+0000","last_undegraded":"2024-09-16T15:39:32.155795+0000","last_fullsized":"2024-09-16T15:39:32.155795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165147+0000","last_change":"2024-09-16T15:39:32.165147+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165147+0000","last_undegraded":"2024-09-16T15:39:32.165147+0000","last_fullsized":"2024-09-16T15:39:32.165147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150200+0000","last_change":"2024-09-16T15:39:32.150200+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150200+0000","last_undegraded":"2024-09-16T15:39:32.150200+0000","last_fullsized":"2024-09-16T15:39:32.150200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.168107+0000","last_change":"2024-09-16T15:39:32.164389+0000","last_active":"2024-09-16T15:39:32.168107+0000","last_peered":"2024-09-16T15:39:32.168107+0000","last_clean":"2024-09-16T15:39:32.168107+0000","last_became_active":"2024-09-16T15:39:32.164146+0000","last_became_peered":"2024-09-16T15:39:32.164146+0000","last_unstale":"2024-09-16T15:39:32.168107+0000","last_undegraded":"2024-09-16T15:39:32.168107+0000","last_fullsized":"2024-09-16T15:39:32.168107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:22.804697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153708+0000","last_change":"2024-09-16T15:39:32.153708+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153708+0000","last_undegraded":"2024-09-16T15:39:32.153708+0000","last_fullsized":"2024-09-16T15:39:32.153708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167838+0000","last_change":"2024-09-16T15:39:32.166998+0000","last_active":"2024-09-16T15:39:32.167838+0000","last_peered":"2024-09-16T15:39:32.167838+0000","last_clean":"2024-09-16T15:39:32.167838+0000","last_became_active":"2024-09-16T15:39:32.166503+0000","last_became_peered":"2024-09-16T15:39:32.166503+0000","last_unstale":"2024-09-16T15:39:32.167838+0000","last_undegraded":"2024-09-16T15:39:32.167838+0000","last_fullsized":"2024-09-16T15:39:32.167838+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:39.190074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151859+0000","last_change":"2024-09-16T15:39:32.151859+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151859+0000","last_undegraded":"2024-09-16T15:39:32.151859+0000","last_fullsized":"2024-09-16T15:39:32.151859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.173010+0000","last_change":"2024-09-16T15:39:32.170881+0000","last_active":"2024-09-16T15:39:32.173010+0000","last_peered":"2024-09-16T15:39:32.173010+0000","last_clean":"2024-09-16T15:39:32.173010+0000","last_became_active":"2024-09-16T15:39:32.170639+0000","last_became_peered":"2024-09-16T15:39:32.170639+0000","last_unstale":"2024-09-16T15:39:32.173010+0000","last_undegraded":"2024-09-16T15:39:32.173010+0000","last_fullsized":"2024-09-16T15:39:32.173010+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:03.240823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165671+0000","last_change":"2024-09-16T15:39:33.174871+0000","last_active":"2024-09-16T15:39:34.165671+0000","last_peered":"2024-09-16T15:39:34.165671+0000","last_clean":"2024-09-16T15:39:34.165671+0000","last_became_active":"2024-09-16T15:39:33.171604+0000","last_became_peered":"2024-09-16T15:39:33.171604+0000","last_unstale":"2024-09-16T15:39:34.165671+0000","last_undegraded":"2024-09-16T15:39:34.165671+0000","last_fullsized":"2024-09-16T15:39:34.165671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:26.792470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172991+0000","last_change":"2024-09-16T15:39:32.171713+0000","last_active":"2024-09-16T15:39:32.172991+0000","last_peered":"2024-09-16T15:39:32.172991+0000","last_clean":"2024-09-16T15:39:32.172991+0000","last_became_active":"2024-09-16T15:39:32.171452+0000","last_became_peered":"2024-09-16T15:39:32.171452+0000","last_unstale":"2024-09-16T15:39:32.172991+0000","last_undegraded":"2024-09-16T15:39:32.172991+0000","last_fullsized":"2024-09-16T15:39:32.172991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:45.685224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165563+0000","last_change":"2024-09-16T15:39:32.165563+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165563+0000","last_undegraded":"2024-09-16T15:39:32.165563+0000","last_fullsized":"2024-09-16T15:39:32.165563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166016+0000","last_change":"2024-09-16T15:39:32.162895+0000","last_active":"2024-09-16T15:39:32.166016+0000","last_peered":"2024-09-16T15:39:32.166016+0000","last_clean":"2024-09-16T15:39:32.166016+0000","last_became_active":"2024-09-16T15:39:32.162267+0000","last_became_peered":"2024-09-16T15:39:32.162267+0000","last_unstale":"2024-09-16T15:39:32.166016+0000","last_undegraded":"2024-09-16T15:39:32.166016+0000","last_fullsized":"2024-09-16T15:39:32.166016+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:35.359481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155925+0000","last_change":"2024-09-16T15:39:32.155925+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155925+0000","last_undegraded":"2024-09-16T15:39:32.155925+0000","last_fullsized":"2024-09-16T15:39:32.155925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166200+0000","last_change":"2024-09-16T15:39:32.163428+0000","last_active":"2024-09-16T15:39:32.166200+0000","last_peered":"2024-09-16T15:39:32.166200+0000","last_clean":"2024-09-16T15:39:32.166200+0000","last_became_active":"2024-09-16T15:39:32.163148+0000","last_became_peered":"2024-09-16T15:39:32.163148+0000","last_unstale":"2024-09-16T15:39:32.166200+0000","last_undegraded":"2024-09-16T15:39:32.166200+0000","last_fullsized":"2024-09-16T15:39:32.166200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:31.750012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150903+0000","last_change":"2024-09-16T15:39:32.150903+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150903+0000","last_undegraded":"2024-09-16T15:39:32.150903+0000","last_fullsized":"2024-09-16T15:39:32.150903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171837+0000","last_change":"2024-09-16T15:39:32.164943+0000","last_active":"2024-09-16T15:39:32.171837+0000","last_peered":"2024-09-16T15:39:32.171837+0000","last_clean":"2024-09-16T15:39:32.171837+0000","last_became_active":"2024-09-16T15:39:32.164576+0000","last_became_peered":"2024-09-16T15:39:32.164576+0000","last_unstale":"2024-09-16T15:39:32.171837+0000","last_undegraded":"2024-09-16T15:39:32.171837+0000","last_fullsized":"2024-09-16T15:39:32.171837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:36.312022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164735+0000","last_change":"2024-09-16T15:39:33.170443+0000","last_active":"2024-09-16T15:39:34.164735+0000","last_peered":"2024-09-16T15:39:34.164735+0000","last_clean":"2024-09-16T15:39:34.164735+0000","last_became_active":"2024-09-16T15:39:33.170165+0000","last_became_peered":"2024-09-16T15:39:33.170165+0000","last_unstale":"2024-09-16T15:39:34.164735+0000","last_undegraded":"2024-09-16T15:39:34.164735+0000","last_fullsized":"2024-09-16T15:39:34.164735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.857670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165602+0000","last_change":"2024-09-16T15:39:32.160722+0000","last_active":"2024-09-16T15:39:32.165602+0000","last_peered":"2024-09-16T15:39:32.165602+0000","last_clean":"2024-09-16T15:39:32.165602+0000","last_became_active":"2024-09-16T15:39:32.160298+0000","last_became_peered":"2024-09-16T15:39:32.160298+0000","last_unstale":"2024-09-16T15:39:32.165602+0000","last_undegraded":"2024-09-16T15:39:32.165602+0000","last_fullsized":"2024-09-16T15:39:32.165602+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:32.567093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156442+0000","last_change":"2024-09-16T15:39:32.156442+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156442+0000","last_undegraded":"2024-09-16T15:39:32.156442+0000","last_fullsized":"2024-09-16T15:39:32.156442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172968+0000","last_change":"2024-09-16T15:39:32.170156+0000","last_active":"2024-09-16T15:39:32.172968+0000","last_peered":"2024-09-16T15:39:32.172968+0000","last_clean":"2024-09-16T15:39:32.172968+0000","last_became_active":"2024-09-16T15:39:32.169818+0000","last_became_peered":"2024-09-16T15:39:32.169818+0000","last_unstale":"2024-09-16T15:39:32.172968+0000","last_undegraded":"2024-09-16T15:39:32.172968+0000","last_fullsized":"2024-09-16T15:39:32.172968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:42.329583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153208+0000","last_change":"2024-09-16T15:39:32.153208+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153208+0000","last_undegraded":"2024-09-16T15:39:32.153208+0000","last_fullsized":"2024-09-16T15:39:32.153208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165089+0000","last_change":"2024-09-16T15:39:32.161560+0000","last_active":"2024-09-16T15:39:32.165089+0000","last_peered":"2024-09-16T15:39:32.165089+0000","last_clean":"2024-09-16T15:39:32.165089+0000","last_became_active":"2024-09-16T15:39:32.161287+0000","last_became_peered":"2024-09-16T15:39:32.161287+0000","last_unstale":"2024-09-16T15:39:32.165089+0000","last_undegraded":"2024-09-16T15:39:32.165089+0000","last_fullsized":"2024-09-16T15:39:32.165089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:33.389071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154532+0000","last_change":"2024-09-16T15:39:32.154532+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154532+0000","last_undegraded":"2024-09-16T15:39:32.154532+0000","last_fullsized":"2024-09-16T15:39:32.154532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.175093+0000","last_change":"2024-09-16T15:39:32.170843+0000","last_active":"2024-09-16T15:39:32.175093+0000","last_peered":"2024-09-16T15:39:32.175093+0000","last_clean":"2024-09-16T15:39:32.175093+0000","last_became_active":"2024-09-16T15:39:32.170538+0000","last_became_peered":"2024-09-16T15:39:32.170538+0000","last_unstale":"2024-09-16T15:39:32.175093+0000","last_undegraded":"2024-09-16T15:39:32.175093+0000","last_fullsized":"2024-09-16T15:39:32.175093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:06.203396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.157250+0000","last_change":"2024-09-16T15:39:32.157250+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.157250+0000","last_undegraded":"2024-09-16T15:39:32.157250+0000","last_fullsized":"2024-09-16T15:39:32.157250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171781+0000","last_change":"2024-09-16T15:39:32.165779+0000","last_active":"2024-09-16T15:39:32.171781+0000","last_peered":"2024-09-16T15:39:32.171781+0000","last_clean":"2024-09-16T15:39:32.171781+0000","last_became_active":"2024-09-16T15:39:32.165510+0000","last_became_peered":"2024-09-16T15:39:32.165510+0000","last_unstale":"2024-09-16T15:39:32.171781+0000","last_undegraded":"2024-09-16T15:39:32.171781+0000","last_fullsized":"2024-09-16T15:39:32.171781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:08.898328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164871+0000","last_change":"2024-09-16T15:39:33.169134+0000","last_active":"2024-09-16T15:39:34.164871+0000","last_peered":"2024-09-16T15:39:34.164871+0000","last_clean":"2024-09-16T15:39:34.164871+0000","last_became_active":"2024-09-16T15:39:33.167382+0000","last_became_peered":"2024-09-16T15:39:33.167382+0000","last_unstale":"2024-09-16T15:39:34.164871+0000","last_undegraded":"2024-09-16T15:39:34.164871+0000","last_fullsized":"2024-09-16T15:39:34.164871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:07:01.249223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166107+0000","last_change":"2024-09-16T15:39:32.163172+0000","last_active":"2024-09-16T15:39:32.166107+0000","last_peered":"2024-09-16T15:39:32.166107+0000","last_clean":"2024-09-16T15:39:32.166107+0000","last_became_active":"2024-09-16T15:39:32.162626+0000","last_became_peered":"2024-09-16T15:39:32.162626+0000","last_unstale":"2024-09-16T15:39:32.166107+0000","last_undegraded":"2024-09-16T15:39:32.166107+0000","last_fullsized":"2024-09-16T15:39:32.166107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:49.229870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154053+0000","last_change":"2024-09-16T15:39:32.154053+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154053+0000","last_undegraded":"2024-09-16T15:39:32.154053+0000","last_fullsized":"2024-09-16T15:39:32.154053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164762+0000","last_change":"2024-09-16T15:39:32.162290+0000","last_active":"2024-09-16T15:39:34.164762+0000","last_peered":"2024-09-16T15:39:34.164762+0000","last_clean":"2024-09-16T15:39:34.164762+0000","last_became_active":"2024-09-16T15:39:32.162042+0000","last_became_peered":"2024-09-16T15:39:32.162042+0000","last_unstale":"2024-09-16T15:39:34.164762+0000","last_undegraded":"2024-09-16T15:39:34.164762+0000","last_fullsized":"2024-09-16T15:39:34.164762+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:23.717646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.163921+0000","last_change":"2024-09-16T15:39:32.163921+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.163921+0000","last_undegraded":"2024-09-16T15:39:32.163921+0000","last_fullsized":"2024-09-16T15:39:32.163921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171868+0000","last_change":"2024-09-16T15:39:32.169367+0000","last_active":"2024-09-16T15:39:32.171868+0000","last_peered":"2024-09-16T15:39:32.171868+0000","last_clean":"2024-09-16T15:39:32.171868+0000","last_became_active":"2024-09-16T15:39:32.169188+0000","last_became_peered":"2024-09-16T15:39:32.169188+0000","last_unstale":"2024-09-16T15:39:32.171868+0000","last_undegraded":"2024-09-16T15:39:32.171868+0000","last_fullsized":"2024-09-16T15:39:32.171868+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:14.156509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151486+0000","last_change":"2024-09-16T15:39:32.151486+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151486+0000","last_undegraded":"2024-09-16T15:39:32.151486+0000","last_fullsized":"2024-09-16T15:39:32.151486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.168920+0000","last_change":"2024-09-16T15:39:32.163551+0000","last_active":"2024-09-16T15:39:32.168920+0000","last_peered":"2024-09-16T15:39:32.168920+0000","last_clean":"2024-09-16T15:39:32.168920+0000","last_became_active":"2024-09-16T15:39:32.163283+0000","last_became_peered":"2024-09-16T15:39:32.163283+0000","last_unstale":"2024-09-16T15:39:32.168920+0000","last_undegraded":"2024-09-16T15:39:32.168920+0000","last_fullsized":"2024-09-16T15:39:32.168920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:54.716352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151031+0000","last_change":"2024-09-16T15:39:32.151031+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151031+0000","last_undegraded":"2024-09-16T15:39:32.151031+0000","last_fullsized":"2024-09-16T15:39:32.151031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165577+0000","last_change":"2024-09-16T15:39:32.161394+0000","last_active":"2024-09-16T15:39:34.165577+0000","last_peered":"2024-09-16T15:39:34.165577+0000","last_clean":"2024-09-16T15:39:34.165577+0000","last_became_active":"2024-09-16T15:39:32.161124+0000","last_became_peered":"2024-09-16T15:39:32.161124+0000","last_unstale":"2024-09-16T15:39:34.165577+0000","last_undegraded":"2024-09-16T15:39:34.165577+0000","last_fullsized":"2024-09-16T15:39:34.165577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:40.912077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.157317+0000","last_change":"2024-09-16T15:39:32.157317+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.157317+0000","last_undegraded":"2024-09-16T15:39:32.157317+0000","last_fullsized":"2024-09-16T15:39:32.157317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166121+0000","last_change":"2024-09-16T15:39:32.163386+0000","last_active":"2024-09-16T15:39:32.166121+0000","last_peered":"2024-09-16T15:39:32.166121+0000","last_clean":"2024-09-16T15:39:32.166121+0000","last_became_active":"2024-09-16T15:39:32.161214+0000","last_became_peered":"2024-09-16T15:39:32.161214+0000","last_unstale":"2024-09-16T15:39:32.166121+0000","last_undegraded":"2024-09-16T15:39:32.166121+0000","last_fullsized":"2024-09-16T15:39:32.166121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:34.093301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165551+0000","last_change":"2024-09-16T15:39:33.175372+0000","last_active":"2024-09-16T15:39:34.165551+0000","last_peered":"2024-09-16T15:39:34.165551+0000","last_clean":"2024-09-16T15:39:34.165551+0000","last_became_active":"2024-09-16T15:39:33.174198+0000","last_became_peered":"2024-09-16T15:39:33.174198+0000","last_unstale":"2024-09-16T15:39:34.165551+0000","last_undegraded":"2024-09-16T15:39:34.165551+0000","last_fullsized":"2024-09-16T15:39:34.165551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:53.410594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.169802+0000","last_change":"2024-09-16T15:39:32.166140+0000","last_active":"2024-09-16T15:39:32.169802+0000","last_peered":"2024-09-16T15:39:32.169802+0000","last_clean":"2024-09-16T15:39:32.169802+0000","last_became_active":"2024-09-16T15:39:32.165774+0000","last_became_peered":"2024-09-16T15:39:32.165774+0000","last_unstale":"2024-09-16T15:39:32.169802+0000","last_undegraded":"2024-09-16T15:39:32.169802+0000","last_fullsized":"2024-09-16T15:39:32.169802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:48.317883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151377+0000","last_change":"2024-09-16T15:39:32.151377+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151377+0000","last_undegraded":"2024-09-16T15:39:32.151377+0000","last_fullsized":"2024-09-16T15:39:32.151377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165944+0000","last_change":"2024-09-16T15:39:32.162492+0000","last_active":"2024-09-16T15:39:32.165944+0000","last_peered":"2024-09-16T15:39:32.165944+0000","last_clean":"2024-09-16T15:39:32.165944+0000","last_became_active":"2024-09-16T15:39:32.162229+0000","last_became_peered":"2024-09-16T15:39:32.162229+0000","last_unstale":"2024-09-16T15:39:32.165944+0000","last_undegraded":"2024-09-16T15:39:32.165944+0000","last_fullsized":"2024-09-16T15:39:32.165944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:19.223491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150686+0000","last_change":"2024-09-16T15:39:32.150686+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150686+0000","last_undegraded":"2024-09-16T15:39:32.150686+0000","last_fullsized":"2024-09-16T15:39:32.150686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.164987+0000","last_change":"2024-09-16T15:39:32.161540+0000","last_active":"2024-09-16T15:39:32.164987+0000","last_peered":"2024-09-16T15:39:32.164987+0000","last_clean":"2024-09-16T15:39:32.164987+0000","last_became_active":"2024-09-16T15:39:32.161225+0000","last_became_peered":"2024-09-16T15:39:32.161225+0000","last_unstale":"2024-09-16T15:39:32.164987+0000","last_undegraded":"2024-09-16T15:39:32.164987+0000","last_fullsized":"2024-09-16T15:39:32.164987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:21:04.290055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151288+0000","last_change":"2024-09-16T15:39:32.151288+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151288+0000","last_undegraded":"2024-09-16T15:39:32.151288+0000","last_fullsized":"2024-09-16T15:39:32.151288+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.160197+0000","last_change":"2024-09-16T15:39:32.157654+0000","last_active":"2024-09-16T15:39:32.160197+0000","last_peered":"2024-09-16T15:39:32.160197+0000","last_clean":"2024-09-16T15:39:32.160197+0000","last_became_active":"2024-09-16T15:39:32.157393+0000","last_became_peered":"2024-09-16T15:39:32.157393+0000","last_unstale":"2024-09-16T15:39:32.160197+0000","last_undegraded":"2024-09-16T15:39:32.160197+0000","last_fullsized":"2024-09-16T15:39:32.160197+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:07.931223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154817+0000","last_change":"2024-09-16T15:39:32.154817+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154817+0000","last_undegraded":"2024-09-16T15:39:32.154817+0000","last_fullsized":"2024-09-16T15:39:32.154817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166535+0000","last_change":"2024-09-16T15:39:32.162100+0000","last_active":"2024-09-16T15:39:32.166535+0000","last_peered":"2024-09-16T15:39:32.166535+0000","last_clean":"2024-09-16T15:39:32.166535+0000","last_became_active":"2024-09-16T15:39:32.161782+0000","last_became_peered":"2024-09-16T15:39:32.161782+0000","last_unstale":"2024-09-16T15:39:32.166535+0000","last_undegraded":"2024-09-16T15:39:32.166535+0000","last_fullsized":"2024-09-16T15:39:32.166535+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:51:50.007487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.161447+0000","last_change":"2024-09-16T15:39:32.161447+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.161447+0000","last_undegraded":"2024-09-16T15:39:32.161447+0000","last_fullsized":"2024-09-16T15:39:32.161447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172551+0000","last_change":"2024-09-16T15:39:32.168491+0000","last_active":"2024-09-16T15:39:32.172551+0000","last_peered":"2024-09-16T15:39:32.172551+0000","last_clean":"2024-09-16T15:39:32.172551+0000","last_became_active":"2024-09-16T15:39:32.168195+0000","last_became_peered":"2024-09-16T15:39:32.168195+0000","last_unstale":"2024-09-16T15:39:32.172551+0000","last_undegraded":"2024-09-16T15:39:32.172551+0000","last_fullsized":"2024-09-16T15:39:32.172551+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:00.428705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150369+0000","last_change":"2024-09-16T15:39:32.150369+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150369+0000","last_undegraded":"2024-09-16T15:39:32.150369+0000","last_fullsized":"2024-09-16T15:39:32.150369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165670+0000","last_change":"2024-09-16T15:39:32.166439+0000","last_active":"2024-09-16T15:39:34.165670+0000","last_peered":"2024-09-16T15:39:34.165670+0000","last_clean":"2024-09-16T15:39:34.165670+0000","last_became_active":"2024-09-16T15:39:32.166175+0000","last_became_peered":"2024-09-16T15:39:32.166175+0000","last_unstale":"2024-09-16T15:39:34.165670+0000","last_undegraded":"2024-09-16T15:39:34.165670+0000","last_fullsized":"2024-09-16T15:39:34.165670+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:39.246388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156915+0000","last_change":"2024-09-16T15:39:32.156915+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156915+0000","last_undegraded":"2024-09-16T15:39:32.156915+0000","last_fullsized":"2024-09-16T15:39:32.156915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.158697+0000","last_change":"2024-09-16T15:39:32.155559+0000","last_active":"2024-09-16T15:39:32.158697+0000","last_peered":"2024-09-16T15:39:32.158697+0000","last_clean":"2024-09-16T15:39:32.158697+0000","last_became_active":"2024-09-16T15:39:32.155286+0000","last_became_peered":"2024-09-16T15:39:32.155286+0000","last_unstale":"2024-09-16T15:39:32.158697+0000","last_undegraded":"2024-09-16T15:39:32.158697+0000","last_fullsized":"2024-09-16T15:39:32.158697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:53:37.151552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150242+0000","last_change":"2024-09-16T15:39:32.150242+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150242+0000","last_undegraded":"2024-09-16T15:39:32.150242+0000","last_fullsized":"2024-09-16T15:39:32.150242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165430+0000","last_change":"2024-09-16T15:39:32.172865+0000","last_active":"2024-09-16T15:39:34.165430+0000","last_peered":"2024-09-16T15:39:34.165430+0000","last_clean":"2024-09-16T15:39:34.165430+0000","last_became_active":"2024-09-16T15:39:32.172616+0000","last_became_peered":"2024-09-16T15:39:32.172616+0000","last_unstale":"2024-09-16T15:39:34.165430+0000","last_undegraded":"2024-09-16T15:39:34.165430+0000","last_fullsized":"2024-09-16T15:39:34.165430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:38.115059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151198+0000","last_change":"2024-09-16T15:39:32.151198+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151198+0000","last_undegraded":"2024-09-16T15:39:32.151198+0000","last_fullsized":"2024-09-16T15:39:32.151198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165849+0000","last_change":"2024-09-16T15:39:32.164165+0000","last_active":"2024-09-16T15:39:32.165849+0000","last_peered":"2024-09-16T15:39:32.165849+0000","last_clean":"2024-09-16T15:39:32.165849+0000","last_became_active":"2024-09-16T15:39:32.163728+0000","last_became_peered":"2024-09-16T15:39:32.163728+0000","last_unstale":"2024-09-16T15:39:32.165849+0000","last_undegraded":"2024-09-16T15:39:32.165849+0000","last_fullsized":"2024-09-16T15:39:32.165849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:07.235894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150024+0000","last_change":"2024-09-16T15:39:32.150024+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150024+0000","last_undegraded":"2024-09-16T15:39:32.150024+0000","last_fullsized":"2024-09-16T15:39:32.150024+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165133+0000","last_change":"2024-09-16T15:39:32.167828+0000","last_active":"2024-09-16T15:39:34.165133+0000","last_peered":"2024-09-16T15:39:34.165133+0000","last_clean":"2024-09-16T15:39:34.165133+0000","last_became_active":"2024-09-16T15:39:32.165651+0000","last_became_peered":"2024-09-16T15:39:32.165651+0000","last_unstale":"2024-09-16T15:39:34.165133+0000","last_undegraded":"2024-09-16T15:39:34.165133+0000","last_fullsized":"2024-09-16T15:39:34.165133+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:29.486963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152940+0000","last_change":"2024-09-16T15:39:32.152940+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152940+0000","last_undegraded":"2024-09-16T15:39:32.152940+0000","last_fullsized":"2024-09-16T15:39:32.152940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172404+0000","last_change":"2024-09-16T15:39:32.169431+0000","last_active":"2024-09-16T15:39:32.172404+0000","last_peered":"2024-09-16T15:39:32.172404+0000","last_clean":"2024-09-16T15:39:32.172404+0000","last_became_active":"2024-09-16T15:39:32.169166+0000","last_became_peered":"2024-09-16T15:39:32.169166+0000","last_unstale":"2024-09-16T15:39:32.172404+0000","last_undegraded":"2024-09-16T15:39:32.172404+0000","last_fullsized":"2024-09-16T15:39:32.172404+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:12.103417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153357+0000","last_change":"2024-09-16T15:39:32.153357+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153357+0000","last_undegraded":"2024-09-16T15:39:32.153357+0000","last_fullsized":"2024-09-16T15:39:32.153357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171856+0000","last_change":"2024-09-16T15:39:32.170091+0000","last_active":"2024-09-16T15:39:32.171856+0000","last_peered":"2024-09-16T15:39:32.171856+0000","last_clean":"2024-09-16T15:39:32.171856+0000","last_became_active":"2024-09-16T15:39:32.169831+0000","last_became_peered":"2024-09-16T15:39:32.169831+0000","last_unstale":"2024-09-16T15:39:32.171856+0000","last_undegraded":"2024-09-16T15:39:32.171856+0000","last_fullsized":"2024-09-16T15:39:32.171856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:18.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159832+0000","last_change":"2024-09-16T15:39:32.159832+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159832+0000","last_undegraded":"2024-09-16T15:39:32.159832+0000","last_fullsized":"2024-09-16T15:39:32.159832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167962+0000","last_change":"2024-09-16T15:39:32.163553+0000","last_active":"2024-09-16T15:39:32.167962+0000","last_peered":"2024-09-16T15:39:32.167962+0000","last_clean":"2024-09-16T15:39:32.167962+0000","last_became_active":"2024-09-16T15:39:32.162649+0000","last_became_peered":"2024-09-16T15:39:32.162649+0000","last_unstale":"2024-09-16T15:39:32.167962+0000","last_undegraded":"2024-09-16T15:39:32.167962+0000","last_fullsized":"2024-09-16T15:39:32.167962+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:12.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":131,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167970+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:39:32.167970+0000","last_peered":"2024-09-16T15:39:32.167970+0000","last_clean":"2024-09-16T15:39:32.167970+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:39:32.167970+0000","last_undegraded":"2024-09-16T15:39:32.167970+0000","last_fullsized":"2024-09-16T15:39:32.167970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":734366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":970752,"data_stored":734366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":417792,"data_stored":158707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:37.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.033+0000 7f1995c00640 1 -- 172.21.15.60:0/619289828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1974004d40 msgr2=0x7f19740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.033+0000 7f1995c00640 1 --2- 172.21.15.60:0/619289828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1974004d40 0x7f19740051c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1988009a90 tx=0x7f1974005830 comp rx=0 tx=0).stop 2024-09-16T15:39:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.033+0000 7f198f400640 1 -- 172.21.15.60:0/619289828 reap_dead start 2024-09-16T15:39:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.034+0000 7f1995c00640 1 -- 172.21.15.60:0/619289828 shutdown_connections 2024-09-16T15:39:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.034+0000 7f1995c00640 1 -- 172.21.15.60:0/619289828 >> 172.21.15.60:0/619289828 conn(0x7f19900b1950 msgr2=0x7f19900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.034+0000 7f1995c00640 1 -- 172.21.15.60:0/619289828 shutdown_connections 2024-09-16T15:39:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.034+0000 7f1995c00640 1 -- 172.21.15.60:0/619289828 wait complete. 2024-09-16T15:39:38.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.035+0000 7f1995c00640 1 Processor -- start 2024-09-16T15:39:38.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.035+0000 7f1995c00640 1 -- start start 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.036+0000 7f1995c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1974004d40 0x7f199015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.036+0000 7f1995c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19901f5d20 0x7f19900cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.036+0000 7f1995c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19901f61a0 0x7f19900ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.036+0000 7f1995c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f199015f080 con 0x7f19901f5d20 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.036+0000 7f1995c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1990073bc0 con 0x7f1974004d40 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.036+0000 7f1995c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f199015f200 con 0x7f19901f61a0 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19901f5d20 0x7f19900cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19901f5d20 0x7f19900cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42118/0 (socket says 172.21.15.60:42118) 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 -- 172.21.15.60:0/419140545 learned_addr learned my addr 172.21.15.60:0/419140545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f198f400640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19901f61a0 0x7f19900ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f198ea00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1974004d40 0x7f199015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 -- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19901f61a0 msgr2=0x7f19900ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19901f61a0 0x7f19900ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 -- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1974004d40 msgr2=0x7f199015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1974004d40 0x7f199015d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 -- 172.21.15.60:0/419140545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1990144fc0 con 0x7f19901f5d20 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f198f400640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19901f61a0 0x7f19900ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f198ea00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1974004d40 0x7f199015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1986000640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19901f5d20 0x7f19900cdc70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f19780066b0 tx=0x7f19780039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.037+0000 7f1987400640 1 -- 172.21.15.60:0/419140545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978015070 con 0x7f19901f5d20 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.038+0000 7f1987400640 1 -- 172.21.15.60:0/419140545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1978011900 con 0x7f19901f5d20 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.038+0000 7f1987400640 1 -- 172.21.15.60:0/419140545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978011d50 con 0x7f19901f5d20 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.038+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19901f6690 con 0x7f19901f5d20 2024-09-16T15:39:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.038+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19901f6c50 con 0x7f19901f5d20 2024-09-16T15:39:38.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.039+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19901f6e50 con 0x7f19901f5d20 2024-09-16T15:39:38.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.040+0000 7f1987400640 1 -- 172.21.15.60:0/419140545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1978010420 con 0x7f19901f5d20 2024-09-16T15:39:38.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.041+0000 7f1987400640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f196c078850 0x7f196c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.041+0000 7f1987400640 1 -- 172.21.15.60:0/419140545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f197809b670 con 0x7f19901f5d20 2024-09-16T15:39:38.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.041+0000 7f198ea00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f196c078850 0x7f196c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.042+0000 7f198ea00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f196c078850 0x7f196c07ad10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1980009b40 tx=0x7f1980005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.045+0000 7f1987400640 1 -- 172.21.15.60:0/419140545 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f19901f6e50 con 0x7f19901f5d20 2024-09-16T15:39:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.240+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f199015e970 con 0x7f196c078850 2024-09-16T15:39:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.245+0000 7f1987400640 1 -- 172.21.15.60:0/419140545 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247320 (secure 0 0 0) 0x7f199015e970 con 0x7f196c078850 2024-09-16T15:39:38.249 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f196c078850 msgr2=0x7f196c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f196c078850 0x7f196c07ad10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1980009b40 tx=0x7f1980005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19901f5d20 msgr2=0x7f19900cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19901f5d20 0x7f19900cdc70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f19780066b0 tx=0x7f19780039c0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 shutdown_connections 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f196c078850 0x7f196c07ad10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19901f61a0 0x7f19900ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19901f5d20 0x7f19900cdc70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 --2- 172.21.15.60:0/419140545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1974004d40 0x7f199015d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 >> 172.21.15.60:0/419140545 conn(0x7f19900b1950 msgr2=0x7f19900aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 shutdown_connections 2024-09-16T15:39:38.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.249+0000 7f1995c00640 1 -- 172.21.15.60:0/419140545 wait complete. 2024-09-16T15:39:38.266 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-16T15:39:37.723221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237024,"kb_used_data":4132,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737696,"statfs":{"total":773094113280,"available":772851400704,"internally_reserved":0,"allocated":4231168,"data_stored":2341162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154188+0000","last_change":"2024-09-16T15:39:32.154188+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154188+0000","last_undegraded":"2024-09-16T15:39:32.154188+0000","last_fullsized":"2024-09-16T15:39:32.154188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159798+0000","last_change":"2024-09-16T15:39:32.159798+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159798+0000","last_undegraded":"2024-09-16T15:39:32.159798+0000","last_fullsized":"2024-09-16T15:39:32.159798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156813+0000","last_change":"2024-09-16T15:39:32.156813+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156813+0000","last_undegraded":"2024-09-16T15:39:32.156813+0000","last_fullsized":"2024-09-16T15:39:32.156813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165367+0000","last_change":"2024-09-16T15:39:33.175477+0000","last_active":"2024-09-16T15:39:34.165367+0000","last_peered":"2024-09-16T15:39:34.165367+0000","last_clean":"2024-09-16T15:39:34.165367+0000","last_became_active":"2024-09-16T15:39:33.174529+0000","last_became_peered":"2024-09-16T15:39:33.174529+0000","last_unstale":"2024-09-16T15:39:34.165367+0000","last_undegraded":"2024-09-16T15:39:34.165367+0000","last_fullsized":"2024-09-16T15:39:34.165367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:16.836397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156011+0000","last_change":"2024-09-16T15:39:32.156011+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156011+0000","last_undegraded":"2024-09-16T15:39:32.156011+0000","last_fullsized":"2024-09-16T15:39:32.156011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.158243+0000","last_change":"2024-09-16T15:39:32.158243+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.158243+0000","last_undegraded":"2024-09-16T15:39:32.158243+0000","last_fullsized":"2024-09-16T15:39:32.158243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159374+0000","last_change":"2024-09-16T15:39:32.159374+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159374+0000","last_undegraded":"2024-09-16T15:39:32.159374+0000","last_fullsized":"2024-09-16T15:39:32.159374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165592+0000","last_change":"2024-09-16T15:39:32.165592+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165592+0000","last_undegraded":"2024-09-16T15:39:32.165592+0000","last_fullsized":"2024-09-16T15:39:32.165592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150787+0000","last_change":"2024-09-16T15:39:32.150787+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150787+0000","last_undegraded":"2024-09-16T15:39:32.150787+0000","last_fullsized":"2024-09-16T15:39:32.150787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165700+0000","last_change":"2024-09-16T15:39:33.177863+0000","last_active":"2024-09-16T15:39:34.165700+0000","last_peered":"2024-09-16T15:39:34.165700+0000","last_clean":"2024-09-16T15:39:34.165700+0000","last_became_active":"2024-09-16T15:39:33.176375+0000","last_became_peered":"2024-09-16T15:39:33.176375+0000","last_unstale":"2024-09-16T15:39:34.165700+0000","last_undegraded":"2024-09-16T15:39:34.165700+0000","last_fullsized":"2024-09-16T15:39:34.165700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:10.080289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165508+0000","last_change":"2024-09-16T15:39:32.165508+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165508+0000","last_undegraded":"2024-09-16T15:39:32.165508+0000","last_fullsized":"2024-09-16T15:39:32.165508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156865+0000","last_change":"2024-09-16T15:39:32.156865+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156865+0000","last_undegraded":"2024-09-16T15:39:32.156865+0000","last_fullsized":"2024-09-16T15:39:32.156865+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151813+0000","last_change":"2024-09-16T15:39:32.151813+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151813+0000","last_undegraded":"2024-09-16T15:39:32.151813+0000","last_fullsized":"2024-09-16T15:39:32.151813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156333+0000","last_change":"2024-09-16T15:39:32.156333+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156333+0000","last_undegraded":"2024-09-16T15:39:32.156333+0000","last_fullsized":"2024-09-16T15:39:32.156333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.161031+0000","last_change":"2024-09-16T15:39:32.161031+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.161031+0000","last_undegraded":"2024-09-16T15:39:32.161031+0000","last_fullsized":"2024-09-16T15:39:32.161031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.158855+0000","last_change":"2024-09-16T15:39:32.158855+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.158855+0000","last_undegraded":"2024-09-16T15:39:32.158855+0000","last_fullsized":"2024-09-16T15:39:32.158855+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.164804+0000","last_change":"2024-09-16T15:39:32.164804+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.164804+0000","last_undegraded":"2024-09-16T15:39:32.164804+0000","last_fullsized":"2024-09-16T15:39:32.164804+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156100+0000","last_change":"2024-09-16T15:39:32.156100+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156100+0000","last_undegraded":"2024-09-16T15:39:32.156100+0000","last_fullsized":"2024-09-16T15:39:32.156100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153593+0000","last_change":"2024-09-16T15:39:32.153593+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153593+0000","last_undegraded":"2024-09-16T15:39:32.153593+0000","last_fullsized":"2024-09-16T15:39:32.153593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152311+0000","last_change":"2024-09-16T15:39:32.152311+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152311+0000","last_undegraded":"2024-09-16T15:39:32.152311+0000","last_fullsized":"2024-09-16T15:39:32.152311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152875+0000","last_change":"2024-09-16T15:39:32.152875+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152875+0000","last_undegraded":"2024-09-16T15:39:32.152875+0000","last_fullsized":"2024-09-16T15:39:32.152875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151657+0000","last_change":"2024-09-16T15:39:32.151657+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151657+0000","last_undegraded":"2024-09-16T15:39:32.151657+0000","last_fullsized":"2024-09-16T15:39:32.151657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151923+0000","last_change":"2024-09-16T15:39:32.151923+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151923+0000","last_undegraded":"2024-09-16T15:39:32.151923+0000","last_fullsized":"2024-09-16T15:39:32.151923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155225+0000","last_change":"2024-09-16T15:39:32.155225+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155225+0000","last_undegraded":"2024-09-16T15:39:32.155225+0000","last_fullsized":"2024-09-16T15:39:32.155225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165596+0000","last_change":"2024-09-16T15:39:33.178266+0000","last_active":"2024-09-16T15:39:34.165596+0000","last_peered":"2024-09-16T15:39:34.165596+0000","last_clean":"2024-09-16T15:39:34.165596+0000","last_became_active":"2024-09-16T15:39:33.177230+0000","last_became_peered":"2024-09-16T15:39:33.177230+0000","last_unstale":"2024-09-16T15:39:34.165596+0000","last_undegraded":"2024-09-16T15:39:34.165596+0000","last_fullsized":"2024-09-16T15:39:34.165596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:59.832311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153291+0000","last_change":"2024-09-16T15:39:32.153291+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153291+0000","last_undegraded":"2024-09-16T15:39:32.153291+0000","last_fullsized":"2024-09-16T15:39:32.153291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165652+0000","last_change":"2024-09-16T15:39:33.176217+0000","last_active":"2024-09-16T15:39:34.165652+0000","last_peered":"2024-09-16T15:39:34.165652+0000","last_clean":"2024-09-16T15:39:34.165652+0000","last_became_active":"2024-09-16T15:39:33.175855+0000","last_became_peered":"2024-09-16T15:39:33.175855+0000","last_unstale":"2024-09-16T15:39:34.165652+0000","last_undegraded":"2024-09-16T15:39:34.165652+0000","last_fullsized":"2024-09-16T15:39:34.165652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:45.876870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156476+0000","last_change":"2024-09-16T15:39:32.156476+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156476+0000","last_undegraded":"2024-09-16T15:39:32.156476+0000","last_fullsized":"2024-09-16T15:39:32.156476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152413+0000","last_change":"2024-09-16T15:39:32.152413+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152413+0000","last_undegraded":"2024-09-16T15:39:32.152413+0000","last_fullsized":"2024-09-16T15:39:32.152413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165973+0000","last_change":"2024-09-16T15:39:32.165973+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165973+0000","last_undegraded":"2024-09-16T15:39:32.165973+0000","last_fullsized":"2024-09-16T15:39:32.165973+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155795+0000","last_change":"2024-09-16T15:39:32.155795+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155795+0000","last_undegraded":"2024-09-16T15:39:32.155795+0000","last_fullsized":"2024-09-16T15:39:32.155795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165147+0000","last_change":"2024-09-16T15:39:32.165147+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165147+0000","last_undegraded":"2024-09-16T15:39:32.165147+0000","last_fullsized":"2024-09-16T15:39:32.165147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150200+0000","last_change":"2024-09-16T15:39:32.150200+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150200+0000","last_undegraded":"2024-09-16T15:39:32.150200+0000","last_fullsized":"2024-09-16T15:39:32.150200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.168107+0000","last_change":"2024-09-16T15:39:32.164389+0000","last_active":"2024-09-16T15:39:32.168107+0000","last_peered":"2024-09-16T15:39:32.168107+0000","last_clean":"2024-09-16T15:39:32.168107+0000","last_became_active":"2024-09-16T15:39:32.164146+0000","last_became_peered":"2024-09-16T15:39:32.164146+0000","last_unstale":"2024-09-16T15:39:32.168107+0000","last_undegraded":"2024-09-16T15:39:32.168107+0000","last_fullsized":"2024-09-16T15:39:32.168107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:22.804697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153708+0000","last_change":"2024-09-16T15:39:32.153708+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153708+0000","last_undegraded":"2024-09-16T15:39:32.153708+0000","last_fullsized":"2024-09-16T15:39:32.153708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167838+0000","last_change":"2024-09-16T15:39:32.166998+0000","last_active":"2024-09-16T15:39:32.167838+0000","last_peered":"2024-09-16T15:39:32.167838+0000","last_clean":"2024-09-16T15:39:32.167838+0000","last_became_active":"2024-09-16T15:39:32.166503+0000","last_became_peered":"2024-09-16T15:39:32.166503+0000","last_unstale":"2024-09-16T15:39:32.167838+0000","last_undegraded":"2024-09-16T15:39:32.167838+0000","last_fullsized":"2024-09-16T15:39:32.167838+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:39.190074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151859+0000","last_change":"2024-09-16T15:39:32.151859+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151859+0000","last_undegraded":"2024-09-16T15:39:32.151859+0000","last_fullsized":"2024-09-16T15:39:32.151859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.173010+0000","last_change":"2024-09-16T15:39:32.170881+0000","last_active":"2024-09-16T15:39:32.173010+0000","last_peered":"2024-09-16T15:39:32.173010+0000","last_clean":"2024-09-16T15:39:32.173010+0000","last_became_active":"2024-09-16T15:39:32.170639+0000","last_became_peered":"2024-09-16T15:39:32.170639+0000","last_unstale":"2024-09-16T15:39:32.173010+0000","last_undegraded":"2024-09-16T15:39:32.173010+0000","last_fullsized":"2024-09-16T15:39:32.173010+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:03.240823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165671+0000","last_change":"2024-09-16T15:39:33.174871+0000","last_active":"2024-09-16T15:39:34.165671+0000","last_peered":"2024-09-16T15:39:34.165671+0000","last_clean":"2024-09-16T15:39:34.165671+0000","last_became_active":"2024-09-16T15:39:33.171604+0000","last_became_peered":"2024-09-16T15:39:33.171604+0000","last_unstale":"2024-09-16T15:39:34.165671+0000","last_undegraded":"2024-09-16T15:39:34.165671+0000","last_fullsized":"2024-09-16T15:39:34.165671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:26.792470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172991+0000","last_change":"2024-09-16T15:39:32.171713+0000","last_active":"2024-09-16T15:39:32.172991+0000","last_peered":"2024-09-16T15:39:32.172991+0000","last_clean":"2024-09-16T15:39:32.172991+0000","last_became_active":"2024-09-16T15:39:32.171452+0000","last_became_peered":"2024-09-16T15:39:32.171452+0000","last_unstale":"2024-09-16T15:39:32.172991+0000","last_undegraded":"2024-09-16T15:39:32.172991+0000","last_fullsized":"2024-09-16T15:39:32.172991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:45.685224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165563+0000","last_change":"2024-09-16T15:39:32.165563+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165563+0000","last_undegraded":"2024-09-16T15:39:32.165563+0000","last_fullsized":"2024-09-16T15:39:32.165563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166016+0000","last_change":"2024-09-16T15:39:32.162895+0000","last_active":"2024-09-16T15:39:32.166016+0000","last_peered":"2024-09-16T15:39:32.166016+0000","last_clean":"2024-09-16T15:39:32.166016+0000","last_became_active":"2024-09-16T15:39:32.162267+0000","last_became_peered":"2024-09-16T15:39:32.162267+0000","last_unstale":"2024-09-16T15:39:32.166016+0000","last_undegraded":"2024-09-16T15:39:32.166016+0000","last_fullsized":"2024-09-16T15:39:32.166016+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:35.359481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155925+0000","last_change":"2024-09-16T15:39:32.155925+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155925+0000","last_undegraded":"2024-09-16T15:39:32.155925+0000","last_fullsized":"2024-09-16T15:39:32.155925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166200+0000","last_change":"2024-09-16T15:39:32.163428+0000","last_active":"2024-09-16T15:39:32.166200+0000","last_peered":"2024-09-16T15:39:32.166200+0000","last_clean":"2024-09-16T15:39:32.166200+0000","last_became_active":"2024-09-16T15:39:32.163148+0000","last_became_peered":"2024-09-16T15:39:32.163148+0000","last_unstale":"2024-09-16T15:39:32.166200+0000","last_undegraded":"2024-09-16T15:39:32.166200+0000","last_fullsized":"2024-09-16T15:39:32.166200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:31.750012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150903+0000","last_change":"2024-09-16T15:39:32.150903+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150903+0000","last_undegraded":"2024-09-16T15:39:32.150903+0000","last_fullsized":"2024-09-16T15:39:32.150903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171837+0000","last_change":"2024-09-16T15:39:32.164943+0000","last_active":"2024-09-16T15:39:32.171837+0000","last_peered":"2024-09-16T15:39:32.171837+0000","last_clean":"2024-09-16T15:39:32.171837+0000","last_became_active":"2024-09-16T15:39:32.164576+0000","last_became_peered":"2024-09-16T15:39:32.164576+0000","last_unstale":"2024-09-16T15:39:32.171837+0000","last_undegraded":"2024-09-16T15:39:32.171837+0000","last_fullsized":"2024-09-16T15:39:32.171837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:36.312022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164735+0000","last_change":"2024-09-16T15:39:33.170443+0000","last_active":"2024-09-16T15:39:34.164735+0000","last_peered":"2024-09-16T15:39:34.164735+0000","last_clean":"2024-09-16T15:39:34.164735+0000","last_became_active":"2024-09-16T15:39:33.170165+0000","last_became_peered":"2024-09-16T15:39:33.170165+0000","last_unstale":"2024-09-16T15:39:34.164735+0000","last_undegraded":"2024-09-16T15:39:34.164735+0000","last_fullsized":"2024-09-16T15:39:34.164735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.857670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165602+0000","last_change":"2024-09-16T15:39:32.160722+0000","last_active":"2024-09-16T15:39:32.165602+0000","last_peered":"2024-09-16T15:39:32.165602+0000","last_clean":"2024-09-16T15:39:32.165602+0000","last_became_active":"2024-09-16T15:39:32.160298+0000","last_became_peered":"2024-09-16T15:39:32.160298+0000","last_unstale":"2024-09-16T15:39:32.165602+0000","last_undegraded":"2024-09-16T15:39:32.165602+0000","last_fullsized":"2024-09-16T15:39:32.165602+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:32.567093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156442+0000","last_change":"2024-09-16T15:39:32.156442+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156442+0000","last_undegraded":"2024-09-16T15:39:32.156442+0000","last_fullsized":"2024-09-16T15:39:32.156442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172968+0000","last_change":"2024-09-16T15:39:32.170156+0000","last_active":"2024-09-16T15:39:32.172968+0000","last_peered":"2024-09-16T15:39:32.172968+0000","last_clean":"2024-09-16T15:39:32.172968+0000","last_became_active":"2024-09-16T15:39:32.169818+0000","last_became_peered":"2024-09-16T15:39:32.169818+0000","last_unstale":"2024-09-16T15:39:32.172968+0000","last_undegraded":"2024-09-16T15:39:32.172968+0000","last_fullsized":"2024-09-16T15:39:32.172968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:42.329583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153208+0000","last_change":"2024-09-16T15:39:32.153208+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153208+0000","last_undegraded":"2024-09-16T15:39:32.153208+0000","last_fullsized":"2024-09-16T15:39:32.153208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165089+0000","last_change":"2024-09-16T15:39:32.161560+0000","last_active":"2024-09-16T15:39:32.165089+0000","last_peered":"2024-09-16T15:39:32.165089+0000","last_clean":"2024-09-16T15:39:32.165089+0000","last_became_active":"2024-09-16T15:39:32.161287+0000","last_became_peered":"2024-09-16T15:39:32.161287+0000","last_unstale":"2024-09-16T15:39:32.165089+0000","last_undegraded":"2024-09-16T15:39:32.165089+0000","last_fullsized":"2024-09-16T15:39:32.165089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:33.389071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154532+0000","last_change":"2024-09-16T15:39:32.154532+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154532+0000","last_undegraded":"2024-09-16T15:39:32.154532+0000","last_fullsized":"2024-09-16T15:39:32.154532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.175093+0000","last_change":"2024-09-16T15:39:32.170843+0000","last_active":"2024-09-16T15:39:32.175093+0000","last_peered":"2024-09-16T15:39:32.175093+0000","last_clean":"2024-09-16T15:39:32.175093+0000","last_became_active":"2024-09-16T15:39:32.170538+0000","last_became_peered":"2024-09-16T15:39:32.170538+0000","last_unstale":"2024-09-16T15:39:32.175093+0000","last_undegraded":"2024-09-16T15:39:32.175093+0000","last_fullsized":"2024-09-16T15:39:32.175093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:06.203396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.157250+0000","last_change":"2024-09-16T15:39:32.157250+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.157250+0000","last_undegraded":"2024-09-16T15:39:32.157250+0000","last_fullsized":"2024-09-16T15:39:32.157250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171781+0000","last_change":"2024-09-16T15:39:32.165779+0000","last_active":"2024-09-16T15:39:32.171781+0000","last_peered":"2024-09-16T15:39:32.171781+0000","last_clean":"2024-09-16T15:39:32.171781+0000","last_became_active":"2024-09-16T15:39:32.165510+0000","last_became_peered":"2024-09-16T15:39:32.165510+0000","last_unstale":"2024-09-16T15:39:32.171781+0000","last_undegraded":"2024-09-16T15:39:32.171781+0000","last_fullsized":"2024-09-16T15:39:32.171781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:08.898328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164871+0000","last_change":"2024-09-16T15:39:33.169134+0000","last_active":"2024-09-16T15:39:34.164871+0000","last_peered":"2024-09-16T15:39:34.164871+0000","last_clean":"2024-09-16T15:39:34.164871+0000","last_became_active":"2024-09-16T15:39:33.167382+0000","last_became_peered":"2024-09-16T15:39:33.167382+0000","last_unstale":"2024-09-16T15:39:34.164871+0000","last_undegraded":"2024-09-16T15:39:34.164871+0000","last_fullsized":"2024-09-16T15:39:34.164871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:07:01.249223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166107+0000","last_change":"2024-09-16T15:39:32.163172+0000","last_active":"2024-09-16T15:39:32.166107+0000","last_peered":"2024-09-16T15:39:32.166107+0000","last_clean":"2024-09-16T15:39:32.166107+0000","last_became_active":"2024-09-16T15:39:32.162626+0000","last_became_peered":"2024-09-16T15:39:32.162626+0000","last_unstale":"2024-09-16T15:39:32.166107+0000","last_undegraded":"2024-09-16T15:39:32.166107+0000","last_fullsized":"2024-09-16T15:39:32.166107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:49.229870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154053+0000","last_change":"2024-09-16T15:39:32.154053+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154053+0000","last_undegraded":"2024-09-16T15:39:32.154053+0000","last_fullsized":"2024-09-16T15:39:32.154053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164762+0000","last_change":"2024-09-16T15:39:32.162290+0000","last_active":"2024-09-16T15:39:34.164762+0000","last_peered":"2024-09-16T15:39:34.164762+0000","last_clean":"2024-09-16T15:39:34.164762+0000","last_became_active":"2024-09-16T15:39:32.162042+0000","last_became_peered":"2024-09-16T15:39:32.162042+0000","last_unstale":"2024-09-16T15:39:34.164762+0000","last_undegraded":"2024-09-16T15:39:34.164762+0000","last_fullsized":"2024-09-16T15:39:34.164762+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:23.717646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.163921+0000","last_change":"2024-09-16T15:39:32.163921+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.163921+0000","last_undegraded":"2024-09-16T15:39:32.163921+0000","last_fullsized":"2024-09-16T15:39:32.163921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171868+0000","last_change":"2024-09-16T15:39:32.169367+0000","last_active":"2024-09-16T15:39:32.171868+0000","last_peered":"2024-09-16T15:39:32.171868+0000","last_clean":"2024-09-16T15:39:32.171868+0000","last_became_active":"2024-09-16T15:39:32.169188+0000","last_became_peered":"2024-09-16T15:39:32.169188+0000","last_unstale":"2024-09-16T15:39:32.171868+0000","last_undegraded":"2024-09-16T15:39:32.171868+0000","last_fullsized":"2024-09-16T15:39:32.171868+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:14.156509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151486+0000","last_change":"2024-09-16T15:39:32.151486+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151486+0000","last_undegraded":"2024-09-16T15:39:32.151486+0000","last_fullsized":"2024-09-16T15:39:32.151486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.168920+0000","last_change":"2024-09-16T15:39:32.163551+0000","last_active":"2024-09-16T15:39:32.168920+0000","last_peered":"2024-09-16T15:39:32.168920+0000","last_clean":"2024-09-16T15:39:32.168920+0000","last_became_active":"2024-09-16T15:39:32.163283+0000","last_became_peered":"2024-09-16T15:39:32.163283+0000","last_unstale":"2024-09-16T15:39:32.168920+0000","last_undegraded":"2024-09-16T15:39:32.168920+0000","last_fullsized":"2024-09-16T15:39:32.168920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:54.716352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151031+0000","last_change":"2024-09-16T15:39:32.151031+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151031+0000","last_undegraded":"2024-09-16T15:39:32.151031+0000","last_fullsized":"2024-09-16T15:39:32.151031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165577+0000","last_change":"2024-09-16T15:39:32.161394+0000","last_active":"2024-09-16T15:39:34.165577+0000","last_peered":"2024-09-16T15:39:34.165577+0000","last_clean":"2024-09-16T15:39:34.165577+0000","last_became_active":"2024-09-16T15:39:32.161124+0000","last_became_peered":"2024-09-16T15:39:32.161124+0000","last_unstale":"2024-09-16T15:39:34.165577+0000","last_undegraded":"2024-09-16T15:39:34.165577+0000","last_fullsized":"2024-09-16T15:39:34.165577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:40.912077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.157317+0000","last_change":"2024-09-16T15:39:32.157317+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.157317+0000","last_undegraded":"2024-09-16T15:39:32.157317+0000","last_fullsized":"2024-09-16T15:39:32.157317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166121+0000","last_change":"2024-09-16T15:39:32.163386+0000","last_active":"2024-09-16T15:39:32.166121+0000","last_peered":"2024-09-16T15:39:32.166121+0000","last_clean":"2024-09-16T15:39:32.166121+0000","last_became_active":"2024-09-16T15:39:32.161214+0000","last_became_peered":"2024-09-16T15:39:32.161214+0000","last_unstale":"2024-09-16T15:39:32.166121+0000","last_undegraded":"2024-09-16T15:39:32.166121+0000","last_fullsized":"2024-09-16T15:39:32.166121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:34.093301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165551+0000","last_change":"2024-09-16T15:39:33.175372+0000","last_active":"2024-09-16T15:39:34.165551+0000","last_peered":"2024-09-16T15:39:34.165551+0000","last_clean":"2024-09-16T15:39:34.165551+0000","last_became_active":"2024-09-16T15:39:33.174198+0000","last_became_peered":"2024-09-16T15:39:33.174198+0000","last_unstale":"2024-09-16T15:39:34.165551+0000","last_undegraded":"2024-09-16T15:39:34.165551+0000","last_fullsized":"2024-09-16T15:39:34.165551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:53.410594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.169802+0000","last_change":"2024-09-16T15:39:32.166140+0000","last_active":"2024-09-16T15:39:32.169802+0000","last_peered":"2024-09-16T15:39:32.169802+0000","last_clean":"2024-09-16T15:39:32.169802+0000","last_became_active":"2024-09-16T15:39:32.165774+0000","last_became_peered":"2024-09-16T15:39:32.165774+0000","last_unstale":"2024-09-16T15:39:32.169802+0000","last_undegraded":"2024-09-16T15:39:32.169802+0000","last_fullsized":"2024-09-16T15:39:32.169802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:48.317883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151377+0000","last_change":"2024-09-16T15:39:32.151377+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151377+0000","last_undegraded":"2024-09-16T15:39:32.151377+0000","last_fullsized":"2024-09-16T15:39:32.151377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165944+0000","last_change":"2024-09-16T15:39:32.162492+0000","last_active":"2024-09-16T15:39:32.165944+0000","last_peered":"2024-09-16T15:39:32.165944+0000","last_clean":"2024-09-16T15:39:32.165944+0000","last_became_active":"2024-09-16T15:39:32.162229+0000","last_became_peered":"2024-09-16T15:39:32.162229+0000","last_unstale":"2024-09-16T15:39:32.165944+0000","last_undegraded":"2024-09-16T15:39:32.165944+0000","last_fullsized":"2024-09-16T15:39:32.165944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:19.223491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150686+0000","last_change":"2024-09-16T15:39:32.150686+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150686+0000","last_undegraded":"2024-09-16T15:39:32.150686+0000","last_fullsized":"2024-09-16T15:39:32.150686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.164987+0000","last_change":"2024-09-16T15:39:32.161540+0000","last_active":"2024-09-16T15:39:32.164987+0000","last_peered":"2024-09-16T15:39:32.164987+0000","last_clean":"2024-09-16T15:39:32.164987+0000","last_became_active":"2024-09-16T15:39:32.161225+0000","last_became_peered":"2024-09-16T15:39:32.161225+0000","last_unstale":"2024-09-16T15:39:32.164987+0000","last_undegraded":"2024-09-16T15:39:32.164987+0000","last_fullsized":"2024-09-16T15:39:32.164987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:21:04.290055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151288+0000","last_change":"2024-09-16T15:39:32.151288+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151288+0000","last_undegraded":"2024-09-16T15:39:32.151288+0000","last_fullsized":"2024-09-16T15:39:32.151288+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.160197+0000","last_change":"2024-09-16T15:39:32.157654+0000","last_active":"2024-09-16T15:39:32.160197+0000","last_peered":"2024-09-16T15:39:32.160197+0000","last_clean":"2024-09-16T15:39:32.160197+0000","last_became_active":"2024-09-16T15:39:32.157393+0000","last_became_peered":"2024-09-16T15:39:32.157393+0000","last_unstale":"2024-09-16T15:39:32.160197+0000","last_undegraded":"2024-09-16T15:39:32.160197+0000","last_fullsized":"2024-09-16T15:39:32.160197+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:07.931223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154817+0000","last_change":"2024-09-16T15:39:32.154817+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154817+0000","last_undegraded":"2024-09-16T15:39:32.154817+0000","last_fullsized":"2024-09-16T15:39:32.154817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166535+0000","last_change":"2024-09-16T15:39:32.162100+0000","last_active":"2024-09-16T15:39:32.166535+0000","last_peered":"2024-09-16T15:39:32.166535+0000","last_clean":"2024-09-16T15:39:32.166535+0000","last_became_active":"2024-09-16T15:39:32.161782+0000","last_became_peered":"2024-09-16T15:39:32.161782+0000","last_unstale":"2024-09-16T15:39:32.166535+0000","last_undegraded":"2024-09-16T15:39:32.166535+0000","last_fullsized":"2024-09-16T15:39:32.166535+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:51:50.007487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.161447+0000","last_change":"2024-09-16T15:39:32.161447+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.161447+0000","last_undegraded":"2024-09-16T15:39:32.161447+0000","last_fullsized":"2024-09-16T15:39:32.161447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172551+0000","last_change":"2024-09-16T15:39:32.168491+0000","last_active":"2024-09-16T15:39:32.172551+0000","last_peered":"2024-09-16T15:39:32.172551+0000","last_clean":"2024-09-16T15:39:32.172551+0000","last_became_active":"2024-09-16T15:39:32.168195+0000","last_became_peered":"2024-09-16T15:39:32.168195+0000","last_unstale":"2024-09-16T15:39:32.172551+0000","last_undegraded":"2024-09-16T15:39:32.172551+0000","last_fullsized":"2024-09-16T15:39:32.172551+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:00.428705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150369+0000","last_change":"2024-09-16T15:39:32.150369+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150369+0000","last_undegraded":"2024-09-16T15:39:32.150369+0000","last_fullsized":"2024-09-16T15:39:32.150369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165670+0000","last_change":"2024-09-16T15:39:32.166439+0000","last_active":"2024-09-16T15:39:34.165670+0000","last_peered":"2024-09-16T15:39:34.165670+0000","last_clean":"2024-09-16T15:39:34.165670+0000","last_became_active":"2024-09-16T15:39:32.166175+0000","last_became_peered":"2024-09-16T15:39:32.166175+0000","last_unstale":"2024-09-16T15:39:34.165670+0000","last_undegraded":"2024-09-16T15:39:34.165670+0000","last_fullsized":"2024-09-16T15:39:34.165670+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:39.246388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156915+0000","last_change":"2024-09-16T15:39:32.156915+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156915+0000","last_undegraded":"2024-09-16T15:39:32.156915+0000","last_fullsized":"2024-09-16T15:39:32.156915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.158697+0000","last_change":"2024-09-16T15:39:32.155559+0000","last_active":"2024-09-16T15:39:32.158697+0000","last_peered":"2024-09-16T15:39:32.158697+0000","last_clean":"2024-09-16T15:39:32.158697+0000","last_became_active":"2024-09-16T15:39:32.155286+0000","last_became_peered":"2024-09-16T15:39:32.155286+0000","last_unstale":"2024-09-16T15:39:32.158697+0000","last_undegraded":"2024-09-16T15:39:32.158697+0000","last_fullsized":"2024-09-16T15:39:32.158697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:53:37.151552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150242+0000","last_change":"2024-09-16T15:39:32.150242+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150242+0000","last_undegraded":"2024-09-16T15:39:32.150242+0000","last_fullsized":"2024-09-16T15:39:32.150242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165430+0000","last_change":"2024-09-16T15:39:32.172865+0000","last_active":"2024-09-16T15:39:34.165430+0000","last_peered":"2024-09-16T15:39:34.165430+0000","last_clean":"2024-09-16T15:39:34.165430+0000","last_became_active":"2024-09-16T15:39:32.172616+0000","last_became_peered":"2024-09-16T15:39:32.172616+0000","last_unstale":"2024-09-16T15:39:34.165430+0000","last_undegraded":"2024-09-16T15:39:34.165430+0000","last_fullsized":"2024-09-16T15:39:34.165430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:38.115059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151198+0000","last_change":"2024-09-16T15:39:32.151198+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151198+0000","last_undegraded":"2024-09-16T15:39:32.151198+0000","last_fullsized":"2024-09-16T15:39:32.151198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165849+0000","last_change":"2024-09-16T15:39:32.164165+0000","last_active":"2024-09-16T15:39:32.165849+0000","last_peered":"2024-09-16T15:39:32.165849+0000","last_clean":"2024-09-16T15:39:32.165849+0000","last_became_active":"2024-09-16T15:39:32.163728+0000","last_became_peered":"2024-09-16T15:39:32.163728+0000","last_unstale":"2024-09-16T15:39:32.165849+0000","last_undegraded":"2024-09-16T15:39:32.165849+0000","last_fullsized":"2024-09-16T15:39:32.165849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:07.235894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150024+0000","last_change":"2024-09-16T15:39:32.150024+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150024+0000","last_undegraded":"2024-09-16T15:39:32.150024+0000","last_fullsized":"2024-09-16T15:39:32.150024+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165133+0000","last_change":"2024-09-16T15:39:32.167828+0000","last_active":"2024-09-16T15:39:34.165133+0000","last_peered":"2024-09-16T15:39:34.165133+0000","last_clean":"2024-09-16T15:39:34.165133+0000","last_became_active":"2024-09-16T15:39:32.165651+0000","last_became_peered":"2024-09-16T15:39:32.165651+0000","last_unstale":"2024-09-16T15:39:34.165133+0000","last_undegraded":"2024-09-16T15:39:34.165133+0000","last_fullsized":"2024-09-16T15:39:34.165133+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:29.486963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152940+0000","last_change":"2024-09-16T15:39:32.152940+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152940+0000","last_undegraded":"2024-09-16T15:39:32.152940+0000","last_fullsized":"2024-09-16T15:39:32.152940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172404+0000","last_change":"2024-09-16T15:39:32.169431+0000","last_active":"2024-09-16T15:39:32.172404+0000","last_peered":"2024-09-16T15:39:32.172404+0000","last_clean":"2024-09-16T15:39:32.172404+0000","last_became_active":"2024-09-16T15:39:32.169166+0000","last_became_peered":"2024-09-16T15:39:32.169166+0000","last_unstale":"2024-09-16T15:39:32.172404+0000","last_undegraded":"2024-09-16T15:39:32.172404+0000","last_fullsized":"2024-09-16T15:39:32.172404+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:12.103417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153357+0000","last_change":"2024-09-16T15:39:32.153357+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153357+0000","last_undegraded":"2024-09-16T15:39:32.153357+0000","last_fullsized":"2024-09-16T15:39:32.153357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171856+0000","last_change":"2024-09-16T15:39:32.170091+0000","last_active":"2024-09-16T15:39:32.171856+0000","last_peered":"2024-09-16T15:39:32.171856+0000","last_clean":"2024-09-16T15:39:32.171856+0000","last_became_active":"2024-09-16T15:39:32.169831+0000","last_became_peered":"2024-09-16T15:39:32.169831+0000","last_unstale":"2024-09-16T15:39:32.171856+0000","last_undegraded":"2024-09-16T15:39:32.171856+0000","last_fullsized":"2024-09-16T15:39:32.171856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:18.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159832+0000","last_change":"2024-09-16T15:39:32.159832+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159832+0000","last_undegraded":"2024-09-16T15:39:32.159832+0000","last_fullsized":"2024-09-16T15:39:32.159832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167962+0000","last_change":"2024-09-16T15:39:32.163553+0000","last_active":"2024-09-16T15:39:32.167962+0000","last_peered":"2024-09-16T15:39:32.167962+0000","last_clean":"2024-09-16T15:39:32.167962+0000","last_became_active":"2024-09-16T15:39:32.162649+0000","last_became_peered":"2024-09-16T15:39:32.162649+0000","last_unstale":"2024-09-16T15:39:32.167962+0000","last_undegraded":"2024-09-16T15:39:32.167962+0000","last_fullsized":"2024-09-16T15:39:32.167962+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:12.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":131,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167970+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:39:32.167970+0000","last_peered":"2024-09-16T15:39:32.167970+0000","last_clean":"2024-09-16T15:39:32.167970+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:39:32.167970+0000","last_undegraded":"2024-09-16T15:39:32.167970+0000","last_fullsized":"2024-09-16T15:39:32.167970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":734366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":970752,"data_stored":734366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":417792,"data_stored":158707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3f is not active+clean 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.154188+0000', 'last_change': '2024-09-16T15:39:32.154188+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.154188+0000', 'last_undegraded': '2024-09-16T15:39:32.154188+0000', 'last_fullsized': '2024-09-16T15:39:32.154188+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.159798+0000', 'last_change': '2024-09-16T15:39:32.159798+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.159798+0000', 'last_undegraded': '2024-09-16T15:39:32.159798+0000', 'last_fullsized': '2024-09-16T15:39:32.159798+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.156813+0000', 'last_change': '2024-09-16T15:39:32.156813+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.156813+0000', 'last_undegraded': '2024-09-16T15:39:32.156813+0000', 'last_fullsized': '2024-09-16T15:39:32.156813+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.156011+0000', 'last_change': '2024-09-16T15:39:32.156011+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.156011+0000', 'last_undegraded': '2024-09-16T15:39:32.156011+0000', 'last_fullsized': '2024-09-16T15:39:32.156011+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3a is not active+clean 2024-09-16T15:39:38.270 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.158243+0000', 'last_change': '2024-09-16T15:39:32.158243+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.158243+0000', 'last_undegraded': '2024-09-16T15:39:32.158243+0000', 'last_fullsized': '2024-09-16T15:39:32.158243+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:PG 5.39 is not active+clean 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.159374+0000', 'last_change': '2024-09-16T15:39:32.159374+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.159374+0000', 'last_undegraded': '2024-09-16T15:39:32.159374+0000', 'last_fullsized': '2024-09-16T15:39:32.159374+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:PG 5.38 is not active+clean 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.165592+0000', 'last_change': '2024-09-16T15:39:32.165592+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.165592+0000', 'last_undegraded': '2024-09-16T15:39:32.165592+0000', 'last_fullsized': '2024-09-16T15:39:32.165592+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.150787+0000', 'last_change': '2024-09-16T15:39:32.150787+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.150787+0000', 'last_undegraded': '2024-09-16T15:39:32.150787+0000', 'last_fullsized': '2024-09-16T15:39:32.150787+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:PG 5.35 is not active+clean 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.165508+0000', 'last_change': '2024-09-16T15:39:32.165508+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.165508+0000', 'last_undegraded': '2024-09-16T15:39:32.165508+0000', 'last_fullsized': '2024-09-16T15:39:32.165508+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.156865+0000', 'last_change': '2024-09-16T15:39:32.156865+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.156865+0000', 'last_undegraded': '2024-09-16T15:39:32.156865+0000', 'last_fullsized': '2024-09-16T15:39:32.156865+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:PG 5.33 is not active+clean 2024-09-16T15:39:38.271 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151813+0000', 'last_change': '2024-09-16T15:39:32.151813+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151813+0000', 'last_undegraded': '2024-09-16T15:39:32.151813+0000', 'last_fullsized': '2024-09-16T15:39:32.151813+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:PG 5.32 is not active+clean 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.156333+0000', 'last_change': '2024-09-16T15:39:32.156333+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.156333+0000', 'last_undegraded': '2024-09-16T15:39:32.156333+0000', 'last_fullsized': '2024-09-16T15:39:32.156333+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:PG 5.31 is not active+clean 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.161031+0000', 'last_change': '2024-09-16T15:39:32.161031+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.161031+0000', 'last_undegraded': '2024-09-16T15:39:32.161031+0000', 'last_fullsized': '2024-09-16T15:39:32.161031+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:PG 5.30 is not active+clean 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.158855+0000', 'last_change': '2024-09-16T15:39:32.158855+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.158855+0000', 'last_undegraded': '2024-09-16T15:39:32.158855+0000', 'last_fullsized': '2024-09-16T15:39:32.158855+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2f is not active+clean 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.164804+0000', 'last_change': '2024-09-16T15:39:32.164804+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.164804+0000', 'last_undegraded': '2024-09-16T15:39:32.164804+0000', 'last_fullsized': '2024-09-16T15:39:32.164804+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2e is not active+clean 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.156100+0000', 'last_change': '2024-09-16T15:39:32.156100+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.156100+0000', 'last_undegraded': '2024-09-16T15:39:32.156100+0000', 'last_fullsized': '2024-09-16T15:39:32.156100+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.272 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2d is not active+clean 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.153593+0000', 'last_change': '2024-09-16T15:39:32.153593+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.153593+0000', 'last_undegraded': '2024-09-16T15:39:32.153593+0000', 'last_fullsized': '2024-09-16T15:39:32.153593+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2c is not active+clean 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.152311+0000', 'last_change': '2024-09-16T15:39:32.152311+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.152311+0000', 'last_undegraded': '2024-09-16T15:39:32.152311+0000', 'last_fullsized': '2024-09-16T15:39:32.152311+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.152875+0000', 'last_change': '2024-09-16T15:39:32.152875+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.152875+0000', 'last_undegraded': '2024-09-16T15:39:32.152875+0000', 'last_fullsized': '2024-09-16T15:39:32.152875+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151657+0000', 'last_change': '2024-09-16T15:39:32.151657+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151657+0000', 'last_undegraded': '2024-09-16T15:39:32.151657+0000', 'last_fullsized': '2024-09-16T15:39:32.151657+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:PG 5.29 is not active+clean 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151923+0000', 'last_change': '2024-09-16T15:39:32.151923+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151923+0000', 'last_undegraded': '2024-09-16T15:39:32.151923+0000', 'last_fullsized': '2024-09-16T15:39:32.151923+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:38.273 INFO:tasks.ceph.ceph_manager.ceph:PG 5.28 is not active+clean 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.155225+0000', 'last_change': '2024-09-16T15:39:32.155225+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.155225+0000', 'last_undegraded': '2024-09-16T15:39:32.155225+0000', 'last_fullsized': '2024-09-16T15:39:32.155225+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:PG 5.26 is not active+clean 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.153291+0000', 'last_change': '2024-09-16T15:39:32.153291+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.153291+0000', 'last_undegraded': '2024-09-16T15:39:32.153291+0000', 'last_fullsized': '2024-09-16T15:39:32.153291+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:PG 5.24 is not active+clean 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.156476+0000', 'last_change': '2024-09-16T15:39:32.156476+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.156476+0000', 'last_undegraded': '2024-09-16T15:39:32.156476+0000', 'last_fullsized': '2024-09-16T15:39:32.156476+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.152413+0000', 'last_change': '2024-09-16T15:39:32.152413+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.152413+0000', 'last_undegraded': '2024-09-16T15:39:32.152413+0000', 'last_fullsized': '2024-09-16T15:39:32.152413+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:PG 5.22 is not active+clean 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.165973+0000', 'last_change': '2024-09-16T15:39:32.165973+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.165973+0000', 'last_undegraded': '2024-09-16T15:39:32.165973+0000', 'last_fullsized': '2024-09-16T15:39:32.165973+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.274 INFO:tasks.ceph.ceph_manager.ceph:PG 5.21 is not active+clean 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.155795+0000', 'last_change': '2024-09-16T15:39:32.155795+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.155795+0000', 'last_undegraded': '2024-09-16T15:39:32.155795+0000', 'last_fullsized': '2024-09-16T15:39:32.155795+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:PG 5.20 is not active+clean 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.165147+0000', 'last_change': '2024-09-16T15:39:32.165147+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.165147+0000', 'last_undegraded': '2024-09-16T15:39:32.165147+0000', 'last_fullsized': '2024-09-16T15:39:32.165147+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.150200+0000', 'last_change': '2024-09-16T15:39:32.150200+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.150200+0000', 'last_undegraded': '2024-09-16T15:39:32.150200+0000', 'last_fullsized': '2024-09-16T15:39:32.150200+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.153708+0000', 'last_change': '2024-09-16T15:39:32.153708+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.153708+0000', 'last_undegraded': '2024-09-16T15:39:32.153708+0000', 'last_fullsized': '2024-09-16T15:39:32.153708+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151859+0000', 'last_change': '2024-09-16T15:39:32.151859+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151859+0000', 'last_undegraded': '2024-09-16T15:39:32.151859+0000', 'last_fullsized': '2024-09-16T15:39:32.151859+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-16T15:39:38.275 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.165563+0000', 'last_change': '2024-09-16T15:39:32.165563+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.165563+0000', 'last_undegraded': '2024-09-16T15:39:32.165563+0000', 'last_fullsized': '2024-09-16T15:39:32.165563+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.155925+0000', 'last_change': '2024-09-16T15:39:32.155925+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.155925+0000', 'last_undegraded': '2024-09-16T15:39:32.155925+0000', 'last_fullsized': '2024-09-16T15:39:32.155925+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.150903+0000', 'last_change': '2024-09-16T15:39:32.150903+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.150903+0000', 'last_undegraded': '2024-09-16T15:39:32.150903+0000', 'last_fullsized': '2024-09-16T15:39:32.150903+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.156442+0000', 'last_change': '2024-09-16T15:39:32.156442+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.156442+0000', 'last_undegraded': '2024-09-16T15:39:32.156442+0000', 'last_fullsized': '2024-09-16T15:39:32.156442+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:PG 5.17 is not active+clean 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.153208+0000', 'last_change': '2024-09-16T15:39:32.153208+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.153208+0000', 'last_undegraded': '2024-09-16T15:39:32.153208+0000', 'last_fullsized': '2024-09-16T15:39:32.153208+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:PG 5.14 is not active+clean 2024-09-16T15:39:38.276 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.154532+0000', 'last_change': '2024-09-16T15:39:32.154532+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.154532+0000', 'last_undegraded': '2024-09-16T15:39:32.154532+0000', 'last_fullsized': '2024-09-16T15:39:32.154532+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.157250+0000', 'last_change': '2024-09-16T15:39:32.157250+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.157250+0000', 'last_undegraded': '2024-09-16T15:39:32.157250+0000', 'last_fullsized': '2024-09-16T15:39:32.157250+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:PG 5.13 is not active+clean 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.154053+0000', 'last_change': '2024-09-16T15:39:32.154053+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.154053+0000', 'last_undegraded': '2024-09-16T15:39:32.154053+0000', 'last_fullsized': '2024-09-16T15:39:32.154053+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.163921+0000', 'last_change': '2024-09-16T15:39:32.163921+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.163921+0000', 'last_undegraded': '2024-09-16T15:39:32.163921+0000', 'last_fullsized': '2024-09-16T15:39:32.163921+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151486+0000', 'last_change': '2024-09-16T15:39:32.151486+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151486+0000', 'last_undegraded': '2024-09-16T15:39:32.151486+0000', 'last_fullsized': '2024-09-16T15:39:32.151486+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151031+0000', 'last_change': '2024-09-16T15:39:32.151031+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151031+0000', 'last_undegraded': '2024-09-16T15:39:32.151031+0000', 'last_fullsized': '2024-09-16T15:39:32.151031+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:38.277 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.157317+0000', 'last_change': '2024-09-16T15:39:32.157317+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.157317+0000', 'last_undegraded': '2024-09-16T15:39:32.157317+0000', 'last_fullsized': '2024-09-16T15:39:32.157317+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151377+0000', 'last_change': '2024-09-16T15:39:32.151377+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151377+0000', 'last_undegraded': '2024-09-16T15:39:32.151377+0000', 'last_fullsized': '2024-09-16T15:39:32.151377+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.150686+0000', 'last_change': '2024-09-16T15:39:32.150686+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.150686+0000', 'last_undegraded': '2024-09-16T15:39:32.150686+0000', 'last_fullsized': '2024-09-16T15:39:32.150686+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151288+0000', 'last_change': '2024-09-16T15:39:32.151288+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151288+0000', 'last_undegraded': '2024-09-16T15:39:32.151288+0000', 'last_fullsized': '2024-09-16T15:39:32.151288+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.154817+0000', 'last_change': '2024-09-16T15:39:32.154817+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.154817+0000', 'last_undegraded': '2024-09-16T15:39:32.154817+0000', 'last_fullsized': '2024-09-16T15:39:32.154817+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:38.278 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.161447+0000', 'last_change': '2024-09-16T15:39:32.161447+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.161447+0000', 'last_undegraded': '2024-09-16T15:39:32.161447+0000', 'last_fullsized': '2024-09-16T15:39:32.161447+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.150369+0000', 'last_change': '2024-09-16T15:39:32.150369+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.150369+0000', 'last_undegraded': '2024-09-16T15:39:32.150369+0000', 'last_fullsized': '2024-09-16T15:39:32.150369+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.156915+0000', 'last_change': '2024-09-16T15:39:32.156915+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.156915+0000', 'last_undegraded': '2024-09-16T15:39:32.156915+0000', 'last_fullsized': '2024-09-16T15:39:32.156915+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.150242+0000', 'last_change': '2024-09-16T15:39:32.150242+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.150242+0000', 'last_undegraded': '2024-09-16T15:39:32.150242+0000', 'last_fullsized': '2024-09-16T15:39:32.150242+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.151198+0000', 'last_change': '2024-09-16T15:39:32.151198+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.151198+0000', 'last_undegraded': '2024-09-16T15:39:32.151198+0000', 'last_fullsized': '2024-09-16T15:39:32.151198+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-16T15:39:38.279 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.150024+0000', 'last_change': '2024-09-16T15:39:32.150024+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.150024+0000', 'last_undegraded': '2024-09-16T15:39:32.150024+0000', 'last_fullsized': '2024-09-16T15:39:32.150024+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:38.280 INFO:tasks.ceph.ceph_manager.ceph:PG 5.0 is not active+clean 2024-09-16T15:39:38.280 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.152940+0000', 'last_change': '2024-09-16T15:39:32.152940+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.152940+0000', 'last_undegraded': '2024-09-16T15:39:32.152940+0000', 'last_fullsized': '2024-09-16T15:39:32.152940+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:38.280 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-16T15:39:38.280 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.153357+0000', 'last_change': '2024-09-16T15:39:32.153357+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.153357+0000', 'last_undegraded': '2024-09-16T15:39:32.153357+0000', 'last_fullsized': '2024-09-16T15:39:32.153357+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:38.280 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-16T15:39:38.280 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:32.159832+0000', 'last_change': '2024-09-16T15:39:32.159832+0000', 'last_active': '2024-09-16T15:39:32.138319+0000', 'last_peered': '2024-09-16T15:39:32.138319+0000', 'last_clean': '2024-09-16T15:39:32.138319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:32.159832+0000', 'last_undegraded': '2024-09-16T15:39:32.159832+0000', 'last_fullsized': '2024-09-16T15:39:32.159832+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:32.138319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:39:38.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:39:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.464+0000 7f5b53e00640 1 -- 172.21.15.60:0/2671225345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d25d0 msgr2=0x7f5b540cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.464+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2671225345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d25d0 0x7f5b540cf9b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5b40005fd0 tx=0x7f5b4002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.464+0000 7f5b53e00640 1 -- 172.21.15.60:0/2671225345 shutdown_connections 2024-09-16T15:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.464+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2671225345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d25d0 0x7f5b540cf9b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.464+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2671225345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b540d1260 0x7f5b540d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.464+0000 7f5b53e00640 1 -- 172.21.15.60:0/2671225345 >> 172.21.15.60:0/2671225345 conn(0x7f5b540d3e40 msgr2=0x7f5b540d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.464+0000 7f5b53e00640 1 -- 172.21.15.60:0/2671225345 shutdown_connections 2024-09-16T15:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.464+0000 7f5b53e00640 1 -- 172.21.15.60:0/2671225345 wait complete. 2024-09-16T15:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.466+0000 7f5b53e00640 1 Processor -- start 2024-09-16T15:39:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.466+0000 7f5b53e00640 1 -- start start 2024-09-16T15:39:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d1260 0x7f5b5409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b540d25d0 0x7f5b5409ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5415e050 0x7f5b5409f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b53e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5b540b70e0 con 0x7f5b540d1260 2024-09-16T15:39:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b53e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5b540b6f60 con 0x7f5b5415e050 2024-09-16T15:39:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b53e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5b540b7260 con 0x7f5b540d25d0 2024-09-16T15:39:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b52a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d1260 0x7f5b5409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b52a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d1260 0x7f5b5409db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42136/0 (socket says 172.21.15.60:42136) 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b52a00640 1 -- 172.21.15.60:0/2046666046 learned_addr learned my addr 172.21.15.60:0/2046666046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b52a00640 1 -- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b540d25d0 msgr2=0x7f5b5409ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b52a00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b540d25d0 0x7f5b5409ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b52a00640 1 -- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5415e050 msgr2=0x7f5b5409f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b52a00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5415e050 0x7f5b5409f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.467+0000 7f5b52a00640 1 -- 172.21.15.60:0/2046666046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5409e350 con 0x7f5b540d1260 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.468+0000 7f5b52a00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d1260 0x7f5b5409db20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400c3d0 tx=0x7f5b4400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.468+0000 7f5b4aa00640 1 -- 172.21.15.60:0/2046666046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44017070 con 0x7f5b540d1260 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.468+0000 7f5b4aa00640 1 -- 172.21.15.60:0/2046666046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b440053c0 con 0x7f5b540d1260 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.468+0000 7f5b4aa00640 1 -- 172.21.15.60:0/2046666046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44003850 con 0x7f5b540d1260 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.468+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5409e630 con 0x7f5b540d1260 2024-09-16T15:39:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.468+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5409ebf0 con 0x7f5b540d1260 2024-09-16T15:39:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.469+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b540b7f40 con 0x7f5b540d1260 2024-09-16T15:39:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.470+0000 7f5b4aa00640 1 -- 172.21.15.60:0/2046666046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5b4401a070 con 0x7f5b540d1260 2024-09-16T15:39:38.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.471+0000 7f5b4aa00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5b34078850 0x7f5b3407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.471+0000 7f5b4aa00640 1 -- 172.21.15.60:0/2046666046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f5b44099d40 con 0x7f5b540d1260 2024-09-16T15:39:38.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.472+0000 7f5b52000640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5b34078850 0x7f5b3407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.472+0000 7f5b52000640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5b34078850 0x7f5b3407ad10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b40033000 tx=0x7f5b400046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.476+0000 7f5b4aa00640 1 -- 172.21.15.60:0/2046666046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5b44062310 con 0x7f5b540d1260 2024-09-16T15:39:38.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.746+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5b540d0630 con 0x7f5b540d1260 2024-09-16T15:39:38.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.747+0000 7f5b4aa00640 1 -- 172.21.15.60:0/2046666046 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1686 (secure 0 0 0) 0x7f5b44061c10 con 0x7f5b540d1260 2024-09-16T15:39:38.748 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:38.748 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":90,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501102,"num_in_osds":8,"osd_in_since":1726501092,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"active+clean","count":41}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590390,"bytes_used":242712576,"bytes_avail":772851400704,"bytes_total":773094113280,"inactive_pgs_ratio":0.57731956243515015,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":17,"btime":"2024-09-16T15:39:36:169518+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4881}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-16T15:39:23.720301+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T15:39:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5b34078850 msgr2=0x7f5b3407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5b34078850 0x7f5b3407ad10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b40033000 tx=0x7f5b400046c0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d1260 msgr2=0x7f5b5409db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d1260 0x7f5b5409db20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400c3d0 tx=0x7f5b4400c890 comp rx=0 tx=0).stop 2024-09-16T15:39:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 shutdown_connections 2024-09-16T15:39:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5b34078850 0x7f5b3407ad10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b5415e050 0x7f5b5409f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b540d25d0 0x7f5b5409ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 --2- 172.21.15.60:0/2046666046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b540d1260 0x7f5b5409db20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 >> 172.21.15.60:0/2046666046 conn(0x7f5b540d3e40 msgr2=0x7f5b540b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 shutdown_connections 2024-09-16T15:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.749+0000 7f5b53e00640 1 -- 172.21.15.60:0/2046666046 wait complete. 2024-09-16T15:39:38.767 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:39:38.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.926+0000 7f2164400640 1 -- 172.21.15.60:0/3109611014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c0ccf00 msgr2=0x7f215c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.926+0000 7f2164400640 1 --2- 172.21.15.60:0/3109611014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c0ccf00 0x7f215c0b2440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2148005fd0 tx=0x7f214802a540 comp rx=0 tx=0).stop 2024-09-16T15:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.926+0000 7f2164400640 1 -- 172.21.15.60:0/3109611014 shutdown_connections 2024-09-16T15:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.926+0000 7f2164400640 1 --2- 172.21.15.60:0/3109611014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c0ccf00 0x7f215c0b2440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.926+0000 7f2164400640 1 -- 172.21.15.60:0/3109611014 >> 172.21.15.60:0/3109611014 conn(0x7f215c0b1950 msgr2=0x7f215c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.926+0000 7f2164400640 1 -- 172.21.15.60:0/3109611014 shutdown_connections 2024-09-16T15:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.926+0000 7f2164400640 1 -- 172.21.15.60:0/3109611014 wait complete. 2024-09-16T15:39:38.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.928+0000 7f2164400640 1 Processor -- start 2024-09-16T15:39:38.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.928+0000 7f2164400640 1 -- start start 2024-09-16T15:39:38.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2164400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f215c073fb0 0x7f215c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2164400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f215c0ccf00 0x7f215c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2164400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15f5a0 0x7f215c0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2164400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f215c06f9f0 con 0x7f215c15f5a0 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2164400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f215c06f870 con 0x7f215c0ccf00 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2164400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f215c074420 con 0x7f215c073fb0 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2162400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15f5a0 0x7f215c0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2162400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15f5a0 0x7f215c0b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57028/0 (socket says 172.21.15.60:57028) 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2162400640 1 -- 172.21.15.60:0/241103776 learned_addr learned my addr 172.21.15.60:0/241103776 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2161000640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f215c0ccf00 0x7f215c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2161a00640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f215c073fb0 0x7f215c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2162400640 1 -- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f215c073fb0 msgr2=0x7f215c1570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2162400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f215c073fb0 0x7f215c1570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.929+0000 7f2162400640 1 -- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f215c0ccf00 msgr2=0x7f215c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.930+0000 7f2162400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f215c0ccf00 0x7f215c0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.930+0000 7f2162400640 1 -- 172.21.15.60:0/241103776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c1f5000 con 0x7f215c15f5a0 2024-09-16T15:39:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.930+0000 7f2162400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15f5a0 0x7f215c0b4510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f215400c7a0 tx=0x7f215400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.930+0000 7f2159600640 1 -- 172.21.15.60:0/241103776 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154003b70 con 0x7f215c15f5a0 2024-09-16T15:39:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.930+0000 7f2159600640 1 -- 172.21.15.60:0/241103776 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2154003d30 con 0x7f215c15f5a0 2024-09-16T15:39:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.930+0000 7f2159600640 1 -- 172.21.15.60:0/241103776 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215400aab0 con 0x7f215c15f5a0 2024-09-16T15:39:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.930+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c1f52e0 con 0x7f215c15f5a0 2024-09-16T15:39:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.930+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c1f58a0 con 0x7f215c15f5a0 2024-09-16T15:39:38.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.931+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215c0b2440 con 0x7f215c15f5a0 2024-09-16T15:39:38.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.932+0000 7f2159600640 1 -- 172.21.15.60:0/241103776 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f215400ac50 con 0x7f215c15f5a0 2024-09-16T15:39:38.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.934+0000 7f2159600640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f213c078850 0x7f213c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.934+0000 7f2159600640 1 -- 172.21.15.60:0/241103776 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f215409a6c0 con 0x7f215c15f5a0 2024-09-16T15:39:38.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.934+0000 7f2161a00640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f213c078850 0x7f213c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.935+0000 7f2161a00640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f213c078850 0x7f213c07ad10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2150006fd0 tx=0x7f2150009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:38.939+0000 7f2159600640 1 -- 172.21.15.60:0/241103776 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2154062640 con 0x7f215c15f5a0 2024-09-16T15:39:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.132+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f215c1f4db0 con 0x7f213c078850 2024-09-16T15:39:39.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.136+0000 7f2159600640 1 -- 172.21.15.60:0/241103776 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247320 (secure 0 0 0) 0x7f215c1f4db0 con 0x7f213c078850 2024-09-16T15:39:39.140 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:39.141 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.140+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f213c078850 msgr2=0x7f213c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.140+0000 7f2164400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f213c078850 0x7f213c07ad10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2150006fd0 tx=0x7f2150009060 comp rx=0 tx=0).stop 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.140+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15f5a0 msgr2=0x7f215c0b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.140+0000 7f2164400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15f5a0 0x7f215c0b4510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f215400c7a0 tx=0x7f215400cc60 comp rx=0 tx=0).stop 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.140+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 shutdown_connections 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.140+0000 7f2164400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f213c078850 0x7f213c07ad10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.140+0000 7f2164400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f215c15f5a0 0x7f215c0b4510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.140+0000 7f2164400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f215c0ccf00 0x7f215c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.141+0000 7f2164400640 1 --2- 172.21.15.60:0/241103776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f215c073fb0 0x7f215c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.141+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 >> 172.21.15.60:0/241103776 conn(0x7f215c0b1950 msgr2=0x7f215c15e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.141+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 shutdown_connections 2024-09-16T15:39:39.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:39.141+0000 7f2164400640 1 -- 172.21.15.60:0/241103776 wait complete. 2024-09-16T15:39:39.158 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-16T15:39:37.723221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237024,"kb_used_data":4132,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737696,"statfs":{"total":773094113280,"available":772851400704,"internally_reserved":0,"allocated":4231168,"data_stored":2341162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154188+0000","last_change":"2024-09-16T15:39:32.154188+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154188+0000","last_undegraded":"2024-09-16T15:39:32.154188+0000","last_fullsized":"2024-09-16T15:39:32.154188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159798+0000","last_change":"2024-09-16T15:39:32.159798+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159798+0000","last_undegraded":"2024-09-16T15:39:32.159798+0000","last_fullsized":"2024-09-16T15:39:32.159798+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156813+0000","last_change":"2024-09-16T15:39:32.156813+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156813+0000","last_undegraded":"2024-09-16T15:39:32.156813+0000","last_fullsized":"2024-09-16T15:39:32.156813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165367+0000","last_change":"2024-09-16T15:39:33.175477+0000","last_active":"2024-09-16T15:39:34.165367+0000","last_peered":"2024-09-16T15:39:34.165367+0000","last_clean":"2024-09-16T15:39:34.165367+0000","last_became_active":"2024-09-16T15:39:33.174529+0000","last_became_peered":"2024-09-16T15:39:33.174529+0000","last_unstale":"2024-09-16T15:39:34.165367+0000","last_undegraded":"2024-09-16T15:39:34.165367+0000","last_fullsized":"2024-09-16T15:39:34.165367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:16.836397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156011+0000","last_change":"2024-09-16T15:39:32.156011+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156011+0000","last_undegraded":"2024-09-16T15:39:32.156011+0000","last_fullsized":"2024-09-16T15:39:32.156011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.158243+0000","last_change":"2024-09-16T15:39:32.158243+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.158243+0000","last_undegraded":"2024-09-16T15:39:32.158243+0000","last_fullsized":"2024-09-16T15:39:32.158243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159374+0000","last_change":"2024-09-16T15:39:32.159374+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159374+0000","last_undegraded":"2024-09-16T15:39:32.159374+0000","last_fullsized":"2024-09-16T15:39:32.159374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165592+0000","last_change":"2024-09-16T15:39:32.165592+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165592+0000","last_undegraded":"2024-09-16T15:39:32.165592+0000","last_fullsized":"2024-09-16T15:39:32.165592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150787+0000","last_change":"2024-09-16T15:39:32.150787+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150787+0000","last_undegraded":"2024-09-16T15:39:32.150787+0000","last_fullsized":"2024-09-16T15:39:32.150787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165700+0000","last_change":"2024-09-16T15:39:33.177863+0000","last_active":"2024-09-16T15:39:34.165700+0000","last_peered":"2024-09-16T15:39:34.165700+0000","last_clean":"2024-09-16T15:39:34.165700+0000","last_became_active":"2024-09-16T15:39:33.176375+0000","last_became_peered":"2024-09-16T15:39:33.176375+0000","last_unstale":"2024-09-16T15:39:34.165700+0000","last_undegraded":"2024-09-16T15:39:34.165700+0000","last_fullsized":"2024-09-16T15:39:34.165700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:10.080289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165508+0000","last_change":"2024-09-16T15:39:32.165508+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165508+0000","last_undegraded":"2024-09-16T15:39:32.165508+0000","last_fullsized":"2024-09-16T15:39:32.165508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156865+0000","last_change":"2024-09-16T15:39:32.156865+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156865+0000","last_undegraded":"2024-09-16T15:39:32.156865+0000","last_fullsized":"2024-09-16T15:39:32.156865+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151813+0000","last_change":"2024-09-16T15:39:32.151813+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151813+0000","last_undegraded":"2024-09-16T15:39:32.151813+0000","last_fullsized":"2024-09-16T15:39:32.151813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156333+0000","last_change":"2024-09-16T15:39:32.156333+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156333+0000","last_undegraded":"2024-09-16T15:39:32.156333+0000","last_fullsized":"2024-09-16T15:39:32.156333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.161031+0000","last_change":"2024-09-16T15:39:32.161031+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.161031+0000","last_undegraded":"2024-09-16T15:39:32.161031+0000","last_fullsized":"2024-09-16T15:39:32.161031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.158855+0000","last_change":"2024-09-16T15:39:32.158855+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.158855+0000","last_undegraded":"2024-09-16T15:39:32.158855+0000","last_fullsized":"2024-09-16T15:39:32.158855+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.164804+0000","last_change":"2024-09-16T15:39:32.164804+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.164804+0000","last_undegraded":"2024-09-16T15:39:32.164804+0000","last_fullsized":"2024-09-16T15:39:32.164804+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156100+0000","last_change":"2024-09-16T15:39:32.156100+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156100+0000","last_undegraded":"2024-09-16T15:39:32.156100+0000","last_fullsized":"2024-09-16T15:39:32.156100+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153593+0000","last_change":"2024-09-16T15:39:32.153593+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153593+0000","last_undegraded":"2024-09-16T15:39:32.153593+0000","last_fullsized":"2024-09-16T15:39:32.153593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152311+0000","last_change":"2024-09-16T15:39:32.152311+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152311+0000","last_undegraded":"2024-09-16T15:39:32.152311+0000","last_fullsized":"2024-09-16T15:39:32.152311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152875+0000","last_change":"2024-09-16T15:39:32.152875+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152875+0000","last_undegraded":"2024-09-16T15:39:32.152875+0000","last_fullsized":"2024-09-16T15:39:32.152875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151657+0000","last_change":"2024-09-16T15:39:32.151657+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151657+0000","last_undegraded":"2024-09-16T15:39:32.151657+0000","last_fullsized":"2024-09-16T15:39:32.151657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151923+0000","last_change":"2024-09-16T15:39:32.151923+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151923+0000","last_undegraded":"2024-09-16T15:39:32.151923+0000","last_fullsized":"2024-09-16T15:39:32.151923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155225+0000","last_change":"2024-09-16T15:39:32.155225+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155225+0000","last_undegraded":"2024-09-16T15:39:32.155225+0000","last_fullsized":"2024-09-16T15:39:32.155225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165596+0000","last_change":"2024-09-16T15:39:33.178266+0000","last_active":"2024-09-16T15:39:34.165596+0000","last_peered":"2024-09-16T15:39:34.165596+0000","last_clean":"2024-09-16T15:39:34.165596+0000","last_became_active":"2024-09-16T15:39:33.177230+0000","last_became_peered":"2024-09-16T15:39:33.177230+0000","last_unstale":"2024-09-16T15:39:34.165596+0000","last_undegraded":"2024-09-16T15:39:34.165596+0000","last_fullsized":"2024-09-16T15:39:34.165596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:59.832311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153291+0000","last_change":"2024-09-16T15:39:32.153291+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153291+0000","last_undegraded":"2024-09-16T15:39:32.153291+0000","last_fullsized":"2024-09-16T15:39:32.153291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165652+0000","last_change":"2024-09-16T15:39:33.176217+0000","last_active":"2024-09-16T15:39:34.165652+0000","last_peered":"2024-09-16T15:39:34.165652+0000","last_clean":"2024-09-16T15:39:34.165652+0000","last_became_active":"2024-09-16T15:39:33.175855+0000","last_became_peered":"2024-09-16T15:39:33.175855+0000","last_unstale":"2024-09-16T15:39:34.165652+0000","last_undegraded":"2024-09-16T15:39:34.165652+0000","last_fullsized":"2024-09-16T15:39:34.165652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:45.876870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156476+0000","last_change":"2024-09-16T15:39:32.156476+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156476+0000","last_undegraded":"2024-09-16T15:39:32.156476+0000","last_fullsized":"2024-09-16T15:39:32.156476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152413+0000","last_change":"2024-09-16T15:39:32.152413+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152413+0000","last_undegraded":"2024-09-16T15:39:32.152413+0000","last_fullsized":"2024-09-16T15:39:32.152413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165973+0000","last_change":"2024-09-16T15:39:32.165973+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165973+0000","last_undegraded":"2024-09-16T15:39:32.165973+0000","last_fullsized":"2024-09-16T15:39:32.165973+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155795+0000","last_change":"2024-09-16T15:39:32.155795+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155795+0000","last_undegraded":"2024-09-16T15:39:32.155795+0000","last_fullsized":"2024-09-16T15:39:32.155795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165147+0000","last_change":"2024-09-16T15:39:32.165147+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165147+0000","last_undegraded":"2024-09-16T15:39:32.165147+0000","last_fullsized":"2024-09-16T15:39:32.165147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150200+0000","last_change":"2024-09-16T15:39:32.150200+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150200+0000","last_undegraded":"2024-09-16T15:39:32.150200+0000","last_fullsized":"2024-09-16T15:39:32.150200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.168107+0000","last_change":"2024-09-16T15:39:32.164389+0000","last_active":"2024-09-16T15:39:32.168107+0000","last_peered":"2024-09-16T15:39:32.168107+0000","last_clean":"2024-09-16T15:39:32.168107+0000","last_became_active":"2024-09-16T15:39:32.164146+0000","last_became_peered":"2024-09-16T15:39:32.164146+0000","last_unstale":"2024-09-16T15:39:32.168107+0000","last_undegraded":"2024-09-16T15:39:32.168107+0000","last_fullsized":"2024-09-16T15:39:32.168107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:22.804697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153708+0000","last_change":"2024-09-16T15:39:32.153708+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153708+0000","last_undegraded":"2024-09-16T15:39:32.153708+0000","last_fullsized":"2024-09-16T15:39:32.153708+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167838+0000","last_change":"2024-09-16T15:39:32.166998+0000","last_active":"2024-09-16T15:39:32.167838+0000","last_peered":"2024-09-16T15:39:32.167838+0000","last_clean":"2024-09-16T15:39:32.167838+0000","last_became_active":"2024-09-16T15:39:32.166503+0000","last_became_peered":"2024-09-16T15:39:32.166503+0000","last_unstale":"2024-09-16T15:39:32.167838+0000","last_undegraded":"2024-09-16T15:39:32.167838+0000","last_fullsized":"2024-09-16T15:39:32.167838+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:39.190074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151859+0000","last_change":"2024-09-16T15:39:32.151859+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151859+0000","last_undegraded":"2024-09-16T15:39:32.151859+0000","last_fullsized":"2024-09-16T15:39:32.151859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.173010+0000","last_change":"2024-09-16T15:39:32.170881+0000","last_active":"2024-09-16T15:39:32.173010+0000","last_peered":"2024-09-16T15:39:32.173010+0000","last_clean":"2024-09-16T15:39:32.173010+0000","last_became_active":"2024-09-16T15:39:32.170639+0000","last_became_peered":"2024-09-16T15:39:32.170639+0000","last_unstale":"2024-09-16T15:39:32.173010+0000","last_undegraded":"2024-09-16T15:39:32.173010+0000","last_fullsized":"2024-09-16T15:39:32.173010+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:03.240823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165671+0000","last_change":"2024-09-16T15:39:33.174871+0000","last_active":"2024-09-16T15:39:34.165671+0000","last_peered":"2024-09-16T15:39:34.165671+0000","last_clean":"2024-09-16T15:39:34.165671+0000","last_became_active":"2024-09-16T15:39:33.171604+0000","last_became_peered":"2024-09-16T15:39:33.171604+0000","last_unstale":"2024-09-16T15:39:34.165671+0000","last_undegraded":"2024-09-16T15:39:34.165671+0000","last_fullsized":"2024-09-16T15:39:34.165671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:26.792470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172991+0000","last_change":"2024-09-16T15:39:32.171713+0000","last_active":"2024-09-16T15:39:32.172991+0000","last_peered":"2024-09-16T15:39:32.172991+0000","last_clean":"2024-09-16T15:39:32.172991+0000","last_became_active":"2024-09-16T15:39:32.171452+0000","last_became_peered":"2024-09-16T15:39:32.171452+0000","last_unstale":"2024-09-16T15:39:32.172991+0000","last_undegraded":"2024-09-16T15:39:32.172991+0000","last_fullsized":"2024-09-16T15:39:32.172991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:45.685224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.165563+0000","last_change":"2024-09-16T15:39:32.165563+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.165563+0000","last_undegraded":"2024-09-16T15:39:32.165563+0000","last_fullsized":"2024-09-16T15:39:32.165563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166016+0000","last_change":"2024-09-16T15:39:32.162895+0000","last_active":"2024-09-16T15:39:32.166016+0000","last_peered":"2024-09-16T15:39:32.166016+0000","last_clean":"2024-09-16T15:39:32.166016+0000","last_became_active":"2024-09-16T15:39:32.162267+0000","last_became_peered":"2024-09-16T15:39:32.162267+0000","last_unstale":"2024-09-16T15:39:32.166016+0000","last_undegraded":"2024-09-16T15:39:32.166016+0000","last_fullsized":"2024-09-16T15:39:32.166016+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:35.359481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.155925+0000","last_change":"2024-09-16T15:39:32.155925+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.155925+0000","last_undegraded":"2024-09-16T15:39:32.155925+0000","last_fullsized":"2024-09-16T15:39:32.155925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166200+0000","last_change":"2024-09-16T15:39:32.163428+0000","last_active":"2024-09-16T15:39:32.166200+0000","last_peered":"2024-09-16T15:39:32.166200+0000","last_clean":"2024-09-16T15:39:32.166200+0000","last_became_active":"2024-09-16T15:39:32.163148+0000","last_became_peered":"2024-09-16T15:39:32.163148+0000","last_unstale":"2024-09-16T15:39:32.166200+0000","last_undegraded":"2024-09-16T15:39:32.166200+0000","last_fullsized":"2024-09-16T15:39:32.166200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:31.750012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150903+0000","last_change":"2024-09-16T15:39:32.150903+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150903+0000","last_undegraded":"2024-09-16T15:39:32.150903+0000","last_fullsized":"2024-09-16T15:39:32.150903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171837+0000","last_change":"2024-09-16T15:39:32.164943+0000","last_active":"2024-09-16T15:39:32.171837+0000","last_peered":"2024-09-16T15:39:32.171837+0000","last_clean":"2024-09-16T15:39:32.171837+0000","last_became_active":"2024-09-16T15:39:32.164576+0000","last_became_peered":"2024-09-16T15:39:32.164576+0000","last_unstale":"2024-09-16T15:39:32.171837+0000","last_undegraded":"2024-09-16T15:39:32.171837+0000","last_fullsized":"2024-09-16T15:39:32.171837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:36.312022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164735+0000","last_change":"2024-09-16T15:39:33.170443+0000","last_active":"2024-09-16T15:39:34.164735+0000","last_peered":"2024-09-16T15:39:34.164735+0000","last_clean":"2024-09-16T15:39:34.164735+0000","last_became_active":"2024-09-16T15:39:33.170165+0000","last_became_peered":"2024-09-16T15:39:33.170165+0000","last_unstale":"2024-09-16T15:39:34.164735+0000","last_undegraded":"2024-09-16T15:39:34.164735+0000","last_fullsized":"2024-09-16T15:39:34.164735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.857670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165602+0000","last_change":"2024-09-16T15:39:32.160722+0000","last_active":"2024-09-16T15:39:32.165602+0000","last_peered":"2024-09-16T15:39:32.165602+0000","last_clean":"2024-09-16T15:39:32.165602+0000","last_became_active":"2024-09-16T15:39:32.160298+0000","last_became_peered":"2024-09-16T15:39:32.160298+0000","last_unstale":"2024-09-16T15:39:32.165602+0000","last_undegraded":"2024-09-16T15:39:32.165602+0000","last_fullsized":"2024-09-16T15:39:32.165602+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:32.567093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156442+0000","last_change":"2024-09-16T15:39:32.156442+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156442+0000","last_undegraded":"2024-09-16T15:39:32.156442+0000","last_fullsized":"2024-09-16T15:39:32.156442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172968+0000","last_change":"2024-09-16T15:39:32.170156+0000","last_active":"2024-09-16T15:39:32.172968+0000","last_peered":"2024-09-16T15:39:32.172968+0000","last_clean":"2024-09-16T15:39:32.172968+0000","last_became_active":"2024-09-16T15:39:32.169818+0000","last_became_peered":"2024-09-16T15:39:32.169818+0000","last_unstale":"2024-09-16T15:39:32.172968+0000","last_undegraded":"2024-09-16T15:39:32.172968+0000","last_fullsized":"2024-09-16T15:39:32.172968+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:42.329583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153208+0000","last_change":"2024-09-16T15:39:32.153208+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153208+0000","last_undegraded":"2024-09-16T15:39:32.153208+0000","last_fullsized":"2024-09-16T15:39:32.153208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165089+0000","last_change":"2024-09-16T15:39:32.161560+0000","last_active":"2024-09-16T15:39:32.165089+0000","last_peered":"2024-09-16T15:39:32.165089+0000","last_clean":"2024-09-16T15:39:32.165089+0000","last_became_active":"2024-09-16T15:39:32.161287+0000","last_became_peered":"2024-09-16T15:39:32.161287+0000","last_unstale":"2024-09-16T15:39:32.165089+0000","last_undegraded":"2024-09-16T15:39:32.165089+0000","last_fullsized":"2024-09-16T15:39:32.165089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:33.389071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154532+0000","last_change":"2024-09-16T15:39:32.154532+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154532+0000","last_undegraded":"2024-09-16T15:39:32.154532+0000","last_fullsized":"2024-09-16T15:39:32.154532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.175093+0000","last_change":"2024-09-16T15:39:32.170843+0000","last_active":"2024-09-16T15:39:32.175093+0000","last_peered":"2024-09-16T15:39:32.175093+0000","last_clean":"2024-09-16T15:39:32.175093+0000","last_became_active":"2024-09-16T15:39:32.170538+0000","last_became_peered":"2024-09-16T15:39:32.170538+0000","last_unstale":"2024-09-16T15:39:32.175093+0000","last_undegraded":"2024-09-16T15:39:32.175093+0000","last_fullsized":"2024-09-16T15:39:32.175093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:06.203396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.157250+0000","last_change":"2024-09-16T15:39:32.157250+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.157250+0000","last_undegraded":"2024-09-16T15:39:32.157250+0000","last_fullsized":"2024-09-16T15:39:32.157250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171781+0000","last_change":"2024-09-16T15:39:32.165779+0000","last_active":"2024-09-16T15:39:32.171781+0000","last_peered":"2024-09-16T15:39:32.171781+0000","last_clean":"2024-09-16T15:39:32.171781+0000","last_became_active":"2024-09-16T15:39:32.165510+0000","last_became_peered":"2024-09-16T15:39:32.165510+0000","last_unstale":"2024-09-16T15:39:32.171781+0000","last_undegraded":"2024-09-16T15:39:32.171781+0000","last_fullsized":"2024-09-16T15:39:32.171781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:08.898328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164871+0000","last_change":"2024-09-16T15:39:33.169134+0000","last_active":"2024-09-16T15:39:34.164871+0000","last_peered":"2024-09-16T15:39:34.164871+0000","last_clean":"2024-09-16T15:39:34.164871+0000","last_became_active":"2024-09-16T15:39:33.167382+0000","last_became_peered":"2024-09-16T15:39:33.167382+0000","last_unstale":"2024-09-16T15:39:34.164871+0000","last_undegraded":"2024-09-16T15:39:34.164871+0000","last_fullsized":"2024-09-16T15:39:34.164871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:07:01.249223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166107+0000","last_change":"2024-09-16T15:39:32.163172+0000","last_active":"2024-09-16T15:39:32.166107+0000","last_peered":"2024-09-16T15:39:32.166107+0000","last_clean":"2024-09-16T15:39:32.166107+0000","last_became_active":"2024-09-16T15:39:32.162626+0000","last_became_peered":"2024-09-16T15:39:32.162626+0000","last_unstale":"2024-09-16T15:39:32.166107+0000","last_undegraded":"2024-09-16T15:39:32.166107+0000","last_fullsized":"2024-09-16T15:39:32.166107+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:49.229870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154053+0000","last_change":"2024-09-16T15:39:32.154053+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154053+0000","last_undegraded":"2024-09-16T15:39:32.154053+0000","last_fullsized":"2024-09-16T15:39:32.154053+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164762+0000","last_change":"2024-09-16T15:39:32.162290+0000","last_active":"2024-09-16T15:39:34.164762+0000","last_peered":"2024-09-16T15:39:34.164762+0000","last_clean":"2024-09-16T15:39:34.164762+0000","last_became_active":"2024-09-16T15:39:32.162042+0000","last_became_peered":"2024-09-16T15:39:32.162042+0000","last_unstale":"2024-09-16T15:39:34.164762+0000","last_undegraded":"2024-09-16T15:39:34.164762+0000","last_fullsized":"2024-09-16T15:39:34.164762+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:23.717646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.163921+0000","last_change":"2024-09-16T15:39:32.163921+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.163921+0000","last_undegraded":"2024-09-16T15:39:32.163921+0000","last_fullsized":"2024-09-16T15:39:32.163921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171868+0000","last_change":"2024-09-16T15:39:32.169367+0000","last_active":"2024-09-16T15:39:32.171868+0000","last_peered":"2024-09-16T15:39:32.171868+0000","last_clean":"2024-09-16T15:39:32.171868+0000","last_became_active":"2024-09-16T15:39:32.169188+0000","last_became_peered":"2024-09-16T15:39:32.169188+0000","last_unstale":"2024-09-16T15:39:32.171868+0000","last_undegraded":"2024-09-16T15:39:32.171868+0000","last_fullsized":"2024-09-16T15:39:32.171868+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:14.156509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151486+0000","last_change":"2024-09-16T15:39:32.151486+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151486+0000","last_undegraded":"2024-09-16T15:39:32.151486+0000","last_fullsized":"2024-09-16T15:39:32.151486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.168920+0000","last_change":"2024-09-16T15:39:32.163551+0000","last_active":"2024-09-16T15:39:32.168920+0000","last_peered":"2024-09-16T15:39:32.168920+0000","last_clean":"2024-09-16T15:39:32.168920+0000","last_became_active":"2024-09-16T15:39:32.163283+0000","last_became_peered":"2024-09-16T15:39:32.163283+0000","last_unstale":"2024-09-16T15:39:32.168920+0000","last_undegraded":"2024-09-16T15:39:32.168920+0000","last_fullsized":"2024-09-16T15:39:32.168920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:54.716352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151031+0000","last_change":"2024-09-16T15:39:32.151031+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151031+0000","last_undegraded":"2024-09-16T15:39:32.151031+0000","last_fullsized":"2024-09-16T15:39:32.151031+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165577+0000","last_change":"2024-09-16T15:39:32.161394+0000","last_active":"2024-09-16T15:39:34.165577+0000","last_peered":"2024-09-16T15:39:34.165577+0000","last_clean":"2024-09-16T15:39:34.165577+0000","last_became_active":"2024-09-16T15:39:32.161124+0000","last_became_peered":"2024-09-16T15:39:32.161124+0000","last_unstale":"2024-09-16T15:39:34.165577+0000","last_undegraded":"2024-09-16T15:39:34.165577+0000","last_fullsized":"2024-09-16T15:39:34.165577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:40.912077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.157317+0000","last_change":"2024-09-16T15:39:32.157317+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.157317+0000","last_undegraded":"2024-09-16T15:39:32.157317+0000","last_fullsized":"2024-09-16T15:39:32.157317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166121+0000","last_change":"2024-09-16T15:39:32.163386+0000","last_active":"2024-09-16T15:39:32.166121+0000","last_peered":"2024-09-16T15:39:32.166121+0000","last_clean":"2024-09-16T15:39:32.166121+0000","last_became_active":"2024-09-16T15:39:32.161214+0000","last_became_peered":"2024-09-16T15:39:32.161214+0000","last_unstale":"2024-09-16T15:39:32.166121+0000","last_undegraded":"2024-09-16T15:39:32.166121+0000","last_fullsized":"2024-09-16T15:39:32.166121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:34.093301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165551+0000","last_change":"2024-09-16T15:39:33.175372+0000","last_active":"2024-09-16T15:39:34.165551+0000","last_peered":"2024-09-16T15:39:34.165551+0000","last_clean":"2024-09-16T15:39:34.165551+0000","last_became_active":"2024-09-16T15:39:33.174198+0000","last_became_peered":"2024-09-16T15:39:33.174198+0000","last_unstale":"2024-09-16T15:39:34.165551+0000","last_undegraded":"2024-09-16T15:39:34.165551+0000","last_fullsized":"2024-09-16T15:39:34.165551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:53.410594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.169802+0000","last_change":"2024-09-16T15:39:32.166140+0000","last_active":"2024-09-16T15:39:32.169802+0000","last_peered":"2024-09-16T15:39:32.169802+0000","last_clean":"2024-09-16T15:39:32.169802+0000","last_became_active":"2024-09-16T15:39:32.165774+0000","last_became_peered":"2024-09-16T15:39:32.165774+0000","last_unstale":"2024-09-16T15:39:32.169802+0000","last_undegraded":"2024-09-16T15:39:32.169802+0000","last_fullsized":"2024-09-16T15:39:32.169802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:48.317883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151377+0000","last_change":"2024-09-16T15:39:32.151377+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151377+0000","last_undegraded":"2024-09-16T15:39:32.151377+0000","last_fullsized":"2024-09-16T15:39:32.151377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165944+0000","last_change":"2024-09-16T15:39:32.162492+0000","last_active":"2024-09-16T15:39:32.165944+0000","last_peered":"2024-09-16T15:39:32.165944+0000","last_clean":"2024-09-16T15:39:32.165944+0000","last_became_active":"2024-09-16T15:39:32.162229+0000","last_became_peered":"2024-09-16T15:39:32.162229+0000","last_unstale":"2024-09-16T15:39:32.165944+0000","last_undegraded":"2024-09-16T15:39:32.165944+0000","last_fullsized":"2024-09-16T15:39:32.165944+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:19.223491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150686+0000","last_change":"2024-09-16T15:39:32.150686+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150686+0000","last_undegraded":"2024-09-16T15:39:32.150686+0000","last_fullsized":"2024-09-16T15:39:32.150686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.164987+0000","last_change":"2024-09-16T15:39:32.161540+0000","last_active":"2024-09-16T15:39:32.164987+0000","last_peered":"2024-09-16T15:39:32.164987+0000","last_clean":"2024-09-16T15:39:32.164987+0000","last_became_active":"2024-09-16T15:39:32.161225+0000","last_became_peered":"2024-09-16T15:39:32.161225+0000","last_unstale":"2024-09-16T15:39:32.164987+0000","last_undegraded":"2024-09-16T15:39:32.164987+0000","last_fullsized":"2024-09-16T15:39:32.164987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:21:04.290055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151288+0000","last_change":"2024-09-16T15:39:32.151288+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151288+0000","last_undegraded":"2024-09-16T15:39:32.151288+0000","last_fullsized":"2024-09-16T15:39:32.151288+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.160197+0000","last_change":"2024-09-16T15:39:32.157654+0000","last_active":"2024-09-16T15:39:32.160197+0000","last_peered":"2024-09-16T15:39:32.160197+0000","last_clean":"2024-09-16T15:39:32.160197+0000","last_became_active":"2024-09-16T15:39:32.157393+0000","last_became_peered":"2024-09-16T15:39:32.157393+0000","last_unstale":"2024-09-16T15:39:32.160197+0000","last_undegraded":"2024-09-16T15:39:32.160197+0000","last_fullsized":"2024-09-16T15:39:32.160197+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:07.931223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.154817+0000","last_change":"2024-09-16T15:39:32.154817+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.154817+0000","last_undegraded":"2024-09-16T15:39:32.154817+0000","last_fullsized":"2024-09-16T15:39:32.154817+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.166535+0000","last_change":"2024-09-16T15:39:32.162100+0000","last_active":"2024-09-16T15:39:32.166535+0000","last_peered":"2024-09-16T15:39:32.166535+0000","last_clean":"2024-09-16T15:39:32.166535+0000","last_became_active":"2024-09-16T15:39:32.161782+0000","last_became_peered":"2024-09-16T15:39:32.161782+0000","last_unstale":"2024-09-16T15:39:32.166535+0000","last_undegraded":"2024-09-16T15:39:32.166535+0000","last_fullsized":"2024-09-16T15:39:32.166535+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:51:50.007487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.161447+0000","last_change":"2024-09-16T15:39:32.161447+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.161447+0000","last_undegraded":"2024-09-16T15:39:32.161447+0000","last_fullsized":"2024-09-16T15:39:32.161447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172551+0000","last_change":"2024-09-16T15:39:32.168491+0000","last_active":"2024-09-16T15:39:32.172551+0000","last_peered":"2024-09-16T15:39:32.172551+0000","last_clean":"2024-09-16T15:39:32.172551+0000","last_became_active":"2024-09-16T15:39:32.168195+0000","last_became_peered":"2024-09-16T15:39:32.168195+0000","last_unstale":"2024-09-16T15:39:32.172551+0000","last_undegraded":"2024-09-16T15:39:32.172551+0000","last_fullsized":"2024-09-16T15:39:32.172551+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:00.428705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150369+0000","last_change":"2024-09-16T15:39:32.150369+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150369+0000","last_undegraded":"2024-09-16T15:39:32.150369+0000","last_fullsized":"2024-09-16T15:39:32.150369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165670+0000","last_change":"2024-09-16T15:39:32.166439+0000","last_active":"2024-09-16T15:39:34.165670+0000","last_peered":"2024-09-16T15:39:34.165670+0000","last_clean":"2024-09-16T15:39:34.165670+0000","last_became_active":"2024-09-16T15:39:32.166175+0000","last_became_peered":"2024-09-16T15:39:32.166175+0000","last_unstale":"2024-09-16T15:39:34.165670+0000","last_undegraded":"2024-09-16T15:39:34.165670+0000","last_fullsized":"2024-09-16T15:39:34.165670+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:39.246388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.156915+0000","last_change":"2024-09-16T15:39:32.156915+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.156915+0000","last_undegraded":"2024-09-16T15:39:32.156915+0000","last_fullsized":"2024-09-16T15:39:32.156915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.158697+0000","last_change":"2024-09-16T15:39:32.155559+0000","last_active":"2024-09-16T15:39:32.158697+0000","last_peered":"2024-09-16T15:39:32.158697+0000","last_clean":"2024-09-16T15:39:32.158697+0000","last_became_active":"2024-09-16T15:39:32.155286+0000","last_became_peered":"2024-09-16T15:39:32.155286+0000","last_unstale":"2024-09-16T15:39:32.158697+0000","last_undegraded":"2024-09-16T15:39:32.158697+0000","last_fullsized":"2024-09-16T15:39:32.158697+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:53:37.151552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150242+0000","last_change":"2024-09-16T15:39:32.150242+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150242+0000","last_undegraded":"2024-09-16T15:39:32.150242+0000","last_fullsized":"2024-09-16T15:39:32.150242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165430+0000","last_change":"2024-09-16T15:39:32.172865+0000","last_active":"2024-09-16T15:39:34.165430+0000","last_peered":"2024-09-16T15:39:34.165430+0000","last_clean":"2024-09-16T15:39:34.165430+0000","last_became_active":"2024-09-16T15:39:32.172616+0000","last_became_peered":"2024-09-16T15:39:32.172616+0000","last_unstale":"2024-09-16T15:39:34.165430+0000","last_undegraded":"2024-09-16T15:39:34.165430+0000","last_fullsized":"2024-09-16T15:39:34.165430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:38.115059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.151198+0000","last_change":"2024-09-16T15:39:32.151198+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.151198+0000","last_undegraded":"2024-09-16T15:39:32.151198+0000","last_fullsized":"2024-09-16T15:39:32.151198+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.165849+0000","last_change":"2024-09-16T15:39:32.164165+0000","last_active":"2024-09-16T15:39:32.165849+0000","last_peered":"2024-09-16T15:39:32.165849+0000","last_clean":"2024-09-16T15:39:32.165849+0000","last_became_active":"2024-09-16T15:39:32.163728+0000","last_became_peered":"2024-09-16T15:39:32.163728+0000","last_unstale":"2024-09-16T15:39:32.165849+0000","last_undegraded":"2024-09-16T15:39:32.165849+0000","last_fullsized":"2024-09-16T15:39:32.165849+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:07.235894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.150024+0000","last_change":"2024-09-16T15:39:32.150024+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.150024+0000","last_undegraded":"2024-09-16T15:39:32.150024+0000","last_fullsized":"2024-09-16T15:39:32.150024+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165133+0000","last_change":"2024-09-16T15:39:32.167828+0000","last_active":"2024-09-16T15:39:34.165133+0000","last_peered":"2024-09-16T15:39:34.165133+0000","last_clean":"2024-09-16T15:39:34.165133+0000","last_became_active":"2024-09-16T15:39:32.165651+0000","last_became_peered":"2024-09-16T15:39:32.165651+0000","last_unstale":"2024-09-16T15:39:34.165133+0000","last_undegraded":"2024-09-16T15:39:34.165133+0000","last_fullsized":"2024-09-16T15:39:34.165133+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:29.486963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.152940+0000","last_change":"2024-09-16T15:39:32.152940+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.152940+0000","last_undegraded":"2024-09-16T15:39:32.152940+0000","last_fullsized":"2024-09-16T15:39:32.152940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.172404+0000","last_change":"2024-09-16T15:39:32.169431+0000","last_active":"2024-09-16T15:39:32.172404+0000","last_peered":"2024-09-16T15:39:32.172404+0000","last_clean":"2024-09-16T15:39:32.172404+0000","last_became_active":"2024-09-16T15:39:32.169166+0000","last_became_peered":"2024-09-16T15:39:32.169166+0000","last_unstale":"2024-09-16T15:39:32.172404+0000","last_undegraded":"2024-09-16T15:39:32.172404+0000","last_fullsized":"2024-09-16T15:39:32.172404+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:12.103417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.153357+0000","last_change":"2024-09-16T15:39:32.153357+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.153357+0000","last_undegraded":"2024-09-16T15:39:32.153357+0000","last_fullsized":"2024-09-16T15:39:32.153357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.171856+0000","last_change":"2024-09-16T15:39:32.170091+0000","last_active":"2024-09-16T15:39:32.171856+0000","last_peered":"2024-09-16T15:39:32.171856+0000","last_clean":"2024-09-16T15:39:32.171856+0000","last_became_active":"2024-09-16T15:39:32.169831+0000","last_became_peered":"2024-09-16T15:39:32.169831+0000","last_unstale":"2024-09-16T15:39:32.171856+0000","last_undegraded":"2024-09-16T15:39:32.171856+0000","last_fullsized":"2024-09-16T15:39:32.171856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:18.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-09-16T15:39:32.159832+0000","last_change":"2024-09-16T15:39:32.159832+0000","last_active":"2024-09-16T15:39:32.138319+0000","last_peered":"2024-09-16T15:39:32.138319+0000","last_clean":"2024-09-16T15:39:32.138319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:32.159832+0000","last_undegraded":"2024-09-16T15:39:32.159832+0000","last_fullsized":"2024-09-16T15:39:32.159832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167962+0000","last_change":"2024-09-16T15:39:32.163553+0000","last_active":"2024-09-16T15:39:32.167962+0000","last_peered":"2024-09-16T15:39:32.167962+0000","last_clean":"2024-09-16T15:39:32.167962+0000","last_became_active":"2024-09-16T15:39:32.162649+0000","last_became_peered":"2024-09-16T15:39:32.162649+0000","last_unstale":"2024-09-16T15:39:32.167962+0000","last_undegraded":"2024-09-16T15:39:32.167962+0000","last_fullsized":"2024-09-16T15:39:32.167962+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:12.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":131,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T15:39:32.167970+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:39:32.167970+0000","last_peered":"2024-09-16T15:39:32.167970+0000","last_clean":"2024-09-16T15:39:32.167970+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:39:32.167970+0000","last_undegraded":"2024-09-16T15:39:32.167970+0000","last_fullsized":"2024-09-16T15:39:32.167970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":734366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":376832,"data_stored":143998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":970752,"data_stored":734366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":417792,"data_stored":158707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:42.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.324+0000 7fee8a600640 1 -- 172.21.15.60:0/1018215029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 msgr2=0x7fee840b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.324+0000 7fee8a600640 1 --2- 172.21.15.60:0/1018215029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 0x7fee840b2440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee70005fd0 tx=0x7fee7002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.324+0000 7fee8a600640 1 -- 172.21.15.60:0/1018215029 shutdown_connections 2024-09-16T15:39:42.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.324+0000 7fee8a600640 1 --2- 172.21.15.60:0/1018215029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 0x7fee840b2440 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.324+0000 7fee8a600640 1 -- 172.21.15.60:0/1018215029 >> 172.21.15.60:0/1018215029 conn(0x7fee840b1950 msgr2=0x7fee840b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.324+0000 7fee8a600640 1 -- 172.21.15.60:0/1018215029 shutdown_connections 2024-09-16T15:39:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.324+0000 7fee8a600640 1 -- 172.21.15.60:0/1018215029 wait complete. 2024-09-16T15:39:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.325+0000 7fee8a600640 1 Processor -- start 2024-09-16T15:39:42.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.325+0000 7fee8a600640 1 -- start start 2024-09-16T15:39:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.326+0000 7fee8a600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee84073fb0 0x7fee84147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.326+0000 7fee8a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 0x7fee84145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.326+0000 7fee8a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee8415f5a0 0x7fee84146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.326+0000 7fee8a600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fee8406f9f0 con 0x7fee840ccf00 2024-09-16T15:39:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.326+0000 7fee8a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee8406f870 con 0x7fee8415f5a0 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.326+0000 7fee8a600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fee84074420 con 0x7fee84073fb0 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 0x7fee84145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 0x7fee84145950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57054/0 (socket says 172.21.15.60:57054) 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 -- 172.21.15.60:0/1063684913 learned_addr learned my addr 172.21.15.60:0/1063684913 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 -- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee84073fb0 msgr2=0x7fee84147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee84073fb0 0x7fee84147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 -- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee8415f5a0 msgr2=0x7fee84146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee8415f5a0 0x7fee84146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 -- 172.21.15.60:0/1063684913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee84146bf0 con 0x7fee840ccf00 2024-09-16T15:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.327+0000 7fee82a00640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 0x7fee84145950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee70006010 tx=0x7fee700039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.328+0000 7fee7b400640 1 -- 172.21.15.60:0/1063684913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7003f070 con 0x7fee840ccf00 2024-09-16T15:39:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.328+0000 7fee7b400640 1 -- 172.21.15.60:0/1063684913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee70004890 con 0x7fee840ccf00 2024-09-16T15:39:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.328+0000 7fee7b400640 1 -- 172.21.15.60:0/1063684913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70035030 con 0x7fee840ccf00 2024-09-16T15:39:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.328+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee84146ed0 con 0x7fee840ccf00 2024-09-16T15:39:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.328+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee84147490 con 0x7fee840ccf00 2024-09-16T15:39:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.329+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee840b2440 con 0x7fee840ccf00 2024-09-16T15:39:42.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.330+0000 7fee7b400640 1 -- 172.21.15.60:0/1063684913 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fee70042070 con 0x7fee840ccf00 2024-09-16T15:39:42.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.331+0000 7fee7b400640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee64078880 0x7fee6407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:42.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.331+0000 7fee7b400640 1 -- 172.21.15.60:0/1063684913 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fee700bb850 con 0x7fee840ccf00 2024-09-16T15:39:42.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.331+0000 7fee83400640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee64078880 0x7fee6407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:42.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.332+0000 7fee83400640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee64078880 0x7fee6407ad40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee74001550 tx=0x7fee74007970 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:42.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.335+0000 7fee7b400640 1 -- 172.21.15.60:0/1063684913 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fee70083eb0 con 0x7fee840ccf00 2024-09-16T15:39:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.527+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee840afa30 con 0x7fee64078880 2024-09-16T15:39:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.532+0000 7fee7b400640 1 -- 172.21.15.60:0/1063684913 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7fee840afa30 con 0x7fee64078880 2024-09-16T15:39:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:42.537 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:39:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.535+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee64078880 msgr2=0x7fee6407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee64078880 0x7fee6407ad40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee74001550 tx=0x7fee74007970 comp rx=0 tx=0).stop 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 msgr2=0x7fee84145950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 0x7fee84145950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee70006010 tx=0x7fee700039e0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 shutdown_connections 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee64078880 0x7fee6407ad40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee8415f5a0 0x7fee84146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee840ccf00 0x7fee84145950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 --2- 172.21.15.60:0/1063684913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee84073fb0 0x7fee84147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 >> 172.21.15.60:0/1063684913 conn(0x7fee840b1950 msgr2=0x7fee840b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 shutdown_connections 2024-09-16T15:39:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.536+0000 7fee8a600640 1 -- 172.21.15.60:0/1063684913 wait complete. 2024-09-16T15:39:42.553 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-16T15:39:41.724488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237248,"kb_used_data":4360,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737472,"statfs":{"total":773094113280,"available":772851171328,"internally_reserved":0,"allocated":4464640,"data_stored":2442140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.576246"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.168043+0000","last_change":"2024-09-16T15:39:33.165877+0000","last_active":"2024-09-16T15:39:34.168043+0000","last_peered":"2024-09-16T15:39:34.168043+0000","last_clean":"2024-09-16T15:39:34.168043+0000","last_became_active":"2024-09-16T15:39:33.165442+0000","last_became_peered":"2024-09-16T15:39:33.165442+0000","last_unstale":"2024-09-16T15:39:34.168043+0000","last_undegraded":"2024-09-16T15:39:34.168043+0000","last_fullsized":"2024-09-16T15:39:34.168043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:59.810876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301341+0000","last_change":"2024-09-16T15:39:33.163845+0000","last_active":"2024-09-16T15:39:34.301341+0000","last_peered":"2024-09-16T15:39:34.301341+0000","last_clean":"2024-09-16T15:39:34.301341+0000","last_became_active":"2024-09-16T15:39:33.162074+0000","last_became_peered":"2024-09-16T15:39:33.162074+0000","last_unstale":"2024-09-16T15:39:34.301341+0000","last_undegraded":"2024-09-16T15:39:34.301341+0000","last_fullsized":"2024-09-16T15:39:34.301341+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:33.501333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.166136+0000","last_change":"2024-09-16T15:39:33.181217+0000","last_active":"2024-09-16T15:39:34.166136+0000","last_peered":"2024-09-16T15:39:34.166136+0000","last_clean":"2024-09-16T15:39:34.166136+0000","last_became_active":"2024-09-16T15:39:33.180921+0000","last_became_peered":"2024-09-16T15:39:33.180921+0000","last_unstale":"2024-09-16T15:39:34.166136+0000","last_undegraded":"2024-09-16T15:39:34.166136+0000","last_fullsized":"2024-09-16T15:39:34.166136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:55.917732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165367+0000","last_change":"2024-09-16T15:39:33.175477+0000","last_active":"2024-09-16T15:39:34.165367+0000","last_peered":"2024-09-16T15:39:34.165367+0000","last_clean":"2024-09-16T15:39:34.165367+0000","last_became_active":"2024-09-16T15:39:33.174529+0000","last_became_peered":"2024-09-16T15:39:33.174529+0000","last_unstale":"2024-09-16T15:39:34.165367+0000","last_undegraded":"2024-09-16T15:39:34.165367+0000","last_fullsized":"2024-09-16T15:39:34.165367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:16.836397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722409+0000","last_change":"2024-09-16T15:39:33.171804+0000","last_active":"2024-09-16T15:39:34.722409+0000","last_peered":"2024-09-16T15:39:34.722409+0000","last_clean":"2024-09-16T15:39:34.722409+0000","last_became_active":"2024-09-16T15:39:33.171488+0000","last_became_peered":"2024-09-16T15:39:33.171488+0000","last_unstale":"2024-09-16T15:39:34.722409+0000","last_undegraded":"2024-09-16T15:39:34.722409+0000","last_fullsized":"2024-09-16T15:39:34.722409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:17.869388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.300826+0000","last_change":"2024-09-16T15:39:33.171565+0000","last_active":"2024-09-16T15:39:34.300826+0000","last_peered":"2024-09-16T15:39:34.300826+0000","last_clean":"2024-09-16T15:39:34.300826+0000","last_became_active":"2024-09-16T15:39:33.169542+0000","last_became_peered":"2024-09-16T15:39:33.169542+0000","last_unstale":"2024-09-16T15:39:34.300826+0000","last_undegraded":"2024-09-16T15:39:34.300826+0000","last_fullsized":"2024-09-16T15:39:34.300826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:15.096028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723548+0000","last_change":"2024-09-16T15:39:33.163317+0000","last_active":"2024-09-16T15:39:34.723548+0000","last_peered":"2024-09-16T15:39:34.723548+0000","last_clean":"2024-09-16T15:39:34.723548+0000","last_became_active":"2024-09-16T15:39:33.162709+0000","last_became_peered":"2024-09-16T15:39:33.162709+0000","last_unstale":"2024-09-16T15:39:34.723548+0000","last_undegraded":"2024-09-16T15:39:34.723548+0000","last_fullsized":"2024-09-16T15:39:34.723548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:10.414778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723070+0000","last_change":"2024-09-16T15:39:33.177916+0000","last_active":"2024-09-16T15:39:34.723070+0000","last_peered":"2024-09-16T15:39:34.723070+0000","last_clean":"2024-09-16T15:39:34.723070+0000","last_became_active":"2024-09-16T15:39:33.177659+0000","last_became_peered":"2024-09-16T15:39:33.177659+0000","last_unstale":"2024-09-16T15:39:34.723070+0000","last_undegraded":"2024-09-16T15:39:34.723070+0000","last_fullsized":"2024-09-16T15:39:34.723070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:45.666047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724622+0000","last_change":"2024-09-16T15:39:33.181204+0000","last_active":"2024-09-16T15:39:34.724622+0000","last_peered":"2024-09-16T15:39:34.724622+0000","last_clean":"2024-09-16T15:39:34.724622+0000","last_became_active":"2024-09-16T15:39:33.180935+0000","last_became_peered":"2024-09-16T15:39:33.180935+0000","last_unstale":"2024-09-16T15:39:34.724622+0000","last_undegraded":"2024-09-16T15:39:34.724622+0000","last_fullsized":"2024-09-16T15:39:34.724622+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:25.525836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165700+0000","last_change":"2024-09-16T15:39:33.177863+0000","last_active":"2024-09-16T15:39:34.165700+0000","last_peered":"2024-09-16T15:39:34.165700+0000","last_clean":"2024-09-16T15:39:34.165700+0000","last_became_active":"2024-09-16T15:39:33.176375+0000","last_became_peered":"2024-09-16T15:39:33.176375+0000","last_unstale":"2024-09-16T15:39:34.165700+0000","last_undegraded":"2024-09-16T15:39:34.165700+0000","last_fullsized":"2024-09-16T15:39:34.165700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:10.080289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723678+0000","last_change":"2024-09-16T15:39:33.177412+0000","last_active":"2024-09-16T15:39:34.723678+0000","last_peered":"2024-09-16T15:39:34.723678+0000","last_clean":"2024-09-16T15:39:34.723678+0000","last_became_active":"2024-09-16T15:39:33.176972+0000","last_became_peered":"2024-09-16T15:39:33.176972+0000","last_unstale":"2024-09-16T15:39:34.723678+0000","last_undegraded":"2024-09-16T15:39:34.723678+0000","last_fullsized":"2024-09-16T15:39:34.723678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:55.917732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.166048+0000","last_change":"2024-09-16T15:39:33.173080+0000","last_active":"2024-09-16T15:39:34.166048+0000","last_peered":"2024-09-16T15:39:34.166048+0000","last_clean":"2024-09-16T15:39:34.166048+0000","last_became_active":"2024-09-16T15:39:33.172778+0000","last_became_peered":"2024-09-16T15:39:33.172778+0000","last_unstale":"2024-09-16T15:39:34.166048+0000","last_undegraded":"2024-09-16T15:39:34.166048+0000","last_fullsized":"2024-09-16T15:39:34.166048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:10.414778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722555+0000","last_change":"2024-09-16T15:39:33.177072+0000","last_active":"2024-09-16T15:39:34.722555+0000","last_peered":"2024-09-16T15:39:34.722555+0000","last_clean":"2024-09-16T15:39:34.722555+0000","last_became_active":"2024-09-16T15:39:33.176814+0000","last_became_peered":"2024-09-16T15:39:33.176814+0000","last_unstale":"2024-09-16T15:39:34.722555+0000","last_undegraded":"2024-09-16T15:39:34.722555+0000","last_fullsized":"2024-09-16T15:39:34.722555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:38.755957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.300845+0000","last_change":"2024-09-16T15:39:33.181131+0000","last_active":"2024-09-16T15:39:34.300845+0000","last_peered":"2024-09-16T15:39:34.300845+0000","last_clean":"2024-09-16T15:39:34.300845+0000","last_became_active":"2024-09-16T15:39:33.180786+0000","last_became_peered":"2024-09-16T15:39:33.180786+0000","last_unstale":"2024-09-16T15:39:34.300845+0000","last_undegraded":"2024-09-16T15:39:34.300845+0000","last_fullsized":"2024-09-16T15:39:34.300845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:04:21.991861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165844+0000","last_change":"2024-09-16T15:39:33.169271+0000","last_active":"2024-09-16T15:39:34.165844+0000","last_peered":"2024-09-16T15:39:34.165844+0000","last_clean":"2024-09-16T15:39:34.165844+0000","last_became_active":"2024-09-16T15:39:33.168975+0000","last_became_peered":"2024-09-16T15:39:33.168975+0000","last_unstale":"2024-09-16T15:39:34.165844+0000","last_undegraded":"2024-09-16T15:39:34.165844+0000","last_fullsized":"2024-09-16T15:39:34.165844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:23.664230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723956+0000","last_change":"2024-09-16T15:39:33.169063+0000","last_active":"2024-09-16T15:39:34.723956+0000","last_peered":"2024-09-16T15:39:34.723956+0000","last_clean":"2024-09-16T15:39:34.723956+0000","last_became_active":"2024-09-16T15:39:33.168776+0000","last_became_peered":"2024-09-16T15:39:33.168776+0000","last_unstale":"2024-09-16T15:39:34.723956+0000","last_undegraded":"2024-09-16T15:39:34.723956+0000","last_fullsized":"2024-09-16T15:39:34.723956+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:50.413223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724596+0000","last_change":"2024-09-16T15:39:33.162971+0000","last_active":"2024-09-16T15:39:34.724596+0000","last_peered":"2024-09-16T15:39:34.724596+0000","last_clean":"2024-09-16T15:39:34.724596+0000","last_became_active":"2024-09-16T15:39:33.162688+0000","last_became_peered":"2024-09-16T15:39:33.162688+0000","last_unstale":"2024-09-16T15:39:34.724596+0000","last_undegraded":"2024-09-16T15:39:34.724596+0000","last_fullsized":"2024-09-16T15:39:34.724596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:00.068097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301332+0000","last_change":"2024-09-16T15:39:33.181419+0000","last_active":"2024-09-16T15:39:34.301332+0000","last_peered":"2024-09-16T15:39:34.301332+0000","last_clean":"2024-09-16T15:39:34.301332+0000","last_became_active":"2024-09-16T15:39:33.180826+0000","last_became_peered":"2024-09-16T15:39:33.180826+0000","last_unstale":"2024-09-16T15:39:34.301332+0000","last_undegraded":"2024-09-16T15:39:34.301332+0000","last_fullsized":"2024-09-16T15:39:34.301332+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:07:36.472178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.300975+0000","last_change":"2024-09-16T15:39:33.169716+0000","last_active":"2024-09-16T15:39:34.300975+0000","last_peered":"2024-09-16T15:39:34.300975+0000","last_clean":"2024-09-16T15:39:34.300975+0000","last_became_active":"2024-09-16T15:39:33.169381+0000","last_became_peered":"2024-09-16T15:39:33.169381+0000","last_unstale":"2024-09-16T15:39:34.300975+0000","last_undegraded":"2024-09-16T15:39:34.300975+0000","last_fullsized":"2024-09-16T15:39:34.300975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:50.413223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722606+0000","last_change":"2024-09-16T15:39:33.178707+0000","last_active":"2024-09-16T15:39:34.722606+0000","last_peered":"2024-09-16T15:39:34.722606+0000","last_clean":"2024-09-16T15:39:34.722606+0000","last_became_active":"2024-09-16T15:39:33.178428+0000","last_became_peered":"2024-09-16T15:39:33.178428+0000","last_unstale":"2024-09-16T15:39:34.722606+0000","last_undegraded":"2024-09-16T15:39:34.722606+0000","last_fullsized":"2024-09-16T15:39:34.722606+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:03.256843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723610+0000","last_change":"2024-09-16T15:39:33.175002+0000","last_active":"2024-09-16T15:39:34.723610+0000","last_peered":"2024-09-16T15:39:34.723610+0000","last_clean":"2024-09-16T15:39:34.723610+0000","last_became_active":"2024-09-16T15:39:33.174772+0000","last_became_peered":"2024-09-16T15:39:33.174772+0000","last_unstale":"2024-09-16T15:39:34.723610+0000","last_undegraded":"2024-09-16T15:39:34.723610+0000","last_fullsized":"2024-09-16T15:39:34.723610+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:46.063293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723852+0000","last_change":"2024-09-16T15:39:33.177521+0000","last_active":"2024-09-16T15:39:34.723852+0000","last_peered":"2024-09-16T15:39:34.723852+0000","last_clean":"2024-09-16T15:39:34.723852+0000","last_became_active":"2024-09-16T15:39:33.177066+0000","last_became_peered":"2024-09-16T15:39:33.177066+0000","last_unstale":"2024-09-16T15:39:34.723852+0000","last_undegraded":"2024-09-16T15:39:34.723852+0000","last_fullsized":"2024-09-16T15:39:34.723852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:45.666047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235374+0000","last_change":"2024-09-16T15:39:33.175156+0000","last_active":"2024-09-16T15:39:34.235374+0000","last_peered":"2024-09-16T15:39:34.235374+0000","last_clean":"2024-09-16T15:39:34.235374+0000","last_became_active":"2024-09-16T15:39:33.174840+0000","last_became_peered":"2024-09-16T15:39:33.174840+0000","last_unstale":"2024-09-16T15:39:34.235374+0000","last_undegraded":"2024-09-16T15:39:34.235374+0000","last_fullsized":"2024-09-16T15:39:34.235374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:36.985244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165822+0000","last_change":"2024-09-16T15:39:33.180553+0000","last_active":"2024-09-16T15:39:34.165822+0000","last_peered":"2024-09-16T15:39:34.165822+0000","last_clean":"2024-09-16T15:39:34.165822+0000","last_became_active":"2024-09-16T15:39:33.180145+0000","last_became_peered":"2024-09-16T15:39:33.180145+0000","last_unstale":"2024-09-16T15:39:34.165822+0000","last_undegraded":"2024-09-16T15:39:34.165822+0000","last_fullsized":"2024-09-16T15:39:34.165822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:25.525836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165596+0000","last_change":"2024-09-16T15:39:33.178266+0000","last_active":"2024-09-16T15:39:34.165596+0000","last_peered":"2024-09-16T15:39:34.165596+0000","last_clean":"2024-09-16T15:39:34.165596+0000","last_became_active":"2024-09-16T15:39:33.177230+0000","last_became_peered":"2024-09-16T15:39:33.177230+0000","last_unstale":"2024-09-16T15:39:34.165596+0000","last_undegraded":"2024-09-16T15:39:34.165596+0000","last_fullsized":"2024-09-16T15:39:34.165596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:59.832311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301532+0000","last_change":"2024-09-16T15:39:33.171239+0000","last_active":"2024-09-16T15:39:34.301532+0000","last_peered":"2024-09-16T15:39:34.301532+0000","last_clean":"2024-09-16T15:39:34.301532+0000","last_became_active":"2024-09-16T15:39:33.170909+0000","last_became_peered":"2024-09-16T15:39:33.170909+0000","last_unstale":"2024-09-16T15:39:34.301532+0000","last_undegraded":"2024-09-16T15:39:34.301532+0000","last_fullsized":"2024-09-16T15:39:34.301532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:55.034002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165652+0000","last_change":"2024-09-16T15:39:33.176217+0000","last_active":"2024-09-16T15:39:34.165652+0000","last_peered":"2024-09-16T15:39:34.165652+0000","last_clean":"2024-09-16T15:39:34.165652+0000","last_became_active":"2024-09-16T15:39:33.175855+0000","last_became_peered":"2024-09-16T15:39:33.175855+0000","last_unstale":"2024-09-16T15:39:34.165652+0000","last_undegraded":"2024-09-16T15:39:34.165652+0000","last_fullsized":"2024-09-16T15:39:34.165652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:45.876870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.166311+0000","last_change":"2024-09-16T15:39:33.172626+0000","last_active":"2024-09-16T15:39:34.166311+0000","last_peered":"2024-09-16T15:39:34.166311+0000","last_clean":"2024-09-16T15:39:34.166311+0000","last_became_active":"2024-09-16T15:39:33.172206+0000","last_became_peered":"2024-09-16T15:39:33.172206+0000","last_unstale":"2024-09-16T15:39:34.166311+0000","last_undegraded":"2024-09-16T15:39:34.166311+0000","last_fullsized":"2024-09-16T15:39:34.166311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:15.096028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723845+0000","last_change":"2024-09-16T15:39:33.182189+0000","last_active":"2024-09-16T15:39:34.723845+0000","last_peered":"2024-09-16T15:39:34.723845+0000","last_clean":"2024-09-16T15:39:34.723845+0000","last_became_active":"2024-09-16T15:39:33.181874+0000","last_became_peered":"2024-09-16T15:39:33.181874+0000","last_unstale":"2024-09-16T15:39:34.723845+0000","last_undegraded":"2024-09-16T15:39:34.723845+0000","last_fullsized":"2024-09-16T15:39:34.723845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:51.244587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723759+0000","last_change":"2024-09-16T15:39:33.163763+0000","last_active":"2024-09-16T15:39:34.723759+0000","last_peered":"2024-09-16T15:39:34.723759+0000","last_clean":"2024-09-16T15:39:34.723759+0000","last_became_active":"2024-09-16T15:39:33.163501+0000","last_became_peered":"2024-09-16T15:39:33.163501+0000","last_unstale":"2024-09-16T15:39:34.723759+0000","last_undegraded":"2024-09-16T15:39:34.723759+0000","last_fullsized":"2024-09-16T15:39:34.723759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:15.096028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.167782+0000","last_change":"2024-09-16T15:39:33.172839+0000","last_active":"2024-09-16T15:39:34.167782+0000","last_peered":"2024-09-16T15:39:34.167782+0000","last_clean":"2024-09-16T15:39:34.167782+0000","last_became_active":"2024-09-16T15:39:33.172444+0000","last_became_peered":"2024-09-16T15:39:33.172444+0000","last_unstale":"2024-09-16T15:39:34.167782+0000","last_undegraded":"2024-09-16T15:39:34.167782+0000","last_fullsized":"2024-09-16T15:39:34.167782+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:04.395310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723055+0000","last_change":"2024-09-16T15:39:33.166874+0000","last_active":"2024-09-16T15:39:34.723055+0000","last_peered":"2024-09-16T15:39:34.723055+0000","last_clean":"2024-09-16T15:39:34.723055+0000","last_became_active":"2024-09-16T15:39:33.165630+0000","last_became_peered":"2024-09-16T15:39:33.165630+0000","last_unstale":"2024-09-16T15:39:34.723055+0000","last_undegraded":"2024-09-16T15:39:34.723055+0000","last_fullsized":"2024-09-16T15:39:34.723055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:49.039555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235750+0000","last_change":"2024-09-16T15:39:33.174678+0000","last_active":"2024-09-16T15:39:34.235750+0000","last_peered":"2024-09-16T15:39:34.235750+0000","last_clean":"2024-09-16T15:39:34.235750+0000","last_became_active":"2024-09-16T15:39:33.174350+0000","last_became_peered":"2024-09-16T15:39:33.174350+0000","last_unstale":"2024-09-16T15:39:34.235750+0000","last_undegraded":"2024-09-16T15:39:34.235750+0000","last_fullsized":"2024-09-16T15:39:34.235750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:59.044396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.168177+0000","last_change":"2024-09-16T15:39:32.164389+0000","last_active":"2024-09-16T15:39:34.168177+0000","last_peered":"2024-09-16T15:39:34.168177+0000","last_clean":"2024-09-16T15:39:34.168177+0000","last_became_active":"2024-09-16T15:39:32.164146+0000","last_became_peered":"2024-09-16T15:39:32.164146+0000","last_unstale":"2024-09-16T15:39:34.168177+0000","last_undegraded":"2024-09-16T15:39:34.168177+0000","last_fullsized":"2024-09-16T15:39:34.168177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:43.243153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.168067+0000","last_change":"2024-09-16T15:39:33.167259+0000","last_active":"2024-09-16T15:39:34.168067+0000","last_peered":"2024-09-16T15:39:34.168067+0000","last_clean":"2024-09-16T15:39:34.168067+0000","last_became_active":"2024-09-16T15:39:33.166799+0000","last_became_peered":"2024-09-16T15:39:33.166799+0000","last_unstale":"2024-09-16T15:39:34.168067+0000","last_undegraded":"2024-09-16T15:39:34.168067+0000","last_fullsized":"2024-09-16T15:39:34.168067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:46.891414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235692+0000","last_change":"2024-09-16T15:39:32.166998+0000","last_active":"2024-09-16T15:39:34.235692+0000","last_peered":"2024-09-16T15:39:34.235692+0000","last_clean":"2024-09-16T15:39:34.235692+0000","last_became_active":"2024-09-16T15:39:32.166503+0000","last_became_peered":"2024-09-16T15:39:32.166503+0000","last_unstale":"2024-09-16T15:39:34.235692+0000","last_undegraded":"2024-09-16T15:39:34.235692+0000","last_fullsized":"2024-09-16T15:39:34.235692+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:16.861797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724359+0000","last_change":"2024-09-16T15:39:33.182201+0000","last_active":"2024-09-16T15:39:34.724359+0000","last_peered":"2024-09-16T15:39:34.724359+0000","last_clean":"2024-09-16T15:39:34.724359+0000","last_became_active":"2024-09-16T15:39:33.181864+0000","last_became_peered":"2024-09-16T15:39:33.181864+0000","last_unstale":"2024-09-16T15:39:34.724359+0000","last_undegraded":"2024-09-16T15:39:34.724359+0000","last_fullsized":"2024-09-16T15:39:34.724359+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:17.921030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301016+0000","last_change":"2024-09-16T15:39:32.170881+0000","last_active":"2024-09-16T15:39:34.301016+0000","last_peered":"2024-09-16T15:39:34.301016+0000","last_clean":"2024-09-16T15:39:34.301016+0000","last_became_active":"2024-09-16T15:39:32.170639+0000","last_became_peered":"2024-09-16T15:39:32.170639+0000","last_unstale":"2024-09-16T15:39:34.301016+0000","last_undegraded":"2024-09-16T15:39:34.301016+0000","last_fullsized":"2024-09-16T15:39:34.301016+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:58.036805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165671+0000","last_change":"2024-09-16T15:39:33.174871+0000","last_active":"2024-09-16T15:39:34.165671+0000","last_peered":"2024-09-16T15:39:34.165671+0000","last_clean":"2024-09-16T15:39:34.165671+0000","last_became_active":"2024-09-16T15:39:33.171604+0000","last_became_peered":"2024-09-16T15:39:33.171604+0000","last_unstale":"2024-09-16T15:39:34.165671+0000","last_undegraded":"2024-09-16T15:39:34.165671+0000","last_fullsized":"2024-09-16T15:39:34.165671+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:26.792470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722550+0000","last_change":"2024-09-16T15:39:32.171713+0000","last_active":"2024-09-16T15:39:34.722550+0000","last_peered":"2024-09-16T15:39:34.722550+0000","last_clean":"2024-09-16T15:39:34.722550+0000","last_became_active":"2024-09-16T15:39:32.171452+0000","last_became_peered":"2024-09-16T15:39:32.171452+0000","last_unstale":"2024-09-16T15:39:34.722550+0000","last_undegraded":"2024-09-16T15:39:34.722550+0000","last_fullsized":"2024-09-16T15:39:34.722550+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:23:49.392761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723929+0000","last_change":"2024-09-16T15:39:33.166679+0000","last_active":"2024-09-16T15:39:34.723929+0000","last_peered":"2024-09-16T15:39:34.723929+0000","last_clean":"2024-09-16T15:39:34.723929+0000","last_became_active":"2024-09-16T15:39:33.166353+0000","last_became_peered":"2024-09-16T15:39:33.166353+0000","last_unstale":"2024-09-16T15:39:34.723929+0000","last_undegraded":"2024-09-16T15:39:34.723929+0000","last_fullsized":"2024-09-16T15:39:34.723929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:04:21.991861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724350+0000","last_change":"2024-09-16T15:39:32.162895+0000","last_active":"2024-09-16T15:39:34.724350+0000","last_peered":"2024-09-16T15:39:34.724350+0000","last_clean":"2024-09-16T15:39:34.724350+0000","last_became_active":"2024-09-16T15:39:32.162267+0000","last_became_peered":"2024-09-16T15:39:32.162267+0000","last_unstale":"2024-09-16T15:39:34.724350+0000","last_undegraded":"2024-09-16T15:39:34.724350+0000","last_fullsized":"2024-09-16T15:39:34.724350+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:14.156509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.166504+0000","last_change":"2024-09-16T15:39:33.175329+0000","last_active":"2024-09-16T15:39:34.166504+0000","last_peered":"2024-09-16T15:39:34.166504+0000","last_clean":"2024-09-16T15:39:34.166504+0000","last_became_active":"2024-09-16T15:39:33.174995+0000","last_became_peered":"2024-09-16T15:39:33.174995+0000","last_unstale":"2024-09-16T15:39:34.166504+0000","last_undegraded":"2024-09-16T15:39:34.166504+0000","last_fullsized":"2024-09-16T15:39:34.166504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:39.743722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723816+0000","last_change":"2024-09-16T15:39:32.163428+0000","last_active":"2024-09-16T15:39:34.723816+0000","last_peered":"2024-09-16T15:39:34.723816+0000","last_clean":"2024-09-16T15:39:34.723816+0000","last_became_active":"2024-09-16T15:39:32.163148+0000","last_became_peered":"2024-09-16T15:39:32.163148+0000","last_unstale":"2024-09-16T15:39:34.723816+0000","last_undegraded":"2024-09-16T15:39:34.723816+0000","last_fullsized":"2024-09-16T15:39:34.723816+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:45:50.931536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723386+0000","last_change":"2024-09-16T15:39:33.173762+0000","last_active":"2024-09-16T15:39:34.723386+0000","last_peered":"2024-09-16T15:39:34.723386+0000","last_clean":"2024-09-16T15:39:34.723386+0000","last_became_active":"2024-09-16T15:39:33.173501+0000","last_became_peered":"2024-09-16T15:39:33.173501+0000","last_unstale":"2024-09-16T15:39:34.723386+0000","last_undegraded":"2024-09-16T15:39:34.723386+0000","last_fullsized":"2024-09-16T15:39:34.723386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:38.003947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.300659+0000","last_change":"2024-09-16T15:39:32.164943+0000","last_active":"2024-09-16T15:39:34.300659+0000","last_peered":"2024-09-16T15:39:34.300659+0000","last_clean":"2024-09-16T15:39:34.300659+0000","last_became_active":"2024-09-16T15:39:32.164576+0000","last_became_peered":"2024-09-16T15:39:32.164576+0000","last_unstale":"2024-09-16T15:39:34.300659+0000","last_undegraded":"2024-09-16T15:39:34.300659+0000","last_fullsized":"2024-09-16T15:39:34.300659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:17.096357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164735+0000","last_change":"2024-09-16T15:39:33.170443+0000","last_active":"2024-09-16T15:39:34.164735+0000","last_peered":"2024-09-16T15:39:34.164735+0000","last_clean":"2024-09-16T15:39:34.164735+0000","last_became_active":"2024-09-16T15:39:33.170165+0000","last_became_peered":"2024-09-16T15:39:33.170165+0000","last_unstale":"2024-09-16T15:39:34.164735+0000","last_undegraded":"2024-09-16T15:39:34.164735+0000","last_fullsized":"2024-09-16T15:39:34.164735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.857670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723303+0000","last_change":"2024-09-16T15:39:32.160722+0000","last_active":"2024-09-16T15:39:34.723303+0000","last_peered":"2024-09-16T15:39:34.723303+0000","last_clean":"2024-09-16T15:39:34.723303+0000","last_became_active":"2024-09-16T15:39:32.160298+0000","last_became_peered":"2024-09-16T15:39:32.160298+0000","last_unstale":"2024-09-16T15:39:34.723303+0000","last_undegraded":"2024-09-16T15:39:34.723303+0000","last_fullsized":"2024-09-16T15:39:34.723303+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:23.168947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.166501+0000","last_change":"2024-09-16T15:39:33.182168+0000","last_active":"2024-09-16T15:39:34.166501+0000","last_peered":"2024-09-16T15:39:34.166501+0000","last_clean":"2024-09-16T15:39:34.166501+0000","last_became_active":"2024-09-16T15:39:33.181860+0000","last_became_peered":"2024-09-16T15:39:33.181860+0000","last_unstale":"2024-09-16T15:39:34.166501+0000","last_undegraded":"2024-09-16T15:39:34.166501+0000","last_fullsized":"2024-09-16T15:39:34.166501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:04:21.991861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301211+0000","last_change":"2024-09-16T15:39:32.170156+0000","last_active":"2024-09-16T15:39:34.301211+0000","last_peered":"2024-09-16T15:39:34.301211+0000","last_clean":"2024-09-16T15:39:34.301211+0000","last_became_active":"2024-09-16T15:39:32.169818+0000","last_became_peered":"2024-09-16T15:39:32.169818+0000","last_unstale":"2024-09-16T15:39:34.301211+0000","last_undegraded":"2024-09-16T15:39:34.301211+0000","last_fullsized":"2024-09-16T15:39:34.301211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:44.958137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301033+0000","last_change":"2024-09-16T15:39:33.171968+0000","last_active":"2024-09-16T15:39:34.301033+0000","last_peered":"2024-09-16T15:39:34.301033+0000","last_clean":"2024-09-16T15:39:34.301033+0000","last_became_active":"2024-09-16T15:39:33.171260+0000","last_became_peered":"2024-09-16T15:39:33.171260+0000","last_unstale":"2024-09-16T15:39:34.301033+0000","last_undegraded":"2024-09-16T15:39:34.301033+0000","last_fullsized":"2024-09-16T15:39:34.301033+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:06.649779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235802+0000","last_change":"2024-09-16T15:39:32.161560+0000","last_active":"2024-09-16T15:39:34.235802+0000","last_peered":"2024-09-16T15:39:34.235802+0000","last_clean":"2024-09-16T15:39:34.235802+0000","last_became_active":"2024-09-16T15:39:32.161287+0000","last_became_peered":"2024-09-16T15:39:32.161287+0000","last_unstale":"2024-09-16T15:39:34.235802+0000","last_undegraded":"2024-09-16T15:39:34.235802+0000","last_fullsized":"2024-09-16T15:39:34.235802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:03.211373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301327+0000","last_change":"2024-09-16T15:39:33.165990+0000","last_active":"2024-09-16T15:39:34.301327+0000","last_peered":"2024-09-16T15:39:34.301327+0000","last_clean":"2024-09-16T15:39:34.301327+0000","last_became_active":"2024-09-16T15:39:33.165597+0000","last_became_peered":"2024-09-16T15:39:33.165597+0000","last_unstale":"2024-09-16T15:39:34.301327+0000","last_undegraded":"2024-09-16T15:39:34.301327+0000","last_fullsized":"2024-09-16T15:39:34.301327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:10.203846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.166176+0000","last_change":"2024-09-16T15:39:32.170843+0000","last_active":"2024-09-16T15:39:34.166176+0000","last_peered":"2024-09-16T15:39:34.166176+0000","last_clean":"2024-09-16T15:39:34.166176+0000","last_became_active":"2024-09-16T15:39:32.170538+0000","last_became_peered":"2024-09-16T15:39:32.170538+0000","last_unstale":"2024-09-16T15:39:34.166176+0000","last_undegraded":"2024-09-16T15:39:34.166176+0000","last_fullsized":"2024-09-16T15:39:34.166176+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:03.908867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165843+0000","last_change":"2024-09-16T15:39:33.181337+0000","last_active":"2024-09-16T15:39:34.165843+0000","last_peered":"2024-09-16T15:39:34.165843+0000","last_clean":"2024-09-16T15:39:34.165843+0000","last_became_active":"2024-09-16T15:39:33.180917+0000","last_became_peered":"2024-09-16T15:39:33.180917+0000","last_unstale":"2024-09-16T15:39:34.165843+0000","last_undegraded":"2024-09-16T15:39:34.165843+0000","last_fullsized":"2024-09-16T15:39:34.165843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:21.149650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":21,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301204+0000","last_change":"2024-09-16T15:39:32.165779+0000","last_active":"2024-09-16T15:39:34.301204+0000","last_peered":"2024-09-16T15:39:34.301204+0000","last_clean":"2024-09-16T15:39:34.301204+0000","last_became_active":"2024-09-16T15:39:32.165510+0000","last_became_peered":"2024-09-16T15:39:32.165510+0000","last_unstale":"2024-09-16T15:39:34.301204+0000","last_undegraded":"2024-09-16T15:39:34.301204+0000","last_fullsized":"2024-09-16T15:39:34.301204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:30.826141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164871+0000","last_change":"2024-09-16T15:39:33.169134+0000","last_active":"2024-09-16T15:39:34.164871+0000","last_peered":"2024-09-16T15:39:34.164871+0000","last_clean":"2024-09-16T15:39:34.164871+0000","last_became_active":"2024-09-16T15:39:33.167382+0000","last_became_peered":"2024-09-16T15:39:33.167382+0000","last_unstale":"2024-09-16T15:39:34.164871+0000","last_undegraded":"2024-09-16T15:39:34.164871+0000","last_fullsized":"2024-09-16T15:39:34.164871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:07:01.249223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724427+0000","last_change":"2024-09-16T15:39:32.163172+0000","last_active":"2024-09-16T15:39:34.724427+0000","last_peered":"2024-09-16T15:39:34.724427+0000","last_clean":"2024-09-16T15:39:34.724427+0000","last_became_active":"2024-09-16T15:39:32.162626+0000","last_became_peered":"2024-09-16T15:39:32.162626+0000","last_unstale":"2024-09-16T15:39:34.724427+0000","last_undegraded":"2024-09-16T15:39:34.724427+0000","last_fullsized":"2024-09-16T15:39:34.724427+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:18.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.301021+0000","last_change":"2024-09-16T15:39:33.166467+0000","last_active":"2024-09-16T15:39:34.301021+0000","last_peered":"2024-09-16T15:39:34.301021+0000","last_clean":"2024-09-16T15:39:34.301021+0000","last_became_active":"2024-09-16T15:39:33.166213+0000","last_became_peered":"2024-09-16T15:39:33.166213+0000","last_unstale":"2024-09-16T15:39:34.301021+0000","last_undegraded":"2024-09-16T15:39:34.301021+0000","last_fullsized":"2024-09-16T15:39:34.301021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:00.068097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.164762+0000","last_change":"2024-09-16T15:39:32.162290+0000","last_active":"2024-09-16T15:39:34.164762+0000","last_peered":"2024-09-16T15:39:34.164762+0000","last_clean":"2024-09-16T15:39:34.164762+0000","last_became_active":"2024-09-16T15:39:32.162042+0000","last_became_peered":"2024-09-16T15:39:32.162042+0000","last_unstale":"2024-09-16T15:39:34.164762+0000","last_undegraded":"2024-09-16T15:39:34.164762+0000","last_fullsized":"2024-09-16T15:39:34.164762+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:23.717646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723324+0000","last_change":"2024-09-16T15:39:33.166957+0000","last_active":"2024-09-16T15:39:34.723324+0000","last_peered":"2024-09-16T15:39:34.723324+0000","last_clean":"2024-09-16T15:39:34.723324+0000","last_became_active":"2024-09-16T15:39:33.165557+0000","last_became_peered":"2024-09-16T15:39:33.165557+0000","last_unstale":"2024-09-16T15:39:34.723324+0000","last_undegraded":"2024-09-16T15:39:34.723324+0000","last_fullsized":"2024-09-16T15:39:34.723324+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:23.664230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.300771+0000","last_change":"2024-09-16T15:39:32.169367+0000","last_active":"2024-09-16T15:39:34.300771+0000","last_peered":"2024-09-16T15:39:34.300771+0000","last_clean":"2024-09-16T15:39:34.300771+0000","last_became_active":"2024-09-16T15:39:32.169188+0000","last_became_peered":"2024-09-16T15:39:32.169188+0000","last_unstale":"2024-09-16T15:39:34.300771+0000","last_undegraded":"2024-09-16T15:39:34.300771+0000","last_fullsized":"2024-09-16T15:39:34.300771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:09.929691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.168005+0000","last_change":"2024-09-16T15:39:33.168131+0000","last_active":"2024-09-16T15:39:34.168005+0000","last_peered":"2024-09-16T15:39:34.168005+0000","last_clean":"2024-09-16T15:39:34.168005+0000","last_became_active":"2024-09-16T15:39:33.167843+0000","last_became_peered":"2024-09-16T15:39:33.167843+0000","last_unstale":"2024-09-16T15:39:34.168005+0000","last_undegraded":"2024-09-16T15:39:34.168005+0000","last_fullsized":"2024-09-16T15:39:34.168005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:29.049665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.300683+0000","last_change":"2024-09-16T15:39:32.163551+0000","last_active":"2024-09-16T15:39:34.300683+0000","last_peered":"2024-09-16T15:39:34.300683+0000","last_clean":"2024-09-16T15:39:34.300683+0000","last_became_active":"2024-09-16T15:39:32.163283+0000","last_became_peered":"2024-09-16T15:39:32.163283+0000","last_unstale":"2024-09-16T15:39:34.300683+0000","last_undegraded":"2024-09-16T15:39:34.300683+0000","last_fullsized":"2024-09-16T15:39:34.300683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:46.721713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724212+0000","last_change":"2024-09-16T15:39:33.169707+0000","last_active":"2024-09-16T15:39:34.724212+0000","last_peered":"2024-09-16T15:39:34.724212+0000","last_clean":"2024-09-16T15:39:34.724212+0000","last_became_active":"2024-09-16T15:39:33.169378+0000","last_became_peered":"2024-09-16T15:39:33.169378+0000","last_unstale":"2024-09-16T15:39:34.724212+0000","last_undegraded":"2024-09-16T15:39:34.724212+0000","last_fullsized":"2024-09-16T15:39:34.724212+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:38:17.682642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165577+0000","last_change":"2024-09-16T15:39:32.161394+0000","last_active":"2024-09-16T15:39:34.165577+0000","last_peered":"2024-09-16T15:39:34.165577+0000","last_clean":"2024-09-16T15:39:34.165577+0000","last_became_active":"2024-09-16T15:39:32.161124+0000","last_became_peered":"2024-09-16T15:39:32.161124+0000","last_unstale":"2024-09-16T15:39:34.165577+0000","last_undegraded":"2024-09-16T15:39:34.165577+0000","last_fullsized":"2024-09-16T15:39:34.165577+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:40.912077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.166781+0000","last_change":"2024-09-16T15:39:33.171451+0000","last_active":"2024-09-16T15:39:34.166781+0000","last_peered":"2024-09-16T15:39:34.166781+0000","last_clean":"2024-09-16T15:39:34.166781+0000","last_became_active":"2024-09-16T15:39:33.171159+0000","last_became_peered":"2024-09-16T15:39:33.171159+0000","last_unstale":"2024-09-16T15:39:34.166781+0000","last_undegraded":"2024-09-16T15:39:34.166781+0000","last_fullsized":"2024-09-16T15:39:34.166781+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:06.649779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724560+0000","last_change":"2024-09-16T15:39:32.163386+0000","last_active":"2024-09-16T15:39:34.724560+0000","last_peered":"2024-09-16T15:39:34.724560+0000","last_clean":"2024-09-16T15:39:34.724560+0000","last_became_active":"2024-09-16T15:39:32.161214+0000","last_became_peered":"2024-09-16T15:39:32.161214+0000","last_unstale":"2024-09-16T15:39:34.724560+0000","last_undegraded":"2024-09-16T15:39:34.724560+0000","last_fullsized":"2024-09-16T15:39:34.724560+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:08.898328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165551+0000","last_change":"2024-09-16T15:39:33.175372+0000","last_active":"2024-09-16T15:39:34.165551+0000","last_peered":"2024-09-16T15:39:34.165551+0000","last_clean":"2024-09-16T15:39:34.165551+0000","last_became_active":"2024-09-16T15:39:33.174198+0000","last_became_peered":"2024-09-16T15:39:33.174198+0000","last_unstale":"2024-09-16T15:39:34.165551+0000","last_undegraded":"2024-09-16T15:39:34.165551+0000","last_fullsized":"2024-09-16T15:39:34.165551+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:53.410594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724331+0000","last_change":"2024-09-16T15:39:32.166140+0000","last_active":"2024-09-16T15:39:34.724331+0000","last_peered":"2024-09-16T15:39:34.724331+0000","last_clean":"2024-09-16T15:39:34.724331+0000","last_became_active":"2024-09-16T15:39:32.165774+0000","last_became_peered":"2024-09-16T15:39:32.165774+0000","last_unstale":"2024-09-16T15:39:34.724331+0000","last_undegraded":"2024-09-16T15:39:34.724331+0000","last_fullsized":"2024-09-16T15:39:34.724331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:54.716352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722666+0000","last_change":"2024-09-16T15:39:33.171115+0000","last_active":"2024-09-16T15:39:34.722666+0000","last_peered":"2024-09-16T15:39:34.722666+0000","last_clean":"2024-09-16T15:39:34.722666+0000","last_became_active":"2024-09-16T15:39:33.170789+0000","last_became_peered":"2024-09-16T15:39:33.170789+0000","last_unstale":"2024-09-16T15:39:34.722666+0000","last_undegraded":"2024-09-16T15:39:34.722666+0000","last_fullsized":"2024-09-16T15:39:34.722666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:43.337174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724079+0000","last_change":"2024-09-16T15:39:32.162492+0000","last_active":"2024-09-16T15:39:34.724079+0000","last_peered":"2024-09-16T15:39:34.724079+0000","last_clean":"2024-09-16T15:39:34.724079+0000","last_became_active":"2024-09-16T15:39:32.162229+0000","last_became_peered":"2024-09-16T15:39:32.162229+0000","last_unstale":"2024-09-16T15:39:34.724079+0000","last_undegraded":"2024-09-16T15:39:34.724079+0000","last_fullsized":"2024-09-16T15:39:34.724079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:23.317340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722386+0000","last_change":"2024-09-16T15:39:33.165270+0000","last_active":"2024-09-16T15:39:34.722386+0000","last_peered":"2024-09-16T15:39:34.722386+0000","last_clean":"2024-09-16T15:39:34.722386+0000","last_became_active":"2024-09-16T15:39:33.164980+0000","last_became_peered":"2024-09-16T15:39:33.164980+0000","last_unstale":"2024-09-16T15:39:34.722386+0000","last_undegraded":"2024-09-16T15:39:34.722386+0000","last_fullsized":"2024-09-16T15:39:34.722386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:27:40.197665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235571+0000","last_change":"2024-09-16T15:39:32.161540+0000","last_active":"2024-09-16T15:39:34.235571+0000","last_peered":"2024-09-16T15:39:34.235571+0000","last_clean":"2024-09-16T15:39:34.235571+0000","last_became_active":"2024-09-16T15:39:32.161225+0000","last_became_peered":"2024-09-16T15:39:32.161225+0000","last_unstale":"2024-09-16T15:39:34.235571+0000","last_undegraded":"2024-09-16T15:39:34.235571+0000","last_fullsized":"2024-09-16T15:39:34.235571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:27.775067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722693+0000","last_change":"2024-09-16T15:39:33.172703+0000","last_active":"2024-09-16T15:39:34.722693+0000","last_peered":"2024-09-16T15:39:34.722693+0000","last_clean":"2024-09-16T15:39:34.722693+0000","last_became_active":"2024-09-16T15:39:33.172417+0000","last_became_peered":"2024-09-16T15:39:33.172417+0000","last_unstale":"2024-09-16T15:39:34.722693+0000","last_undegraded":"2024-09-16T15:39:34.722693+0000","last_fullsized":"2024-09-16T15:39:34.722693+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:01.436296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.167838+0000","last_change":"2024-09-16T15:39:32.157654+0000","last_active":"2024-09-16T15:39:34.167838+0000","last_peered":"2024-09-16T15:39:34.167838+0000","last_clean":"2024-09-16T15:39:34.167838+0000","last_became_active":"2024-09-16T15:39:32.157393+0000","last_became_peered":"2024-09-16T15:39:32.157393+0000","last_unstale":"2024-09-16T15:39:34.167838+0000","last_undegraded":"2024-09-16T15:39:34.167838+0000","last_fullsized":"2024-09-16T15:39:34.167838+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:02.941075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722440+0000","last_change":"2024-09-16T15:39:33.183025+0000","last_active":"2024-09-16T15:39:34.722440+0000","last_peered":"2024-09-16T15:39:34.722440+0000","last_clean":"2024-09-16T15:39:34.722440+0000","last_became_active":"2024-09-16T15:39:33.182661+0000","last_became_peered":"2024-09-16T15:39:33.182661+0000","last_unstale":"2024-09-16T15:39:34.722440+0000","last_undegraded":"2024-09-16T15:39:34.722440+0000","last_fullsized":"2024-09-16T15:39:34.722440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:04.218963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723612+0000","last_change":"2024-09-16T15:39:32.162100+0000","last_active":"2024-09-16T15:39:34.723612+0000","last_peered":"2024-09-16T15:39:34.723612+0000","last_clean":"2024-09-16T15:39:34.723612+0000","last_became_active":"2024-09-16T15:39:32.161782+0000","last_became_peered":"2024-09-16T15:39:32.161782+0000","last_unstale":"2024-09-16T15:39:34.723612+0000","last_undegraded":"2024-09-16T15:39:34.723612+0000","last_fullsized":"2024-09-16T15:39:34.723612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:12:26.270346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723133+0000","last_change":"2024-09-16T15:39:33.163394+0000","last_active":"2024-09-16T15:39:34.723133+0000","last_peered":"2024-09-16T15:39:34.723133+0000","last_clean":"2024-09-16T15:39:34.723133+0000","last_became_active":"2024-09-16T15:39:33.163086+0000","last_became_peered":"2024-09-16T15:39:33.163086+0000","last_unstale":"2024-09-16T15:39:34.723133+0000","last_undegraded":"2024-09-16T15:39:34.723133+0000","last_fullsized":"2024-09-16T15:39:34.723133+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:21.149650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165672+0000","last_change":"2024-09-16T15:39:32.168491+0000","last_active":"2024-09-16T15:39:34.165672+0000","last_peered":"2024-09-16T15:39:34.165672+0000","last_clean":"2024-09-16T15:39:34.165672+0000","last_became_active":"2024-09-16T15:39:32.168195+0000","last_became_peered":"2024-09-16T15:39:32.168195+0000","last_unstale":"2024-09-16T15:39:34.165672+0000","last_undegraded":"2024-09-16T15:39:34.165672+0000","last_fullsized":"2024-09-16T15:39:34.165672+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:17.715641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.722614+0000","last_change":"2024-09-16T15:39:33.170935+0000","last_active":"2024-09-16T15:39:34.722614+0000","last_peered":"2024-09-16T15:39:34.722614+0000","last_clean":"2024-09-16T15:39:34.722614+0000","last_became_active":"2024-09-16T15:39:33.170519+0000","last_became_peered":"2024-09-16T15:39:33.170519+0000","last_unstale":"2024-09-16T15:39:34.722614+0000","last_undegraded":"2024-09-16T15:39:34.722614+0000","last_fullsized":"2024-09-16T15:39:34.722614+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:28.782657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165670+0000","last_change":"2024-09-16T15:39:32.166439+0000","last_active":"2024-09-16T15:39:34.165670+0000","last_peered":"2024-09-16T15:39:34.165670+0000","last_clean":"2024-09-16T15:39:34.165670+0000","last_became_active":"2024-09-16T15:39:32.166175+0000","last_became_peered":"2024-09-16T15:39:32.166175+0000","last_unstale":"2024-09-16T15:39:34.165670+0000","last_undegraded":"2024-09-16T15:39:34.165670+0000","last_fullsized":"2024-09-16T15:39:34.165670+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:39.246388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.166884+0000","last_change":"2024-09-16T15:39:33.181043+0000","last_active":"2024-09-16T15:39:34.166884+0000","last_peered":"2024-09-16T15:39:34.166884+0000","last_clean":"2024-09-16T15:39:34.166884+0000","last_became_active":"2024-09-16T15:39:33.180740+0000","last_became_peered":"2024-09-16T15:39:33.180740+0000","last_unstale":"2024-09-16T15:39:34.166884+0000","last_undegraded":"2024-09-16T15:39:34.166884+0000","last_fullsized":"2024-09-16T15:39:34.166884+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:00.068097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235893+0000","last_change":"2024-09-16T15:39:32.155559+0000","last_active":"2024-09-16T15:39:34.235893+0000","last_peered":"2024-09-16T15:39:34.235893+0000","last_clean":"2024-09-16T15:39:34.235893+0000","last_became_active":"2024-09-16T15:39:32.155286+0000","last_became_peered":"2024-09-16T15:39:32.155286+0000","last_unstale":"2024-09-16T15:39:34.235893+0000","last_undegraded":"2024-09-16T15:39:34.235893+0000","last_fullsized":"2024-09-16T15:39:34.235893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:37.748367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235517+0000","last_change":"2024-09-16T15:39:33.187024+0000","last_active":"2024-09-16T15:39:34.235517+0000","last_peered":"2024-09-16T15:39:34.235517+0000","last_clean":"2024-09-16T15:39:34.235517+0000","last_became_active":"2024-09-16T15:39:33.186739+0000","last_became_peered":"2024-09-16T15:39:33.186739+0000","last_unstale":"2024-09-16T15:39:34.235517+0000","last_undegraded":"2024-09-16T15:39:34.235517+0000","last_fullsized":"2024-09-16T15:39:34.235517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:10.937282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165430+0000","last_change":"2024-09-16T15:39:32.172865+0000","last_active":"2024-09-16T15:39:34.165430+0000","last_peered":"2024-09-16T15:39:34.165430+0000","last_clean":"2024-09-16T15:39:34.165430+0000","last_became_active":"2024-09-16T15:39:32.172616+0000","last_became_peered":"2024-09-16T15:39:32.172616+0000","last_unstale":"2024-09-16T15:39:34.165430+0000","last_undegraded":"2024-09-16T15:39:34.165430+0000","last_fullsized":"2024-09-16T15:39:34.165430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:38.115059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.167894+0000","last_change":"2024-09-16T15:39:33.165299+0000","last_active":"2024-09-16T15:39:34.167894+0000","last_peered":"2024-09-16T15:39:34.167894+0000","last_clean":"2024-09-16T15:39:34.167894+0000","last_became_active":"2024-09-16T15:39:33.164315+0000","last_became_peered":"2024-09-16T15:39:33.164315+0000","last_unstale":"2024-09-16T15:39:34.167894+0000","last_undegraded":"2024-09-16T15:39:34.167894+0000","last_fullsized":"2024-09-16T15:39:34.167894+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:59:45.623638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235776+0000","last_change":"2024-09-16T15:39:32.164165+0000","last_active":"2024-09-16T15:39:34.235776+0000","last_peered":"2024-09-16T15:39:34.235776+0000","last_clean":"2024-09-16T15:39:34.235776+0000","last_became_active":"2024-09-16T15:39:32.163728+0000","last_became_peered":"2024-09-16T15:39:32.163728+0000","last_unstale":"2024-09-16T15:39:34.235776+0000","last_undegraded":"2024-09-16T15:39:34.235776+0000","last_fullsized":"2024-09-16T15:39:34.235776+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:02.249253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.235587+0000","last_change":"2024-09-16T15:39:33.174726+0000","last_active":"2024-09-16T15:39:34.235587+0000","last_peered":"2024-09-16T15:39:34.235587+0000","last_clean":"2024-09-16T15:39:34.235587+0000","last_became_active":"2024-09-16T15:39:33.174375+0000","last_became_peered":"2024-09-16T15:39:33.174375+0000","last_unstale":"2024-09-16T15:39:34.235587+0000","last_undegraded":"2024-09-16T15:39:34.235587+0000","last_fullsized":"2024-09-16T15:39:34.235587+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:45.965728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.165133+0000","last_change":"2024-09-16T15:39:32.167828+0000","last_active":"2024-09-16T15:39:34.165133+0000","last_peered":"2024-09-16T15:39:34.165133+0000","last_clean":"2024-09-16T15:39:34.165133+0000","last_became_active":"2024-09-16T15:39:32.165651+0000","last_became_peered":"2024-09-16T15:39:32.165651+0000","last_unstale":"2024-09-16T15:39:34.165133+0000","last_undegraded":"2024-09-16T15:39:34.165133+0000","last_fullsized":"2024-09-16T15:39:34.165133+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:29.486963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.300667+0000","last_change":"2024-09-16T15:39:33.184537+0000","last_active":"2024-09-16T15:39:34.300667+0000","last_peered":"2024-09-16T15:39:34.300667+0000","last_clean":"2024-09-16T15:39:34.300667+0000","last_became_active":"2024-09-16T15:39:33.184263+0000","last_became_peered":"2024-09-16T15:39:33.184263+0000","last_unstale":"2024-09-16T15:39:34.300667+0000","last_undegraded":"2024-09-16T15:39:34.300667+0000","last_fullsized":"2024-09-16T15:39:34.300667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:55.917732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723737+0000","last_change":"2024-09-16T15:39:32.169431+0000","last_active":"2024-09-16T15:39:34.723737+0000","last_peered":"2024-09-16T15:39:34.723737+0000","last_clean":"2024-09-16T15:39:34.723737+0000","last_became_active":"2024-09-16T15:39:32.169166+0000","last_became_peered":"2024-09-16T15:39:32.169166+0000","last_unstale":"2024-09-16T15:39:34.723737+0000","last_undegraded":"2024-09-16T15:39:34.723737+0000","last_fullsized":"2024-09-16T15:39:34.723737+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:36.312022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723451+0000","last_change":"2024-09-16T15:39:33.177663+0000","last_active":"2024-09-16T15:39:34.723451+0000","last_peered":"2024-09-16T15:39:34.723451+0000","last_clean":"2024-09-16T15:39:34.723451+0000","last_became_active":"2024-09-16T15:39:33.177154+0000","last_became_peered":"2024-09-16T15:39:33.177154+0000","last_unstale":"2024-09-16T15:39:34.723451+0000","last_undegraded":"2024-09-16T15:39:34.723451+0000","last_fullsized":"2024-09-16T15:39:34.723451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:54.183993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.300790+0000","last_change":"2024-09-16T15:39:32.170091+0000","last_active":"2024-09-16T15:39:34.300790+0000","last_peered":"2024-09-16T15:39:34.300790+0000","last_clean":"2024-09-16T15:39:34.300790+0000","last_became_active":"2024-09-16T15:39:32.169831+0000","last_became_peered":"2024-09-16T15:39:32.169831+0000","last_unstale":"2024-09-16T15:39:34.300790+0000","last_undegraded":"2024-09-16T15:39:34.300790+0000","last_fullsized":"2024-09-16T15:39:34.300790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:35.977654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.724533+0000","last_change":"2024-09-16T15:39:33.178022+0000","last_active":"2024-09-16T15:39:34.724533+0000","last_peered":"2024-09-16T15:39:34.724533+0000","last_clean":"2024-09-16T15:39:34.724533+0000","last_became_active":"2024-09-16T15:39:33.177705+0000","last_became_peered":"2024-09-16T15:39:33.177705+0000","last_unstale":"2024-09-16T15:39:34.724533+0000","last_undegraded":"2024-09-16T15:39:34.724533+0000","last_fullsized":"2024-09-16T15:39:34.724533+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:32.138319+0000","last_clean_scrub_stamp":"2024-09-16T15:39:32.138319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:06.649779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.167805+0000","last_change":"2024-09-16T15:39:32.163553+0000","last_active":"2024-09-16T15:39:34.167805+0000","last_peered":"2024-09-16T15:39:34.167805+0000","last_clean":"2024-09-16T15:39:34.167805+0000","last_became_active":"2024-09-16T15:39:32.162649+0000","last_became_peered":"2024-09-16T15:39:32.162649+0000","last_unstale":"2024-09-16T15:39:34.167805+0000","last_undegraded":"2024-09-16T15:39:34.167805+0000","last_fullsized":"2024-09-16T15:39:34.167805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:31.130728+0000","last_clean_scrub_stamp":"2024-09-16T15:39:31.130728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:06.959987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":135,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T15:39:34.723484+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:39:34.723484+0000","last_peered":"2024-09-16T15:39:34.723484+0000","last_clean":"2024-09-16T15:39:34.723484+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:39:34.723484+0000","last_undegraded":"2024-09-16T15:39:34.723484+0000","last_fullsized":"2024-09-16T15:39:34.723484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":747531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":409600,"data_stored":157689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":409600,"data_stored":157745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":409600,"data_stored":157177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":417792,"data_stored":158707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":405504,"data_stored":157143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":747441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":417792,"data_stored":158707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:42.557 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:39:42.557 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:39:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.761+0000 7f997a600640 1 -- 172.21.15.60:0/1636237544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99740c5730 msgr2=0x7f99740c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.761+0000 7f997a600640 1 --2- 172.21.15.60:0/1636237544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99740c5730 0x7f99740c6430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f995c005fd0 tx=0x7f995c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.761+0000 7f997a600640 1 -- 172.21.15.60:0/1636237544 shutdown_connections 2024-09-16T15:39:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.761+0000 7f997a600640 1 --2- 172.21.15.60:0/1636237544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99740c5730 0x7f99740c6430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.761+0000 7f997a600640 1 --2- 172.21.15.60:0/1636237544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f997406e660 0x7f997406eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.761+0000 7f997a600640 1 -- 172.21.15.60:0/1636237544 >> 172.21.15.60:0/1636237544 conn(0x7f99740cc490 msgr2=0x7f99740cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.761+0000 7f997a600640 1 -- 172.21.15.60:0/1636237544 shutdown_connections 2024-09-16T15:39:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.761+0000 7f997a600640 1 -- 172.21.15.60:0/1636237544 wait complete. 2024-09-16T15:39:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.763+0000 7f997a600640 1 Processor -- start 2024-09-16T15:39:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.763+0000 7f997a600640 1 -- start start 2024-09-16T15:39:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.764+0000 7f997a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f997406e660 0x7f99740b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.764+0000 7f997a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99740c5730 0x7f9974145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.764+0000 7f997a600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f997415f670 0x7f9974159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.764+0000 7f997a600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f99740ca4f0 con 0x7f99740c5730 2024-09-16T15:39:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.764+0000 7f997a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99740ca370 con 0x7f997406e660 2024-09-16T15:39:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.764+0000 7f997a600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f997406f000 con 0x7f997415f670 2024-09-16T15:39:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f996be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99740c5730 0x7f9974145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f997406e660 0x7f99740b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f997406e660 0x7f99740b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37728/0 (socket says 172.21.15.60:37728) 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 -- 172.21.15.60:0/2861506077 learned_addr learned my addr 172.21.15.60:0/2861506077 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f996be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99740c5730 0x7f9974145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57064/0 (socket says 172.21.15.60:57064) 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979c00640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f997415f670 0x7f9974159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 -- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f997415f670 msgr2=0x7f9974159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f997415f670 0x7f9974159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 -- 172.21.15.60:0/2861506077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99740c5730 msgr2=0x7f9974145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99740c5730 0x7f9974145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 -- 172.21.15.60:0/2861506077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997415b170 con 0x7f997406e660 2024-09-16T15:39:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9979200640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f997406e660 0x7f99740b37c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f996400c370 tx=0x7f996400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9971600640 1 -- 172.21.15.60:0/2861506077 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99640052d0 con 0x7f997406e660 2024-09-16T15:39:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.765+0000 7f9971600640 1 -- 172.21.15.60:0/2861506077 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9964007020 con 0x7f997406e660 2024-09-16T15:39:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.766+0000 7f9971600640 1 -- 172.21.15.60:0/2861506077 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964007470 con 0x7f997406e660 2024-09-16T15:39:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.766+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99740b3fe0 con 0x7f997406e660 2024-09-16T15:39:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.766+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99740b45a0 con 0x7f997406e660 2024-09-16T15:39:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.767+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99740b47e0 con 0x7f997406e660 2024-09-16T15:39:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.768+0000 7f9971600640 1 -- 172.21.15.60:0/2861506077 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f99640067d0 con 0x7f997406e660 2024-09-16T15:39:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.769+0000 7f9971600640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9950078850 0x7f995007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.769+0000 7f9971600640 1 -- 172.21.15.60:0/2861506077 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9964099d10 con 0x7f997406e660 2024-09-16T15:39:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.769+0000 7f996be00640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9950078850 0x7f995007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.770+0000 7f996be00640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9950078850 0x7f995007ad10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f995c033000 tx=0x7f995c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:42.773+0000 7f9971600640 1 -- 172.21.15.60:0/2861506077 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f99740b47e0 con 0x7f997406e660 2024-09-16T15:39:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.040+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f99740b3d00 con 0x7f997406e660 2024-09-16T15:39:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.045+0000 7f9971600640 1 -- 172.21.15.60:0/2861506077 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f996409f020 con 0x7f997406e660 2024-09-16T15:39:43.047 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:39:43.047 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:39:43.047 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:39:43.047 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:39:43.047 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:39:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:39:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9950078850 msgr2=0x7f995007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9950078850 0x7f995007ad10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f995c033000 tx=0x7f995c004730 comp rx=0 tx=0).stop 2024-09-16T15:39:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f997406e660 msgr2=0x7f99740b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f997406e660 0x7f99740b37c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f996400c370 tx=0x7f996400c830 comp rx=0 tx=0).stop 2024-09-16T15:39:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 shutdown_connections 2024-09-16T15:39:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9950078850 0x7f995007ad10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f997415f670 0x7f9974159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f99740c5730 0x7f9974145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 --2- 172.21.15.60:0/2861506077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f997406e660 0x7f99740b37c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 >> 172.21.15.60:0/2861506077 conn(0x7f99740cc490 msgr2=0x7f99740af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 shutdown_connections 2024-09-16T15:39:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.048+0000 7f997a600640 1 -- 172.21.15.60:0/2861506077 wait complete. 2024-09-16T15:39:43.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.224+0000 7f59df600640 1 -- 172.21.15.60:0/2941071065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d81445d0 msgr2=0x7f59d80cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.224+0000 7f59df600640 1 --2- 172.21.15.60:0/2941071065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d81445d0 0x7f59d80cebf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009860 tx=0x7f59cc02a540 comp rx=0 tx=0).stop 2024-09-16T15:39:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.224+0000 7f59df600640 1 -- 172.21.15.60:0/2941071065 shutdown_connections 2024-09-16T15:39:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.224+0000 7f59df600640 1 --2- 172.21.15.60:0/2941071065 >> v2:172.21.15.100:3300/0 conn(0x7f59d80b4ca0 0x7f59d80cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.224+0000 7f59df600640 1 --2- 172.21.15.60:0/2941071065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d81445d0 0x7f59d80cebf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.224+0000 7f59df600640 1 -- 172.21.15.60:0/2941071065 >> 172.21.15.60:0/2941071065 conn(0x7f59d80b1950 msgr2=0x7f59d80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.224+0000 7f59df600640 1 -- 172.21.15.60:0/2941071065 shutdown_connections 2024-09-16T15:39:43.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.224+0000 7f59df600640 1 -- 172.21.15.60:0/2941071065 wait complete. 2024-09-16T15:39:43.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.226+0000 7f59df600640 1 Processor -- start 2024-09-16T15:39:43.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.226+0000 7f59df600640 1 -- start start 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59df600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d80b4ca0 0x7f59d81419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59df600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d80ccfd0 0x7f59d81f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59df600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d81445d0 0x7f59d80afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59df600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f59d815f080 con 0x7f59d80b4ca0 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59df600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59d8073bc0 con 0x7f59d81445d0 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59df600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f59d815f200 con 0x7f59d80ccfd0 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59dcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d80b4ca0 0x7f59d81419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59dcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d80b4ca0 0x7f59d81419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57070/0 (socket says 172.21.15.60:57070) 2024-09-16T15:39:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.227+0000 7f59dcc00640 1 -- 172.21.15.60:0/3485305694 learned_addr learned my addr 172.21.15.60:0/3485305694 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59d7e00640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d80ccfd0 0x7f59d81f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59dd600640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d81445d0 0x7f59d80afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59dcc00640 1 -- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d80ccfd0 msgr2=0x7f59d81f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59dcc00640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d80ccfd0 0x7f59d81f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59dcc00640 1 -- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d81445d0 msgr2=0x7f59d80afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59dcc00640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d81445d0 0x7f59d80afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59dcc00640 1 -- 172.21.15.60:0/3485305694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d81f6620 con 0x7f59d80b4ca0 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59dcc00640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d80b4ca0 0x7f59d81419c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009780 tx=0x7f59cc033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.228+0000 7f59d4c00640 1 -- 172.21.15.60:0/3485305694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc031b50 con 0x7f59d80b4ca0 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.229+0000 7f59d4c00640 1 -- 172.21.15.60:0/3485305694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59cc031e00 con 0x7f59d80b4ca0 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.229+0000 7f59d4c00640 1 -- 172.21.15.60:0/3485305694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc005280 con 0x7f59d80b4ca0 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.229+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d81f6900 con 0x7f59d80b4ca0 2024-09-16T15:39:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.229+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d81f2b80 con 0x7f59d80b4ca0 2024-09-16T15:39:43.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.230+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d815f6b0 con 0x7f59d80b4ca0 2024-09-16T15:39:43.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.231+0000 7f59d4c00640 1 -- 172.21.15.60:0/3485305694 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f59cc005420 con 0x7f59d80b4ca0 2024-09-16T15:39:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.232+0000 7f59d4c00640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f59b8078850 0x7f59b807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.232+0000 7f59d4c00640 1 -- 172.21.15.60:0/3485305694 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f59cc0bb780 con 0x7f59d80b4ca0 2024-09-16T15:39:43.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.232+0000 7f59d7e00640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f59b8078850 0x7f59b807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:43.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.233+0000 7f59d7e00640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f59b8078850 0x7f59b807ad10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f59c4007fd0 tx=0x7f59c4001410 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:43.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.236+0000 7f59d4c00640 1 -- 172.21.15.60:0/3485305694 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f59cc03f030 con 0x7f59d80b4ca0 2024-09-16T15:39:43.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.468+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59d8070a20 con 0x7f59d80b4ca0 2024-09-16T15:39:43.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.468+0000 7f59d4c00640 1 -- 172.21.15.60:0/3485305694 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f59cc083790 con 0x7f59d80b4ca0 2024-09-16T15:39:43.470 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:43.470 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"btime":"2024-09-16T15:39:36:169518+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4866,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2562305658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2562305658},{"type":"v1","addr":"172.21.15.60:6833","nonce":2562305658}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/664909622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":664909622},{"type":"v1","addr":"172.21.15.100:6836","nonce":664909622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/801262237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":801262237},{"type":"v1","addr":"172.21.15.100:6837","nonce":801262237}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:32.603941+0000","modified":"2024-09-16T15:39:35.605148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/846626570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":846626570},{"type":"v1","addr":"172.21.15.60:6835","nonce":846626570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4881,"qdb_cluster":[4881]},"id":2}]} 2024-09-16T15:39:43.470 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 17 2024-09-16T15:39:43.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f59b8078850 msgr2=0x7f59b807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:43.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f59b8078850 0x7f59b807ad10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f59c4007fd0 tx=0x7f59c4001410 comp rx=0 tx=0).stop 2024-09-16T15:39:43.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d80b4ca0 msgr2=0x7f59d81419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d80b4ca0 0x7f59d81419c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009780 tx=0x7f59cc033ba0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 shutdown_connections 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f59b8078850 0x7f59b807ad10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59d81445d0 0x7f59d80afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59d80ccfd0 0x7f59d81f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 --2- 172.21.15.60:0/3485305694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59d80b4ca0 0x7f59d81419c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 >> 172.21.15.60:0/3485305694 conn(0x7f59d80b1950 msgr2=0x7f59d80b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 shutdown_connections 2024-09-16T15:39:43.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:43.471+0000 7f59df600640 1 -- 172.21.15.60:0/3485305694 wait complete. 2024-09-16T15:39:43.890 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:39:32.603941+0000', 'modified': '2024-09-16T15:39:35.605148+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4881}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4881': {'gid': 4881, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/846626570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 846626570}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 846626570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4881, 'qdb_cluster': [4881]} 2024-09-16T15:39:43.890 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:39:43.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:39:44.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.053+0000 7f88d0600640 1 -- 172.21.15.60:0/2786550351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 msgr2=0x7f88c80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.053+0000 7f88d0600640 1 --2- 172.21.15.60:0/2786550351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 0x7f88c80b2440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009830 tx=0x7f88b402a530 comp rx=0 tx=0).stop 2024-09-16T15:39:44.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.053+0000 7f88d0600640 1 -- 172.21.15.60:0/2786550351 shutdown_connections 2024-09-16T15:39:44.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.053+0000 7f88d0600640 1 --2- 172.21.15.60:0/2786550351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 0x7f88c80b2440 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.053+0000 7f88d0600640 1 -- 172.21.15.60:0/2786550351 >> 172.21.15.60:0/2786550351 conn(0x7f88c80b1950 msgr2=0x7f88c80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.053+0000 7f88d0600640 1 -- 172.21.15.60:0/2786550351 shutdown_connections 2024-09-16T15:39:44.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.053+0000 7f88d0600640 1 -- 172.21.15.60:0/2786550351 wait complete. 2024-09-16T15:39:44.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.055+0000 7f88d0600640 1 Processor -- start 2024-09-16T15:39:44.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.055+0000 7f88d0600640 1 -- start start 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88d0600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c8073fb0 0x7f88c81f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88d0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 0x7f88c80c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88d0600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c815f5a0 0x7f88c80c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88d0600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88c806f9f0 con 0x7f88c8073fb0 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88d0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88c806f870 con 0x7f88c80ccf00 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88d0600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88c8074420 con 0x7f88c815f5a0 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88cdc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c8073fb0 0x7f88c81f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88cdc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c8073fb0 0x7f88c81f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57082/0 (socket says 172.21.15.60:57082) 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88cdc00640 1 -- 172.21.15.60:0/3153320733 learned_addr learned my addr 172.21.15.60:0/3153320733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:44.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88cd200640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 0x7f88c80c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.056+0000 7f88ce600640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c815f5a0 0x7f88c80c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88cdc00640 1 -- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c815f5a0 msgr2=0x7f88c80c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88cdc00640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c815f5a0 0x7f88c80c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88cdc00640 1 -- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 msgr2=0x7f88c80c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88cdc00640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 0x7f88c80c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88cdc00640 1 -- 172.21.15.60:0/3153320733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c809b400 con 0x7f88c8073fb0 2024-09-16T15:39:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88cd200640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 0x7f88c80c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88cdc00640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c8073fb0 0x7f88c81f5920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c430 tx=0x7f88bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88c5600640 1 -- 172.21.15.60:0/3153320733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc011850 con 0x7f88c8073fb0 2024-09-16T15:39:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88c5600640 1 -- 172.21.15.60:0/3153320733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88bc017070 con 0x7f88c8073fb0 2024-09-16T15:39:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88c5600640 1 -- 172.21.15.60:0/3153320733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc011da0 con 0x7f88c8073fb0 2024-09-16T15:39:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.057+0000 7f88d0600640 1 -- 172.21.15.60:0/3153320733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c809b6e0 con 0x7f88c8073fb0 2024-09-16T15:39:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.058+0000 7f88d0600640 1 -- 172.21.15.60:0/3153320733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c809bca0 con 0x7f88c8073fb0 2024-09-16T15:39:44.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.060+0000 7f88c5600640 1 -- 172.21.15.60:0/3153320733 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f88bc01c460 con 0x7f88c8073fb0 2024-09-16T15:39:44.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.061+0000 7f88c5600640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88a8078850 0x7f88a807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.061+0000 7f88d3620740 1 Processor -- start 2024-09-16T15:39:44.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.061+0000 7f88c5600640 1 -- 172.21.15.60:0/3153320733 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f88bc09a700 con 0x7f88c8073fb0 2024-09-16T15:39:44.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.061+0000 7f88d3620740 1 -- start start 2024-09-16T15:39:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.061+0000 7f88cd200640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88a8078850 0x7f88a807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88d3620740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a0e8b0 0x558869a0ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88d3620740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588699c9a20 0x558869a0d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88d3620740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x558869a0d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88d3620740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55886994c430 con 0x5588699c9a20 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88d3620740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55886993c5d0 con 0x5588699c9fa0 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88d3620740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55886992ba00 con 0x558869a0e8b0 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88cd200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588699c9a20 0x558869a0d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88cd200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588699c9a20 0x558869a0d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57104/0 (socket says 172.21.15.60:57104) 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88cd200640 1 -- 172.21.15.60:0/3351036041 learned_addr learned my addr 172.21.15.60:0/3351036041 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88ce600640 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x558869a0d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88cd200640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88a8078850 0x7f88a807ad10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f88b4034000 tx=0x7f88b4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.062+0000 7f88cdc00640 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a0e8b0 0x558869a0ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88cd200640 1 -- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a0e8b0 msgr2=0x558869a0ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88cd200640 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a0e8b0 0x558869a0ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88cd200640 1 -- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 msgr2=0x558869a0d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88cd200640 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x558869a0d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88cd200640 1 -- 172.21.15.60:0/3351036041 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588698f4520 con 0x5588699c9a20 2024-09-16T15:39:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88cdc00640 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a0e8b0 0x558869a0ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88cd200640 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588699c9a20 0x558869a0d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f88b402ab20 tx=0x7f88b403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9c37b329aa09f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88ba000640 1 -- 172.21.15.60:0/3351036041 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b403d630 con 0x5588699c9a20 2024-09-16T15:39:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88ba000640 1 -- 172.21.15.60:0/3351036041 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88b403d8c0 con 0x5588699c9a20 2024-09-16T15:39:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.063+0000 7f88ba000640 1 -- 172.21.15.60:0/3351036041 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b403dcf0 con 0x5588699c9a20 2024-09-16T15:39:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 -- 172.21.15.60:0/3351036041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588699c9a20 msgr2=0x558869a0d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588699c9a20 0x558869a0d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f88b402ab20 tx=0x7f88b403f000 comp rx=0 tx=0).stop 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 -- 172.21.15.60:0/3351036041 shutdown_connections 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x558869a0d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588699c9a20 0x558869a0d390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 --2- 172.21.15.60:0/3351036041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a0e8b0 0x558869a0ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 -- 172.21.15.60:0/3351036041 >> 172.21.15.60:0/3351036041 conn(0x5588699c4ff0 msgr2=0x5588699c8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 -- 172.21.15.60:0/3351036041 shutdown_connections 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.064+0000 7f88d3620740 1 -- 172.21.15.60:0/3351036041 wait complete. 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.065+0000 7f88d3620740 1 -- start start 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.065+0000 7f88d3620740 1 Processor -- start 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88d3620740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x5588699f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88d3620740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a20410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88d3620740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a20950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88d3620740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5588698d2330 con 0x558869a0e8b0 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88d3620740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55886992f810 con 0x5588699c9fa0 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88d3620740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55886969c620 con 0x558869a07df0 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88d3620740 10 client.0 _ll_register_callbacks cb 0x5588698d7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88cd200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a20410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88cd200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a20410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57106/0 (socket says 172.21.15.60:57106) 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.066+0000 7f88cd200640 1 -- 172.21.15.60:0/2389921127 learned_addr learned my addr 172.21.15.60:0/2389921127 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88cdc00640 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x5588699f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88ce600640 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a20950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88cd200640 1 -- 172.21.15.60:0/2389921127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 msgr2=0x558869a20950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88cd200640 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a20950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88cd200640 1 -- 172.21.15.60:0/2389921127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 msgr2=0x5588699f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88cd200640 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x5588699f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88cd200640 1 -- 172.21.15.60:0/2389921127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886968d850 con 0x558869a0e8b0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88cd200640 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a20410 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f88b4035040 tx=0x7f88b403dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553b419ff5b73646 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88aea00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88aea00640 1 -- 172.21.15.60:0/2389921127 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b404a630 con 0x558869a0e8b0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88aea00640 1 -- 172.21.15.60:0/2389921127 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f88b404a7d0 con 0x558869a0e8b0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.067+0000 7f88d3620740 1 -- 172.21.15.60:0/2389921127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b403d8c0 con 0x558869a0e8b0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88aea00640 1 -- 172.21.15.60:0/2389921127 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f88b4063050 con 0x558869a0e8b0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88d3620740 10 client.5061 fetch_fsmap learned FSMap version 17 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88d3620740 1 -- 172.21.15.60:0/2389921127 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5588697046b0 con 0x558869a0e8b0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88aea00640 1 -- 172.21.15.60:0/2389921127 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88b403d8c0 con 0x558869a0e8b0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88aea00640 1 -- 172.21.15.60:0/2389921127 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f88b4066020 con 0x558869a0e8b0 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88d3620740 10 client.5061 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88d3620740 10 client.5061 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88d3620740 20 client.5061 populate_metadata read hostname 'smithi060' 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88d3620740 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] conn(0x558869a1f210 0x558869a1f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88d3620740 4 client.5061 mds_command: new command op to 4881 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:39:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88d3620740 1 -- 172.21.15.60:0/2389921127 --> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55886993e520 con 0x558869a1f210 2024-09-16T15:39:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.068+0000 7f88cdc00640 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] conn(0x558869a1f210 0x558869a1f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.069+0000 7f88cdc00640 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] conn(0x558869a1f210 0x558869a1f5f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3489555576964d7 server_cookie=19470786a7629d9b in_seq=0 out_seq=0 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.069+0000 7f88aea00640 10 client.5061 ms_handle_connect on v2:172.21.15.60:6834/846626570 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88aea00640 1 -- 172.21.15.60:0/2389921127 <== mds.0 v2:172.21.15.60:6834/846626570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55886993e520 con 0x558869a1f210 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88aea00640 10 client.5061 handle_command_reply: tid=0 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 client.5061 shutdown 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 -- 172.21.15.60:0/2389921127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 msgr2=0x558869a20410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a20410 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f88b4035040 tx=0x7f88b403dca0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 -- 172.21.15.60:0/2389921127 shutdown_connections 2024-09-16T15:39:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] conn(0x558869a1f210 0x558869a1f5f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a20950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a20410 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 --2- 172.21.15.60:0/2389921127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x5588699f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88aea00640 0 client.5061 ms_handle_reset on v2:172.21.15.60:6834/846626570 2024-09-16T15:39:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 -- 172.21.15.60:0/2389921127 >> 172.21.15.60:0/2389921127 conn(0x5588699c9a20 msgr2=0x5588699c2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 -- 172.21.15.60:0/2389921127 shutdown_connections 2024-09-16T15:39:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 1 -- 172.21.15.60:0/2389921127 wait complete. 2024-09-16T15:39:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.070+0000 7f88d3620740 20 client.5061 trim_cache size 0 max 16384 2024-09-16T15:39:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.089+0000 7f88d3620740 1 Processor -- start 2024-09-16T15:39:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.089+0000 7f88d3620740 1 -- start start 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88d3620740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x5588699c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88d3620740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a04830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88d3620740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a04d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88d3620740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88c82001c0 con 0x558869a0e8b0 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88d3620740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81f8b00 con 0x5588699c9fa0 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88d3620740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88c814a6d0 con 0x558869a07df0 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88ce600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a04d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88ce600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a04d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57122/0 (socket says 172.21.15.60:57122) 2024-09-16T15:39:44.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88ce600640 1 -- 172.21.15.60:0/1307983787 learned_addr learned my addr 172.21.15.60:0/1307983787 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88cdc00640 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x5588699c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.090+0000 7f88cd200640 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a04830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88ce600640 1 -- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 msgr2=0x558869a04830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88ce600640 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a04830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88ce600640 1 -- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 msgr2=0x5588699c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88ce600640 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x5588699c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88ce600640 1 -- 172.21.15.60:0/1307983787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558869a17e20 con 0x558869a0e8b0 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88cd200640 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a04830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88ce600640 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a04d70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f88c0003fb0 tx=0x7f88c0001e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342b6a68cc01362a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88b9600640 1 -- 172.21.15.60:0/1307983787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c001a020 con 0x558869a0e8b0 2024-09-16T15:39:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88b9600640 1 -- 172.21.15.60:0/1307983787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c00020b0 con 0x558869a0e8b0 2024-09-16T15:39:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.091+0000 7f88b9600640 1 -- 172.21.15.60:0/1307983787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0016420 con 0x558869a0e8b0 2024-09-16T15:39:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 -- 172.21.15.60:0/1307983787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 msgr2=0x558869a04d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a04d70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f88c0003fb0 tx=0x7f88c0001e60 comp rx=0 tx=0).stop 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 -- 172.21.15.60:0/1307983787 shutdown_connections 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x558869a04d70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a04830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 --2- 172.21.15.60:0/1307983787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x5588699c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 -- 172.21.15.60:0/1307983787 >> 172.21.15.60:0/1307983787 conn(0x5588699c9a20 msgr2=0x558869a20090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 -- 172.21.15.60:0/1307983787 shutdown_connections 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.092+0000 7f88d3620740 1 -- 172.21.15.60:0/1307983787 wait complete. 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.093+0000 7f88d3620740 1 -- start start 2024-09-16T15:39:44.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.093+0000 7f88d3620740 1 Processor -- start 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88d3620740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x558869a03820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88d3620740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a74e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88d3620740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x5588699c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88d3620740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x558869a063b0 con 0x558869a0e8b0 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88d3620740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558869a05df0 con 0x5588699c9fa0 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88d3620740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x558869a0dd90 con 0x558869a07df0 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88d3620740 10 client.0 _ll_register_callbacks cb 0x7f88b40035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88ce600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x5588699c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88ce600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x5588699c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57136/0 (socket says 172.21.15.60:57136) 2024-09-16T15:39:44.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88ce600640 1 -- 172.21.15.60:0/2771980643 learned_addr learned my addr 172.21.15.60:0/2771980643 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88cdc00640 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x558869a03820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88cd200640 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a74e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88ce600640 1 -- 172.21.15.60:0/2771980643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 msgr2=0x558869a74e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88ce600640 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a74e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88ce600640 1 -- 172.21.15.60:0/2771980643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 msgr2=0x558869a03820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88ce600640 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x558869a03820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.094+0000 7f88ce600640 1 -- 172.21.15.60:0/2771980643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558869a93410 con 0x558869a0e8b0 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.095+0000 7f88ce600640 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x5588699c3b30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f88c0001330 tx=0x7f88c00208a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc71b144e696ff8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.095+0000 7f88aea00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.095+0000 7f88aea00640 1 -- 172.21.15.60:0/2771980643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c001a070 con 0x558869a0e8b0 2024-09-16T15:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.095+0000 7f88aea00640 1 -- 172.21.15.60:0/2771980643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f88c0016420 con 0x558869a0e8b0 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.095+0000 7f88d3620740 1 -- 172.21.15.60:0/2771980643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c00020b0 con 0x558869a0e8b0 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.095+0000 7f88aea00640 1 -- 172.21.15.60:0/2771980643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f88c0016660 con 0x558869a0e8b0 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.095+0000 7f88d3620740 10 client.5067 fetch_fsmap learned FSMap version 17 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.095+0000 7f88d3620740 1 -- 172.21.15.60:0/2771980643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x558869a20a50 con 0x558869a0e8b0 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88aea00640 1 -- 172.21.15.60:0/2771980643 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c00020b0 con 0x558869a0e8b0 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88aea00640 1 -- 172.21.15.60:0/2771980643 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f88c0016800 con 0x558869a0e8b0 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88d3620740 10 client.5067 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88d3620740 10 client.5067 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88d3620740 20 client.5067 populate_metadata read hostname 'smithi060' 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88d3620740 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] conn(0x558869a8fe00 0x558869a280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88d3620740 4 client.5067 mds_command: new command op to 4881 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88d3620740 1 -- 172.21.15.60:0/2771980643 --> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] -- command(tid 0: {"prefix": "status"}) -- 0x558869939fd0 con 0x558869a8fe00 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88cdc00640 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] conn(0x558869a8fe00 0x558869a280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88cdc00640 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] conn(0x558869a8fe00 0x558869a280a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b6c839cf4efa79 server_cookie=eb3a7728e9e43046 in_seq=0 out_seq=0 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.096+0000 7f88aea00640 10 client.5067 ms_handle_connect on v2:172.21.15.60:6834/846626570 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88aea00640 1 -- 172.21.15.60:0/2771980643 <== mds.0 v2:172.21.15.60:6834/846626570 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558869939fd0 con 0x558869a8fe00 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88aea00640 10 client.5067 handle_command_reply: tid=0 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 client.5067 shutdown 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 -- 172.21.15.60:0/2771980643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 msgr2=0x5588699c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x5588699c3b30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f88c0001330 tx=0x7f88c00208a0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 -- 172.21.15.60:0/2771980643 shutdown_connections 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.60:6834/846626570,v1:172.21.15.60:6835/846626570] conn(0x558869a8fe00 0x558869a280a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558869a0e8b0 0x5588699c3b30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558869a07df0 0x558869a74e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 --2- 172.21.15.60:0/2771980643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588699c9fa0 0x558869a03820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88aea00640 0 client.5067 ms_handle_reset on v2:172.21.15.60:6834/846626570 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 -- 172.21.15.60:0/2771980643 >> 172.21.15.60:0/2771980643 conn(0x5588699c9a20 msgr2=0x5588699eec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 -- 172.21.15.60:0/2771980643 shutdown_connections 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 1 -- 172.21.15.60:0/2771980643 wait complete. 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.097+0000 7f88d3620740 20 client.5067 trim_cache size 0 max 16384 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4881, 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:39:44.103 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:39:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 10.950086926999999, 2024-09-16T15:39:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 17, 2024-09-16T15:39:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 36, 2024-09-16T15:39:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 36, 2024-09-16T15:39:44.104 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 19.811539583999998 2024-09-16T15:39:44.104 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 -- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88a8078850 msgr2=0x7f88a807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88a8078850 0x7f88a807ad10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f88b4034000 tx=0x7f88b4004730 comp rx=0 tx=0).stop 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 -- 172.21.15.60:0/3153320733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c8073fb0 msgr2=0x7f88c81f5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c8073fb0 0x7f88c81f5920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c430 tx=0x7f88bc00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 -- 172.21.15.60:0/3153320733 shutdown_connections 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f88a8078850 0x7f88a807ad10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88c815f5a0 0x7f88c80c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88c80ccf00 0x7f88c80c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 --2- 172.21.15.60:0/3153320733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88c8073fb0 0x7f88c81f5920 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 -- 172.21.15.60:0/3153320733 >> 172.21.15.60:0/3153320733 conn(0x7f88c80b1950 msgr2=0x7f88c815cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 -- 172.21.15.60:0/3153320733 shutdown_connections 2024-09-16T15:39:44.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.098+0000 7f88d0600640 1 -- 172.21.15.60:0/3153320733 wait complete. 2024-09-16T15:39:44.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:39:44.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.277+0000 7f7616c00640 1 -- 172.21.15.60:0/4135175844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76100d1050 msgr2=0x7f76100d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.277+0000 7f7616c00640 1 --2- 172.21.15.60:0/4135175844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76100d1050 0x7f76100d23e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7608005fd0 tx=0x7f760802a930 comp rx=0 tx=0).stop 2024-09-16T15:39:44.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.277+0000 7f7616c00640 1 -- 172.21.15.60:0/4135175844 shutdown_connections 2024-09-16T15:39:44.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.277+0000 7f7616c00640 1 --2- 172.21.15.60:0/4135175844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76100d1050 0x7f76100d23e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.277+0000 7f7616c00640 1 -- 172.21.15.60:0/4135175844 >> 172.21.15.60:0/4135175844 conn(0x7f76100d3fa0 msgr2=0x7f76100d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.277+0000 7f7616c00640 1 -- 172.21.15.60:0/4135175844 shutdown_connections 2024-09-16T15:39:44.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.277+0000 7f7616c00640 1 -- 172.21.15.60:0/4135175844 wait complete. 2024-09-16T15:39:44.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.279+0000 7f7616c00640 1 Processor -- start 2024-09-16T15:39:44.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.279+0000 7f7616c00640 1 -- start start 2024-09-16T15:39:44.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f7616c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76100d1050 0x7f76101f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f7616c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76100d4ac0 0x7f76100b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f7616c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f761015e090 0x7f761014f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f7616c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f76100cf960 con 0x7f76100d4ac0 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f7616c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76100d2920 con 0x7f761015e090 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f7616c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f76100cfae0 con 0x7f76100d1050 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f760f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76100d4ac0 0x7f76100b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f760f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76100d4ac0 0x7f76100b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57142/0 (socket says 172.21.15.60:57142) 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.280+0000 7f760f400640 1 -- 172.21.15.60:0/4067110439 learned_addr learned my addr 172.21.15.60:0/4067110439 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f760fe00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76100d1050 0x7f76101f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f7614c00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f761015e090 0x7f761014f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f760f400640 1 -- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76100d1050 msgr2=0x7f76101f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f760f400640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76100d1050 0x7f76101f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f760f400640 1 -- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f761015e090 msgr2=0x7f761014f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f760f400640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f761015e090 0x7f761014f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f760f400640 1 -- 172.21.15.60:0/4067110439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76100b77c0 con 0x7f76100d4ac0 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f760f400640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76100d4ac0 0x7f76100b7280 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00c460 tx=0x7f75fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.281+0000 7f7607e00640 1 -- 172.21.15.60:0/4067110439 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc011790 con 0x7f76100d4ac0 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.282+0000 7f7607e00640 1 -- 172.21.15.60:0/4067110439 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75fc017070 con 0x7f76100d4ac0 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.282+0000 7f7607e00640 1 -- 172.21.15.60:0/4067110439 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc011cc0 con 0x7f76100d4ac0 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.282+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761014f910 con 0x7f76100d4ac0 2024-09-16T15:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.282+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76101f4df0 con 0x7f76100d4ac0 2024-09-16T15:39:44.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.283+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76100cff90 con 0x7f76100d4ac0 2024-09-16T15:39:44.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.284+0000 7f7607e00640 1 -- 172.21.15.60:0/4067110439 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f75fc01c460 con 0x7f76100d4ac0 2024-09-16T15:39:44.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.285+0000 7f7607e00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f75f0078850 0x7f75f007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.285+0000 7f7607e00640 1 -- 172.21.15.60:0/4067110439 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f75fc09a6e0 con 0x7f76100d4ac0 2024-09-16T15:39:44.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.285+0000 7f760fe00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f75f0078850 0x7f75f007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.286+0000 7f760fe00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f75f0078850 0x7f75f007ad10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7600000f60 tx=0x7f7600009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.289+0000 7f7607e00640 1 -- 172.21.15.60:0/4067110439 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f75fc062660 con 0x7f76100d4ac0 2024-09-16T15:39:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.527+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f76100696a0 con 0x7f76100d4ac0 2024-09-16T15:39:44.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.527+0000 7f7607e00640 1 -- 172.21.15.60:0/4067110439 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f75fc067090 con 0x7f76100d4ac0 2024-09-16T15:39:44.529 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f75f0078850 msgr2=0x7f75f007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f75f0078850 0x7f75f007ad10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7600000f60 tx=0x7f7600009040 comp rx=0 tx=0).stop 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76100d4ac0 msgr2=0x7f76100b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76100d4ac0 0x7f76100b7280 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00c460 tx=0x7f75fc00c920 comp rx=0 tx=0).stop 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 shutdown_connections 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f75f0078850 0x7f75f007ad10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f761015e090 0x7f761014f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76100d4ac0 0x7f76100b7280 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 --2- 172.21.15.60:0/4067110439 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76100d1050 0x7f76101f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 >> 172.21.15.60:0/4067110439 conn(0x7f76100d3fa0 msgr2=0x7f76100b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 shutdown_connections 2024-09-16T15:39:44.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44.530+0000 7f7616c00640 1 -- 172.21.15.60:0/4067110439 wait complete. 2024-09-16T15:39:44.548 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:39:44.548 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:39:44.548 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:39:44.548 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:39:44.548 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:44.548 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:39:44.548 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:39:44.548 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:39:44.549 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:39:44.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:44.549 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:39:44.584 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:39:44.585 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42665sec preferred_lft 42665sec 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.586 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:44.587 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:39:44.678 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:39:44.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:44.681 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:39:44.718 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:39:44.718 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:39:44.718 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:39:44.718 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:39:44.718 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:39:44.719 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:44.719 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:39:44.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:39:44.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:39:44.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:39:44.719 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:39:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:44.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:44.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:44.917 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:39:44.953 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:39:44.953 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:39:44.953 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:39:44.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:44.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:39:45.049 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:45.049 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:45.085 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:39:45.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:45.085 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:45.159 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:39:45.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:45.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:45.288 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:45.289 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:39:45.316 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:45.316 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:39:45.316 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:39:45.316 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:39:45.316 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:39:45.317 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:39:45.318 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:39:45.318 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:39:45.318 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:39:45.318 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:39:45.321 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:39:45.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:45.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:45.377 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:45.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:45.475 INFO:teuthology.orchestra.run.smithi060.stdout:client5095 2024-09-16T15:39:45.476 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:45.476 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:45.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:39:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.711+0000 7fb14ee00640 1 -- 172.21.15.60:0/2061614464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 msgr2=0x7fb148074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.711+0000 7fb14ee00640 1 --2- 172.21.15.60:0/2061614464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 0x7fb148074440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb140005fd0 tx=0x7fb14002aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.711+0000 7fb14ee00640 1 -- 172.21.15.60:0/2061614464 shutdown_connections 2024-09-16T15:39:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.711+0000 7fb14ee00640 1 --2- 172.21.15.60:0/2061614464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 0x7fb148074440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.711+0000 7fb14ee00640 1 --2- 172.21.15.60:0/2061614464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1480ccf00 0x7fb1480b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.711+0000 7fb14ee00640 1 -- 172.21.15.60:0/2061614464 >> 172.21.15.60:0/2061614464 conn(0x7fb1480b1950 msgr2=0x7fb1480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.711+0000 7fb14ee00640 1 -- 172.21.15.60:0/2061614464 shutdown_connections 2024-09-16T15:39:45.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.711+0000 7fb14ee00640 1 -- 172.21.15.60:0/2061614464 wait complete. 2024-09-16T15:39:45.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.713+0000 7fb14ee00640 1 Processor -- start 2024-09-16T15:39:45.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.713+0000 7fb14ee00640 1 -- start start 2024-09-16T15:39:45.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb14ee00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 0x7fb1481f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb14ee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1480ccf00 0x7fb1481f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb14ee00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb14815f5a0 0x7fb1481ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb14ee00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb14806f2e0 con 0x7fb148073fb0 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb14ee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb14806f160 con 0x7fb1480ccf00 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb14ee00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb14806f460 con 0x7fb14815f5a0 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb147e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 0x7fb1481f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb147e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 0x7fb1481f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57168/0 (socket says 172.21.15.60:57168) 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb147e00640 1 -- 172.21.15.60:0/50301531 learned_addr learned my addr 172.21.15.60:0/50301531 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.714+0000 7fb13f400640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1480ccf00 0x7fb1481f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb14ce00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb14815f5a0 0x7fb1481ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb147e00640 1 -- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb14815f5a0 msgr2=0x7fb1481ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb147e00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb14815f5a0 0x7fb1481ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb147e00640 1 -- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1480ccf00 msgr2=0x7fb1481f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb147e00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1480ccf00 0x7fb1481f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:45.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb147e00640 1 -- 172.21.15.60:0/50301531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1481f58b0 con 0x7fb148073fb0 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb14ce00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb14815f5a0 0x7fb1481ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb13f400640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1480ccf00 0x7fb1481f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb147e00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 0x7fb1481f5370 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb13800c3a0 tx=0x7fb13800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb144c00640 1 -- 172.21.15.60:0/50301531 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138017070 con 0x7fb148073fb0 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb144c00640 1 -- 172.21.15.60:0/50301531 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1380053c0 con 0x7fb148073fb0 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb144c00640 1 -- 172.21.15.60:0/50301531 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138003850 con 0x7fb148073fb0 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1480cebd0 con 0x7fb148073fb0 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.715+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1481590b0 con 0x7fb148073fb0 2024-09-16T15:39:45.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.716+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14806f9f0 con 0x7fb148073fb0 2024-09-16T15:39:45.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.718+0000 7fb144c00640 1 -- 172.21.15.60:0/50301531 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb13801a070 con 0x7fb148073fb0 2024-09-16T15:39:45.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.719+0000 7fb144c00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb124078880 0x7fb12407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:45.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.719+0000 7fb144c00640 1 -- 172.21.15.60:0/50301531 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fb138099d70 con 0x7fb148073fb0 2024-09-16T15:39:45.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.719+0000 7fb13f400640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb124078880 0x7fb12407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:45.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.720+0000 7fb13f400640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb124078880 0x7fb12407ad40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1300058c0 tx=0x7fb130005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:45.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.723+0000 7fb144c00640 1 -- 172.21.15.60:0/50301531 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb1481f58b0 con 0x7fb148073fb0 2024-09-16T15:39:45.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.958+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb1480b4170 con 0x7fb148073fb0 2024-09-16T15:39:45.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.958+0000 7fb144c00640 1 -- 172.21.15.60:0/50301531 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fb138062340 con 0x7fb148073fb0 2024-09-16T15:39:45.959 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb124078880 msgr2=0x7fb12407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb124078880 0x7fb12407ad40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1300058c0 tx=0x7fb130005850 comp rx=0 tx=0).stop 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 msgr2=0x7fb1481f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 0x7fb1481f5370 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb13800c3a0 tx=0x7fb13800c860 comp rx=0 tx=0).stop 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 shutdown_connections 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb124078880 0x7fb12407ad40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb14815f5a0 0x7fb1481ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1480ccf00 0x7fb1481f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:45.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.960+0000 7fb14ee00640 1 --2- 172.21.15.60:0/50301531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb148073fb0 0x7fb1481f5370 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:45.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.961+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 >> 172.21.15.60:0/50301531 conn(0x7fb1480b1950 msgr2=0x7fb14809ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:45.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.961+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 shutdown_connections 2024-09-16T15:39:45.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:45.961+0000 7fb14ee00640 1 -- 172.21.15.60:0/50301531 wait complete. 2024-09-16T15:39:45.978 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:39:45.978 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:39:45.978 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:39:45.978 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:39:45.978 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:45.978 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:39:45.978 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:39:45.978 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:39:45.979 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:39:45.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:45.979 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:46.021 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42663sec preferred_lft 42663sec 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:39:46.022 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:46.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:46.024 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:39:46.118 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:39:46.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:46.121 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:39:46.158 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:39:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:39:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:39:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:39:46.159 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:39:46.159 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:46.159 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:39:46.159 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:39:46.159 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:39:46.159 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:39:46.159 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:39:46.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:46.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:46.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:46.357 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:39:46.392 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:39:46.392 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:39:46.392 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:39:46.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:46.392 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:39:46.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:46.488 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:46.523 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:39:46.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:46.523 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:46.597 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:39:46.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:46.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:46.726 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:46.726 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:39:46.755 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:39:46.755 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:39:46.755 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:39:46.755 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:39:46.756 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:39:46.757 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:39:46.757 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:39:46.757 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:39:46.757 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:39:46.757 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:39:46.760 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:39:46.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:46.760 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:46.816 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:46.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:46.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:39:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:46.907 INFO:teuthology.orchestra.run.smithi100.stdout:client5082 2024-09-16T15:39:46.907 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:46.907 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:46.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:39:47.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.119+0000 7f4f0fc00640 1 -- 172.21.15.60:0/1578149681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f0806f220 msgr2=0x7f4f0806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:47.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.119+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/1578149681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f0806f220 0x7f4f0806f6b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4f080b2680 tx=0x7f4f0002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.119+0000 7f4f0fc00640 1 -- 172.21.15.60:0/1578149681 shutdown_connections 2024-09-16T15:39:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.119+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/1578149681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f0806f220 0x7f4f0806f6b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.119+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/1578149681 >> v2:172.21.15.100:3300/0 conn(0x7f4f08159aa0 0x7f4f080741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.119+0000 7f4f0fc00640 1 -- 172.21.15.60:0/1578149681 >> 172.21.15.60:0/1578149681 conn(0x7f4f080b18e0 msgr2=0x7f4f080b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.119+0000 7f4f0fc00640 1 -- 172.21.15.60:0/1578149681 shutdown_connections 2024-09-16T15:39:47.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.119+0000 7f4f0fc00640 1 -- 172.21.15.60:0/1578149681 wait complete. 2024-09-16T15:39:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.121+0000 7f4f0fc00640 1 Processor -- start 2024-09-16T15:39:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.121+0000 7f4f0fc00640 1 -- start start 2024-09-16T15:39:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0806f220 0x7f4f080ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0fc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f08159aa0 0x7f4f081ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0fc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f0815cff0 0x7f4f080d0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0fc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4f080cc520 con 0x7f4f0815cff0 2024-09-16T15:39:47.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f080cc3a0 con 0x7f4f0806f220 2024-09-16T15:39:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0fc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4f080cc6a0 con 0x7f4f08159aa0 2024-09-16T15:39:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0806f220 0x7f4f080ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4effe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f08159aa0 0x7f4f081ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f0815cff0 0x7f4f080d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4effe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f08159aa0 0x7f4f081ebaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59284/0 (socket says 172.21.15.60:59284) 2024-09-16T15:39:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4f0d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0806f220 0x7f4f080ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37842/0 (socket says 172.21.15.60:37842) 2024-09-16T15:39:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.122+0000 7f4effe00640 1 -- 172.21.15.60:0/3495976008 learned_addr learned my addr 172.21.15.60:0/3495976008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4effe00640 1 -- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0806f220 msgr2=0x7f4f080ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4effe00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0806f220 0x7f4f080ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4effe00640 1 -- 172.21.15.60:0/3495976008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f0815cff0 msgr2=0x7f4f080d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4effe00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f0815cff0 0x7f4f080d0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4effe00640 1 -- 172.21.15.60:0/3495976008 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f080d09d0 con 0x7f4f08159aa0 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4effe00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f08159aa0 0x7f4f081ebaa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c430 tx=0x7f4ef000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4f05600640 1 -- 172.21.15.60:0/3495976008 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0011790 con 0x7f4f08159aa0 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4f05600640 1 -- 172.21.15.60:0/3495976008 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ef0017070 con 0x7f4f08159aa0 2024-09-16T15:39:47.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4f05600640 1 -- 172.21.15.60:0/3495976008 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0011ca0 con 0x7f4f08159aa0 2024-09-16T15:39:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f080ced60 con 0x7f4f08159aa0 2024-09-16T15:39:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.123+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f080b32d0 con 0x7f4f08159aa0 2024-09-16T15:39:47.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.124+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08069360 con 0x7f4f08159aa0 2024-09-16T15:39:47.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.126+0000 7f4f05600640 1 -- 172.21.15.60:0/3495976008 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4ef001c460 con 0x7f4f08159aa0 2024-09-16T15:39:47.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.127+0000 7f4f05600640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4ee4078850 0x7f4ee407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:47.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.127+0000 7f4f05600640 1 -- 172.21.15.60:0/3495976008 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f4ef009a9b0 con 0x7f4f08159aa0 2024-09-16T15:39:47.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.127+0000 7f4f0d200640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4ee4078850 0x7f4ee407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:47.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.128+0000 7f4f0d200640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4ee4078850 0x7f4ee407ad10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f080b2e00 tx=0x7f4ef8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.131+0000 7f4f05600640 1 -- 172.21.15.60:0/3495976008 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4ef00629c0 con 0x7f4f08159aa0 2024-09-16T15:39:47.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:47.368+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f4f080ccba0 con 0x7f4f08159aa0 2024-09-16T15:39:48.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.219+0000 7f4f05600640 1 -- 172.21.15.60:0/3495976008 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7f4ef00673f0 con 0x7f4f08159aa0 2024-09-16T15:39:48.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4ee4078850 msgr2=0x7f4ee407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4ee4078850 0x7f4ee407ad10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f080b2e00 tx=0x7f4ef8005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f08159aa0 msgr2=0x7f4f081ebaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f08159aa0 0x7f4f081ebaa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c430 tx=0x7f4ef000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 shutdown_connections 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4ee4078850 0x7f4ee407ad10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f0815cff0 0x7f4f080d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f08159aa0 0x7f4f081ebaa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 --2- 172.21.15.60:0/3495976008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0806f220 0x7f4f080ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 >> 172.21.15.60:0/3495976008 conn(0x7f4f080b18e0 msgr2=0x7f4f080b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 shutdown_connections 2024-09-16T15:39:48.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.222+0000 7f4f0fc00640 1 -- 172.21.15.60:0/3495976008 wait complete. 2024-09-16T15:39:48.243 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:39:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.408+0000 7f53b1800640 1 -- 172.21.15.60:0/2280305946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac15f5a0 msgr2=0x7f53ac0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.408+0000 7f53b1800640 1 --2- 172.21.15.60:0/2280305946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac15f5a0 0x7f53ac0b2440 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f53ac06dac0 tx=0x7f539c02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.408+0000 7f53b1800640 1 -- 172.21.15.60:0/2280305946 shutdown_connections 2024-09-16T15:39:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.408+0000 7f53b1800640 1 --2- 172.21.15.60:0/2280305946 >> v2:172.21.15.100:3300/0 conn(0x7f53ac073fb0 0x7f53ac074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.408+0000 7f53b1800640 1 --2- 172.21.15.60:0/2280305946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac15f5a0 0x7f53ac0b2440 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.409+0000 7f53b1800640 1 -- 172.21.15.60:0/2280305946 >> 172.21.15.60:0/2280305946 conn(0x7f53ac0b1950 msgr2=0x7f53ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.409+0000 7f53b1800640 1 -- 172.21.15.60:0/2280305946 shutdown_connections 2024-09-16T15:39:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.409+0000 7f53b1800640 1 -- 172.21.15.60:0/2280305946 wait complete. 2024-09-16T15:39:48.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.410+0000 7f53b1800640 1 Processor -- start 2024-09-16T15:39:48.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.410+0000 7f53b1800640 1 -- start start 2024-09-16T15:39:48.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.411+0000 7f53b1800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac073fb0 0x7f53ac14bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.411+0000 7f53b1800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ccf00 0x7f53ac0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.411+0000 7f53b1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15f5a0 0x7f53ac0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.411+0000 7f53b1800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac06f2e0 con 0x7f53ac0ccf00 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.411+0000 7f53b1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac06f160 con 0x7f53ac15f5a0 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.411+0000 7f53b1800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f53ac06f460 con 0x7f53ac073fb0 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.411+0000 7f53a2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ccf00 0x7f53ac0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ccf00 0x7f53ac0b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57208/0 (socket says 172.21.15.60:57208) 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a2000640 1 -- 172.21.15.60:0/3102090628 learned_addr learned my addr 172.21.15.60:0/3102090628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53aaa00640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac073fb0 0x7f53ac14bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53ab400640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15f5a0 0x7f53ac0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:48.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a2000640 1 -- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac073fb0 msgr2=0x7f53ac14bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a2000640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac073fb0 0x7f53ac14bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a2000640 1 -- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15f5a0 msgr2=0x7f53ac0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a2000640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15f5a0 0x7f53ac0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a2000640 1 -- 172.21.15.60:0/3102090628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac1f2ad0 con 0x7f53ac0ccf00 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a2000640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ccf00 0x7f53ac0b1d30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f539400c500 tx=0x7f539400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a3400640 1 -- 172.21.15.60:0/3102090628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394017070 con 0x7f53ac0ccf00 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a3400640 1 -- 172.21.15.60:0/3102090628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53940035f0 con 0x7f53ac0ccf00 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ac151b10 con 0x7f53ac0ccf00 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.412+0000 7f53a3400640 1 -- 172.21.15.60:0/3102090628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394003a40 con 0x7f53ac0ccf00 2024-09-16T15:39:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.413+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ac1520d0 con 0x7f53ac0ccf00 2024-09-16T15:39:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.414+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ac06f9f0 con 0x7f53ac0ccf00 2024-09-16T15:39:48.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.415+0000 7f53a3400640 1 -- 172.21.15.60:0/3102090628 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f539401a070 con 0x7f53ac0ccf00 2024-09-16T15:39:48.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.416+0000 7f53a3400640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5388078850 0x7f538807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:48.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.416+0000 7f53a3400640 1 -- 172.21.15.60:0/3102090628 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f5394099d70 con 0x7f53ac0ccf00 2024-09-16T15:39:48.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.416+0000 7f53aaa00640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5388078850 0x7f538807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:48.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.417+0000 7f53aaa00640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5388078850 0x7f538807ad10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f539c034000 tx=0x7f539c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:48.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.420+0000 7f53a3400640 1 -- 172.21.15.60:0/3102090628 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5394061d80 con 0x7f53ac0ccf00 2024-09-16T15:39:48.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.638+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f53ac0b0160 con 0x7f5388078850 2024-09-16T15:39:48.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.640+0000 7f53a3400640 1 -- 172.21.15.60:0/3102090628 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f53ac0b0160 con 0x7f5388078850 2024-09-16T15:39:48.642 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:39:48.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:39:48.642 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:39:48.642 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:39:48.642 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:39:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5388078850 msgr2=0x7f538807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5388078850 0x7f538807ad10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f539c034000 tx=0x7f539c004730 comp rx=0 tx=0).stop 2024-09-16T15:39:48.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ccf00 msgr2=0x7f53ac0b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ccf00 0x7f53ac0b1d30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f539400c500 tx=0x7f539400c9c0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 shutdown_connections 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5388078850 0x7f538807ad10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53ac15f5a0 0x7f53ac0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53ac0ccf00 0x7f53ac0b1d30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 --2- 172.21.15.60:0/3102090628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53ac073fb0 0x7f53ac14bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 >> 172.21.15.60:0/3102090628 conn(0x7f53ac0b1950 msgr2=0x7f53ac0ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 shutdown_connections 2024-09-16T15:39:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.643+0000 7f53b1800640 1 -- 172.21.15.60:0/3102090628 wait complete. 2024-09-16T15:39:48.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:39:48.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.823+0000 7f976f400640 1 -- 172.21.15.60:0/3060571997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97680cccd0 msgr2=0x7f9768073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.823+0000 7f976f400640 1 --2- 172.21.15.60:0/3060571997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97680cccd0 0x7f9768073440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9750009860 tx=0x7f975002a510 comp rx=0 tx=0).stop 2024-09-16T15:39:48.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.823+0000 7f976f400640 1 -- 172.21.15.60:0/3060571997 shutdown_connections 2024-09-16T15:39:48.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.823+0000 7f976f400640 1 --2- 172.21.15.60:0/3060571997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97680cccd0 0x7f9768073440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.823+0000 7f976f400640 1 -- 172.21.15.60:0/3060571997 >> 172.21.15.60:0/3060571997 conn(0x7f97680b1840 msgr2=0x7f97680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:48.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.823+0000 7f976f400640 1 -- 172.21.15.60:0/3060571997 shutdown_connections 2024-09-16T15:39:48.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.823+0000 7f976f400640 1 -- 172.21.15.60:0/3060571997 wait complete. 2024-09-16T15:39:48.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.824+0000 7f976f400640 1 Processor -- start 2024-09-16T15:39:48.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.825+0000 7f976f400640 1 -- start start 2024-09-16T15:39:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.825+0000 7f976f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f976806efa0 0x7f97681f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97680cccd0 0x7f97680c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976f400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9768145750 0x7f9768141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976f400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f97680ce6d0 con 0x7f97680cccd0 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97680ce550 con 0x7f976806efa0 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976f400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f97680ce850 con 0x7f9768145750 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f975fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97680cccd0 0x7f97680c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f975fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97680cccd0 0x7f97680c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59506/0 (socket says 172.21.15.60:59506) 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f975fe00640 1 -- 172.21.15.60:0/2757357283 learned_addr learned my addr 172.21.15.60:0/2757357283 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976ca00640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f976806efa0 0x7f97681f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f975fe00640 1 -- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9768145750 msgr2=0x7f9768141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976d400640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9768145750 0x7f9768141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f975fe00640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9768145750 0x7f9768141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f975fe00640 1 -- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f976806efa0 msgr2=0x7f97681f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f975fe00640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f976806efa0 0x7f97681f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f975fe00640 1 -- 172.21.15.60:0/2757357283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97680c59f0 con 0x7f97680cccd0 2024-09-16T15:39:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976ca00640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f976806efa0 0x7f97681f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.826+0000 7f976d400640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9768145750 0x7f9768141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.827+0000 7f975fe00640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97680cccd0 0x7f97680c4f10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9750009980 tx=0x7f9750032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.827+0000 7f9765600640 1 -- 172.21.15.60:0/2757357283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750040070 con 0x7f97680cccd0 2024-09-16T15:39:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.827+0000 7f9765600640 1 -- 172.21.15.60:0/2757357283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97500038e0 con 0x7f97680cccd0 2024-09-16T15:39:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.827+0000 7f9765600640 1 -- 172.21.15.60:0/2757357283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750003cf0 con 0x7f97680cccd0 2024-09-16T15:39:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.827+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97680c5cd0 con 0x7f97680cccd0 2024-09-16T15:39:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.827+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97680c6290 con 0x7f97680cccd0 2024-09-16T15:39:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.828+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97680ced00 con 0x7f97680cccd0 2024-09-16T15:39:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.829+0000 7f9765600640 1 -- 172.21.15.60:0/2757357283 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9750039040 con 0x7f97680cccd0 2024-09-16T15:39:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.830+0000 7f9765600640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9744078850 0x7f974407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.830+0000 7f9765600640 1 -- 172.21.15.60:0/2757357283 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f97500bbd90 con 0x7f97680cccd0 2024-09-16T15:39:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.831+0000 7f976ca00640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9744078850 0x7f974407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.832+0000 7f976ca00640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9744078850 0x7f974407ad10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9758001120 tx=0x7f9758009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:48.835+0000 7f9765600640 1 -- 172.21.15.60:0/2757357283 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9750084360 con 0x7f97680cccd0 2024-09-16T15:39:49.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.032+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f97680cf390 con 0x7f97680cccd0 2024-09-16T15:39:49.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.033+0000 7f9765600640 1 -- 172.21.15.60:0/2757357283 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f9750083c60 con 0x7f97680cccd0 2024-09-16T15:39:49.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9744078850 msgr2=0x7f974407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:49.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9744078850 0x7f974407ad10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9758001120 tx=0x7f9758009040 comp rx=0 tx=0).stop 2024-09-16T15:39:49.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97680cccd0 msgr2=0x7f97680c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:49.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97680cccd0 0x7f97680c4f10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9750009980 tx=0x7f9750032b90 comp rx=0 tx=0).stop 2024-09-16T15:39:49.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 shutdown_connections 2024-09-16T15:39:49.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9744078850 0x7f974407ad10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9768145750 0x7f9768141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97680cccd0 0x7f97680c4f10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 --2- 172.21.15.60:0/2757357283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f976806efa0 0x7f97681f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.035+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 >> 172.21.15.60:0/2757357283 conn(0x7f97680b1840 msgr2=0x7f97680aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:49.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.036+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 shutdown_connections 2024-09-16T15:39:49.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.036+0000 7f976f400640 1 -- 172.21.15.60:0/2757357283 wait complete. 2024-09-16T15:39:49.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:49.053 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp42980/subvol81730) 2024-09-16T15:39:49.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:49.118 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:39:49.154 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:39:49.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:49.155 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp42980/subvol81730) 2024-09-16T15:39:49.263 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:49.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp42980/subvol81730/.meta 2024-09-16T15:39:49.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol81730 subvol_grp42980 2024-09-16T15:39:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.528+0000 7f2a14800640 1 -- 172.21.15.60:0/1643674214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c15cff0 msgr2=0x7f2a0c06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.528+0000 7f2a14800640 1 --2- 172.21.15.60:0/1643674214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c15cff0 0x7f2a0c06e6f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009830 tx=0x7f29fc02a510 comp rx=0 tx=0).stop 2024-09-16T15:39:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.528+0000 7f2a14800640 1 -- 172.21.15.60:0/1643674214 shutdown_connections 2024-09-16T15:39:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.528+0000 7f2a14800640 1 --2- 172.21.15.60:0/1643674214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a0c06f220 0x7f2a0c06f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.528+0000 7f2a14800640 1 --2- 172.21.15.60:0/1643674214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c15cff0 0x7f2a0c06e6f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.528+0000 7f2a14800640 1 -- 172.21.15.60:0/1643674214 >> 172.21.15.60:0/1643674214 conn(0x7f2a0c0b18e0 msgr2=0x7f2a0c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.528+0000 7f2a14800640 1 -- 172.21.15.60:0/1643674214 shutdown_connections 2024-09-16T15:39:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.528+0000 7f2a14800640 1 -- 172.21.15.60:0/1643674214 wait complete. 2024-09-16T15:39:49.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.530+0000 7f2a14800640 1 Processor -- start 2024-09-16T15:39:49.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.530+0000 7f2a14800640 1 -- start start 2024-09-16T15:39:49.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a14800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a0c06f220 0x7f2a0c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:49.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a14800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c159aa0 0x7f2a0c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a14800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a0c15cff0 0x7f2a0c1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a14800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c0cc520 con 0x7f2a0c159aa0 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a14800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c0cc3a0 con 0x7f2a0c15cff0 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a14800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0c0cc6a0 con 0x7f2a0c06f220 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a11400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c159aa0 0x7f2a0c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a11400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c159aa0 0x7f2a0c0ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59514/0 (socket says 172.21.15.60:59514) 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a11400640 1 -- 172.21.15.60:0/3787947996 learned_addr learned my addr 172.21.15.60:0/3787947996 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a12800640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a0c15cff0 0x7f2a0c1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:49.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.531+0000 7f2a11e00640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a0c06f220 0x7f2a0c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:49.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a11400640 1 -- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a0c06f220 msgr2=0x7f2a0c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a11400640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a0c06f220 0x7f2a0c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a11400640 1 -- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a0c15cff0 msgr2=0x7f2a0c1f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a11400640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a0c15cff0 0x7f2a0c1f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a11400640 1 -- 172.21.15.60:0/3787947996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c0b05a0 con 0x7f2a0c159aa0 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a11400640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c159aa0 0x7f2a0c0ae2c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009950 tx=0x7f29fc033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a06000640 1 -- 172.21.15.60:0/3787947996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc005c90 con 0x7f2a0c159aa0 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a06000640 1 -- 172.21.15.60:0/3787947996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29fc031e50 con 0x7f2a0c159aa0 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.532+0000 7f2a06000640 1 -- 172.21.15.60:0/3787947996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0332f0 con 0x7f2a0c159aa0 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.533+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c0b0880 con 0x7f2a0c159aa0 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.533+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c157370 con 0x7f2a0c159aa0 2024-09-16T15:39:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.534+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c0ccbc0 con 0x7f2a0c159aa0 2024-09-16T15:39:49.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.535+0000 7f2a06000640 1 -- 172.21.15.60:0/3787947996 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f29fc033490 con 0x7f2a0c159aa0 2024-09-16T15:39:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.536+0000 7f2a06000640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29f0078850 0x7f29f007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.536+0000 7f2a06000640 1 -- 172.21.15.60:0/3787947996 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f29fc0bbb70 con 0x7f2a0c159aa0 2024-09-16T15:39:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.536+0000 7f2a11e00640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29f0078850 0x7f29f007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:49.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.537+0000 7f2a11e00640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29f0078850 0x7f29f007ad10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2a00007fd0 tx=0x7f2a00009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:49.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.540+0000 7f2a06000640 1 -- 172.21.15.60:0/3787947996 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f29fc03f030 con 0x7f2a0c159aa0 2024-09-16T15:39:49.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.764+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol81730", "group_name": "subvol_grp42980", "target": ["mon-mgr", ""]}) -- 0x7f2a0c156bc0 con 0x7f29f0078850 2024-09-16T15:39:49.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:49.771+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:49.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:49.771+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:49.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:49.771+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:49.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:49.771+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:49.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:49.771+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:49.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.813+0000 7f2a06000640 1 -- 172.21.15.60:0/3787947996 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2a0c156bc0 con 0x7f29f0078850 2024-09-16T15:39:49.814 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/subvol_grp42980/subvol81730 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29f0078850 msgr2=0x7f29f007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29f0078850 0x7f29f007ad10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2a00007fd0 tx=0x7f2a00009040 comp rx=0 tx=0).stop 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c159aa0 msgr2=0x7f2a0c0ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c159aa0 0x7f2a0c0ae2c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009950 tx=0x7f29fc033c90 comp rx=0 tx=0).stop 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 shutdown_connections 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f29f0078850 0x7f29f007ad10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a0c15cff0 0x7f2a0c1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0c159aa0 0x7f2a0c0ae2c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 --2- 172.21.15.60:0/3787947996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a0c06f220 0x7f2a0c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.815+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 >> 172.21.15.60:0/3787947996 conn(0x7f2a0c0b18e0 msgr2=0x7f2a0c15c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.816+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 shutdown_connections 2024-09-16T15:39:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.816+0000 7f2a14800640 1 -- 172.21.15.60:0/3787947996 wait complete. 2024-09-16T15:39:49.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol81730 subvol_grp42980 2024-09-16T15:39:49.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.997+0000 7fc9d5e00640 1 -- 172.21.15.60:0/2041599311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4004d40 msgr2=0x7fc9b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:49.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.997+0000 7fc9d5e00640 1 --2- 172.21.15.60:0/2041599311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9b4004d40 0x7fc9b40051c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800c610 tx=0x7fc9b4005830 comp rx=0 tx=0).stop 2024-09-16T15:39:49.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.997+0000 7fc9d5400640 1 -- 172.21.15.60:0/2041599311 reap_dead start 2024-09-16T15:39:49.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.997+0000 7fc9d5e00640 1 -- 172.21.15.60:0/2041599311 shutdown_connections 2024-09-16T15:39:49.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.997+0000 7fc9d5e00640 1 -- 172.21.15.60:0/2041599311 >> 172.21.15.60:0/2041599311 conn(0x7fc9d00cfa70 msgr2=0x7fc9d00cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:49.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.997+0000 7fc9d5e00640 1 -- 172.21.15.60:0/2041599311 shutdown_connections 2024-09-16T15:39:49.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.997+0000 7fc9d5e00640 1 -- 172.21.15.60:0/2041599311 wait complete. 2024-09-16T15:39:50.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.999+0000 7fc9d5e00640 1 Processor -- start 2024-09-16T15:39:50.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:49.999+0000 7fc9d5e00640 1 -- start start 2024-09-16T15:39:50.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b4004d40 0x7fc9d015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:50.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d01f6520 0x7fc9d01f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9d5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d00c8040 0x7fc9d00c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9d5e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0073010 con 0x7fc9d01f6520 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9d5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0073190 con 0x7fc9d00c8040 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9d5e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc9d0073310 con 0x7fc9b4004d40 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9c7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d01f6520 0x7fc9d01f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9c7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d01f6520 0x7fc9d01f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59538/0 (socket says 172.21.15.60:59538) 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9c7e00640 1 -- 172.21.15.60:0/3586517174 learned_addr learned my addr 172.21.15.60:0/3586517174 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9d4a00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b4004d40 0x7fc9d015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.000+0000 7fc9d5400640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d00c8040 0x7fc9d00c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9c7e00640 1 -- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b4004d40 msgr2=0x7fc9d015ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9c7e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b4004d40 0x7fc9d015ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9c7e00640 1 -- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d00c8040 msgr2=0x7fc9d00c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9c7e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d00c8040 0x7fc9d00c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9c7e00640 1 -- 172.21.15.60:0/3586517174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d015e0d0 con 0x7fc9d01f6520 2024-09-16T15:39:50.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9c7e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d01f6520 0x7fc9d01f1d30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8011ae0 tx=0x7fc9b8011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:50.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9cd600640 1 -- 172.21.15.60:0/3586517174 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b800fd30 con 0x7fc9d01f6520 2024-09-16T15:39:50.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9cd600640 1 -- 172.21.15.60:0/3586517174 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9b8004020 con 0x7fc9d01f6520 2024-09-16T15:39:50.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d015e3b0 con 0x7fc9d01f6520 2024-09-16T15:39:50.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9cd600640 1 -- 172.21.15.60:0/3586517174 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8004430 con 0x7fc9d01f6520 2024-09-16T15:39:50.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.001+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d015ead0 con 0x7fc9d01f6520 2024-09-16T15:39:50.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.002+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d006fe40 con 0x7fc9d01f6520 2024-09-16T15:39:50.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.004+0000 7fc9cd600640 1 -- 172.21.15.60:0/3586517174 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc9b8020060 con 0x7fc9d01f6520 2024-09-16T15:39:50.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.005+0000 7fc9cd600640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc9ac078850 0x7fc9ac07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:50.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.005+0000 7fc9cd600640 1 -- 172.21.15.60:0/3586517174 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc9b801e070 con 0x7fc9d01f6520 2024-09-16T15:39:50.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.005+0000 7fc9d4a00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc9ac078850 0x7fc9ac07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:50.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.006+0000 7fc9d4a00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc9ac078850 0x7fc9ac07ad10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009c50 tx=0x7fc9c0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:50.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.009+0000 7fc9cd600640 1 -- 172.21.15.60:0/3586517174 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc9b80637c0 con 0x7fc9d01f6520 2024-09-16T15:39:50.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.232+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol81730", "group_name": "subvol_grp42980", "target": ["mon-mgr", ""]}) -- 0x7fc9d01f6a90 con 0x7fc9ac078850 2024-09-16T15:39:50.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.238+0000 7fc9cd600640 1 -- 172.21.15.60:0/3586517174 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc9d01f6a90 con 0x7fc9ac078850 2024-09-16T15:39:50.239 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/subvol_grp42980/subvol81730 2024-09-16T15:39:50.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc9ac078850 msgr2=0x7fc9ac07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc9ac078850 0x7fc9ac07ad10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009c50 tx=0x7fc9c0005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d01f6520 msgr2=0x7fc9d01f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d01f6520 0x7fc9d01f1d30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8011ae0 tx=0x7fc9b8011ee0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 shutdown_connections 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc9ac078850 0x7fc9ac07ad10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d00c8040 0x7fc9d00c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9d01f6520 0x7fc9d01f1d30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 --2- 172.21.15.60:0/3586517174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9b4004d40 0x7fc9d015ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 >> 172.21.15.60:0/3586517174 conn(0x7fc9d00cfa70 msgr2=0x7fc9d015b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 shutdown_connections 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:50.241+0000 7fc9d5e00640 1 -- 172.21.15.60:0/3586517174 wait complete. 2024-09-16T15:39:50.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:50.259 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/ca9e8352451c745ac5d6b8c0117b2157.meta) 2024-09-16T15:39:54.298 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:39:54.298 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:39:54.298 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:39:54.298 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/subvol_grp42980/subvol81730 2024-09-16T15:39:54.298 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:39:54.298 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:54.302 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol81730 subvol_grp42980 2024-09-16T15:39:54.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.476+0000 7fa538600640 1 -- 172.21.15.60:0/2371287706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53015f870 msgr2=0x7fa53015fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.476+0000 7fa538600640 1 --2- 172.21.15.60:0/2371287706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53015f870 0x7fa53015fc50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa524005fd0 tx=0x7fa52402aa90 comp rx=0 tx=0).stop 2024-09-16T15:39:54.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.476+0000 7fa538600640 1 -- 172.21.15.60:0/2371287706 shutdown_connections 2024-09-16T15:39:54.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.476+0000 7fa538600640 1 --2- 172.21.15.60:0/2371287706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5301565b0 0x7fa53006f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.476+0000 7fa538600640 1 --2- 172.21.15.60:0/2371287706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa53015f870 0x7fa53015fc50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.476+0000 7fa538600640 1 -- 172.21.15.60:0/2371287706 >> 172.21.15.60:0/2371287706 conn(0x7fa5300b18e0 msgr2=0x7fa5300b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:54.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.476+0000 7fa538600640 1 -- 172.21.15.60:0/2371287706 shutdown_connections 2024-09-16T15:39:54.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.476+0000 7fa538600640 1 -- 172.21.15.60:0/2371287706 wait complete. 2024-09-16T15:39:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.478+0000 7fa538600640 1 Processor -- start 2024-09-16T15:39:54.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.478+0000 7fa538600640 1 -- start start 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa538600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300ccb00 0x7fa530152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa538600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5301565b0 0x7fa5300d0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa538600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53015f870 0x7fa530151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa538600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa5300cd320 con 0x7fa5301565b0 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa538600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5300cd1a0 con 0x7fa5300ccb00 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa538600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa5300739a0 con 0x7fa53015f870 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa535200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5301565b0 0x7fa5300d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa535200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5301565b0 0x7fa5300d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59540/0 (socket says 172.21.15.60:59540) 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa535200640 1 -- 172.21.15.60:0/3814615590 learned_addr learned my addr 172.21.15.60:0/3814615590 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa535c00640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300ccb00 0x7fa530152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.479+0000 7fa536600640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53015f870 0x7fa530151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa535200640 1 -- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53015f870 msgr2=0x7fa530151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa535200640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53015f870 0x7fa530151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa535200640 1 -- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300ccb00 msgr2=0x7fa530152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa535200640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300ccb00 0x7fa530152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa535200640 1 -- 172.21.15.60:0/3814615590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5301f53e0 con 0x7fa5301565b0 2024-09-16T15:39:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa535c00640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300ccb00 0x7fa530152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa535200640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5301565b0 0x7fa5300d0300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00ea20 tx=0x7fa51c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa52d600640 1 -- 172.21.15.60:0/3814615590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00cd20 con 0x7fa5301565b0 2024-09-16T15:39:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa52d600640 1 -- 172.21.15.60:0/3814615590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa51c010430 con 0x7fa5301565b0 2024-09-16T15:39:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa52d600640 1 -- 172.21.15.60:0/3814615590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c010840 con 0x7fa5301565b0 2024-09-16T15:39:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5301f56c0 con 0x7fa5301565b0 2024-09-16T15:39:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.480+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301f5c80 con 0x7fa5301565b0 2024-09-16T15:39:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.481+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa530069360 con 0x7fa5301565b0 2024-09-16T15:39:54.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.483+0000 7fa52d600640 1 -- 172.21.15.60:0/3814615590 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 6) ==== 101338+0+0 (secure 0 0 0) 0x7fa51c0109e0 con 0x7fa5301565b0 2024-09-16T15:39:54.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.483+0000 7fa52d600640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5100788e0 0x7fa51007ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.484+0000 7fa52d600640 1 -- 172.21.15.60:0/3814615590 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa51c09b500 con 0x7fa5301565b0 2024-09-16T15:39:54.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.484+0000 7fa535c00640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5100788e0 0x7fa51007ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.485+0000 7fa535c00640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5100788e0 0x7fa51007ada0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa524035000 tx=0x7fa524038040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:54.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.488+0000 7fa52d600640 1 -- 172.21.15.60:0/3814615590 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa51c0633b0 con 0x7fa5301565b0 2024-09-16T15:39:54.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.711+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol81730", "group_name": "subvol_grp42980", "target": ["mon-mgr", ""]}) -- 0x7fa53006f810 con 0x7fa5100788e0 2024-09-16T15:39:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.746+0000 7fa52d600640 1 -- 172.21.15.60:0/3814615590 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa53006f810 con 0x7fa5100788e0 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5100788e0 msgr2=0x7fa51007ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5100788e0 0x7fa51007ada0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa524035000 tx=0x7fa524038040 comp rx=0 tx=0).stop 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5301565b0 msgr2=0x7fa5300d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5301565b0 0x7fa5300d0300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00ea20 tx=0x7fa51c00eee0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 shutdown_connections 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5100788e0 0x7fa51007ada0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa53015f870 0x7fa530151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5301565b0 0x7fa5300d0300 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.748+0000 7fa538600640 1 --2- 172.21.15.60:0/3814615590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5300ccb00 0x7fa530152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.749+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 >> 172.21.15.60:0/3814615590 conn(0x7fa5300b18e0 msgr2=0x7fa530153510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:54.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.749+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 shutdown_connections 2024-09-16T15:39:54.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54.749+0000 7fa538600640 1 -- 172.21.15.60:0/3814615590 wait complete. 2024-09-16T15:39:54.752 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.751+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.753 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.751+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.753 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.751+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.753 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.751+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.753 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.751+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.767 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:39:54.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.772+0000 7fa50de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.772+0000 7fa50de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.772+0000 7fa50de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.773 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.772+0000 7fa50de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.774 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:39:54.772+0000 7fa50de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:39:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:54.854 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-16T15:39:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:54.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp42980 2024-09-16T15:39:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.027+0000 7f3bc9c00640 1 -- 172.21.15.60:0/3441684493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 msgr2=0x7f3bc4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.027+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/3441684493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 0x7f3bc4073440 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3bac005fd0 tx=0x7f3bac02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.027+0000 7f3bc9c00640 1 -- 172.21.15.60:0/3441684493 shutdown_connections 2024-09-16T15:39:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.028+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/3441684493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 0x7f3bc4073440 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.028+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/3441684493 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc406efa0 0x7f3bc40cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.028+0000 7f3bc9c00640 1 -- 172.21.15.60:0/3441684493 >> 172.21.15.60:0/3441684493 conn(0x7f3bc40b1840 msgr2=0x7f3bc40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.028+0000 7f3bc9c00640 1 -- 172.21.15.60:0/3441684493 shutdown_connections 2024-09-16T15:39:55.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.028+0000 7f3bc9c00640 1 -- 172.21.15.60:0/3441684493 wait complete. 2024-09-16T15:39:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.029+0000 7f3bc9c00640 1 Processor -- start 2024-09-16T15:39:55.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.030+0000 7f3bc9c00640 1 -- start start 2024-09-16T15:39:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.030+0000 7f3bc9c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc406efa0 0x7f3bc41f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.030+0000 7f3bc9c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 0x7f3bc4140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.030+0000 7f3bc9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4145750 0x7f3bc413e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bc9c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40ce480 con 0x7f3bc40cccd0 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bc9c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40ce300 con 0x7f3bc4145750 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bc9c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc40ce600 con 0x7f3bc406efa0 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bba000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 0x7f3bc4140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bba000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 0x7f3bc4140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59560/0 (socket says 172.21.15.60:59560) 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bba000640 1 -- 172.21.15.60:0/2442181603 learned_addr learned my addr 172.21.15.60:0/2442181603 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bc2a00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc406efa0 0x7f3bc41f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bc3400640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4145750 0x7f3bc413e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bba000640 1 -- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc406efa0 msgr2=0x7f3bc41f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bba000640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc406efa0 0x7f3bc41f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bba000640 1 -- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4145750 msgr2=0x7f3bc413e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bba000640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4145750 0x7f3bc413e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.031+0000 7f3bba000640 1 -- 172.21.15.60:0/2442181603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc40c4ab0 con 0x7f3bc40cccd0 2024-09-16T15:39:55.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.032+0000 7f3bba000640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 0x7f3bc4140580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009850 tx=0x7f3bac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:55.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.032+0000 7f3bbb400640 1 -- 172.21.15.60:0/2442181603 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac03f070 con 0x7f3bc40cccd0 2024-09-16T15:39:55.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.032+0000 7f3bbb400640 1 -- 172.21.15.60:0/2442181603 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bac0047a0 con 0x7f3bc40cccd0 2024-09-16T15:39:55.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.032+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc40c4d90 con 0x7f3bc40cccd0 2024-09-16T15:39:55.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.032+0000 7f3bbb400640 1 -- 172.21.15.60:0/2442181603 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac035030 con 0x7f3bc40cccd0 2024-09-16T15:39:55.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.032+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc40c5350 con 0x7f3bc40cccd0 2024-09-16T15:39:55.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.033+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc40c5840 con 0x7f3bc40cccd0 2024-09-16T15:39:55.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.034+0000 7f3bbb400640 1 -- 172.21.15.60:0/2442181603 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 6) ==== 101338+0+0 (secure 0 0 0) 0x7f3bac042070 con 0x7f3bc40cccd0 2024-09-16T15:39:55.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.035+0000 7f3bbb400640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ba00788e0 0x7f3ba007ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.035+0000 7f3bbb400640 1 -- 172.21.15.60:0/2442181603 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f3bac0bb910 con 0x7f3bc40cccd0 2024-09-16T15:39:55.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.036+0000 7f3bc2a00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ba00788e0 0x7f3ba007ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.036+0000 7f3bc2a00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ba00788e0 0x7f3ba007ada0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006fd0 tx=0x7f3bb4008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:55.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.039+0000 7f3bbb400640 1 -- 172.21.15.60:0/2442181603 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3bac083e10 con 0x7f3bc40cccd0 2024-09-16T15:39:55.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.259+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp42980", "target": ["mon-mgr", ""]}) -- 0x7f3bc40cec30 con 0x7f3ba00788e0 2024-09-16T15:39:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.264+0000 7f3bbb400640 1 -- 172.21.15.60:0/2442181603 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3bc40cec30 con 0x7f3ba00788e0 2024-09-16T15:39:55.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ba00788e0 msgr2=0x7f3ba007ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ba00788e0 0x7f3ba007ada0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006fd0 tx=0x7f3bb4008040 comp rx=0 tx=0).stop 2024-09-16T15:39:55.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 msgr2=0x7f3bc4140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 0x7f3bc4140580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009850 tx=0x7f3bac0038a0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 shutdown_connections 2024-09-16T15:39:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ba00788e0 0x7f3ba007ada0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc4145750 0x7f3bc413e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc40cccd0 0x7f3bc4140580 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 --2- 172.21.15.60:0/2442181603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc406efa0 0x7f3bc41f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 >> 172.21.15.60:0/2442181603 conn(0x7f3bc40b1840 msgr2=0x7f3bc40d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 shutdown_connections 2024-09-16T15:39:55.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.266+0000 7f3bc9c00640 1 -- 172.21.15.60:0/2442181603 wait complete. 2024-09-16T15:39:55.283 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:39:55.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:39:55.381 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:39:55.481 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:39:55.481 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:39:55.481 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:55.481 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:55.516 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:55.516 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:55.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.793+0000 7f8944c00640 1 -- 172.21.15.60:0/1448374051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400cc8d0 msgr2=0x7f894015cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.793+0000 7f8944c00640 1 --2- 172.21.15.60:0/1448374051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400cc8d0 0x7f894015cb40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8940074b40 tx=0x7f893002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.794+0000 7f8944c00640 1 -- 172.21.15.60:0/1448374051 shutdown_connections 2024-09-16T15:39:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.794+0000 7f8944c00640 1 --2- 172.21.15.60:0/1448374051 >> v2:172.21.15.100:3300/0 conn(0x7f8940073ee0 0x7f894006f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.794+0000 7f8944c00640 1 --2- 172.21.15.60:0/1448374051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400cc8d0 0x7f894015cb40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.794+0000 7f8944c00640 1 -- 172.21.15.60:0/1448374051 >> 172.21.15.60:0/1448374051 conn(0x7f89400d0b30 msgr2=0x7f89400d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.794+0000 7f8944c00640 1 -- 172.21.15.60:0/1448374051 shutdown_connections 2024-09-16T15:39:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.794+0000 7f8944c00640 1 -- 172.21.15.60:0/1448374051 wait complete. 2024-09-16T15:39:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.795+0000 7f8944c00640 1 Processor -- start 2024-09-16T15:39:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.795+0000 7f8944c00640 1 -- start start 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.796+0000 7f8944c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8940073ee0 0x7f89401f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.796+0000 7f8944c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89400cc8d0 0x7f894014fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.796+0000 7f8944c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400d1620 0x7f89401f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.796+0000 7f8944c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f89400744b0 con 0x7f89400d1620 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.796+0000 7f8944c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8940074330 con 0x7f89400cc8d0 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.796+0000 7f8944c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8940074630 con 0x7f8940073ee0 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400d1620 0x7f89401f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400d1620 0x7f89401f85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59582/0 (socket says 172.21.15.60:59582) 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 -- 172.21.15.60:0/2047775647 learned_addr learned my addr 172.21.15.60:0/2047775647 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893f400640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8940073ee0 0x7f89401f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f8936a00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89400cc8d0 0x7f894014fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 -- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8940073ee0 msgr2=0x7f89401f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8940073ee0 0x7f89401f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 -- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89400cc8d0 msgr2=0x7f894014fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89400cc8d0 0x7f894014fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 -- 172.21.15.60:0/2047775647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89401f51e0 con 0x7f89400d1620 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f8936a00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89400cc8d0 0x7f894014fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.797+0000 7f893fe00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400d1620 0x7f89401f85e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f893800eb40 tx=0x7f893800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.798+0000 7f8937e00640 1 -- 172.21.15.60:0/2047775647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938004b20 con 0x7f89400d1620 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.798+0000 7f8937e00640 1 -- 172.21.15.60:0/2047775647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8938004ce0 con 0x7f89400d1620 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.798+0000 7f8937e00640 1 -- 172.21.15.60:0/2047775647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938005280 con 0x7f89400d1620 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.798+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89401f9030 con 0x7f89400d1620 2024-09-16T15:39:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.798+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89401f5f00 con 0x7f89400d1620 2024-09-16T15:39:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.799+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89401f4ff0 con 0x7f89400d1620 2024-09-16T15:39:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.800+0000 7f8937e00640 1 -- 172.21.15.60:0/2047775647 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 6) ==== 101338+0+0 (secure 0 0 0) 0x7f8938005420 con 0x7f89400d1620 2024-09-16T15:39:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.801+0000 7f8937e00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f891c0788e0 0x7f891c07ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.801+0000 7f8937e00640 1 -- 172.21.15.60:0/2047775647 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f893809b6c0 con 0x7f89400d1620 2024-09-16T15:39:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.801+0000 7f893f400640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f891c0788e0 0x7f891c07ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.802+0000 7f893f400640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f891c0788e0 0x7f891c07ada0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8930033000 tx=0x7f8930004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:55.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:55.805+0000 7f8937e00640 1 -- 172.21.15.60:0/2047775647 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8938063bc0 con 0x7f89400d1620 2024-09-16T15:39:56.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.011+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f89401535f0 con 0x7f89400d1620 2024-09-16T15:39:56.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.012+0000 7f8937e00640 1 -- 172.21.15.60:0/2047775647 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f89380634c0 con 0x7f89400d1620 2024-09-16T15:39:56.013 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f891c0788e0 msgr2=0x7f891c07ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f891c0788e0 0x7f891c07ada0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8930033000 tx=0x7f8930004730 comp rx=0 tx=0).stop 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400d1620 msgr2=0x7f89401f85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400d1620 0x7f89401f85e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f893800eb40 tx=0x7f893800c2d0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 shutdown_connections 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f891c0788e0 0x7f891c07ada0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89400d1620 0x7f89401f85e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89400cc8d0 0x7f894014fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 --2- 172.21.15.60:0/2047775647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8940073ee0 0x7f89401f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 >> 172.21.15.60:0/2047775647 conn(0x7f89400d0b30 msgr2=0x7f89400b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:56.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.014+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 shutdown_connections 2024-09-16T15:39:56.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.015+0000 7f8944c00640 1 -- 172.21.15.60:0/2047775647 wait complete. 2024-09-16T15:39:56.032 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:39:56.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:56.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:56.120 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:39:56.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:56.162 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:39:56.200 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:39:56.200 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:56.200 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:56.237 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:56.238 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:56.314 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.476+0000 7fe40e800640 1 -- 172.21.15.60:0/2021681648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080d1ec0 msgr2=0x7fe4080d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.476+0000 7fe40e800640 1 --2- 172.21.15.60:0/2021681648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080d1ec0 0x7fe4080d22a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009830 tx=0x7fe3fc02a510 comp rx=0 tx=0).stop 2024-09-16T15:39:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.477+0000 7fe40e800640 1 -- 172.21.15.60:0/2021681648 shutdown_connections 2024-09-16T15:39:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.477+0000 7fe40e800640 1 --2- 172.21.15.60:0/2021681648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080d1ec0 0x7fe4080d22a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.477+0000 7fe40e800640 1 -- 172.21.15.60:0/2021681648 >> 172.21.15.60:0/2021681648 conn(0x7fe4080d3de0 msgr2=0x7fe4080d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.477+0000 7fe40e800640 1 -- 172.21.15.60:0/2021681648 shutdown_connections 2024-09-16T15:39:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.477+0000 7fe40e800640 1 -- 172.21.15.60:0/2021681648 wait complete. 2024-09-16T15:39:56.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.478+0000 7fe40e800640 1 Processor -- start 2024-09-16T15:39:56.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.478+0000 7fe40e800640 1 -- start start 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.479+0000 7fe40e800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4080b28d0 0x7fe40815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.479+0000 7fe40e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4d10 0x7fe40815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.479+0000 7fe40e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080d1ec0 0x7fe4081f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.479+0000 7fe40e800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe4080b4b20 con 0x7fe4080b4d10 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.479+0000 7fe40e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4080b49a0 con 0x7fe4080d1ec0 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.479+0000 7fe40e800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe4081692e0 con 0x7fe4080b28d0 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4d10 0x7fe40815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4d10 0x7fe40815e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59606/0 (socket says 172.21.15.60:59606) 2024-09-16T15:39:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 -- 172.21.15.60:0/1029060893 learned_addr learned my addr 172.21.15.60:0/1029060893 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40de00640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080d1ec0 0x7fe4081f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40d400640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4080b28d0 0x7fe40815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 -- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4080b28d0 msgr2=0x7fe40815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4080b28d0 0x7fe40815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 -- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080d1ec0 msgr2=0x7fe4081f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080d1ec0 0x7fe4081f48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 -- 172.21.15.60:0/1029060893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4080cdb10 con 0x7fe4080b4d10 2024-09-16T15:39:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.480+0000 7fe40ca00640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4d10 0x7fe40815e1d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400c490 tx=0x7fe3f400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.481+0000 7fe405600640 1 -- 172.21.15.60:0/1029060893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4016020 con 0x7fe4080b4d10 2024-09-16T15:39:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.481+0000 7fe405600640 1 -- 172.21.15.60:0/1029060893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3f4012940 con 0x7fe4080b4d10 2024-09-16T15:39:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.481+0000 7fe405600640 1 -- 172.21.15.60:0/1029060893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4012d90 con 0x7fe4080b4d10 2024-09-16T15:39:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.481+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40815b9c0 con 0x7fe4080b4d10 2024-09-16T15:39:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.481+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40815bc60 con 0x7fe4080b4d10 2024-09-16T15:39:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.482+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4080d22a0 con 0x7fe4080b4d10 2024-09-16T15:39:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.483+0000 7fe405600640 1 -- 172.21.15.60:0/1029060893 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fe3f4019020 con 0x7fe4080b4d10 2024-09-16T15:39:56.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.484+0000 7fe405600640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe3e8078a00 0x7fe3e807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:56.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.484+0000 7fe405600640 1 -- 172.21.15.60:0/1029060893 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fe3f409aac0 con 0x7fe4080b4d10 2024-09-16T15:39:56.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.484+0000 7fe40d400640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe3e8078a00 0x7fe3e807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:56.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.485+0000 7fe40d400640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe3e8078a00 0x7fe3e807aec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc034000 tx=0x7fe3fc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:56.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.489+0000 7fe405600640 1 -- 172.21.15.60:0/1029060893 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe3f4020030 con 0x7fe4080b4d10 2024-09-16T15:39:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.696+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe408157e10 con 0x7fe4080b4d10 2024-09-16T15:39:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.697+0000 7fe405600640 1 -- 172.21.15.60:0/1029060893 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7fe3f4062d90 con 0x7fe4080b4d10 2024-09-16T15:39:56.698 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:39:56.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe3e8078a00 msgr2=0x7fe3e807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe3e8078a00 0x7fe3e807aec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc034000 tx=0x7fe3fc004730 comp rx=0 tx=0).stop 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4d10 msgr2=0x7fe40815e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4d10 0x7fe40815e1d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400c490 tx=0x7fe3f400c950 comp rx=0 tx=0).stop 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 shutdown_connections 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe3e8078a00 0x7fe3e807aec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4080d1ec0 0x7fe4081f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4080b4d10 0x7fe40815e1d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 --2- 172.21.15.60:0/1029060893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4080b28d0 0x7fe40815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 >> 172.21.15.60:0/1029060893 conn(0x7fe4080d3de0 msgr2=0x7fe4080d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 shutdown_connections 2024-09-16T15:39:56.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:56.699+0000 7fe40e800640 1 -- 172.21.15.60:0/1029060893 wait complete. 2024-09-16T15:39:56.715 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:39:56.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:56.716 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:56.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:39:56.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:56.779 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:39:56.814 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:39:56.815 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:39:56.815 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:56.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:39:56.850 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:39:56.850 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:39:56.850 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:56.889 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:39:56.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:57.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.049+0000 7f9906c00640 1 -- 172.21.15.60:0/2055340219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 msgr2=0x7f99000b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.049+0000 7f9906c00640 1 --2- 172.21.15.60:0/2055340219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 0x7f99000b28f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f98f4005fd0 tx=0x7f98f402a670 comp rx=0 tx=0).stop 2024-09-16T15:39:57.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.049+0000 7f9906c00640 1 -- 172.21.15.60:0/2055340219 shutdown_connections 2024-09-16T15:39:57.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.049+0000 7f9906c00640 1 --2- 172.21.15.60:0/2055340219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 0x7f99000b28f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.049+0000 7f9906c00640 1 -- 172.21.15.60:0/2055340219 >> 172.21.15.60:0/2055340219 conn(0x7f99000cfa20 msgr2=0x7f99000cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.049+0000 7f9906c00640 1 -- 172.21.15.60:0/2055340219 shutdown_connections 2024-09-16T15:39:57.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.049+0000 7f9906c00640 1 -- 172.21.15.60:0/2055340219 wait complete. 2024-09-16T15:39:57.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.051+0000 7f9906c00640 1 Processor -- start 2024-09-16T15:39:57.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.051+0000 7f9906c00640 1 -- start start 2024-09-16T15:39:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9906c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99000d0540 0x7f99001f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9906c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 0x7f99001f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9906c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000cd4f0 0x7f99001ee420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9906c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9900074660 con 0x7f990015bc30 2024-09-16T15:39:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9906c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f990006fa30 con 0x7f99000d0540 2024-09-16T15:39:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9906c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f99001f8c60 con 0x7f99000cd4f0 2024-09-16T15:39:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9905800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99000d0540 0x7f99001f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9904e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 0x7f99001f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9905800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99000d0540 0x7f99001f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45434/0 (socket says 172.21.15.60:45434) 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.052+0000 7f9905800640 1 -- 172.21.15.60:0/3907197556 learned_addr learned my addr 172.21.15.60:0/3907197556 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f9906200640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000cd4f0 0x7f99001ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f9905800640 1 -- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000cd4f0 msgr2=0x7f99001ee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f9905800640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000cd4f0 0x7f99001ee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f9905800640 1 -- 172.21.15.60:0/3907197556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 msgr2=0x7f99001f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f9905800640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 0x7f99001f8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f9905800640 1 -- 172.21.15.60:0/3907197556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99001f4db0 con 0x7f99000d0540 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f9904e00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 0x7f99001f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:57.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f9905800640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99000d0540 0x7f99001f4720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f98f4034040 tx=0x7f98f40039e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.053+0000 7f98fd600640 1 -- 172.21.15.60:0/3907197556 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f403e030 con 0x7f99000d0540 2024-09-16T15:39:57.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.054+0000 7f98fd600640 1 -- 172.21.15.60:0/3907197556 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98f40047a0 con 0x7f99000d0540 2024-09-16T15:39:57.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.054+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99001f5090 con 0x7f99000d0540 2024-09-16T15:39:57.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.054+0000 7f98fd600640 1 -- 172.21.15.60:0/3907197556 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4035040 con 0x7f99000d0540 2024-09-16T15:39:57.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.054+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99001f5650 con 0x7f99000d0540 2024-09-16T15:39:57.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.055+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99000b28f0 con 0x7f99000d0540 2024-09-16T15:39:57.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.056+0000 7f98fd600640 1 -- 172.21.15.60:0/3907197556 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f98f4041030 con 0x7f99000d0540 2024-09-16T15:39:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.057+0000 7f98fd600640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98e0078a00 0x7f98e007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.057+0000 7f98fd600640 1 -- 172.21.15.60:0/3907197556 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f98f40bb3a0 con 0x7f99000d0540 2024-09-16T15:39:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.057+0000 7f9904e00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98e0078a00 0x7f98e007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.058+0000 7f9904e00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98e0078a00 0x7f98e007aec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f99000c8d80 tx=0x7f98ec009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.062+0000 7f98fd600640 1 -- 172.21.15.60:0/3907197556 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f98f4083040 con 0x7f99000d0540 2024-09-16T15:39:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.297+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99000b4450 con 0x7f99000d0540 2024-09-16T15:39:57.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.298+0000 7f98fd600640 1 -- 172.21.15.60:0/3907197556 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5220 (secure 0 0 0) 0x7f98f4087a70 con 0x7f99000d0540 2024-09-16T15:39:57.300 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:57.300 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":18,"btime":"2024-09-16T15:39:48:209143+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4866,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2562305658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2562305658},{"type":"v1","addr":"172.21.15.60:6833","nonce":2562305658}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/664909622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":664909622},{"type":"v1","addr":"172.21.15.100:6836","nonce":664909622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/801262237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":801262237},{"type":"v1","addr":"172.21.15.100:6837","nonce":801262237}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:32.603941+0000","modified":"2024-09-16T15:39:35.605148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/846626570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":846626570},{"type":"v1","addr":"172.21.15.60:6835","nonce":846626570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4881,"qdb_cluster":[4881]},"id":2}]} 2024-09-16T15:39:57.300 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 18 2024-09-16T15:39:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98e0078a00 msgr2=0x7f98e007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98e0078a00 0x7f98e007aec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f99000c8d80 tx=0x7f98ec009040 comp rx=0 tx=0).stop 2024-09-16T15:39:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99000d0540 msgr2=0x7f99001f4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99000d0540 0x7f99001f4720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f98f4034040 tx=0x7f98f40039e0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 shutdown_connections 2024-09-16T15:39:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98e0078a00 0x7f98e007aec0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99000cd4f0 0x7f99001ee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f990015bc30 0x7f99001f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 --2- 172.21.15.60:0/3907197556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f99000d0540 0x7f99001f4720 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 >> 172.21.15.60:0/3907197556 conn(0x7f99000cfa20 msgr2=0x7f9900150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 shutdown_connections 2024-09-16T15:39:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.301+0000 7f9906c00640 1 -- 172.21.15.60:0/3907197556 wait complete. 2024-09-16T15:39:57.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.477+0000 7f9fae000640 1 -- 172.21.15.60:0/3146608144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa815f5a0 msgr2=0x7f9fa80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.477+0000 7f9fae000640 1 --2- 172.21.15.60:0/3146608144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa815f5a0 0x7f9fa80b2440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f98005fd0 tx=0x7f9f9802a540 comp rx=0 tx=0).stop 2024-09-16T15:39:57.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.477+0000 7f9fae000640 1 -- 172.21.15.60:0/3146608144 shutdown_connections 2024-09-16T15:39:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.477+0000 7f9fae000640 1 --2- 172.21.15.60:0/3146608144 >> v2:172.21.15.100:3300/0 conn(0x7f9fa80ccf00 0x7f9fa80acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.477+0000 7f9fae000640 1 --2- 172.21.15.60:0/3146608144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa815f5a0 0x7f9fa80b2440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.477+0000 7f9fae000640 1 -- 172.21.15.60:0/3146608144 >> 172.21.15.60:0/3146608144 conn(0x7f9fa80b1950 msgr2=0x7f9fa80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.477+0000 7f9fae000640 1 -- 172.21.15.60:0/3146608144 shutdown_connections 2024-09-16T15:39:57.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.477+0000 7f9fae000640 1 -- 172.21.15.60:0/3146608144 wait complete. 2024-09-16T15:39:57.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.479+0000 7f9fae000640 1 Processor -- start 2024-09-16T15:39:57.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.479+0000 7f9fae000640 1 -- start start 2024-09-16T15:39:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fae000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa80ad270 0x7f9fa814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fae000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fa80ccf00 0x7f9fa81505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fae000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fa815f5a0 0x7f9fa80c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fae000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa806f2e0 con 0x7f9fa80ccf00 2024-09-16T15:39:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fae000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa806f160 con 0x7f9fa815f5a0 2024-09-16T15:39:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fae000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa806f460 con 0x7f9fa80ad270 2024-09-16T15:39:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fa80ccf00 0x7f9fa81505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa80ad270 0x7f9fa814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa80ad270 0x7f9fa814ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35890/0 (socket says 172.21.15.60:35890) 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fa815f5a0 0x7f9fa80c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7400640 1 -- 172.21.15.60:0/433404173 learned_addr learned my addr 172.21.15.60:0/433404173 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7400640 1 -- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fa815f5a0 msgr2=0x7f9fa80c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7400640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fa815f5a0 0x7f9fa80c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7400640 1 -- 172.21.15.60:0/433404173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fa80ccf00 msgr2=0x7f9fa81505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7400640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fa80ccf00 0x7f9fa81505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa7400640 1 -- 172.21.15.60:0/433404173 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa80cdc80 con 0x7f9fa80ad270 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.480+0000 7f9fa6a00640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fa80ccf00 0x7f9fa81505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.481+0000 7f9fa7400640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa80ad270 0x7f9fa814ea90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f98034040 tx=0x7f9f980038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.481+0000 7f9f9f400640 1 -- 172.21.15.60:0/433404173 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9803e030 con 0x7f9fa80ad270 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.481+0000 7f9f9f400640 1 -- 172.21.15.60:0/433404173 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f98003d90 con 0x7f9fa80ad270 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.481+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa80cdf60 con 0x7f9fa80ad270 2024-09-16T15:39:57.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.481+0000 7f9f9f400640 1 -- 172.21.15.60:0/433404173 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98035040 con 0x7f9fa80ad270 2024-09-16T15:39:57.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.481+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa80ce520 con 0x7f9fa80ad270 2024-09-16T15:39:57.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.482+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa806f980 con 0x7f9fa80ad270 2024-09-16T15:39:57.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.483+0000 7f9f9f400640 1 -- 172.21.15.60:0/433404173 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f9f98041030 con 0x7f9fa80ad270 2024-09-16T15:39:57.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.484+0000 7f9f9f400640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9f88078a00 0x7f9f8807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.484+0000 7f9f9f400640 1 -- 172.21.15.60:0/433404173 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9f980bb540 con 0x7f9fa80ad270 2024-09-16T15:39:57.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.484+0000 7f9fa6a00640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9f88078a00 0x7f9f8807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.485+0000 7f9fa6a00640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9f88078a00 0x7f9f8807aec0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8152890 tx=0x7f9f94005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.489+0000 7f9f9f400640 1 -- 172.21.15.60:0/433404173 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9f98083270 con 0x7f9fa80ad270 2024-09-16T15:39:57.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.722+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fa80ce760 con 0x7f9fa80ad270 2024-09-16T15:39:57.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.723+0000 7f9f9f400640 1 -- 172.21.15.60:0/433404173 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5220 (secure 0 0 0) 0x7f9f98087ca0 con 0x7f9fa80ad270 2024-09-16T15:39:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:57.725 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":18,"btime":"2024-09-16T15:39:48:209143+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4866,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2562305658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2562305658},{"type":"v1","addr":"172.21.15.60:6833","nonce":2562305658}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/664909622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":664909622},{"type":"v1","addr":"172.21.15.100:6836","nonce":664909622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/801262237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":801262237},{"type":"v1","addr":"172.21.15.100:6837","nonce":801262237}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:32.603941+0000","modified":"2024-09-16T15:39:35.605148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/846626570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":846626570},{"type":"v1","addr":"172.21.15.60:6835","nonce":846626570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4881,"qdb_cluster":[4881]},"id":2}]} 2024-09-16T15:39:57.725 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 18 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9f88078a00 msgr2=0x7f9f8807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9f88078a00 0x7f9f8807aec0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8152890 tx=0x7f9f94005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa80ad270 msgr2=0x7f9fa814ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa80ad270 0x7f9fa814ea90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f98034040 tx=0x7f9f980038a0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 shutdown_connections 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9f88078a00 0x7f9f8807aec0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fa815f5a0 0x7f9fa80c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9fa80ccf00 0x7f9fa81505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 --2- 172.21.15.60:0/433404173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9fa80ad270 0x7f9fa814ea90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 >> 172.21.15.60:0/433404173 conn(0x7f9fa80b1950 msgr2=0x7f9fa80b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 shutdown_connections 2024-09-16T15:39:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.726+0000 7f9fae000640 1 -- 172.21.15.60:0/433404173 wait complete. 2024-09-16T15:39:57.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:57.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.900+0000 7f9040c00640 1 -- 172.21.15.60:0/2216623559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 msgr2=0x7f903c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.900+0000 7f9040c00640 1 --2- 172.21.15.60:0/2216623559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 0x7f903c0b2440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f902c009860 tx=0x7f902c02a510 comp rx=0 tx=0).stop 2024-09-16T15:39:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.900+0000 7f9040c00640 1 -- 172.21.15.60:0/2216623559 shutdown_connections 2024-09-16T15:39:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.900+0000 7f9040c00640 1 --2- 172.21.15.60:0/2216623559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 0x7f903c0b2440 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.900+0000 7f9040c00640 1 -- 172.21.15.60:0/2216623559 >> 172.21.15.60:0/2216623559 conn(0x7f903c0b1950 msgr2=0x7f903c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.900+0000 7f9040c00640 1 -- 172.21.15.60:0/2216623559 shutdown_connections 2024-09-16T15:39:57.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.901+0000 7f9040c00640 1 -- 172.21.15.60:0/2216623559 wait complete. 2024-09-16T15:39:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.902+0000 7f9040c00640 1 Processor -- start 2024-09-16T15:39:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.902+0000 7f9040c00640 1 -- start start 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9040c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f903c073fb0 0x7f903c0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9040c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f903c0ccf00 0x7f903c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9040c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 0x7f903c1f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9040c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f903c06ef30 con 0x7f903c073fb0 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9040c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f903c06fa80 con 0x7f903c0ccf00 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9040c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f903c074460 con 0x7f903c15f5a0 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9039e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f903c073fb0 0x7f903c0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9039e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f903c073fb0 0x7f903c0cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59646/0 (socket says 172.21.15.60:59646) 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.903+0000 7f9039e00640 1 -- 172.21.15.60:0/2564222162 learned_addr learned my addr 172.21.15.60:0/2564222162 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9031400640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f903c0ccf00 0x7f903c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f903a800640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 0x7f903c1f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9039e00640 1 -- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 msgr2=0x7f903c1f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9039e00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 0x7f903c1f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9039e00640 1 -- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f903c0ccf00 msgr2=0x7f903c0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9039e00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f903c0ccf00 0x7f903c0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9039e00640 1 -- 172.21.15.60:0/2564222162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c0b45e0 con 0x7f903c073fb0 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f903a800640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 0x7f903c1f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9031400640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f903c0ccf00 0x7f903c0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9039e00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f903c073fb0 0x7f903c0cba60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f902c009980 tx=0x7f902c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9032a00640 1 -- 172.21.15.60:0/2564222162 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c005c90 con 0x7f903c073fb0 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9032a00640 1 -- 172.21.15.60:0/2564222162 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f902c031e50 con 0x7f903c073fb0 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9032a00640 1 -- 172.21.15.60:0/2564222162 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c005260 con 0x7f903c073fb0 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.904+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c0c9190 con 0x7f903c073fb0 2024-09-16T15:39:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.905+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c0cbfa0 con 0x7f903c073fb0 2024-09-16T15:39:57.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.906+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f903c0b2440 con 0x7f903c073fb0 2024-09-16T15:39:57.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.907+0000 7f9032a00640 1 -- 172.21.15.60:0/2564222162 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f902c005400 con 0x7f903c073fb0 2024-09-16T15:39:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.908+0000 7f9032a00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9018078a30 0x7f901807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.908+0000 7f9032a00640 1 -- 172.21.15.60:0/2564222162 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f902c0bbc30 con 0x7f903c073fb0 2024-09-16T15:39:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.908+0000 7f9031400640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9018078a30 0x7f901807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.909+0000 7f9031400640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9018078a30 0x7f901807aef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f903c0b4f30 tx=0x7f9024009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:57.912+0000 7f9032a00640 1 -- 172.21.15.60:0/2564222162 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f902c03f030 con 0x7f903c073fb0 2024-09-16T15:39:58.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.108+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f903c0b2070 con 0x7f903c073fb0 2024-09-16T15:39:58.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.109+0000 7f9032a00640 1 -- 172.21.15.60:0/2564222162 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16729 (secure 0 0 0) 0x7f902c083f00 con 0x7f903c073fb0 2024-09-16T15:39:58.113 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:58.113 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":36,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:39:34.155062+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:30.169700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:31.550025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:58.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9018078a30 msgr2=0x7f901807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9018078a30 0x7f901807aef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f903c0b4f30 tx=0x7f9024009040 comp rx=0 tx=0).stop 2024-09-16T15:39:58.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f903c073fb0 msgr2=0x7f903c0cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f903c073fb0 0x7f903c0cba60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f902c009980 tx=0x7f902c033c20 comp rx=0 tx=0).stop 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 shutdown_connections 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9018078a30 0x7f901807aef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f903c15f5a0 0x7f903c1f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f903c0ccf00 0x7f903c0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 --2- 172.21.15.60:0/2564222162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f903c073fb0 0x7f903c0cba60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 >> 172.21.15.60:0/2564222162 conn(0x7f903c0b1950 msgr2=0x7f903c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 shutdown_connections 2024-09-16T15:39:58.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.111+0000 7f9040c00640 1 -- 172.21.15.60:0/2564222162 wait complete. 2024-09-16T15:39:58.128 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:39:58.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:58.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.291+0000 7f7ab8200640 1 -- 172.21.15.60:0/3151406684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 msgr2=0x7f7ab00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.291+0000 7f7ab8200640 1 --2- 172.21.15.60:0/3151406684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 0x7f7ab00b2440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005fd0 tx=0x7f7aa402a540 comp rx=0 tx=0).stop 2024-09-16T15:39:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.291+0000 7f7ab8200640 1 -- 172.21.15.60:0/3151406684 shutdown_connections 2024-09-16T15:39:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.291+0000 7f7ab8200640 1 --2- 172.21.15.60:0/3151406684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ab00ccf00 0x7f7ab00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.291+0000 7f7ab8200640 1 --2- 172.21.15.60:0/3151406684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 0x7f7ab00b2440 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.291+0000 7f7ab8200640 1 -- 172.21.15.60:0/3151406684 >> 172.21.15.60:0/3151406684 conn(0x7f7ab00b1950 msgr2=0x7f7ab00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.291+0000 7f7ab8200640 1 -- 172.21.15.60:0/3151406684 shutdown_connections 2024-09-16T15:39:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.291+0000 7f7ab8200640 1 -- 172.21.15.60:0/3151406684 wait complete. 2024-09-16T15:39:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.293+0000 7f7ab8200640 1 Processor -- start 2024-09-16T15:39:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.293+0000 7f7ab8200640 1 -- start start 2024-09-16T15:39:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab8200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ab00ad270 0x7f7ab014c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab8200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ab00ccf00 0x7f7ab014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab8200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 0x7f7ab014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab8200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab006f2e0 con 0x7f7ab015f5a0 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab8200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab006f160 con 0x7f7ab00ad270 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab8200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab006f460 con 0x7f7ab00ccf00 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab6200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 0x7f7ab014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab4e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ab00ccf00 0x7f7ab014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab6200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 0x7f7ab014f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59674/0 (socket says 172.21.15.60:59674) 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab6200640 1 -- 172.21.15.60:0/1689121922 learned_addr learned my addr 172.21.15.60:0/1689121922 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.294+0000 7f7ab5800640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ab00ad270 0x7f7ab014c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab6200640 1 -- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ab00ccf00 msgr2=0x7f7ab014ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab6200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ab00ccf00 0x7f7ab014ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab6200640 1 -- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ab00ad270 msgr2=0x7f7ab014c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab6200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ab00ad270 0x7f7ab014c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab6200640 1 -- 172.21.15.60:0/1689121922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab014e820 con 0x7f7ab015f5a0 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab4e00640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ab00ccf00 0x7f7ab014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab6200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 0x7f7ab014f700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c770 tx=0x7f7aa800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7aad600640 1 -- 172.21.15.60:0/1689121922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8017070 con 0x7f7ab015f5a0 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7aad600640 1 -- 172.21.15.60:0/1689121922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7aa8005070 con 0x7f7ab015f5a0 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7aad600640 1 -- 172.21.15.60:0/1689121922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80054c0 con 0x7f7ab015f5a0 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab014eb00 con 0x7f7ab015f5a0 2024-09-16T15:39:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.295+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab0150cc0 con 0x7f7ab015f5a0 2024-09-16T15:39:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.296+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab0150e90 con 0x7f7ab015f5a0 2024-09-16T15:39:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.297+0000 7f7aad600640 1 -- 172.21.15.60:0/1689121922 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f7aa801a070 con 0x7f7ab015f5a0 2024-09-16T15:39:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.298+0000 7f7aad600640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a90078a00 0x7f7a9007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.298+0000 7f7aad600640 1 -- 172.21.15.60:0/1689121922 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f7aa809a2d0 con 0x7f7ab015f5a0 2024-09-16T15:39:58.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.299+0000 7f7ab5800640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a90078a00 0x7f7a9007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.300+0000 7f7ab5800640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a90078a00 0x7f7a9007aec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4033000 tx=0x7f7aa4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:58.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.302+0000 7f7aad600640 1 -- 172.21.15.60:0/1689121922 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7aa8061f80 con 0x7f7ab015f5a0 2024-09-16T15:39:58.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.534+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ab00743a0 con 0x7f7ab015f5a0 2024-09-16T15:39:58.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.535+0000 7f7aad600640 1 -- 172.21.15.60:0/1689121922 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5220 (secure 0 0 0) 0x7f7aa80669b0 con 0x7f7ab015f5a0 2024-09-16T15:39:58.537 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:58.537 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":18,"btime":"2024-09-16T15:39:48:209143+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4866,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2562305658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2562305658},{"type":"v1","addr":"172.21.15.60:6833","nonce":2562305658}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/664909622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":664909622},{"type":"v1","addr":"172.21.15.100:6836","nonce":664909622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/801262237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":801262237},{"type":"v1","addr":"172.21.15.100:6837","nonce":801262237}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:32.603941+0000","modified":"2024-09-16T15:39:35.605148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/846626570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":846626570},{"type":"v1","addr":"172.21.15.60:6835","nonce":846626570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4881,"qdb_cluster":[4881]},"id":2}]} 2024-09-16T15:39:58.537 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 18 2024-09-16T15:39:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a90078a00 msgr2=0x7f7a9007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a90078a00 0x7f7a9007aec0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4033000 tx=0x7f7aa4004730 comp rx=0 tx=0).stop 2024-09-16T15:39:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 msgr2=0x7f7ab014f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 0x7f7ab014f700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c770 tx=0x7f7aa800cc30 comp rx=0 tx=0).stop 2024-09-16T15:39:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 shutdown_connections 2024-09-16T15:39:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a90078a00 0x7f7a9007aec0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ab015f5a0 0x7f7ab014f700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ab00ccf00 0x7f7ab014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 --2- 172.21.15.60:0/1689121922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ab00ad270 0x7f7ab014c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 >> 172.21.15.60:0/1689121922 conn(0x7f7ab00b1950 msgr2=0x7f7ab015e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 shutdown_connections 2024-09-16T15:39:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.537+0000 7f7ab8200640 1 -- 172.21.15.60:0/1689121922 wait complete. 2024-09-16T15:39:58.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:58.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.713+0000 7f58e9000640 1 -- 172.21.15.60:0/2091161954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 msgr2=0x7f58e40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.713+0000 7f58e9000640 1 --2- 172.21.15.60:0/2091161954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 0x7f58e40b2440 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f58e40cfbb0 tx=0x7f58d402a580 comp rx=0 tx=0).stop 2024-09-16T15:39:58.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.713+0000 7f58e9000640 1 -- 172.21.15.60:0/2091161954 shutdown_connections 2024-09-16T15:39:58.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.713+0000 7f58e9000640 1 --2- 172.21.15.60:0/2091161954 >> v2:172.21.15.100:3300/0 conn(0x7f58e40ccf00 0x7f58e40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.713+0000 7f58e9000640 1 --2- 172.21.15.60:0/2091161954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 0x7f58e40b2440 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.713+0000 7f58e9000640 1 -- 172.21.15.60:0/2091161954 >> 172.21.15.60:0/2091161954 conn(0x7f58e40b1950 msgr2=0x7f58e40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:58.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.713+0000 7f58e9000640 1 -- 172.21.15.60:0/2091161954 shutdown_connections 2024-09-16T15:39:58.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.713+0000 7f58e9000640 1 -- 172.21.15.60:0/2091161954 wait complete. 2024-09-16T15:39:58.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.715+0000 7f58e9000640 1 Processor -- start 2024-09-16T15:39:58.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.715+0000 7f58e9000640 1 -- start start 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e9000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e40ad270 0x7f58e40cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e40ccf00 0x7f58e40c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 0x7f58e40c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e9000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f58e406f160 con 0x7f58e415f5a0 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e9000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58e406f2e0 con 0x7f58e40ccf00 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e9000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f58e406f460 con 0x7f58e40ad270 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 0x7f58e40c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e40ad270 0x7f58e40cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58d9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e40ccf00 0x7f58e40c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 0x7f58e40c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59688/0 (socket says 172.21.15.60:59688) 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e40ad270 0x7f58e40cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35938/0 (socket says 172.21.15.60:35938) 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.716+0000 7f58e2800640 1 -- 172.21.15.60:0/935638749 learned_addr learned my addr 172.21.15.60:0/935638749 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:58.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58d9400640 1 -- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e40ad270 msgr2=0x7f58e40cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58d9400640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e40ad270 0x7f58e40cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58d9400640 1 -- 172.21.15.60:0/935638749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 msgr2=0x7f58e40c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58d9400640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 0x7f58e40c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58d9400640 1 -- 172.21.15.60:0/935638749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e4153eb0 con 0x7f58e40ccf00 2024-09-16T15:39:58.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58d9400640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e40ccf00 0x7f58e40c4eb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00d6d0 tx=0x7f58cc00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:58.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58daa00640 1 -- 172.21.15.60:0/935638749 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc017070 con 0x7f58e40ccf00 2024-09-16T15:39:58.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58daa00640 1 -- 172.21.15.60:0/935638749 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58cc005310 con 0x7f58e40ccf00 2024-09-16T15:39:58.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58daa00640 1 -- 172.21.15.60:0/935638749 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0097c0 con 0x7f58e40ccf00 2024-09-16T15:39:58.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.717+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e4154190 con 0x7f58e40ccf00 2024-09-16T15:39:58.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.718+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e4154750 con 0x7f58e40ccf00 2024-09-16T15:39:58.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.719+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e406f9f0 con 0x7f58e40ccf00 2024-09-16T15:39:58.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.720+0000 7f58daa00640 1 -- 172.21.15.60:0/935638749 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f58cc009b70 con 0x7f58e40ccf00 2024-09-16T15:39:58.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.721+0000 7f58daa00640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58c0078a00 0x7f58c007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.721+0000 7f58daa00640 1 -- 172.21.15.60:0/935638749 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f58cc09ae20 con 0x7f58e40ccf00 2024-09-16T15:39:58.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.721+0000 7f58e1e00640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58c0078a00 0x7f58c007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.722+0000 7f58e1e00640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58c0078a00 0x7f58c007aec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f58d4033000 tx=0x7f58d4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:58.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.725+0000 7f58daa00640 1 -- 172.21.15.60:0/935638749 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f58cc062b50 con 0x7f58e40ccf00 2024-09-16T15:39:58.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.955+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58e4155e50 con 0x7f58e40ccf00 2024-09-16T15:39:58.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.956+0000 7f58daa00640 1 -- 172.21.15.60:0/935638749 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5220 (secure 0 0 0) 0x7f58cc067580 con 0x7f58e40ccf00 2024-09-16T15:39:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:58.958 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":18,"btime":"2024-09-16T15:39:48:209143+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4866,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2562305658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2562305658},{"type":"v1","addr":"172.21.15.60:6833","nonce":2562305658}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/664909622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":664909622},{"type":"v1","addr":"172.21.15.100:6836","nonce":664909622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/801262237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":801262237},{"type":"v1","addr":"172.21.15.100:6837","nonce":801262237}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:32.603941+0000","modified":"2024-09-16T15:39:35.605148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/846626570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":846626570},{"type":"v1","addr":"172.21.15.60:6835","nonce":846626570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4881,"qdb_cluster":[4881]},"id":2}]} 2024-09-16T15:39:58.958 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 18 2024-09-16T15:39:58.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58c0078a00 msgr2=0x7f58c007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58c0078a00 0x7f58c007aec0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f58d4033000 tx=0x7f58d4004730 comp rx=0 tx=0).stop 2024-09-16T15:39:58.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e40ccf00 msgr2=0x7f58e40c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e40ccf00 0x7f58e40c4eb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00d6d0 tx=0x7f58cc00db90 comp rx=0 tx=0).stop 2024-09-16T15:39:58.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 shutdown_connections 2024-09-16T15:39:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f58c0078a00 0x7f58c007aec0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58e415f5a0 0x7f58e40c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58e40ccf00 0x7f58e40c4eb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 --2- 172.21.15.60:0/935638749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58e40ad270 0x7f58e40cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 >> 172.21.15.60:0/935638749 conn(0x7f58e40b1950 msgr2=0x7f58e40ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 shutdown_connections 2024-09-16T15:39:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:58.959+0000 7f58e9000640 1 -- 172.21.15.60:0/935638749 wait complete. 2024-09-16T15:39:58.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.135+0000 7f92e6e00640 1 -- 172.21.15.60:0/329555873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92e00d1ec0 msgr2=0x7f92e00d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.135+0000 7f92e6e00640 1 --2- 172.21.15.60:0/329555873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92e00d1ec0 0x7f92e00d22a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009830 tx=0x7f92d402a530 comp rx=0 tx=0).stop 2024-09-16T15:39:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.135+0000 7f92e6e00640 1 -- 172.21.15.60:0/329555873 shutdown_connections 2024-09-16T15:39:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.135+0000 7f92e6e00640 1 --2- 172.21.15.60:0/329555873 >> v2:172.21.15.100:3300/0 conn(0x7f92e00b28d0 0x7f92e0162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.135+0000 7f92e6e00640 1 --2- 172.21.15.60:0/329555873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92e00d1ec0 0x7f92e00d22a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.135+0000 7f92e6e00640 1 -- 172.21.15.60:0/329555873 >> 172.21.15.60:0/329555873 conn(0x7f92e00d3de0 msgr2=0x7f92e00d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.135+0000 7f92e6e00640 1 -- 172.21.15.60:0/329555873 shutdown_connections 2024-09-16T15:39:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.135+0000 7f92e6e00640 1 -- 172.21.15.60:0/329555873 wait complete. 2024-09-16T15:39:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.137+0000 7f92e6e00640 1 Processor -- start 2024-09-16T15:39:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.137+0000 7f92e6e00640 1 -- start start 2024-09-16T15:39:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.137+0000 7f92e6e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92e00b28d0 0x7f92e0156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.137+0000 7f92e6e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e00b4d10 0x7f92e01459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e6e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92e00d1ec0 0x7f92e01479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e6e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f92e00b3a80 con 0x7f92e00b28d0 2024-09-16T15:39:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e6e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92e00b3900 con 0x7f92e00b4d10 2024-09-16T15:39:59.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e6e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f92e00b3c00 con 0x7f92e00d1ec0 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92e00b28d0 0x7f92e0156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92e00b28d0 0x7f92e0156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50378/0 (socket says 172.21.15.60:50378) 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5a00640 1 -- 172.21.15.60:0/1667652627 learned_addr learned my addr 172.21.15.60:0/1667652627 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e6400640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92e00d1ec0 0x7f92e01479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5000640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e00b4d10 0x7f92e01459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5a00640 1 -- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92e00d1ec0 msgr2=0x7f92e01479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5a00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92e00d1ec0 0x7f92e01479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5a00640 1 -- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e00b4d10 msgr2=0x7f92e01459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5a00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e00b4d10 0x7f92e01459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.138+0000 7f92e5a00640 1 -- 172.21.15.60:0/1667652627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e0147f20 con 0x7f92e00b28d0 2024-09-16T15:39:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.139+0000 7f92e5a00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92e00b28d0 0x7f92e0156500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92d4000c00 tx=0x7f92d4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.139+0000 7f92dd600640 1 -- 172.21.15.60:0/1667652627 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4031d60 con 0x7f92e00b28d0 2024-09-16T15:39:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.139+0000 7f92dd600640 1 -- 172.21.15.60:0/1667652627 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92d4033e10 con 0x7f92e00b28d0 2024-09-16T15:39:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.139+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e00b88d0 con 0x7f92e00b28d0 2024-09-16T15:39:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.139+0000 7f92dd600640 1 -- 172.21.15.60:0/1667652627 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4033310 con 0x7f92e00b28d0 2024-09-16T15:39:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.139+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e0148470 con 0x7f92e00b28d0 2024-09-16T15:39:59.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.140+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e00d22a0 con 0x7f92e00b28d0 2024-09-16T15:39:59.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.141+0000 7f92dd600640 1 -- 172.21.15.60:0/1667652627 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f92d40334b0 con 0x7f92e00b28d0 2024-09-16T15:39:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.142+0000 7f92dd600640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f92c0078a00 0x7f92c007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.142+0000 7f92dd600640 1 -- 172.21.15.60:0/1667652627 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f92d40bbf90 con 0x7f92e00b28d0 2024-09-16T15:39:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.143+0000 7f92e5000640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f92c0078a00 0x7f92c007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.143+0000 7f92e5000640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f92c0078a00 0x7f92c007aec0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f92e00d2640 tx=0x7f92cc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.147+0000 7f92dd600640 1 -- 172.21.15.60:0/1667652627 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f92e00b88d0 con 0x7f92e00b28d0 2024-09-16T15:39:59.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.347+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92e015dd10 con 0x7f92e00b28d0 2024-09-16T15:39:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.348+0000 7f92dd600640 1 -- 172.21.15.60:0/1667652627 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16729 (secure 0 0 0) 0x7f92d403f030 con 0x7f92e00b28d0 2024-09-16T15:39:59.350 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:39:59.350 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":36,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:39:34.155062+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:30.169700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:31.550025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.350+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f92c0078a00 msgr2=0x7f92c007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.350+0000 7f92e6e00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f92c0078a00 0x7f92c007aec0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f92e00d2640 tx=0x7f92cc009040 comp rx=0 tx=0).stop 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.350+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92e00b28d0 msgr2=0x7f92e0156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.350+0000 7f92e6e00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92e00b28d0 0x7f92e0156500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92d4000c00 tx=0x7f92d4033c20 comp rx=0 tx=0).stop 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.351+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 shutdown_connections 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.351+0000 7f92e6e00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f92c0078a00 0x7f92c007aec0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.351+0000 7f92e6e00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92e00d1ec0 0x7f92e01479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.351+0000 7f92e6e00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92e00b4d10 0x7f92e01459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.351+0000 7f92e6e00640 1 --2- 172.21.15.60:0/1667652627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92e00b28d0 0x7f92e0156500 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.351+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 >> 172.21.15.60:0/1667652627 conn(0x7f92e00d3de0 msgr2=0x7f92e015e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.351+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 shutdown_connections 2024-09-16T15:39:59.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.351+0000 7f92e6e00640 1 -- 172.21.15.60:0/1667652627 wait complete. 2024-09-16T15:39:59.369 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:39:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.529+0000 7f9c9e000640 1 -- 172.21.15.60:0/2452782440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c98073fb0 msgr2=0x7f9c98074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.529+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2452782440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c98073fb0 0x7f9c98074460 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9c90005fd0 tx=0x7f9c9002aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.529+0000 7f9c9e000640 1 -- 172.21.15.60:0/2452782440 shutdown_connections 2024-09-16T15:39:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.529+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2452782440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c98073fb0 0x7f9c98074460 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.529+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2452782440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c9815f5a0 0x7f9c980b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.529+0000 7f9c9e000640 1 -- 172.21.15.60:0/2452782440 >> 172.21.15.60:0/2452782440 conn(0x7f9c980b1950 msgr2=0x7f9c980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.529+0000 7f9c9e000640 1 -- 172.21.15.60:0/2452782440 shutdown_connections 2024-09-16T15:39:59.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.529+0000 7f9c9e000640 1 -- 172.21.15.60:0/2452782440 wait complete. 2024-09-16T15:39:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.531+0000 7f9c9e000640 1 Processor -- start 2024-09-16T15:39:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.531+0000 7f9c9e000640 1 -- start start 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c98073fb0 0x7f9c9814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c980ccf00 0x7f9c981489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c9815f5a0 0x7f9c9815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c9e000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9806f2e0 con 0x7f9c9815f5a0 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c9e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9806f160 con 0x7f9c98073fb0 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c9e000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9806f460 con 0x7f9c980ccf00 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c97e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c9815f5a0 0x7f9c9815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c97e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c9815f5a0 0x7f9c9815da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50394/0 (socket says 172.21.15.60:50394) 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.532+0000 7f9c97e00640 1 -- 172.21.15.60:0/2779845717 learned_addr learned my addr 172.21.15.60:0/2779845717 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c8ea00640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c980ccf00 0x7f9c981489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c97400640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c98073fb0 0x7f9c9814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c97e00640 1 -- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c980ccf00 msgr2=0x7f9c981489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c97e00640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c980ccf00 0x7f9c981489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c97e00640 1 -- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c98073fb0 msgr2=0x7f9c9814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c97e00640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c98073fb0 0x7f9c9814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c97e00640 1 -- 172.21.15.60:0/2779845717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9814b880 con 0x7f9c9815f5a0 2024-09-16T15:39:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c8ea00640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c980ccf00 0x7f9c981489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c97e00640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c9815f5a0 0x7f9c9815da20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006010 tx=0x7f9c90003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c8fe00640 1 -- 172.21.15.60:0/2779845717 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90047070 con 0x7f9c9815f5a0 2024-09-16T15:39:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.533+0000 7f9c8fe00640 1 -- 172.21.15.60:0/2779845717 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c9002f670 con 0x7f9c9815f5a0 2024-09-16T15:39:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.534+0000 7f9c8fe00640 1 -- 172.21.15.60:0/2779845717 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90033050 con 0x7f9c9815f5a0 2024-09-16T15:39:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.534+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9814bb60 con 0x7f9c9815f5a0 2024-09-16T15:39:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.534+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c98149290 con 0x7f9c9815f5a0 2024-09-16T15:39:59.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.535+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9806f9f0 con 0x7f9c9815f5a0 2024-09-16T15:39:59.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.536+0000 7f9c8fe00640 1 -- 172.21.15.60:0/2779845717 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f9c9004a070 con 0x7f9c9815f5a0 2024-09-16T15:39:59.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.537+0000 7f9c8fe00640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9c74078a00 0x7f9c7407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.537+0000 7f9c8fe00640 1 -- 172.21.15.60:0/2779845717 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f9c900bbca0 con 0x7f9c9815f5a0 2024-09-16T15:39:59.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.537+0000 7f9c97400640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9c74078a00 0x7f9c7407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.538+0000 7f9c97400640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9c74078a00 0x7f9c7407aec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009b30 tx=0x7f9c88005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.541+0000 7f9c8fe00640 1 -- 172.21.15.60:0/2779845717 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c900839d0 con 0x7f9c9815f5a0 2024-09-16T15:39:59.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.778+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9c980b16b0 con 0x7f9c9815f5a0 2024-09-16T15:39:59.779 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:39:59.778+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:39:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.780+0000 7f9c8fe00640 1 -- 172.21.15.60:0/2779845717 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f9c90088400 con 0x7f9c9815f5a0 2024-09-16T15:39:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:39:59.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9c74078a00 msgr2=0x7f9c7407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9c74078a00 0x7f9c7407aec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009b30 tx=0x7f9c88005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c9815f5a0 msgr2=0x7f9c9815da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c9815f5a0 0x7f9c9815da20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006010 tx=0x7f9c90003ef0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 shutdown_connections 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9c74078a00 0x7f9c7407aec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c9815f5a0 0x7f9c9815da20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c980ccf00 0x7f9c981489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 --2- 172.21.15.60:0/2779845717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c98073fb0 0x7f9c9814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 >> 172.21.15.60:0/2779845717 conn(0x7f9c980b1950 msgr2=0x7f9c980acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 shutdown_connections 2024-09-16T15:39:59.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.784+0000 7f9c9e000640 1 -- 172.21.15.60:0/2779845717 wait complete. 2024-09-16T15:39:59.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:39:59.852 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:39:59.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 -- 172.21.15.60:0/1638118742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 msgr2=0x7f5370073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 --2- 172.21.15.60:0/1638118742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 0x7f5370073ef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f53700c9ff0 tx=0x7f535c02a560 comp rx=0 tx=0).stop 2024-09-16T15:39:59.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 -- 172.21.15.60:0/1638118742 shutdown_connections 2024-09-16T15:39:59.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 --2- 172.21.15.60:0/1638118742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53700cb4f0 0x7f53700cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 --2- 172.21.15.60:0/1638118742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 0x7f5370073ef0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 --2- 172.21.15.60:0/1638118742 >> v2:172.21.15.100:3300/0 conn(0x7f537015bec0 0x7f537006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 -- 172.21.15.60:0/1638118742 >> 172.21.15.60:0/1638118742 conn(0x7f53700cf9f0 msgr2=0x7f53700cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 -- 172.21.15.60:0/1638118742 shutdown_connections 2024-09-16T15:39:59.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.970+0000 7f5376000640 1 -- 172.21.15.60:0/1638118742 wait complete. 2024-09-16T15:39:59.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.972+0000 7f5376000640 1 Processor -- start 2024-09-16T15:39:59.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.972+0000 7f5376000640 1 -- start start 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5376000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 0x7f53700a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5376000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53700cb4f0 0x7f537015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5376000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537015bec0 0x7f53700aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5376000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53700ae850 con 0x7f5370072560 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5376000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53700ae6d0 con 0x7f537015bec0 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5376000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f53700ae9d0 con 0x7f53700cb4f0 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5374c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 0x7f53700a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5374c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 0x7f53700a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50440/0 (socket says 172.21.15.60:50440) 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5374c00640 1 -- 172.21.15.60:0/3239592212 learned_addr learned my addr 172.21.15.60:0/3239592212 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f5375600640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537015bec0 0x7f53700aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.973+0000 7f536fe00640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53700cb4f0 0x7f537015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f5374c00640 1 -- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53700cb4f0 msgr2=0x7f537015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f5374c00640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53700cb4f0 0x7f537015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f5374c00640 1 -- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537015bec0 msgr2=0x7f53700aaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f5374c00640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537015bec0 0x7f53700aaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f5374c00640 1 -- 172.21.15.60:0/3239592212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53701f60d0 con 0x7f5370072560 2024-09-16T15:39:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f5374c00640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 0x7f53700a9470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f536400e790 tx=0x7f536400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f536cc00640 1 -- 172.21.15.60:0/3239592212 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536400cab0 con 0x7f5370072560 2024-09-16T15:39:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f536cc00640 1 -- 172.21.15.60:0/3239592212 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f536400cd40 con 0x7f5370072560 2024-09-16T15:39:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53701f63b0 con 0x7f5370072560 2024-09-16T15:39:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.974+0000 7f536cc00640 1 -- 172.21.15.60:0/3239592212 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364010690 con 0x7f5370072560 2024-09-16T15:39:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.975+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53701f6970 con 0x7f5370072560 2024-09-16T15:39:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.976+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53700b4a10 con 0x7f5370072560 2024-09-16T15:39:59.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.977+0000 7f536cc00640 1 -- 172.21.15.60:0/3239592212 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f5364010830 con 0x7f5370072560 2024-09-16T15:39:59.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.978+0000 7f536cc00640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5350078a00 0x7f535007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.978+0000 7f536cc00640 1 -- 172.21.15.60:0/3239592212 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7f536409b6b0 con 0x7f5370072560 2024-09-16T15:39:59.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.978+0000 7f536fe00640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5350078a00 0x7f535007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.979+0000 7f536fe00640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5350078a00 0x7f535007aec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f535c033000 tx=0x7f535c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:39:59.982+0000 7f536cc00640 1 -- 172.21.15.60:0/3239592212 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f53700b4a10 con 0x7f5370072560 2024-09-16T15:40:00.000 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:39:59.999+0000 7f2a14200640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-09-16T15:40:00.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.217+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f53700aeef0 con 0x7f5370072560 2024-09-16T15:40:00.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:40:00.269 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:40:00.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.782+0000 7f536cc00640 1 -- 172.21.15.60:0/3239592212 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f5364063240 con 0x7f5370072560 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5350078a00 msgr2=0x7f535007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5350078a00 0x7f535007aec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f535c033000 tx=0x7f535c004730 comp rx=0 tx=0).stop 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 msgr2=0x7f53700a9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 0x7f53700a9470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f536400e790 tx=0x7f536400eb90 comp rx=0 tx=0).stop 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 shutdown_connections 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5350078a00 0x7f535007aec0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f537015bec0 0x7f53700aaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53700cb4f0 0x7f537015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 --2- 172.21.15.60:0/3239592212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5370072560 0x7f53700a9470 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 >> 172.21.15.60:0/3239592212 conn(0x7f53700cf9f0 msgr2=0x7f53700ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:00.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.785+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 shutdown_connections 2024-09-16T15:40:00.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.786+0000 7f5376000640 1 -- 172.21.15.60:0/3239592212 wait complete. 2024-09-16T15:40:00.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:40:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.970+0000 7f6c3e400640 1 -- 172.21.15.60:0/1417682556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3806f220 msgr2=0x7f6c3806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.970+0000 7f6c3e400640 1 --2- 172.21.15.60:0/1417682556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3806f220 0x7f6c3806f6b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b2680 tx=0x7f6c3002aac0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.971+0000 7f6c3e400640 1 -- 172.21.15.60:0/1417682556 shutdown_connections 2024-09-16T15:40:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.971+0000 7f6c3e400640 1 --2- 172.21.15.60:0/1417682556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3806f220 0x7f6c3806f6b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.971+0000 7f6c3e400640 1 --2- 172.21.15.60:0/1417682556 >> v2:172.21.15.100:3300/0 conn(0x7f6c38159aa0 0x7f6c380741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.971+0000 7f6c3e400640 1 -- 172.21.15.60:0/1417682556 >> 172.21.15.60:0/1417682556 conn(0x7f6c380b18e0 msgr2=0x7f6c380b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.971+0000 7f6c3e400640 1 -- 172.21.15.60:0/1417682556 shutdown_connections 2024-09-16T15:40:00.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.971+0000 7f6c3e400640 1 -- 172.21.15.60:0/1417682556 wait complete. 2024-09-16T15:40:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.972+0000 7f6c3e400640 1 Processor -- start 2024-09-16T15:40:00.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.973+0000 7f6c3e400640 1 -- start start 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.973+0000 7f6c3e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c3806f220 0x7f6c380d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.973+0000 7f6c3e400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c38159aa0 0x7f6c3809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.973+0000 7f6c3e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3815cff0 0x7f6c380b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.973+0000 7f6c3e400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6c380cc520 con 0x7f6c3815cff0 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.973+0000 7f6c3e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c380cc3a0 con 0x7f6c3806f220 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.973+0000 7f6c3e400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6c380cc6a0 con 0x7f6c38159aa0 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3815cff0 0x7f6c380b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3815cff0 0x7f6c380b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50460/0 (socket says 172.21.15.60:50460) 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 -- 172.21.15.60:0/4263716251 learned_addr learned my addr 172.21.15.60:0/4263716251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c3806f220 0x7f6c380d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c2fe00640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c38159aa0 0x7f6c3809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 -- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c38159aa0 msgr2=0x7f6c3809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c38159aa0 0x7f6c3809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 -- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c3806f220 msgr2=0x7f6c380d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c3806f220 0x7f6c380d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 -- 172.21.15.60:0/4263716251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c381f4ce0 con 0x7f6c3815cff0 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c2fe00640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c38159aa0 0x7f6c3809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.974+0000 7f6c37e00640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3815cff0 0x7f6c380b30c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009d60 tx=0x7f6c30003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.975+0000 7f6c2f400640 1 -- 172.21.15.60:0/4263716251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30044070 con 0x7f6c3815cff0 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.975+0000 7f6c2f400640 1 -- 172.21.15.60:0/4263716251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c30004250 con 0x7f6c3815cff0 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.975+0000 7f6c2f400640 1 -- 172.21.15.60:0/4263716251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30004680 con 0x7f6c3815cff0 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.975+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c381f4fc0 con 0x7f6c3815cff0 2024-09-16T15:40:00.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.975+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c381f5580 con 0x7f6c3815cff0 2024-09-16T15:40:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.976+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c38069360 con 0x7f6c3815cff0 2024-09-16T15:40:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.977+0000 7f6c2f400640 1 -- 172.21.15.60:0/4263716251 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f6c3002c840 con 0x7f6c3815cff0 2024-09-16T15:40:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.978+0000 7f6c2f400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6c14078a00 0x7f6c1407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.978+0000 7f6c2f400640 1 -- 172.21.15.60:0/4263716251 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5795+0+0 (secure 0 0 0) 0x7f6c300bb380 con 0x7f6c3815cff0 2024-09-16T15:40:00.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.978+0000 7f6c37400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6c14078a00 0x7f6c1407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.979+0000 7f6c37400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6c14078a00 0x7f6c1407aec0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005fd0 tx=0x7f6c28005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:00.982+0000 7f6c2f400640 1 -- 172.21.15.60:0/4263716251 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6c30032030 con 0x7f6c3815cff0 2024-09-16T15:40:01.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.177+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6c380ccbf0 con 0x7f6c3815cff0 2024-09-16T15:40:01.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.785+0000 7f6c2f400640 1 -- 172.21.15.60:0/4263716251 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) ==== 181+0+0 (secure 0 0 0) 0x7f6c30004820 con 0x7f6c3815cff0 2024-09-16T15:40:01.786 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:40:01.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.788+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6c14078a00 msgr2=0x7f6c1407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6c14078a00 0x7f6c1407aec0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005fd0 tx=0x7f6c28005850 comp rx=0 tx=0).stop 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3815cff0 msgr2=0x7f6c380b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3815cff0 0x7f6c380b30c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009d60 tx=0x7f6c30003f10 comp rx=0 tx=0).stop 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 shutdown_connections 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6c14078a00 0x7f6c1407aec0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c3815cff0 0x7f6c380b30c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c38159aa0 0x7f6c3809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 --2- 172.21.15.60:0/4263716251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c3806f220 0x7f6c380d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 >> 172.21.15.60:0/4263716251 conn(0x7f6c380b18e0 msgr2=0x7f6c380aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 shutdown_connections 2024-09-16T15:40:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.789+0000 7f6c3e400640 1 -- 172.21.15.60:0/4263716251 wait complete. 2024-09-16T15:40:01.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:40:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.978+0000 7ff1b8a00640 1 -- 172.21.15.60:0/654284989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1b415dfc0 msgr2=0x7ff1b406f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.978+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/654284989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1b415dfc0 0x7ff1b406f300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4005fd0 tx=0x7ff1a402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.978+0000 7ff1b8a00640 1 -- 172.21.15.60:0/654284989 shutdown_connections 2024-09-16T15:40:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.978+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/654284989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1b40b4770 0x7ff1b406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.978+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/654284989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1b415dfc0 0x7ff1b406f300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.978+0000 7ff1b8a00640 1 -- 172.21.15.60:0/654284989 >> 172.21.15.60:0/654284989 conn(0x7ff1b40b18e0 msgr2=0x7ff1b40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.978+0000 7ff1b8a00640 1 -- 172.21.15.60:0/654284989 shutdown_connections 2024-09-16T15:40:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.978+0000 7ff1b8a00640 1 -- 172.21.15.60:0/654284989 wait complete. 2024-09-16T15:40:01.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.980+0000 7ff1b8a00640 1 Processor -- start 2024-09-16T15:40:01.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.980+0000 7ff1b8a00640 1 -- start start 2024-09-16T15:40:01.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1b406ec30 0x7ff1b41f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1b40b4770 0x7ff1b414a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b8a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1b415dfc0 0x7ff1b415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b8a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41622c0 con 0x7ff1b406ec30 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b40ae8f0 con 0x7ff1b40b4770 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b8a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b4162440 con 0x7ff1b415dfc0 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1b406ec30 0x7ff1b41f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1b406ec30 0x7ff1b41f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50470/0 (socket says 172.21.15.60:50470) 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b1e00640 1 -- 172.21.15.60:0/3503579891 learned_addr learned my addr 172.21.15.60:0/3503579891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1a9400640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1b40b4770 0x7ff1b414a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.981+0000 7ff1b2800640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1b415dfc0 0x7ff1b415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1b1e00640 1 -- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1b415dfc0 msgr2=0x7ff1b415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1b1e00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1b415dfc0 0x7ff1b415b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1b1e00640 1 -- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1b40b4770 msgr2=0x7ff1b414a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1b1e00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1b40b4770 0x7ff1b414a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1b1e00640 1 -- 172.21.15.60:0/3503579891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b41f5600 con 0x7ff1b406ec30 2024-09-16T15:40:01.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1b1e00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1b406ec30 0x7ff1b41f4e60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4034040 tx=0x7ff1a40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1aaa00640 1 -- 172.21.15.60:0/3503579891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a403e030 con 0x7ff1b406ec30 2024-09-16T15:40:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1aaa00640 1 -- 172.21.15.60:0/3503579891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1a4003ae0 con 0x7ff1b406ec30 2024-09-16T15:40:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1aaa00640 1 -- 172.21.15.60:0/3503579891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4035040 con 0x7ff1b406ec30 2024-09-16T15:40:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b41f58e0 con 0x7ff1b406ec30 2024-09-16T15:40:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.982+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b414aa80 con 0x7ff1b406ec30 2024-09-16T15:40:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.983+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b4069360 con 0x7ff1b406ec30 2024-09-16T15:40:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.985+0000 7ff1aaa00640 1 -- 172.21.15.60:0/3503579891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7ff1a4041030 con 0x7ff1b406ec30 2024-09-16T15:40:01.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.986+0000 7ff1aaa00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff190078a00 0x7ff19007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.986+0000 7ff1aaa00640 1 -- 172.21.15.60:0/3503579891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5357+0+0 (secure 0 0 0) 0x7ff1a40bb2f0 con 0x7ff1b406ec30 2024-09-16T15:40:01.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.986+0000 7ff1a9400640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff190078a00 0x7ff19007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.987+0000 7ff1a9400640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff190078a00 0x7ff19007aec0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40cd360 tx=0x7ff19c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:01.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:01.990+0000 7ff1aaa00640 1 -- 172.21.15.60:0/3503579891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff1a4083110 con 0x7ff1b406ec30 2024-09-16T15:40:02.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.193+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff1b41f5e00 con 0x7ff1b406ec30 2024-09-16T15:40:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.787+0000 7ff1aaa00640 1 -- 172.21.15.60:0/3503579891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) ==== 169+0+0 (secure 0 0 0) 0x7ff1b41f5e00 con 0x7ff1b406ec30 2024-09-16T15:40:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:40:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff190078a00 msgr2=0x7ff19007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff190078a00 0x7ff19007aec0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40cd360 tx=0x7ff19c005f90 comp rx=0 tx=0).stop 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1b406ec30 msgr2=0x7ff1b41f4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1b406ec30 0x7ff1b41f4e60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4034040 tx=0x7ff1a40035f0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 shutdown_connections 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff190078a00 0x7ff19007aec0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1b415dfc0 0x7ff1b415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1b40b4770 0x7ff1b414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 --2- 172.21.15.60:0/3503579891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1b406ec30 0x7ff1b41f4e60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 >> 172.21.15.60:0/3503579891 conn(0x7ff1b40b18e0 msgr2=0x7ff1b40af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 shutdown_connections 2024-09-16T15:40:02.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.791+0000 7ff1b8a00640 1 -- 172.21.15.60:0/3503579891 wait complete. 2024-09-16T15:40:02.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:40:02.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.978+0000 7f4496400640 1 -- 172.21.15.60:0/2092020162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 msgr2=0x7f4490074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.978+0000 7f4496400640 1 --2- 172.21.15.60:0/2092020162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 0x7f4490074420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f44900cfbb0 tx=0x7f448802aa30 comp rx=0 tx=0).stop 2024-09-16T15:40:02.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.978+0000 7f4496400640 1 -- 172.21.15.60:0/2092020162 shutdown_connections 2024-09-16T15:40:02.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.978+0000 7f4496400640 1 --2- 172.21.15.60:0/2092020162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 0x7f4490074420 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.978+0000 7f4496400640 1 --2- 172.21.15.60:0/2092020162 >> v2:172.21.15.100:3300/0 conn(0x7f44900ccf00 0x7f44900b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.978+0000 7f4496400640 1 -- 172.21.15.60:0/2092020162 >> 172.21.15.60:0/2092020162 conn(0x7f44900b1950 msgr2=0x7f44900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:02.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.978+0000 7f4496400640 1 -- 172.21.15.60:0/2092020162 shutdown_connections 2024-09-16T15:40:02.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.978+0000 7f4496400640 1 -- 172.21.15.60:0/2092020162 wait complete. 2024-09-16T15:40:02.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.980+0000 7f4496400640 1 Processor -- start 2024-09-16T15:40:02.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.980+0000 7f4496400640 1 -- start start 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f4496400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 0x7f449014f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f4496400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900ccf00 0x7f4490148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f4496400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f449015f5a0 0x7f44900d0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f4496400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f449006f2e0 con 0x7f4490073fb0 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f4496400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f449006f160 con 0x7f44900ccf00 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f4496400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f449006f460 con 0x7f449015f5a0 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 0x7f449014f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 0x7f449014f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50482/0 (socket says 172.21.15.60:50482) 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448f400640 1 -- 172.21.15.60:0/3414344503 learned_addr learned my addr 172.21.15.60:0/3414344503 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448ea00640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900ccf00 0x7f4490148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448fe00640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f449015f5a0 0x7f44900d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448f400640 1 -- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f449015f5a0 msgr2=0x7f44900d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448f400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f449015f5a0 0x7f44900d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448f400640 1 -- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900ccf00 msgr2=0x7f4490148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448f400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900ccf00 0x7f4490148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.981+0000 7f448f400640 1 -- 172.21.15.60:0/3414344503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44900d0ad0 con 0x7f4490073fb0 2024-09-16T15:40:02.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.982+0000 7f448f400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 0x7f449014f630 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f448000c3a0 tx=0x7f448000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.982+0000 7f4487400640 1 -- 172.21.15.60:0/3414344503 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480017070 con 0x7f4490073fb0 2024-09-16T15:40:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.982+0000 7f4487400640 1 -- 172.21.15.60:0/3414344503 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44800053c0 con 0x7f4490073fb0 2024-09-16T15:40:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.982+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449009cfc0 con 0x7f4490073fb0 2024-09-16T15:40:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.982+0000 7f4487400640 1 -- 172.21.15.60:0/3414344503 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480003850 con 0x7f4490073fb0 2024-09-16T15:40:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.982+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f449009cce0 con 0x7f4490073fb0 2024-09-16T15:40:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.983+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449006f9f0 con 0x7f4490073fb0 2024-09-16T15:40:02.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.984+0000 7f4487400640 1 -- 172.21.15.60:0/3414344503 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f448001a070 con 0x7f4490073fb0 2024-09-16T15:40:02.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.985+0000 7f4487400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4470078a00 0x7f447007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:02.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.985+0000 7f4487400640 1 -- 172.21.15.60:0/3414344503 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f4480099d50 con 0x7f4490073fb0 2024-09-16T15:40:02.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.986+0000 7f448ea00640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4470078a00 0x7f447007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:02.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.987+0000 7f448ea00640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4470078a00 0x7f447007aec0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f44900ccea0 tx=0x7f447c00b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:02.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:02.990+0000 7f4487400640 1 -- 172.21.15.60:0/3414344503 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4480061cf0 con 0x7f4490073fb0 2024-09-16T15:40:03.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.183+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f44900b3900 con 0x7f4490073fb0 2024-09-16T15:40:03.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.187+0000 7f4487400640 1 -- 172.21.15.60:0/3414344503 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v29) ==== 71+0+0 (secure 0 0 0) 0x7f4480066720 con 0x7f4490073fb0 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4470078a00 msgr2=0x7f447007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4470078a00 0x7f447007aec0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f44900ccea0 tx=0x7f447c00b390 comp rx=0 tx=0).stop 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 msgr2=0x7f449014f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 0x7f449014f630 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f448000c3a0 tx=0x7f448000c860 comp rx=0 tx=0).stop 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 shutdown_connections 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4470078a00 0x7f447007aec0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f449015f5a0 0x7f44900d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44900ccf00 0x7f4490148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 --2- 172.21.15.60:0/3414344503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4490073fb0 0x7f449014f630 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.189+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 >> 172.21.15.60:0/3414344503 conn(0x7f44900b1950 msgr2=0x7f44900b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.190+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 shutdown_connections 2024-09-16T15:40:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.190+0000 7f4496400640 1 -- 172.21.15.60:0/3414344503 wait complete. 2024-09-16T15:40:03.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-16T15:40:03.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.372+0000 7fdc11c00640 1 -- 172.21.15.60:0/3014334708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c0b4020 msgr2=0x7fdc0c0b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.372+0000 7fdc11c00640 1 --2- 172.21.15.60:0/3014334708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c0b4020 0x7fdc0c0b1770 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0d4ea0 tx=0x7fdc0402a9f0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.372+0000 7fdc11c00640 1 -- 172.21.15.60:0/3014334708 shutdown_connections 2024-09-16T15:40:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.372+0000 7fdc11c00640 1 --2- 172.21.15.60:0/3014334708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c0b4020 0x7fdc0c0b1770 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.372+0000 7fdc11c00640 1 --2- 172.21.15.60:0/3014334708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc0c161440 0x7fdc0c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.372+0000 7fdc11c00640 1 -- 172.21.15.60:0/3014334708 >> 172.21.15.60:0/3014334708 conn(0x7fdc0c0d3df0 msgr2=0x7fdc0c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.372+0000 7fdc11c00640 1 -- 172.21.15.60:0/3014334708 shutdown_connections 2024-09-16T15:40:03.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.372+0000 7fdc11c00640 1 -- 172.21.15.60:0/3014334708 wait complete. 2024-09-16T15:40:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.374+0000 7fdc11c00640 1 Processor -- start 2024-09-16T15:40:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.374+0000 7fdc11c00640 1 -- start start 2024-09-16T15:40:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc0c0b34b0 0x7fdc0c0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc0c0b4020 0x7fdc0c06e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c161440 0x7fdc0c1f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c0b2040 con 0x7fdc0c161440 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c0b1ec0 con 0x7fdc0c0b34b0 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0c0b21c0 con 0x7fdc0c0b4020 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c161440 0x7fdc0c1f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c161440 0x7fdc0c1f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50488/0 (socket says 172.21.15.60:50488) 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc11200640 1 -- 172.21.15.60:0/2571820845 learned_addr learned my addr 172.21.15.60:0/2571820845 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc0be00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc0c0b34b0 0x7fdc0c0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.375+0000 7fdc0b400640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc0c0b4020 0x7fdc0c06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc11200640 1 -- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc0c0b4020 msgr2=0x7fdc0c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc11200640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc0c0b4020 0x7fdc0c06e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc11200640 1 -- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc0c0b34b0 msgr2=0x7fdc0c0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc11200640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc0c0b34b0 0x7fdc0c0caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc11200640 1 -- 172.21.15.60:0/2571820845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c0ccaa0 con 0x7fdc0c161440 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc11200640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c161440 0x7fdc0c1f3850 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009d60 tx=0x7fdc04003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc03e00640 1 -- 172.21.15.60:0/2571820845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04047070 con 0x7fdc0c161440 2024-09-16T15:40:03.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc03e00640 1 -- 172.21.15.60:0/2571820845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc04004250 con 0x7fdc0c161440 2024-09-16T15:40:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc03e00640 1 -- 172.21.15.60:0/2571820845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04033050 con 0x7fdc0c161440 2024-09-16T15:40:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c0b72f0 con 0x7fdc0c161440 2024-09-16T15:40:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.376+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c0ccd20 con 0x7fdc0c161440 2024-09-16T15:40:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.377+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0c0b1770 con 0x7fdc0c161440 2024-09-16T15:40:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.379+0000 7fdc03e00640 1 -- 172.21.15.60:0/2571820845 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fdc0404a070 con 0x7fdc0c161440 2024-09-16T15:40:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.380+0000 7fdc03e00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbec078a00 0x7fdbec07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.380+0000 7fdc03e00640 1 -- 172.21.15.60:0/2571820845 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fdc040bba80 con 0x7fdc0c161440 2024-09-16T15:40:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.380+0000 7fdc0be00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbec078a00 0x7fdbec07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.381+0000 7fdc0be00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbec078a00 0x7fdbec07aec0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc004630 tx=0x7fdbfc00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:03.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.384+0000 7fdc03e00640 1 -- 172.21.15.60:0/2571820845 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdc0c0ccaa0 con 0x7fdc0c161440 2024-09-16T15:40:03.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.659+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fdc0c0cb400 con 0x7fdc0c161440 2024-09-16T15:40:03.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.795+0000 7fdc03e00640 1 -- 172.21.15.60:0/2571820845 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v105) ==== 158+0+0 (secure 0 0 0) 0x7fdc04083ac0 con 0x7fdc0c161440 2024-09-16T15:40:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbec078a00 msgr2=0x7fdbec07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbec078a00 0x7fdbec07aec0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc004630 tx=0x7fdbfc00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c161440 msgr2=0x7fdc0c1f3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c161440 0x7fdc0c1f3850 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009d60 tx=0x7fdc04003f10 comp rx=0 tx=0).stop 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 shutdown_connections 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbec078a00 0x7fdbec07aec0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc0c161440 0x7fdc0c1f3850 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc0c0b4020 0x7fdc0c06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 --2- 172.21.15.60:0/2571820845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc0c0b34b0 0x7fdc0c0caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 >> 172.21.15.60:0/2571820845 conn(0x7fdc0c0d3df0 msgr2=0x7fdc0c0b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 shutdown_connections 2024-09-16T15:40:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.798+0000 7fdc11c00640 1 -- 172.21.15.60:0/2571820845 wait complete. 2024-09-16T15:40:03.815 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-16T15:40:03.816 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:40:03.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.975+0000 7f9d80c00640 1 -- 172.21.15.60:0/1659506585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c073e00 msgr2=0x7f9d7c168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.975+0000 7f9d80c00640 1 --2- 172.21.15.60:0/1659506585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c073e00 0x7f9d7c168e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0b4890 tx=0x7f9d7402aa90 comp rx=0 tx=0).stop 2024-09-16T15:40:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.975+0000 7f9d80c00640 1 -- 172.21.15.60:0/1659506585 shutdown_connections 2024-09-16T15:40:03.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.975+0000 7f9d80c00640 1 --2- 172.21.15.60:0/1659506585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c073e00 0x7f9d7c168e10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.975+0000 7f9d80c00640 1 -- 172.21.15.60:0/1659506585 >> 172.21.15.60:0/1659506585 conn(0x7f9d7c0b1950 msgr2=0x7f9d7c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.975+0000 7f9d80c00640 1 -- 172.21.15.60:0/1659506585 shutdown_connections 2024-09-16T15:40:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.975+0000 7f9d80c00640 1 -- 172.21.15.60:0/1659506585 wait complete. 2024-09-16T15:40:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.976+0000 7f9d80c00640 1 Processor -- start 2024-09-16T15:40:03.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.976+0000 7f9d80c00640 1 -- start start 2024-09-16T15:40:03.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.977+0000 7f9d80c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d7c073e00 0x7f9d7c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.977+0000 7f9d80c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7c0af3d0 0x7f9d7c1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.977+0000 7f9d80c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c161550 0x7f9d7c148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.977+0000 7f9d80c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1694d0 con 0x7f9d7c161550 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.977+0000 7f9d80c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c169350 con 0x7f9d7c073e00 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.977+0000 7f9d80c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7c169650 con 0x7f9d7c0af3d0 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c161550 0x7f9d7c148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c161550 0x7f9d7c148230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50504/0 (socket says 172.21.15.60:50504) 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 -- 172.21.15.60:0/923591509 learned_addr learned my addr 172.21.15.60:0/923591509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d71400640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7c0af3d0 0x7f9d7c1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d79e00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d7c073e00 0x7f9d7c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 -- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7c0af3d0 msgr2=0x7f9d7c1525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7c0af3d0 0x7f9d7c1525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 -- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d7c073e00 msgr2=0x7f9d7c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d7c073e00 0x7f9d7c157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 -- 172.21.15.60:0/923591509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c156090 con 0x7f9d7c161550 2024-09-16T15:40:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.978+0000 7f9d7a800640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c161550 0x7f9d7c148230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009d60 tx=0x7f9d74003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.979+0000 7f9d72a00640 1 -- 172.21.15.60:0/923591509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74046070 con 0x7f9d7c161550 2024-09-16T15:40:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.979+0000 7f9d72a00640 1 -- 172.21.15.60:0/923591509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d74004250 con 0x7f9d7c161550 2024-09-16T15:40:03.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.979+0000 7f9d72a00640 1 -- 172.21.15.60:0/923591509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74004680 con 0x7f9d7c161550 2024-09-16T15:40:03.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.979+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c156370 con 0x7f9d7c161550 2024-09-16T15:40:03.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.979+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c153d50 con 0x7f9d7c161550 2024-09-16T15:40:03.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.980+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7c154120 con 0x7f9d7c161550 2024-09-16T15:40:03.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.981+0000 7f9d72a00640 1 -- 172.21.15.60:0/923591509 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f9d74049070 con 0x7f9d7c161550 2024-09-16T15:40:03.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.982+0000 7f9d72a00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a00 0x7f9d5807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.982+0000 7f9d72a00640 1 -- 172.21.15.60:0/923591509 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f9d740bb9a0 con 0x7f9d7c161550 2024-09-16T15:40:03.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.983+0000 7f9d79e00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a00 0x7f9d5807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.983+0000 7f9d79e00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a00 0x7f9d5807aec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c000f60 tx=0x7f9d6c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:03.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:03.986+0000 7f9d72a00640 1 -- 172.21.15.60:0/923591509 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d74032030 con 0x7f9d7c161550 2024-09-16T15:40:04.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.181+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d7c06f710 con 0x7f9d7c161550 2024-09-16T15:40:04.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.182+0000 7f9d72a00640 1 -- 172.21.15.60:0/923591509 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13030 (secure 0 0 0) 0x7f9d74004820 con 0x7f9d7c161550 2024-09-16T15:40:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:04.184 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:40:02.785524+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/846626570":"2024-09-17T15:39:59.779200+0000","172.21.15.60:6834/846626570":"2024-09-17T15:39:59.779200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a00 msgr2=0x7f9d5807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a00 0x7f9d5807aec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c000f60 tx=0x7f9d6c009040 comp rx=0 tx=0).stop 2024-09-16T15:40:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c161550 msgr2=0x7f9d7c148230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c161550 0x7f9d7c148230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009d60 tx=0x7f9d74003f10 comp rx=0 tx=0).stop 2024-09-16T15:40:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 shutdown_connections 2024-09-16T15:40:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a00 0x7f9d5807aec0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7c161550 0x7f9d7c148230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7c0af3d0 0x7f9d7c1525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 --2- 172.21.15.60:0/923591509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d7c073e00 0x7f9d7c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 >> 172.21.15.60:0/923591509 conn(0x7f9d7c0b1950 msgr2=0x7f9d7c150e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 shutdown_connections 2024-09-16T15:40:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.185+0000 7f9d80c00640 1 -- 172.21.15.60:0/923591509 wait complete. 2024-09-16T15:40:04.202 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:40:04.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:40:04.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.364+0000 7fc7cd000640 1 -- 172.21.15.60:0/729768847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c80af190 msgr2=0x7fc7c80b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.364+0000 7fc7cd000640 1 --2- 172.21.15.60:0/729768847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c80af190 0x7fc7c80b2b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80cf760 tx=0x7fc7b402a540 comp rx=0 tx=0).stop 2024-09-16T15:40:04.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.365+0000 7fc7cd000640 1 -- 172.21.15.60:0/729768847 shutdown_connections 2024-09-16T15:40:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.365+0000 7fc7cd000640 1 --2- 172.21.15.60:0/729768847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7c80cec60 0x7fc7c80cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.365+0000 7fc7cd000640 1 --2- 172.21.15.60:0/729768847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c80af190 0x7fc7c80b2b40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.365+0000 7fc7cd000640 1 --2- 172.21.15.60:0/729768847 >> v2:172.21.15.100:3300/0 conn(0x7fc7c8161760 0x7fc7c80ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.365+0000 7fc7cd000640 1 -- 172.21.15.60:0/729768847 >> 172.21.15.60:0/729768847 conn(0x7fc7c80b1840 msgr2=0x7fc7c80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.365+0000 7fc7cd000640 1 -- 172.21.15.60:0/729768847 shutdown_connections 2024-09-16T15:40:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.365+0000 7fc7cd000640 1 -- 172.21.15.60:0/729768847 wait complete. 2024-09-16T15:40:04.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.366+0000 7fc7cd000640 1 Processor -- start 2024-09-16T15:40:04.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.366+0000 7fc7cd000640 1 -- start start 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.367+0000 7fc7cd000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7c80af190 0x7fc7c81428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.367+0000 7fc7cd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7c80cec60 0x7fc7c81f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.367+0000 7fc7cd000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c8161760 0x7fc7c8141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.367+0000 7fc7cd000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c80b3200 con 0x7fc7c8161760 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.367+0000 7fc7cd000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c80b3080 con 0x7fc7c80cec60 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.367+0000 7fc7cd000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c8162190 con 0x7fc7c80af190 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c8161760 0x7fc7c8141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c8161760 0x7fc7c8141d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50512/0 (socket says 172.21.15.60:50512) 2024-09-16T15:40:04.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 -- 172.21.15.60:0/2548759001 learned_addr learned my addr 172.21.15.60:0/2548759001 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c5e00640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7c80af190 0x7fc7c81428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c5400640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7c80cec60 0x7fc7c81f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 -- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7c80af190 msgr2=0x7fc7c81428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7c80af190 0x7fc7c81428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 -- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7c80cec60 msgr2=0x7fc7c81f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7c80cec60 0x7fc7c81f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 -- 172.21.15.60:0/2548759001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c81f91c0 con 0x7fc7c8161760 2024-09-16T15:40:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7c6800640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c8161760 0x7fc7c8141d30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000e980 tx=0x7fc7c000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.368+0000 7fc7be000640 1 -- 172.21.15.60:0/2548759001 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0004d20 con 0x7fc7c8161760 2024-09-16T15:40:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.369+0000 7fc7be000640 1 -- 172.21.15.60:0/2548759001 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7c000cb20 con 0x7fc7c8161760 2024-09-16T15:40:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.369+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c8155650 con 0x7fc7c8161760 2024-09-16T15:40:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.369+0000 7fc7be000640 1 -- 172.21.15.60:0/2548759001 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0005020 con 0x7fc7c8161760 2024-09-16T15:40:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.369+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c81458c0 con 0x7fc7c8161760 2024-09-16T15:40:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.370+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c80cf070 con 0x7fc7c8161760 2024-09-16T15:40:04.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.371+0000 7fc7be000640 1 -- 172.21.15.60:0/2548759001 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fc7c00051c0 con 0x7fc7c8161760 2024-09-16T15:40:04.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.372+0000 7fc7be000640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc7a8078a00 0x7fc7a807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.372+0000 7fc7be000640 1 -- 172.21.15.60:0/2548759001 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fc7c009b280 con 0x7fc7c8161760 2024-09-16T15:40:04.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.372+0000 7fc7c5e00640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc7a8078a00 0x7fc7a807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.373+0000 7fc7c5e00640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc7a8078a00 0x7fc7a807aec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800bb50 tx=0x7fc7b800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:04.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.376+0000 7fc7be000640 1 -- 172.21.15.60:0/2548759001 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc7c81f91c0 con 0x7fc7c8161760 2024-09-16T15:40:04.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.569+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7c806ac60 con 0x7fc7c8161760 2024-09-16T15:40:04.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.570+0000 7fc7be000640 1 -- 172.21.15.60:0/2548759001 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7fc7c0016030 con 0x7fc7c8161760 2024-09-16T15:40:04.571 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc7a8078a00 msgr2=0x7fc7a807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc7a8078a00 0x7fc7a807aec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800bb50 tx=0x7fc7b800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c8161760 msgr2=0x7fc7c8141d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c8161760 0x7fc7c8141d30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000e980 tx=0x7fc7c000ee40 comp rx=0 tx=0).stop 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 shutdown_connections 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc7a8078a00 0x7fc7a807aec0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7c8161760 0x7fc7c8141d30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7c80cec60 0x7fc7c81f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 --2- 172.21.15.60:0/2548759001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7c80af190 0x7fc7c81428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 >> 172.21.15.60:0/2548759001 conn(0x7fc7c80b1840 msgr2=0x7fc7c8072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.572+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 shutdown_connections 2024-09-16T15:40:04.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.573+0000 7fc7cd000640 1 -- 172.21.15.60:0/2548759001 wait complete. 2024-09-16T15:40:04.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-16T15:40:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.750+0000 7fc7c3200640 1 -- 172.21.15.60:0/3352751438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7bc0d1b70 msgr2=0x7fc7bc0d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.750+0000 7fc7c3200640 1 --2- 172.21.15.60:0/3352751438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7bc0d1b70 0x7fc7bc0d2730 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009860 tx=0x7fc7a802a5a0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.750+0000 7fc7c3200640 1 -- 172.21.15.60:0/3352751438 shutdown_connections 2024-09-16T15:40:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.750+0000 7fc7c3200640 1 --2- 172.21.15.60:0/3352751438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7bc0d1b70 0x7fc7bc0d2730 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.750+0000 7fc7c3200640 1 -- 172.21.15.60:0/3352751438 >> 172.21.15.60:0/3352751438 conn(0x7fc7bc0d3fa0 msgr2=0x7fc7bc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.750+0000 7fc7c3200640 1 -- 172.21.15.60:0/3352751438 shutdown_connections 2024-09-16T15:40:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.750+0000 7fc7c3200640 1 -- 172.21.15.60:0/3352751438 wait complete. 2024-09-16T15:40:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.752+0000 7fc7c3200640 1 Processor -- start 2024-09-16T15:40:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.752+0000 7fc7c3200640 1 -- start start 2024-09-16T15:40:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7c3200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7bc0b7720 0x7fc7bc15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7c3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7bc0d1b70 0x7fc7bc0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7c3200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7bc15e1b0 0x7fc7bc1f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7c3200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc0b83e0 con 0x7fc7bc0b7720 2024-09-16T15:40:04.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7c3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc0b8260 con 0x7fc7bc0d1b70 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7c3200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc7bc0b8560 con 0x7fc7bc15e1b0 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7bbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7bc0b7720 0x7fc7bc15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7bbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7bc0b7720 0x7fc7bc15c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50518/0 (socket says 172.21.15.60:50518) 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7bbe00640 1 -- 172.21.15.60:0/2610848636 learned_addr learned my addr 172.21.15.60:0/2610848636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7c1200640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7bc15e1b0 0x7fc7bc1f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.753+0000 7fc7bb400640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7bc0d1b70 0x7fc7bc0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7bbe00640 1 -- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7bc15e1b0 msgr2=0x7fc7bc1f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7bbe00640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7bc15e1b0 0x7fc7bc1f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7bbe00640 1 -- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7bc0d1b70 msgr2=0x7fc7bc0c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7bbe00640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7bc0d1b70 0x7fc7bc0c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7bbe00640 1 -- 172.21.15.60:0/2610848636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc0c8ec0 con 0x7fc7bc0b7720 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7bbe00640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7bc0b7720 0x7fc7bc15c410 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c410 tx=0x7fc7ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:04.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7b3e00640 1 -- 172.21.15.60:0/2610848636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac017070 con 0x7fc7bc0b7720 2024-09-16T15:40:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7b3e00640 1 -- 172.21.15.60:0/2610848636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7ac0035f0 con 0x7fc7bc0b7720 2024-09-16T15:40:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7b3e00640 1 -- 172.21.15.60:0/2610848636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac003a40 con 0x7fc7bc0b7720 2024-09-16T15:40:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc1f8930 con 0x7fc7bc0b7720 2024-09-16T15:40:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.754+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc0a0980 con 0x7fc7bc0b7720 2024-09-16T15:40:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.755+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc0b7b50 con 0x7fc7bc0b7720 2024-09-16T15:40:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.756+0000 7fc7b3e00640 1 -- 172.21.15.60:0/2610848636 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fc7ac01a070 con 0x7fc7bc0b7720 2024-09-16T15:40:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.757+0000 7fc7b3e00640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc79c078a00 0x7fc79c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.758+0000 7fc7b3e00640 1 -- 172.21.15.60:0/2610848636 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fc7ac099c40 con 0x7fc7bc0b7720 2024-09-16T15:40:04.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.758+0000 7fc7bb400640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc79c078a00 0x7fc79c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.759+0000 7fc7bb400640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc79c078a00 0x7fc79c07aec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8034000 tx=0x7fc7a8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:04.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:04.762+0000 7fc7b3e00640 1 -- 172.21.15.60:0/2610848636 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc7bc1f8930 con 0x7fc7bc0b7720 2024-09-16T15:40:05.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.030+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7fc7bc0cedd0 con 0x7fc7bc0b7720 2024-09-16T15:40:05.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.803+0000 7fc7b3e00640 1 -- 172.21.15.60:0/2610848636 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v107) ==== 143+0+0 (secure 0 0 0) 0x7fc7ac061c70 con 0x7fc7bc0b7720 2024-09-16T15:40:05.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc79c078a00 msgr2=0x7fc79c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc79c078a00 0x7fc79c07aec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8034000 tx=0x7fc7a8004730 comp rx=0 tx=0).stop 2024-09-16T15:40:05.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7bc0b7720 msgr2=0x7fc7bc15c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7bc0b7720 0x7fc7bc15c410 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c410 tx=0x7fc7ac00c8d0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 shutdown_connections 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc79c078a00 0x7fc79c07aec0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7bc15e1b0 0x7fc7bc1f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7bc0d1b70 0x7fc7bc0c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 --2- 172.21.15.60:0/2610848636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7bc0b7720 0x7fc7bc15c410 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 >> 172.21.15.60:0/2610848636 conn(0x7fc7bc0d3fa0 msgr2=0x7fc7bc06ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 shutdown_connections 2024-09-16T15:40:05.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.806+0000 7fc7c3200640 1 -- 172.21.15.60:0/2610848636 wait complete. 2024-09-16T15:40:05.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:40:05.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.990+0000 7f4ac3a00640 1 -- 172.21.15.60:0/1478298770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc15f5a0 msgr2=0x7f4abc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.990+0000 7f4ac3a00640 1 --2- 172.21.15.60:0/1478298770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc15f5a0 0x7f4abc0b2440 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0005fd0 tx=0x7f4ab002a540 comp rx=0 tx=0).stop 2024-09-16T15:40:05.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.990+0000 7f4ac3a00640 1 -- 172.21.15.60:0/1478298770 shutdown_connections 2024-09-16T15:40:05.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.990+0000 7f4ac3a00640 1 --2- 172.21.15.60:0/1478298770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc15f5a0 0x7f4abc0b2440 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.990+0000 7f4ac3a00640 1 -- 172.21.15.60:0/1478298770 >> 172.21.15.60:0/1478298770 conn(0x7f4abc0b1950 msgr2=0x7f4abc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:05.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.990+0000 7f4ac3a00640 1 -- 172.21.15.60:0/1478298770 shutdown_connections 2024-09-16T15:40:05.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.990+0000 7f4ac3a00640 1 -- 172.21.15.60:0/1478298770 wait complete. 2024-09-16T15:40:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.992+0000 7f4ac3a00640 1 Processor -- start 2024-09-16T15:40:05.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.992+0000 7f4ac3a00640 1 -- start start 2024-09-16T15:40:05.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac3a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4abc073fb0 0x7f4abc1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac3a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc0ccf00 0x7f4abc0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac3a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4abc15f5a0 0x7f4abc0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac3a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc06f9f0 con 0x7f4abc0ccf00 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac3a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc06f870 con 0x7f4abc073fb0 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac3a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4abc074420 con 0x7f4abc15f5a0 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc0ccf00 0x7f4abc0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc0ccf00 0x7f4abc0b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50536/0 (socket says 172.21.15.60:50536) 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4abbe00640 1 -- 172.21.15.60:0/4069826571 learned_addr learned my addr 172.21.15.60:0/4069826571 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac1a00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4abc15f5a0 0x7f4abc0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac1000640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4abc073fb0 0x7f4abc1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4abbe00640 1 -- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4abc15f5a0 msgr2=0x7f4abc0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4abbe00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4abc15f5a0 0x7f4abc0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4abbe00640 1 -- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4abc073fb0 msgr2=0x7f4abc1532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4abbe00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4abc073fb0 0x7f4abc1532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4abbe00640 1 -- 172.21.15.60:0/4069826571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc0cfad0 con 0x7f4abc0ccf00 2024-09-16T15:40:05.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.993+0000 7f4ac1a00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4abc15f5a0 0x7f4abc0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:05.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.994+0000 7f4abbe00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc0ccf00 0x7f4abc0b42d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800c440 tx=0x7f4aa800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:05.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.994+0000 7f4ab8c00640 1 -- 172.21.15.60:0/4069826571 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8017070 con 0x7f4abc0ccf00 2024-09-16T15:40:05.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.994+0000 7f4ab8c00640 1 -- 172.21.15.60:0/4069826571 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4aa80035f0 con 0x7f4abc0ccf00 2024-09-16T15:40:05.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.994+0000 7f4ab8c00640 1 -- 172.21.15.60:0/4069826571 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8003a40 con 0x7f4abc0ccf00 2024-09-16T15:40:05.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.994+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4abc153820 con 0x7f4abc0ccf00 2024-09-16T15:40:05.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.994+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4abc0cde60 con 0x7f4abc0ccf00 2024-09-16T15:40:05.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.995+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4abc09dc50 con 0x7f4abc0ccf00 2024-09-16T15:40:05.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.996+0000 7f4ab8c00640 1 -- 172.21.15.60:0/4069826571 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f4aa8003be0 con 0x7f4abc0ccf00 2024-09-16T15:40:05.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.997+0000 7f4ab8c00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a9c078a00 0x7f4a9c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:05.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.997+0000 7f4ab8c00640 1 -- 172.21.15.60:0/4069826571 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f4aa8099e50 con 0x7f4abc0ccf00 2024-09-16T15:40:05.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.998+0000 7f4ac1000640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a9c078a00 0x7f4a9c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:05.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:05.998+0000 7f4ac1000640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a9c078a00 0x7f4a9c07aec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0033000 tx=0x7f4ab0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.001+0000 7f4ab8c00640 1 -- 172.21.15.60:0/4069826571 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4aa8062390 con 0x7f4abc0ccf00 2024-09-16T15:40:06.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.196+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4abc075d60 con 0x7f4abc0ccf00 2024-09-16T15:40:06.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.197+0000 7f4ab8c00640 1 -- 172.21.15.60:0/4069826571 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v29) ==== 59+0+1998 (secure 0 0 0) 0x7f4aa80010c0 con 0x7f4abc0ccf00 2024-09-16T15:40:06.198 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:40:06.198 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:40:06.198 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:40:06.198 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:40:06.199 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:40:06.200 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:40:06.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a9c078a00 msgr2=0x7f4a9c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a9c078a00 0x7f4a9c07aec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0033000 tx=0x7f4ab0004730 comp rx=0 tx=0).stop 2024-09-16T15:40:06.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc0ccf00 msgr2=0x7f4abc0b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc0ccf00 0x7f4abc0b42d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800c440 tx=0x7f4aa800c900 comp rx=0 tx=0).stop 2024-09-16T15:40:06.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 shutdown_connections 2024-09-16T15:40:06.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4a9c078a00 0x7f4a9c07aec0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4abc15f5a0 0x7f4abc0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4abc0ccf00 0x7f4abc0b42d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 --2- 172.21.15.60:0/4069826571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4abc073fb0 0x7f4abc1532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 >> 172.21.15.60:0/4069826571 conn(0x7f4abc0b1950 msgr2=0x7f4abc154ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:06.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 shutdown_connections 2024-09-16T15:40:06.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.199+0000 7f4ac3a00640 1 -- 172.21.15.60:0/4069826571 wait complete. 2024-09-16T15:40:06.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:40:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 -- 172.21.15.60:0/2400743574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2006efa0 msgr2=0x7f7f200cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 --2- 172.21.15.60:0/2400743574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2006efa0 0x7f7f200cc5c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f200d0cd0 tx=0x7f7f0c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 -- 172.21.15.60:0/2400743574 shutdown_connections 2024-09-16T15:40:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 --2- 172.21.15.60:0/2400743574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f200ccb00 0x7f7f2015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 --2- 172.21.15.60:0/2400743574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2006efa0 0x7f7f200cc5c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 --2- 172.21.15.60:0/2400743574 >> v2:172.21.15.100:3300/0 conn(0x7f7f20145750 0x7f7f20073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 -- 172.21.15.60:0/2400743574 >> 172.21.15.60:0/2400743574 conn(0x7f7f200b1840 msgr2=0x7f7f200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:06.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 -- 172.21.15.60:0/2400743574 shutdown_connections 2024-09-16T15:40:06.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.378+0000 7f7f28000640 1 -- 172.21.15.60:0/2400743574 wait complete. 2024-09-16T15:40:06.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.380+0000 7f7f28000640 1 Processor -- start 2024-09-16T15:40:06.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.380+0000 7f7f28000640 1 -- start start 2024-09-16T15:40:06.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f28000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2006efa0 0x7f7f20141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f28000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f200ccb00 0x7f7f2013acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f28000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f20145750 0x7f7f2013d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f28000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7f201602c0 con 0x7f7f200ccb00 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f28000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20160140 con 0x7f7f20145750 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f28000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7f20160440 con 0x7f7f2006efa0 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f24c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f200ccb00 0x7f7f2013acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f24c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f200ccb00 0x7f7f2013acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50542/0 (socket says 172.21.15.60:50542) 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f24c00640 1 -- 172.21.15.60:0/2638645273 learned_addr learned my addr 172.21.15.60:0/2638645273 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.381+0000 7f7f25600640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2006efa0 0x7f7f20141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f26000640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f20145750 0x7f7f2013d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f24c00640 1 -- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2006efa0 msgr2=0x7f7f20141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f24c00640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2006efa0 0x7f7f20141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f24c00640 1 -- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f20145750 msgr2=0x7f7f2013d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f24c00640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f20145750 0x7f7f2013d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f24c00640 1 -- 172.21.15.60:0/2638645273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f201f89f0 con 0x7f7f200ccb00 2024-09-16T15:40:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f26000640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f20145750 0x7f7f2013d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f24c00640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f200ccb00 0x7f7f2013acc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009850 tx=0x7f7f0c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f1d600640 1 -- 172.21.15.60:0/2638645273 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c03f070 con 0x7f7f200ccb00 2024-09-16T15:40:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f1d600640 1 -- 172.21.15.60:0/2638645273 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f0c0047a0 con 0x7f7f200ccb00 2024-09-16T15:40:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f1d600640 1 -- 172.21.15.60:0/2638645273 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c035030 con 0x7f7f200ccb00 2024-09-16T15:40:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.382+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2013b200 con 0x7f7f200ccb00 2024-09-16T15:40:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.383+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f201f80c0 con 0x7f7f200ccb00 2024-09-16T15:40:06.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.384+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f2015f9f0 con 0x7f7f200ccb00 2024-09-16T15:40:06.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.385+0000 7f7f1d600640 1 -- 172.21.15.60:0/2638645273 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f7f0c042070 con 0x7f7f200ccb00 2024-09-16T15:40:06.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.386+0000 7f7f1d600640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7f00078a30 0x7f7f0007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.387+0000 7f7f1d600640 1 -- 172.21.15.60:0/2638645273 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f7f0c0bb8b0 con 0x7f7f200ccb00 2024-09-16T15:40:06.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.387+0000 7f7f25600640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7f00078a30 0x7f7f0007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.388+0000 7f7f25600640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7f00078a30 0x7f7f0007aef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400bb50 tx=0x7f7f1400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:06.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.391+0000 7f7f1d600640 1 -- 172.21.15.60:0/2638645273 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7f0c0837a0 con 0x7f7f200ccb00 2024-09-16T15:40:06.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.588+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7f200b3390 con 0x7f7f200ccb00 2024-09-16T15:40:06.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.592+0000 7f7f1d600640 1 -- 172.21.15.60:0/2638645273 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f7f0c0881d0 con 0x7f7f200ccb00 2024-09-16T15:40:06.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7f00078a30 msgr2=0x7f7f0007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7f00078a30 0x7f7f0007aef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400bb50 tx=0x7f7f1400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f200ccb00 msgr2=0x7f7f2013acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f200ccb00 0x7f7f2013acc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009850 tx=0x7f7f0c003910 comp rx=0 tx=0).stop 2024-09-16T15:40:06.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 shutdown_connections 2024-09-16T15:40:06.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7f00078a30 0x7f7f0007aef0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f20145750 0x7f7f2013d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f200ccb00 0x7f7f2013acc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 --2- 172.21.15.60:0/2638645273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2006efa0 0x7f7f20141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 >> 172.21.15.60:0/2638645273 conn(0x7f7f200b1840 msgr2=0x7f7f200cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:06.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 shutdown_connections 2024-09-16T15:40:06.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:06.595+0000 7f7f28000640 1 -- 172.21.15.60:0/2638645273 wait complete. 2024-09-16T15:40:06.612 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:40:06.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:40:06.697 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:40:06.786 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:40:06.786 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:40:06.786 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:06.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:40:06.822 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:40:06.822 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:06.858 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:40:06.858 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:40:06.859 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:06.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:40:06.897 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:40:06.897 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:06.933 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:40:06.934 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:40:06.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.095+0000 7f1c19a00640 1 -- 172.21.15.60:0/255280497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 msgr2=0x7f1c140d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.095+0000 7f1c19a00640 1 --2- 172.21.15.60:0/255280497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 0x7f1c140d1620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009830 tx=0x7f1c0c02a510 comp rx=0 tx=0).stop 2024-09-16T15:40:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.095+0000 7f1c19a00640 1 -- 172.21.15.60:0/255280497 shutdown_connections 2024-09-16T15:40:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.095+0000 7f1c19a00640 1 --2- 172.21.15.60:0/255280497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 0x7f1c140d1620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.096+0000 7f1c19a00640 1 -- 172.21.15.60:0/255280497 >> 172.21.15.60:0/255280497 conn(0x7f1c140d0b30 msgr2=0x7f1c140d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:07.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.096+0000 7f1c19a00640 1 -- 172.21.15.60:0/255280497 shutdown_connections 2024-09-16T15:40:07.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.096+0000 7f1c19a00640 1 -- 172.21.15.60:0/255280497 wait complete. 2024-09-16T15:40:07.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.097+0000 7f1c19a00640 1 Processor -- start 2024-09-16T15:40:07.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.097+0000 7f1c19a00640 1 -- start start 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c19a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 0x7f1c140c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c19a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1406f740 0x7f1c140c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c19a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c140ce7e0 0x7f1c141f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c19a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14073f70 con 0x7f1c1406d6f0 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c19a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14073df0 con 0x7f1c140ce7e0 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c19a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1c140740f0 con 0x7f1c1406f740 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c13e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 0x7f1c140c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c13e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 0x7f1c140c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50550/0 (socket says 172.21.15.60:50550) 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.098+0000 7f1c13e00640 1 -- 172.21.15.60:0/1304206993 learned_addr learned my addr 172.21.15.60:0/1304206993 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:07.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c19000640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c140ce7e0 0x7f1c141f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c13400640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1406f740 0x7f1c140c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c13e00640 1 -- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1406f740 msgr2=0x7f1c140c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c13e00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1406f740 0x7f1c140c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c13e00640 1 -- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c140ce7e0 msgr2=0x7f1c141f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c13e00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c140ce7e0 0x7f1c141f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c13e00640 1 -- 172.21.15.60:0/1304206993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c140c6620 con 0x7f1c1406d6f0 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c13e00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 0x7f1c140c6fb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400c7a0 tx=0x7f1c0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.099+0000 7f1c0be00640 1 -- 172.21.15.60:0/1304206993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04017070 con 0x7f1c1406d6f0 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.100+0000 7f1c0be00640 1 -- 172.21.15.60:0/1304206993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c04005070 con 0x7f1c1406d6f0 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.100+0000 7f1c0be00640 1 -- 172.21.15.60:0/1304206993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04005500 con 0x7f1c1406d6f0 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.100+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c140c6900 con 0x7f1c1406d6f0 2024-09-16T15:40:07.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.100+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c14157f60 con 0x7f1c1406d6f0 2024-09-16T15:40:07.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.101+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c140b0360 con 0x7f1c1406d6f0 2024-09-16T15:40:07.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.102+0000 7f1c0be00640 1 -- 172.21.15.60:0/1304206993 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f1c040056a0 con 0x7f1c1406d6f0 2024-09-16T15:40:07.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.103+0000 7f1c0be00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1bf4078a00 0x7f1bf407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.103+0000 7f1c0be00640 1 -- 172.21.15.60:0/1304206993 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f1c04099f70 con 0x7f1c1406d6f0 2024-09-16T15:40:07.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.104+0000 7f1c13400640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1bf4078a00 0x7f1bf407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.104+0000 7f1c13400640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1bf4078a00 0x7f1bf407aec0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1c000015c0 tx=0x7f1c00009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:07.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.107+0000 7f1c0be00640 1 -- 172.21.15.60:0/1304206993 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1c04062540 con 0x7f1c1406d6f0 2024-09-16T15:40:07.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.343+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c140d1620 con 0x7f1c1406d6f0 2024-09-16T15:40:07.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.343+0000 7f1c0be00640 1 -- 172.21.15.60:0/1304206993 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3908 (secure 0 0 0) 0x7f1c04061e40 con 0x7f1c1406d6f0 2024-09-16T15:40:07.345 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:07.345 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":20,"btime":"2024-09-16T15:40:00:781597+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4866,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2562305658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2562305658},{"type":"v1","addr":"172.21.15.60:6833","nonce":2562305658}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4869,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/664909622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":664909622},{"type":"v1","addr":"172.21.15.100:6836","nonce":664909622}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/801262237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":801262237},{"type":"v1","addr":"172.21.15.100:6837","nonce":801262237}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/186340465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":186340465},{"type":"v1","addr":"172.21.15.60:6835","nonce":186340465}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-16T15:40:07.345 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 20 2024-09-16T15:40:07.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1bf4078a00 msgr2=0x7f1bf407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1bf4078a00 0x7f1bf407aec0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1c000015c0 tx=0x7f1c00009040 comp rx=0 tx=0).stop 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 msgr2=0x7f1c140c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 0x7f1c140c6fb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400c7a0 tx=0x7f1c0400cc60 comp rx=0 tx=0).stop 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 shutdown_connections 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1bf4078a00 0x7f1bf407aec0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c140ce7e0 0x7f1c141f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c1406f740 0x7f1c140c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 --2- 172.21.15.60:0/1304206993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c1406d6f0 0x7f1c140c6fb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 >> 172.21.15.60:0/1304206993 conn(0x7f1c140d0b30 msgr2=0x7f1c140b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 shutdown_connections 2024-09-16T15:40:07.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.346+0000 7f1c19a00640 1 -- 172.21.15.60:0/1304206993 wait complete. 2024-09-16T15:40:07.363 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:40:07.363 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:40:07.364 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:40:07.364 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:40:07.364 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:40:07.364 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:40:07.364 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:40:07.364 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:40:07.364 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:40:07.365 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:40:07.365 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:40:07.365 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:40:07.365 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:40:07.365 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:40:07.365 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:40:07.365 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:40:07.379 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:40:07.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:40:07.380 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:40:07.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:40:07.381 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:40:07.381 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:40:07.382 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:40:07.382 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:40:07.423 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:40:07.423 DEBUG:teuthology.parallel:result is None 2024-09-16T15:40:07.425 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:40:07.425 DEBUG:teuthology.parallel:result is None 2024-09-16T15:40:07.426 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:40:07.426 DEBUG:teuthology.parallel:result is None 2024-09-16T15:40:07.427 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:40:07.427 DEBUG:teuthology.parallel:result is None 2024-09-16T15:40:07.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:40:07.554 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:40:07.555 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:40:07.556 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:40:07.556 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-09-16T15:40:07.555+0000 7f7649c00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-16T15:40:07.559 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:40:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.597+0000 7f6de2400640 1 -- 172.21.15.60:0/862045077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc1438b0 msgr2=0x7f6ddc06ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.597+0000 7f6de2400640 1 --2- 172.21.15.60:0/862045077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc1438b0 0x7f6ddc06ff60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8005fd0 tx=0x7f6dc802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.597+0000 7f6de2400640 1 -- 172.21.15.60:0/862045077 shutdown_connections 2024-09-16T15:40:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.597+0000 7f6de2400640 1 --2- 172.21.15.60:0/862045077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc1438b0 0x7f6ddc06ff60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.597+0000 7f6de2400640 1 --2- 172.21.15.60:0/862045077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ddc0b3f00 0x7f6ddc0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.597+0000 7f6de2400640 1 -- 172.21.15.60:0/862045077 >> 172.21.15.60:0/862045077 conn(0x7f6ddc0b1950 msgr2=0x7f6ddc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:07.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.597+0000 7f6de2400640 1 -- 172.21.15.60:0/862045077 shutdown_connections 2024-09-16T15:40:07.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.597+0000 7f6de2400640 1 -- 172.21.15.60:0/862045077 wait complete. 2024-09-16T15:40:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.599+0000 7f6de2400640 1 Processor -- start 2024-09-16T15:40:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.599+0000 7f6de2400640 1 -- start start 2024-09-16T15:40:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6de2400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc070920 0x7f6ddc09cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6de2400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ddc0b3f00 0x7f6ddc09b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6de2400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ddc1438b0 0x7f6ddc1f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6de2400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0b4da0 con 0x7f6ddc0b3f00 2024-09-16T15:40:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6de2400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0b4c20 con 0x7f6ddc070920 2024-09-16T15:40:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6de2400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc0b4f20 con 0x7f6ddc1438b0 2024-09-16T15:40:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6ddaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ddc0b3f00 0x7f6ddc09b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6ddb400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc070920 0x7f6ddc09cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6ddaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ddc0b3f00 0x7f6ddc09b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:50580/0 (socket says 172.21.15.60:50580) 2024-09-16T15:40:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6ddaa00640 1 -- 172.21.15.60:0/3435434790 learned_addr learned my addr 172.21.15.60:0/3435434790 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.600+0000 7f6ddbe00640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ddc1438b0 0x7f6ddc1f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6ddb400640 1 -- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ddc1438b0 msgr2=0x7f6ddc1f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6ddb400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ddc1438b0 0x7f6ddc1f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6ddb400640 1 -- 172.21.15.60:0/3435434790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ddc0b3f00 msgr2=0x7f6ddc09b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6ddb400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ddc0b3f00 0x7f6ddc09b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6ddb400640 1 -- 172.21.15.60:0/3435434790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc1f4e60 con 0x7f6ddc070920 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6ddaa00640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ddc0b3f00 0x7f6ddc09b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6ddb400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc070920 0x7f6ddc09cde0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c3c0 tx=0x7f6dcc00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6dd3400640 1 -- 172.21.15.60:0/3435434790 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc017070 con 0x7f6ddc070920 2024-09-16T15:40:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6dd3400640 1 -- 172.21.15.60:0/3435434790 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dcc0053c0 con 0x7f6ddc070920 2024-09-16T15:40:07.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6dd3400640 1 -- 172.21.15.60:0/3435434790 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc003850 con 0x7f6ddc070920 2024-09-16T15:40:07.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc0c9be0 con 0x7f6ddc070920 2024-09-16T15:40:07.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.601+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc09d750 con 0x7f6ddc070920 2024-09-16T15:40:07.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.602+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc0b4390 con 0x7f6ddc070920 2024-09-16T15:40:07.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.604+0000 7f6dd3400640 1 -- 172.21.15.60:0/3435434790 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f6dcc01a070 con 0x7f6ddc070920 2024-09-16T15:40:07.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.604+0000 7f6dd3400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6dbc078a30 0x7f6dbc07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.605+0000 7f6dd3400640 1 -- 172.21.15.60:0/3435434790 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f6dcc099be0 con 0x7f6ddc070920 2024-09-16T15:40:07.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.605+0000 7f6ddaa00640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6dbc078a30 0x7f6dbc07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.605+0000 7f6ddaa00640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6dbc078a30 0x7f6dbc07aef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009230 tx=0x7f6dc8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:07.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.609+0000 7f6dd3400640 1 -- 172.21.15.60:0/3435434790 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6ddc0cdae0 con 0x7f6ddc070920 2024-09-16T15:40:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:07.821+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6ddc0cd430 con 0x7f6ddc070920 2024-09-16T15:40:08.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.832+0000 7f6dd3400640 1 -- 172.21.15.60:0/3435434790 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) ==== 97+0+0 (secure 0 0 0) 0x7f6dcc061b80 con 0x7f6ddc070920 2024-09-16T15:40:08.834 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:40:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6dbc078a30 msgr2=0x7f6dbc07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6dbc078a30 0x7f6dbc07aef0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009230 tx=0x7f6dc8004730 comp rx=0 tx=0).stop 2024-09-16T15:40:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc070920 msgr2=0x7f6ddc09cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc070920 0x7f6ddc09cde0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c3c0 tx=0x7f6dcc00c880 comp rx=0 tx=0).stop 2024-09-16T15:40:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 shutdown_connections 2024-09-16T15:40:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6dbc078a30 0x7f6dbc07aef0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ddc1438b0 0x7f6ddc1f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ddc0b3f00 0x7f6ddc09b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 --2- 172.21.15.60:0/3435434790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ddc070920 0x7f6ddc09cde0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 >> 172.21.15.60:0/3435434790 conn(0x7f6ddc0b1950 msgr2=0x7f6ddc15e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:08.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 shutdown_connections 2024-09-16T15:40:08.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:08.836+0000 7f6de2400640 1 -- 172.21.15.60:0/3435434790 wait complete. 2024-09-16T15:40:08.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:40:09.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.020+0000 7f8aab000640 1 -- 172.21.15.60:0/3078063977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 msgr2=0x7f8aa40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.020+0000 7f8aab000640 1 --2- 172.21.15.60:0/3078063977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 0x7f8aa40b2440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8aa406dac0 tx=0x7f8a9402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.020+0000 7f8aab000640 1 -- 172.21.15.60:0/3078063977 shutdown_connections 2024-09-16T15:40:09.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.020+0000 7f8aab000640 1 --2- 172.21.15.60:0/3078063977 >> v2:172.21.15.100:3300/0 conn(0x7f8aa4073fb0 0x7f8aa4074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.020+0000 7f8aab000640 1 --2- 172.21.15.60:0/3078063977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 0x7f8aa40b2440 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.020+0000 7f8aab000640 1 -- 172.21.15.60:0/3078063977 >> 172.21.15.60:0/3078063977 conn(0x7f8aa40b1950 msgr2=0x7f8aa40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.020+0000 7f8aab000640 1 -- 172.21.15.60:0/3078063977 shutdown_connections 2024-09-16T15:40:09.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.020+0000 7f8aab000640 1 -- 172.21.15.60:0/3078063977 wait complete. 2024-09-16T15:40:09.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.021+0000 7f8aab000640 1 Processor -- start 2024-09-16T15:40:09.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.022+0000 7f8aab000640 1 -- start start 2024-09-16T15:40:09.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.022+0000 7f8aab000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa4073fb0 0x7f8aa41f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aab000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8aa40ccf00 0x7f8aa409d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aab000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 0x7f8aa40b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aab000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa406f2e0 con 0x7f8aa415f5a0 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aab000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa406f160 con 0x7f8aa4073fb0 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aab000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa406f460 con 0x7f8aa40ccf00 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 0x7f8aa40b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 0x7f8aa40b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37262/0 (socket says 172.21.15.60:37262) 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa9000640 1 -- 172.21.15.60:0/1742026676 learned_addr learned my addr 172.21.15.60:0/1742026676 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8a9be00640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8aa40ccf00 0x7f8aa409d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa3e00640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa4073fb0 0x7f8aa41f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa9000640 1 -- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8aa40ccf00 msgr2=0x7f8aa409d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa9000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8aa40ccf00 0x7f8aa409d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa9000640 1 -- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa4073fb0 msgr2=0x7f8aa41f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa9000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa4073fb0 0x7f8aa41f8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa9000640 1 -- 172.21.15.60:0/1742026676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa409d730 con 0x7f8aa415f5a0 2024-09-16T15:40:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.023+0000 7f8aa3e00640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa4073fb0 0x7f8aa41f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:09.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.024+0000 7f8aa9000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 0x7f8aa40b1fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b970 tx=0x7f8a9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.024+0000 7f8aa0c00640 1 -- 172.21.15.60:0/1742026676 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c019070 con 0x7f8aa415f5a0 2024-09-16T15:40:09.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.024+0000 7f8aa0c00640 1 -- 172.21.15.60:0/1742026676 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a9c00cc50 con 0x7f8aa415f5a0 2024-09-16T15:40:09.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.024+0000 7f8aa0c00640 1 -- 172.21.15.60:0/1742026676 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0146a0 con 0x7f8aa415f5a0 2024-09-16T15:40:09.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.024+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa41f4de0 con 0x7f8aa415f5a0 2024-09-16T15:40:09.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.025+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa41f53a0 con 0x7f8aa415f5a0 2024-09-16T15:40:09.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.026+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa406fa10 con 0x7f8aa415f5a0 2024-09-16T15:40:09.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.027+0000 7f8aa0c00640 1 -- 172.21.15.60:0/1742026676 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f8a9c01c070 con 0x7f8aa415f5a0 2024-09-16T15:40:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.028+0000 7f8aa0c00640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8a80078a50 0x7f8a8007af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.029+0000 7f8aa0c00640 1 -- 172.21.15.60:0/1742026676 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f8a9c09a600 con 0x7f8aa415f5a0 2024-09-16T15:40:09.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.029+0000 7f8aa3e00640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8a80078a50 0x7f8a8007af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.030+0000 7f8aa3e00640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8a80078a50 0x7f8a8007af10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a94033000 tx=0x7f8a94004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.033+0000 7f8aa0c00640 1 -- 172.21.15.60:0/1742026676 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8a9c062650 con 0x7f8aa415f5a0 2024-09-16T15:40:09.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.302+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8aa4154b70 con 0x7f8aa415f5a0 2024-09-16T15:40:09.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.303+0000 7f8aa0c00640 1 -- 172.21.15.60:0/1742026676 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f8a9c067080 con 0x7f8aa415f5a0 2024-09-16T15:40:09.304 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:09.304 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:40:09.304 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:40:09.305 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:40:09.306 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:40:09.307 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.309 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.310 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:40:09.312 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:40:09.314 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:40:09.315 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.316 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.317 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:40:09.317 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8a80078a50 msgr2=0x7f8a8007af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8a80078a50 0x7f8a8007af10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a94033000 tx=0x7f8a94004730 comp rx=0 tx=0).stop 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 msgr2=0x7f8aa40b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 0x7f8aa40b1fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b970 tx=0x7f8a9c00be30 comp rx=0 tx=0).stop 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 shutdown_connections 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8a80078a50 0x7f8a8007af10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8aa415f5a0 0x7f8aa40b1fa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8aa40ccf00 0x7f8aa409d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 --2- 172.21.15.60:0/1742026676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8aa4073fb0 0x7f8aa41f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 >> 172.21.15.60:0/1742026676 conn(0x7f8aa40b1950 msgr2=0x7f8aa40ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 shutdown_connections 2024-09-16T15:40:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.305+0000 7f8aab000640 1 -- 172.21.15.60:0/1742026676 wait complete. 2024-09-16T15:40:09.322 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:40:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.480+0000 7f49ee200640 1 -- 172.21.15.60:0/1534023701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e8073fb0 msgr2=0x7f49e8074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.480+0000 7f49ee200640 1 --2- 172.21.15.60:0/1534023701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e8073fb0 0x7f49e8074460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f49e0005fd0 tx=0x7f49e002a970 comp rx=0 tx=0).stop 2024-09-16T15:40:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.480+0000 7f49ee200640 1 -- 172.21.15.60:0/1534023701 shutdown_connections 2024-09-16T15:40:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.480+0000 7f49ee200640 1 --2- 172.21.15.60:0/1534023701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e8073fb0 0x7f49e8074460 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.480+0000 7f49ee200640 1 --2- 172.21.15.60:0/1534023701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e815f5a0 0x7f49e80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.480+0000 7f49ee200640 1 -- 172.21.15.60:0/1534023701 >> 172.21.15.60:0/1534023701 conn(0x7f49e80b1950 msgr2=0x7f49e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.480+0000 7f49ee200640 1 -- 172.21.15.60:0/1534023701 shutdown_connections 2024-09-16T15:40:09.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.480+0000 7f49ee200640 1 -- 172.21.15.60:0/1534023701 wait complete. 2024-09-16T15:40:09.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.482+0000 7f49ee200640 1 Processor -- start 2024-09-16T15:40:09.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.482+0000 7f49ee200640 1 -- start start 2024-09-16T15:40:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49ee200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e8073fb0 0x7f49e80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49ee200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e80ccf00 0x7f49e8148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49ee200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f5a0 0x7f49e81487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49ee200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f49e806f2e0 con 0x7f49e815f5a0 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49ee200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f49e806f160 con 0x7f49e80ccf00 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49ee200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f49e806f460 con 0x7f49e8073fb0 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49e7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f5a0 0x7f49e81487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49e7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f5a0 0x7f49e81487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37270/0 (socket says 172.21.15.60:37270) 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49e7e00640 1 -- 172.21.15.60:0/677080940 learned_addr learned my addr 172.21.15.60:0/677080940 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.483+0000 7f49e7400640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e8073fb0 0x7f49e80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49dea00640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e80ccf00 0x7f49e8148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49e7e00640 1 -- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e8073fb0 msgr2=0x7f49e80b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49e7e00640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e8073fb0 0x7f49e80b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49e7e00640 1 -- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e80ccf00 msgr2=0x7f49e8148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49e7e00640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e80ccf00 0x7f49e8148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49e7e00640 1 -- 172.21.15.60:0/677080940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e80b4500 con 0x7f49e815f5a0 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49e7400640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e8073fb0 0x7f49e80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49e7e00640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f5a0 0x7f49e81487b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f49e0006010 tx=0x7f49e0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49dfe00640 1 -- 172.21.15.60:0/677080940 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0034070 con 0x7f49e815f5a0 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49dfe00640 1 -- 172.21.15.60:0/677080940 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49e00045c0 con 0x7f49e815f5a0 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49dfe00640 1 -- 172.21.15.60:0/677080940 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0039450 con 0x7f49e815f5a0 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.484+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e80c4c60 con 0x7f49e815f5a0 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.485+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e80c5220 con 0x7f49e815f5a0 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.486+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e806f9f0 con 0x7f49e815f5a0 2024-09-16T15:40:09.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.487+0000 7f49dfe00640 1 -- 172.21.15.60:0/677080940 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f49e0037020 con 0x7f49e815f5a0 2024-09-16T15:40:09.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.488+0000 7f49dfe00640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f49c4078a00 0x7f49c407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.488+0000 7f49dfe00640 1 -- 172.21.15.60:0/677080940 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f49e00bbfe0 con 0x7f49e815f5a0 2024-09-16T15:40:09.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.489+0000 7f49e7400640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f49c4078a00 0x7f49c407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.489+0000 7f49e7400640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f49c4078a00 0x7f49c407aec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49d8005fd0 tx=0x7f49d800f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.492+0000 7f49dfe00640 1 -- 172.21.15.60:0/677080940 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f49e00845b0 con 0x7f49e815f5a0 2024-09-16T15:40:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.687+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f49e80ae710 con 0x7f49e815f5a0 2024-09-16T15:40:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.688+0000 7f49dfe00640 1 -- 172.21.15.60:0/677080940 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) ==== 84+0+65 (secure 0 0 0) 0x7f49e0083eb0 con 0x7f49e815f5a0 2024-09-16T15:40:09.689 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:09.689 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:40:09.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:09.689 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:40:09.690 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:40:09.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:09.690 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:40:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f49c4078a00 msgr2=0x7f49c407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f49c4078a00 0x7f49c407aec0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49d8005fd0 tx=0x7f49d800f360 comp rx=0 tx=0).stop 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f5a0 msgr2=0x7f49e81487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f5a0 0x7f49e81487b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f49e0006010 tx=0x7f49e0003f10 comp rx=0 tx=0).stop 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 shutdown_connections 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f49c4078a00 0x7f49c407aec0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49e815f5a0 0x7f49e81487b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49e80ccf00 0x7f49e8148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 --2- 172.21.15.60:0/677080940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49e8073fb0 0x7f49e80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 >> 172.21.15.60:0/677080940 conn(0x7f49e80b1950 msgr2=0x7f49e815e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 shutdown_connections 2024-09-16T15:40:09.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.690+0000 7f49ee200640 1 -- 172.21.15.60:0/677080940 wait complete. 2024-09-16T15:40:09.708 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:40:09.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:40:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.872+0000 7f7a8d200640 1 -- 172.21.15.60:0/1476054287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a88161750 msgr2=0x7f7a880b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.872+0000 7f7a8d200640 1 --2- 172.21.15.60:0/1476054287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a88161750 0x7f7a880b2cf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7a8809fdd0 tx=0x7f7a7802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.873+0000 7f7a8d200640 1 -- 172.21.15.60:0/1476054287 shutdown_connections 2024-09-16T15:40:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.873+0000 7f7a8d200640 1 --2- 172.21.15.60:0/1476054287 >> v2:172.21.15.100:3300/0 conn(0x7f7a880d0080 0x7f7a880d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.873+0000 7f7a8d200640 1 --2- 172.21.15.60:0/1476054287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a88161750 0x7f7a880b2cf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.873+0000 7f7a8d200640 1 -- 172.21.15.60:0/1476054287 >> 172.21.15.60:0/1476054287 conn(0x7f7a880d1e30 msgr2=0x7f7a880d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.873+0000 7f7a8d200640 1 -- 172.21.15.60:0/1476054287 shutdown_connections 2024-09-16T15:40:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.873+0000 7f7a8d200640 1 -- 172.21.15.60:0/1476054287 wait complete. 2024-09-16T15:40:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.874+0000 7f7a8d200640 1 Processor -- start 2024-09-16T15:40:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.874+0000 7f7a8d200640 1 -- start start 2024-09-16T15:40:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.875+0000 7f7a8d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd550 0x7f7a88151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.875+0000 7f7a8d200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a880d0080 0x7f7a8809df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.875+0000 7f7a8d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88161750 0x7f7a8809e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.875+0000 7f7a8d200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8806eca0 con 0x7f7a880cd550 2024-09-16T15:40:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.875+0000 7f7a8d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8806eb20 con 0x7f7a88161750 2024-09-16T15:40:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.875+0000 7f7a8d200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8806ee20 con 0x7f7a880d0080 2024-09-16T15:40:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd550 0x7f7a88151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd550 0x7f7a88151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37282/0 (socket says 172.21.15.60:37282) 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 -- 172.21.15.60:0/3424773428 learned_addr learned my addr 172.21.15.60:0/3424773428 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a86800640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88161750 0x7f7a8809e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85400640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a880d0080 0x7f7a8809df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 -- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a880d0080 msgr2=0x7f7a8809df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a880d0080 0x7f7a8809df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 -- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88161750 msgr2=0x7f7a8809e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88161750 0x7f7a8809e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 -- 172.21.15.60:0/3424773428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a880c9d60 con 0x7f7a880cd550 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85400640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a880d0080 0x7f7a8809df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.876+0000 7f7a85e00640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd550 0x7f7a88151190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009780 tx=0x7f7a78033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.877+0000 7f7a7e000640 1 -- 172.21.15.60:0/3424773428 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78031d60 con 0x7f7a880cd550 2024-09-16T15:40:09.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.877+0000 7f7a7e000640 1 -- 172.21.15.60:0/3424773428 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a78033e10 con 0x7f7a880cd550 2024-09-16T15:40:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.877+0000 7f7a7e000640 1 -- 172.21.15.60:0/3424773428 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78005260 con 0x7f7a880cd550 2024-09-16T15:40:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.877+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a880ca040 con 0x7f7a880cd550 2024-09-16T15:40:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.877+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a880ca600 con 0x7f7a880cd550 2024-09-16T15:40:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.878+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a880d04f0 con 0x7f7a880cd550 2024-09-16T15:40:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.879+0000 7f7a7e000640 1 -- 172.21.15.60:0/3424773428 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f7a78005400 con 0x7f7a880cd550 2024-09-16T15:40:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.880+0000 7f7a7e000640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a68078a00 0x7f7a6807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.880+0000 7f7a7e000640 1 -- 172.21.15.60:0/3424773428 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f7a780bba80 con 0x7f7a880cd550 2024-09-16T15:40:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.880+0000 7f7a85400640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a68078a00 0x7f7a6807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.881+0000 7f7a85400640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a68078a00 0x7f7a6807aec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a740058c0 tx=0x7f7a74005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:09.884+0000 7f7a7e000640 1 -- 172.21.15.60:0/3424773428 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7a780bde20 con 0x7f7a880cd550 2024-09-16T15:40:10.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.080+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7a880d22b0 con 0x7f7a880cd550 2024-09-16T15:40:10.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.834+0000 7f7a7e000640 1 -- 172.21.15.60:0/3424773428 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) ==== 138+0+0 (secure 0 0 0) 0x7f7a7803f030 con 0x7f7a880cd550 2024-09-16T15:40:10.836 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:40:10.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a68078a00 msgr2=0x7f7a6807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:10.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a68078a00 0x7f7a6807aec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a740058c0 tx=0x7f7a74005850 comp rx=0 tx=0).stop 2024-09-16T15:40:10.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd550 msgr2=0x7f7a88151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:10.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd550 0x7f7a88151190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009780 tx=0x7f7a78033c20 comp rx=0 tx=0).stop 2024-09-16T15:40:10.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 shutdown_connections 2024-09-16T15:40:10.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a68078a00 0x7f7a6807aec0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a88161750 0x7f7a8809e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a880d0080 0x7f7a8809df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 --2- 172.21.15.60:0/3424773428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a880cd550 0x7f7a88151190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 >> 172.21.15.60:0/3424773428 conn(0x7f7a880d1e30 msgr2=0x7f7a8815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:10.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.838+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 shutdown_connections 2024-09-16T15:40:10.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:10.839+0000 7f7a8d200640 1 -- 172.21.15.60:0/3424773428 wait complete. 2024-09-16T15:40:10.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:40:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.034+0000 7f6be3c00640 1 -- 172.21.15.60:0/2056968437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc1437c0 msgr2=0x7f6bdc0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.034+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2056968437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc1437c0 0x7f6bdc0cdad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc076510 tx=0x7f6bd002b1c0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.034+0000 7f6be3c00640 1 -- 172.21.15.60:0/2056968437 shutdown_connections 2024-09-16T15:40:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.034+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2056968437 >> v2:172.21.15.100:3300/0 conn(0x7f6bdc0b4400 0x7f6bdc15db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.034+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2056968437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bdc0b4bd0 0x7f6bdc0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.034+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2056968437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc1437c0 0x7f6bdc0cdad0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.034+0000 7f6be3c00640 1 -- 172.21.15.60:0/2056968437 >> 172.21.15.60:0/2056968437 conn(0x7f6bdc0b1840 msgr2=0x7f6bdc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.035+0000 7f6be3c00640 1 -- 172.21.15.60:0/2056968437 shutdown_connections 2024-09-16T15:40:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.035+0000 7f6be3c00640 1 -- 172.21.15.60:0/2056968437 wait complete. 2024-09-16T15:40:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.036+0000 7f6be3c00640 1 Processor -- start 2024-09-16T15:40:11.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.036+0000 7f6be3c00640 1 -- start start 2024-09-16T15:40:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.037+0000 7f6be3c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc0b4400 0x7f6bdc155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.037+0000 7f6be3c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bdc0b4bd0 0x7f6bdc1f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.037+0000 7f6be3c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1437c0 0x7f6bdc0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:11.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.037+0000 7f6be3c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc15e090 con 0x7f6bdc0b4400 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.037+0000 7f6be3c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc070a80 con 0x7f6bdc1437c0 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.037+0000 7f6be3c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6bdc15e210 con 0x7f6bdc0b4bd0 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.037+0000 7f6be1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc0b4400 0x7f6bdc155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc0b4400 0x7f6bdc155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37308/0 (socket says 172.21.15.60:37308) 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1200640 1 -- 172.21.15.60:0/2256504903 learned_addr learned my addr 172.21.15.60:0/2256504903 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1c00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1437c0 0x7f6bdc0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6bdbe00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bdc0b4bd0 0x7f6bdc1f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1200640 1 -- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bdc0b4bd0 msgr2=0x7f6bdc1f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1200640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bdc0b4bd0 0x7f6bdc1f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1200640 1 -- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1437c0 msgr2=0x7f6bdc0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1200640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1437c0 0x7f6bdc0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1200640 1 -- 172.21.15.60:0/2256504903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc135490 con 0x7f6bdc0b4400 2024-09-16T15:40:11.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.038+0000 7f6be1200640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc0b4400 0x7f6bdc155630 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00075a0 tx=0x7f6bd00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.039+0000 7f6bd8c00640 1 -- 172.21.15.60:0/2256504903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd002bb00 con 0x7f6bdc0b4400 2024-09-16T15:40:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.039+0000 7f6bd8c00640 1 -- 172.21.15.60:0/2256504903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bd002be50 con 0x7f6bdc0b4400 2024-09-16T15:40:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.039+0000 7f6bd8c00640 1 -- 172.21.15.60:0/2256504903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0005d50 con 0x7f6bdc0b4400 2024-09-16T15:40:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.039+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc135770 con 0x7f6bdc0b4400 2024-09-16T15:40:11.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.039+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc135d30 con 0x7f6bdc0b4400 2024-09-16T15:40:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.040+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bdc15e6c0 con 0x7f6bdc0b4400 2024-09-16T15:40:11.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.041+0000 7f6bd8c00640 1 -- 172.21.15.60:0/2256504903 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f6bd0007cb0 con 0x7f6bdc0b4400 2024-09-16T15:40:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.042+0000 7f6bd8c00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6bbc078a00 0x7f6bbc07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:11.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.042+0000 7f6bd8c00640 1 -- 172.21.15.60:0/2256504903 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5227+0+0 (secure 0 0 0) 0x7f6bd00ba410 con 0x7f6bdc0b4400 2024-09-16T15:40:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.043+0000 7f6bdbe00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6bbc078a00 0x7f6bbc07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.043+0000 7f6bdbe00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6bbc078a00 0x7f6bbc07aec0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc1f5db0 tx=0x7f6bc8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.047+0000 7f6bd8c00640 1 -- 172.21.15.60:0/2256504903 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6bd0082830 con 0x7f6bdc0b4400 2024-09-16T15:40:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.245+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6bdc155b70 con 0x7f6bdc0b4400 2024-09-16T15:40:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.843+0000 7f6bd8c00640 1 -- 172.21.15.60:0/2256504903 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) ==== 170+0+0 (secure 0 0 0) 0x7f6bd0082130 con 0x7f6bdc0b4400 2024-09-16T15:40:11.844 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:40:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.847+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6bbc078a00 msgr2=0x7f6bbc07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.847+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6bbc078a00 0x7f6bbc07aec0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc1f5db0 tx=0x7f6bc8005f90 comp rx=0 tx=0).stop 2024-09-16T15:40:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.847+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc0b4400 msgr2=0x7f6bdc155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.847+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc0b4400 0x7f6bdc155630 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00075a0 tx=0x7f6bd00038a0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.848+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 shutdown_connections 2024-09-16T15:40:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.848+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6bbc078a00 0x7f6bbc07aec0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.848+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6bdc1437c0 0x7f6bdc0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.848+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6bdc0b4bd0 0x7f6bdc1f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.848+0000 7f6be3c00640 1 --2- 172.21.15.60:0/2256504903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6bdc0b4400 0x7f6bdc155630 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.848+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 >> 172.21.15.60:0/2256504903 conn(0x7f6bdc0b1840 msgr2=0x7f6bdc06f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.848+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 shutdown_connections 2024-09-16T15:40:11.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:11.848+0000 7f6be3c00640 1 -- 172.21.15.60:0/2256504903 wait complete. 2024-09-16T15:40:11.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:40:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.048+0000 7f9727200640 1 -- 172.21.15.60:0/4015049041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 msgr2=0x7f972806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.048+0000 7f9727200640 1 --2- 172.21.15.60:0/4015049041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 0x7f972806fce0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97280b5600 tx=0x7f971802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.048+0000 7f9727200640 1 -- 172.21.15.60:0/4015049041 shutdown_connections 2024-09-16T15:40:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.048+0000 7f9727200640 1 --2- 172.21.15.60:0/4015049041 >> v2:172.21.15.100:3300/0 conn(0x7f97280cbc00 0x7f9728073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.048+0000 7f9727200640 1 --2- 172.21.15.60:0/4015049041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 0x7f972806fce0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.048+0000 7f9727200640 1 -- 172.21.15.60:0/4015049041 >> 172.21.15.60:0/4015049041 conn(0x7f97280cf9f0 msgr2=0x7f97280cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.048+0000 7f9727200640 1 -- 172.21.15.60:0/4015049041 shutdown_connections 2024-09-16T15:40:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.048+0000 7f9727200640 1 -- 172.21.15.60:0/4015049041 wait complete. 2024-09-16T15:40:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.049+0000 7f9727200640 1 Processor -- start 2024-09-16T15:40:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.049+0000 7f9727200640 1 -- start start 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.050+0000 7f9727200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9728072560 0x7f97281f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.050+0000 7f9727200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97280cbc00 0x7f97280a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.050+0000 7f9727200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 0x7f97280c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.050+0000 7f9727200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f97280ae930 con 0x7f972815bec0 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.050+0000 7f9727200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97280ae7b0 con 0x7f9728072560 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.050+0000 7f9727200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f97280aeab0 con 0x7f97280cbc00 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9726800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 0x7f97280c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9726800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 0x7f97280c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37312/0 (socket says 172.21.15.60:37312) 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9726800640 1 -- 172.21.15.60:0/1803518279 learned_addr learned my addr 172.21.15.60:0/1803518279 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9725e00640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9728072560 0x7f97281f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9725400640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97280cbc00 0x7f97280a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9726800640 1 -- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97280cbc00 msgr2=0x7f97280a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9726800640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97280cbc00 0x7f97280a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9726800640 1 -- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9728072560 msgr2=0x7f97281f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9726800640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9728072560 0x7f97281f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.051+0000 7f9726800640 1 -- 172.21.15.60:0/1803518279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97280c6d10 con 0x7f972815bec0 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.052+0000 7f9726800640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 0x7f97280c8d50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f972000ccb0 tx=0x7f972000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.052+0000 7f971e000640 1 -- 172.21.15.60:0/1803518279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972000f450 con 0x7f972815bec0 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.052+0000 7f971e000640 1 -- 172.21.15.60:0/1803518279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f972000f700 con 0x7f972815bec0 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.052+0000 7f971e000640 1 -- 172.21.15.60:0/1803518279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972000fb90 con 0x7f972815bec0 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.052+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97280c6ff0 con 0x7f972815bec0 2024-09-16T15:40:12.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.052+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97280c9290 con 0x7f972815bec0 2024-09-16T15:40:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.053+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97280c6aa0 con 0x7f972815bec0 2024-09-16T15:40:12.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.054+0000 7f971e000640 1 -- 172.21.15.60:0/1803518279 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f97200213f0 con 0x7f972815bec0 2024-09-16T15:40:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.055+0000 7f971e000640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9708078a00 0x7f970807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.055+0000 7f971e000640 1 -- 172.21.15.60:0/1803518279 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5609+0+0 (secure 0 0 0) 0x7f972009b750 con 0x7f972815bec0 2024-09-16T15:40:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.056+0000 7f9725e00640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9708078a00 0x7f970807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.056+0000 7f9725e00640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9708078a00 0x7f970807aec0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9718033000 tx=0x7f9718004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:12.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.060+0000 7f971e000640 1 -- 172.21.15.60:0/1803518279 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9720014030 con 0x7f972815bec0 2024-09-16T15:40:12.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.293+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f97280aef60 con 0x7f972815bec0 2024-09-16T15:40:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.852+0000 7f971e000640 1 -- 172.21.15.60:0/1803518279 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7f9720063430 con 0x7f972815bec0 2024-09-16T15:40:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 6 and data pool 7 2024-09-16T15:40:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.857+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9708078a00 msgr2=0x7f970807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.857+0000 7f9727200640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9708078a00 0x7f970807aec0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9718033000 tx=0x7f9718004730 comp rx=0 tx=0).stop 2024-09-16T15:40:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.857+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 msgr2=0x7f97280c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.857+0000 7f9727200640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 0x7f97280c8d50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f972000ccb0 tx=0x7f972000a620 comp rx=0 tx=0).stop 2024-09-16T15:40:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.858+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 shutdown_connections 2024-09-16T15:40:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.858+0000 7f9727200640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9708078a00 0x7f970807aec0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.858+0000 7f9727200640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f972815bec0 0x7f97280c8d50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.858+0000 7f9727200640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97280cbc00 0x7f97280a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.858+0000 7f9727200640 1 --2- 172.21.15.60:0/1803518279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9728072560 0x7f97281f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.858+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 >> 172.21.15.60:0/1803518279 conn(0x7f97280cf9f0 msgr2=0x7f97280b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.859+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 shutdown_connections 2024-09-16T15:40:12.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:12.859+0000 7f9727200640 1 -- 172.21.15.60:0/1803518279 wait complete. 2024-09-16T15:40:12.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.058+0000 7f9a3e000640 1 -- 172.21.15.60:0/1552926986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a38073fb0 msgr2=0x7f9a38074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.058+0000 7f9a3e000640 1 --2- 172.21.15.60:0/1552926986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a38073fb0 0x7f9a38074460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9a30005fd0 tx=0x7f9a3002aa00 comp rx=0 tx=0).stop 2024-09-16T15:40:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.058+0000 7f9a3e000640 1 -- 172.21.15.60:0/1552926986 shutdown_connections 2024-09-16T15:40:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.058+0000 7f9a3e000640 1 --2- 172.21.15.60:0/1552926986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a38073fb0 0x7f9a38074460 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.058+0000 7f9a3e000640 1 --2- 172.21.15.60:0/1552926986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a3815f5a0 0x7f9a380b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.058+0000 7f9a3e000640 1 -- 172.21.15.60:0/1552926986 >> 172.21.15.60:0/1552926986 conn(0x7f9a380b1950 msgr2=0x7f9a380b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.058+0000 7f9a3e000640 1 -- 172.21.15.60:0/1552926986 shutdown_connections 2024-09-16T15:40:13.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.058+0000 7f9a3e000640 1 -- 172.21.15.60:0/1552926986 wait complete. 2024-09-16T15:40:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.060+0000 7f9a3e000640 1 Processor -- start 2024-09-16T15:40:13.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.060+0000 7f9a3e000640 1 -- start start 2024-09-16T15:40:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a3e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a38073fb0 0x7f9a3814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a3e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a380ccf00 0x7f9a381489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a3e000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a3815f5a0 0x7f9a3815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a3e000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3806f2e0 con 0x7f9a380ccf00 2024-09-16T15:40:13.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a3e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3806f160 con 0x7f9a38073fb0 2024-09-16T15:40:13.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a3e000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3806f460 con 0x7f9a3815f5a0 2024-09-16T15:40:13.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a36a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a380ccf00 0x7f9a381489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a36a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a380ccf00 0x7f9a381489a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37328/0 (socket says 172.21.15.60:37328) 2024-09-16T15:40:13.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.061+0000 7f9a36a00640 1 -- 172.21.15.60:0/98437722 learned_addr learned my addr 172.21.15.60:0/98437722 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a37e00640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a3815f5a0 0x7f9a3815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a37400640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a38073fb0 0x7f9a3814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a36a00640 1 -- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a3815f5a0 msgr2=0x7f9a3815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a36a00640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a3815f5a0 0x7f9a3815da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a36a00640 1 -- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a38073fb0 msgr2=0x7f9a3814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a36a00640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a38073fb0 0x7f9a3814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a36a00640 1 -- 172.21.15.60:0/98437722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3814b880 con 0x7f9a380ccf00 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a37e00640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a3815f5a0 0x7f9a3815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a37400640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a38073fb0 0x7f9a3814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a36a00640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a380ccf00 0x7f9a381489a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400c420 tx=0x7f9a2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a2f400640 1 -- 172.21.15.60:0/98437722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24017070 con 0x7f9a380ccf00 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a2f400640 1 -- 172.21.15.60:0/98437722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a240053c0 con 0x7f9a380ccf00 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a2f400640 1 -- 172.21.15.60:0/98437722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24003850 con 0x7f9a380ccf00 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.062+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a3814bb60 con 0x7f9a380ccf00 2024-09-16T15:40:13.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.063+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a38149290 con 0x7f9a380ccf00 2024-09-16T15:40:13.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.064+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a3806f9f0 con 0x7f9a380ccf00 2024-09-16T15:40:13.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.065+0000 7f9a2f400640 1 -- 172.21.15.60:0/98437722 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f9a2401a070 con 0x7f9a380ccf00 2024-09-16T15:40:13.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.066+0000 7f9a2f400640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9a18078a00 0x7f9a1807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.066+0000 7f9a2f400640 1 -- 172.21.15.60:0/98437722 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f9a2409a100 con 0x7f9a380ccf00 2024-09-16T15:40:13.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.066+0000 7f9a37400640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9a18078a00 0x7f9a1807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.067+0000 7f9a37400640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9a18078a00 0x7f9a1807aec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009aa0 tx=0x7f9a28005db0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.070+0000 7f9a2f400640 1 -- 172.21.15.60:0/98437722 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9a240623d0 con 0x7f9a380ccf00 2024-09-16T15:40:13.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.263+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a38149460 con 0x7f9a380ccf00 2024-09-16T15:40:13.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.264+0000 7f9a2f400640 1 -- 172.21.15.60:0/98437722 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16686 (secure 0 0 0) 0x7f9a24061cd0 con 0x7f9a380ccf00 2024-09-16T15:40:13.266 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:13.266 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:40:12.294673+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.089921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:11.255279+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9a18078a00 msgr2=0x7f9a1807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9a18078a00 0x7f9a1807aec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009aa0 tx=0x7f9a28005db0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a380ccf00 msgr2=0x7f9a381489a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a380ccf00 0x7f9a381489a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400c420 tx=0x7f9a2400c8e0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 shutdown_connections 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9a18078a00 0x7f9a1807aec0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a3815f5a0 0x7f9a3815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a380ccf00 0x7f9a381489a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 --2- 172.21.15.60:0/98437722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a38073fb0 0x7f9a3814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 >> 172.21.15.60:0/98437722 conn(0x7f9a380b1950 msgr2=0x7f9a380acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 shutdown_connections 2024-09-16T15:40:13.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.267+0000 7f9a3e000640 1 -- 172.21.15.60:0/98437722 wait complete. 2024-09-16T15:40:13.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.444+0000 7efc52e00640 1 -- 172.21.15.60:0/2305810797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c073fb0 msgr2=0x7efc4c074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.444+0000 7efc52e00640 1 --2- 172.21.15.60:0/2305810797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c073fb0 0x7efc4c074420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0cfbb0 tx=0x7efc4402a9a0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.444+0000 7efc52e00640 1 -- 172.21.15.60:0/2305810797 shutdown_connections 2024-09-16T15:40:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.444+0000 7efc52e00640 1 --2- 172.21.15.60:0/2305810797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c073fb0 0x7efc4c074420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.444+0000 7efc52e00640 1 --2- 172.21.15.60:0/2305810797 >> v2:172.21.15.100:3300/0 conn(0x7efc4c0ccf00 0x7efc4c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.444+0000 7efc52e00640 1 -- 172.21.15.60:0/2305810797 >> 172.21.15.60:0/2305810797 conn(0x7efc4c0b1950 msgr2=0x7efc4c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.444+0000 7efc52e00640 1 -- 172.21.15.60:0/2305810797 shutdown_connections 2024-09-16T15:40:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.444+0000 7efc52e00640 1 -- 172.21.15.60:0/2305810797 wait complete. 2024-09-16T15:40:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.445+0000 7efc52e00640 1 Processor -- start 2024-09-16T15:40:13.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.446+0000 7efc52e00640 1 -- start start 2024-09-16T15:40:13.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.446+0000 7efc52e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc4c073fb0 0x7efc4c156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.446+0000 7efc52e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c0ccf00 0x7efc4c1f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.446+0000 7efc52e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc4c15f5a0 0x7efc4c14c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.446+0000 7efc52e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c06f2e0 con 0x7efc4c0ccf00 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.446+0000 7efc52e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c06f160 con 0x7efc4c073fb0 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.446+0000 7efc52e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7efc4c06f460 con 0x7efc4c15f5a0 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c0ccf00 0x7efc4c1f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c0ccf00 0x7efc4c1f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37356/0 (socket says 172.21.15.60:37356) 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4b400640 1 -- 172.21.15.60:0/1453332570 learned_addr learned my addr 172.21.15.60:0/1453332570 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4be00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc4c073fb0 0x7efc4c156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc50e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc4c15f5a0 0x7efc4c14c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4b400640 1 -- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc4c15f5a0 msgr2=0x7efc4c14c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4b400640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc4c15f5a0 0x7efc4c14c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4b400640 1 -- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc4c073fb0 msgr2=0x7efc4c156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4b400640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc4c073fb0 0x7efc4c156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc4b400640 1 -- 172.21.15.60:0/1453332570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c15a9f0 con 0x7efc4c0ccf00 2024-09-16T15:40:13.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.447+0000 7efc50e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc4c15f5a0 0x7efc4c14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:13.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.448+0000 7efc4b400640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c0ccf00 0x7efc4c1f3610 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efc3800d770 tx=0x7efc3800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.448+0000 7efc43e00640 1 -- 172.21.15.60:0/1453332570 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38017070 con 0x7efc4c0ccf00 2024-09-16T15:40:13.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.448+0000 7efc43e00640 1 -- 172.21.15.60:0/1453332570 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc38009d20 con 0x7efc4c0ccf00 2024-09-16T15:40:13.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.448+0000 7efc43e00640 1 -- 172.21.15.60:0/1453332570 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38005280 con 0x7efc4c0ccf00 2024-09-16T15:40:13.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.448+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c15af70 con 0x7efc4c0ccf00 2024-09-16T15:40:13.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.448+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c15b530 con 0x7efc4c0ccf00 2024-09-16T15:40:13.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.449+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c15b730 con 0x7efc4c0ccf00 2024-09-16T15:40:13.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.450+0000 7efc43e00640 1 -- 172.21.15.60:0/1453332570 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7efc38005420 con 0x7efc4c0ccf00 2024-09-16T15:40:13.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.451+0000 7efc43e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efc2c078a00 0x7efc2c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.451+0000 7efc43e00640 1 -- 172.21.15.60:0/1453332570 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7efc3809aeb0 con 0x7efc4c0ccf00 2024-09-16T15:40:13.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.452+0000 7efc4be00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efc2c078a00 0x7efc2c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.452+0000 7efc4be00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efc2c078a00 0x7efc2c07aec0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc3c006fd0 tx=0x7efc3c009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.455+0000 7efc43e00640 1 -- 172.21.15.60:0/1453332570 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc4c15b730 con 0x7efc4c0ccf00 2024-09-16T15:40:13.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.651+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc4c06f9f0 con 0x7efc4c0ccf00 2024-09-16T15:40:13.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.652+0000 7efc43e00640 1 -- 172.21.15.60:0/1453332570 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16686 (secure 0 0 0) 0x7efc38062aa0 con 0x7efc4c0ccf00 2024-09-16T15:40:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:13.654 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:40:12.294673+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.089921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:11.255279+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efc2c078a00 msgr2=0x7efc2c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efc2c078a00 0x7efc2c07aec0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc3c006fd0 tx=0x7efc3c009060 comp rx=0 tx=0).stop 2024-09-16T15:40:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c0ccf00 msgr2=0x7efc4c1f3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c0ccf00 0x7efc4c1f3610 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efc3800d770 tx=0x7efc3800dc30 comp rx=0 tx=0).stop 2024-09-16T15:40:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 shutdown_connections 2024-09-16T15:40:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efc2c078a00 0x7efc2c07aec0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc4c15f5a0 0x7efc4c14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc4c0ccf00 0x7efc4c1f3610 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 --2- 172.21.15.60:0/1453332570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc4c073fb0 0x7efc4c156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 >> 172.21.15.60:0/1453332570 conn(0x7efc4c0b1950 msgr2=0x7efc4c15dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 shutdown_connections 2024-09-16T15:40:13.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.655+0000 7efc52e00640 1 -- 172.21.15.60:0/1453332570 wait complete. 2024-09-16T15:40:13.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:40:13.829 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-09-16T15:40:13.828+0000 7f41aa600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:40:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.836+0000 7f83a2c00640 1 -- 172.21.15.60:0/1918018229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f839c0cccd0 msgr2=0x7f839c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.836+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1918018229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f839c0cccd0 0x7f839c0cdbb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8394005fd0 tx=0x7f839402aa10 comp rx=0 tx=0).stop 2024-09-16T15:40:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.837+0000 7f83a2c00640 1 -- 172.21.15.60:0/1918018229 shutdown_connections 2024-09-16T15:40:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.837+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1918018229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f839c0cccd0 0x7f839c0cdbb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.837+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1918018229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f839c145750 0x7f839c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.837+0000 7f83a2c00640 1 -- 172.21.15.60:0/1918018229 >> 172.21.15.60:0/1918018229 conn(0x7f839c0b1840 msgr2=0x7f839c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.837+0000 7f83a2c00640 1 -- 172.21.15.60:0/1918018229 shutdown_connections 2024-09-16T15:40:13.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.837+0000 7f83a2c00640 1 -- 172.21.15.60:0/1918018229 wait complete. 2024-09-16T15:40:13.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.838+0000 7f83a2c00640 1 Processor -- start 2024-09-16T15:40:13.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.838+0000 7f83a2c00640 1 -- start start 2024-09-16T15:40:13.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f83a2c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839c06efa0 0x7f839c15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f83a2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f839c0cccd0 0x7f839c0b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f83a2c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f839c145750 0x7f839c12abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f83a2c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0ce480 con 0x7f839c06efa0 2024-09-16T15:40:13.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f83a2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0ce300 con 0x7f839c0cccd0 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f83a2c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f839c0ce600 con 0x7f839c145750 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f839be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839c06efa0 0x7f839c15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f839be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839c06efa0 0x7f839c15eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37372/0 (socket says 172.21.15.60:37372) 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f839be00640 1 -- 172.21.15.60:0/1310512139 learned_addr learned my addr 172.21.15.60:0/1310512139 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f8393e00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f839c0cccd0 0x7f839c0b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f83a0c00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f839c145750 0x7f839c12abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f839be00640 1 -- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f839c145750 msgr2=0x7f839c12abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f839be00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f839c145750 0x7f839c12abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f839be00640 1 -- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f839c0cccd0 msgr2=0x7f839c0b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f839be00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f839c0cccd0 0x7f839c0b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.840+0000 7f839be00640 1 -- 172.21.15.60:0/1310512139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0b4830 con 0x7f839c06efa0 2024-09-16T15:40:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.841+0000 7f839be00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839c06efa0 0x7f839c15eca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e9f0 tx=0x7f838c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.842+0000 7f8398c00640 1 -- 172.21.15.60:0/1310512139 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c00cd80 con 0x7f839c06efa0 2024-09-16T15:40:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.842+0000 7f8398c00640 1 -- 172.21.15.60:0/1310512139 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f838c004020 con 0x7f839c06efa0 2024-09-16T15:40:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.842+0000 7f8398c00640 1 -- 172.21.15.60:0/1310512139 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c004470 con 0x7f839c06efa0 2024-09-16T15:40:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.842+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c0b4b10 con 0x7f839c06efa0 2024-09-16T15:40:13.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.842+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c0adff0 con 0x7f839c06efa0 2024-09-16T15:40:13.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.843+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c0cdbb0 con 0x7f839c06efa0 2024-09-16T15:40:13.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.844+0000 7f8398c00640 1 -- 172.21.15.60:0/1310512139 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f838c0035f0 con 0x7f839c06efa0 2024-09-16T15:40:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.845+0000 7f8398c00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8378078a00 0x7f837807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.845+0000 7f8398c00640 1 -- 172.21.15.60:0/1310512139 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f838c09b790 con 0x7f839c06efa0 2024-09-16T15:40:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.846+0000 7f8393e00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8378078a00 0x7f837807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.846+0000 7f8393e00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8378078a00 0x7f837807aec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f839c1400e0 tx=0x7f8384009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:13.849+0000 7f8398c00640 1 -- 172.21.15.60:0/1310512139 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f838c063380 con 0x7f839c06efa0 2024-09-16T15:40:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.086+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f839c15f240 con 0x7f839c06efa0 2024-09-16T15:40:14.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.857+0000 7f8398c00640 1 -- 172.21.15.60:0/1310512139 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f838c067db0 con 0x7f839c06efa0 2024-09-16T15:40:14.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8378078a00 msgr2=0x7f837807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:14.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8378078a00 0x7f837807aec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f839c1400e0 tx=0x7f8384009040 comp rx=0 tx=0).stop 2024-09-16T15:40:14.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839c06efa0 msgr2=0x7f839c15eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:14.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839c06efa0 0x7f839c15eca0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e9f0 tx=0x7f838c00eeb0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 shutdown_connections 2024-09-16T15:40:14.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8378078a00 0x7f837807aec0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f839c145750 0x7f839c12abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f839c0cccd0 0x7f839c0b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 --2- 172.21.15.60:0/1310512139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839c06efa0 0x7f839c15eca0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 >> 172.21.15.60:0/1310512139 conn(0x7f839c0b1840 msgr2=0x7f839c144160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:14.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 shutdown_connections 2024-09-16T15:40:14.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:14.861+0000 7f83a2c00640 1 -- 172.21.15.60:0/1310512139 wait complete. 2024-09-16T15:40:14.882 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:40:14.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:40:15.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.049+0000 7f0457200640 1 -- 172.21.15.60:0/1791308309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0458161460 msgr2=0x7f04580d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.049+0000 7f0457200640 1 --2- 172.21.15.60:0/1791308309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0458161460 0x7f04580d22f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0448009830 tx=0x7f044802a480 comp rx=0 tx=0).stop 2024-09-16T15:40:15.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.049+0000 7f0457200640 1 -- 172.21.15.60:0/1791308309 shutdown_connections 2024-09-16T15:40:15.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.049+0000 7f0457200640 1 --2- 172.21.15.60:0/1791308309 >> v2:172.21.15.100:3300/0 conn(0x7f04580b43c0 0x7f04580d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.049+0000 7f0457200640 1 --2- 172.21.15.60:0/1791308309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0458161460 0x7f04580d22f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.049+0000 7f0457200640 1 -- 172.21.15.60:0/1791308309 >> 172.21.15.60:0/1791308309 conn(0x7f045809d0b0 msgr2=0x7f04580d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:15.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.049+0000 7f0457200640 1 -- 172.21.15.60:0/1791308309 shutdown_connections 2024-09-16T15:40:15.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.049+0000 7f0457200640 1 -- 172.21.15.60:0/1791308309 wait complete. 2024-09-16T15:40:15.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.051+0000 7f0457200640 1 Processor -- start 2024-09-16T15:40:15.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.051+0000 7f0457200640 1 -- start start 2024-09-16T15:40:15.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0457200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04580b43c0 0x7f045814bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:15.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0457200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04580b5480 0x7f04580b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:15.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0457200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0458161460 0x7f04580b9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:15.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0457200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f04580cf0b0 con 0x7f0458161460 2024-09-16T15:40:15.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0457200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04580cef30 con 0x7f04580b5480 2024-09-16T15:40:15.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0457200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f04580cf230 con 0x7f04580b43c0 2024-09-16T15:40:15.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0456800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0458161460 0x7f04580b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:15.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0456800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0458161460 0x7f04580b9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37380/0 (socket says 172.21.15.60:37380) 2024-09-16T15:40:15.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0456800640 1 -- 172.21.15.60:0/2465066033 learned_addr learned my addr 172.21.15.60:0/2465066033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:15.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0455e00640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04580b43c0 0x7f045814bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:15.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0455400640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04580b5480 0x7f04580b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0456800640 1 -- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04580b43c0 msgr2=0x7f045814bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0456800640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04580b43c0 0x7f045814bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0456800640 1 -- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04580b5480 msgr2=0x7f04580b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0456800640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04580b5480 0x7f04580b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.052+0000 7f0456800640 1 -- 172.21.15.60:0/2465066033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045814c460 con 0x7f0458161460 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.053+0000 7f0456800640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0458161460 0x7f04580b9380 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f045000ef60 tx=0x7f045000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.053+0000 7f044e000640 1 -- 172.21.15.60:0/2465066033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450004b20 con 0x7f0458161460 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.053+0000 7f044e000640 1 -- 172.21.15.60:0/2465066033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0450004dd0 con 0x7f0458161460 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.053+0000 7f044e000640 1 -- 172.21.15.60:0/2465066033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450003bd0 con 0x7f0458161460 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.053+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04581f8a30 con 0x7f0458161460 2024-09-16T15:40:15.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.053+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04581f8ff0 con 0x7f0458161460 2024-09-16T15:40:15.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.054+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04580cf750 con 0x7f0458161460 2024-09-16T15:40:15.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.055+0000 7f044e000640 1 -- 172.21.15.60:0/2465066033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f045001d020 con 0x7f0458161460 2024-09-16T15:40:15.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.057+0000 7f044e000640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0438078a00 0x7f043807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:15.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.057+0000 7f044e000640 1 -- 172.21.15.60:0/2465066033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f045009b280 con 0x7f0458161460 2024-09-16T15:40:15.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.057+0000 7f0455e00640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0438078a00 0x7f043807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:15.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.058+0000 7f0455e00640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0438078a00 0x7f043807aec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0448034000 tx=0x7f0448004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:15.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.061+0000 7f044e000640 1 -- 172.21.15.60:0/2465066033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0450062f00 con 0x7f0458161460 2024-09-16T15:40:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.297+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f045806aaf0 con 0x7f0458161460 2024-09-16T15:40:15.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.863+0000 7f044e000640 1 -- 172.21.15.60:0/2465066033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f0450067930 con 0x7f0458161460 2024-09-16T15:40:15.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0438078a00 msgr2=0x7f043807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0438078a00 0x7f043807aec0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0448034000 tx=0x7f0448004730 comp rx=0 tx=0).stop 2024-09-16T15:40:15.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0458161460 msgr2=0x7f04580b9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0458161460 0x7f04580b9380 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f045000ef60 tx=0x7f045000ef90 comp rx=0 tx=0).stop 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 shutdown_connections 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0438078a00 0x7f043807aec0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0458161460 0x7f04580b9380 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04580b5480 0x7f04580b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 --2- 172.21.15.60:0/2465066033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04580b43c0 0x7f045814bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 >> 172.21.15.60:0/2465066033 conn(0x7f045809d0b0 msgr2=0x7f045815f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 shutdown_connections 2024-09-16T15:40:15.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:15.866+0000 7f0457200640 1 -- 172.21.15.60:0/2465066033 wait complete. 2024-09-16T15:40:15.885 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:40:15.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:16.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.048+0000 7f0c2ac00640 1 -- 172.21.15.60:0/4004397684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 msgr2=0x7f0c240d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.048+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/4004397684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 0x7f0c240d2760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c240cc220 tx=0x7f0c1802a560 comp rx=0 tx=0).stop 2024-09-16T15:40:16.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.049+0000 7f0c2ac00640 1 -- 172.21.15.60:0/4004397684 shutdown_connections 2024-09-16T15:40:16.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.049+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/4004397684 >> v2:172.21.15.100:3300/0 conn(0x7f0c240b1680 0x7f0c240b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.049+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/4004397684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 0x7f0c240d2760 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.049+0000 7f0c2ac00640 1 -- 172.21.15.60:0/4004397684 >> 172.21.15.60:0/4004397684 conn(0x7f0c240d1c40 msgr2=0x7f0c240d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.049+0000 7f0c2ac00640 1 -- 172.21.15.60:0/4004397684 shutdown_connections 2024-09-16T15:40:16.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.049+0000 7f0c2ac00640 1 -- 172.21.15.60:0/4004397684 wait complete. 2024-09-16T15:40:16.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.050+0000 7f0c2ac00640 1 Processor -- start 2024-09-16T15:40:16.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.050+0000 7f0c2ac00640 1 -- start start 2024-09-16T15:40:16.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.051+0000 7f0c2ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c240b1680 0x7f0c240cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.051+0000 7f0c2ac00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c240cff60 0x7f0c240c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.051+0000 7f0c2ac00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 0x7f0c240b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.051+0000 7f0c2ac00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0c240b21f0 con 0x7f0c2415bd30 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.051+0000 7f0c2ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0c240b2070 con 0x7f0c240b1680 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.051+0000 7f0c2ac00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0c240b2370 con 0x7f0c240cff60 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 0x7f0c240b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 0x7f0c240b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37396/0 (socket says 172.21.15.60:37396) 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 -- 172.21.15.60:0/270968840 learned_addr learned my addr 172.21.15.60:0/270968840 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c29800640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c240b1680 0x7f0c240cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c28e00640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c240cff60 0x7f0c240c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 -- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c240cff60 msgr2=0x7f0c240c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c240cff60 0x7f0c240c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 -- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c240b1680 msgr2=0x7f0c240cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c240b1680 0x7f0c240cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 -- 172.21.15.60:0/270968840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c241f5af0 con 0x7f0c2415bd30 2024-09-16T15:40:16.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.052+0000 7f0c2a200640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 0x7f0c240b6ed0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00ead0 tx=0x7f0c1c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.053+0000 7f0c21600640 1 -- 172.21.15.60:0/270968840 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00cde0 con 0x7f0c2415bd30 2024-09-16T15:40:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.053+0000 7f0c21600640 1 -- 172.21.15.60:0/270968840 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c1c004c10 con 0x7f0c2415bd30 2024-09-16T15:40:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.053+0000 7f0c21600640 1 -- 172.21.15.60:0/270968840 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c005020 con 0x7f0c2415bd30 2024-09-16T15:40:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.053+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c241f5db0 con 0x7f0c2415bd30 2024-09-16T15:40:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.053+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c241f6370 con 0x7f0c2415bd30 2024-09-16T15:40:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.054+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c240b2820 con 0x7f0c2415bd30 2024-09-16T15:40:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.055+0000 7f0c21600640 1 -- 172.21.15.60:0/270968840 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f0c1c0051c0 con 0x7f0c2415bd30 2024-09-16T15:40:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.056+0000 7f0c21600640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0c04078a00 0x7f0c0407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.057+0000 7f0c21600640 1 -- 172.21.15.60:0/270968840 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f0c1c09b530 con 0x7f0c2415bd30 2024-09-16T15:40:16.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.057+0000 7f0c29800640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0c04078a00 0x7f0c0407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.058+0000 7f0c29800640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0c04078a00 0x7f0c0407aec0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c18033000 tx=0x7f0c18004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.061+0000 7f0c21600640 1 -- 172.21.15.60:0/270968840 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0c1c016030 con 0x7f0c2415bd30 2024-09-16T15:40:16.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.296+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c240cde20 con 0x7f0c2415bd30 2024-09-16T15:40:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.297+0000 7f0c21600640 1 -- 172.21.15.60:0/270968840 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7f0c1c0631d0 con 0x7f0c2415bd30 2024-09-16T15:40:16.298 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:16.298 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-09-16T15:40:15:858549+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-09-16T15:40:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0c04078a00 msgr2=0x7f0c0407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0c04078a00 0x7f0c0407aec0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0c18033000 tx=0x7f0c18004730 comp rx=0 tx=0).stop 2024-09-16T15:40:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 msgr2=0x7f0c240b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 0x7f0c240b6ed0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00ead0 tx=0x7f0c1c00ef90 comp rx=0 tx=0).stop 2024-09-16T15:40:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 shutdown_connections 2024-09-16T15:40:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0c04078a00 0x7f0c0407aec0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c2415bd30 0x7f0c240b6ed0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c240cff60 0x7f0c240c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 --2- 172.21.15.60:0/270968840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c240b1680 0x7f0c240cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 >> 172.21.15.60:0/270968840 conn(0x7f0c240d1c40 msgr2=0x7f0c24159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 shutdown_connections 2024-09-16T15:40:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.299+0000 7f0c2ac00640 1 -- 172.21.15.60:0/270968840 wait complete. 2024-09-16T15:40:16.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:16.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.476+0000 7fa106800640 1 -- 172.21.15.60:0/103706535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ccf00 msgr2=0x7fa1000acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.476+0000 7fa106800640 1 --2- 172.21.15.60:0/103706535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ccf00 0x7fa1000acd30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec005fd0 tx=0x7fa0ec02a4b0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.476+0000 7fa106800640 1 -- 172.21.15.60:0/103706535 shutdown_connections 2024-09-16T15:40:16.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.476+0000 7fa106800640 1 --2- 172.21.15.60:0/103706535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ccf00 0x7fa1000acd30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.476+0000 7fa106800640 1 --2- 172.21.15.60:0/103706535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa10015f5a0 0x7fa1000b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.476+0000 7fa106800640 1 -- 172.21.15.60:0/103706535 >> 172.21.15.60:0/103706535 conn(0x7fa1000b1950 msgr2=0x7fa1000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.476+0000 7fa106800640 1 -- 172.21.15.60:0/103706535 shutdown_connections 2024-09-16T15:40:16.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.476+0000 7fa106800640 1 -- 172.21.15.60:0/103706535 wait complete. 2024-09-16T15:40:16.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.477+0000 7fa106800640 1 Processor -- start 2024-09-16T15:40:16.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.478+0000 7fa106800640 1 -- start start 2024-09-16T15:40:16.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.478+0000 7fa106800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ad270 0x7fa1000cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.478+0000 7fa106800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1000ccf00 0x7fa1001f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.478+0000 7fa106800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa10015f5a0 0x7fa100152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa106800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa10006f2e0 con 0x7fa1000ad270 2024-09-16T15:40:16.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa106800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa10006f160 con 0x7fa1000ccf00 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa106800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa10006f460 con 0x7fa10015f5a0 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ff400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ad270 0x7fa1000cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ff400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ad270 0x7fa1000cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37418/0 (socket says 172.21.15.60:37418) 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ff400640 1 -- 172.21.15.60:0/507562644 learned_addr learned my addr 172.21.15.60:0/507562644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ffe00640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa10015f5a0 0x7fa100152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0fea00640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1000ccf00 0x7fa1001f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ff400640 1 -- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa10015f5a0 msgr2=0x7fa100152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ff400640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa10015f5a0 0x7fa100152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ff400640 1 -- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1000ccf00 msgr2=0x7fa1001f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ff400640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1000ccf00 0x7fa1001f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0ff400640 1 -- 172.21.15.60:0/507562644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1001f5790 con 0x7fa1000ad270 2024-09-16T15:40:16.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.479+0000 7fa0fea00640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1000ccf00 0x7fa1001f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:16.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.480+0000 7fa0ff400640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ad270 0x7fa1000cb1b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000d700 tx=0x7fa0f000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.480+0000 7fa0f7400640 1 -- 172.21.15.60:0/507562644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0017070 con 0x7fa1000ad270 2024-09-16T15:40:16.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.480+0000 7fa0f7400640 1 -- 172.21.15.60:0/507562644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0f0005310 con 0x7fa1000ad270 2024-09-16T15:40:16.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.480+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1001f5a70 con 0x7fa1000ad270 2024-09-16T15:40:16.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.480+0000 7fa0f7400640 1 -- 172.21.15.60:0/507562644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00097c0 con 0x7fa1000ad270 2024-09-16T15:40:16.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.480+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1000cb6f0 con 0x7fa1000ad270 2024-09-16T15:40:16.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.481+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10006f980 con 0x7fa1000ad270 2024-09-16T15:40:16.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.482+0000 7fa0f7400640 1 -- 172.21.15.60:0/507562644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fa0f0009960 con 0x7fa1000ad270 2024-09-16T15:40:16.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.483+0000 7fa0f7400640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa0e0078a00 0x7fa0e007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.483+0000 7fa0f7400640 1 -- 172.21.15.60:0/507562644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa0f009adc0 con 0x7fa1000ad270 2024-09-16T15:40:16.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.484+0000 7fa0fea00640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa0e0078a00 0x7fa0e007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.484+0000 7fa0fea00640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa0e0078a00 0x7fa0e007aec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec033000 tx=0x7fa0ec004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.487+0000 7fa0f7400640 1 -- 172.21.15.60:0/507562644 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa0f00629b0 con 0x7fa1000ad270 2024-09-16T15:40:16.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.689+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa10015edc0 con 0x7fa1000ad270 2024-09-16T15:40:16.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.690+0000 7fa0f7400640 1 -- 172.21.15.60:0/507562644 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16673 (secure 0 0 0) 0x7fa0f00673e0 con 0x7fa1000ad270 2024-09-16T15:40:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:16.692 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:40:13.850086+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.089921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:11.255279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa0e0078a00 msgr2=0x7fa0e007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa0e0078a00 0x7fa0e007aec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec033000 tx=0x7fa0ec004730 comp rx=0 tx=0).stop 2024-09-16T15:40:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ad270 msgr2=0x7fa1000cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ad270 0x7fa1000cb1b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000d700 tx=0x7fa0f000dbc0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 shutdown_connections 2024-09-16T15:40:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa0e0078a00 0x7fa0e007aec0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa10015f5a0 0x7fa100152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1000ccf00 0x7fa1001f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 --2- 172.21.15.60:0/507562644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1000ad270 0x7fa1000cb1b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 >> 172.21.15.60:0/507562644 conn(0x7fa1000b1950 msgr2=0x7fa1000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 shutdown_connections 2024-09-16T15:40:16.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.693+0000 7fa106800640 1 -- 172.21.15.60:0/507562644 wait complete. 2024-09-16T15:40:16.710 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:40:16.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:16.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.872+0000 7fba53400640 1 -- 172.21.15.60:0/2554441559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 msgr2=0x7fba4c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.872+0000 7fba53400640 1 --2- 172.21.15.60:0/2554441559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 0x7fba4c0b2440 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba34009830 tx=0x7fba3402a530 comp rx=0 tx=0).stop 2024-09-16T15:40:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.872+0000 7fba53400640 1 -- 172.21.15.60:0/2554441559 shutdown_connections 2024-09-16T15:40:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.872+0000 7fba53400640 1 --2- 172.21.15.60:0/2554441559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 0x7fba4c0b2440 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.872+0000 7fba53400640 1 -- 172.21.15.60:0/2554441559 >> 172.21.15.60:0/2554441559 conn(0x7fba4c0b1950 msgr2=0x7fba4c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.873+0000 7fba53400640 1 -- 172.21.15.60:0/2554441559 shutdown_connections 2024-09-16T15:40:16.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.873+0000 7fba53400640 1 -- 172.21.15.60:0/2554441559 wait complete. 2024-09-16T15:40:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.874+0000 7fba53400640 1 Processor -- start 2024-09-16T15:40:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.874+0000 7fba53400640 1 -- start start 2024-09-16T15:40:16.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.875+0000 7fba53400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba4c073fb0 0x7fba4c1f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.875+0000 7fba53400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 0x7fba4c14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.875+0000 7fba53400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba4c15f5a0 0x7fba4c15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.875+0000 7fba53400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c06f9f0 con 0x7fba4c073fb0 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.875+0000 7fba53400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c06f870 con 0x7fba4c0ccf00 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.875+0000 7fba53400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fba4c074420 con 0x7fba4c15f5a0 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.875+0000 7fba50a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba4c073fb0 0x7fba4c1f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba50a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba4c073fb0 0x7fba4c1f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37424/0 (socket says 172.21.15.60:37424) 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba50a00640 1 -- 172.21.15.60:0/1466615476 learned_addr learned my addr 172.21.15.60:0/1466615476 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba43e00640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 0x7fba4c14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba51400640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba4c15f5a0 0x7fba4c15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba50a00640 1 -- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba4c15f5a0 msgr2=0x7fba4c15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba50a00640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba4c15f5a0 0x7fba4c15cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba50a00640 1 -- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 msgr2=0x7fba4c14dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba50a00640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 0x7fba4c14dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba50a00640 1 -- 172.21.15.60:0/1466615476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c14ea30 con 0x7fba4c073fb0 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba43e00640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 0x7fba4c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba51400640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba4c15f5a0 0x7fba4c15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba50a00640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba4c073fb0 0x7fba4c1f85b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00c400 tx=0x7fba3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.876+0000 7fba49600640 1 -- 172.21.15.60:0/1466615476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c017070 con 0x7fba4c073fb0 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.877+0000 7fba49600640 1 -- 172.21.15.60:0/1466615476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba3c0053c0 con 0x7fba4c073fb0 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.877+0000 7fba49600640 1 -- 172.21.15.60:0/1466615476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c003850 con 0x7fba4c073fb0 2024-09-16T15:40:16.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.877+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c1f8af0 con 0x7fba4c073fb0 2024-09-16T15:40:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.877+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c1f4dc0 con 0x7fba4c073fb0 2024-09-16T15:40:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.878+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4c0b2440 con 0x7fba4c073fb0 2024-09-16T15:40:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.879+0000 7fba49600640 1 -- 172.21.15.60:0/1466615476 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fba3c01a070 con 0x7fba4c073fb0 2024-09-16T15:40:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.880+0000 7fba49600640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fba28078a00 0x7fba2807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.880+0000 7fba49600640 1 -- 172.21.15.60:0/1466615476 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fba3c09a080 con 0x7fba4c073fb0 2024-09-16T15:40:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.880+0000 7fba43e00640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fba28078a00 0x7fba2807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.881+0000 7fba43e00640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fba28078a00 0x7fba2807aec0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fba34034000 tx=0x7fba34004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:16.884+0000 7fba49600640 1 -- 172.21.15.60:0/1466615476 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fba4c1f8af0 con 0x7fba4c073fb0 2024-09-16T15:40:17.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.076+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba4c0b1dd0 con 0x7fba28078a00 2024-09-16T15:40:17.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.081+0000 7fba49600640 1 -- 172.21.15.60:0/1466615476 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251694 (secure 0 0 0) 0x7fba4c0b1dd0 con 0x7fba28078a00 2024-09-16T15:40:17.086 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fba28078a00 msgr2=0x7fba2807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fba28078a00 0x7fba2807aec0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fba34034000 tx=0x7fba34004730 comp rx=0 tx=0).stop 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba4c073fb0 msgr2=0x7fba4c1f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba4c073fb0 0x7fba4c1f85b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00c400 tx=0x7fba3c00c8c0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 shutdown_connections 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fba28078a00 0x7fba2807aec0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba4c15f5a0 0x7fba4c15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba4c0ccf00 0x7fba4c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 --2- 172.21.15.60:0/1466615476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba4c073fb0 0x7fba4c1f85b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 >> 172.21.15.60:0/1466615476 conn(0x7fba4c0b1950 msgr2=0x7fba4c0b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 shutdown_connections 2024-09-16T15:40:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.085+0000 7fba53400640 1 -- 172.21.15.60:0/1466615476 wait complete. 2024-09-16T15:40:17.102 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-16T15:40:15.733953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238240,"kb_used_data":5340,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736480,"statfs":{"total":773094113280,"available":772850155520,"internally_reserved":0,"allocated":5468160,"data_stored":2888711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.893876"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872674+0000","last_change":"2024-09-16T15:40:12.868607+0000","last_active":"2024-09-16T15:40:12.872674+0000","last_peered":"2024-09-16T15:40:12.872674+0000","last_clean":"2024-09-16T15:40:12.872674+0000","last_became_active":"2024-09-16T15:40:12.868140+0000","last_became_peered":"2024-09-16T15:40:12.868140+0000","last_unstale":"2024-09-16T15:40:12.872674+0000","last_undegraded":"2024-09-16T15:40:12.872674+0000","last_fullsized":"2024-09-16T15:40:12.872674+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:29.669859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872203+0000","last_change":"2024-09-16T15:40:12.869358+0000","last_active":"2024-09-16T15:40:12.872203+0000","last_peered":"2024-09-16T15:40:12.872203+0000","last_clean":"2024-09-16T15:40:12.872203+0000","last_became_active":"2024-09-16T15:40:12.868998+0000","last_became_peered":"2024-09-16T15:40:12.868998+0000","last_unstale":"2024-09-16T15:40:12.872203+0000","last_undegraded":"2024-09-16T15:40:12.872203+0000","last_fullsized":"2024-09-16T15:40:12.872203+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874339+0000","last_change":"2024-09-16T15:40:12.868681+0000","last_active":"2024-09-16T15:40:12.874339+0000","last_peered":"2024-09-16T15:40:12.874339+0000","last_clean":"2024-09-16T15:40:12.874339+0000","last_became_active":"2024-09-16T15:40:12.867980+0000","last_became_peered":"2024-09-16T15:40:12.867980+0000","last_unstale":"2024-09-16T15:40:12.874339+0000","last_undegraded":"2024-09-16T15:40:12.874339+0000","last_fullsized":"2024-09-16T15:40:12.874339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:39.160061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883276+0000","last_change":"2024-09-16T15:40:12.873202+0000","last_active":"2024-09-16T15:40:12.883276+0000","last_peered":"2024-09-16T15:40:12.883276+0000","last_clean":"2024-09-16T15:40:12.883276+0000","last_became_active":"2024-09-16T15:40:12.872682+0000","last_became_peered":"2024-09-16T15:40:12.872682+0000","last_unstale":"2024-09-16T15:40:12.883276+0000","last_undegraded":"2024-09-16T15:40:12.883276+0000","last_fullsized":"2024-09-16T15:40:12.883276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:23.455972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874812+0000","last_change":"2024-09-16T15:40:12.870112+0000","last_active":"2024-09-16T15:40:12.874812+0000","last_peered":"2024-09-16T15:40:12.874812+0000","last_clean":"2024-09-16T15:40:12.874812+0000","last_became_active":"2024-09-16T15:40:12.869753+0000","last_became_peered":"2024-09-16T15:40:12.869753+0000","last_unstale":"2024-09-16T15:40:12.874812+0000","last_undegraded":"2024-09-16T15:40:12.874812+0000","last_fullsized":"2024-09-16T15:40:12.874812+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875406+0000","last_change":"2024-09-16T15:40:12.866164+0000","last_active":"2024-09-16T15:40:12.875406+0000","last_peered":"2024-09-16T15:40:12.875406+0000","last_clean":"2024-09-16T15:40:12.875406+0000","last_became_active":"2024-09-16T15:40:12.865904+0000","last_became_peered":"2024-09-16T15:40:12.865904+0000","last_unstale":"2024-09-16T15:40:12.875406+0000","last_undegraded":"2024-09-16T15:40:12.875406+0000","last_fullsized":"2024-09-16T15:40:12.875406+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:44.336056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.869132+0000","last_change":"2024-09-16T15:40:12.865180+0000","last_active":"2024-09-16T15:40:12.869132+0000","last_peered":"2024-09-16T15:40:12.869132+0000","last_clean":"2024-09-16T15:40:12.869132+0000","last_became_active":"2024-09-16T15:40:12.864842+0000","last_became_peered":"2024-09-16T15:40:12.864842+0000","last_unstale":"2024-09-16T15:40:12.869132+0000","last_undegraded":"2024-09-16T15:40:12.869132+0000","last_fullsized":"2024-09-16T15:40:12.869132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:36.820991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880732+0000","last_change":"2024-09-16T15:40:12.861554+0000","last_active":"2024-09-16T15:40:12.880732+0000","last_peered":"2024-09-16T15:40:12.880732+0000","last_clean":"2024-09-16T15:40:12.880732+0000","last_became_active":"2024-09-16T15:40:12.861215+0000","last_became_peered":"2024-09-16T15:40:12.861215+0000","last_unstale":"2024-09-16T15:40:12.880732+0000","last_undegraded":"2024-09-16T15:40:12.880732+0000","last_fullsized":"2024-09-16T15:40:12.880732+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:16.169482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882156+0000","last_change":"2024-09-16T15:40:12.864597+0000","last_active":"2024-09-16T15:40:12.882156+0000","last_peered":"2024-09-16T15:40:12.882156+0000","last_clean":"2024-09-16T15:40:12.882156+0000","last_became_active":"2024-09-16T15:40:12.864345+0000","last_became_peered":"2024-09-16T15:40:12.864345+0000","last_unstale":"2024-09-16T15:40:12.882156+0000","last_undegraded":"2024-09-16T15:40:12.882156+0000","last_fullsized":"2024-09-16T15:40:12.882156+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:43.902508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882148+0000","last_change":"2024-09-16T15:40:12.865537+0000","last_active":"2024-09-16T15:40:12.882148+0000","last_peered":"2024-09-16T15:40:12.882148+0000","last_clean":"2024-09-16T15:40:12.882148+0000","last_became_active":"2024-09-16T15:40:12.865293+0000","last_became_peered":"2024-09-16T15:40:12.865293+0000","last_unstale":"2024-09-16T15:40:12.882148+0000","last_undegraded":"2024-09-16T15:40:12.882148+0000","last_fullsized":"2024-09-16T15:40:12.882148+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:15.882606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883443+0000","last_change":"2024-09-16T15:40:12.875755+0000","last_active":"2024-09-16T15:40:12.883443+0000","last_peered":"2024-09-16T15:40:12.883443+0000","last_clean":"2024-09-16T15:40:12.883443+0000","last_became_active":"2024-09-16T15:40:12.875484+0000","last_became_peered":"2024-09-16T15:40:12.875484+0000","last_unstale":"2024-09-16T15:40:12.883443+0000","last_undegraded":"2024-09-16T15:40:12.883443+0000","last_fullsized":"2024-09-16T15:40:12.883443+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:57.204826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870792+0000","last_change":"2024-09-16T15:40:12.866408+0000","last_active":"2024-09-16T15:40:12.870792+0000","last_peered":"2024-09-16T15:40:12.870792+0000","last_clean":"2024-09-16T15:40:12.870792+0000","last_became_active":"2024-09-16T15:40:12.865787+0000","last_became_peered":"2024-09-16T15:40:12.865787+0000","last_unstale":"2024-09-16T15:40:12.870792+0000","last_undegraded":"2024-09-16T15:40:12.870792+0000","last_fullsized":"2024-09-16T15:40:12.870792+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:39.299743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867785+0000","last_change":"2024-09-16T15:40:12.865889+0000","last_active":"2024-09-16T15:40:12.867785+0000","last_peered":"2024-09-16T15:40:12.867785+0000","last_clean":"2024-09-16T15:40:12.867785+0000","last_became_active":"2024-09-16T15:40:12.865611+0000","last_became_peered":"2024-09-16T15:40:12.865611+0000","last_unstale":"2024-09-16T15:40:12.867785+0000","last_undegraded":"2024-09-16T15:40:12.867785+0000","last_fullsized":"2024-09-16T15:40:12.867785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:28.736859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881374+0000","last_change":"2024-09-16T15:40:12.863378+0000","last_active":"2024-09-16T15:40:12.881374+0000","last_peered":"2024-09-16T15:40:12.881374+0000","last_clean":"2024-09-16T15:40:12.881374+0000","last_became_active":"2024-09-16T15:40:12.863108+0000","last_became_peered":"2024-09-16T15:40:12.863108+0000","last_unstale":"2024-09-16T15:40:12.881374+0000","last_undegraded":"2024-09-16T15:40:12.881374+0000","last_fullsized":"2024-09-16T15:40:12.881374+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:34.731306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873670+0000","last_change":"2024-09-16T15:40:12.861959+0000","last_active":"2024-09-16T15:40:12.873670+0000","last_peered":"2024-09-16T15:40:12.873670+0000","last_clean":"2024-09-16T15:40:12.873670+0000","last_became_active":"2024-09-16T15:40:12.861657+0000","last_became_peered":"2024-09-16T15:40:12.861657+0000","last_unstale":"2024-09-16T15:40:12.873670+0000","last_undegraded":"2024-09-16T15:40:12.873670+0000","last_fullsized":"2024-09-16T15:40:12.873670+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:55.582229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871103+0000","last_change":"2024-09-16T15:40:12.865876+0000","last_active":"2024-09-16T15:40:12.871103+0000","last_peered":"2024-09-16T15:40:12.871103+0000","last_clean":"2024-09-16T15:40:12.871103+0000","last_became_active":"2024-09-16T15:40:12.865493+0000","last_became_peered":"2024-09-16T15:40:12.865493+0000","last_unstale":"2024-09-16T15:40:12.871103+0000","last_undegraded":"2024-09-16T15:40:12.871103+0000","last_fullsized":"2024-09-16T15:40:12.871103+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:44.257723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867152+0000","last_change":"2024-09-16T15:40:12.861173+0000","last_active":"2024-09-16T15:40:12.867152+0000","last_peered":"2024-09-16T15:40:12.867152+0000","last_clean":"2024-09-16T15:40:12.867152+0000","last_became_active":"2024-09-16T15:40:12.860306+0000","last_became_peered":"2024-09-16T15:40:12.860306+0000","last_unstale":"2024-09-16T15:40:12.867152+0000","last_undegraded":"2024-09-16T15:40:12.867152+0000","last_fullsized":"2024-09-16T15:40:12.867152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:38.251940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875035+0000","last_change":"2024-09-16T15:40:12.873790+0000","last_active":"2024-09-16T15:40:12.875035+0000","last_peered":"2024-09-16T15:40:12.875035+0000","last_clean":"2024-09-16T15:40:12.875035+0000","last_became_active":"2024-09-16T15:40:12.873524+0000","last_became_peered":"2024-09-16T15:40:12.873524+0000","last_unstale":"2024-09-16T15:40:12.875035+0000","last_undegraded":"2024-09-16T15:40:12.875035+0000","last_fullsized":"2024-09-16T15:40:12.875035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:48.530545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880065+0000","last_change":"2024-09-16T15:40:12.872623+0000","last_active":"2024-09-16T15:40:12.880065+0000","last_peered":"2024-09-16T15:40:12.880065+0000","last_clean":"2024-09-16T15:40:12.880065+0000","last_became_active":"2024-09-16T15:40:12.872346+0000","last_became_peered":"2024-09-16T15:40:12.872346+0000","last_unstale":"2024-09-16T15:40:12.880065+0000","last_undegraded":"2024-09-16T15:40:12.880065+0000","last_fullsized":"2024-09-16T15:40:12.880065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:45.262493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.885065+0000","last_change":"2024-09-16T15:40:12.882504+0000","last_active":"2024-09-16T15:40:12.885065+0000","last_peered":"2024-09-16T15:40:12.885065+0000","last_clean":"2024-09-16T15:40:12.885065+0000","last_became_active":"2024-09-16T15:40:12.882113+0000","last_became_peered":"2024-09-16T15:40:12.882113+0000","last_unstale":"2024-09-16T15:40:12.885065+0000","last_undegraded":"2024-09-16T15:40:12.885065+0000","last_fullsized":"2024-09-16T15:40:12.885065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:35.919623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.869373+0000","last_change":"2024-09-16T15:40:12.857122+0000","last_active":"2024-09-16T15:40:12.869373+0000","last_peered":"2024-09-16T15:40:12.869373+0000","last_clean":"2024-09-16T15:40:12.869373+0000","last_became_active":"2024-09-16T15:40:12.856815+0000","last_became_peered":"2024-09-16T15:40:12.856815+0000","last_unstale":"2024-09-16T15:40:12.869373+0000","last_undegraded":"2024-09-16T15:40:12.869373+0000","last_fullsized":"2024-09-16T15:40:12.869373+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:41.290208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882619+0000","last_change":"2024-09-16T15:40:12.868687+0000","last_active":"2024-09-16T15:40:12.882619+0000","last_peered":"2024-09-16T15:40:12.882619+0000","last_clean":"2024-09-16T15:40:12.882619+0000","last_became_active":"2024-09-16T15:40:12.868063+0000","last_became_peered":"2024-09-16T15:40:12.868063+0000","last_unstale":"2024-09-16T15:40:12.882619+0000","last_undegraded":"2024-09-16T15:40:12.882619+0000","last_fullsized":"2024-09-16T15:40:12.882619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:53.819296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870525+0000","last_change":"2024-09-16T15:40:12.864426+0000","last_active":"2024-09-16T15:40:12.870525+0000","last_peered":"2024-09-16T15:40:12.870525+0000","last_clean":"2024-09-16T15:40:12.870525+0000","last_became_active":"2024-09-16T15:40:12.863762+0000","last_became_peered":"2024-09-16T15:40:12.863762+0000","last_unstale":"2024-09-16T15:40:12.870525+0000","last_undegraded":"2024-09-16T15:40:12.870525+0000","last_fullsized":"2024-09-16T15:40:12.870525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:38.955827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871085+0000","last_change":"2024-09-16T15:40:12.863693+0000","last_active":"2024-09-16T15:40:12.871085+0000","last_peered":"2024-09-16T15:40:12.871085+0000","last_clean":"2024-09-16T15:40:12.871085+0000","last_became_active":"2024-09-16T15:40:12.863383+0000","last_became_peered":"2024-09-16T15:40:12.863383+0000","last_unstale":"2024-09-16T15:40:12.871085+0000","last_undegraded":"2024-09-16T15:40:12.871085+0000","last_fullsized":"2024-09-16T15:40:12.871085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:44.940019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.863957+0000","last_change":"2024-09-16T15:40:12.856737+0000","last_active":"2024-09-16T15:40:12.863957+0000","last_peered":"2024-09-16T15:40:12.863957+0000","last_clean":"2024-09-16T15:40:12.863957+0000","last_became_active":"2024-09-16T15:40:12.856426+0000","last_became_peered":"2024-09-16T15:40:12.856426+0000","last_unstale":"2024-09-16T15:40:12.863957+0000","last_undegraded":"2024-09-16T15:40:12.863957+0000","last_fullsized":"2024-09-16T15:40:12.863957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:58.240556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879492+0000","last_change":"2024-09-16T15:40:12.876499+0000","last_active":"2024-09-16T15:40:12.879492+0000","last_peered":"2024-09-16T15:40:12.879492+0000","last_clean":"2024-09-16T15:40:12.879492+0000","last_became_active":"2024-09-16T15:40:12.876247+0000","last_became_peered":"2024-09-16T15:40:12.876247+0000","last_unstale":"2024-09-16T15:40:12.879492+0000","last_undegraded":"2024-09-16T15:40:12.879492+0000","last_fullsized":"2024-09-16T15:40:12.879492+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:42:34.521097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867366+0000","last_change":"2024-09-16T15:40:12.858149+0000","last_active":"2024-09-16T15:40:12.867366+0000","last_peered":"2024-09-16T15:40:12.867366+0000","last_clean":"2024-09-16T15:40:12.867366+0000","last_became_active":"2024-09-16T15:40:12.857902+0000","last_became_peered":"2024-09-16T15:40:12.857902+0000","last_unstale":"2024-09-16T15:40:12.867366+0000","last_undegraded":"2024-09-16T15:40:12.867366+0000","last_fullsized":"2024-09-16T15:40:12.867366+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:04.893938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870724+0000","last_change":"2024-09-16T15:40:12.865956+0000","last_active":"2024-09-16T15:40:12.870724+0000","last_peered":"2024-09-16T15:40:12.870724+0000","last_clean":"2024-09-16T15:40:12.870724+0000","last_became_active":"2024-09-16T15:40:12.865681+0000","last_became_peered":"2024-09-16T15:40:12.865681+0000","last_unstale":"2024-09-16T15:40:12.870724+0000","last_undegraded":"2024-09-16T15:40:12.870724+0000","last_fullsized":"2024-09-16T15:40:12.870724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:14.857566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.864031+0000","last_change":"2024-09-16T15:40:12.856711+0000","last_active":"2024-09-16T15:40:12.864031+0000","last_peered":"2024-09-16T15:40:12.864031+0000","last_clean":"2024-09-16T15:40:12.864031+0000","last_became_active":"2024-09-16T15:40:12.856290+0000","last_became_peered":"2024-09-16T15:40:12.856290+0000","last_unstale":"2024-09-16T15:40:12.864031+0000","last_undegraded":"2024-09-16T15:40:12.864031+0000","last_fullsized":"2024-09-16T15:40:12.864031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:27.797378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867021+0000","last_change":"2024-09-16T15:40:12.859686+0000","last_active":"2024-09-16T15:40:12.867021+0000","last_peered":"2024-09-16T15:40:12.867021+0000","last_clean":"2024-09-16T15:40:12.867021+0000","last_became_active":"2024-09-16T15:40:12.859399+0000","last_became_peered":"2024-09-16T15:40:12.859399+0000","last_unstale":"2024-09-16T15:40:12.867021+0000","last_undegraded":"2024-09-16T15:40:12.867021+0000","last_fullsized":"2024-09-16T15:40:12.867021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:45.345188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867284+0000","last_change":"2024-09-16T15:40:12.857344+0000","last_active":"2024-09-16T15:40:12.867284+0000","last_peered":"2024-09-16T15:40:12.867284+0000","last_clean":"2024-09-16T15:40:12.867284+0000","last_became_active":"2024-09-16T15:40:12.857044+0000","last_became_peered":"2024-09-16T15:40:12.857044+0000","last_unstale":"2024-09-16T15:40:12.867284+0000","last_undegraded":"2024-09-16T15:40:12.867284+0000","last_fullsized":"2024-09-16T15:40:12.867284+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:28:28.848082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867593+0000","last_change":"2024-09-16T15:40:12.860869+0000","last_active":"2024-09-16T15:40:12.867593+0000","last_peered":"2024-09-16T15:40:12.867593+0000","last_clean":"2024-09-16T15:40:12.867593+0000","last_became_active":"2024-09-16T15:40:12.860241+0000","last_became_peered":"2024-09-16T15:40:12.860241+0000","last_unstale":"2024-09-16T15:40:12.867593+0000","last_undegraded":"2024-09-16T15:40:12.867593+0000","last_fullsized":"2024-09-16T15:40:12.867593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:37.559530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:40:11.850246+0000","last_change":"2024-09-16T15:40:11.850246+0000","last_active":"2024-09-16T15:40:11.835623+0000","last_peered":"2024-09-16T15:40:11.835623+0000","last_clean":"2024-09-16T15:40:11.835623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.850246+0000","last_undegraded":"2024-09-16T15:40:11.850246+0000","last_fullsized":"2024-09-16T15:40:11.850246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875888+0000","last_change":"2024-09-16T15:40:11.859931+0000","last_active":"2024-09-16T15:40:12.875888+0000","last_peered":"2024-09-16T15:40:12.875888+0000","last_clean":"2024-09-16T15:40:12.875888+0000","last_became_active":"2024-09-16T15:40:11.859675+0000","last_became_peered":"2024-09-16T15:40:11.859675+0000","last_unstale":"2024-09-16T15:40:12.875888+0000","last_undegraded":"2024-09-16T15:40:12.875888+0000","last_fullsized":"2024-09-16T15:40:12.875888+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:28.283914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867675+0000","last_change":"2024-09-16T15:40:12.861984+0000","last_active":"2024-09-16T15:40:12.867675+0000","last_peered":"2024-09-16T15:40:12.867675+0000","last_clean":"2024-09-16T15:40:12.867675+0000","last_became_active":"2024-09-16T15:40:12.861759+0000","last_became_peered":"2024-09-16T15:40:12.861759+0000","last_unstale":"2024-09-16T15:40:12.867675+0000","last_undegraded":"2024-09-16T15:40:12.867675+0000","last_fullsized":"2024-09-16T15:40:12.867675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:23.370252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880884+0000","last_change":"2024-09-16T15:40:11.863371+0000","last_active":"2024-09-16T15:40:12.880884+0000","last_peered":"2024-09-16T15:40:12.880884+0000","last_clean":"2024-09-16T15:40:12.880884+0000","last_became_active":"2024-09-16T15:40:11.860269+0000","last_became_peered":"2024-09-16T15:40:11.860269+0000","last_unstale":"2024-09-16T15:40:12.880884+0000","last_undegraded":"2024-09-16T15:40:12.880884+0000","last_fullsized":"2024-09-16T15:40:12.880884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:36.282583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.858764+0000","last_change":"2024-09-16T15:40:12.855039+0000","last_active":"2024-09-16T15:40:12.858764+0000","last_peered":"2024-09-16T15:40:12.858764+0000","last_clean":"2024-09-16T15:40:12.858764+0000","last_became_active":"2024-09-16T15:40:12.854141+0000","last_became_peered":"2024-09-16T15:40:12.854141+0000","last_unstale":"2024-09-16T15:40:12.858764+0000","last_undegraded":"2024-09-16T15:40:12.858764+0000","last_fullsized":"2024-09-16T15:40:12.858764+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:26.535001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:40:11.871707+0000","last_change":"2024-09-16T15:40:11.869853+0000","last_active":"2024-09-16T15:40:11.871707+0000","last_peered":"2024-09-16T15:40:11.871707+0000","last_clean":"2024-09-16T15:40:11.871707+0000","last_became_active":"2024-09-16T15:40:11.869583+0000","last_became_peered":"2024-09-16T15:40:11.869583+0000","last_unstale":"2024-09-16T15:40:11.871707+0000","last_undegraded":"2024-09-16T15:40:11.871707+0000","last_fullsized":"2024-09-16T15:40:11.871707+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:12.986348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:40:11.852066+0000","last_change":"2024-09-16T15:40:11.852066+0000","last_active":"2024-09-16T15:40:11.835623+0000","last_peered":"2024-09-16T15:40:11.835623+0000","last_clean":"2024-09-16T15:40:11.835623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.852066+0000","last_undegraded":"2024-09-16T15:40:11.852066+0000","last_fullsized":"2024-09-16T15:40:11.852066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880313+0000","last_change":"2024-09-16T15:40:11.868506+0000","last_active":"2024-09-16T15:40:12.880313+0000","last_peered":"2024-09-16T15:40:12.880313+0000","last_clean":"2024-09-16T15:40:12.880313+0000","last_became_active":"2024-09-16T15:40:11.868014+0000","last_became_peered":"2024-09-16T15:40:11.868014+0000","last_unstale":"2024-09-16T15:40:12.880313+0000","last_undegraded":"2024-09-16T15:40:12.880313+0000","last_fullsized":"2024-09-16T15:40:12.880313+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.292320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874345+0000","last_change":"2024-09-16T15:40:12.867667+0000","last_active":"2024-09-16T15:40:12.874345+0000","last_peered":"2024-09-16T15:40:12.874345+0000","last_clean":"2024-09-16T15:40:12.874345+0000","last_became_active":"2024-09-16T15:40:12.866213+0000","last_became_peered":"2024-09-16T15:40:12.866213+0000","last_unstale":"2024-09-16T15:40:12.874345+0000","last_undegraded":"2024-09-16T15:40:12.874345+0000","last_fullsized":"2024-09-16T15:40:12.874345+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:06.413661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.877703+0000","last_change":"2024-09-16T15:40:11.871006+0000","last_active":"2024-09-16T15:40:12.877703+0000","last_peered":"2024-09-16T15:40:12.877703+0000","last_clean":"2024-09-16T15:40:12.877703+0000","last_became_active":"2024-09-16T15:40:11.870728+0000","last_became_peered":"2024-09-16T15:40:11.870728+0000","last_unstale":"2024-09-16T15:40:12.877703+0000","last_undegraded":"2024-09-16T15:40:12.877703+0000","last_fullsized":"2024-09-16T15:40:12.877703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:26.890614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882663+0000","last_change":"2024-09-16T15:40:12.869124+0000","last_active":"2024-09-16T15:40:12.882663+0000","last_peered":"2024-09-16T15:40:12.882663+0000","last_clean":"2024-09-16T15:40:12.882663+0000","last_became_active":"2024-09-16T15:40:12.868857+0000","last_became_peered":"2024-09-16T15:40:12.868857+0000","last_unstale":"2024-09-16T15:40:12.882663+0000","last_undegraded":"2024-09-16T15:40:12.882663+0000","last_fullsized":"2024-09-16T15:40:12.882663+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:59.975480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879734+0000","last_change":"2024-09-16T15:40:11.865117+0000","last_active":"2024-09-16T15:40:12.879734+0000","last_peered":"2024-09-16T15:40:12.879734+0000","last_clean":"2024-09-16T15:40:12.879734+0000","last_became_active":"2024-09-16T15:40:11.864766+0000","last_became_peered":"2024-09-16T15:40:11.864766+0000","last_unstale":"2024-09-16T15:40:12.879734+0000","last_undegraded":"2024-09-16T15:40:12.879734+0000","last_fullsized":"2024-09-16T15:40:12.879734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:07.999976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880707+0000","last_change":"2024-09-16T15:40:12.860867+0000","last_active":"2024-09-16T15:40:12.880707+0000","last_peered":"2024-09-16T15:40:12.880707+0000","last_clean":"2024-09-16T15:40:12.880707+0000","last_became_active":"2024-09-16T15:40:12.860648+0000","last_became_peered":"2024-09-16T15:40:12.860648+0000","last_unstale":"2024-09-16T15:40:12.880707+0000","last_undegraded":"2024-09-16T15:40:12.880707+0000","last_fullsized":"2024-09-16T15:40:12.880707+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:49.901150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.844800+0000","last_change":"2024-09-16T15:40:11.868600+0000","last_active":"2024-09-16T15:40:12.844800+0000","last_peered":"2024-09-16T15:40:12.844800+0000","last_clean":"2024-09-16T15:40:12.844800+0000","last_became_active":"2024-09-16T15:40:11.868271+0000","last_became_peered":"2024-09-16T15:40:11.868271+0000","last_unstale":"2024-09-16T15:40:12.844800+0000","last_undegraded":"2024-09-16T15:40:12.844800+0000","last_fullsized":"2024-09-16T15:40:12.844800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:56.197404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870499+0000","last_change":"2024-09-16T15:40:12.864230+0000","last_active":"2024-09-16T15:40:12.870499+0000","last_peered":"2024-09-16T15:40:12.870499+0000","last_clean":"2024-09-16T15:40:12.870499+0000","last_became_active":"2024-09-16T15:40:12.863371+0000","last_became_peered":"2024-09-16T15:40:12.863371+0000","last_unstale":"2024-09-16T15:40:12.870499+0000","last_undegraded":"2024-09-16T15:40:12.870499+0000","last_fullsized":"2024-09-16T15:40:12.870499+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:26.451623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.847205+0000","last_change":"2024-09-16T15:40:11.864603+0000","last_active":"2024-09-16T15:40:12.847205+0000","last_peered":"2024-09-16T15:40:12.847205+0000","last_clean":"2024-09-16T15:40:12.847205+0000","last_became_active":"2024-09-16T15:40:11.864256+0000","last_became_peered":"2024-09-16T15:40:11.864256+0000","last_unstale":"2024-09-16T15:40:12.847205+0000","last_undegraded":"2024-09-16T15:40:12.847205+0000","last_fullsized":"2024-09-16T15:40:12.847205+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:34.607613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875009+0000","last_change":"2024-09-16T15:40:12.873524+0000","last_active":"2024-09-16T15:40:12.875009+0000","last_peered":"2024-09-16T15:40:12.875009+0000","last_clean":"2024-09-16T15:40:12.875009+0000","last_became_active":"2024-09-16T15:40:12.873256+0000","last_became_peered":"2024-09-16T15:40:12.873256+0000","last_unstale":"2024-09-16T15:40:12.875009+0000","last_undegraded":"2024-09-16T15:40:12.875009+0000","last_fullsized":"2024-09-16T15:40:12.875009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:15.479730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.851806+0000","last_change":"2024-09-16T15:40:11.860367+0000","last_active":"2024-09-16T15:40:12.851806+0000","last_peered":"2024-09-16T15:40:12.851806+0000","last_clean":"2024-09-16T15:40:12.851806+0000","last_became_active":"2024-09-16T15:40:11.858703+0000","last_became_peered":"2024-09-16T15:40:11.858703+0000","last_unstale":"2024-09-16T15:40:12.851806+0000","last_undegraded":"2024-09-16T15:40:12.851806+0000","last_fullsized":"2024-09-16T15:40:12.851806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:47.251591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.868881+0000","last_change":"2024-09-16T15:40:12.865158+0000","last_active":"2024-09-16T15:40:12.868881+0000","last_peered":"2024-09-16T15:40:12.868881+0000","last_clean":"2024-09-16T15:40:12.868881+0000","last_became_active":"2024-09-16T15:40:12.864675+0000","last_became_peered":"2024-09-16T15:40:12.864675+0000","last_unstale":"2024-09-16T15:40:12.868881+0000","last_undegraded":"2024-09-16T15:40:12.868881+0000","last_fullsized":"2024-09-16T15:40:12.868881+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.849200+0000","last_change":"2024-09-16T15:40:11.863611+0000","last_active":"2024-09-16T15:40:12.849200+0000","last_peered":"2024-09-16T15:40:12.849200+0000","last_clean":"2024-09-16T15:40:12.849200+0000","last_became_active":"2024-09-16T15:40:11.863346+0000","last_became_peered":"2024-09-16T15:40:11.863346+0000","last_unstale":"2024-09-16T15:40:12.849200+0000","last_undegraded":"2024-09-16T15:40:12.849200+0000","last_fullsized":"2024-09-16T15:40:12.849200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:00.681742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879244+0000","last_change":"2024-09-16T15:40:12.876615+0000","last_active":"2024-09-16T15:40:12.879244+0000","last_peered":"2024-09-16T15:40:12.879244+0000","last_clean":"2024-09-16T15:40:12.879244+0000","last_became_active":"2024-09-16T15:40:12.876385+0000","last_became_peered":"2024-09-16T15:40:12.876385+0000","last_unstale":"2024-09-16T15:40:12.879244+0000","last_undegraded":"2024-09-16T15:40:12.879244+0000","last_fullsized":"2024-09-16T15:40:12.879244+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:35.883048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881290+0000","last_change":"2024-09-16T15:40:11.864331+0000","last_active":"2024-09-16T15:40:12.881290+0000","last_peered":"2024-09-16T15:40:12.881290+0000","last_clean":"2024-09-16T15:40:12.881290+0000","last_became_active":"2024-09-16T15:40:11.864083+0000","last_became_peered":"2024-09-16T15:40:11.864083+0000","last_unstale":"2024-09-16T15:40:12.881290+0000","last_undegraded":"2024-09-16T15:40:12.881290+0000","last_fullsized":"2024-09-16T15:40:12.881290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:44.080258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.868336+0000","last_change":"2024-09-16T15:40:12.865124+0000","last_active":"2024-09-16T15:40:12.868336+0000","last_peered":"2024-09-16T15:40:12.868336+0000","last_clean":"2024-09-16T15:40:12.868336+0000","last_became_active":"2024-09-16T15:40:12.864809+0000","last_became_peered":"2024-09-16T15:40:12.864809+0000","last_unstale":"2024-09-16T15:40:12.868336+0000","last_undegraded":"2024-09-16T15:40:12.868336+0000","last_fullsized":"2024-09-16T15:40:12.868336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.885109+0000","last_change":"2024-09-16T15:40:11.865511+0000","last_active":"2024-09-16T15:40:12.885109+0000","last_peered":"2024-09-16T15:40:12.885109+0000","last_clean":"2024-09-16T15:40:12.885109+0000","last_became_active":"2024-09-16T15:40:11.863715+0000","last_became_peered":"2024-09-16T15:40:11.863715+0000","last_unstale":"2024-09-16T15:40:12.885109+0000","last_undegraded":"2024-09-16T15:40:12.885109+0000","last_fullsized":"2024-09-16T15:40:12.885109+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:18.604944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875276+0000","last_change":"2024-09-16T15:40:12.871040+0000","last_active":"2024-09-16T15:40:12.875276+0000","last_peered":"2024-09-16T15:40:12.875276+0000","last_clean":"2024-09-16T15:40:12.875276+0000","last_became_active":"2024-09-16T15:40:12.870459+0000","last_became_peered":"2024-09-16T15:40:12.870459+0000","last_unstale":"2024-09-16T15:40:12.875276+0000","last_undegraded":"2024-09-16T15:40:12.875276+0000","last_fullsized":"2024-09-16T15:40:12.875276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:53.575639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.884353+0000","last_change":"2024-09-16T15:40:11.870606+0000","last_active":"2024-09-16T15:40:12.884353+0000","last_peered":"2024-09-16T15:40:12.884353+0000","last_clean":"2024-09-16T15:40:12.884353+0000","last_became_active":"2024-09-16T15:40:11.868146+0000","last_became_peered":"2024-09-16T15:40:11.868146+0000","last_unstale":"2024-09-16T15:40:12.884353+0000","last_undegraded":"2024-09-16T15:40:12.884353+0000","last_fullsized":"2024-09-16T15:40:12.884353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:36.745807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880157+0000","last_change":"2024-09-16T15:40:12.860151+0000","last_active":"2024-09-16T15:40:12.880157+0000","last_peered":"2024-09-16T15:40:12.880157+0000","last_clean":"2024-09-16T15:40:12.880157+0000","last_became_active":"2024-09-16T15:40:12.859179+0000","last_became_peered":"2024-09-16T15:40:12.859179+0000","last_unstale":"2024-09-16T15:40:12.880157+0000","last_undegraded":"2024-09-16T15:40:12.880157+0000","last_fullsized":"2024-09-16T15:40:12.880157+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:39.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855830+0000","last_change":"2024-09-16T15:40:11.863101+0000","last_active":"2024-09-16T15:40:12.855830+0000","last_peered":"2024-09-16T15:40:12.855830+0000","last_clean":"2024-09-16T15:40:12.855830+0000","last_became_active":"2024-09-16T15:40:11.862829+0000","last_became_peered":"2024-09-16T15:40:11.862829+0000","last_unstale":"2024-09-16T15:40:12.855830+0000","last_undegraded":"2024-09-16T15:40:12.855830+0000","last_fullsized":"2024-09-16T15:40:12.855830+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:25.527579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.876819+0000","last_change":"2024-09-16T15:40:12.871218+0000","last_active":"2024-09-16T15:40:12.876819+0000","last_peered":"2024-09-16T15:40:12.876819+0000","last_clean":"2024-09-16T15:40:12.876819+0000","last_became_active":"2024-09-16T15:40:12.870997+0000","last_became_peered":"2024-09-16T15:40:12.870997+0000","last_unstale":"2024-09-16T15:40:12.876819+0000","last_undegraded":"2024-09-16T15:40:12.876819+0000","last_fullsized":"2024-09-16T15:40:12.876819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:27.010957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.852972+0000","last_change":"2024-09-16T15:40:11.867251+0000","last_active":"2024-09-16T15:40:12.852972+0000","last_peered":"2024-09-16T15:40:12.852972+0000","last_clean":"2024-09-16T15:40:12.852972+0000","last_became_active":"2024-09-16T15:40:11.866730+0000","last_became_peered":"2024-09-16T15:40:11.866730+0000","last_unstale":"2024-09-16T15:40:12.852972+0000","last_undegraded":"2024-09-16T15:40:12.852972+0000","last_fullsized":"2024-09-16T15:40:12.852972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:55.152872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873844+0000","last_change":"2024-09-16T15:40:12.869762+0000","last_active":"2024-09-16T15:40:12.873844+0000","last_peered":"2024-09-16T15:40:12.873844+0000","last_clean":"2024-09-16T15:40:12.873844+0000","last_became_active":"2024-09-16T15:40:12.869420+0000","last_became_peered":"2024-09-16T15:40:12.869420+0000","last_unstale":"2024-09-16T15:40:12.873844+0000","last_undegraded":"2024-09-16T15:40:12.873844+0000","last_fullsized":"2024-09-16T15:40:12.873844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:51.377166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.888763+0000","last_change":"2024-09-16T15:40:11.868078+0000","last_active":"2024-09-16T15:40:12.888763+0000","last_peered":"2024-09-16T15:40:12.888763+0000","last_clean":"2024-09-16T15:40:12.888763+0000","last_became_active":"2024-09-16T15:40:11.867825+0000","last_became_peered":"2024-09-16T15:40:11.867825+0000","last_unstale":"2024-09-16T15:40:12.888763+0000","last_undegraded":"2024-09-16T15:40:12.888763+0000","last_fullsized":"2024-09-16T15:40:12.888763+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:29.103104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872014+0000","last_change":"2024-09-16T15:40:12.867357+0000","last_active":"2024-09-16T15:40:12.872014+0000","last_peered":"2024-09-16T15:40:12.872014+0000","last_clean":"2024-09-16T15:40:12.872014+0000","last_became_active":"2024-09-16T15:40:12.867072+0000","last_became_peered":"2024-09-16T15:40:12.867072+0000","last_unstale":"2024-09-16T15:40:12.872014+0000","last_undegraded":"2024-09-16T15:40:12.872014+0000","last_fullsized":"2024-09-16T15:40:12.872014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:17.395905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.886756+0000","last_change":"2024-09-16T15:40:11.866281+0000","last_active":"2024-09-16T15:40:12.886756+0000","last_peered":"2024-09-16T15:40:12.886756+0000","last_clean":"2024-09-16T15:40:12.886756+0000","last_became_active":"2024-09-16T15:40:11.865322+0000","last_became_peered":"2024-09-16T15:40:11.865322+0000","last_unstale":"2024-09-16T15:40:12.886756+0000","last_undegraded":"2024-09-16T15:40:12.886756+0000","last_fullsized":"2024-09-16T15:40:12.886756+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:14:13.585151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883782+0000","last_change":"2024-09-16T15:40:12.874698+0000","last_active":"2024-09-16T15:40:12.883782+0000","last_peered":"2024-09-16T15:40:12.883782+0000","last_clean":"2024-09-16T15:40:12.883782+0000","last_became_active":"2024-09-16T15:40:12.874345+0000","last_became_peered":"2024-09-16T15:40:12.874345+0000","last_unstale":"2024-09-16T15:40:12.883782+0000","last_undegraded":"2024-09-16T15:40:12.883782+0000","last_fullsized":"2024-09-16T15:40:12.883782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:49.306539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880542+0000","last_change":"2024-09-16T15:40:11.859504+0000","last_active":"2024-09-16T15:40:12.880542+0000","last_peered":"2024-09-16T15:40:12.880542+0000","last_clean":"2024-09-16T15:40:12.880542+0000","last_became_active":"2024-09-16T15:40:11.859183+0000","last_became_peered":"2024-09-16T15:40:11.859183+0000","last_unstale":"2024-09-16T15:40:12.880542+0000","last_undegraded":"2024-09-16T15:40:12.880542+0000","last_fullsized":"2024-09-16T15:40:12.880542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:57.973947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.863982+0000","last_change":"2024-09-16T15:40:12.856434+0000","last_active":"2024-09-16T15:40:12.863982+0000","last_peered":"2024-09-16T15:40:12.863982+0000","last_clean":"2024-09-16T15:40:12.863982+0000","last_became_active":"2024-09-16T15:40:12.856073+0000","last_became_peered":"2024-09-16T15:40:12.856073+0000","last_unstale":"2024-09-16T15:40:12.863982+0000","last_undegraded":"2024-09-16T15:40:12.863982+0000","last_fullsized":"2024-09-16T15:40:12.863982+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:22.248689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.878307+0000","last_change":"2024-09-16T15:40:11.869221+0000","last_active":"2024-09-16T15:40:12.878307+0000","last_peered":"2024-09-16T15:40:12.878307+0000","last_clean":"2024-09-16T15:40:12.878307+0000","last_became_active":"2024-09-16T15:40:11.868892+0000","last_became_peered":"2024-09-16T15:40:11.868892+0000","last_unstale":"2024-09-16T15:40:12.878307+0000","last_undegraded":"2024-09-16T15:40:12.878307+0000","last_fullsized":"2024-09-16T15:40:12.878307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.865500+0000","last_change":"2024-09-16T15:40:12.859954+0000","last_active":"2024-09-16T15:40:12.865500+0000","last_peered":"2024-09-16T15:40:12.865500+0000","last_clean":"2024-09-16T15:40:12.865500+0000","last_became_active":"2024-09-16T15:40:12.859695+0000","last_became_peered":"2024-09-16T15:40:12.859695+0000","last_unstale":"2024-09-16T15:40:12.865500+0000","last_undegraded":"2024-09-16T15:40:12.865500+0000","last_fullsized":"2024-09-16T15:40:12.865500+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:50.839867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.851789+0000","last_change":"2024-09-16T15:40:11.863949+0000","last_active":"2024-09-16T15:40:12.851789+0000","last_peered":"2024-09-16T15:40:12.851789+0000","last_clean":"2024-09-16T15:40:12.851789+0000","last_became_active":"2024-09-16T15:40:11.863620+0000","last_became_peered":"2024-09-16T15:40:11.863620+0000","last_unstale":"2024-09-16T15:40:12.851789+0000","last_undegraded":"2024-09-16T15:40:12.851789+0000","last_fullsized":"2024-09-16T15:40:12.851789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:18.083324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.877926+0000","last_change":"2024-09-16T15:40:12.873993+0000","last_active":"2024-09-16T15:40:12.877926+0000","last_peered":"2024-09-16T15:40:12.877926+0000","last_clean":"2024-09-16T15:40:12.877926+0000","last_became_active":"2024-09-16T15:40:12.873720+0000","last_became_peered":"2024-09-16T15:40:12.873720+0000","last_unstale":"2024-09-16T15:40:12.877926+0000","last_undegraded":"2024-09-16T15:40:12.877926+0000","last_fullsized":"2024-09-16T15:40:12.877926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:47.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880354+0000","last_change":"2024-09-16T15:40:11.871266+0000","last_active":"2024-09-16T15:40:12.880354+0000","last_peered":"2024-09-16T15:40:12.880354+0000","last_clean":"2024-09-16T15:40:12.880354+0000","last_became_active":"2024-09-16T15:40:11.871013+0000","last_became_peered":"2024-09-16T15:40:11.871013+0000","last_unstale":"2024-09-16T15:40:12.880354+0000","last_undegraded":"2024-09-16T15:40:12.880354+0000","last_fullsized":"2024-09-16T15:40:12.880354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:10.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:40:11.850702+0000","last_change":"2024-09-16T15:40:11.850702+0000","last_active":"2024-09-16T15:40:11.835623+0000","last_peered":"2024-09-16T15:40:11.835623+0000","last_clean":"2024-09-16T15:40:11.835623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.850702+0000","last_undegraded":"2024-09-16T15:40:11.850702+0000","last_fullsized":"2024-09-16T15:40:11.850702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.845323+0000","last_change":"2024-09-16T15:40:11.865942+0000","last_active":"2024-09-16T15:40:12.845323+0000","last_peered":"2024-09-16T15:40:12.845323+0000","last_clean":"2024-09-16T15:40:12.845323+0000","last_became_active":"2024-09-16T15:40:11.864112+0000","last_became_peered":"2024-09-16T15:40:11.864112+0000","last_unstale":"2024-09-16T15:40:12.845323+0000","last_undegraded":"2024-09-16T15:40:12.845323+0000","last_fullsized":"2024-09-16T15:40:12.845323+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:13.011662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867632+0000","last_change":"2024-09-16T15:40:12.861166+0000","last_active":"2024-09-16T15:40:12.867632+0000","last_peered":"2024-09-16T15:40:12.867632+0000","last_clean":"2024-09-16T15:40:12.867632+0000","last_became_active":"2024-09-16T15:40:12.860619+0000","last_became_peered":"2024-09-16T15:40:12.860619+0000","last_unstale":"2024-09-16T15:40:12.867632+0000","last_undegraded":"2024-09-16T15:40:12.867632+0000","last_fullsized":"2024-09-16T15:40:12.867632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:54.024305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.862595+0000","last_change":"2024-09-16T15:40:11.862288+0000","last_active":"2024-09-16T15:40:12.862595+0000","last_peered":"2024-09-16T15:40:12.862595+0000","last_clean":"2024-09-16T15:40:12.862595+0000","last_became_active":"2024-09-16T15:40:11.862002+0000","last_became_peered":"2024-09-16T15:40:11.862002+0000","last_unstale":"2024-09-16T15:40:12.862595+0000","last_undegraded":"2024-09-16T15:40:12.862595+0000","last_fullsized":"2024-09-16T15:40:12.862595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:12.191215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.884800+0000","last_change":"2024-09-16T15:40:12.879430+0000","last_active":"2024-09-16T15:40:12.884800+0000","last_peered":"2024-09-16T15:40:12.884800+0000","last_clean":"2024-09-16T15:40:12.884800+0000","last_became_active":"2024-09-16T15:40:12.879172+0000","last_became_peered":"2024-09-16T15:40:12.879172+0000","last_unstale":"2024-09-16T15:40:12.884800+0000","last_undegraded":"2024-09-16T15:40:12.884800+0000","last_fullsized":"2024-09-16T15:40:12.884800+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:14.019085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.854691+0000","last_change":"2024-09-16T15:40:11.863062+0000","last_active":"2024-09-16T15:40:12.854691+0000","last_peered":"2024-09-16T15:40:12.854691+0000","last_clean":"2024-09-16T15:40:12.854691+0000","last_became_active":"2024-09-16T15:40:11.860714+0000","last_became_peered":"2024-09-16T15:40:11.860714+0000","last_unstale":"2024-09-16T15:40:12.854691+0000","last_undegraded":"2024-09-16T15:40:12.854691+0000","last_fullsized":"2024-09-16T15:40:12.854691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883782+0000","last_change":"2024-09-16T15:40:12.883082+0000","last_active":"2024-09-16T15:40:12.883782+0000","last_peered":"2024-09-16T15:40:12.883782+0000","last_clean":"2024-09-16T15:40:12.883782+0000","last_became_active":"2024-09-16T15:40:12.882695+0000","last_became_peered":"2024-09-16T15:40:12.882695+0000","last_unstale":"2024-09-16T15:40:12.883782+0000","last_undegraded":"2024-09-16T15:40:12.883782+0000","last_fullsized":"2024-09-16T15:40:12.883782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:21.231163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.853585+0000","last_change":"2024-09-16T15:40:11.867238+0000","last_active":"2024-09-16T15:40:12.853585+0000","last_peered":"2024-09-16T15:40:12.853585+0000","last_clean":"2024-09-16T15:40:12.853585+0000","last_became_active":"2024-09-16T15:40:11.865996+0000","last_became_peered":"2024-09-16T15:40:11.865996+0000","last_unstale":"2024-09-16T15:40:12.853585+0000","last_undegraded":"2024-09-16T15:40:12.853585+0000","last_fullsized":"2024-09-16T15:40:12.853585+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:23.953426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883245+0000","last_change":"2024-09-16T15:40:12.872931+0000","last_active":"2024-09-16T15:40:12.883245+0000","last_peered":"2024-09-16T15:40:12.883245+0000","last_clean":"2024-09-16T15:40:12.883245+0000","last_became_active":"2024-09-16T15:40:12.872344+0000","last_became_peered":"2024-09-16T15:40:12.872344+0000","last_unstale":"2024-09-16T15:40:12.883245+0000","last_undegraded":"2024-09-16T15:40:12.883245+0000","last_fullsized":"2024-09-16T15:40:12.883245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:28.977918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855957+0000","last_change":"2024-09-16T15:40:11.865216+0000","last_active":"2024-09-16T15:40:12.855957+0000","last_peered":"2024-09-16T15:40:12.855957+0000","last_clean":"2024-09-16T15:40:12.855957+0000","last_became_active":"2024-09-16T15:40:11.864945+0000","last_became_peered":"2024-09-16T15:40:11.864945+0000","last_unstale":"2024-09-16T15:40:12.855957+0000","last_undegraded":"2024-09-16T15:40:12.855957+0000","last_fullsized":"2024-09-16T15:40:12.855957+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:13.850144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873339+0000","last_change":"2024-09-16T15:40:12.868439+0000","last_active":"2024-09-16T15:40:12.873339+0000","last_peered":"2024-09-16T15:40:12.873339+0000","last_clean":"2024-09-16T15:40:12.873339+0000","last_became_active":"2024-09-16T15:40:12.867651+0000","last_became_peered":"2024-09-16T15:40:12.867651+0000","last_unstale":"2024-09-16T15:40:12.873339+0000","last_undegraded":"2024-09-16T15:40:12.873339+0000","last_fullsized":"2024-09-16T15:40:12.873339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:12.149041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:40:11.871863+0000","last_change":"2024-09-16T15:40:11.869272+0000","last_active":"2024-09-16T15:40:11.871863+0000","last_peered":"2024-09-16T15:40:11.871863+0000","last_clean":"2024-09-16T15:40:11.871863+0000","last_became_active":"2024-09-16T15:40:11.868909+0000","last_became_peered":"2024-09-16T15:40:11.868909+0000","last_unstale":"2024-09-16T15:40:11.871863+0000","last_undegraded":"2024-09-16T15:40:11.871863+0000","last_fullsized":"2024-09-16T15:40:11.871863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:31.920861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874248+0000","last_change":"2024-09-16T15:40:12.862103+0000","last_active":"2024-09-16T15:40:12.874248+0000","last_peered":"2024-09-16T15:40:12.874248+0000","last_clean":"2024-09-16T15:40:12.874248+0000","last_became_active":"2024-09-16T15:40:12.861788+0000","last_became_peered":"2024-09-16T15:40:12.861788+0000","last_unstale":"2024-09-16T15:40:12.874248+0000","last_undegraded":"2024-09-16T15:40:12.874248+0000","last_fullsized":"2024-09-16T15:40:12.874248+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:38.051709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855069+0000","last_change":"2024-09-16T15:40:11.867570+0000","last_active":"2024-09-16T15:40:12.855069+0000","last_peered":"2024-09-16T15:40:12.855069+0000","last_clean":"2024-09-16T15:40:12.855069+0000","last_became_active":"2024-09-16T15:40:11.867301+0000","last_became_peered":"2024-09-16T15:40:11.867301+0000","last_unstale":"2024-09-16T15:40:12.855069+0000","last_undegraded":"2024-09-16T15:40:12.855069+0000","last_fullsized":"2024-09-16T15:40:12.855069+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879331+0000","last_change":"2024-09-16T15:40:12.875892+0000","last_active":"2024-09-16T15:40:12.879331+0000","last_peered":"2024-09-16T15:40:12.879331+0000","last_clean":"2024-09-16T15:40:12.879331+0000","last_became_active":"2024-09-16T15:40:12.875619+0000","last_became_peered":"2024-09-16T15:40:12.875619+0000","last_unstale":"2024-09-16T15:40:12.879331+0000","last_undegraded":"2024-09-16T15:40:12.879331+0000","last_fullsized":"2024-09-16T15:40:12.879331+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:15.128749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.849889+0000","last_change":"2024-09-16T15:40:11.867392+0000","last_active":"2024-09-16T15:40:12.849889+0000","last_peered":"2024-09-16T15:40:12.849889+0000","last_clean":"2024-09-16T15:40:12.849889+0000","last_became_active":"2024-09-16T15:40:11.867003+0000","last_became_peered":"2024-09-16T15:40:11.867003+0000","last_unstale":"2024-09-16T15:40:12.849889+0000","last_undegraded":"2024-09-16T15:40:12.849889+0000","last_fullsized":"2024-09-16T15:40:12.849889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:00.706403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.857550+0000","last_change":"2024-09-16T15:40:12.850142+0000","last_active":"2024-09-16T15:40:12.857550+0000","last_peered":"2024-09-16T15:40:12.857550+0000","last_clean":"2024-09-16T15:40:12.857550+0000","last_became_active":"2024-09-16T15:40:12.849859+0000","last_became_peered":"2024-09-16T15:40:12.849859+0000","last_unstale":"2024-09-16T15:40:12.857550+0000","last_undegraded":"2024-09-16T15:40:12.857550+0000","last_fullsized":"2024-09-16T15:40:12.857550+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:45.087681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:40:11.863988+0000","last_change":"2024-09-16T15:40:11.860188+0000","last_active":"2024-09-16T15:40:11.863988+0000","last_peered":"2024-09-16T15:40:11.863988+0000","last_clean":"2024-09-16T15:40:11.863988+0000","last_became_active":"2024-09-16T15:40:11.859924+0000","last_became_peered":"2024-09-16T15:40:11.859924+0000","last_unstale":"2024-09-16T15:40:11.863988+0000","last_undegraded":"2024-09-16T15:40:11.863988+0000","last_fullsized":"2024-09-16T15:40:11.863988+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:30.152062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867386+0000","last_change":"2024-09-16T15:40:12.862284+0000","last_active":"2024-09-16T15:40:12.867386+0000","last_peered":"2024-09-16T15:40:12.867386+0000","last_clean":"2024-09-16T15:40:12.867386+0000","last_became_active":"2024-09-16T15:40:12.861890+0000","last_became_peered":"2024-09-16T15:40:12.861890+0000","last_unstale":"2024-09-16T15:40:12.867386+0000","last_undegraded":"2024-09-16T15:40:12.867386+0000","last_fullsized":"2024-09-16T15:40:12.867386+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:52.757331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.847519+0000","last_change":"2024-09-16T15:40:11.869696+0000","last_active":"2024-09-16T15:40:12.847519+0000","last_peered":"2024-09-16T15:40:12.847519+0000","last_clean":"2024-09-16T15:40:12.847519+0000","last_became_active":"2024-09-16T15:40:11.869403+0000","last_became_peered":"2024-09-16T15:40:11.869403+0000","last_unstale":"2024-09-16T15:40:12.847519+0000","last_undegraded":"2024-09-16T15:40:12.847519+0000","last_fullsized":"2024-09-16T15:40:12.847519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:53.785910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871439+0000","last_change":"2024-09-16T15:40:12.859022+0000","last_active":"2024-09-16T15:40:12.871439+0000","last_peered":"2024-09-16T15:40:12.871439+0000","last_clean":"2024-09-16T15:40:12.871439+0000","last_became_active":"2024-09-16T15:40:12.858683+0000","last_became_peered":"2024-09-16T15:40:12.858683+0000","last_unstale":"2024-09-16T15:40:12.871439+0000","last_undegraded":"2024-09-16T15:40:12.871439+0000","last_fullsized":"2024-09-16T15:40:12.871439+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:46.347083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881285+0000","last_change":"2024-09-16T15:40:11.872583+0000","last_active":"2024-09-16T15:40:12.881285+0000","last_peered":"2024-09-16T15:40:12.881285+0000","last_clean":"2024-09-16T15:40:12.881285+0000","last_became_active":"2024-09-16T15:40:11.872290+0000","last_became_peered":"2024-09-16T15:40:11.872290+0000","last_unstale":"2024-09-16T15:40:12.881285+0000","last_undegraded":"2024-09-16T15:40:12.881285+0000","last_fullsized":"2024-09-16T15:40:12.881285+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.854788+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:40:12.854788+0000","last_peered":"2024-09-16T15:40:12.854788+0000","last_clean":"2024-09-16T15:40:12.854788+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:40:12.854788+0000","last_undegraded":"2024-09-16T15:40:12.854788+0000","last_fullsized":"2024-09-16T15:40:12.854788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1126400,"data_stored":802873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":548864,"data_stored":219862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":219350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":802873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":520192,"data_stored":206238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:17.106 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:17.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.301+0000 7fd8c3e00640 1 -- 172.21.15.60:0/3947575423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c40af040 msgr2=0x7fd8c406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.301+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/3947575423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c40af040 0x7fd8c406f770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005fd0 tx=0x7fd8bc02aa90 comp rx=0 tx=0).stop 2024-09-16T15:40:17.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.301+0000 7fd8c3e00640 1 -- 172.21.15.60:0/3947575423 shutdown_connections 2024-09-16T15:40:17.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.301+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/3947575423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c40af040 0x7fd8c406f770 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.301+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/3947575423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c415a620 0x7fd8c40cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.301+0000 7fd8c3e00640 1 -- 172.21.15.60:0/3947575423 >> 172.21.15.60:0/3947575423 conn(0x7fd8c40cfa20 msgr2=0x7fd8c40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.301+0000 7fd8c3e00640 1 -- 172.21.15.60:0/3947575423 shutdown_connections 2024-09-16T15:40:17.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.301+0000 7fd8c3e00640 1 -- 172.21.15.60:0/3947575423 wait complete. 2024-09-16T15:40:17.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.303+0000 7fd8c3e00640 1 Processor -- start 2024-09-16T15:40:17.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.303+0000 7fd8c3e00640 1 -- start start 2024-09-16T15:40:17.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c40aec20 0x7fd8c414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c40af040 0x7fd8c414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c415a620 0x7fd8c414f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c406fcb0 con 0x7fd8c415a620 2024-09-16T15:40:17.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c4070950 con 0x7fd8c40af040 2024-09-16T15:40:17.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c406fe30 con 0x7fd8c40aec20 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c415a620 0x7fd8c414f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c415a620 0x7fd8c414f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37440/0 (socket says 172.21.15.60:37440) 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c3400640 1 -- 172.21.15.60:0/254033615 learned_addr learned my addr 172.21.15.60:0/254033615 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8c2a00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c40aec20 0x7fd8c414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.304+0000 7fd8ba000640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c40af040 0x7fd8c414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.305+0000 7fd8c3400640 1 -- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c40aec20 msgr2=0x7fd8c414e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.305+0000 7fd8c3400640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c40aec20 0x7fd8c414e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.305+0000 7fd8c3400640 1 -- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c40af040 msgr2=0x7fd8c414f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.305+0000 7fd8c3400640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c40af040 0x7fd8c414f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.305+0000 7fd8c3400640 1 -- 172.21.15.60:0/254033615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c414e880 con 0x7fd8c415a620 2024-09-16T15:40:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.305+0000 7fd8c3400640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c415a620 0x7fd8c414f620 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc006010 tx=0x7fd8bc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.306+0000 7fd8bb400640 1 -- 172.21.15.60:0/254033615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc046070 con 0x7fd8c415a620 2024-09-16T15:40:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.306+0000 7fd8bb400640 1 -- 172.21.15.60:0/254033615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8bc003e50 con 0x7fd8c415a620 2024-09-16T15:40:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.306+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c4072570 con 0x7fd8c415a620 2024-09-16T15:40:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.306+0000 7fd8bb400640 1 -- 172.21.15.60:0/254033615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0044b0 con 0x7fd8c415a620 2024-09-16T15:40:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.306+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c4072af0 con 0x7fd8c415a620 2024-09-16T15:40:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.307+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c406f770 con 0x7fd8c415a620 2024-09-16T15:40:17.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.308+0000 7fd8bb400640 1 -- 172.21.15.60:0/254033615 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fd8bc049070 con 0x7fd8c415a620 2024-09-16T15:40:17.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.309+0000 7fd8bb400640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a0078a30 0x7fd8a007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.309+0000 7fd8bb400640 1 -- 172.21.15.60:0/254033615 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fd8bc0bbb00 con 0x7fd8c415a620 2024-09-16T15:40:17.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.310+0000 7fd8c2a00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a0078a30 0x7fd8a007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.310+0000 7fd8c2a00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a0078a30 0x7fd8a007aef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4004630 tx=0x7fd8b400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.313+0000 7fd8bb400640 1 -- 172.21.15.60:0/254033615 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8c414e880 con 0x7fd8c415a620 2024-09-16T15:40:17.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.507+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8c40cc020 con 0x7fd8a0078a30 2024-09-16T15:40:17.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.512+0000 7fd8bb400640 1 -- 172.21.15.60:0/254033615 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251694 (secure 0 0 0) 0x7fd8c40cc020 con 0x7fd8a0078a30 2024-09-16T15:40:17.516 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:17.518 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.516+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a0078a30 msgr2=0x7fd8a007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.516+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a0078a30 0x7fd8a007aef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4004630 tx=0x7fd8b400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.516+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c415a620 msgr2=0x7fd8c414f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.516+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c415a620 0x7fd8c414f620 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc006010 tx=0x7fd8bc003ce0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.517+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 shutdown_connections 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.517+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8a0078a30 0x7fd8a007aef0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.517+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c415a620 0x7fd8c414f620 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.517+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c40af040 0x7fd8c414f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.517+0000 7fd8c3e00640 1 --2- 172.21.15.60:0/254033615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c40aec20 0x7fd8c414e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.517+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 >> 172.21.15.60:0/254033615 conn(0x7fd8c40cfa20 msgr2=0x7fd8c40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.517+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 shutdown_connections 2024-09-16T15:40:17.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.517+0000 7fd8c3e00640 1 -- 172.21.15.60:0/254033615 wait complete. 2024-09-16T15:40:17.534 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-16T15:40:15.733953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238240,"kb_used_data":5340,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736480,"statfs":{"total":773094113280,"available":772850155520,"internally_reserved":0,"allocated":5468160,"data_stored":2888711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.893876"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872674+0000","last_change":"2024-09-16T15:40:12.868607+0000","last_active":"2024-09-16T15:40:12.872674+0000","last_peered":"2024-09-16T15:40:12.872674+0000","last_clean":"2024-09-16T15:40:12.872674+0000","last_became_active":"2024-09-16T15:40:12.868140+0000","last_became_peered":"2024-09-16T15:40:12.868140+0000","last_unstale":"2024-09-16T15:40:12.872674+0000","last_undegraded":"2024-09-16T15:40:12.872674+0000","last_fullsized":"2024-09-16T15:40:12.872674+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:29.669859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872203+0000","last_change":"2024-09-16T15:40:12.869358+0000","last_active":"2024-09-16T15:40:12.872203+0000","last_peered":"2024-09-16T15:40:12.872203+0000","last_clean":"2024-09-16T15:40:12.872203+0000","last_became_active":"2024-09-16T15:40:12.868998+0000","last_became_peered":"2024-09-16T15:40:12.868998+0000","last_unstale":"2024-09-16T15:40:12.872203+0000","last_undegraded":"2024-09-16T15:40:12.872203+0000","last_fullsized":"2024-09-16T15:40:12.872203+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874339+0000","last_change":"2024-09-16T15:40:12.868681+0000","last_active":"2024-09-16T15:40:12.874339+0000","last_peered":"2024-09-16T15:40:12.874339+0000","last_clean":"2024-09-16T15:40:12.874339+0000","last_became_active":"2024-09-16T15:40:12.867980+0000","last_became_peered":"2024-09-16T15:40:12.867980+0000","last_unstale":"2024-09-16T15:40:12.874339+0000","last_undegraded":"2024-09-16T15:40:12.874339+0000","last_fullsized":"2024-09-16T15:40:12.874339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:39.160061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883276+0000","last_change":"2024-09-16T15:40:12.873202+0000","last_active":"2024-09-16T15:40:12.883276+0000","last_peered":"2024-09-16T15:40:12.883276+0000","last_clean":"2024-09-16T15:40:12.883276+0000","last_became_active":"2024-09-16T15:40:12.872682+0000","last_became_peered":"2024-09-16T15:40:12.872682+0000","last_unstale":"2024-09-16T15:40:12.883276+0000","last_undegraded":"2024-09-16T15:40:12.883276+0000","last_fullsized":"2024-09-16T15:40:12.883276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:23.455972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874812+0000","last_change":"2024-09-16T15:40:12.870112+0000","last_active":"2024-09-16T15:40:12.874812+0000","last_peered":"2024-09-16T15:40:12.874812+0000","last_clean":"2024-09-16T15:40:12.874812+0000","last_became_active":"2024-09-16T15:40:12.869753+0000","last_became_peered":"2024-09-16T15:40:12.869753+0000","last_unstale":"2024-09-16T15:40:12.874812+0000","last_undegraded":"2024-09-16T15:40:12.874812+0000","last_fullsized":"2024-09-16T15:40:12.874812+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875406+0000","last_change":"2024-09-16T15:40:12.866164+0000","last_active":"2024-09-16T15:40:12.875406+0000","last_peered":"2024-09-16T15:40:12.875406+0000","last_clean":"2024-09-16T15:40:12.875406+0000","last_became_active":"2024-09-16T15:40:12.865904+0000","last_became_peered":"2024-09-16T15:40:12.865904+0000","last_unstale":"2024-09-16T15:40:12.875406+0000","last_undegraded":"2024-09-16T15:40:12.875406+0000","last_fullsized":"2024-09-16T15:40:12.875406+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:44.336056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.869132+0000","last_change":"2024-09-16T15:40:12.865180+0000","last_active":"2024-09-16T15:40:12.869132+0000","last_peered":"2024-09-16T15:40:12.869132+0000","last_clean":"2024-09-16T15:40:12.869132+0000","last_became_active":"2024-09-16T15:40:12.864842+0000","last_became_peered":"2024-09-16T15:40:12.864842+0000","last_unstale":"2024-09-16T15:40:12.869132+0000","last_undegraded":"2024-09-16T15:40:12.869132+0000","last_fullsized":"2024-09-16T15:40:12.869132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:36.820991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880732+0000","last_change":"2024-09-16T15:40:12.861554+0000","last_active":"2024-09-16T15:40:12.880732+0000","last_peered":"2024-09-16T15:40:12.880732+0000","last_clean":"2024-09-16T15:40:12.880732+0000","last_became_active":"2024-09-16T15:40:12.861215+0000","last_became_peered":"2024-09-16T15:40:12.861215+0000","last_unstale":"2024-09-16T15:40:12.880732+0000","last_undegraded":"2024-09-16T15:40:12.880732+0000","last_fullsized":"2024-09-16T15:40:12.880732+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:16.169482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882156+0000","last_change":"2024-09-16T15:40:12.864597+0000","last_active":"2024-09-16T15:40:12.882156+0000","last_peered":"2024-09-16T15:40:12.882156+0000","last_clean":"2024-09-16T15:40:12.882156+0000","last_became_active":"2024-09-16T15:40:12.864345+0000","last_became_peered":"2024-09-16T15:40:12.864345+0000","last_unstale":"2024-09-16T15:40:12.882156+0000","last_undegraded":"2024-09-16T15:40:12.882156+0000","last_fullsized":"2024-09-16T15:40:12.882156+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:43.902508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882148+0000","last_change":"2024-09-16T15:40:12.865537+0000","last_active":"2024-09-16T15:40:12.882148+0000","last_peered":"2024-09-16T15:40:12.882148+0000","last_clean":"2024-09-16T15:40:12.882148+0000","last_became_active":"2024-09-16T15:40:12.865293+0000","last_became_peered":"2024-09-16T15:40:12.865293+0000","last_unstale":"2024-09-16T15:40:12.882148+0000","last_undegraded":"2024-09-16T15:40:12.882148+0000","last_fullsized":"2024-09-16T15:40:12.882148+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:15.882606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883443+0000","last_change":"2024-09-16T15:40:12.875755+0000","last_active":"2024-09-16T15:40:12.883443+0000","last_peered":"2024-09-16T15:40:12.883443+0000","last_clean":"2024-09-16T15:40:12.883443+0000","last_became_active":"2024-09-16T15:40:12.875484+0000","last_became_peered":"2024-09-16T15:40:12.875484+0000","last_unstale":"2024-09-16T15:40:12.883443+0000","last_undegraded":"2024-09-16T15:40:12.883443+0000","last_fullsized":"2024-09-16T15:40:12.883443+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:57.204826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870792+0000","last_change":"2024-09-16T15:40:12.866408+0000","last_active":"2024-09-16T15:40:12.870792+0000","last_peered":"2024-09-16T15:40:12.870792+0000","last_clean":"2024-09-16T15:40:12.870792+0000","last_became_active":"2024-09-16T15:40:12.865787+0000","last_became_peered":"2024-09-16T15:40:12.865787+0000","last_unstale":"2024-09-16T15:40:12.870792+0000","last_undegraded":"2024-09-16T15:40:12.870792+0000","last_fullsized":"2024-09-16T15:40:12.870792+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:39.299743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867785+0000","last_change":"2024-09-16T15:40:12.865889+0000","last_active":"2024-09-16T15:40:12.867785+0000","last_peered":"2024-09-16T15:40:12.867785+0000","last_clean":"2024-09-16T15:40:12.867785+0000","last_became_active":"2024-09-16T15:40:12.865611+0000","last_became_peered":"2024-09-16T15:40:12.865611+0000","last_unstale":"2024-09-16T15:40:12.867785+0000","last_undegraded":"2024-09-16T15:40:12.867785+0000","last_fullsized":"2024-09-16T15:40:12.867785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:28.736859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881374+0000","last_change":"2024-09-16T15:40:12.863378+0000","last_active":"2024-09-16T15:40:12.881374+0000","last_peered":"2024-09-16T15:40:12.881374+0000","last_clean":"2024-09-16T15:40:12.881374+0000","last_became_active":"2024-09-16T15:40:12.863108+0000","last_became_peered":"2024-09-16T15:40:12.863108+0000","last_unstale":"2024-09-16T15:40:12.881374+0000","last_undegraded":"2024-09-16T15:40:12.881374+0000","last_fullsized":"2024-09-16T15:40:12.881374+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:34.731306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873670+0000","last_change":"2024-09-16T15:40:12.861959+0000","last_active":"2024-09-16T15:40:12.873670+0000","last_peered":"2024-09-16T15:40:12.873670+0000","last_clean":"2024-09-16T15:40:12.873670+0000","last_became_active":"2024-09-16T15:40:12.861657+0000","last_became_peered":"2024-09-16T15:40:12.861657+0000","last_unstale":"2024-09-16T15:40:12.873670+0000","last_undegraded":"2024-09-16T15:40:12.873670+0000","last_fullsized":"2024-09-16T15:40:12.873670+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:55.582229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871103+0000","last_change":"2024-09-16T15:40:12.865876+0000","last_active":"2024-09-16T15:40:12.871103+0000","last_peered":"2024-09-16T15:40:12.871103+0000","last_clean":"2024-09-16T15:40:12.871103+0000","last_became_active":"2024-09-16T15:40:12.865493+0000","last_became_peered":"2024-09-16T15:40:12.865493+0000","last_unstale":"2024-09-16T15:40:12.871103+0000","last_undegraded":"2024-09-16T15:40:12.871103+0000","last_fullsized":"2024-09-16T15:40:12.871103+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:44.257723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867152+0000","last_change":"2024-09-16T15:40:12.861173+0000","last_active":"2024-09-16T15:40:12.867152+0000","last_peered":"2024-09-16T15:40:12.867152+0000","last_clean":"2024-09-16T15:40:12.867152+0000","last_became_active":"2024-09-16T15:40:12.860306+0000","last_became_peered":"2024-09-16T15:40:12.860306+0000","last_unstale":"2024-09-16T15:40:12.867152+0000","last_undegraded":"2024-09-16T15:40:12.867152+0000","last_fullsized":"2024-09-16T15:40:12.867152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:38.251940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875035+0000","last_change":"2024-09-16T15:40:12.873790+0000","last_active":"2024-09-16T15:40:12.875035+0000","last_peered":"2024-09-16T15:40:12.875035+0000","last_clean":"2024-09-16T15:40:12.875035+0000","last_became_active":"2024-09-16T15:40:12.873524+0000","last_became_peered":"2024-09-16T15:40:12.873524+0000","last_unstale":"2024-09-16T15:40:12.875035+0000","last_undegraded":"2024-09-16T15:40:12.875035+0000","last_fullsized":"2024-09-16T15:40:12.875035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:48.530545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880065+0000","last_change":"2024-09-16T15:40:12.872623+0000","last_active":"2024-09-16T15:40:12.880065+0000","last_peered":"2024-09-16T15:40:12.880065+0000","last_clean":"2024-09-16T15:40:12.880065+0000","last_became_active":"2024-09-16T15:40:12.872346+0000","last_became_peered":"2024-09-16T15:40:12.872346+0000","last_unstale":"2024-09-16T15:40:12.880065+0000","last_undegraded":"2024-09-16T15:40:12.880065+0000","last_fullsized":"2024-09-16T15:40:12.880065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:45.262493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.885065+0000","last_change":"2024-09-16T15:40:12.882504+0000","last_active":"2024-09-16T15:40:12.885065+0000","last_peered":"2024-09-16T15:40:12.885065+0000","last_clean":"2024-09-16T15:40:12.885065+0000","last_became_active":"2024-09-16T15:40:12.882113+0000","last_became_peered":"2024-09-16T15:40:12.882113+0000","last_unstale":"2024-09-16T15:40:12.885065+0000","last_undegraded":"2024-09-16T15:40:12.885065+0000","last_fullsized":"2024-09-16T15:40:12.885065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:35.919623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.869373+0000","last_change":"2024-09-16T15:40:12.857122+0000","last_active":"2024-09-16T15:40:12.869373+0000","last_peered":"2024-09-16T15:40:12.869373+0000","last_clean":"2024-09-16T15:40:12.869373+0000","last_became_active":"2024-09-16T15:40:12.856815+0000","last_became_peered":"2024-09-16T15:40:12.856815+0000","last_unstale":"2024-09-16T15:40:12.869373+0000","last_undegraded":"2024-09-16T15:40:12.869373+0000","last_fullsized":"2024-09-16T15:40:12.869373+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:41.290208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882619+0000","last_change":"2024-09-16T15:40:12.868687+0000","last_active":"2024-09-16T15:40:12.882619+0000","last_peered":"2024-09-16T15:40:12.882619+0000","last_clean":"2024-09-16T15:40:12.882619+0000","last_became_active":"2024-09-16T15:40:12.868063+0000","last_became_peered":"2024-09-16T15:40:12.868063+0000","last_unstale":"2024-09-16T15:40:12.882619+0000","last_undegraded":"2024-09-16T15:40:12.882619+0000","last_fullsized":"2024-09-16T15:40:12.882619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:53.819296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870525+0000","last_change":"2024-09-16T15:40:12.864426+0000","last_active":"2024-09-16T15:40:12.870525+0000","last_peered":"2024-09-16T15:40:12.870525+0000","last_clean":"2024-09-16T15:40:12.870525+0000","last_became_active":"2024-09-16T15:40:12.863762+0000","last_became_peered":"2024-09-16T15:40:12.863762+0000","last_unstale":"2024-09-16T15:40:12.870525+0000","last_undegraded":"2024-09-16T15:40:12.870525+0000","last_fullsized":"2024-09-16T15:40:12.870525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:38.955827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871085+0000","last_change":"2024-09-16T15:40:12.863693+0000","last_active":"2024-09-16T15:40:12.871085+0000","last_peered":"2024-09-16T15:40:12.871085+0000","last_clean":"2024-09-16T15:40:12.871085+0000","last_became_active":"2024-09-16T15:40:12.863383+0000","last_became_peered":"2024-09-16T15:40:12.863383+0000","last_unstale":"2024-09-16T15:40:12.871085+0000","last_undegraded":"2024-09-16T15:40:12.871085+0000","last_fullsized":"2024-09-16T15:40:12.871085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:44.940019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.863957+0000","last_change":"2024-09-16T15:40:12.856737+0000","last_active":"2024-09-16T15:40:12.863957+0000","last_peered":"2024-09-16T15:40:12.863957+0000","last_clean":"2024-09-16T15:40:12.863957+0000","last_became_active":"2024-09-16T15:40:12.856426+0000","last_became_peered":"2024-09-16T15:40:12.856426+0000","last_unstale":"2024-09-16T15:40:12.863957+0000","last_undegraded":"2024-09-16T15:40:12.863957+0000","last_fullsized":"2024-09-16T15:40:12.863957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:58.240556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879492+0000","last_change":"2024-09-16T15:40:12.876499+0000","last_active":"2024-09-16T15:40:12.879492+0000","last_peered":"2024-09-16T15:40:12.879492+0000","last_clean":"2024-09-16T15:40:12.879492+0000","last_became_active":"2024-09-16T15:40:12.876247+0000","last_became_peered":"2024-09-16T15:40:12.876247+0000","last_unstale":"2024-09-16T15:40:12.879492+0000","last_undegraded":"2024-09-16T15:40:12.879492+0000","last_fullsized":"2024-09-16T15:40:12.879492+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:42:34.521097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867366+0000","last_change":"2024-09-16T15:40:12.858149+0000","last_active":"2024-09-16T15:40:12.867366+0000","last_peered":"2024-09-16T15:40:12.867366+0000","last_clean":"2024-09-16T15:40:12.867366+0000","last_became_active":"2024-09-16T15:40:12.857902+0000","last_became_peered":"2024-09-16T15:40:12.857902+0000","last_unstale":"2024-09-16T15:40:12.867366+0000","last_undegraded":"2024-09-16T15:40:12.867366+0000","last_fullsized":"2024-09-16T15:40:12.867366+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:04.893938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870724+0000","last_change":"2024-09-16T15:40:12.865956+0000","last_active":"2024-09-16T15:40:12.870724+0000","last_peered":"2024-09-16T15:40:12.870724+0000","last_clean":"2024-09-16T15:40:12.870724+0000","last_became_active":"2024-09-16T15:40:12.865681+0000","last_became_peered":"2024-09-16T15:40:12.865681+0000","last_unstale":"2024-09-16T15:40:12.870724+0000","last_undegraded":"2024-09-16T15:40:12.870724+0000","last_fullsized":"2024-09-16T15:40:12.870724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:14.857566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.864031+0000","last_change":"2024-09-16T15:40:12.856711+0000","last_active":"2024-09-16T15:40:12.864031+0000","last_peered":"2024-09-16T15:40:12.864031+0000","last_clean":"2024-09-16T15:40:12.864031+0000","last_became_active":"2024-09-16T15:40:12.856290+0000","last_became_peered":"2024-09-16T15:40:12.856290+0000","last_unstale":"2024-09-16T15:40:12.864031+0000","last_undegraded":"2024-09-16T15:40:12.864031+0000","last_fullsized":"2024-09-16T15:40:12.864031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:27.797378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867021+0000","last_change":"2024-09-16T15:40:12.859686+0000","last_active":"2024-09-16T15:40:12.867021+0000","last_peered":"2024-09-16T15:40:12.867021+0000","last_clean":"2024-09-16T15:40:12.867021+0000","last_became_active":"2024-09-16T15:40:12.859399+0000","last_became_peered":"2024-09-16T15:40:12.859399+0000","last_unstale":"2024-09-16T15:40:12.867021+0000","last_undegraded":"2024-09-16T15:40:12.867021+0000","last_fullsized":"2024-09-16T15:40:12.867021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:45.345188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867284+0000","last_change":"2024-09-16T15:40:12.857344+0000","last_active":"2024-09-16T15:40:12.867284+0000","last_peered":"2024-09-16T15:40:12.867284+0000","last_clean":"2024-09-16T15:40:12.867284+0000","last_became_active":"2024-09-16T15:40:12.857044+0000","last_became_peered":"2024-09-16T15:40:12.857044+0000","last_unstale":"2024-09-16T15:40:12.867284+0000","last_undegraded":"2024-09-16T15:40:12.867284+0000","last_fullsized":"2024-09-16T15:40:12.867284+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:28:28.848082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867593+0000","last_change":"2024-09-16T15:40:12.860869+0000","last_active":"2024-09-16T15:40:12.867593+0000","last_peered":"2024-09-16T15:40:12.867593+0000","last_clean":"2024-09-16T15:40:12.867593+0000","last_became_active":"2024-09-16T15:40:12.860241+0000","last_became_peered":"2024-09-16T15:40:12.860241+0000","last_unstale":"2024-09-16T15:40:12.867593+0000","last_undegraded":"2024-09-16T15:40:12.867593+0000","last_fullsized":"2024-09-16T15:40:12.867593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:37.559530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:40:11.850246+0000","last_change":"2024-09-16T15:40:11.850246+0000","last_active":"2024-09-16T15:40:11.835623+0000","last_peered":"2024-09-16T15:40:11.835623+0000","last_clean":"2024-09-16T15:40:11.835623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.850246+0000","last_undegraded":"2024-09-16T15:40:11.850246+0000","last_fullsized":"2024-09-16T15:40:11.850246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875888+0000","last_change":"2024-09-16T15:40:11.859931+0000","last_active":"2024-09-16T15:40:12.875888+0000","last_peered":"2024-09-16T15:40:12.875888+0000","last_clean":"2024-09-16T15:40:12.875888+0000","last_became_active":"2024-09-16T15:40:11.859675+0000","last_became_peered":"2024-09-16T15:40:11.859675+0000","last_unstale":"2024-09-16T15:40:12.875888+0000","last_undegraded":"2024-09-16T15:40:12.875888+0000","last_fullsized":"2024-09-16T15:40:12.875888+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:28.283914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867675+0000","last_change":"2024-09-16T15:40:12.861984+0000","last_active":"2024-09-16T15:40:12.867675+0000","last_peered":"2024-09-16T15:40:12.867675+0000","last_clean":"2024-09-16T15:40:12.867675+0000","last_became_active":"2024-09-16T15:40:12.861759+0000","last_became_peered":"2024-09-16T15:40:12.861759+0000","last_unstale":"2024-09-16T15:40:12.867675+0000","last_undegraded":"2024-09-16T15:40:12.867675+0000","last_fullsized":"2024-09-16T15:40:12.867675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:23.370252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880884+0000","last_change":"2024-09-16T15:40:11.863371+0000","last_active":"2024-09-16T15:40:12.880884+0000","last_peered":"2024-09-16T15:40:12.880884+0000","last_clean":"2024-09-16T15:40:12.880884+0000","last_became_active":"2024-09-16T15:40:11.860269+0000","last_became_peered":"2024-09-16T15:40:11.860269+0000","last_unstale":"2024-09-16T15:40:12.880884+0000","last_undegraded":"2024-09-16T15:40:12.880884+0000","last_fullsized":"2024-09-16T15:40:12.880884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:36.282583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.858764+0000","last_change":"2024-09-16T15:40:12.855039+0000","last_active":"2024-09-16T15:40:12.858764+0000","last_peered":"2024-09-16T15:40:12.858764+0000","last_clean":"2024-09-16T15:40:12.858764+0000","last_became_active":"2024-09-16T15:40:12.854141+0000","last_became_peered":"2024-09-16T15:40:12.854141+0000","last_unstale":"2024-09-16T15:40:12.858764+0000","last_undegraded":"2024-09-16T15:40:12.858764+0000","last_fullsized":"2024-09-16T15:40:12.858764+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:26.535001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:40:11.871707+0000","last_change":"2024-09-16T15:40:11.869853+0000","last_active":"2024-09-16T15:40:11.871707+0000","last_peered":"2024-09-16T15:40:11.871707+0000","last_clean":"2024-09-16T15:40:11.871707+0000","last_became_active":"2024-09-16T15:40:11.869583+0000","last_became_peered":"2024-09-16T15:40:11.869583+0000","last_unstale":"2024-09-16T15:40:11.871707+0000","last_undegraded":"2024-09-16T15:40:11.871707+0000","last_fullsized":"2024-09-16T15:40:11.871707+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:12.986348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:40:11.852066+0000","last_change":"2024-09-16T15:40:11.852066+0000","last_active":"2024-09-16T15:40:11.835623+0000","last_peered":"2024-09-16T15:40:11.835623+0000","last_clean":"2024-09-16T15:40:11.835623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.852066+0000","last_undegraded":"2024-09-16T15:40:11.852066+0000","last_fullsized":"2024-09-16T15:40:11.852066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880313+0000","last_change":"2024-09-16T15:40:11.868506+0000","last_active":"2024-09-16T15:40:12.880313+0000","last_peered":"2024-09-16T15:40:12.880313+0000","last_clean":"2024-09-16T15:40:12.880313+0000","last_became_active":"2024-09-16T15:40:11.868014+0000","last_became_peered":"2024-09-16T15:40:11.868014+0000","last_unstale":"2024-09-16T15:40:12.880313+0000","last_undegraded":"2024-09-16T15:40:12.880313+0000","last_fullsized":"2024-09-16T15:40:12.880313+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.292320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874345+0000","last_change":"2024-09-16T15:40:12.867667+0000","last_active":"2024-09-16T15:40:12.874345+0000","last_peered":"2024-09-16T15:40:12.874345+0000","last_clean":"2024-09-16T15:40:12.874345+0000","last_became_active":"2024-09-16T15:40:12.866213+0000","last_became_peered":"2024-09-16T15:40:12.866213+0000","last_unstale":"2024-09-16T15:40:12.874345+0000","last_undegraded":"2024-09-16T15:40:12.874345+0000","last_fullsized":"2024-09-16T15:40:12.874345+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:06.413661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.877703+0000","last_change":"2024-09-16T15:40:11.871006+0000","last_active":"2024-09-16T15:40:12.877703+0000","last_peered":"2024-09-16T15:40:12.877703+0000","last_clean":"2024-09-16T15:40:12.877703+0000","last_became_active":"2024-09-16T15:40:11.870728+0000","last_became_peered":"2024-09-16T15:40:11.870728+0000","last_unstale":"2024-09-16T15:40:12.877703+0000","last_undegraded":"2024-09-16T15:40:12.877703+0000","last_fullsized":"2024-09-16T15:40:12.877703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:26.890614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882663+0000","last_change":"2024-09-16T15:40:12.869124+0000","last_active":"2024-09-16T15:40:12.882663+0000","last_peered":"2024-09-16T15:40:12.882663+0000","last_clean":"2024-09-16T15:40:12.882663+0000","last_became_active":"2024-09-16T15:40:12.868857+0000","last_became_peered":"2024-09-16T15:40:12.868857+0000","last_unstale":"2024-09-16T15:40:12.882663+0000","last_undegraded":"2024-09-16T15:40:12.882663+0000","last_fullsized":"2024-09-16T15:40:12.882663+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:59.975480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879734+0000","last_change":"2024-09-16T15:40:11.865117+0000","last_active":"2024-09-16T15:40:12.879734+0000","last_peered":"2024-09-16T15:40:12.879734+0000","last_clean":"2024-09-16T15:40:12.879734+0000","last_became_active":"2024-09-16T15:40:11.864766+0000","last_became_peered":"2024-09-16T15:40:11.864766+0000","last_unstale":"2024-09-16T15:40:12.879734+0000","last_undegraded":"2024-09-16T15:40:12.879734+0000","last_fullsized":"2024-09-16T15:40:12.879734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:07.999976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880707+0000","last_change":"2024-09-16T15:40:12.860867+0000","last_active":"2024-09-16T15:40:12.880707+0000","last_peered":"2024-09-16T15:40:12.880707+0000","last_clean":"2024-09-16T15:40:12.880707+0000","last_became_active":"2024-09-16T15:40:12.860648+0000","last_became_peered":"2024-09-16T15:40:12.860648+0000","last_unstale":"2024-09-16T15:40:12.880707+0000","last_undegraded":"2024-09-16T15:40:12.880707+0000","last_fullsized":"2024-09-16T15:40:12.880707+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:49.901150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.844800+0000","last_change":"2024-09-16T15:40:11.868600+0000","last_active":"2024-09-16T15:40:12.844800+0000","last_peered":"2024-09-16T15:40:12.844800+0000","last_clean":"2024-09-16T15:40:12.844800+0000","last_became_active":"2024-09-16T15:40:11.868271+0000","last_became_peered":"2024-09-16T15:40:11.868271+0000","last_unstale":"2024-09-16T15:40:12.844800+0000","last_undegraded":"2024-09-16T15:40:12.844800+0000","last_fullsized":"2024-09-16T15:40:12.844800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:56.197404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870499+0000","last_change":"2024-09-16T15:40:12.864230+0000","last_active":"2024-09-16T15:40:12.870499+0000","last_peered":"2024-09-16T15:40:12.870499+0000","last_clean":"2024-09-16T15:40:12.870499+0000","last_became_active":"2024-09-16T15:40:12.863371+0000","last_became_peered":"2024-09-16T15:40:12.863371+0000","last_unstale":"2024-09-16T15:40:12.870499+0000","last_undegraded":"2024-09-16T15:40:12.870499+0000","last_fullsized":"2024-09-16T15:40:12.870499+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:26.451623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.847205+0000","last_change":"2024-09-16T15:40:11.864603+0000","last_active":"2024-09-16T15:40:12.847205+0000","last_peered":"2024-09-16T15:40:12.847205+0000","last_clean":"2024-09-16T15:40:12.847205+0000","last_became_active":"2024-09-16T15:40:11.864256+0000","last_became_peered":"2024-09-16T15:40:11.864256+0000","last_unstale":"2024-09-16T15:40:12.847205+0000","last_undegraded":"2024-09-16T15:40:12.847205+0000","last_fullsized":"2024-09-16T15:40:12.847205+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:34.607613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875009+0000","last_change":"2024-09-16T15:40:12.873524+0000","last_active":"2024-09-16T15:40:12.875009+0000","last_peered":"2024-09-16T15:40:12.875009+0000","last_clean":"2024-09-16T15:40:12.875009+0000","last_became_active":"2024-09-16T15:40:12.873256+0000","last_became_peered":"2024-09-16T15:40:12.873256+0000","last_unstale":"2024-09-16T15:40:12.875009+0000","last_undegraded":"2024-09-16T15:40:12.875009+0000","last_fullsized":"2024-09-16T15:40:12.875009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:15.479730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.851806+0000","last_change":"2024-09-16T15:40:11.860367+0000","last_active":"2024-09-16T15:40:12.851806+0000","last_peered":"2024-09-16T15:40:12.851806+0000","last_clean":"2024-09-16T15:40:12.851806+0000","last_became_active":"2024-09-16T15:40:11.858703+0000","last_became_peered":"2024-09-16T15:40:11.858703+0000","last_unstale":"2024-09-16T15:40:12.851806+0000","last_undegraded":"2024-09-16T15:40:12.851806+0000","last_fullsized":"2024-09-16T15:40:12.851806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:47.251591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.868881+0000","last_change":"2024-09-16T15:40:12.865158+0000","last_active":"2024-09-16T15:40:12.868881+0000","last_peered":"2024-09-16T15:40:12.868881+0000","last_clean":"2024-09-16T15:40:12.868881+0000","last_became_active":"2024-09-16T15:40:12.864675+0000","last_became_peered":"2024-09-16T15:40:12.864675+0000","last_unstale":"2024-09-16T15:40:12.868881+0000","last_undegraded":"2024-09-16T15:40:12.868881+0000","last_fullsized":"2024-09-16T15:40:12.868881+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.849200+0000","last_change":"2024-09-16T15:40:11.863611+0000","last_active":"2024-09-16T15:40:12.849200+0000","last_peered":"2024-09-16T15:40:12.849200+0000","last_clean":"2024-09-16T15:40:12.849200+0000","last_became_active":"2024-09-16T15:40:11.863346+0000","last_became_peered":"2024-09-16T15:40:11.863346+0000","last_unstale":"2024-09-16T15:40:12.849200+0000","last_undegraded":"2024-09-16T15:40:12.849200+0000","last_fullsized":"2024-09-16T15:40:12.849200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:00.681742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879244+0000","last_change":"2024-09-16T15:40:12.876615+0000","last_active":"2024-09-16T15:40:12.879244+0000","last_peered":"2024-09-16T15:40:12.879244+0000","last_clean":"2024-09-16T15:40:12.879244+0000","last_became_active":"2024-09-16T15:40:12.876385+0000","last_became_peered":"2024-09-16T15:40:12.876385+0000","last_unstale":"2024-09-16T15:40:12.879244+0000","last_undegraded":"2024-09-16T15:40:12.879244+0000","last_fullsized":"2024-09-16T15:40:12.879244+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:35.883048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881290+0000","last_change":"2024-09-16T15:40:11.864331+0000","last_active":"2024-09-16T15:40:12.881290+0000","last_peered":"2024-09-16T15:40:12.881290+0000","last_clean":"2024-09-16T15:40:12.881290+0000","last_became_active":"2024-09-16T15:40:11.864083+0000","last_became_peered":"2024-09-16T15:40:11.864083+0000","last_unstale":"2024-09-16T15:40:12.881290+0000","last_undegraded":"2024-09-16T15:40:12.881290+0000","last_fullsized":"2024-09-16T15:40:12.881290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:44.080258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.868336+0000","last_change":"2024-09-16T15:40:12.865124+0000","last_active":"2024-09-16T15:40:12.868336+0000","last_peered":"2024-09-16T15:40:12.868336+0000","last_clean":"2024-09-16T15:40:12.868336+0000","last_became_active":"2024-09-16T15:40:12.864809+0000","last_became_peered":"2024-09-16T15:40:12.864809+0000","last_unstale":"2024-09-16T15:40:12.868336+0000","last_undegraded":"2024-09-16T15:40:12.868336+0000","last_fullsized":"2024-09-16T15:40:12.868336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.885109+0000","last_change":"2024-09-16T15:40:11.865511+0000","last_active":"2024-09-16T15:40:12.885109+0000","last_peered":"2024-09-16T15:40:12.885109+0000","last_clean":"2024-09-16T15:40:12.885109+0000","last_became_active":"2024-09-16T15:40:11.863715+0000","last_became_peered":"2024-09-16T15:40:11.863715+0000","last_unstale":"2024-09-16T15:40:12.885109+0000","last_undegraded":"2024-09-16T15:40:12.885109+0000","last_fullsized":"2024-09-16T15:40:12.885109+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:18.604944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875276+0000","last_change":"2024-09-16T15:40:12.871040+0000","last_active":"2024-09-16T15:40:12.875276+0000","last_peered":"2024-09-16T15:40:12.875276+0000","last_clean":"2024-09-16T15:40:12.875276+0000","last_became_active":"2024-09-16T15:40:12.870459+0000","last_became_peered":"2024-09-16T15:40:12.870459+0000","last_unstale":"2024-09-16T15:40:12.875276+0000","last_undegraded":"2024-09-16T15:40:12.875276+0000","last_fullsized":"2024-09-16T15:40:12.875276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:53.575639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.884353+0000","last_change":"2024-09-16T15:40:11.870606+0000","last_active":"2024-09-16T15:40:12.884353+0000","last_peered":"2024-09-16T15:40:12.884353+0000","last_clean":"2024-09-16T15:40:12.884353+0000","last_became_active":"2024-09-16T15:40:11.868146+0000","last_became_peered":"2024-09-16T15:40:11.868146+0000","last_unstale":"2024-09-16T15:40:12.884353+0000","last_undegraded":"2024-09-16T15:40:12.884353+0000","last_fullsized":"2024-09-16T15:40:12.884353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:36.745807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880157+0000","last_change":"2024-09-16T15:40:12.860151+0000","last_active":"2024-09-16T15:40:12.880157+0000","last_peered":"2024-09-16T15:40:12.880157+0000","last_clean":"2024-09-16T15:40:12.880157+0000","last_became_active":"2024-09-16T15:40:12.859179+0000","last_became_peered":"2024-09-16T15:40:12.859179+0000","last_unstale":"2024-09-16T15:40:12.880157+0000","last_undegraded":"2024-09-16T15:40:12.880157+0000","last_fullsized":"2024-09-16T15:40:12.880157+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:39.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855830+0000","last_change":"2024-09-16T15:40:11.863101+0000","last_active":"2024-09-16T15:40:12.855830+0000","last_peered":"2024-09-16T15:40:12.855830+0000","last_clean":"2024-09-16T15:40:12.855830+0000","last_became_active":"2024-09-16T15:40:11.862829+0000","last_became_peered":"2024-09-16T15:40:11.862829+0000","last_unstale":"2024-09-16T15:40:12.855830+0000","last_undegraded":"2024-09-16T15:40:12.855830+0000","last_fullsized":"2024-09-16T15:40:12.855830+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:25.527579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.876819+0000","last_change":"2024-09-16T15:40:12.871218+0000","last_active":"2024-09-16T15:40:12.876819+0000","last_peered":"2024-09-16T15:40:12.876819+0000","last_clean":"2024-09-16T15:40:12.876819+0000","last_became_active":"2024-09-16T15:40:12.870997+0000","last_became_peered":"2024-09-16T15:40:12.870997+0000","last_unstale":"2024-09-16T15:40:12.876819+0000","last_undegraded":"2024-09-16T15:40:12.876819+0000","last_fullsized":"2024-09-16T15:40:12.876819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:27.010957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.852972+0000","last_change":"2024-09-16T15:40:11.867251+0000","last_active":"2024-09-16T15:40:12.852972+0000","last_peered":"2024-09-16T15:40:12.852972+0000","last_clean":"2024-09-16T15:40:12.852972+0000","last_became_active":"2024-09-16T15:40:11.866730+0000","last_became_peered":"2024-09-16T15:40:11.866730+0000","last_unstale":"2024-09-16T15:40:12.852972+0000","last_undegraded":"2024-09-16T15:40:12.852972+0000","last_fullsized":"2024-09-16T15:40:12.852972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:55.152872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873844+0000","last_change":"2024-09-16T15:40:12.869762+0000","last_active":"2024-09-16T15:40:12.873844+0000","last_peered":"2024-09-16T15:40:12.873844+0000","last_clean":"2024-09-16T15:40:12.873844+0000","last_became_active":"2024-09-16T15:40:12.869420+0000","last_became_peered":"2024-09-16T15:40:12.869420+0000","last_unstale":"2024-09-16T15:40:12.873844+0000","last_undegraded":"2024-09-16T15:40:12.873844+0000","last_fullsized":"2024-09-16T15:40:12.873844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:51.377166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.888763+0000","last_change":"2024-09-16T15:40:11.868078+0000","last_active":"2024-09-16T15:40:12.888763+0000","last_peered":"2024-09-16T15:40:12.888763+0000","last_clean":"2024-09-16T15:40:12.888763+0000","last_became_active":"2024-09-16T15:40:11.867825+0000","last_became_peered":"2024-09-16T15:40:11.867825+0000","last_unstale":"2024-09-16T15:40:12.888763+0000","last_undegraded":"2024-09-16T15:40:12.888763+0000","last_fullsized":"2024-09-16T15:40:12.888763+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:29.103104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872014+0000","last_change":"2024-09-16T15:40:12.867357+0000","last_active":"2024-09-16T15:40:12.872014+0000","last_peered":"2024-09-16T15:40:12.872014+0000","last_clean":"2024-09-16T15:40:12.872014+0000","last_became_active":"2024-09-16T15:40:12.867072+0000","last_became_peered":"2024-09-16T15:40:12.867072+0000","last_unstale":"2024-09-16T15:40:12.872014+0000","last_undegraded":"2024-09-16T15:40:12.872014+0000","last_fullsized":"2024-09-16T15:40:12.872014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:17.395905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.886756+0000","last_change":"2024-09-16T15:40:11.866281+0000","last_active":"2024-09-16T15:40:12.886756+0000","last_peered":"2024-09-16T15:40:12.886756+0000","last_clean":"2024-09-16T15:40:12.886756+0000","last_became_active":"2024-09-16T15:40:11.865322+0000","last_became_peered":"2024-09-16T15:40:11.865322+0000","last_unstale":"2024-09-16T15:40:12.886756+0000","last_undegraded":"2024-09-16T15:40:12.886756+0000","last_fullsized":"2024-09-16T15:40:12.886756+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:14:13.585151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883782+0000","last_change":"2024-09-16T15:40:12.874698+0000","last_active":"2024-09-16T15:40:12.883782+0000","last_peered":"2024-09-16T15:40:12.883782+0000","last_clean":"2024-09-16T15:40:12.883782+0000","last_became_active":"2024-09-16T15:40:12.874345+0000","last_became_peered":"2024-09-16T15:40:12.874345+0000","last_unstale":"2024-09-16T15:40:12.883782+0000","last_undegraded":"2024-09-16T15:40:12.883782+0000","last_fullsized":"2024-09-16T15:40:12.883782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:49.306539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880542+0000","last_change":"2024-09-16T15:40:11.859504+0000","last_active":"2024-09-16T15:40:12.880542+0000","last_peered":"2024-09-16T15:40:12.880542+0000","last_clean":"2024-09-16T15:40:12.880542+0000","last_became_active":"2024-09-16T15:40:11.859183+0000","last_became_peered":"2024-09-16T15:40:11.859183+0000","last_unstale":"2024-09-16T15:40:12.880542+0000","last_undegraded":"2024-09-16T15:40:12.880542+0000","last_fullsized":"2024-09-16T15:40:12.880542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:57.973947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.863982+0000","last_change":"2024-09-16T15:40:12.856434+0000","last_active":"2024-09-16T15:40:12.863982+0000","last_peered":"2024-09-16T15:40:12.863982+0000","last_clean":"2024-09-16T15:40:12.863982+0000","last_became_active":"2024-09-16T15:40:12.856073+0000","last_became_peered":"2024-09-16T15:40:12.856073+0000","last_unstale":"2024-09-16T15:40:12.863982+0000","last_undegraded":"2024-09-16T15:40:12.863982+0000","last_fullsized":"2024-09-16T15:40:12.863982+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:22.248689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.878307+0000","last_change":"2024-09-16T15:40:11.869221+0000","last_active":"2024-09-16T15:40:12.878307+0000","last_peered":"2024-09-16T15:40:12.878307+0000","last_clean":"2024-09-16T15:40:12.878307+0000","last_became_active":"2024-09-16T15:40:11.868892+0000","last_became_peered":"2024-09-16T15:40:11.868892+0000","last_unstale":"2024-09-16T15:40:12.878307+0000","last_undegraded":"2024-09-16T15:40:12.878307+0000","last_fullsized":"2024-09-16T15:40:12.878307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.865500+0000","last_change":"2024-09-16T15:40:12.859954+0000","last_active":"2024-09-16T15:40:12.865500+0000","last_peered":"2024-09-16T15:40:12.865500+0000","last_clean":"2024-09-16T15:40:12.865500+0000","last_became_active":"2024-09-16T15:40:12.859695+0000","last_became_peered":"2024-09-16T15:40:12.859695+0000","last_unstale":"2024-09-16T15:40:12.865500+0000","last_undegraded":"2024-09-16T15:40:12.865500+0000","last_fullsized":"2024-09-16T15:40:12.865500+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:50.839867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.851789+0000","last_change":"2024-09-16T15:40:11.863949+0000","last_active":"2024-09-16T15:40:12.851789+0000","last_peered":"2024-09-16T15:40:12.851789+0000","last_clean":"2024-09-16T15:40:12.851789+0000","last_became_active":"2024-09-16T15:40:11.863620+0000","last_became_peered":"2024-09-16T15:40:11.863620+0000","last_unstale":"2024-09-16T15:40:12.851789+0000","last_undegraded":"2024-09-16T15:40:12.851789+0000","last_fullsized":"2024-09-16T15:40:12.851789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:18.083324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.877926+0000","last_change":"2024-09-16T15:40:12.873993+0000","last_active":"2024-09-16T15:40:12.877926+0000","last_peered":"2024-09-16T15:40:12.877926+0000","last_clean":"2024-09-16T15:40:12.877926+0000","last_became_active":"2024-09-16T15:40:12.873720+0000","last_became_peered":"2024-09-16T15:40:12.873720+0000","last_unstale":"2024-09-16T15:40:12.877926+0000","last_undegraded":"2024-09-16T15:40:12.877926+0000","last_fullsized":"2024-09-16T15:40:12.877926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:47.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880354+0000","last_change":"2024-09-16T15:40:11.871266+0000","last_active":"2024-09-16T15:40:12.880354+0000","last_peered":"2024-09-16T15:40:12.880354+0000","last_clean":"2024-09-16T15:40:12.880354+0000","last_became_active":"2024-09-16T15:40:11.871013+0000","last_became_peered":"2024-09-16T15:40:11.871013+0000","last_unstale":"2024-09-16T15:40:12.880354+0000","last_undegraded":"2024-09-16T15:40:12.880354+0000","last_fullsized":"2024-09-16T15:40:12.880354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:10.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:40:11.850702+0000","last_change":"2024-09-16T15:40:11.850702+0000","last_active":"2024-09-16T15:40:11.835623+0000","last_peered":"2024-09-16T15:40:11.835623+0000","last_clean":"2024-09-16T15:40:11.835623+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.850702+0000","last_undegraded":"2024-09-16T15:40:11.850702+0000","last_fullsized":"2024-09-16T15:40:11.850702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.845323+0000","last_change":"2024-09-16T15:40:11.865942+0000","last_active":"2024-09-16T15:40:12.845323+0000","last_peered":"2024-09-16T15:40:12.845323+0000","last_clean":"2024-09-16T15:40:12.845323+0000","last_became_active":"2024-09-16T15:40:11.864112+0000","last_became_peered":"2024-09-16T15:40:11.864112+0000","last_unstale":"2024-09-16T15:40:12.845323+0000","last_undegraded":"2024-09-16T15:40:12.845323+0000","last_fullsized":"2024-09-16T15:40:12.845323+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:13.011662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867632+0000","last_change":"2024-09-16T15:40:12.861166+0000","last_active":"2024-09-16T15:40:12.867632+0000","last_peered":"2024-09-16T15:40:12.867632+0000","last_clean":"2024-09-16T15:40:12.867632+0000","last_became_active":"2024-09-16T15:40:12.860619+0000","last_became_peered":"2024-09-16T15:40:12.860619+0000","last_unstale":"2024-09-16T15:40:12.867632+0000","last_undegraded":"2024-09-16T15:40:12.867632+0000","last_fullsized":"2024-09-16T15:40:12.867632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:54.024305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.862595+0000","last_change":"2024-09-16T15:40:11.862288+0000","last_active":"2024-09-16T15:40:12.862595+0000","last_peered":"2024-09-16T15:40:12.862595+0000","last_clean":"2024-09-16T15:40:12.862595+0000","last_became_active":"2024-09-16T15:40:11.862002+0000","last_became_peered":"2024-09-16T15:40:11.862002+0000","last_unstale":"2024-09-16T15:40:12.862595+0000","last_undegraded":"2024-09-16T15:40:12.862595+0000","last_fullsized":"2024-09-16T15:40:12.862595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:12.191215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.884800+0000","last_change":"2024-09-16T15:40:12.879430+0000","last_active":"2024-09-16T15:40:12.884800+0000","last_peered":"2024-09-16T15:40:12.884800+0000","last_clean":"2024-09-16T15:40:12.884800+0000","last_became_active":"2024-09-16T15:40:12.879172+0000","last_became_peered":"2024-09-16T15:40:12.879172+0000","last_unstale":"2024-09-16T15:40:12.884800+0000","last_undegraded":"2024-09-16T15:40:12.884800+0000","last_fullsized":"2024-09-16T15:40:12.884800+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:14.019085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.854691+0000","last_change":"2024-09-16T15:40:11.863062+0000","last_active":"2024-09-16T15:40:12.854691+0000","last_peered":"2024-09-16T15:40:12.854691+0000","last_clean":"2024-09-16T15:40:12.854691+0000","last_became_active":"2024-09-16T15:40:11.860714+0000","last_became_peered":"2024-09-16T15:40:11.860714+0000","last_unstale":"2024-09-16T15:40:12.854691+0000","last_undegraded":"2024-09-16T15:40:12.854691+0000","last_fullsized":"2024-09-16T15:40:12.854691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883782+0000","last_change":"2024-09-16T15:40:12.883082+0000","last_active":"2024-09-16T15:40:12.883782+0000","last_peered":"2024-09-16T15:40:12.883782+0000","last_clean":"2024-09-16T15:40:12.883782+0000","last_became_active":"2024-09-16T15:40:12.882695+0000","last_became_peered":"2024-09-16T15:40:12.882695+0000","last_unstale":"2024-09-16T15:40:12.883782+0000","last_undegraded":"2024-09-16T15:40:12.883782+0000","last_fullsized":"2024-09-16T15:40:12.883782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:21.231163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.853585+0000","last_change":"2024-09-16T15:40:11.867238+0000","last_active":"2024-09-16T15:40:12.853585+0000","last_peered":"2024-09-16T15:40:12.853585+0000","last_clean":"2024-09-16T15:40:12.853585+0000","last_became_active":"2024-09-16T15:40:11.865996+0000","last_became_peered":"2024-09-16T15:40:11.865996+0000","last_unstale":"2024-09-16T15:40:12.853585+0000","last_undegraded":"2024-09-16T15:40:12.853585+0000","last_fullsized":"2024-09-16T15:40:12.853585+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:23.953426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883245+0000","last_change":"2024-09-16T15:40:12.872931+0000","last_active":"2024-09-16T15:40:12.883245+0000","last_peered":"2024-09-16T15:40:12.883245+0000","last_clean":"2024-09-16T15:40:12.883245+0000","last_became_active":"2024-09-16T15:40:12.872344+0000","last_became_peered":"2024-09-16T15:40:12.872344+0000","last_unstale":"2024-09-16T15:40:12.883245+0000","last_undegraded":"2024-09-16T15:40:12.883245+0000","last_fullsized":"2024-09-16T15:40:12.883245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:28.977918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855957+0000","last_change":"2024-09-16T15:40:11.865216+0000","last_active":"2024-09-16T15:40:12.855957+0000","last_peered":"2024-09-16T15:40:12.855957+0000","last_clean":"2024-09-16T15:40:12.855957+0000","last_became_active":"2024-09-16T15:40:11.864945+0000","last_became_peered":"2024-09-16T15:40:11.864945+0000","last_unstale":"2024-09-16T15:40:12.855957+0000","last_undegraded":"2024-09-16T15:40:12.855957+0000","last_fullsized":"2024-09-16T15:40:12.855957+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:13.850144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873339+0000","last_change":"2024-09-16T15:40:12.868439+0000","last_active":"2024-09-16T15:40:12.873339+0000","last_peered":"2024-09-16T15:40:12.873339+0000","last_clean":"2024-09-16T15:40:12.873339+0000","last_became_active":"2024-09-16T15:40:12.867651+0000","last_became_peered":"2024-09-16T15:40:12.867651+0000","last_unstale":"2024-09-16T15:40:12.873339+0000","last_undegraded":"2024-09-16T15:40:12.873339+0000","last_fullsized":"2024-09-16T15:40:12.873339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:12.149041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:40:11.871863+0000","last_change":"2024-09-16T15:40:11.869272+0000","last_active":"2024-09-16T15:40:11.871863+0000","last_peered":"2024-09-16T15:40:11.871863+0000","last_clean":"2024-09-16T15:40:11.871863+0000","last_became_active":"2024-09-16T15:40:11.868909+0000","last_became_peered":"2024-09-16T15:40:11.868909+0000","last_unstale":"2024-09-16T15:40:11.871863+0000","last_undegraded":"2024-09-16T15:40:11.871863+0000","last_fullsized":"2024-09-16T15:40:11.871863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:31.920861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874248+0000","last_change":"2024-09-16T15:40:12.862103+0000","last_active":"2024-09-16T15:40:12.874248+0000","last_peered":"2024-09-16T15:40:12.874248+0000","last_clean":"2024-09-16T15:40:12.874248+0000","last_became_active":"2024-09-16T15:40:12.861788+0000","last_became_peered":"2024-09-16T15:40:12.861788+0000","last_unstale":"2024-09-16T15:40:12.874248+0000","last_undegraded":"2024-09-16T15:40:12.874248+0000","last_fullsized":"2024-09-16T15:40:12.874248+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:38.051709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855069+0000","last_change":"2024-09-16T15:40:11.867570+0000","last_active":"2024-09-16T15:40:12.855069+0000","last_peered":"2024-09-16T15:40:12.855069+0000","last_clean":"2024-09-16T15:40:12.855069+0000","last_became_active":"2024-09-16T15:40:11.867301+0000","last_became_peered":"2024-09-16T15:40:11.867301+0000","last_unstale":"2024-09-16T15:40:12.855069+0000","last_undegraded":"2024-09-16T15:40:12.855069+0000","last_fullsized":"2024-09-16T15:40:12.855069+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879331+0000","last_change":"2024-09-16T15:40:12.875892+0000","last_active":"2024-09-16T15:40:12.879331+0000","last_peered":"2024-09-16T15:40:12.879331+0000","last_clean":"2024-09-16T15:40:12.879331+0000","last_became_active":"2024-09-16T15:40:12.875619+0000","last_became_peered":"2024-09-16T15:40:12.875619+0000","last_unstale":"2024-09-16T15:40:12.879331+0000","last_undegraded":"2024-09-16T15:40:12.879331+0000","last_fullsized":"2024-09-16T15:40:12.879331+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:15.128749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.849889+0000","last_change":"2024-09-16T15:40:11.867392+0000","last_active":"2024-09-16T15:40:12.849889+0000","last_peered":"2024-09-16T15:40:12.849889+0000","last_clean":"2024-09-16T15:40:12.849889+0000","last_became_active":"2024-09-16T15:40:11.867003+0000","last_became_peered":"2024-09-16T15:40:11.867003+0000","last_unstale":"2024-09-16T15:40:12.849889+0000","last_undegraded":"2024-09-16T15:40:12.849889+0000","last_fullsized":"2024-09-16T15:40:12.849889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:00.706403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.857550+0000","last_change":"2024-09-16T15:40:12.850142+0000","last_active":"2024-09-16T15:40:12.857550+0000","last_peered":"2024-09-16T15:40:12.857550+0000","last_clean":"2024-09-16T15:40:12.857550+0000","last_became_active":"2024-09-16T15:40:12.849859+0000","last_became_peered":"2024-09-16T15:40:12.849859+0000","last_unstale":"2024-09-16T15:40:12.857550+0000","last_undegraded":"2024-09-16T15:40:12.857550+0000","last_fullsized":"2024-09-16T15:40:12.857550+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:45.087681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:40:11.863988+0000","last_change":"2024-09-16T15:40:11.860188+0000","last_active":"2024-09-16T15:40:11.863988+0000","last_peered":"2024-09-16T15:40:11.863988+0000","last_clean":"2024-09-16T15:40:11.863988+0000","last_became_active":"2024-09-16T15:40:11.859924+0000","last_became_peered":"2024-09-16T15:40:11.859924+0000","last_unstale":"2024-09-16T15:40:11.863988+0000","last_undegraded":"2024-09-16T15:40:11.863988+0000","last_fullsized":"2024-09-16T15:40:11.863988+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:30.152062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867386+0000","last_change":"2024-09-16T15:40:12.862284+0000","last_active":"2024-09-16T15:40:12.867386+0000","last_peered":"2024-09-16T15:40:12.867386+0000","last_clean":"2024-09-16T15:40:12.867386+0000","last_became_active":"2024-09-16T15:40:12.861890+0000","last_became_peered":"2024-09-16T15:40:12.861890+0000","last_unstale":"2024-09-16T15:40:12.867386+0000","last_undegraded":"2024-09-16T15:40:12.867386+0000","last_fullsized":"2024-09-16T15:40:12.867386+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:52.757331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.847519+0000","last_change":"2024-09-16T15:40:11.869696+0000","last_active":"2024-09-16T15:40:12.847519+0000","last_peered":"2024-09-16T15:40:12.847519+0000","last_clean":"2024-09-16T15:40:12.847519+0000","last_became_active":"2024-09-16T15:40:11.869403+0000","last_became_peered":"2024-09-16T15:40:11.869403+0000","last_unstale":"2024-09-16T15:40:12.847519+0000","last_undegraded":"2024-09-16T15:40:12.847519+0000","last_fullsized":"2024-09-16T15:40:12.847519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:53.785910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871439+0000","last_change":"2024-09-16T15:40:12.859022+0000","last_active":"2024-09-16T15:40:12.871439+0000","last_peered":"2024-09-16T15:40:12.871439+0000","last_clean":"2024-09-16T15:40:12.871439+0000","last_became_active":"2024-09-16T15:40:12.858683+0000","last_became_peered":"2024-09-16T15:40:12.858683+0000","last_unstale":"2024-09-16T15:40:12.871439+0000","last_undegraded":"2024-09-16T15:40:12.871439+0000","last_fullsized":"2024-09-16T15:40:12.871439+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:46.347083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881285+0000","last_change":"2024-09-16T15:40:11.872583+0000","last_active":"2024-09-16T15:40:12.881285+0000","last_peered":"2024-09-16T15:40:12.881285+0000","last_clean":"2024-09-16T15:40:12.881285+0000","last_became_active":"2024-09-16T15:40:11.872290+0000","last_became_peered":"2024-09-16T15:40:11.872290+0000","last_unstale":"2024-09-16T15:40:12.881285+0000","last_undegraded":"2024-09-16T15:40:12.881285+0000","last_fullsized":"2024-09-16T15:40:12.881285+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.854788+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:40:12.854788+0000","last_peered":"2024-09-16T15:40:12.854788+0000","last_clean":"2024-09-16T15:40:12.854788+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:40:12.854788+0000","last_undegraded":"2024-09-16T15:40:12.854788+0000","last_fullsized":"2024-09-16T15:40:12.854788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1126400,"data_stored":802873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":548864,"data_stored":219862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":219350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":802873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":520192,"data_stored":206238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:17.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.733+0000 7f826e200640 1 -- 172.21.15.60:0/2784957086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 msgr2=0x7f82680cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.733+0000 7f826e200640 1 --2- 172.21.15.60:0/2784957086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 0x7f82680cc890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8258005fd0 tx=0x7f825802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.733+0000 7f826e200640 1 -- 172.21.15.60:0/2784957086 shutdown_connections 2024-09-16T15:40:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.733+0000 7f826e200640 1 --2- 172.21.15.60:0/2784957086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82680d0ab0 0x7f82680b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.733+0000 7f826e200640 1 --2- 172.21.15.60:0/2784957086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 0x7f82680cc890 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.733+0000 7f826e200640 1 -- 172.21.15.60:0/2784957086 >> 172.21.15.60:0/2784957086 conn(0x7f82680d1dc0 msgr2=0x7f82680d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.733+0000 7f826e200640 1 -- 172.21.15.60:0/2784957086 shutdown_connections 2024-09-16T15:40:17.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.734+0000 7f826e200640 1 -- 172.21.15.60:0/2784957086 wait complete. 2024-09-16T15:40:17.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.735+0000 7f826e200640 1 Processor -- start 2024-09-16T15:40:17.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.735+0000 7f826e200640 1 -- start start 2024-09-16T15:40:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.736+0000 7f826e200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82680ccdd0 0x7f82681f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.736+0000 7f826e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82680d0ab0 0x7f82680ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.736+0000 7f826e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 0x7f82681f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.736+0000 7f826e200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f82680b2440 con 0x7f82680d0ab0 2024-09-16T15:40:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.736+0000 7f826e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82680b22c0 con 0x7f826815f450 2024-09-16T15:40:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.736+0000 7f826e200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f82680b25c0 con 0x7f82680ccdd0 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82680d0ab0 0x7f82680ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82680d0ab0 0x7f82680ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37456/0 (socket says 172.21.15.60:37456) 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 -- 172.21.15.60:0/2770885937 learned_addr learned my addr 172.21.15.60:0/2770885937 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f8267400640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82680ccdd0 0x7f82681f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f8267e00640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 0x7f82681f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 -- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82680ccdd0 msgr2=0x7f82681f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82680ccdd0 0x7f82681f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 -- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 msgr2=0x7f82681f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 0x7f82681f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 -- 172.21.15.60:0/2770885937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82681f55d0 con 0x7f82680d0ab0 2024-09-16T15:40:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f8267e00640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 0x7f82681f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:17.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.737+0000 7f825ea00640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82680d0ab0 0x7f82680ca4e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f825000c450 tx=0x7f825000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:17.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.738+0000 7f825fe00640 1 -- 172.21.15.60:0/2770885937 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82500052d0 con 0x7f82680d0ab0 2024-09-16T15:40:17.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.738+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82681f58b0 con 0x7f82680d0ab0 2024-09-16T15:40:17.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.738+0000 7f825fe00640 1 -- 172.21.15.60:0/2770885937 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8250005470 con 0x7f82680d0ab0 2024-09-16T15:40:17.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.738+0000 7f825fe00640 1 -- 172.21.15.60:0/2770885937 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250012870 con 0x7f82680d0ab0 2024-09-16T15:40:17.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.738+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82681f8630 con 0x7f82680d0ab0 2024-09-16T15:40:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.739+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82681f5dd0 con 0x7f82680d0ab0 2024-09-16T15:40:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.740+0000 7f825fe00640 1 -- 172.21.15.60:0/2770885937 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f82500035f0 con 0x7f82680d0ab0 2024-09-16T15:40:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.741+0000 7f825fe00640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8244078a00 0x7f824407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.741+0000 7f825fe00640 1 -- 172.21.15.60:0/2770885937 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f825009a900 con 0x7f82680d0ab0 2024-09-16T15:40:17.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.742+0000 7f8267400640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8244078a00 0x7f824407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.742+0000 7f8267400640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8244078a00 0x7f824407aec0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8258033000 tx=0x7f8258004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:17.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.745+0000 7f825fe00640 1 -- 172.21.15.60:0/2770885937 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f82681f5dd0 con 0x7f82680d0ab0 2024-09-16T15:40:17.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.938+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82681f8800 con 0x7f8244078a00 2024-09-16T15:40:17.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.943+0000 7f825fe00640 1 -- 172.21.15.60:0/2770885937 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7f82681f8800 con 0x7f8244078a00 2024-09-16T15:40:17.947 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:17.948 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:40:17.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8244078a00 msgr2=0x7f824407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8244078a00 0x7f824407aec0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8258033000 tx=0x7f8258004730 comp rx=0 tx=0).stop 2024-09-16T15:40:17.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82680d0ab0 msgr2=0x7f82680ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82680d0ab0 0x7f82680ca4e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f825000c450 tx=0x7f825000c910 comp rx=0 tx=0).stop 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 shutdown_connections 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8244078a00 0x7f824407aec0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f826815f450 0x7f82681f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82680d0ab0 0x7f82680ca4e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 --2- 172.21.15.60:0/2770885937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82680ccdd0 0x7f82681f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 >> 172.21.15.60:0/2770885937 conn(0x7f82680d1dc0 msgr2=0x7f826806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 shutdown_connections 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:17.947+0000 7f826e200640 1 -- 172.21.15.60:0/2770885937 wait complete. 2024-09-16T15:40:17.964 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-16T15:40:17.734369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238284,"kb_used_data":5380,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736436,"statfs":{"total":773094113280,"available":772850110464,"internally_reserved":0,"allocated":5509120,"data_stored":2908123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001649"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872674+0000","last_change":"2024-09-16T15:40:12.868607+0000","last_active":"2024-09-16T15:40:12.872674+0000","last_peered":"2024-09-16T15:40:12.872674+0000","last_clean":"2024-09-16T15:40:12.872674+0000","last_became_active":"2024-09-16T15:40:12.868140+0000","last_became_peered":"2024-09-16T15:40:12.868140+0000","last_unstale":"2024-09-16T15:40:12.872674+0000","last_undegraded":"2024-09-16T15:40:12.872674+0000","last_fullsized":"2024-09-16T15:40:12.872674+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:29.669859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872203+0000","last_change":"2024-09-16T15:40:12.869358+0000","last_active":"2024-09-16T15:40:12.872203+0000","last_peered":"2024-09-16T15:40:12.872203+0000","last_clean":"2024-09-16T15:40:12.872203+0000","last_became_active":"2024-09-16T15:40:12.868998+0000","last_became_peered":"2024-09-16T15:40:12.868998+0000","last_unstale":"2024-09-16T15:40:12.872203+0000","last_undegraded":"2024-09-16T15:40:12.872203+0000","last_fullsized":"2024-09-16T15:40:12.872203+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874339+0000","last_change":"2024-09-16T15:40:12.868681+0000","last_active":"2024-09-16T15:40:12.874339+0000","last_peered":"2024-09-16T15:40:12.874339+0000","last_clean":"2024-09-16T15:40:12.874339+0000","last_became_active":"2024-09-16T15:40:12.867980+0000","last_became_peered":"2024-09-16T15:40:12.867980+0000","last_unstale":"2024-09-16T15:40:12.874339+0000","last_undegraded":"2024-09-16T15:40:12.874339+0000","last_fullsized":"2024-09-16T15:40:12.874339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:39.160061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883276+0000","last_change":"2024-09-16T15:40:12.873202+0000","last_active":"2024-09-16T15:40:12.883276+0000","last_peered":"2024-09-16T15:40:12.883276+0000","last_clean":"2024-09-16T15:40:12.883276+0000","last_became_active":"2024-09-16T15:40:12.872682+0000","last_became_peered":"2024-09-16T15:40:12.872682+0000","last_unstale":"2024-09-16T15:40:12.883276+0000","last_undegraded":"2024-09-16T15:40:12.883276+0000","last_fullsized":"2024-09-16T15:40:12.883276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:23.455972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874812+0000","last_change":"2024-09-16T15:40:12.870112+0000","last_active":"2024-09-16T15:40:12.874812+0000","last_peered":"2024-09-16T15:40:12.874812+0000","last_clean":"2024-09-16T15:40:12.874812+0000","last_became_active":"2024-09-16T15:40:12.869753+0000","last_became_peered":"2024-09-16T15:40:12.869753+0000","last_unstale":"2024-09-16T15:40:12.874812+0000","last_undegraded":"2024-09-16T15:40:12.874812+0000","last_fullsized":"2024-09-16T15:40:12.874812+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875406+0000","last_change":"2024-09-16T15:40:12.866164+0000","last_active":"2024-09-16T15:40:12.875406+0000","last_peered":"2024-09-16T15:40:12.875406+0000","last_clean":"2024-09-16T15:40:12.875406+0000","last_became_active":"2024-09-16T15:40:12.865904+0000","last_became_peered":"2024-09-16T15:40:12.865904+0000","last_unstale":"2024-09-16T15:40:12.875406+0000","last_undegraded":"2024-09-16T15:40:12.875406+0000","last_fullsized":"2024-09-16T15:40:12.875406+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:44.336056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.869132+0000","last_change":"2024-09-16T15:40:12.865180+0000","last_active":"2024-09-16T15:40:12.869132+0000","last_peered":"2024-09-16T15:40:12.869132+0000","last_clean":"2024-09-16T15:40:12.869132+0000","last_became_active":"2024-09-16T15:40:12.864842+0000","last_became_peered":"2024-09-16T15:40:12.864842+0000","last_unstale":"2024-09-16T15:40:12.869132+0000","last_undegraded":"2024-09-16T15:40:12.869132+0000","last_fullsized":"2024-09-16T15:40:12.869132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:36.820991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880732+0000","last_change":"2024-09-16T15:40:12.861554+0000","last_active":"2024-09-16T15:40:12.880732+0000","last_peered":"2024-09-16T15:40:12.880732+0000","last_clean":"2024-09-16T15:40:12.880732+0000","last_became_active":"2024-09-16T15:40:12.861215+0000","last_became_peered":"2024-09-16T15:40:12.861215+0000","last_unstale":"2024-09-16T15:40:12.880732+0000","last_undegraded":"2024-09-16T15:40:12.880732+0000","last_fullsized":"2024-09-16T15:40:12.880732+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:16.169482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882156+0000","last_change":"2024-09-16T15:40:12.864597+0000","last_active":"2024-09-16T15:40:12.882156+0000","last_peered":"2024-09-16T15:40:12.882156+0000","last_clean":"2024-09-16T15:40:12.882156+0000","last_became_active":"2024-09-16T15:40:12.864345+0000","last_became_peered":"2024-09-16T15:40:12.864345+0000","last_unstale":"2024-09-16T15:40:12.882156+0000","last_undegraded":"2024-09-16T15:40:12.882156+0000","last_fullsized":"2024-09-16T15:40:12.882156+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:43.902508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882148+0000","last_change":"2024-09-16T15:40:12.865537+0000","last_active":"2024-09-16T15:40:12.882148+0000","last_peered":"2024-09-16T15:40:12.882148+0000","last_clean":"2024-09-16T15:40:12.882148+0000","last_became_active":"2024-09-16T15:40:12.865293+0000","last_became_peered":"2024-09-16T15:40:12.865293+0000","last_unstale":"2024-09-16T15:40:12.882148+0000","last_undegraded":"2024-09-16T15:40:12.882148+0000","last_fullsized":"2024-09-16T15:40:12.882148+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:15.882606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883443+0000","last_change":"2024-09-16T15:40:12.875755+0000","last_active":"2024-09-16T15:40:12.883443+0000","last_peered":"2024-09-16T15:40:12.883443+0000","last_clean":"2024-09-16T15:40:12.883443+0000","last_became_active":"2024-09-16T15:40:12.875484+0000","last_became_peered":"2024-09-16T15:40:12.875484+0000","last_unstale":"2024-09-16T15:40:12.883443+0000","last_undegraded":"2024-09-16T15:40:12.883443+0000","last_fullsized":"2024-09-16T15:40:12.883443+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:57.204826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870792+0000","last_change":"2024-09-16T15:40:12.866408+0000","last_active":"2024-09-16T15:40:12.870792+0000","last_peered":"2024-09-16T15:40:12.870792+0000","last_clean":"2024-09-16T15:40:12.870792+0000","last_became_active":"2024-09-16T15:40:12.865787+0000","last_became_peered":"2024-09-16T15:40:12.865787+0000","last_unstale":"2024-09-16T15:40:12.870792+0000","last_undegraded":"2024-09-16T15:40:12.870792+0000","last_fullsized":"2024-09-16T15:40:12.870792+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:39.299743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867785+0000","last_change":"2024-09-16T15:40:12.865889+0000","last_active":"2024-09-16T15:40:12.867785+0000","last_peered":"2024-09-16T15:40:12.867785+0000","last_clean":"2024-09-16T15:40:12.867785+0000","last_became_active":"2024-09-16T15:40:12.865611+0000","last_became_peered":"2024-09-16T15:40:12.865611+0000","last_unstale":"2024-09-16T15:40:12.867785+0000","last_undegraded":"2024-09-16T15:40:12.867785+0000","last_fullsized":"2024-09-16T15:40:12.867785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:28.736859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881374+0000","last_change":"2024-09-16T15:40:12.863378+0000","last_active":"2024-09-16T15:40:12.881374+0000","last_peered":"2024-09-16T15:40:12.881374+0000","last_clean":"2024-09-16T15:40:12.881374+0000","last_became_active":"2024-09-16T15:40:12.863108+0000","last_became_peered":"2024-09-16T15:40:12.863108+0000","last_unstale":"2024-09-16T15:40:12.881374+0000","last_undegraded":"2024-09-16T15:40:12.881374+0000","last_fullsized":"2024-09-16T15:40:12.881374+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:34.731306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873670+0000","last_change":"2024-09-16T15:40:12.861959+0000","last_active":"2024-09-16T15:40:12.873670+0000","last_peered":"2024-09-16T15:40:12.873670+0000","last_clean":"2024-09-16T15:40:12.873670+0000","last_became_active":"2024-09-16T15:40:12.861657+0000","last_became_peered":"2024-09-16T15:40:12.861657+0000","last_unstale":"2024-09-16T15:40:12.873670+0000","last_undegraded":"2024-09-16T15:40:12.873670+0000","last_fullsized":"2024-09-16T15:40:12.873670+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:55.582229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871103+0000","last_change":"2024-09-16T15:40:12.865876+0000","last_active":"2024-09-16T15:40:12.871103+0000","last_peered":"2024-09-16T15:40:12.871103+0000","last_clean":"2024-09-16T15:40:12.871103+0000","last_became_active":"2024-09-16T15:40:12.865493+0000","last_became_peered":"2024-09-16T15:40:12.865493+0000","last_unstale":"2024-09-16T15:40:12.871103+0000","last_undegraded":"2024-09-16T15:40:12.871103+0000","last_fullsized":"2024-09-16T15:40:12.871103+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:44.257723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867152+0000","last_change":"2024-09-16T15:40:12.861173+0000","last_active":"2024-09-16T15:40:12.867152+0000","last_peered":"2024-09-16T15:40:12.867152+0000","last_clean":"2024-09-16T15:40:12.867152+0000","last_became_active":"2024-09-16T15:40:12.860306+0000","last_became_peered":"2024-09-16T15:40:12.860306+0000","last_unstale":"2024-09-16T15:40:12.867152+0000","last_undegraded":"2024-09-16T15:40:12.867152+0000","last_fullsized":"2024-09-16T15:40:12.867152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:38.251940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875035+0000","last_change":"2024-09-16T15:40:12.873790+0000","last_active":"2024-09-16T15:40:12.875035+0000","last_peered":"2024-09-16T15:40:12.875035+0000","last_clean":"2024-09-16T15:40:12.875035+0000","last_became_active":"2024-09-16T15:40:12.873524+0000","last_became_peered":"2024-09-16T15:40:12.873524+0000","last_unstale":"2024-09-16T15:40:12.875035+0000","last_undegraded":"2024-09-16T15:40:12.875035+0000","last_fullsized":"2024-09-16T15:40:12.875035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:48.530545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880065+0000","last_change":"2024-09-16T15:40:12.872623+0000","last_active":"2024-09-16T15:40:12.880065+0000","last_peered":"2024-09-16T15:40:12.880065+0000","last_clean":"2024-09-16T15:40:12.880065+0000","last_became_active":"2024-09-16T15:40:12.872346+0000","last_became_peered":"2024-09-16T15:40:12.872346+0000","last_unstale":"2024-09-16T15:40:12.880065+0000","last_undegraded":"2024-09-16T15:40:12.880065+0000","last_fullsized":"2024-09-16T15:40:12.880065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:45.262493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.885065+0000","last_change":"2024-09-16T15:40:12.882504+0000","last_active":"2024-09-16T15:40:12.885065+0000","last_peered":"2024-09-16T15:40:12.885065+0000","last_clean":"2024-09-16T15:40:12.885065+0000","last_became_active":"2024-09-16T15:40:12.882113+0000","last_became_peered":"2024-09-16T15:40:12.882113+0000","last_unstale":"2024-09-16T15:40:12.885065+0000","last_undegraded":"2024-09-16T15:40:12.885065+0000","last_fullsized":"2024-09-16T15:40:12.885065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:35.919623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.869373+0000","last_change":"2024-09-16T15:40:12.857122+0000","last_active":"2024-09-16T15:40:12.869373+0000","last_peered":"2024-09-16T15:40:12.869373+0000","last_clean":"2024-09-16T15:40:12.869373+0000","last_became_active":"2024-09-16T15:40:12.856815+0000","last_became_peered":"2024-09-16T15:40:12.856815+0000","last_unstale":"2024-09-16T15:40:12.869373+0000","last_undegraded":"2024-09-16T15:40:12.869373+0000","last_fullsized":"2024-09-16T15:40:12.869373+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:41.290208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882619+0000","last_change":"2024-09-16T15:40:12.868687+0000","last_active":"2024-09-16T15:40:12.882619+0000","last_peered":"2024-09-16T15:40:12.882619+0000","last_clean":"2024-09-16T15:40:12.882619+0000","last_became_active":"2024-09-16T15:40:12.868063+0000","last_became_peered":"2024-09-16T15:40:12.868063+0000","last_unstale":"2024-09-16T15:40:12.882619+0000","last_undegraded":"2024-09-16T15:40:12.882619+0000","last_fullsized":"2024-09-16T15:40:12.882619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:53.819296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870525+0000","last_change":"2024-09-16T15:40:12.864426+0000","last_active":"2024-09-16T15:40:12.870525+0000","last_peered":"2024-09-16T15:40:12.870525+0000","last_clean":"2024-09-16T15:40:12.870525+0000","last_became_active":"2024-09-16T15:40:12.863762+0000","last_became_peered":"2024-09-16T15:40:12.863762+0000","last_unstale":"2024-09-16T15:40:12.870525+0000","last_undegraded":"2024-09-16T15:40:12.870525+0000","last_fullsized":"2024-09-16T15:40:12.870525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:38.955827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871085+0000","last_change":"2024-09-16T15:40:12.863693+0000","last_active":"2024-09-16T15:40:12.871085+0000","last_peered":"2024-09-16T15:40:12.871085+0000","last_clean":"2024-09-16T15:40:12.871085+0000","last_became_active":"2024-09-16T15:40:12.863383+0000","last_became_peered":"2024-09-16T15:40:12.863383+0000","last_unstale":"2024-09-16T15:40:12.871085+0000","last_undegraded":"2024-09-16T15:40:12.871085+0000","last_fullsized":"2024-09-16T15:40:12.871085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:44.940019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.863957+0000","last_change":"2024-09-16T15:40:12.856737+0000","last_active":"2024-09-16T15:40:12.863957+0000","last_peered":"2024-09-16T15:40:12.863957+0000","last_clean":"2024-09-16T15:40:12.863957+0000","last_became_active":"2024-09-16T15:40:12.856426+0000","last_became_peered":"2024-09-16T15:40:12.856426+0000","last_unstale":"2024-09-16T15:40:12.863957+0000","last_undegraded":"2024-09-16T15:40:12.863957+0000","last_fullsized":"2024-09-16T15:40:12.863957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:58.240556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879492+0000","last_change":"2024-09-16T15:40:12.876499+0000","last_active":"2024-09-16T15:40:12.879492+0000","last_peered":"2024-09-16T15:40:12.879492+0000","last_clean":"2024-09-16T15:40:12.879492+0000","last_became_active":"2024-09-16T15:40:12.876247+0000","last_became_peered":"2024-09-16T15:40:12.876247+0000","last_unstale":"2024-09-16T15:40:12.879492+0000","last_undegraded":"2024-09-16T15:40:12.879492+0000","last_fullsized":"2024-09-16T15:40:12.879492+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:42:34.521097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867366+0000","last_change":"2024-09-16T15:40:12.858149+0000","last_active":"2024-09-16T15:40:12.867366+0000","last_peered":"2024-09-16T15:40:12.867366+0000","last_clean":"2024-09-16T15:40:12.867366+0000","last_became_active":"2024-09-16T15:40:12.857902+0000","last_became_peered":"2024-09-16T15:40:12.857902+0000","last_unstale":"2024-09-16T15:40:12.867366+0000","last_undegraded":"2024-09-16T15:40:12.867366+0000","last_fullsized":"2024-09-16T15:40:12.867366+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:04.893938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870724+0000","last_change":"2024-09-16T15:40:12.865956+0000","last_active":"2024-09-16T15:40:12.870724+0000","last_peered":"2024-09-16T15:40:12.870724+0000","last_clean":"2024-09-16T15:40:12.870724+0000","last_became_active":"2024-09-16T15:40:12.865681+0000","last_became_peered":"2024-09-16T15:40:12.865681+0000","last_unstale":"2024-09-16T15:40:12.870724+0000","last_undegraded":"2024-09-16T15:40:12.870724+0000","last_fullsized":"2024-09-16T15:40:12.870724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:14.857566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.864031+0000","last_change":"2024-09-16T15:40:12.856711+0000","last_active":"2024-09-16T15:40:12.864031+0000","last_peered":"2024-09-16T15:40:12.864031+0000","last_clean":"2024-09-16T15:40:12.864031+0000","last_became_active":"2024-09-16T15:40:12.856290+0000","last_became_peered":"2024-09-16T15:40:12.856290+0000","last_unstale":"2024-09-16T15:40:12.864031+0000","last_undegraded":"2024-09-16T15:40:12.864031+0000","last_fullsized":"2024-09-16T15:40:12.864031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:27.797378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867021+0000","last_change":"2024-09-16T15:40:12.859686+0000","last_active":"2024-09-16T15:40:12.867021+0000","last_peered":"2024-09-16T15:40:12.867021+0000","last_clean":"2024-09-16T15:40:12.867021+0000","last_became_active":"2024-09-16T15:40:12.859399+0000","last_became_peered":"2024-09-16T15:40:12.859399+0000","last_unstale":"2024-09-16T15:40:12.867021+0000","last_undegraded":"2024-09-16T15:40:12.867021+0000","last_fullsized":"2024-09-16T15:40:12.867021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:45.345188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867284+0000","last_change":"2024-09-16T15:40:12.857344+0000","last_active":"2024-09-16T15:40:12.867284+0000","last_peered":"2024-09-16T15:40:12.867284+0000","last_clean":"2024-09-16T15:40:12.867284+0000","last_became_active":"2024-09-16T15:40:12.857044+0000","last_became_peered":"2024-09-16T15:40:12.857044+0000","last_unstale":"2024-09-16T15:40:12.867284+0000","last_undegraded":"2024-09-16T15:40:12.867284+0000","last_fullsized":"2024-09-16T15:40:12.867284+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:28:28.848082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867593+0000","last_change":"2024-09-16T15:40:12.860869+0000","last_active":"2024-09-16T15:40:12.867593+0000","last_peered":"2024-09-16T15:40:12.867593+0000","last_clean":"2024-09-16T15:40:12.867593+0000","last_became_active":"2024-09-16T15:40:12.860241+0000","last_became_peered":"2024-09-16T15:40:12.860241+0000","last_unstale":"2024-09-16T15:40:12.867593+0000","last_undegraded":"2024-09-16T15:40:12.867593+0000","last_fullsized":"2024-09-16T15:40:12.867593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:37.559530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147666+0000","last_change":"2024-09-16T15:40:12.856388+0000","last_active":"2024-09-16T15:40:14.147666+0000","last_peered":"2024-09-16T15:40:14.147666+0000","last_clean":"2024-09-16T15:40:14.147666+0000","last_became_active":"2024-09-16T15:40:12.854943+0000","last_became_peered":"2024-09-16T15:40:12.854943+0000","last_unstale":"2024-09-16T15:40:14.147666+0000","last_undegraded":"2024-09-16T15:40:14.147666+0000","last_fullsized":"2024-09-16T15:40:14.147666+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:10.844833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875888+0000","last_change":"2024-09-16T15:40:11.859931+0000","last_active":"2024-09-16T15:40:12.875888+0000","last_peered":"2024-09-16T15:40:12.875888+0000","last_clean":"2024-09-16T15:40:12.875888+0000","last_became_active":"2024-09-16T15:40:11.859675+0000","last_became_peered":"2024-09-16T15:40:11.859675+0000","last_unstale":"2024-09-16T15:40:12.875888+0000","last_undegraded":"2024-09-16T15:40:12.875888+0000","last_fullsized":"2024-09-16T15:40:12.875888+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:28.283914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867675+0000","last_change":"2024-09-16T15:40:12.861984+0000","last_active":"2024-09-16T15:40:12.867675+0000","last_peered":"2024-09-16T15:40:12.867675+0000","last_clean":"2024-09-16T15:40:12.867675+0000","last_became_active":"2024-09-16T15:40:12.861759+0000","last_became_peered":"2024-09-16T15:40:12.861759+0000","last_unstale":"2024-09-16T15:40:12.867675+0000","last_undegraded":"2024-09-16T15:40:12.867675+0000","last_fullsized":"2024-09-16T15:40:12.867675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:23.370252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880884+0000","last_change":"2024-09-16T15:40:11.863371+0000","last_active":"2024-09-16T15:40:12.880884+0000","last_peered":"2024-09-16T15:40:12.880884+0000","last_clean":"2024-09-16T15:40:12.880884+0000","last_became_active":"2024-09-16T15:40:11.860269+0000","last_became_peered":"2024-09-16T15:40:11.860269+0000","last_unstale":"2024-09-16T15:40:12.880884+0000","last_undegraded":"2024-09-16T15:40:12.880884+0000","last_fullsized":"2024-09-16T15:40:12.880884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:36.282583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.858764+0000","last_change":"2024-09-16T15:40:12.855039+0000","last_active":"2024-09-16T15:40:12.858764+0000","last_peered":"2024-09-16T15:40:12.858764+0000","last_clean":"2024-09-16T15:40:12.858764+0000","last_became_active":"2024-09-16T15:40:12.854141+0000","last_became_peered":"2024-09-16T15:40:12.854141+0000","last_unstale":"2024-09-16T15:40:12.858764+0000","last_undegraded":"2024-09-16T15:40:12.858764+0000","last_fullsized":"2024-09-16T15:40:12.858764+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:26.535001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147699+0000","last_change":"2024-09-16T15:40:11.869853+0000","last_active":"2024-09-16T15:40:14.147699+0000","last_peered":"2024-09-16T15:40:14.147699+0000","last_clean":"2024-09-16T15:40:14.147699+0000","last_became_active":"2024-09-16T15:40:11.869583+0000","last_became_peered":"2024-09-16T15:40:11.869583+0000","last_unstale":"2024-09-16T15:40:14.147699+0000","last_undegraded":"2024-09-16T15:40:14.147699+0000","last_fullsized":"2024-09-16T15:40:14.147699+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:50.423580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147687+0000","last_change":"2024-09-16T15:40:12.853403+0000","last_active":"2024-09-16T15:40:14.147687+0000","last_peered":"2024-09-16T15:40:14.147687+0000","last_clean":"2024-09-16T15:40:14.147687+0000","last_became_active":"2024-09-16T15:40:12.853161+0000","last_became_peered":"2024-09-16T15:40:12.853161+0000","last_unstale":"2024-09-16T15:40:14.147687+0000","last_undegraded":"2024-09-16T15:40:14.147687+0000","last_fullsized":"2024-09-16T15:40:14.147687+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:30.287843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880313+0000","last_change":"2024-09-16T15:40:11.868506+0000","last_active":"2024-09-16T15:40:12.880313+0000","last_peered":"2024-09-16T15:40:12.880313+0000","last_clean":"2024-09-16T15:40:12.880313+0000","last_became_active":"2024-09-16T15:40:11.868014+0000","last_became_peered":"2024-09-16T15:40:11.868014+0000","last_unstale":"2024-09-16T15:40:12.880313+0000","last_undegraded":"2024-09-16T15:40:12.880313+0000","last_fullsized":"2024-09-16T15:40:12.880313+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.292320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874345+0000","last_change":"2024-09-16T15:40:12.867667+0000","last_active":"2024-09-16T15:40:12.874345+0000","last_peered":"2024-09-16T15:40:12.874345+0000","last_clean":"2024-09-16T15:40:12.874345+0000","last_became_active":"2024-09-16T15:40:12.866213+0000","last_became_peered":"2024-09-16T15:40:12.866213+0000","last_unstale":"2024-09-16T15:40:12.874345+0000","last_undegraded":"2024-09-16T15:40:12.874345+0000","last_fullsized":"2024-09-16T15:40:12.874345+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:06.413661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.877703+0000","last_change":"2024-09-16T15:40:11.871006+0000","last_active":"2024-09-16T15:40:12.877703+0000","last_peered":"2024-09-16T15:40:12.877703+0000","last_clean":"2024-09-16T15:40:12.877703+0000","last_became_active":"2024-09-16T15:40:11.870728+0000","last_became_peered":"2024-09-16T15:40:11.870728+0000","last_unstale":"2024-09-16T15:40:12.877703+0000","last_undegraded":"2024-09-16T15:40:12.877703+0000","last_fullsized":"2024-09-16T15:40:12.877703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:26.890614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882663+0000","last_change":"2024-09-16T15:40:12.869124+0000","last_active":"2024-09-16T15:40:12.882663+0000","last_peered":"2024-09-16T15:40:12.882663+0000","last_clean":"2024-09-16T15:40:12.882663+0000","last_became_active":"2024-09-16T15:40:12.868857+0000","last_became_peered":"2024-09-16T15:40:12.868857+0000","last_unstale":"2024-09-16T15:40:12.882663+0000","last_undegraded":"2024-09-16T15:40:12.882663+0000","last_fullsized":"2024-09-16T15:40:12.882663+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:59.975480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879734+0000","last_change":"2024-09-16T15:40:11.865117+0000","last_active":"2024-09-16T15:40:12.879734+0000","last_peered":"2024-09-16T15:40:12.879734+0000","last_clean":"2024-09-16T15:40:12.879734+0000","last_became_active":"2024-09-16T15:40:11.864766+0000","last_became_peered":"2024-09-16T15:40:11.864766+0000","last_unstale":"2024-09-16T15:40:12.879734+0000","last_undegraded":"2024-09-16T15:40:12.879734+0000","last_fullsized":"2024-09-16T15:40:12.879734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:07.999976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880707+0000","last_change":"2024-09-16T15:40:12.860867+0000","last_active":"2024-09-16T15:40:12.880707+0000","last_peered":"2024-09-16T15:40:12.880707+0000","last_clean":"2024-09-16T15:40:12.880707+0000","last_became_active":"2024-09-16T15:40:12.860648+0000","last_became_peered":"2024-09-16T15:40:12.860648+0000","last_unstale":"2024-09-16T15:40:12.880707+0000","last_undegraded":"2024-09-16T15:40:12.880707+0000","last_fullsized":"2024-09-16T15:40:12.880707+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:49.901150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.844800+0000","last_change":"2024-09-16T15:40:11.868600+0000","last_active":"2024-09-16T15:40:12.844800+0000","last_peered":"2024-09-16T15:40:12.844800+0000","last_clean":"2024-09-16T15:40:12.844800+0000","last_became_active":"2024-09-16T15:40:11.868271+0000","last_became_peered":"2024-09-16T15:40:11.868271+0000","last_unstale":"2024-09-16T15:40:12.844800+0000","last_undegraded":"2024-09-16T15:40:12.844800+0000","last_fullsized":"2024-09-16T15:40:12.844800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:56.197404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870499+0000","last_change":"2024-09-16T15:40:12.864230+0000","last_active":"2024-09-16T15:40:12.870499+0000","last_peered":"2024-09-16T15:40:12.870499+0000","last_clean":"2024-09-16T15:40:12.870499+0000","last_became_active":"2024-09-16T15:40:12.863371+0000","last_became_peered":"2024-09-16T15:40:12.863371+0000","last_unstale":"2024-09-16T15:40:12.870499+0000","last_undegraded":"2024-09-16T15:40:12.870499+0000","last_fullsized":"2024-09-16T15:40:12.870499+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:26.451623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.847205+0000","last_change":"2024-09-16T15:40:11.864603+0000","last_active":"2024-09-16T15:40:12.847205+0000","last_peered":"2024-09-16T15:40:12.847205+0000","last_clean":"2024-09-16T15:40:12.847205+0000","last_became_active":"2024-09-16T15:40:11.864256+0000","last_became_peered":"2024-09-16T15:40:11.864256+0000","last_unstale":"2024-09-16T15:40:12.847205+0000","last_undegraded":"2024-09-16T15:40:12.847205+0000","last_fullsized":"2024-09-16T15:40:12.847205+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:34.607613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875009+0000","last_change":"2024-09-16T15:40:12.873524+0000","last_active":"2024-09-16T15:40:12.875009+0000","last_peered":"2024-09-16T15:40:12.875009+0000","last_clean":"2024-09-16T15:40:12.875009+0000","last_became_active":"2024-09-16T15:40:12.873256+0000","last_became_peered":"2024-09-16T15:40:12.873256+0000","last_unstale":"2024-09-16T15:40:12.875009+0000","last_undegraded":"2024-09-16T15:40:12.875009+0000","last_fullsized":"2024-09-16T15:40:12.875009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:15.479730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.851806+0000","last_change":"2024-09-16T15:40:11.860367+0000","last_active":"2024-09-16T15:40:12.851806+0000","last_peered":"2024-09-16T15:40:12.851806+0000","last_clean":"2024-09-16T15:40:12.851806+0000","last_became_active":"2024-09-16T15:40:11.858703+0000","last_became_peered":"2024-09-16T15:40:11.858703+0000","last_unstale":"2024-09-16T15:40:12.851806+0000","last_undegraded":"2024-09-16T15:40:12.851806+0000","last_fullsized":"2024-09-16T15:40:12.851806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:47.251591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.868881+0000","last_change":"2024-09-16T15:40:12.865158+0000","last_active":"2024-09-16T15:40:12.868881+0000","last_peered":"2024-09-16T15:40:12.868881+0000","last_clean":"2024-09-16T15:40:12.868881+0000","last_became_active":"2024-09-16T15:40:12.864675+0000","last_became_peered":"2024-09-16T15:40:12.864675+0000","last_unstale":"2024-09-16T15:40:12.868881+0000","last_undegraded":"2024-09-16T15:40:12.868881+0000","last_fullsized":"2024-09-16T15:40:12.868881+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.849200+0000","last_change":"2024-09-16T15:40:11.863611+0000","last_active":"2024-09-16T15:40:12.849200+0000","last_peered":"2024-09-16T15:40:12.849200+0000","last_clean":"2024-09-16T15:40:12.849200+0000","last_became_active":"2024-09-16T15:40:11.863346+0000","last_became_peered":"2024-09-16T15:40:11.863346+0000","last_unstale":"2024-09-16T15:40:12.849200+0000","last_undegraded":"2024-09-16T15:40:12.849200+0000","last_fullsized":"2024-09-16T15:40:12.849200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:00.681742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879244+0000","last_change":"2024-09-16T15:40:12.876615+0000","last_active":"2024-09-16T15:40:12.879244+0000","last_peered":"2024-09-16T15:40:12.879244+0000","last_clean":"2024-09-16T15:40:12.879244+0000","last_became_active":"2024-09-16T15:40:12.876385+0000","last_became_peered":"2024-09-16T15:40:12.876385+0000","last_unstale":"2024-09-16T15:40:12.879244+0000","last_undegraded":"2024-09-16T15:40:12.879244+0000","last_fullsized":"2024-09-16T15:40:12.879244+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:35.883048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881290+0000","last_change":"2024-09-16T15:40:11.864331+0000","last_active":"2024-09-16T15:40:12.881290+0000","last_peered":"2024-09-16T15:40:12.881290+0000","last_clean":"2024-09-16T15:40:12.881290+0000","last_became_active":"2024-09-16T15:40:11.864083+0000","last_became_peered":"2024-09-16T15:40:11.864083+0000","last_unstale":"2024-09-16T15:40:12.881290+0000","last_undegraded":"2024-09-16T15:40:12.881290+0000","last_fullsized":"2024-09-16T15:40:12.881290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:44.080258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.868336+0000","last_change":"2024-09-16T15:40:12.865124+0000","last_active":"2024-09-16T15:40:12.868336+0000","last_peered":"2024-09-16T15:40:12.868336+0000","last_clean":"2024-09-16T15:40:12.868336+0000","last_became_active":"2024-09-16T15:40:12.864809+0000","last_became_peered":"2024-09-16T15:40:12.864809+0000","last_unstale":"2024-09-16T15:40:12.868336+0000","last_undegraded":"2024-09-16T15:40:12.868336+0000","last_fullsized":"2024-09-16T15:40:12.868336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.885109+0000","last_change":"2024-09-16T15:40:11.865511+0000","last_active":"2024-09-16T15:40:12.885109+0000","last_peered":"2024-09-16T15:40:12.885109+0000","last_clean":"2024-09-16T15:40:12.885109+0000","last_became_active":"2024-09-16T15:40:11.863715+0000","last_became_peered":"2024-09-16T15:40:11.863715+0000","last_unstale":"2024-09-16T15:40:12.885109+0000","last_undegraded":"2024-09-16T15:40:12.885109+0000","last_fullsized":"2024-09-16T15:40:12.885109+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:18.604944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875276+0000","last_change":"2024-09-16T15:40:12.871040+0000","last_active":"2024-09-16T15:40:12.875276+0000","last_peered":"2024-09-16T15:40:12.875276+0000","last_clean":"2024-09-16T15:40:12.875276+0000","last_became_active":"2024-09-16T15:40:12.870459+0000","last_became_peered":"2024-09-16T15:40:12.870459+0000","last_unstale":"2024-09-16T15:40:12.875276+0000","last_undegraded":"2024-09-16T15:40:12.875276+0000","last_fullsized":"2024-09-16T15:40:12.875276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:53.575639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.884353+0000","last_change":"2024-09-16T15:40:11.870606+0000","last_active":"2024-09-16T15:40:12.884353+0000","last_peered":"2024-09-16T15:40:12.884353+0000","last_clean":"2024-09-16T15:40:12.884353+0000","last_became_active":"2024-09-16T15:40:11.868146+0000","last_became_peered":"2024-09-16T15:40:11.868146+0000","last_unstale":"2024-09-16T15:40:12.884353+0000","last_undegraded":"2024-09-16T15:40:12.884353+0000","last_fullsized":"2024-09-16T15:40:12.884353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:36.745807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880157+0000","last_change":"2024-09-16T15:40:12.860151+0000","last_active":"2024-09-16T15:40:12.880157+0000","last_peered":"2024-09-16T15:40:12.880157+0000","last_clean":"2024-09-16T15:40:12.880157+0000","last_became_active":"2024-09-16T15:40:12.859179+0000","last_became_peered":"2024-09-16T15:40:12.859179+0000","last_unstale":"2024-09-16T15:40:12.880157+0000","last_undegraded":"2024-09-16T15:40:12.880157+0000","last_fullsized":"2024-09-16T15:40:12.880157+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:39.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855830+0000","last_change":"2024-09-16T15:40:11.863101+0000","last_active":"2024-09-16T15:40:12.855830+0000","last_peered":"2024-09-16T15:40:12.855830+0000","last_clean":"2024-09-16T15:40:12.855830+0000","last_became_active":"2024-09-16T15:40:11.862829+0000","last_became_peered":"2024-09-16T15:40:11.862829+0000","last_unstale":"2024-09-16T15:40:12.855830+0000","last_undegraded":"2024-09-16T15:40:12.855830+0000","last_fullsized":"2024-09-16T15:40:12.855830+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:25.527579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.876819+0000","last_change":"2024-09-16T15:40:12.871218+0000","last_active":"2024-09-16T15:40:12.876819+0000","last_peered":"2024-09-16T15:40:12.876819+0000","last_clean":"2024-09-16T15:40:12.876819+0000","last_became_active":"2024-09-16T15:40:12.870997+0000","last_became_peered":"2024-09-16T15:40:12.870997+0000","last_unstale":"2024-09-16T15:40:12.876819+0000","last_undegraded":"2024-09-16T15:40:12.876819+0000","last_fullsized":"2024-09-16T15:40:12.876819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:27.010957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.852972+0000","last_change":"2024-09-16T15:40:11.867251+0000","last_active":"2024-09-16T15:40:12.852972+0000","last_peered":"2024-09-16T15:40:12.852972+0000","last_clean":"2024-09-16T15:40:12.852972+0000","last_became_active":"2024-09-16T15:40:11.866730+0000","last_became_peered":"2024-09-16T15:40:11.866730+0000","last_unstale":"2024-09-16T15:40:12.852972+0000","last_undegraded":"2024-09-16T15:40:12.852972+0000","last_fullsized":"2024-09-16T15:40:12.852972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:55.152872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873844+0000","last_change":"2024-09-16T15:40:12.869762+0000","last_active":"2024-09-16T15:40:12.873844+0000","last_peered":"2024-09-16T15:40:12.873844+0000","last_clean":"2024-09-16T15:40:12.873844+0000","last_became_active":"2024-09-16T15:40:12.869420+0000","last_became_peered":"2024-09-16T15:40:12.869420+0000","last_unstale":"2024-09-16T15:40:12.873844+0000","last_undegraded":"2024-09-16T15:40:12.873844+0000","last_fullsized":"2024-09-16T15:40:12.873844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:51.377166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.888763+0000","last_change":"2024-09-16T15:40:11.868078+0000","last_active":"2024-09-16T15:40:12.888763+0000","last_peered":"2024-09-16T15:40:12.888763+0000","last_clean":"2024-09-16T15:40:12.888763+0000","last_became_active":"2024-09-16T15:40:11.867825+0000","last_became_peered":"2024-09-16T15:40:11.867825+0000","last_unstale":"2024-09-16T15:40:12.888763+0000","last_undegraded":"2024-09-16T15:40:12.888763+0000","last_fullsized":"2024-09-16T15:40:12.888763+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:29.103104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872014+0000","last_change":"2024-09-16T15:40:12.867357+0000","last_active":"2024-09-16T15:40:12.872014+0000","last_peered":"2024-09-16T15:40:12.872014+0000","last_clean":"2024-09-16T15:40:12.872014+0000","last_became_active":"2024-09-16T15:40:12.867072+0000","last_became_peered":"2024-09-16T15:40:12.867072+0000","last_unstale":"2024-09-16T15:40:12.872014+0000","last_undegraded":"2024-09-16T15:40:12.872014+0000","last_fullsized":"2024-09-16T15:40:12.872014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:17.395905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.886756+0000","last_change":"2024-09-16T15:40:11.866281+0000","last_active":"2024-09-16T15:40:12.886756+0000","last_peered":"2024-09-16T15:40:12.886756+0000","last_clean":"2024-09-16T15:40:12.886756+0000","last_became_active":"2024-09-16T15:40:11.865322+0000","last_became_peered":"2024-09-16T15:40:11.865322+0000","last_unstale":"2024-09-16T15:40:12.886756+0000","last_undegraded":"2024-09-16T15:40:12.886756+0000","last_fullsized":"2024-09-16T15:40:12.886756+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:14:13.585151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883782+0000","last_change":"2024-09-16T15:40:12.874698+0000","last_active":"2024-09-16T15:40:12.883782+0000","last_peered":"2024-09-16T15:40:12.883782+0000","last_clean":"2024-09-16T15:40:12.883782+0000","last_became_active":"2024-09-16T15:40:12.874345+0000","last_became_peered":"2024-09-16T15:40:12.874345+0000","last_unstale":"2024-09-16T15:40:12.883782+0000","last_undegraded":"2024-09-16T15:40:12.883782+0000","last_fullsized":"2024-09-16T15:40:12.883782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:49.306539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880542+0000","last_change":"2024-09-16T15:40:11.859504+0000","last_active":"2024-09-16T15:40:12.880542+0000","last_peered":"2024-09-16T15:40:12.880542+0000","last_clean":"2024-09-16T15:40:12.880542+0000","last_became_active":"2024-09-16T15:40:11.859183+0000","last_became_peered":"2024-09-16T15:40:11.859183+0000","last_unstale":"2024-09-16T15:40:12.880542+0000","last_undegraded":"2024-09-16T15:40:12.880542+0000","last_fullsized":"2024-09-16T15:40:12.880542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:57.973947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.863982+0000","last_change":"2024-09-16T15:40:12.856434+0000","last_active":"2024-09-16T15:40:12.863982+0000","last_peered":"2024-09-16T15:40:12.863982+0000","last_clean":"2024-09-16T15:40:12.863982+0000","last_became_active":"2024-09-16T15:40:12.856073+0000","last_became_peered":"2024-09-16T15:40:12.856073+0000","last_unstale":"2024-09-16T15:40:12.863982+0000","last_undegraded":"2024-09-16T15:40:12.863982+0000","last_fullsized":"2024-09-16T15:40:12.863982+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:22.248689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.878307+0000","last_change":"2024-09-16T15:40:11.869221+0000","last_active":"2024-09-16T15:40:12.878307+0000","last_peered":"2024-09-16T15:40:12.878307+0000","last_clean":"2024-09-16T15:40:12.878307+0000","last_became_active":"2024-09-16T15:40:11.868892+0000","last_became_peered":"2024-09-16T15:40:11.868892+0000","last_unstale":"2024-09-16T15:40:12.878307+0000","last_undegraded":"2024-09-16T15:40:12.878307+0000","last_fullsized":"2024-09-16T15:40:12.878307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.865500+0000","last_change":"2024-09-16T15:40:12.859954+0000","last_active":"2024-09-16T15:40:12.865500+0000","last_peered":"2024-09-16T15:40:12.865500+0000","last_clean":"2024-09-16T15:40:12.865500+0000","last_became_active":"2024-09-16T15:40:12.859695+0000","last_became_peered":"2024-09-16T15:40:12.859695+0000","last_unstale":"2024-09-16T15:40:12.865500+0000","last_undegraded":"2024-09-16T15:40:12.865500+0000","last_fullsized":"2024-09-16T15:40:12.865500+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:50.839867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.851789+0000","last_change":"2024-09-16T15:40:11.863949+0000","last_active":"2024-09-16T15:40:12.851789+0000","last_peered":"2024-09-16T15:40:12.851789+0000","last_clean":"2024-09-16T15:40:12.851789+0000","last_became_active":"2024-09-16T15:40:11.863620+0000","last_became_peered":"2024-09-16T15:40:11.863620+0000","last_unstale":"2024-09-16T15:40:12.851789+0000","last_undegraded":"2024-09-16T15:40:12.851789+0000","last_fullsized":"2024-09-16T15:40:12.851789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:18.083324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.877926+0000","last_change":"2024-09-16T15:40:12.873993+0000","last_active":"2024-09-16T15:40:12.877926+0000","last_peered":"2024-09-16T15:40:12.877926+0000","last_clean":"2024-09-16T15:40:12.877926+0000","last_became_active":"2024-09-16T15:40:12.873720+0000","last_became_peered":"2024-09-16T15:40:12.873720+0000","last_unstale":"2024-09-16T15:40:12.877926+0000","last_undegraded":"2024-09-16T15:40:12.877926+0000","last_fullsized":"2024-09-16T15:40:12.877926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:47.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880354+0000","last_change":"2024-09-16T15:40:11.871266+0000","last_active":"2024-09-16T15:40:12.880354+0000","last_peered":"2024-09-16T15:40:12.880354+0000","last_clean":"2024-09-16T15:40:12.880354+0000","last_became_active":"2024-09-16T15:40:11.871013+0000","last_became_peered":"2024-09-16T15:40:11.871013+0000","last_unstale":"2024-09-16T15:40:12.880354+0000","last_undegraded":"2024-09-16T15:40:12.880354+0000","last_fullsized":"2024-09-16T15:40:12.880354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:10.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147184+0000","last_change":"2024-09-16T15:40:12.850407+0000","last_active":"2024-09-16T15:40:14.147184+0000","last_peered":"2024-09-16T15:40:14.147184+0000","last_clean":"2024-09-16T15:40:14.147184+0000","last_became_active":"2024-09-16T15:40:12.849693+0000","last_became_peered":"2024-09-16T15:40:12.849693+0000","last_unstale":"2024-09-16T15:40:14.147184+0000","last_undegraded":"2024-09-16T15:40:14.147184+0000","last_fullsized":"2024-09-16T15:40:14.147184+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:48.268979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.845323+0000","last_change":"2024-09-16T15:40:11.865942+0000","last_active":"2024-09-16T15:40:12.845323+0000","last_peered":"2024-09-16T15:40:12.845323+0000","last_clean":"2024-09-16T15:40:12.845323+0000","last_became_active":"2024-09-16T15:40:11.864112+0000","last_became_peered":"2024-09-16T15:40:11.864112+0000","last_unstale":"2024-09-16T15:40:12.845323+0000","last_undegraded":"2024-09-16T15:40:12.845323+0000","last_fullsized":"2024-09-16T15:40:12.845323+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:13.011662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867632+0000","last_change":"2024-09-16T15:40:12.861166+0000","last_active":"2024-09-16T15:40:12.867632+0000","last_peered":"2024-09-16T15:40:12.867632+0000","last_clean":"2024-09-16T15:40:12.867632+0000","last_became_active":"2024-09-16T15:40:12.860619+0000","last_became_peered":"2024-09-16T15:40:12.860619+0000","last_unstale":"2024-09-16T15:40:12.867632+0000","last_undegraded":"2024-09-16T15:40:12.867632+0000","last_fullsized":"2024-09-16T15:40:12.867632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:54.024305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.862595+0000","last_change":"2024-09-16T15:40:11.862288+0000","last_active":"2024-09-16T15:40:12.862595+0000","last_peered":"2024-09-16T15:40:12.862595+0000","last_clean":"2024-09-16T15:40:12.862595+0000","last_became_active":"2024-09-16T15:40:11.862002+0000","last_became_peered":"2024-09-16T15:40:11.862002+0000","last_unstale":"2024-09-16T15:40:12.862595+0000","last_undegraded":"2024-09-16T15:40:12.862595+0000","last_fullsized":"2024-09-16T15:40:12.862595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:12.191215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.884800+0000","last_change":"2024-09-16T15:40:12.879430+0000","last_active":"2024-09-16T15:40:12.884800+0000","last_peered":"2024-09-16T15:40:12.884800+0000","last_clean":"2024-09-16T15:40:12.884800+0000","last_became_active":"2024-09-16T15:40:12.879172+0000","last_became_peered":"2024-09-16T15:40:12.879172+0000","last_unstale":"2024-09-16T15:40:12.884800+0000","last_undegraded":"2024-09-16T15:40:12.884800+0000","last_fullsized":"2024-09-16T15:40:12.884800+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:14.019085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.854691+0000","last_change":"2024-09-16T15:40:11.863062+0000","last_active":"2024-09-16T15:40:12.854691+0000","last_peered":"2024-09-16T15:40:12.854691+0000","last_clean":"2024-09-16T15:40:12.854691+0000","last_became_active":"2024-09-16T15:40:11.860714+0000","last_became_peered":"2024-09-16T15:40:11.860714+0000","last_unstale":"2024-09-16T15:40:12.854691+0000","last_undegraded":"2024-09-16T15:40:12.854691+0000","last_fullsized":"2024-09-16T15:40:12.854691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883782+0000","last_change":"2024-09-16T15:40:12.883082+0000","last_active":"2024-09-16T15:40:12.883782+0000","last_peered":"2024-09-16T15:40:12.883782+0000","last_clean":"2024-09-16T15:40:12.883782+0000","last_became_active":"2024-09-16T15:40:12.882695+0000","last_became_peered":"2024-09-16T15:40:12.882695+0000","last_unstale":"2024-09-16T15:40:12.883782+0000","last_undegraded":"2024-09-16T15:40:12.883782+0000","last_fullsized":"2024-09-16T15:40:12.883782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:21.231163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.853585+0000","last_change":"2024-09-16T15:40:11.867238+0000","last_active":"2024-09-16T15:40:12.853585+0000","last_peered":"2024-09-16T15:40:12.853585+0000","last_clean":"2024-09-16T15:40:12.853585+0000","last_became_active":"2024-09-16T15:40:11.865996+0000","last_became_peered":"2024-09-16T15:40:11.865996+0000","last_unstale":"2024-09-16T15:40:12.853585+0000","last_undegraded":"2024-09-16T15:40:12.853585+0000","last_fullsized":"2024-09-16T15:40:12.853585+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:23.953426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883245+0000","last_change":"2024-09-16T15:40:12.872931+0000","last_active":"2024-09-16T15:40:12.883245+0000","last_peered":"2024-09-16T15:40:12.883245+0000","last_clean":"2024-09-16T15:40:12.883245+0000","last_became_active":"2024-09-16T15:40:12.872344+0000","last_became_peered":"2024-09-16T15:40:12.872344+0000","last_unstale":"2024-09-16T15:40:12.883245+0000","last_undegraded":"2024-09-16T15:40:12.883245+0000","last_fullsized":"2024-09-16T15:40:12.883245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:28.977918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855957+0000","last_change":"2024-09-16T15:40:11.865216+0000","last_active":"2024-09-16T15:40:12.855957+0000","last_peered":"2024-09-16T15:40:12.855957+0000","last_clean":"2024-09-16T15:40:12.855957+0000","last_became_active":"2024-09-16T15:40:11.864945+0000","last_became_peered":"2024-09-16T15:40:11.864945+0000","last_unstale":"2024-09-16T15:40:12.855957+0000","last_undegraded":"2024-09-16T15:40:12.855957+0000","last_fullsized":"2024-09-16T15:40:12.855957+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:13.850144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873339+0000","last_change":"2024-09-16T15:40:12.868439+0000","last_active":"2024-09-16T15:40:12.873339+0000","last_peered":"2024-09-16T15:40:12.873339+0000","last_clean":"2024-09-16T15:40:12.873339+0000","last_became_active":"2024-09-16T15:40:12.867651+0000","last_became_peered":"2024-09-16T15:40:12.867651+0000","last_unstale":"2024-09-16T15:40:12.873339+0000","last_undegraded":"2024-09-16T15:40:12.873339+0000","last_fullsized":"2024-09-16T15:40:12.873339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:12.149041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146873+0000","last_change":"2024-09-16T15:40:11.869272+0000","last_active":"2024-09-16T15:40:14.146873+0000","last_peered":"2024-09-16T15:40:14.146873+0000","last_clean":"2024-09-16T15:40:14.146873+0000","last_became_active":"2024-09-16T15:40:11.868909+0000","last_became_peered":"2024-09-16T15:40:11.868909+0000","last_unstale":"2024-09-16T15:40:14.146873+0000","last_undegraded":"2024-09-16T15:40:14.146873+0000","last_fullsized":"2024-09-16T15:40:14.146873+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:57.684406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874248+0000","last_change":"2024-09-16T15:40:12.862103+0000","last_active":"2024-09-16T15:40:12.874248+0000","last_peered":"2024-09-16T15:40:12.874248+0000","last_clean":"2024-09-16T15:40:12.874248+0000","last_became_active":"2024-09-16T15:40:12.861788+0000","last_became_peered":"2024-09-16T15:40:12.861788+0000","last_unstale":"2024-09-16T15:40:12.874248+0000","last_undegraded":"2024-09-16T15:40:12.874248+0000","last_fullsized":"2024-09-16T15:40:12.874248+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:38.051709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855069+0000","last_change":"2024-09-16T15:40:11.867570+0000","last_active":"2024-09-16T15:40:12.855069+0000","last_peered":"2024-09-16T15:40:12.855069+0000","last_clean":"2024-09-16T15:40:12.855069+0000","last_became_active":"2024-09-16T15:40:11.867301+0000","last_became_peered":"2024-09-16T15:40:11.867301+0000","last_unstale":"2024-09-16T15:40:12.855069+0000","last_undegraded":"2024-09-16T15:40:12.855069+0000","last_fullsized":"2024-09-16T15:40:12.855069+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879331+0000","last_change":"2024-09-16T15:40:12.875892+0000","last_active":"2024-09-16T15:40:12.879331+0000","last_peered":"2024-09-16T15:40:12.879331+0000","last_clean":"2024-09-16T15:40:12.879331+0000","last_became_active":"2024-09-16T15:40:12.875619+0000","last_became_peered":"2024-09-16T15:40:12.875619+0000","last_unstale":"2024-09-16T15:40:12.879331+0000","last_undegraded":"2024-09-16T15:40:12.879331+0000","last_fullsized":"2024-09-16T15:40:12.879331+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:15.128749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.849889+0000","last_change":"2024-09-16T15:40:11.867392+0000","last_active":"2024-09-16T15:40:12.849889+0000","last_peered":"2024-09-16T15:40:12.849889+0000","last_clean":"2024-09-16T15:40:12.849889+0000","last_became_active":"2024-09-16T15:40:11.867003+0000","last_became_peered":"2024-09-16T15:40:11.867003+0000","last_unstale":"2024-09-16T15:40:12.849889+0000","last_undegraded":"2024-09-16T15:40:12.849889+0000","last_fullsized":"2024-09-16T15:40:12.849889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:00.706403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.857550+0000","last_change":"2024-09-16T15:40:12.850142+0000","last_active":"2024-09-16T15:40:12.857550+0000","last_peered":"2024-09-16T15:40:12.857550+0000","last_clean":"2024-09-16T15:40:12.857550+0000","last_became_active":"2024-09-16T15:40:12.849859+0000","last_became_peered":"2024-09-16T15:40:12.849859+0000","last_unstale":"2024-09-16T15:40:12.857550+0000","last_undegraded":"2024-09-16T15:40:12.857550+0000","last_fullsized":"2024-09-16T15:40:12.857550+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:45.087681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146222+0000","last_change":"2024-09-16T15:40:11.860188+0000","last_active":"2024-09-16T15:40:14.146222+0000","last_peered":"2024-09-16T15:40:14.146222+0000","last_clean":"2024-09-16T15:40:14.146222+0000","last_became_active":"2024-09-16T15:40:11.859924+0000","last_became_peered":"2024-09-16T15:40:11.859924+0000","last_unstale":"2024-09-16T15:40:14.146222+0000","last_undegraded":"2024-09-16T15:40:14.146222+0000","last_fullsized":"2024-09-16T15:40:14.146222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:47.670989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867386+0000","last_change":"2024-09-16T15:40:12.862284+0000","last_active":"2024-09-16T15:40:12.867386+0000","last_peered":"2024-09-16T15:40:12.867386+0000","last_clean":"2024-09-16T15:40:12.867386+0000","last_became_active":"2024-09-16T15:40:12.861890+0000","last_became_peered":"2024-09-16T15:40:12.861890+0000","last_unstale":"2024-09-16T15:40:12.867386+0000","last_undegraded":"2024-09-16T15:40:12.867386+0000","last_fullsized":"2024-09-16T15:40:12.867386+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:52.757331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.847519+0000","last_change":"2024-09-16T15:40:11.869696+0000","last_active":"2024-09-16T15:40:12.847519+0000","last_peered":"2024-09-16T15:40:12.847519+0000","last_clean":"2024-09-16T15:40:12.847519+0000","last_became_active":"2024-09-16T15:40:11.869403+0000","last_became_peered":"2024-09-16T15:40:11.869403+0000","last_unstale":"2024-09-16T15:40:12.847519+0000","last_undegraded":"2024-09-16T15:40:12.847519+0000","last_fullsized":"2024-09-16T15:40:12.847519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:53.785910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871439+0000","last_change":"2024-09-16T15:40:12.859022+0000","last_active":"2024-09-16T15:40:12.871439+0000","last_peered":"2024-09-16T15:40:12.871439+0000","last_clean":"2024-09-16T15:40:12.871439+0000","last_became_active":"2024-09-16T15:40:12.858683+0000","last_became_peered":"2024-09-16T15:40:12.858683+0000","last_unstale":"2024-09-16T15:40:12.871439+0000","last_undegraded":"2024-09-16T15:40:12.871439+0000","last_fullsized":"2024-09-16T15:40:12.871439+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:46.347083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881285+0000","last_change":"2024-09-16T15:40:11.872583+0000","last_active":"2024-09-16T15:40:12.881285+0000","last_peered":"2024-09-16T15:40:12.881285+0000","last_clean":"2024-09-16T15:40:12.881285+0000","last_became_active":"2024-09-16T15:40:11.872290+0000","last_became_peered":"2024-09-16T15:40:11.872290+0000","last_unstale":"2024-09-16T15:40:12.881285+0000","last_undegraded":"2024-09-16T15:40:12.881285+0000","last_fullsized":"2024-09-16T15:40:12.881285+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.854788+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:40:12.854788+0000","last_peered":"2024-09-16T15:40:12.854788+0000","last_clean":"2024-09-16T15:40:12.854788+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:40:12.854788+0000","last_undegraded":"2024-09-16T15:40:12.854788+0000","last_fullsized":"2024-09-16T15:40:12.854788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1126400,"data_stored":802873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":548864,"data_stored":219862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":219350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":802873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":561152,"data_stored":225650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:18.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:40:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.178+0000 7f48a3200640 1 -- 172.21.15.60:0/3372664194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a40ccba0 msgr2=0x7f48a4073280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.178+0000 7f48a3200640 1 --2- 172.21.15.60:0/3372664194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a40ccba0 0x7f48a4073280 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f48a4070580 tx=0x7f489c02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.179+0000 7f48a3200640 1 -- 172.21.15.60:0/3372664194 shutdown_connections 2024-09-16T15:40:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.179+0000 7f48a3200640 1 --2- 172.21.15.60:0/3372664194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a40ccba0 0x7f48a4073280 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.179+0000 7f48a3200640 1 --2- 172.21.15.60:0/3372664194 >> v2:172.21.15.100:3300/0 conn(0x7f48a406e9b0 0x7f48a40cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.179+0000 7f48a3200640 1 -- 172.21.15.60:0/3372664194 >> 172.21.15.60:0/3372664194 conn(0x7f48a40b18e0 msgr2=0x7f48a40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:18.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.179+0000 7f48a3200640 1 -- 172.21.15.60:0/3372664194 shutdown_connections 2024-09-16T15:40:18.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.179+0000 7f48a3200640 1 -- 172.21.15.60:0/3372664194 wait complete. 2024-09-16T15:40:18.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.180+0000 7f48a3200640 1 Processor -- start 2024-09-16T15:40:18.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.180+0000 7f48a3200640 1 -- start start 2024-09-16T15:40:18.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.181+0000 7f48a3200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a406e9b0 0x7f48a409d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.181+0000 7f48a3200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40ccba0 0x7f48a409b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.181+0000 7f48a3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41563a0 0x7f48a40cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.181+0000 7f48a3200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4073b50 con 0x7f48a406e9b0 2024-09-16T15:40:18.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.181+0000 7f48a3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f48a40739d0 con 0x7f48a41563a0 2024-09-16T15:40:18.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.181+0000 7f48a3200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f48a4073cd0 con 0x7f48a40ccba0 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.181+0000 7f48a1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a406e9b0 0x7f48a409d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a406e9b0 0x7f48a409d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37470/0 (socket says 172.21.15.60:37470) 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a1e00640 1 -- 172.21.15.60:0/1495091193 learned_addr learned my addr 172.21.15.60:0/1495091193 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a2800640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41563a0 0x7f48a40cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f489be00640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40ccba0 0x7f48a409b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a1e00640 1 -- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40ccba0 msgr2=0x7f48a409b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a1e00640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40ccba0 0x7f48a409b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a1e00640 1 -- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41563a0 msgr2=0x7f48a40cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a1e00640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41563a0 0x7f48a40cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a1e00640 1 -- 172.21.15.60:0/1495091193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a41f3140 con 0x7f48a406e9b0 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a2800640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41563a0 0x7f48a40cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f48a1e00640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a406e9b0 0x7f48a409d8c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f489400c3a0 tx=0x7f489400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:18.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.182+0000 7f489a000640 1 -- 172.21.15.60:0/1495091193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894011790 con 0x7f48a406e9b0 2024-09-16T15:40:18.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.183+0000 7f489a000640 1 -- 172.21.15.60:0/1495091193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4894017070 con 0x7f48a406e9b0 2024-09-16T15:40:18.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.183+0000 7f489a000640 1 -- 172.21.15.60:0/1495091193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894011cc0 con 0x7f48a406e9b0 2024-09-16T15:40:18.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.183+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a409cde0 con 0x7f48a406e9b0 2024-09-16T15:40:18.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.183+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a409d3a0 con 0x7f48a406e9b0 2024-09-16T15:40:18.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.184+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a40cc660 con 0x7f48a406e9b0 2024-09-16T15:40:18.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.185+0000 7f489a000640 1 -- 172.21.15.60:0/1495091193 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f489401c460 con 0x7f48a406e9b0 2024-09-16T15:40:18.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.186+0000 7f489a000640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4880078a00 0x7f488007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.187+0000 7f489a000640 1 -- 172.21.15.60:0/1495091193 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f489409ab60 con 0x7f48a406e9b0 2024-09-16T15:40:18.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.187+0000 7f489be00640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4880078a00 0x7f488007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.188+0000 7f489be00640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4880078a00 0x7f488007aec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f488c005e10 tx=0x7f488c005da0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:18.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.191+0000 7f489a000640 1 -- 172.21.15.60:0/1495091193 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4894062da0 con 0x7f48a406e9b0 2024-09-16T15:40:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.459+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f48a4070ae0 con 0x7f48a406e9b0 2024-09-16T15:40:18.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.460+0000 7f489a000640 1 -- 172.21.15.60:0/1495091193 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1603 (secure 0 0 0) 0x7f48940626a0 con 0x7f48a406e9b0 2024-09-16T15:40:18.461 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:18.461 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":129,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501102,"num_in_osds":8,"osd_in_since":1726501092,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":244002816,"bytes_avail":772850110464,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":26,"btime":"2024-09-16T15:40:15:858549+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5310}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-16T15:39:23.720301+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T15:40:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4880078a00 msgr2=0x7f488007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4880078a00 0x7f488007aec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f488c005e10 tx=0x7f488c005da0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a406e9b0 msgr2=0x7f48a409d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a406e9b0 0x7f48a409d8c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f489400c3a0 tx=0x7f489400c860 comp rx=0 tx=0).stop 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 shutdown_connections 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4880078a00 0x7f488007aec0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a41563a0 0x7f48a40cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a40ccba0 0x7f48a409b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 --2- 172.21.15.60:0/1495091193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a406e9b0 0x7f48a409d8c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 >> 172.21.15.60:0/1495091193 conn(0x7f48a40b18e0 msgr2=0x7f48a415e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 shutdown_connections 2024-09-16T15:40:18.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.462+0000 7f48a3200640 1 -- 172.21.15.60:0/1495091193 wait complete. 2024-09-16T15:40:18.479 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:40:18.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:18.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.639+0000 7fc5fee00640 1 -- 172.21.15.60:0/3280619041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5f8073fb0 msgr2=0x7fc5f8074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.639+0000 7fc5fee00640 1 --2- 172.21.15.60:0/3280619041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5f8073fb0 0x7fc5f8074440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0005fd0 tx=0x7fc5f002a820 comp rx=0 tx=0).stop 2024-09-16T15:40:18.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.639+0000 7fc5fee00640 1 -- 172.21.15.60:0/3280619041 shutdown_connections 2024-09-16T15:40:18.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.639+0000 7fc5fee00640 1 --2- 172.21.15.60:0/3280619041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5f8073fb0 0x7fc5f8074440 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.639+0000 7fc5fee00640 1 --2- 172.21.15.60:0/3280619041 >> v2:172.21.15.100:3300/0 conn(0x7fc5f815f5a0 0x7fc5f80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.639+0000 7fc5fee00640 1 -- 172.21.15.60:0/3280619041 >> 172.21.15.60:0/3280619041 conn(0x7fc5f80b1950 msgr2=0x7fc5f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:18.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.639+0000 7fc5fee00640 1 -- 172.21.15.60:0/3280619041 shutdown_connections 2024-09-16T15:40:18.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.639+0000 7fc5fee00640 1 -- 172.21.15.60:0/3280619041 wait complete. 2024-09-16T15:40:18.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.641+0000 7fc5fee00640 1 Processor -- start 2024-09-16T15:40:18.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.641+0000 7fc5fee00640 1 -- start start 2024-09-16T15:40:18.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5fee00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5f8073fb0 0x7fc5f80cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5fee00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f80ccf00 0x7fc5f81537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5fee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f815f5a0 0x7fc5f8159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5fee00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f806f2e0 con 0x7fc5f80ccf00 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5fee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f806f160 con 0x7fc5f815f5a0 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5fee00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f806f460 con 0x7fc5f8073fb0 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f80ccf00 0x7fc5f81537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f80ccf00 0x7fc5f81537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37482/0 (socket says 172.21.15.60:37482) 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7400640 1 -- 172.21.15.60:0/825236674 learned_addr learned my addr 172.21.15.60:0/825236674 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7e00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5f8073fb0 0x7fc5f80cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5fce00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f815f5a0 0x7fc5f8159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7400640 1 -- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5f8073fb0 msgr2=0x7fc5f80cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7400640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5f8073fb0 0x7fc5f80cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7400640 1 -- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f815f5a0 msgr2=0x7fc5f8159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7400640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f815f5a0 0x7fc5f8159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5f7400640 1 -- 172.21.15.60:0/825236674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f80cc200 con 0x7fc5f80ccf00 2024-09-16T15:40:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.642+0000 7fc5fce00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f815f5a0 0x7fc5f8159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.643+0000 7fc5f7400640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f80ccf00 0x7fc5f81537c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c450 tx=0x7fc5e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.643+0000 7fc5efe00640 1 -- 172.21.15.60:0/825236674 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40052d0 con 0x7fc5f80ccf00 2024-09-16T15:40:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.643+0000 7fc5efe00640 1 -- 172.21.15.60:0/825236674 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5e4005470 con 0x7fc5f80ccf00 2024-09-16T15:40:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.643+0000 7fc5efe00640 1 -- 172.21.15.60:0/825236674 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4012870 con 0x7fc5f80ccf00 2024-09-16T15:40:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.643+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f80cc480 con 0x7fc5f80ccf00 2024-09-16T15:40:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.643+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f8159030 con 0x7fc5f80ccf00 2024-09-16T15:40:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.644+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f806f9f0 con 0x7fc5f80ccf00 2024-09-16T15:40:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.645+0000 7fc5efe00640 1 -- 172.21.15.60:0/825236674 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fc5e40035f0 con 0x7fc5f80ccf00 2024-09-16T15:40:18.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.646+0000 7fc5efe00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc5d8078a00 0x7fc5d807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.646+0000 7fc5efe00640 1 -- 172.21.15.60:0/825236674 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fc5e409a7f0 con 0x7fc5f80ccf00 2024-09-16T15:40:18.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.647+0000 7fc5f7e00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc5d8078a00 0x7fc5d807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.647+0000 7fc5f7e00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc5d8078a00 0x7fc5d807aec0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80096c0 tx=0x7fc5e8009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:18.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.651+0000 7fc5efe00640 1 -- 172.21.15.60:0/825236674 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc5f80cc200 con 0x7fc5f80ccf00 2024-09-16T15:40:18.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.845+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5f80cf100 con 0x7fc5d8078a00 2024-09-16T15:40:18.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.850+0000 7fc5efe00640 1 -- 172.21.15.60:0/825236674 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7fc5f80cf100 con 0x7fc5d8078a00 2024-09-16T15:40:18.854 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:18.855 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:40:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc5d8078a00 msgr2=0x7fc5d807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc5d8078a00 0x7fc5d807aec0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80096c0 tx=0x7fc5e8009260 comp rx=0 tx=0).stop 2024-09-16T15:40:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f80ccf00 msgr2=0x7fc5f81537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f80ccf00 0x7fc5f81537c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c450 tx=0x7fc5e400c910 comp rx=0 tx=0).stop 2024-09-16T15:40:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 shutdown_connections 2024-09-16T15:40:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc5d8078a00 0x7fc5d807aec0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5f815f5a0 0x7fc5f8159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5f80ccf00 0x7fc5f81537c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 --2- 172.21.15.60:0/825236674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5f8073fb0 0x7fc5f80cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 >> 172.21.15.60:0/825236674 conn(0x7fc5f80b1950 msgr2=0x7fc5f80ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 shutdown_connections 2024-09-16T15:40:18.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:18.854+0000 7fc5fee00640 1 -- 172.21.15.60:0/825236674 wait complete. 2024-09-16T15:40:18.871 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-16T15:40:17.734369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238284,"kb_used_data":5380,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736436,"statfs":{"total":773094113280,"available":772850110464,"internally_reserved":0,"allocated":5509120,"data_stored":2908123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001649"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872674+0000","last_change":"2024-09-16T15:40:12.868607+0000","last_active":"2024-09-16T15:40:12.872674+0000","last_peered":"2024-09-16T15:40:12.872674+0000","last_clean":"2024-09-16T15:40:12.872674+0000","last_became_active":"2024-09-16T15:40:12.868140+0000","last_became_peered":"2024-09-16T15:40:12.868140+0000","last_unstale":"2024-09-16T15:40:12.872674+0000","last_undegraded":"2024-09-16T15:40:12.872674+0000","last_fullsized":"2024-09-16T15:40:12.872674+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:29.669859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872203+0000","last_change":"2024-09-16T15:40:12.869358+0000","last_active":"2024-09-16T15:40:12.872203+0000","last_peered":"2024-09-16T15:40:12.872203+0000","last_clean":"2024-09-16T15:40:12.872203+0000","last_became_active":"2024-09-16T15:40:12.868998+0000","last_became_peered":"2024-09-16T15:40:12.868998+0000","last_unstale":"2024-09-16T15:40:12.872203+0000","last_undegraded":"2024-09-16T15:40:12.872203+0000","last_fullsized":"2024-09-16T15:40:12.872203+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874339+0000","last_change":"2024-09-16T15:40:12.868681+0000","last_active":"2024-09-16T15:40:12.874339+0000","last_peered":"2024-09-16T15:40:12.874339+0000","last_clean":"2024-09-16T15:40:12.874339+0000","last_became_active":"2024-09-16T15:40:12.867980+0000","last_became_peered":"2024-09-16T15:40:12.867980+0000","last_unstale":"2024-09-16T15:40:12.874339+0000","last_undegraded":"2024-09-16T15:40:12.874339+0000","last_fullsized":"2024-09-16T15:40:12.874339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:39.160061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883276+0000","last_change":"2024-09-16T15:40:12.873202+0000","last_active":"2024-09-16T15:40:12.883276+0000","last_peered":"2024-09-16T15:40:12.883276+0000","last_clean":"2024-09-16T15:40:12.883276+0000","last_became_active":"2024-09-16T15:40:12.872682+0000","last_became_peered":"2024-09-16T15:40:12.872682+0000","last_unstale":"2024-09-16T15:40:12.883276+0000","last_undegraded":"2024-09-16T15:40:12.883276+0000","last_fullsized":"2024-09-16T15:40:12.883276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:23.455972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874812+0000","last_change":"2024-09-16T15:40:12.870112+0000","last_active":"2024-09-16T15:40:12.874812+0000","last_peered":"2024-09-16T15:40:12.874812+0000","last_clean":"2024-09-16T15:40:12.874812+0000","last_became_active":"2024-09-16T15:40:12.869753+0000","last_became_peered":"2024-09-16T15:40:12.869753+0000","last_unstale":"2024-09-16T15:40:12.874812+0000","last_undegraded":"2024-09-16T15:40:12.874812+0000","last_fullsized":"2024-09-16T15:40:12.874812+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875406+0000","last_change":"2024-09-16T15:40:12.866164+0000","last_active":"2024-09-16T15:40:12.875406+0000","last_peered":"2024-09-16T15:40:12.875406+0000","last_clean":"2024-09-16T15:40:12.875406+0000","last_became_active":"2024-09-16T15:40:12.865904+0000","last_became_peered":"2024-09-16T15:40:12.865904+0000","last_unstale":"2024-09-16T15:40:12.875406+0000","last_undegraded":"2024-09-16T15:40:12.875406+0000","last_fullsized":"2024-09-16T15:40:12.875406+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:44.336056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.869132+0000","last_change":"2024-09-16T15:40:12.865180+0000","last_active":"2024-09-16T15:40:12.869132+0000","last_peered":"2024-09-16T15:40:12.869132+0000","last_clean":"2024-09-16T15:40:12.869132+0000","last_became_active":"2024-09-16T15:40:12.864842+0000","last_became_peered":"2024-09-16T15:40:12.864842+0000","last_unstale":"2024-09-16T15:40:12.869132+0000","last_undegraded":"2024-09-16T15:40:12.869132+0000","last_fullsized":"2024-09-16T15:40:12.869132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:36.820991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880732+0000","last_change":"2024-09-16T15:40:12.861554+0000","last_active":"2024-09-16T15:40:12.880732+0000","last_peered":"2024-09-16T15:40:12.880732+0000","last_clean":"2024-09-16T15:40:12.880732+0000","last_became_active":"2024-09-16T15:40:12.861215+0000","last_became_peered":"2024-09-16T15:40:12.861215+0000","last_unstale":"2024-09-16T15:40:12.880732+0000","last_undegraded":"2024-09-16T15:40:12.880732+0000","last_fullsized":"2024-09-16T15:40:12.880732+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:16.169482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882156+0000","last_change":"2024-09-16T15:40:12.864597+0000","last_active":"2024-09-16T15:40:12.882156+0000","last_peered":"2024-09-16T15:40:12.882156+0000","last_clean":"2024-09-16T15:40:12.882156+0000","last_became_active":"2024-09-16T15:40:12.864345+0000","last_became_peered":"2024-09-16T15:40:12.864345+0000","last_unstale":"2024-09-16T15:40:12.882156+0000","last_undegraded":"2024-09-16T15:40:12.882156+0000","last_fullsized":"2024-09-16T15:40:12.882156+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:43.902508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882148+0000","last_change":"2024-09-16T15:40:12.865537+0000","last_active":"2024-09-16T15:40:12.882148+0000","last_peered":"2024-09-16T15:40:12.882148+0000","last_clean":"2024-09-16T15:40:12.882148+0000","last_became_active":"2024-09-16T15:40:12.865293+0000","last_became_peered":"2024-09-16T15:40:12.865293+0000","last_unstale":"2024-09-16T15:40:12.882148+0000","last_undegraded":"2024-09-16T15:40:12.882148+0000","last_fullsized":"2024-09-16T15:40:12.882148+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:15.882606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883443+0000","last_change":"2024-09-16T15:40:12.875755+0000","last_active":"2024-09-16T15:40:12.883443+0000","last_peered":"2024-09-16T15:40:12.883443+0000","last_clean":"2024-09-16T15:40:12.883443+0000","last_became_active":"2024-09-16T15:40:12.875484+0000","last_became_peered":"2024-09-16T15:40:12.875484+0000","last_unstale":"2024-09-16T15:40:12.883443+0000","last_undegraded":"2024-09-16T15:40:12.883443+0000","last_fullsized":"2024-09-16T15:40:12.883443+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:57.204826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870792+0000","last_change":"2024-09-16T15:40:12.866408+0000","last_active":"2024-09-16T15:40:12.870792+0000","last_peered":"2024-09-16T15:40:12.870792+0000","last_clean":"2024-09-16T15:40:12.870792+0000","last_became_active":"2024-09-16T15:40:12.865787+0000","last_became_peered":"2024-09-16T15:40:12.865787+0000","last_unstale":"2024-09-16T15:40:12.870792+0000","last_undegraded":"2024-09-16T15:40:12.870792+0000","last_fullsized":"2024-09-16T15:40:12.870792+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:39.299743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867785+0000","last_change":"2024-09-16T15:40:12.865889+0000","last_active":"2024-09-16T15:40:12.867785+0000","last_peered":"2024-09-16T15:40:12.867785+0000","last_clean":"2024-09-16T15:40:12.867785+0000","last_became_active":"2024-09-16T15:40:12.865611+0000","last_became_peered":"2024-09-16T15:40:12.865611+0000","last_unstale":"2024-09-16T15:40:12.867785+0000","last_undegraded":"2024-09-16T15:40:12.867785+0000","last_fullsized":"2024-09-16T15:40:12.867785+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:28.736859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881374+0000","last_change":"2024-09-16T15:40:12.863378+0000","last_active":"2024-09-16T15:40:12.881374+0000","last_peered":"2024-09-16T15:40:12.881374+0000","last_clean":"2024-09-16T15:40:12.881374+0000","last_became_active":"2024-09-16T15:40:12.863108+0000","last_became_peered":"2024-09-16T15:40:12.863108+0000","last_unstale":"2024-09-16T15:40:12.881374+0000","last_undegraded":"2024-09-16T15:40:12.881374+0000","last_fullsized":"2024-09-16T15:40:12.881374+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:34.731306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873670+0000","last_change":"2024-09-16T15:40:12.861959+0000","last_active":"2024-09-16T15:40:12.873670+0000","last_peered":"2024-09-16T15:40:12.873670+0000","last_clean":"2024-09-16T15:40:12.873670+0000","last_became_active":"2024-09-16T15:40:12.861657+0000","last_became_peered":"2024-09-16T15:40:12.861657+0000","last_unstale":"2024-09-16T15:40:12.873670+0000","last_undegraded":"2024-09-16T15:40:12.873670+0000","last_fullsized":"2024-09-16T15:40:12.873670+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:55.582229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871103+0000","last_change":"2024-09-16T15:40:12.865876+0000","last_active":"2024-09-16T15:40:12.871103+0000","last_peered":"2024-09-16T15:40:12.871103+0000","last_clean":"2024-09-16T15:40:12.871103+0000","last_became_active":"2024-09-16T15:40:12.865493+0000","last_became_peered":"2024-09-16T15:40:12.865493+0000","last_unstale":"2024-09-16T15:40:12.871103+0000","last_undegraded":"2024-09-16T15:40:12.871103+0000","last_fullsized":"2024-09-16T15:40:12.871103+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:44.257723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867152+0000","last_change":"2024-09-16T15:40:12.861173+0000","last_active":"2024-09-16T15:40:12.867152+0000","last_peered":"2024-09-16T15:40:12.867152+0000","last_clean":"2024-09-16T15:40:12.867152+0000","last_became_active":"2024-09-16T15:40:12.860306+0000","last_became_peered":"2024-09-16T15:40:12.860306+0000","last_unstale":"2024-09-16T15:40:12.867152+0000","last_undegraded":"2024-09-16T15:40:12.867152+0000","last_fullsized":"2024-09-16T15:40:12.867152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:38.251940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875035+0000","last_change":"2024-09-16T15:40:12.873790+0000","last_active":"2024-09-16T15:40:12.875035+0000","last_peered":"2024-09-16T15:40:12.875035+0000","last_clean":"2024-09-16T15:40:12.875035+0000","last_became_active":"2024-09-16T15:40:12.873524+0000","last_became_peered":"2024-09-16T15:40:12.873524+0000","last_unstale":"2024-09-16T15:40:12.875035+0000","last_undegraded":"2024-09-16T15:40:12.875035+0000","last_fullsized":"2024-09-16T15:40:12.875035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:48.530545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880065+0000","last_change":"2024-09-16T15:40:12.872623+0000","last_active":"2024-09-16T15:40:12.880065+0000","last_peered":"2024-09-16T15:40:12.880065+0000","last_clean":"2024-09-16T15:40:12.880065+0000","last_became_active":"2024-09-16T15:40:12.872346+0000","last_became_peered":"2024-09-16T15:40:12.872346+0000","last_unstale":"2024-09-16T15:40:12.880065+0000","last_undegraded":"2024-09-16T15:40:12.880065+0000","last_fullsized":"2024-09-16T15:40:12.880065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:45.262493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.885065+0000","last_change":"2024-09-16T15:40:12.882504+0000","last_active":"2024-09-16T15:40:12.885065+0000","last_peered":"2024-09-16T15:40:12.885065+0000","last_clean":"2024-09-16T15:40:12.885065+0000","last_became_active":"2024-09-16T15:40:12.882113+0000","last_became_peered":"2024-09-16T15:40:12.882113+0000","last_unstale":"2024-09-16T15:40:12.885065+0000","last_undegraded":"2024-09-16T15:40:12.885065+0000","last_fullsized":"2024-09-16T15:40:12.885065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:35.919623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.869373+0000","last_change":"2024-09-16T15:40:12.857122+0000","last_active":"2024-09-16T15:40:12.869373+0000","last_peered":"2024-09-16T15:40:12.869373+0000","last_clean":"2024-09-16T15:40:12.869373+0000","last_became_active":"2024-09-16T15:40:12.856815+0000","last_became_peered":"2024-09-16T15:40:12.856815+0000","last_unstale":"2024-09-16T15:40:12.869373+0000","last_undegraded":"2024-09-16T15:40:12.869373+0000","last_fullsized":"2024-09-16T15:40:12.869373+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:41.290208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882619+0000","last_change":"2024-09-16T15:40:12.868687+0000","last_active":"2024-09-16T15:40:12.882619+0000","last_peered":"2024-09-16T15:40:12.882619+0000","last_clean":"2024-09-16T15:40:12.882619+0000","last_became_active":"2024-09-16T15:40:12.868063+0000","last_became_peered":"2024-09-16T15:40:12.868063+0000","last_unstale":"2024-09-16T15:40:12.882619+0000","last_undegraded":"2024-09-16T15:40:12.882619+0000","last_fullsized":"2024-09-16T15:40:12.882619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:53.819296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870525+0000","last_change":"2024-09-16T15:40:12.864426+0000","last_active":"2024-09-16T15:40:12.870525+0000","last_peered":"2024-09-16T15:40:12.870525+0000","last_clean":"2024-09-16T15:40:12.870525+0000","last_became_active":"2024-09-16T15:40:12.863762+0000","last_became_peered":"2024-09-16T15:40:12.863762+0000","last_unstale":"2024-09-16T15:40:12.870525+0000","last_undegraded":"2024-09-16T15:40:12.870525+0000","last_fullsized":"2024-09-16T15:40:12.870525+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:38.955827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871085+0000","last_change":"2024-09-16T15:40:12.863693+0000","last_active":"2024-09-16T15:40:12.871085+0000","last_peered":"2024-09-16T15:40:12.871085+0000","last_clean":"2024-09-16T15:40:12.871085+0000","last_became_active":"2024-09-16T15:40:12.863383+0000","last_became_peered":"2024-09-16T15:40:12.863383+0000","last_unstale":"2024-09-16T15:40:12.871085+0000","last_undegraded":"2024-09-16T15:40:12.871085+0000","last_fullsized":"2024-09-16T15:40:12.871085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:44.940019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.863957+0000","last_change":"2024-09-16T15:40:12.856737+0000","last_active":"2024-09-16T15:40:12.863957+0000","last_peered":"2024-09-16T15:40:12.863957+0000","last_clean":"2024-09-16T15:40:12.863957+0000","last_became_active":"2024-09-16T15:40:12.856426+0000","last_became_peered":"2024-09-16T15:40:12.856426+0000","last_unstale":"2024-09-16T15:40:12.863957+0000","last_undegraded":"2024-09-16T15:40:12.863957+0000","last_fullsized":"2024-09-16T15:40:12.863957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:58.240556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879492+0000","last_change":"2024-09-16T15:40:12.876499+0000","last_active":"2024-09-16T15:40:12.879492+0000","last_peered":"2024-09-16T15:40:12.879492+0000","last_clean":"2024-09-16T15:40:12.879492+0000","last_became_active":"2024-09-16T15:40:12.876247+0000","last_became_peered":"2024-09-16T15:40:12.876247+0000","last_unstale":"2024-09-16T15:40:12.879492+0000","last_undegraded":"2024-09-16T15:40:12.879492+0000","last_fullsized":"2024-09-16T15:40:12.879492+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:42:34.521097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867366+0000","last_change":"2024-09-16T15:40:12.858149+0000","last_active":"2024-09-16T15:40:12.867366+0000","last_peered":"2024-09-16T15:40:12.867366+0000","last_clean":"2024-09-16T15:40:12.867366+0000","last_became_active":"2024-09-16T15:40:12.857902+0000","last_became_peered":"2024-09-16T15:40:12.857902+0000","last_unstale":"2024-09-16T15:40:12.867366+0000","last_undegraded":"2024-09-16T15:40:12.867366+0000","last_fullsized":"2024-09-16T15:40:12.867366+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:04.893938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870724+0000","last_change":"2024-09-16T15:40:12.865956+0000","last_active":"2024-09-16T15:40:12.870724+0000","last_peered":"2024-09-16T15:40:12.870724+0000","last_clean":"2024-09-16T15:40:12.870724+0000","last_became_active":"2024-09-16T15:40:12.865681+0000","last_became_peered":"2024-09-16T15:40:12.865681+0000","last_unstale":"2024-09-16T15:40:12.870724+0000","last_undegraded":"2024-09-16T15:40:12.870724+0000","last_fullsized":"2024-09-16T15:40:12.870724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:14.857566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.864031+0000","last_change":"2024-09-16T15:40:12.856711+0000","last_active":"2024-09-16T15:40:12.864031+0000","last_peered":"2024-09-16T15:40:12.864031+0000","last_clean":"2024-09-16T15:40:12.864031+0000","last_became_active":"2024-09-16T15:40:12.856290+0000","last_became_peered":"2024-09-16T15:40:12.856290+0000","last_unstale":"2024-09-16T15:40:12.864031+0000","last_undegraded":"2024-09-16T15:40:12.864031+0000","last_fullsized":"2024-09-16T15:40:12.864031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:27.797378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867021+0000","last_change":"2024-09-16T15:40:12.859686+0000","last_active":"2024-09-16T15:40:12.867021+0000","last_peered":"2024-09-16T15:40:12.867021+0000","last_clean":"2024-09-16T15:40:12.867021+0000","last_became_active":"2024-09-16T15:40:12.859399+0000","last_became_peered":"2024-09-16T15:40:12.859399+0000","last_unstale":"2024-09-16T15:40:12.867021+0000","last_undegraded":"2024-09-16T15:40:12.867021+0000","last_fullsized":"2024-09-16T15:40:12.867021+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:56:45.345188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867284+0000","last_change":"2024-09-16T15:40:12.857344+0000","last_active":"2024-09-16T15:40:12.867284+0000","last_peered":"2024-09-16T15:40:12.867284+0000","last_clean":"2024-09-16T15:40:12.867284+0000","last_became_active":"2024-09-16T15:40:12.857044+0000","last_became_peered":"2024-09-16T15:40:12.857044+0000","last_unstale":"2024-09-16T15:40:12.867284+0000","last_undegraded":"2024-09-16T15:40:12.867284+0000","last_fullsized":"2024-09-16T15:40:12.867284+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:28:28.848082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867593+0000","last_change":"2024-09-16T15:40:12.860869+0000","last_active":"2024-09-16T15:40:12.867593+0000","last_peered":"2024-09-16T15:40:12.867593+0000","last_clean":"2024-09-16T15:40:12.867593+0000","last_became_active":"2024-09-16T15:40:12.860241+0000","last_became_peered":"2024-09-16T15:40:12.860241+0000","last_unstale":"2024-09-16T15:40:12.867593+0000","last_undegraded":"2024-09-16T15:40:12.867593+0000","last_fullsized":"2024-09-16T15:40:12.867593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:37.559530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147666+0000","last_change":"2024-09-16T15:40:12.856388+0000","last_active":"2024-09-16T15:40:14.147666+0000","last_peered":"2024-09-16T15:40:14.147666+0000","last_clean":"2024-09-16T15:40:14.147666+0000","last_became_active":"2024-09-16T15:40:12.854943+0000","last_became_peered":"2024-09-16T15:40:12.854943+0000","last_unstale":"2024-09-16T15:40:14.147666+0000","last_undegraded":"2024-09-16T15:40:14.147666+0000","last_fullsized":"2024-09-16T15:40:14.147666+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:10.844833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875888+0000","last_change":"2024-09-16T15:40:11.859931+0000","last_active":"2024-09-16T15:40:12.875888+0000","last_peered":"2024-09-16T15:40:12.875888+0000","last_clean":"2024-09-16T15:40:12.875888+0000","last_became_active":"2024-09-16T15:40:11.859675+0000","last_became_peered":"2024-09-16T15:40:11.859675+0000","last_unstale":"2024-09-16T15:40:12.875888+0000","last_undegraded":"2024-09-16T15:40:12.875888+0000","last_fullsized":"2024-09-16T15:40:12.875888+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:28.283914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867675+0000","last_change":"2024-09-16T15:40:12.861984+0000","last_active":"2024-09-16T15:40:12.867675+0000","last_peered":"2024-09-16T15:40:12.867675+0000","last_clean":"2024-09-16T15:40:12.867675+0000","last_became_active":"2024-09-16T15:40:12.861759+0000","last_became_peered":"2024-09-16T15:40:12.861759+0000","last_unstale":"2024-09-16T15:40:12.867675+0000","last_undegraded":"2024-09-16T15:40:12.867675+0000","last_fullsized":"2024-09-16T15:40:12.867675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:23.370252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880884+0000","last_change":"2024-09-16T15:40:11.863371+0000","last_active":"2024-09-16T15:40:12.880884+0000","last_peered":"2024-09-16T15:40:12.880884+0000","last_clean":"2024-09-16T15:40:12.880884+0000","last_became_active":"2024-09-16T15:40:11.860269+0000","last_became_peered":"2024-09-16T15:40:11.860269+0000","last_unstale":"2024-09-16T15:40:12.880884+0000","last_undegraded":"2024-09-16T15:40:12.880884+0000","last_fullsized":"2024-09-16T15:40:12.880884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:36.282583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.858764+0000","last_change":"2024-09-16T15:40:12.855039+0000","last_active":"2024-09-16T15:40:12.858764+0000","last_peered":"2024-09-16T15:40:12.858764+0000","last_clean":"2024-09-16T15:40:12.858764+0000","last_became_active":"2024-09-16T15:40:12.854141+0000","last_became_peered":"2024-09-16T15:40:12.854141+0000","last_unstale":"2024-09-16T15:40:12.858764+0000","last_undegraded":"2024-09-16T15:40:12.858764+0000","last_fullsized":"2024-09-16T15:40:12.858764+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:26.535001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147699+0000","last_change":"2024-09-16T15:40:11.869853+0000","last_active":"2024-09-16T15:40:14.147699+0000","last_peered":"2024-09-16T15:40:14.147699+0000","last_clean":"2024-09-16T15:40:14.147699+0000","last_became_active":"2024-09-16T15:40:11.869583+0000","last_became_peered":"2024-09-16T15:40:11.869583+0000","last_unstale":"2024-09-16T15:40:14.147699+0000","last_undegraded":"2024-09-16T15:40:14.147699+0000","last_fullsized":"2024-09-16T15:40:14.147699+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:50.423580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147687+0000","last_change":"2024-09-16T15:40:12.853403+0000","last_active":"2024-09-16T15:40:14.147687+0000","last_peered":"2024-09-16T15:40:14.147687+0000","last_clean":"2024-09-16T15:40:14.147687+0000","last_became_active":"2024-09-16T15:40:12.853161+0000","last_became_peered":"2024-09-16T15:40:12.853161+0000","last_unstale":"2024-09-16T15:40:14.147687+0000","last_undegraded":"2024-09-16T15:40:14.147687+0000","last_fullsized":"2024-09-16T15:40:14.147687+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:30.287843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880313+0000","last_change":"2024-09-16T15:40:11.868506+0000","last_active":"2024-09-16T15:40:12.880313+0000","last_peered":"2024-09-16T15:40:12.880313+0000","last_clean":"2024-09-16T15:40:12.880313+0000","last_became_active":"2024-09-16T15:40:11.868014+0000","last_became_peered":"2024-09-16T15:40:11.868014+0000","last_unstale":"2024-09-16T15:40:12.880313+0000","last_undegraded":"2024-09-16T15:40:12.880313+0000","last_fullsized":"2024-09-16T15:40:12.880313+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.292320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874345+0000","last_change":"2024-09-16T15:40:12.867667+0000","last_active":"2024-09-16T15:40:12.874345+0000","last_peered":"2024-09-16T15:40:12.874345+0000","last_clean":"2024-09-16T15:40:12.874345+0000","last_became_active":"2024-09-16T15:40:12.866213+0000","last_became_peered":"2024-09-16T15:40:12.866213+0000","last_unstale":"2024-09-16T15:40:12.874345+0000","last_undegraded":"2024-09-16T15:40:12.874345+0000","last_fullsized":"2024-09-16T15:40:12.874345+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:06.413661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.877703+0000","last_change":"2024-09-16T15:40:11.871006+0000","last_active":"2024-09-16T15:40:12.877703+0000","last_peered":"2024-09-16T15:40:12.877703+0000","last_clean":"2024-09-16T15:40:12.877703+0000","last_became_active":"2024-09-16T15:40:11.870728+0000","last_became_peered":"2024-09-16T15:40:11.870728+0000","last_unstale":"2024-09-16T15:40:12.877703+0000","last_undegraded":"2024-09-16T15:40:12.877703+0000","last_fullsized":"2024-09-16T15:40:12.877703+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:26.890614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.882663+0000","last_change":"2024-09-16T15:40:12.869124+0000","last_active":"2024-09-16T15:40:12.882663+0000","last_peered":"2024-09-16T15:40:12.882663+0000","last_clean":"2024-09-16T15:40:12.882663+0000","last_became_active":"2024-09-16T15:40:12.868857+0000","last_became_peered":"2024-09-16T15:40:12.868857+0000","last_unstale":"2024-09-16T15:40:12.882663+0000","last_undegraded":"2024-09-16T15:40:12.882663+0000","last_fullsized":"2024-09-16T15:40:12.882663+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:59.975480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879734+0000","last_change":"2024-09-16T15:40:11.865117+0000","last_active":"2024-09-16T15:40:12.879734+0000","last_peered":"2024-09-16T15:40:12.879734+0000","last_clean":"2024-09-16T15:40:12.879734+0000","last_became_active":"2024-09-16T15:40:11.864766+0000","last_became_peered":"2024-09-16T15:40:11.864766+0000","last_unstale":"2024-09-16T15:40:12.879734+0000","last_undegraded":"2024-09-16T15:40:12.879734+0000","last_fullsized":"2024-09-16T15:40:12.879734+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:07.999976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880707+0000","last_change":"2024-09-16T15:40:12.860867+0000","last_active":"2024-09-16T15:40:12.880707+0000","last_peered":"2024-09-16T15:40:12.880707+0000","last_clean":"2024-09-16T15:40:12.880707+0000","last_became_active":"2024-09-16T15:40:12.860648+0000","last_became_peered":"2024-09-16T15:40:12.860648+0000","last_unstale":"2024-09-16T15:40:12.880707+0000","last_undegraded":"2024-09-16T15:40:12.880707+0000","last_fullsized":"2024-09-16T15:40:12.880707+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:49.901150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.844800+0000","last_change":"2024-09-16T15:40:11.868600+0000","last_active":"2024-09-16T15:40:12.844800+0000","last_peered":"2024-09-16T15:40:12.844800+0000","last_clean":"2024-09-16T15:40:12.844800+0000","last_became_active":"2024-09-16T15:40:11.868271+0000","last_became_peered":"2024-09-16T15:40:11.868271+0000","last_unstale":"2024-09-16T15:40:12.844800+0000","last_undegraded":"2024-09-16T15:40:12.844800+0000","last_fullsized":"2024-09-16T15:40:12.844800+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:56.197404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.870499+0000","last_change":"2024-09-16T15:40:12.864230+0000","last_active":"2024-09-16T15:40:12.870499+0000","last_peered":"2024-09-16T15:40:12.870499+0000","last_clean":"2024-09-16T15:40:12.870499+0000","last_became_active":"2024-09-16T15:40:12.863371+0000","last_became_peered":"2024-09-16T15:40:12.863371+0000","last_unstale":"2024-09-16T15:40:12.870499+0000","last_undegraded":"2024-09-16T15:40:12.870499+0000","last_fullsized":"2024-09-16T15:40:12.870499+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:26.451623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.847205+0000","last_change":"2024-09-16T15:40:11.864603+0000","last_active":"2024-09-16T15:40:12.847205+0000","last_peered":"2024-09-16T15:40:12.847205+0000","last_clean":"2024-09-16T15:40:12.847205+0000","last_became_active":"2024-09-16T15:40:11.864256+0000","last_became_peered":"2024-09-16T15:40:11.864256+0000","last_unstale":"2024-09-16T15:40:12.847205+0000","last_undegraded":"2024-09-16T15:40:12.847205+0000","last_fullsized":"2024-09-16T15:40:12.847205+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:34.607613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875009+0000","last_change":"2024-09-16T15:40:12.873524+0000","last_active":"2024-09-16T15:40:12.875009+0000","last_peered":"2024-09-16T15:40:12.875009+0000","last_clean":"2024-09-16T15:40:12.875009+0000","last_became_active":"2024-09-16T15:40:12.873256+0000","last_became_peered":"2024-09-16T15:40:12.873256+0000","last_unstale":"2024-09-16T15:40:12.875009+0000","last_undegraded":"2024-09-16T15:40:12.875009+0000","last_fullsized":"2024-09-16T15:40:12.875009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:15.479730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.851806+0000","last_change":"2024-09-16T15:40:11.860367+0000","last_active":"2024-09-16T15:40:12.851806+0000","last_peered":"2024-09-16T15:40:12.851806+0000","last_clean":"2024-09-16T15:40:12.851806+0000","last_became_active":"2024-09-16T15:40:11.858703+0000","last_became_peered":"2024-09-16T15:40:11.858703+0000","last_unstale":"2024-09-16T15:40:12.851806+0000","last_undegraded":"2024-09-16T15:40:12.851806+0000","last_fullsized":"2024-09-16T15:40:12.851806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:47.251591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.868881+0000","last_change":"2024-09-16T15:40:12.865158+0000","last_active":"2024-09-16T15:40:12.868881+0000","last_peered":"2024-09-16T15:40:12.868881+0000","last_clean":"2024-09-16T15:40:12.868881+0000","last_became_active":"2024-09-16T15:40:12.864675+0000","last_became_peered":"2024-09-16T15:40:12.864675+0000","last_unstale":"2024-09-16T15:40:12.868881+0000","last_undegraded":"2024-09-16T15:40:12.868881+0000","last_fullsized":"2024-09-16T15:40:12.868881+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.849200+0000","last_change":"2024-09-16T15:40:11.863611+0000","last_active":"2024-09-16T15:40:12.849200+0000","last_peered":"2024-09-16T15:40:12.849200+0000","last_clean":"2024-09-16T15:40:12.849200+0000","last_became_active":"2024-09-16T15:40:11.863346+0000","last_became_peered":"2024-09-16T15:40:11.863346+0000","last_unstale":"2024-09-16T15:40:12.849200+0000","last_undegraded":"2024-09-16T15:40:12.849200+0000","last_fullsized":"2024-09-16T15:40:12.849200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:00.681742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879244+0000","last_change":"2024-09-16T15:40:12.876615+0000","last_active":"2024-09-16T15:40:12.879244+0000","last_peered":"2024-09-16T15:40:12.879244+0000","last_clean":"2024-09-16T15:40:12.879244+0000","last_became_active":"2024-09-16T15:40:12.876385+0000","last_became_peered":"2024-09-16T15:40:12.876385+0000","last_unstale":"2024-09-16T15:40:12.879244+0000","last_undegraded":"2024-09-16T15:40:12.879244+0000","last_fullsized":"2024-09-16T15:40:12.879244+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:35.883048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881290+0000","last_change":"2024-09-16T15:40:11.864331+0000","last_active":"2024-09-16T15:40:12.881290+0000","last_peered":"2024-09-16T15:40:12.881290+0000","last_clean":"2024-09-16T15:40:12.881290+0000","last_became_active":"2024-09-16T15:40:11.864083+0000","last_became_peered":"2024-09-16T15:40:11.864083+0000","last_unstale":"2024-09-16T15:40:12.881290+0000","last_undegraded":"2024-09-16T15:40:12.881290+0000","last_fullsized":"2024-09-16T15:40:12.881290+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:44.080258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.868336+0000","last_change":"2024-09-16T15:40:12.865124+0000","last_active":"2024-09-16T15:40:12.868336+0000","last_peered":"2024-09-16T15:40:12.868336+0000","last_clean":"2024-09-16T15:40:12.868336+0000","last_became_active":"2024-09-16T15:40:12.864809+0000","last_became_peered":"2024-09-16T15:40:12.864809+0000","last_unstale":"2024-09-16T15:40:12.868336+0000","last_undegraded":"2024-09-16T15:40:12.868336+0000","last_fullsized":"2024-09-16T15:40:12.868336+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.885109+0000","last_change":"2024-09-16T15:40:11.865511+0000","last_active":"2024-09-16T15:40:12.885109+0000","last_peered":"2024-09-16T15:40:12.885109+0000","last_clean":"2024-09-16T15:40:12.885109+0000","last_became_active":"2024-09-16T15:40:11.863715+0000","last_became_peered":"2024-09-16T15:40:11.863715+0000","last_unstale":"2024-09-16T15:40:12.885109+0000","last_undegraded":"2024-09-16T15:40:12.885109+0000","last_fullsized":"2024-09-16T15:40:12.885109+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:18.604944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.875276+0000","last_change":"2024-09-16T15:40:12.871040+0000","last_active":"2024-09-16T15:40:12.875276+0000","last_peered":"2024-09-16T15:40:12.875276+0000","last_clean":"2024-09-16T15:40:12.875276+0000","last_became_active":"2024-09-16T15:40:12.870459+0000","last_became_peered":"2024-09-16T15:40:12.870459+0000","last_unstale":"2024-09-16T15:40:12.875276+0000","last_undegraded":"2024-09-16T15:40:12.875276+0000","last_fullsized":"2024-09-16T15:40:12.875276+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:53.575639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.884353+0000","last_change":"2024-09-16T15:40:11.870606+0000","last_active":"2024-09-16T15:40:12.884353+0000","last_peered":"2024-09-16T15:40:12.884353+0000","last_clean":"2024-09-16T15:40:12.884353+0000","last_became_active":"2024-09-16T15:40:11.868146+0000","last_became_peered":"2024-09-16T15:40:11.868146+0000","last_unstale":"2024-09-16T15:40:12.884353+0000","last_undegraded":"2024-09-16T15:40:12.884353+0000","last_fullsized":"2024-09-16T15:40:12.884353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:36.745807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880157+0000","last_change":"2024-09-16T15:40:12.860151+0000","last_active":"2024-09-16T15:40:12.880157+0000","last_peered":"2024-09-16T15:40:12.880157+0000","last_clean":"2024-09-16T15:40:12.880157+0000","last_became_active":"2024-09-16T15:40:12.859179+0000","last_became_peered":"2024-09-16T15:40:12.859179+0000","last_unstale":"2024-09-16T15:40:12.880157+0000","last_undegraded":"2024-09-16T15:40:12.880157+0000","last_fullsized":"2024-09-16T15:40:12.880157+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:39.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855830+0000","last_change":"2024-09-16T15:40:11.863101+0000","last_active":"2024-09-16T15:40:12.855830+0000","last_peered":"2024-09-16T15:40:12.855830+0000","last_clean":"2024-09-16T15:40:12.855830+0000","last_became_active":"2024-09-16T15:40:11.862829+0000","last_became_peered":"2024-09-16T15:40:11.862829+0000","last_unstale":"2024-09-16T15:40:12.855830+0000","last_undegraded":"2024-09-16T15:40:12.855830+0000","last_fullsized":"2024-09-16T15:40:12.855830+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:25.527579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.876819+0000","last_change":"2024-09-16T15:40:12.871218+0000","last_active":"2024-09-16T15:40:12.876819+0000","last_peered":"2024-09-16T15:40:12.876819+0000","last_clean":"2024-09-16T15:40:12.876819+0000","last_became_active":"2024-09-16T15:40:12.870997+0000","last_became_peered":"2024-09-16T15:40:12.870997+0000","last_unstale":"2024-09-16T15:40:12.876819+0000","last_undegraded":"2024-09-16T15:40:12.876819+0000","last_fullsized":"2024-09-16T15:40:12.876819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:27.010957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.852972+0000","last_change":"2024-09-16T15:40:11.867251+0000","last_active":"2024-09-16T15:40:12.852972+0000","last_peered":"2024-09-16T15:40:12.852972+0000","last_clean":"2024-09-16T15:40:12.852972+0000","last_became_active":"2024-09-16T15:40:11.866730+0000","last_became_peered":"2024-09-16T15:40:11.866730+0000","last_unstale":"2024-09-16T15:40:12.852972+0000","last_undegraded":"2024-09-16T15:40:12.852972+0000","last_fullsized":"2024-09-16T15:40:12.852972+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:55.152872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873844+0000","last_change":"2024-09-16T15:40:12.869762+0000","last_active":"2024-09-16T15:40:12.873844+0000","last_peered":"2024-09-16T15:40:12.873844+0000","last_clean":"2024-09-16T15:40:12.873844+0000","last_became_active":"2024-09-16T15:40:12.869420+0000","last_became_peered":"2024-09-16T15:40:12.869420+0000","last_unstale":"2024-09-16T15:40:12.873844+0000","last_undegraded":"2024-09-16T15:40:12.873844+0000","last_fullsized":"2024-09-16T15:40:12.873844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:51.377166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.888763+0000","last_change":"2024-09-16T15:40:11.868078+0000","last_active":"2024-09-16T15:40:12.888763+0000","last_peered":"2024-09-16T15:40:12.888763+0000","last_clean":"2024-09-16T15:40:12.888763+0000","last_became_active":"2024-09-16T15:40:11.867825+0000","last_became_peered":"2024-09-16T15:40:11.867825+0000","last_unstale":"2024-09-16T15:40:12.888763+0000","last_undegraded":"2024-09-16T15:40:12.888763+0000","last_fullsized":"2024-09-16T15:40:12.888763+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:29.103104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.872014+0000","last_change":"2024-09-16T15:40:12.867357+0000","last_active":"2024-09-16T15:40:12.872014+0000","last_peered":"2024-09-16T15:40:12.872014+0000","last_clean":"2024-09-16T15:40:12.872014+0000","last_became_active":"2024-09-16T15:40:12.867072+0000","last_became_peered":"2024-09-16T15:40:12.867072+0000","last_unstale":"2024-09-16T15:40:12.872014+0000","last_undegraded":"2024-09-16T15:40:12.872014+0000","last_fullsized":"2024-09-16T15:40:12.872014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:17.395905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.886756+0000","last_change":"2024-09-16T15:40:11.866281+0000","last_active":"2024-09-16T15:40:12.886756+0000","last_peered":"2024-09-16T15:40:12.886756+0000","last_clean":"2024-09-16T15:40:12.886756+0000","last_became_active":"2024-09-16T15:40:11.865322+0000","last_became_peered":"2024-09-16T15:40:11.865322+0000","last_unstale":"2024-09-16T15:40:12.886756+0000","last_undegraded":"2024-09-16T15:40:12.886756+0000","last_fullsized":"2024-09-16T15:40:12.886756+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:14:13.585151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883782+0000","last_change":"2024-09-16T15:40:12.874698+0000","last_active":"2024-09-16T15:40:12.883782+0000","last_peered":"2024-09-16T15:40:12.883782+0000","last_clean":"2024-09-16T15:40:12.883782+0000","last_became_active":"2024-09-16T15:40:12.874345+0000","last_became_peered":"2024-09-16T15:40:12.874345+0000","last_unstale":"2024-09-16T15:40:12.883782+0000","last_undegraded":"2024-09-16T15:40:12.883782+0000","last_fullsized":"2024-09-16T15:40:12.883782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:49.306539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880542+0000","last_change":"2024-09-16T15:40:11.859504+0000","last_active":"2024-09-16T15:40:12.880542+0000","last_peered":"2024-09-16T15:40:12.880542+0000","last_clean":"2024-09-16T15:40:12.880542+0000","last_became_active":"2024-09-16T15:40:11.859183+0000","last_became_peered":"2024-09-16T15:40:11.859183+0000","last_unstale":"2024-09-16T15:40:12.880542+0000","last_undegraded":"2024-09-16T15:40:12.880542+0000","last_fullsized":"2024-09-16T15:40:12.880542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:57.973947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.863982+0000","last_change":"2024-09-16T15:40:12.856434+0000","last_active":"2024-09-16T15:40:12.863982+0000","last_peered":"2024-09-16T15:40:12.863982+0000","last_clean":"2024-09-16T15:40:12.863982+0000","last_became_active":"2024-09-16T15:40:12.856073+0000","last_became_peered":"2024-09-16T15:40:12.856073+0000","last_unstale":"2024-09-16T15:40:12.863982+0000","last_undegraded":"2024-09-16T15:40:12.863982+0000","last_fullsized":"2024-09-16T15:40:12.863982+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:22.248689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.878307+0000","last_change":"2024-09-16T15:40:11.869221+0000","last_active":"2024-09-16T15:40:12.878307+0000","last_peered":"2024-09-16T15:40:12.878307+0000","last_clean":"2024-09-16T15:40:12.878307+0000","last_became_active":"2024-09-16T15:40:11.868892+0000","last_became_peered":"2024-09-16T15:40:11.868892+0000","last_unstale":"2024-09-16T15:40:12.878307+0000","last_undegraded":"2024-09-16T15:40:12.878307+0000","last_fullsized":"2024-09-16T15:40:12.878307+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.865500+0000","last_change":"2024-09-16T15:40:12.859954+0000","last_active":"2024-09-16T15:40:12.865500+0000","last_peered":"2024-09-16T15:40:12.865500+0000","last_clean":"2024-09-16T15:40:12.865500+0000","last_became_active":"2024-09-16T15:40:12.859695+0000","last_became_peered":"2024-09-16T15:40:12.859695+0000","last_unstale":"2024-09-16T15:40:12.865500+0000","last_undegraded":"2024-09-16T15:40:12.865500+0000","last_fullsized":"2024-09-16T15:40:12.865500+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:50.839867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.851789+0000","last_change":"2024-09-16T15:40:11.863949+0000","last_active":"2024-09-16T15:40:12.851789+0000","last_peered":"2024-09-16T15:40:12.851789+0000","last_clean":"2024-09-16T15:40:12.851789+0000","last_became_active":"2024-09-16T15:40:11.863620+0000","last_became_peered":"2024-09-16T15:40:11.863620+0000","last_unstale":"2024-09-16T15:40:12.851789+0000","last_undegraded":"2024-09-16T15:40:12.851789+0000","last_fullsized":"2024-09-16T15:40:12.851789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:18.083324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.877926+0000","last_change":"2024-09-16T15:40:12.873993+0000","last_active":"2024-09-16T15:40:12.877926+0000","last_peered":"2024-09-16T15:40:12.877926+0000","last_clean":"2024-09-16T15:40:12.877926+0000","last_became_active":"2024-09-16T15:40:12.873720+0000","last_became_peered":"2024-09-16T15:40:12.873720+0000","last_unstale":"2024-09-16T15:40:12.877926+0000","last_undegraded":"2024-09-16T15:40:12.877926+0000","last_fullsized":"2024-09-16T15:40:12.877926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:47.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.880354+0000","last_change":"2024-09-16T15:40:11.871266+0000","last_active":"2024-09-16T15:40:12.880354+0000","last_peered":"2024-09-16T15:40:12.880354+0000","last_clean":"2024-09-16T15:40:12.880354+0000","last_became_active":"2024-09-16T15:40:11.871013+0000","last_became_peered":"2024-09-16T15:40:11.871013+0000","last_unstale":"2024-09-16T15:40:12.880354+0000","last_undegraded":"2024-09-16T15:40:12.880354+0000","last_fullsized":"2024-09-16T15:40:12.880354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:10.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147184+0000","last_change":"2024-09-16T15:40:12.850407+0000","last_active":"2024-09-16T15:40:14.147184+0000","last_peered":"2024-09-16T15:40:14.147184+0000","last_clean":"2024-09-16T15:40:14.147184+0000","last_became_active":"2024-09-16T15:40:12.849693+0000","last_became_peered":"2024-09-16T15:40:12.849693+0000","last_unstale":"2024-09-16T15:40:14.147184+0000","last_undegraded":"2024-09-16T15:40:14.147184+0000","last_fullsized":"2024-09-16T15:40:14.147184+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:48.268979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.845323+0000","last_change":"2024-09-16T15:40:11.865942+0000","last_active":"2024-09-16T15:40:12.845323+0000","last_peered":"2024-09-16T15:40:12.845323+0000","last_clean":"2024-09-16T15:40:12.845323+0000","last_became_active":"2024-09-16T15:40:11.864112+0000","last_became_peered":"2024-09-16T15:40:11.864112+0000","last_unstale":"2024-09-16T15:40:12.845323+0000","last_undegraded":"2024-09-16T15:40:12.845323+0000","last_fullsized":"2024-09-16T15:40:12.845323+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:13.011662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867632+0000","last_change":"2024-09-16T15:40:12.861166+0000","last_active":"2024-09-16T15:40:12.867632+0000","last_peered":"2024-09-16T15:40:12.867632+0000","last_clean":"2024-09-16T15:40:12.867632+0000","last_became_active":"2024-09-16T15:40:12.860619+0000","last_became_peered":"2024-09-16T15:40:12.860619+0000","last_unstale":"2024-09-16T15:40:12.867632+0000","last_undegraded":"2024-09-16T15:40:12.867632+0000","last_fullsized":"2024-09-16T15:40:12.867632+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:54.024305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.862595+0000","last_change":"2024-09-16T15:40:11.862288+0000","last_active":"2024-09-16T15:40:12.862595+0000","last_peered":"2024-09-16T15:40:12.862595+0000","last_clean":"2024-09-16T15:40:12.862595+0000","last_became_active":"2024-09-16T15:40:11.862002+0000","last_became_peered":"2024-09-16T15:40:11.862002+0000","last_unstale":"2024-09-16T15:40:12.862595+0000","last_undegraded":"2024-09-16T15:40:12.862595+0000","last_fullsized":"2024-09-16T15:40:12.862595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:12.191215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.884800+0000","last_change":"2024-09-16T15:40:12.879430+0000","last_active":"2024-09-16T15:40:12.884800+0000","last_peered":"2024-09-16T15:40:12.884800+0000","last_clean":"2024-09-16T15:40:12.884800+0000","last_became_active":"2024-09-16T15:40:12.879172+0000","last_became_peered":"2024-09-16T15:40:12.879172+0000","last_unstale":"2024-09-16T15:40:12.884800+0000","last_undegraded":"2024-09-16T15:40:12.884800+0000","last_fullsized":"2024-09-16T15:40:12.884800+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:14.019085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.854691+0000","last_change":"2024-09-16T15:40:11.863062+0000","last_active":"2024-09-16T15:40:12.854691+0000","last_peered":"2024-09-16T15:40:12.854691+0000","last_clean":"2024-09-16T15:40:12.854691+0000","last_became_active":"2024-09-16T15:40:11.860714+0000","last_became_peered":"2024-09-16T15:40:11.860714+0000","last_unstale":"2024-09-16T15:40:12.854691+0000","last_undegraded":"2024-09-16T15:40:12.854691+0000","last_fullsized":"2024-09-16T15:40:12.854691+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883782+0000","last_change":"2024-09-16T15:40:12.883082+0000","last_active":"2024-09-16T15:40:12.883782+0000","last_peered":"2024-09-16T15:40:12.883782+0000","last_clean":"2024-09-16T15:40:12.883782+0000","last_became_active":"2024-09-16T15:40:12.882695+0000","last_became_peered":"2024-09-16T15:40:12.882695+0000","last_unstale":"2024-09-16T15:40:12.883782+0000","last_undegraded":"2024-09-16T15:40:12.883782+0000","last_fullsized":"2024-09-16T15:40:12.883782+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:21.231163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.853585+0000","last_change":"2024-09-16T15:40:11.867238+0000","last_active":"2024-09-16T15:40:12.853585+0000","last_peered":"2024-09-16T15:40:12.853585+0000","last_clean":"2024-09-16T15:40:12.853585+0000","last_became_active":"2024-09-16T15:40:11.865996+0000","last_became_peered":"2024-09-16T15:40:11.865996+0000","last_unstale":"2024-09-16T15:40:12.853585+0000","last_undegraded":"2024-09-16T15:40:12.853585+0000","last_fullsized":"2024-09-16T15:40:12.853585+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:23.953426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.883245+0000","last_change":"2024-09-16T15:40:12.872931+0000","last_active":"2024-09-16T15:40:12.883245+0000","last_peered":"2024-09-16T15:40:12.883245+0000","last_clean":"2024-09-16T15:40:12.883245+0000","last_became_active":"2024-09-16T15:40:12.872344+0000","last_became_peered":"2024-09-16T15:40:12.872344+0000","last_unstale":"2024-09-16T15:40:12.883245+0000","last_undegraded":"2024-09-16T15:40:12.883245+0000","last_fullsized":"2024-09-16T15:40:12.883245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:28.977918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855957+0000","last_change":"2024-09-16T15:40:11.865216+0000","last_active":"2024-09-16T15:40:12.855957+0000","last_peered":"2024-09-16T15:40:12.855957+0000","last_clean":"2024-09-16T15:40:12.855957+0000","last_became_active":"2024-09-16T15:40:11.864945+0000","last_became_peered":"2024-09-16T15:40:11.864945+0000","last_unstale":"2024-09-16T15:40:12.855957+0000","last_undegraded":"2024-09-16T15:40:12.855957+0000","last_fullsized":"2024-09-16T15:40:12.855957+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:13.850144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.873339+0000","last_change":"2024-09-16T15:40:12.868439+0000","last_active":"2024-09-16T15:40:12.873339+0000","last_peered":"2024-09-16T15:40:12.873339+0000","last_clean":"2024-09-16T15:40:12.873339+0000","last_became_active":"2024-09-16T15:40:12.867651+0000","last_became_peered":"2024-09-16T15:40:12.867651+0000","last_unstale":"2024-09-16T15:40:12.873339+0000","last_undegraded":"2024-09-16T15:40:12.873339+0000","last_fullsized":"2024-09-16T15:40:12.873339+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:12.149041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146873+0000","last_change":"2024-09-16T15:40:11.869272+0000","last_active":"2024-09-16T15:40:14.146873+0000","last_peered":"2024-09-16T15:40:14.146873+0000","last_clean":"2024-09-16T15:40:14.146873+0000","last_became_active":"2024-09-16T15:40:11.868909+0000","last_became_peered":"2024-09-16T15:40:11.868909+0000","last_unstale":"2024-09-16T15:40:14.146873+0000","last_undegraded":"2024-09-16T15:40:14.146873+0000","last_fullsized":"2024-09-16T15:40:14.146873+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:57.684406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.874248+0000","last_change":"2024-09-16T15:40:12.862103+0000","last_active":"2024-09-16T15:40:12.874248+0000","last_peered":"2024-09-16T15:40:12.874248+0000","last_clean":"2024-09-16T15:40:12.874248+0000","last_became_active":"2024-09-16T15:40:12.861788+0000","last_became_peered":"2024-09-16T15:40:12.861788+0000","last_unstale":"2024-09-16T15:40:12.874248+0000","last_undegraded":"2024-09-16T15:40:12.874248+0000","last_fullsized":"2024-09-16T15:40:12.874248+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:38.051709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.855069+0000","last_change":"2024-09-16T15:40:11.867570+0000","last_active":"2024-09-16T15:40:12.855069+0000","last_peered":"2024-09-16T15:40:12.855069+0000","last_clean":"2024-09-16T15:40:12.855069+0000","last_became_active":"2024-09-16T15:40:11.867301+0000","last_became_peered":"2024-09-16T15:40:11.867301+0000","last_unstale":"2024-09-16T15:40:12.855069+0000","last_undegraded":"2024-09-16T15:40:12.855069+0000","last_fullsized":"2024-09-16T15:40:12.855069+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.879331+0000","last_change":"2024-09-16T15:40:12.875892+0000","last_active":"2024-09-16T15:40:12.879331+0000","last_peered":"2024-09-16T15:40:12.879331+0000","last_clean":"2024-09-16T15:40:12.879331+0000","last_became_active":"2024-09-16T15:40:12.875619+0000","last_became_peered":"2024-09-16T15:40:12.875619+0000","last_unstale":"2024-09-16T15:40:12.879331+0000","last_undegraded":"2024-09-16T15:40:12.879331+0000","last_fullsized":"2024-09-16T15:40:12.879331+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:15.128749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.849889+0000","last_change":"2024-09-16T15:40:11.867392+0000","last_active":"2024-09-16T15:40:12.849889+0000","last_peered":"2024-09-16T15:40:12.849889+0000","last_clean":"2024-09-16T15:40:12.849889+0000","last_became_active":"2024-09-16T15:40:11.867003+0000","last_became_peered":"2024-09-16T15:40:11.867003+0000","last_unstale":"2024-09-16T15:40:12.849889+0000","last_undegraded":"2024-09-16T15:40:12.849889+0000","last_fullsized":"2024-09-16T15:40:12.849889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:00.706403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.857550+0000","last_change":"2024-09-16T15:40:12.850142+0000","last_active":"2024-09-16T15:40:12.857550+0000","last_peered":"2024-09-16T15:40:12.857550+0000","last_clean":"2024-09-16T15:40:12.857550+0000","last_became_active":"2024-09-16T15:40:12.849859+0000","last_became_peered":"2024-09-16T15:40:12.849859+0000","last_unstale":"2024-09-16T15:40:12.857550+0000","last_undegraded":"2024-09-16T15:40:12.857550+0000","last_fullsized":"2024-09-16T15:40:12.857550+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:45.087681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146222+0000","last_change":"2024-09-16T15:40:11.860188+0000","last_active":"2024-09-16T15:40:14.146222+0000","last_peered":"2024-09-16T15:40:14.146222+0000","last_clean":"2024-09-16T15:40:14.146222+0000","last_became_active":"2024-09-16T15:40:11.859924+0000","last_became_peered":"2024-09-16T15:40:11.859924+0000","last_unstale":"2024-09-16T15:40:14.146222+0000","last_undegraded":"2024-09-16T15:40:14.146222+0000","last_fullsized":"2024-09-16T15:40:14.146222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:47.670989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.867386+0000","last_change":"2024-09-16T15:40:12.862284+0000","last_active":"2024-09-16T15:40:12.867386+0000","last_peered":"2024-09-16T15:40:12.867386+0000","last_clean":"2024-09-16T15:40:12.867386+0000","last_became_active":"2024-09-16T15:40:12.861890+0000","last_became_peered":"2024-09-16T15:40:12.861890+0000","last_unstale":"2024-09-16T15:40:12.867386+0000","last_undegraded":"2024-09-16T15:40:12.867386+0000","last_fullsized":"2024-09-16T15:40:12.867386+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:52.757331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.847519+0000","last_change":"2024-09-16T15:40:11.869696+0000","last_active":"2024-09-16T15:40:12.847519+0000","last_peered":"2024-09-16T15:40:12.847519+0000","last_clean":"2024-09-16T15:40:12.847519+0000","last_became_active":"2024-09-16T15:40:11.869403+0000","last_became_peered":"2024-09-16T15:40:11.869403+0000","last_unstale":"2024-09-16T15:40:12.847519+0000","last_undegraded":"2024-09-16T15:40:12.847519+0000","last_fullsized":"2024-09-16T15:40:12.847519+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:53.785910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.871439+0000","last_change":"2024-09-16T15:40:12.859022+0000","last_active":"2024-09-16T15:40:12.871439+0000","last_peered":"2024-09-16T15:40:12.871439+0000","last_clean":"2024-09-16T15:40:12.871439+0000","last_became_active":"2024-09-16T15:40:12.858683+0000","last_became_peered":"2024-09-16T15:40:12.858683+0000","last_unstale":"2024-09-16T15:40:12.871439+0000","last_undegraded":"2024-09-16T15:40:12.871439+0000","last_fullsized":"2024-09-16T15:40:12.871439+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:46.347083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.881285+0000","last_change":"2024-09-16T15:40:11.872583+0000","last_active":"2024-09-16T15:40:12.881285+0000","last_peered":"2024-09-16T15:40:12.881285+0000","last_clean":"2024-09-16T15:40:12.881285+0000","last_became_active":"2024-09-16T15:40:11.872290+0000","last_became_peered":"2024-09-16T15:40:11.872290+0000","last_unstale":"2024-09-16T15:40:12.881285+0000","last_undegraded":"2024-09-16T15:40:12.881285+0000","last_fullsized":"2024-09-16T15:40:12.881285+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:40:12.854788+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:40:12.854788+0000","last_peered":"2024-09-16T15:40:12.854788+0000","last_clean":"2024-09-16T15:40:12.854788+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:40:12.854788+0000","last_undegraded":"2024-09-16T15:40:12.854788+0000","last_fullsized":"2024-09-16T15:40:12.854788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28136,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343704,"statfs":{"total":96636764160,"available":96607952896,"internally_reserved":0,"allocated":1126400,"data_stored":802873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":548864,"data_stored":219862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":219350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":532480,"data_stored":212505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":802873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":561152,"data_stored":225650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:21.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:22.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.048+0000 7fd521400640 1 -- 172.21.15.60:0/1995019278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5000051c0 msgr2=0x7fd50000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.048+0000 7fd521400640 1 --2- 172.21.15.60:0/1995019278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5000051c0 0x7fd50000b9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd51400c590 tx=0x7fd500005830 comp rx=0 tx=0).stop 2024-09-16T15:40:22.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.048+0000 7fd520a00640 1 -- 172.21.15.60:0/1995019278 reap_dead start 2024-09-16T15:40:22.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.048+0000 7fd521400640 1 -- 172.21.15.60:0/1995019278 shutdown_connections 2024-09-16T15:40:22.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.048+0000 7fd521400640 1 -- 172.21.15.60:0/1995019278 >> 172.21.15.60:0/1995019278 conn(0x7fd51c0cfa20 msgr2=0x7fd51c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:22.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.048+0000 7fd521400640 1 -- 172.21.15.60:0/1995019278 shutdown_connections 2024-09-16T15:40:22.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.049+0000 7fd521400640 1 -- 172.21.15.60:0/1995019278 wait complete. 2024-09-16T15:40:22.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.050+0000 7fd521400640 1 Processor -- start 2024-09-16T15:40:22.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.050+0000 7fd521400640 1 -- start start 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd521400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5000051c0 0x7fd51c1f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd521400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd51c146ea0 0x7fd51c0c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd521400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd51c156d80 0x7fd51c157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd521400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c06fbb0 con 0x7fd51c146ea0 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd521400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c06fa30 con 0x7fd51c156d80 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd521400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c06fd30 con 0x7fd5000051c0 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd513400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd51c146ea0 0x7fd51c0c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd513400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd51c146ea0 0x7fd51c0c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40406/0 (socket says 172.21.15.60:40406) 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.051+0000 7fd513400640 1 -- 172.21.15.60:0/2205109277 learned_addr learned my addr 172.21.15.60:0/2205109277 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd51be00640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5000051c0 0x7fd51c1f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd520a00640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd51c156d80 0x7fd51c157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd513400640 1 -- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5000051c0 msgr2=0x7fd51c1f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd513400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5000051c0 0x7fd51c1f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd513400640 1 -- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd51c156d80 msgr2=0x7fd51c157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd513400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd51c156d80 0x7fd51c157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd513400640 1 -- 172.21.15.60:0/2205109277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c0c7030 con 0x7fd51c146ea0 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd51be00640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5000051c0 0x7fd51c1f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd520a00640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd51c156d80 0x7fd51c157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd513400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd51c146ea0 0x7fd51c0c6af0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd504011b40 tx=0x7fd504009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd518c00640 1 -- 172.21.15.60:0/2205109277 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50401e070 con 0x7fd51c146ea0 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd518c00640 1 -- 172.21.15.60:0/2205109277 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd50400fe70 con 0x7fd51c146ea0 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c0c8370 con 0x7fd51c146ea0 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.052+0000 7fd518c00640 1 -- 172.21.15.60:0/2205109277 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504004290 con 0x7fd51c146ea0 2024-09-16T15:40:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.053+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c0c72b0 con 0x7fd51c146ea0 2024-09-16T15:40:22.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.054+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c0725d0 con 0x7fd51c146ea0 2024-09-16T15:40:22.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.055+0000 7fd518c00640 1 -- 172.21.15.60:0/2205109277 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fd504021070 con 0x7fd51c146ea0 2024-09-16T15:40:22.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.056+0000 7fd518c00640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd4f8078a00 0x7fd4f807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.056+0000 7fd518c00640 1 -- 172.21.15.60:0/2205109277 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fd50409b850 con 0x7fd51c146ea0 2024-09-16T15:40:22.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.056+0000 7fd51be00640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd4f8078a00 0x7fd4f807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.057+0000 7fd51be00640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd4f8078a00 0x7fd4f807aec0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd51c1f57d0 tx=0x7fd50c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:22.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.060+0000 7fd518c00640 1 -- 172.21.15.60:0/2205109277 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd51c0c7030 con 0x7fd51c146ea0 2024-09-16T15:40:22.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.251+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd51c0b5700 con 0x7fd4f8078a00 2024-09-16T15:40:22.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.255+0000 7fd518c00640 1 -- 172.21.15.60:0/2205109277 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7fd51c0b5700 con 0x7fd4f8078a00 2024-09-16T15:40:22.259 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:22.261 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:40:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.259+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd4f8078a00 msgr2=0x7fd4f807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.259+0000 7fd521400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd4f8078a00 0x7fd4f807aec0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd51c1f57d0 tx=0x7fd50c005f90 comp rx=0 tx=0).stop 2024-09-16T15:40:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.259+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd51c146ea0 msgr2=0x7fd51c0c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.259+0000 7fd521400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd51c146ea0 0x7fd51c0c6af0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd504011b40 tx=0x7fd504009e90 comp rx=0 tx=0).stop 2024-09-16T15:40:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.260+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 shutdown_connections 2024-09-16T15:40:22.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.260+0000 7fd521400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd4f8078a00 0x7fd4f807aec0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.260+0000 7fd521400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd51c156d80 0x7fd51c157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.260+0000 7fd521400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd51c146ea0 0x7fd51c0c6af0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.260+0000 7fd521400640 1 --2- 172.21.15.60:0/2205109277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5000051c0 0x7fd51c1f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.260+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 >> 172.21.15.60:0/2205109277 conn(0x7fd51c0cfa20 msgr2=0x7fd51c0b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.260+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 shutdown_connections 2024-09-16T15:40:22.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.260+0000 7fd521400640 1 -- 172.21.15.60:0/2205109277 wait complete. 2024-09-16T15:40:22.277 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-16T15:40:21.735702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242636,"kb_used_data":5576,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732084,"statfs":{"total":773094113280,"available":772845654016,"internally_reserved":0,"allocated":5709824,"data_stored":2990196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.895406"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855039+0000","last_change":"2024-09-16T15:40:12.868607+0000","last_active":"2024-09-16T15:40:13.855039+0000","last_peered":"2024-09-16T15:40:13.855039+0000","last_clean":"2024-09-16T15:40:13.855039+0000","last_became_active":"2024-09-16T15:40:12.868140+0000","last_became_peered":"2024-09-16T15:40:12.868140+0000","last_unstale":"2024-09-16T15:40:13.855039+0000","last_undegraded":"2024-09-16T15:40:13.855039+0000","last_fullsized":"2024-09-16T15:40:13.855039+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:10.213025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146249+0000","last_change":"2024-09-16T15:40:12.869358+0000","last_active":"2024-09-16T15:40:14.146249+0000","last_peered":"2024-09-16T15:40:14.146249+0000","last_clean":"2024-09-16T15:40:14.146249+0000","last_became_active":"2024-09-16T15:40:12.868998+0000","last_became_peered":"2024-09-16T15:40:12.868998+0000","last_unstale":"2024-09-16T15:40:14.146249+0000","last_undegraded":"2024-09-16T15:40:14.146249+0000","last_fullsized":"2024-09-16T15:40:14.146249+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:10.213025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861228+0000","last_change":"2024-09-16T15:40:12.868681+0000","last_active":"2024-09-16T15:40:13.861228+0000","last_peered":"2024-09-16T15:40:13.861228+0000","last_clean":"2024-09-16T15:40:13.861228+0000","last_became_active":"2024-09-16T15:40:12.867980+0000","last_became_peered":"2024-09-16T15:40:12.867980+0000","last_unstale":"2024-09-16T15:40:13.861228+0000","last_undegraded":"2024-09-16T15:40:13.861228+0000","last_fullsized":"2024-09-16T15:40:13.861228+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:49.421540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146071+0000","last_change":"2024-09-16T15:40:12.873202+0000","last_active":"2024-09-16T15:40:14.146071+0000","last_peered":"2024-09-16T15:40:14.146071+0000","last_clean":"2024-09-16T15:40:14.146071+0000","last_became_active":"2024-09-16T15:40:12.872682+0000","last_became_peered":"2024-09-16T15:40:12.872682+0000","last_unstale":"2024-09-16T15:40:14.146071+0000","last_undegraded":"2024-09-16T15:40:14.146071+0000","last_fullsized":"2024-09-16T15:40:14.146071+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:39.299743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145922+0000","last_change":"2024-09-16T15:40:12.870112+0000","last_active":"2024-09-16T15:40:14.145922+0000","last_peered":"2024-09-16T15:40:14.145922+0000","last_clean":"2024-09-16T15:40:14.145922+0000","last_became_active":"2024-09-16T15:40:12.869753+0000","last_became_peered":"2024-09-16T15:40:12.869753+0000","last_unstale":"2024-09-16T15:40:14.145922+0000","last_undegraded":"2024-09-16T15:40:14.145922+0000","last_fullsized":"2024-09-16T15:40:14.145922+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:12.149041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854332+0000","last_change":"2024-09-16T15:40:12.866164+0000","last_active":"2024-09-16T15:40:13.854332+0000","last_peered":"2024-09-16T15:40:13.854332+0000","last_clean":"2024-09-16T15:40:13.854332+0000","last_became_active":"2024-09-16T15:40:12.865904+0000","last_became_peered":"2024-09-16T15:40:12.865904+0000","last_unstale":"2024-09-16T15:40:13.854332+0000","last_undegraded":"2024-09-16T15:40:13.854332+0000","last_fullsized":"2024-09-16T15:40:13.854332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:06.300562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855678+0000","last_change":"2024-09-16T15:40:12.865180+0000","last_active":"2024-09-16T15:40:13.855678+0000","last_peered":"2024-09-16T15:40:13.855678+0000","last_clean":"2024-09-16T15:40:13.855678+0000","last_became_active":"2024-09-16T15:40:12.864842+0000","last_became_peered":"2024-09-16T15:40:12.864842+0000","last_unstale":"2024-09-16T15:40:13.855678+0000","last_undegraded":"2024-09-16T15:40:13.855678+0000","last_fullsized":"2024-09-16T15:40:13.855678+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:14.746612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145685+0000","last_change":"2024-09-16T15:40:12.861554+0000","last_active":"2024-09-16T15:40:14.145685+0000","last_peered":"2024-09-16T15:40:14.145685+0000","last_clean":"2024-09-16T15:40:14.145685+0000","last_became_active":"2024-09-16T15:40:12.861215+0000","last_became_peered":"2024-09-16T15:40:12.861215+0000","last_unstale":"2024-09-16T15:40:14.145685+0000","last_undegraded":"2024-09-16T15:40:14.145685+0000","last_fullsized":"2024-09-16T15:40:14.145685+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:59.999158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147288+0000","last_change":"2024-09-16T15:40:12.864597+0000","last_active":"2024-09-16T15:40:14.147288+0000","last_peered":"2024-09-16T15:40:14.147288+0000","last_clean":"2024-09-16T15:40:14.147288+0000","last_became_active":"2024-09-16T15:40:12.864345+0000","last_became_peered":"2024-09-16T15:40:12.864345+0000","last_unstale":"2024-09-16T15:40:14.147288+0000","last_undegraded":"2024-09-16T15:40:14.147288+0000","last_fullsized":"2024-09-16T15:40:14.147288+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:29.669859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147038+0000","last_change":"2024-09-16T15:40:12.865537+0000","last_active":"2024-09-16T15:40:14.147038+0000","last_peered":"2024-09-16T15:40:14.147038+0000","last_clean":"2024-09-16T15:40:14.147038+0000","last_became_active":"2024-09-16T15:40:12.865293+0000","last_became_peered":"2024-09-16T15:40:12.865293+0000","last_unstale":"2024-09-16T15:40:14.147038+0000","last_undegraded":"2024-09-16T15:40:14.147038+0000","last_fullsized":"2024-09-16T15:40:14.147038+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146143+0000","last_change":"2024-09-16T15:40:12.875755+0000","last_active":"2024-09-16T15:40:14.146143+0000","last_peered":"2024-09-16T15:40:14.146143+0000","last_clean":"2024-09-16T15:40:14.146143+0000","last_became_active":"2024-09-16T15:40:12.875484+0000","last_became_peered":"2024-09-16T15:40:12.875484+0000","last_unstale":"2024-09-16T15:40:14.146143+0000","last_undegraded":"2024-09-16T15:40:14.146143+0000","last_fullsized":"2024-09-16T15:40:14.146143+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:29.291337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146022+0000","last_change":"2024-09-16T15:40:12.866408+0000","last_active":"2024-09-16T15:40:14.146022+0000","last_peered":"2024-09-16T15:40:14.146022+0000","last_clean":"2024-09-16T15:40:14.146022+0000","last_became_active":"2024-09-16T15:40:12.865787+0000","last_became_peered":"2024-09-16T15:40:12.865787+0000","last_unstale":"2024-09-16T15:40:14.146022+0000","last_undegraded":"2024-09-16T15:40:14.146022+0000","last_fullsized":"2024-09-16T15:40:14.146022+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:29.669859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146660+0000","last_change":"2024-09-16T15:40:12.865889+0000","last_active":"2024-09-16T15:40:14.146660+0000","last_peered":"2024-09-16T15:40:14.146660+0000","last_clean":"2024-09-16T15:40:14.146660+0000","last_became_active":"2024-09-16T15:40:12.865611+0000","last_became_peered":"2024-09-16T15:40:12.865611+0000","last_unstale":"2024-09-16T15:40:14.146660+0000","last_undegraded":"2024-09-16T15:40:14.146660+0000","last_fullsized":"2024-09-16T15:40:14.146660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:16.283138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146411+0000","last_change":"2024-09-16T15:40:12.863378+0000","last_active":"2024-09-16T15:40:14.146411+0000","last_peered":"2024-09-16T15:40:14.146411+0000","last_clean":"2024-09-16T15:40:14.146411+0000","last_became_active":"2024-09-16T15:40:12.863108+0000","last_became_peered":"2024-09-16T15:40:12.863108+0000","last_unstale":"2024-09-16T15:40:14.146411+0000","last_undegraded":"2024-09-16T15:40:14.146411+0000","last_fullsized":"2024-09-16T15:40:14.146411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:14.857566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.853784+0000","last_change":"2024-09-16T15:40:12.861959+0000","last_active":"2024-09-16T15:40:13.853784+0000","last_peered":"2024-09-16T15:40:13.853784+0000","last_clean":"2024-09-16T15:40:13.853784+0000","last_became_active":"2024-09-16T15:40:12.861657+0000","last_became_peered":"2024-09-16T15:40:12.861657+0000","last_unstale":"2024-09-16T15:40:13.853784+0000","last_undegraded":"2024-09-16T15:40:13.853784+0000","last_fullsized":"2024-09-16T15:40:13.853784+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:01.800635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854807+0000","last_change":"2024-09-16T15:40:12.865876+0000","last_active":"2024-09-16T15:40:13.854807+0000","last_peered":"2024-09-16T15:40:13.854807+0000","last_clean":"2024-09-16T15:40:13.854807+0000","last_became_active":"2024-09-16T15:40:12.865493+0000","last_became_peered":"2024-09-16T15:40:12.865493+0000","last_unstale":"2024-09-16T15:40:13.854807+0000","last_undegraded":"2024-09-16T15:40:13.854807+0000","last_fullsized":"2024-09-16T15:40:13.854807+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:52.111603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861503+0000","last_change":"2024-09-16T15:40:12.861173+0000","last_active":"2024-09-16T15:40:13.861503+0000","last_peered":"2024-09-16T15:40:13.861503+0000","last_clean":"2024-09-16T15:40:13.861503+0000","last_became_active":"2024-09-16T15:40:12.860306+0000","last_became_peered":"2024-09-16T15:40:12.860306+0000","last_unstale":"2024-09-16T15:40:13.861503+0000","last_undegraded":"2024-09-16T15:40:13.861503+0000","last_fullsized":"2024-09-16T15:40:13.861503+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:14.176641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861703+0000","last_change":"2024-09-16T15:40:12.873790+0000","last_active":"2024-09-16T15:40:13.861703+0000","last_peered":"2024-09-16T15:40:13.861703+0000","last_clean":"2024-09-16T15:40:13.861703+0000","last_became_active":"2024-09-16T15:40:12.873524+0000","last_became_peered":"2024-09-16T15:40:12.873524+0000","last_unstale":"2024-09-16T15:40:13.861703+0000","last_undegraded":"2024-09-16T15:40:13.861703+0000","last_fullsized":"2024-09-16T15:40:13.861703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:56.160294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.856255+0000","last_change":"2024-09-16T15:40:12.872623+0000","last_active":"2024-09-16T15:40:13.856255+0000","last_peered":"2024-09-16T15:40:13.856255+0000","last_clean":"2024-09-16T15:40:13.856255+0000","last_became_active":"2024-09-16T15:40:12.872346+0000","last_became_peered":"2024-09-16T15:40:12.872346+0000","last_unstale":"2024-09-16T15:40:13.856255+0000","last_undegraded":"2024-09-16T15:40:13.856255+0000","last_fullsized":"2024-09-16T15:40:13.856255+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:53.737499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146904+0000","last_change":"2024-09-16T15:40:12.882504+0000","last_active":"2024-09-16T15:40:14.146904+0000","last_peered":"2024-09-16T15:40:14.146904+0000","last_clean":"2024-09-16T15:40:14.146904+0000","last_became_active":"2024-09-16T15:40:12.882113+0000","last_became_peered":"2024-09-16T15:40:12.882113+0000","last_unstale":"2024-09-16T15:40:14.146904+0000","last_undegraded":"2024-09-16T15:40:14.146904+0000","last_fullsized":"2024-09-16T15:40:14.146904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:32.674720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854660+0000","last_change":"2024-09-16T15:40:12.857122+0000","last_active":"2024-09-16T15:40:13.854660+0000","last_peered":"2024-09-16T15:40:13.854660+0000","last_clean":"2024-09-16T15:40:13.854660+0000","last_became_active":"2024-09-16T15:40:12.856815+0000","last_became_peered":"2024-09-16T15:40:12.856815+0000","last_unstale":"2024-09-16T15:40:13.854660+0000","last_undegraded":"2024-09-16T15:40:13.854660+0000","last_fullsized":"2024-09-16T15:40:13.854660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:59.787752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147170+0000","last_change":"2024-09-16T15:40:12.868687+0000","last_active":"2024-09-16T15:40:14.147170+0000","last_peered":"2024-09-16T15:40:14.147170+0000","last_clean":"2024-09-16T15:40:14.147170+0000","last_became_active":"2024-09-16T15:40:12.868063+0000","last_became_peered":"2024-09-16T15:40:12.868063+0000","last_unstale":"2024-09-16T15:40:14.147170+0000","last_undegraded":"2024-09-16T15:40:14.147170+0000","last_fullsized":"2024-09-16T15:40:14.147170+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:10.213025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855108+0000","last_change":"2024-09-16T15:40:12.864426+0000","last_active":"2024-09-16T15:40:13.855108+0000","last_peered":"2024-09-16T15:40:13.855108+0000","last_clean":"2024-09-16T15:40:13.855108+0000","last_became_active":"2024-09-16T15:40:12.863762+0000","last_became_peered":"2024-09-16T15:40:12.863762+0000","last_unstale":"2024-09-16T15:40:13.855108+0000","last_undegraded":"2024-09-16T15:40:13.855108+0000","last_fullsized":"2024-09-16T15:40:13.855108+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:41.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855077+0000","last_change":"2024-09-16T15:40:12.863693+0000","last_active":"2024-09-16T15:40:13.855077+0000","last_peered":"2024-09-16T15:40:13.855077+0000","last_clean":"2024-09-16T15:40:13.855077+0000","last_became_active":"2024-09-16T15:40:12.863383+0000","last_became_peered":"2024-09-16T15:40:12.863383+0000","last_unstale":"2024-09-16T15:40:13.855077+0000","last_undegraded":"2024-09-16T15:40:13.855077+0000","last_fullsized":"2024-09-16T15:40:13.855077+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:09.305321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.856251+0000","last_change":"2024-09-16T15:40:12.856737+0000","last_active":"2024-09-16T15:40:13.856251+0000","last_peered":"2024-09-16T15:40:13.856251+0000","last_clean":"2024-09-16T15:40:13.856251+0000","last_became_active":"2024-09-16T15:40:12.856426+0000","last_became_peered":"2024-09-16T15:40:12.856426+0000","last_unstale":"2024-09-16T15:40:13.856251+0000","last_undegraded":"2024-09-16T15:40:13.856251+0000","last_fullsized":"2024-09-16T15:40:13.856251+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:42.495596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146723+0000","last_change":"2024-09-16T15:40:12.876499+0000","last_active":"2024-09-16T15:40:14.146723+0000","last_peered":"2024-09-16T15:40:14.146723+0000","last_clean":"2024-09-16T15:40:14.146723+0000","last_became_active":"2024-09-16T15:40:12.876247+0000","last_became_peered":"2024-09-16T15:40:12.876247+0000","last_unstale":"2024-09-16T15:40:14.146723+0000","last_undegraded":"2024-09-16T15:40:14.146723+0000","last_fullsized":"2024-09-16T15:40:14.146723+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:19.441026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861402+0000","last_change":"2024-09-16T15:40:12.858149+0000","last_active":"2024-09-16T15:40:13.861402+0000","last_peered":"2024-09-16T15:40:13.861402+0000","last_clean":"2024-09-16T15:40:13.861402+0000","last_became_active":"2024-09-16T15:40:12.857902+0000","last_became_peered":"2024-09-16T15:40:12.857902+0000","last_unstale":"2024-09-16T15:40:13.861402+0000","last_undegraded":"2024-09-16T15:40:13.861402+0000","last_fullsized":"2024-09-16T15:40:13.861402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:37.753229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146163+0000","last_change":"2024-09-16T15:40:12.865956+0000","last_active":"2024-09-16T15:40:14.146163+0000","last_peered":"2024-09-16T15:40:14.146163+0000","last_clean":"2024-09-16T15:40:14.146163+0000","last_became_active":"2024-09-16T15:40:12.865681+0000","last_became_peered":"2024-09-16T15:40:12.865681+0000","last_unstale":"2024-09-16T15:40:14.146163+0000","last_undegraded":"2024-09-16T15:40:14.146163+0000","last_fullsized":"2024-09-16T15:40:14.146163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:51.377166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855836+0000","last_change":"2024-09-16T15:40:12.856711+0000","last_active":"2024-09-16T15:40:13.855836+0000","last_peered":"2024-09-16T15:40:13.855836+0000","last_clean":"2024-09-16T15:40:13.855836+0000","last_became_active":"2024-09-16T15:40:12.856290+0000","last_became_peered":"2024-09-16T15:40:12.856290+0000","last_unstale":"2024-09-16T15:40:13.855836+0000","last_undegraded":"2024-09-16T15:40:13.855836+0000","last_fullsized":"2024-09-16T15:40:13.855836+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:17.385382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861184+0000","last_change":"2024-09-16T15:40:12.859686+0000","last_active":"2024-09-16T15:40:13.861184+0000","last_peered":"2024-09-16T15:40:13.861184+0000","last_clean":"2024-09-16T15:40:13.861184+0000","last_became_active":"2024-09-16T15:40:12.859399+0000","last_became_peered":"2024-09-16T15:40:12.859399+0000","last_unstale":"2024-09-16T15:40:13.861184+0000","last_undegraded":"2024-09-16T15:40:13.861184+0000","last_fullsized":"2024-09-16T15:40:13.861184+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:45.087681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.853812+0000","last_change":"2024-09-16T15:40:12.857344+0000","last_active":"2024-09-16T15:40:13.853812+0000","last_peered":"2024-09-16T15:40:13.853812+0000","last_clean":"2024-09-16T15:40:13.853812+0000","last_became_active":"2024-09-16T15:40:12.857044+0000","last_became_peered":"2024-09-16T15:40:12.857044+0000","last_unstale":"2024-09-16T15:40:13.853812+0000","last_undegraded":"2024-09-16T15:40:13.853812+0000","last_fullsized":"2024-09-16T15:40:13.853812+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:28:04.141653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146470+0000","last_change":"2024-09-16T15:40:12.860869+0000","last_active":"2024-09-16T15:40:14.146470+0000","last_peered":"2024-09-16T15:40:14.146470+0000","last_clean":"2024-09-16T15:40:14.146470+0000","last_became_active":"2024-09-16T15:40:12.860241+0000","last_became_peered":"2024-09-16T15:40:12.860241+0000","last_unstale":"2024-09-16T15:40:14.146470+0000","last_undegraded":"2024-09-16T15:40:14.146470+0000","last_fullsized":"2024-09-16T15:40:14.146470+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:04:49.946427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147666+0000","last_change":"2024-09-16T15:40:12.856388+0000","last_active":"2024-09-16T15:40:14.147666+0000","last_peered":"2024-09-16T15:40:14.147666+0000","last_clean":"2024-09-16T15:40:14.147666+0000","last_became_active":"2024-09-16T15:40:12.854943+0000","last_became_peered":"2024-09-16T15:40:12.854943+0000","last_unstale":"2024-09-16T15:40:14.147666+0000","last_undegraded":"2024-09-16T15:40:14.147666+0000","last_fullsized":"2024-09-16T15:40:14.147666+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:10.844833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854411+0000","last_change":"2024-09-16T15:40:11.859931+0000","last_active":"2024-09-16T15:40:13.854411+0000","last_peered":"2024-09-16T15:40:13.854411+0000","last_clean":"2024-09-16T15:40:13.854411+0000","last_became_active":"2024-09-16T15:40:11.859675+0000","last_became_peered":"2024-09-16T15:40:11.859675+0000","last_unstale":"2024-09-16T15:40:13.854411+0000","last_undegraded":"2024-09-16T15:40:13.854411+0000","last_fullsized":"2024-09-16T15:40:13.854411+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:28.283914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.856289+0000","last_change":"2024-09-16T15:40:12.861984+0000","last_active":"2024-09-16T15:40:13.856289+0000","last_peered":"2024-09-16T15:40:13.856289+0000","last_clean":"2024-09-16T15:40:13.856289+0000","last_became_active":"2024-09-16T15:40:12.861759+0000","last_became_peered":"2024-09-16T15:40:12.861759+0000","last_unstale":"2024-09-16T15:40:13.856289+0000","last_undegraded":"2024-09-16T15:40:13.856289+0000","last_fullsized":"2024-09-16T15:40:13.856289+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:37.094606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861649+0000","last_change":"2024-09-16T15:40:11.863371+0000","last_active":"2024-09-16T15:40:13.861649+0000","last_peered":"2024-09-16T15:40:13.861649+0000","last_clean":"2024-09-16T15:40:13.861649+0000","last_became_active":"2024-09-16T15:40:11.860269+0000","last_became_peered":"2024-09-16T15:40:11.860269+0000","last_unstale":"2024-09-16T15:40:13.861649+0000","last_undegraded":"2024-09-16T15:40:13.861649+0000","last_fullsized":"2024-09-16T15:40:13.861649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:36.282583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146340+0000","last_change":"2024-09-16T15:40:12.855039+0000","last_active":"2024-09-16T15:40:14.146340+0000","last_peered":"2024-09-16T15:40:14.146340+0000","last_clean":"2024-09-16T15:40:14.146340+0000","last_became_active":"2024-09-16T15:40:12.854141+0000","last_became_peered":"2024-09-16T15:40:12.854141+0000","last_unstale":"2024-09-16T15:40:14.146340+0000","last_undegraded":"2024-09-16T15:40:14.146340+0000","last_fullsized":"2024-09-16T15:40:14.146340+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:35.868247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147699+0000","last_change":"2024-09-16T15:40:11.869853+0000","last_active":"2024-09-16T15:40:14.147699+0000","last_peered":"2024-09-16T15:40:14.147699+0000","last_clean":"2024-09-16T15:40:14.147699+0000","last_became_active":"2024-09-16T15:40:11.869583+0000","last_became_peered":"2024-09-16T15:40:11.869583+0000","last_unstale":"2024-09-16T15:40:14.147699+0000","last_undegraded":"2024-09-16T15:40:14.147699+0000","last_fullsized":"2024-09-16T15:40:14.147699+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:50.423580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147687+0000","last_change":"2024-09-16T15:40:12.853403+0000","last_active":"2024-09-16T15:40:14.147687+0000","last_peered":"2024-09-16T15:40:14.147687+0000","last_clean":"2024-09-16T15:40:14.147687+0000","last_became_active":"2024-09-16T15:40:12.853161+0000","last_became_peered":"2024-09-16T15:40:12.853161+0000","last_unstale":"2024-09-16T15:40:14.147687+0000","last_undegraded":"2024-09-16T15:40:14.147687+0000","last_fullsized":"2024-09-16T15:40:14.147687+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:30.287843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854937+0000","last_change":"2024-09-16T15:40:11.868506+0000","last_active":"2024-09-16T15:40:13.854937+0000","last_peered":"2024-09-16T15:40:13.854937+0000","last_clean":"2024-09-16T15:40:13.854937+0000","last_became_active":"2024-09-16T15:40:11.868014+0000","last_became_peered":"2024-09-16T15:40:11.868014+0000","last_unstale":"2024-09-16T15:40:13.854937+0000","last_undegraded":"2024-09-16T15:40:13.854937+0000","last_fullsized":"2024-09-16T15:40:13.854937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:38.292320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861226+0000","last_change":"2024-09-16T15:40:12.867667+0000","last_active":"2024-09-16T15:40:13.861226+0000","last_peered":"2024-09-16T15:40:13.861226+0000","last_clean":"2024-09-16T15:40:13.861226+0000","last_became_active":"2024-09-16T15:40:12.866213+0000","last_became_peered":"2024-09-16T15:40:12.866213+0000","last_unstale":"2024-09-16T15:40:13.861226+0000","last_undegraded":"2024-09-16T15:40:13.861226+0000","last_fullsized":"2024-09-16T15:40:13.861226+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:27.898036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854443+0000","last_change":"2024-09-16T15:40:11.871006+0000","last_active":"2024-09-16T15:40:13.854443+0000","last_peered":"2024-09-16T15:40:13.854443+0000","last_clean":"2024-09-16T15:40:13.854443+0000","last_became_active":"2024-09-16T15:40:11.870728+0000","last_became_peered":"2024-09-16T15:40:11.870728+0000","last_unstale":"2024-09-16T15:40:13.854443+0000","last_undegraded":"2024-09-16T15:40:13.854443+0000","last_fullsized":"2024-09-16T15:40:13.854443+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:26.890614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145878+0000","last_change":"2024-09-16T15:40:12.869124+0000","last_active":"2024-09-16T15:40:14.145878+0000","last_peered":"2024-09-16T15:40:14.145878+0000","last_clean":"2024-09-16T15:40:14.145878+0000","last_became_active":"2024-09-16T15:40:12.868857+0000","last_became_peered":"2024-09-16T15:40:12.868857+0000","last_unstale":"2024-09-16T15:40:14.145878+0000","last_undegraded":"2024-09-16T15:40:14.145878+0000","last_fullsized":"2024-09-16T15:40:14.145878+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:26.535001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861362+0000","last_change":"2024-09-16T15:40:11.865117+0000","last_active":"2024-09-16T15:40:13.861362+0000","last_peered":"2024-09-16T15:40:13.861362+0000","last_clean":"2024-09-16T15:40:13.861362+0000","last_became_active":"2024-09-16T15:40:11.864766+0000","last_became_peered":"2024-09-16T15:40:11.864766+0000","last_unstale":"2024-09-16T15:40:13.861362+0000","last_undegraded":"2024-09-16T15:40:13.861362+0000","last_fullsized":"2024-09-16T15:40:13.861362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:07.999976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145776+0000","last_change":"2024-09-16T15:40:12.860867+0000","last_active":"2024-09-16T15:40:14.145776+0000","last_peered":"2024-09-16T15:40:14.145776+0000","last_clean":"2024-09-16T15:40:14.145776+0000","last_became_active":"2024-09-16T15:40:12.860648+0000","last_became_peered":"2024-09-16T15:40:12.860648+0000","last_unstale":"2024-09-16T15:40:14.145776+0000","last_undegraded":"2024-09-16T15:40:14.145776+0000","last_fullsized":"2024-09-16T15:40:14.145776+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:45.087681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145793+0000","last_change":"2024-09-16T15:40:11.868600+0000","last_active":"2024-09-16T15:40:14.145793+0000","last_peered":"2024-09-16T15:40:14.145793+0000","last_clean":"2024-09-16T15:40:14.145793+0000","last_became_active":"2024-09-16T15:40:11.868271+0000","last_became_peered":"2024-09-16T15:40:11.868271+0000","last_unstale":"2024-09-16T15:40:14.145793+0000","last_undegraded":"2024-09-16T15:40:14.145793+0000","last_fullsized":"2024-09-16T15:40:14.145793+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:56.197404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854868+0000","last_change":"2024-09-16T15:40:12.864230+0000","last_active":"2024-09-16T15:40:13.854868+0000","last_peered":"2024-09-16T15:40:13.854868+0000","last_clean":"2024-09-16T15:40:13.854868+0000","last_became_active":"2024-09-16T15:40:12.863371+0000","last_became_peered":"2024-09-16T15:40:12.863371+0000","last_unstale":"2024-09-16T15:40:13.854868+0000","last_undegraded":"2024-09-16T15:40:13.854868+0000","last_fullsized":"2024-09-16T15:40:13.854868+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:53:57.882302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145864+0000","last_change":"2024-09-16T15:40:11.864603+0000","last_active":"2024-09-16T15:40:14.145864+0000","last_peered":"2024-09-16T15:40:14.145864+0000","last_clean":"2024-09-16T15:40:14.145864+0000","last_became_active":"2024-09-16T15:40:11.864256+0000","last_became_peered":"2024-09-16T15:40:11.864256+0000","last_unstale":"2024-09-16T15:40:14.145864+0000","last_undegraded":"2024-09-16T15:40:14.145864+0000","last_fullsized":"2024-09-16T15:40:14.145864+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:34.607613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861500+0000","last_change":"2024-09-16T15:40:12.873524+0000","last_active":"2024-09-16T15:40:13.861500+0000","last_peered":"2024-09-16T15:40:13.861500+0000","last_clean":"2024-09-16T15:40:13.861500+0000","last_became_active":"2024-09-16T15:40:12.873256+0000","last_became_peered":"2024-09-16T15:40:12.873256+0000","last_unstale":"2024-09-16T15:40:13.861500+0000","last_undegraded":"2024-09-16T15:40:13.861500+0000","last_fullsized":"2024-09-16T15:40:13.861500+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:11.366608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861542+0000","last_change":"2024-09-16T15:40:11.860367+0000","last_active":"2024-09-16T15:40:13.861542+0000","last_peered":"2024-09-16T15:40:13.861542+0000","last_clean":"2024-09-16T15:40:13.861542+0000","last_became_active":"2024-09-16T15:40:11.858703+0000","last_became_peered":"2024-09-16T15:40:11.858703+0000","last_unstale":"2024-09-16T15:40:13.861542+0000","last_undegraded":"2024-09-16T15:40:13.861542+0000","last_fullsized":"2024-09-16T15:40:13.861542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:47.251591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855264+0000","last_change":"2024-09-16T15:40:12.865158+0000","last_active":"2024-09-16T15:40:13.855264+0000","last_peered":"2024-09-16T15:40:13.855264+0000","last_clean":"2024-09-16T15:40:13.855264+0000","last_became_active":"2024-09-16T15:40:12.864675+0000","last_became_peered":"2024-09-16T15:40:12.864675+0000","last_unstale":"2024-09-16T15:40:13.855264+0000","last_undegraded":"2024-09-16T15:40:13.855264+0000","last_fullsized":"2024-09-16T15:40:13.855264+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:35.868247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147193+0000","last_change":"2024-09-16T15:40:11.863611+0000","last_active":"2024-09-16T15:40:14.147193+0000","last_peered":"2024-09-16T15:40:14.147193+0000","last_clean":"2024-09-16T15:40:14.147193+0000","last_became_active":"2024-09-16T15:40:11.863346+0000","last_became_peered":"2024-09-16T15:40:11.863346+0000","last_unstale":"2024-09-16T15:40:14.147193+0000","last_undegraded":"2024-09-16T15:40:14.147193+0000","last_fullsized":"2024-09-16T15:40:14.147193+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:00.681742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854387+0000","last_change":"2024-09-16T15:40:12.876615+0000","last_active":"2024-09-16T15:40:13.854387+0000","last_peered":"2024-09-16T15:40:13.854387+0000","last_clean":"2024-09-16T15:40:13.854387+0000","last_became_active":"2024-09-16T15:40:12.876385+0000","last_became_peered":"2024-09-16T15:40:12.876385+0000","last_unstale":"2024-09-16T15:40:13.854387+0000","last_undegraded":"2024-09-16T15:40:13.854387+0000","last_fullsized":"2024-09-16T15:40:13.854387+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:10.231811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855675+0000","last_change":"2024-09-16T15:40:11.864331+0000","last_active":"2024-09-16T15:40:13.855675+0000","last_peered":"2024-09-16T15:40:13.855675+0000","last_clean":"2024-09-16T15:40:13.855675+0000","last_became_active":"2024-09-16T15:40:11.864083+0000","last_became_peered":"2024-09-16T15:40:11.864083+0000","last_unstale":"2024-09-16T15:40:13.855675+0000","last_undegraded":"2024-09-16T15:40:13.855675+0000","last_fullsized":"2024-09-16T15:40:13.855675+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:44.080258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855491+0000","last_change":"2024-09-16T15:40:12.865124+0000","last_active":"2024-09-16T15:40:13.855491+0000","last_peered":"2024-09-16T15:40:13.855491+0000","last_clean":"2024-09-16T15:40:13.855491+0000","last_became_active":"2024-09-16T15:40:12.864809+0000","last_became_peered":"2024-09-16T15:40:12.864809+0000","last_unstale":"2024-09-16T15:40:13.855491+0000","last_undegraded":"2024-09-16T15:40:13.855491+0000","last_fullsized":"2024-09-16T15:40:13.855491+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:56.172520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":21,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146897+0000","last_change":"2024-09-16T15:40:11.865511+0000","last_active":"2024-09-16T15:40:14.146897+0000","last_peered":"2024-09-16T15:40:14.146897+0000","last_clean":"2024-09-16T15:40:14.146897+0000","last_became_active":"2024-09-16T15:40:11.863715+0000","last_became_peered":"2024-09-16T15:40:11.863715+0000","last_unstale":"2024-09-16T15:40:14.146897+0000","last_undegraded":"2024-09-16T15:40:14.146897+0000","last_fullsized":"2024-09-16T15:40:14.146897+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:18.604944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855064+0000","last_change":"2024-09-16T15:40:12.871040+0000","last_active":"2024-09-16T15:40:13.855064+0000","last_peered":"2024-09-16T15:40:13.855064+0000","last_clean":"2024-09-16T15:40:13.855064+0000","last_became_active":"2024-09-16T15:40:12.870459+0000","last_became_peered":"2024-09-16T15:40:12.870459+0000","last_unstale":"2024-09-16T15:40:13.855064+0000","last_undegraded":"2024-09-16T15:40:13.855064+0000","last_fullsized":"2024-09-16T15:40:13.855064+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:39.789180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854549+0000","last_change":"2024-09-16T15:40:11.870606+0000","last_active":"2024-09-16T15:40:13.854549+0000","last_peered":"2024-09-16T15:40:13.854549+0000","last_clean":"2024-09-16T15:40:13.854549+0000","last_became_active":"2024-09-16T15:40:11.868146+0000","last_became_peered":"2024-09-16T15:40:11.868146+0000","last_unstale":"2024-09-16T15:40:13.854549+0000","last_undegraded":"2024-09-16T15:40:13.854549+0000","last_fullsized":"2024-09-16T15:40:13.854549+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:36.745807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146322+0000","last_change":"2024-09-16T15:40:12.860151+0000","last_active":"2024-09-16T15:40:14.146322+0000","last_peered":"2024-09-16T15:40:14.146322+0000","last_clean":"2024-09-16T15:40:14.146322+0000","last_became_active":"2024-09-16T15:40:12.859179+0000","last_became_peered":"2024-09-16T15:40:12.859179+0000","last_unstale":"2024-09-16T15:40:14.146322+0000","last_undegraded":"2024-09-16T15:40:14.146322+0000","last_fullsized":"2024-09-16T15:40:14.146322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:12.149041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855449+0000","last_change":"2024-09-16T15:40:11.863101+0000","last_active":"2024-09-16T15:40:13.855449+0000","last_peered":"2024-09-16T15:40:13.855449+0000","last_clean":"2024-09-16T15:40:13.855449+0000","last_became_active":"2024-09-16T15:40:11.862829+0000","last_became_peered":"2024-09-16T15:40:11.862829+0000","last_unstale":"2024-09-16T15:40:13.855449+0000","last_undegraded":"2024-09-16T15:40:13.855449+0000","last_fullsized":"2024-09-16T15:40:13.855449+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:25.527579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.853763+0000","last_change":"2024-09-16T15:40:12.871218+0000","last_active":"2024-09-16T15:40:13.853763+0000","last_peered":"2024-09-16T15:40:13.853763+0000","last_clean":"2024-09-16T15:40:13.853763+0000","last_became_active":"2024-09-16T15:40:12.870997+0000","last_became_peered":"2024-09-16T15:40:12.870997+0000","last_unstale":"2024-09-16T15:40:13.853763+0000","last_undegraded":"2024-09-16T15:40:13.853763+0000","last_fullsized":"2024-09-16T15:40:13.853763+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:53.048708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.853934+0000","last_change":"2024-09-16T15:40:11.867251+0000","last_active":"2024-09-16T15:40:13.853934+0000","last_peered":"2024-09-16T15:40:13.853934+0000","last_clean":"2024-09-16T15:40:13.853934+0000","last_became_active":"2024-09-16T15:40:11.866730+0000","last_became_peered":"2024-09-16T15:40:11.866730+0000","last_unstale":"2024-09-16T15:40:13.853934+0000","last_undegraded":"2024-09-16T15:40:13.853934+0000","last_fullsized":"2024-09-16T15:40:13.853934+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:55.152872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855524+0000","last_change":"2024-09-16T15:40:12.869762+0000","last_active":"2024-09-16T15:40:13.855524+0000","last_peered":"2024-09-16T15:40:13.855524+0000","last_clean":"2024-09-16T15:40:13.855524+0000","last_became_active":"2024-09-16T15:40:12.869420+0000","last_became_peered":"2024-09-16T15:40:12.869420+0000","last_unstale":"2024-09-16T15:40:13.855524+0000","last_undegraded":"2024-09-16T15:40:13.855524+0000","last_fullsized":"2024-09-16T15:40:13.855524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:59.226336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":23,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146065+0000","last_change":"2024-09-16T15:40:11.868078+0000","last_active":"2024-09-16T15:40:14.146065+0000","last_peered":"2024-09-16T15:40:14.146065+0000","last_clean":"2024-09-16T15:40:14.146065+0000","last_became_active":"2024-09-16T15:40:11.867825+0000","last_became_peered":"2024-09-16T15:40:11.867825+0000","last_unstale":"2024-09-16T15:40:14.146065+0000","last_undegraded":"2024-09-16T15:40:14.146065+0000","last_fullsized":"2024-09-16T15:40:14.146065+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:23:29.103104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146916+0000","last_change":"2024-09-16T15:40:12.867357+0000","last_active":"2024-09-16T15:40:14.146916+0000","last_peered":"2024-09-16T15:40:14.146916+0000","last_clean":"2024-09-16T15:40:14.146916+0000","last_became_active":"2024-09-16T15:40:12.867072+0000","last_became_peered":"2024-09-16T15:40:12.867072+0000","last_unstale":"2024-09-16T15:40:14.146916+0000","last_undegraded":"2024-09-16T15:40:14.146916+0000","last_fullsized":"2024-09-16T15:40:14.146916+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:46.347083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147101+0000","last_change":"2024-09-16T15:40:11.866281+0000","last_active":"2024-09-16T15:40:14.147101+0000","last_peered":"2024-09-16T15:40:14.147101+0000","last_clean":"2024-09-16T15:40:14.147101+0000","last_became_active":"2024-09-16T15:40:11.865322+0000","last_became_peered":"2024-09-16T15:40:11.865322+0000","last_unstale":"2024-09-16T15:40:14.147101+0000","last_undegraded":"2024-09-16T15:40:14.147101+0000","last_fullsized":"2024-09-16T15:40:14.147101+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:14:13.585151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147357+0000","last_change":"2024-09-16T15:40:12.874698+0000","last_active":"2024-09-16T15:40:14.147357+0000","last_peered":"2024-09-16T15:40:14.147357+0000","last_clean":"2024-09-16T15:40:14.147357+0000","last_became_active":"2024-09-16T15:40:12.874345+0000","last_became_peered":"2024-09-16T15:40:12.874345+0000","last_unstale":"2024-09-16T15:40:14.147357+0000","last_undegraded":"2024-09-16T15:40:14.147357+0000","last_fullsized":"2024-09-16T15:40:14.147357+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:35.868247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861755+0000","last_change":"2024-09-16T15:40:11.859504+0000","last_active":"2024-09-16T15:40:13.861755+0000","last_peered":"2024-09-16T15:40:13.861755+0000","last_clean":"2024-09-16T15:40:13.861755+0000","last_became_active":"2024-09-16T15:40:11.859183+0000","last_became_peered":"2024-09-16T15:40:11.859183+0000","last_unstale":"2024-09-16T15:40:13.861755+0000","last_undegraded":"2024-09-16T15:40:13.861755+0000","last_fullsized":"2024-09-16T15:40:13.861755+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:57.973947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.856008+0000","last_change":"2024-09-16T15:40:12.856434+0000","last_active":"2024-09-16T15:40:13.856008+0000","last_peered":"2024-09-16T15:40:13.856008+0000","last_clean":"2024-09-16T15:40:13.856008+0000","last_became_active":"2024-09-16T15:40:12.856073+0000","last_became_peered":"2024-09-16T15:40:12.856073+0000","last_unstale":"2024-09-16T15:40:13.856008+0000","last_undegraded":"2024-09-16T15:40:13.856008+0000","last_fullsized":"2024-09-16T15:40:13.856008+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:30.624614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146142+0000","last_change":"2024-09-16T15:40:11.869221+0000","last_active":"2024-09-16T15:40:14.146142+0000","last_peered":"2024-09-16T15:40:14.146142+0000","last_clean":"2024-09-16T15:40:14.146142+0000","last_became_active":"2024-09-16T15:40:11.868892+0000","last_became_peered":"2024-09-16T15:40:11.868892+0000","last_unstale":"2024-09-16T15:40:14.146142+0000","last_undegraded":"2024-09-16T15:40:14.146142+0000","last_fullsized":"2024-09-16T15:40:14.146142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.856163+0000","last_change":"2024-09-16T15:40:12.859954+0000","last_active":"2024-09-16T15:40:13.856163+0000","last_peered":"2024-09-16T15:40:13.856163+0000","last_clean":"2024-09-16T15:40:13.856163+0000","last_became_active":"2024-09-16T15:40:12.859695+0000","last_became_peered":"2024-09-16T15:40:12.859695+0000","last_unstale":"2024-09-16T15:40:13.856163+0000","last_undegraded":"2024-09-16T15:40:13.856163+0000","last_fullsized":"2024-09-16T15:40:13.856163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:26.182820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854102+0000","last_change":"2024-09-16T15:40:11.863949+0000","last_active":"2024-09-16T15:40:13.854102+0000","last_peered":"2024-09-16T15:40:13.854102+0000","last_clean":"2024-09-16T15:40:13.854102+0000","last_became_active":"2024-09-16T15:40:11.863620+0000","last_became_peered":"2024-09-16T15:40:11.863620+0000","last_unstale":"2024-09-16T15:40:13.854102+0000","last_undegraded":"2024-09-16T15:40:13.854102+0000","last_fullsized":"2024-09-16T15:40:13.854102+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:18.083324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.853981+0000","last_change":"2024-09-16T15:40:12.873993+0000","last_active":"2024-09-16T15:40:13.853981+0000","last_peered":"2024-09-16T15:40:13.853981+0000","last_clean":"2024-09-16T15:40:13.853981+0000","last_became_active":"2024-09-16T15:40:12.873720+0000","last_became_peered":"2024-09-16T15:40:12.873720+0000","last_unstale":"2024-09-16T15:40:13.853981+0000","last_undegraded":"2024-09-16T15:40:13.853981+0000","last_fullsized":"2024-09-16T15:40:13.853981+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:17.565042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854752+0000","last_change":"2024-09-16T15:40:11.871266+0000","last_active":"2024-09-16T15:40:13.854752+0000","last_peered":"2024-09-16T15:40:13.854752+0000","last_clean":"2024-09-16T15:40:13.854752+0000","last_became_active":"2024-09-16T15:40:11.871013+0000","last_became_peered":"2024-09-16T15:40:11.871013+0000","last_unstale":"2024-09-16T15:40:13.854752+0000","last_undegraded":"2024-09-16T15:40:13.854752+0000","last_fullsized":"2024-09-16T15:40:13.854752+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:10.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147184+0000","last_change":"2024-09-16T15:40:12.850407+0000","last_active":"2024-09-16T15:40:14.147184+0000","last_peered":"2024-09-16T15:40:14.147184+0000","last_clean":"2024-09-16T15:40:14.147184+0000","last_became_active":"2024-09-16T15:40:12.849693+0000","last_became_peered":"2024-09-16T15:40:12.849693+0000","last_unstale":"2024-09-16T15:40:14.147184+0000","last_undegraded":"2024-09-16T15:40:14.147184+0000","last_fullsized":"2024-09-16T15:40:14.147184+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:48.268979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145822+0000","last_change":"2024-09-16T15:40:11.865942+0000","last_active":"2024-09-16T15:40:14.145822+0000","last_peered":"2024-09-16T15:40:14.145822+0000","last_clean":"2024-09-16T15:40:14.145822+0000","last_became_active":"2024-09-16T15:40:11.864112+0000","last_became_peered":"2024-09-16T15:40:11.864112+0000","last_unstale":"2024-09-16T15:40:14.145822+0000","last_undegraded":"2024-09-16T15:40:14.145822+0000","last_fullsized":"2024-09-16T15:40:14.145822+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:13.011662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146509+0000","last_change":"2024-09-16T15:40:12.861166+0000","last_active":"2024-09-16T15:40:14.146509+0000","last_peered":"2024-09-16T15:40:14.146509+0000","last_clean":"2024-09-16T15:40:14.146509+0000","last_became_active":"2024-09-16T15:40:12.860619+0000","last_became_peered":"2024-09-16T15:40:12.860619+0000","last_unstale":"2024-09-16T15:40:14.146509+0000","last_undegraded":"2024-09-16T15:40:14.146509+0000","last_fullsized":"2024-09-16T15:40:14.146509+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:22.890221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145837+0000","last_change":"2024-09-16T15:40:11.862288+0000","last_active":"2024-09-16T15:40:14.145837+0000","last_peered":"2024-09-16T15:40:14.145837+0000","last_clean":"2024-09-16T15:40:14.145837+0000","last_became_active":"2024-09-16T15:40:11.862002+0000","last_became_peered":"2024-09-16T15:40:11.862002+0000","last_unstale":"2024-09-16T15:40:14.145837+0000","last_undegraded":"2024-09-16T15:40:14.145837+0000","last_fullsized":"2024-09-16T15:40:14.145837+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:12.191215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145739+0000","last_change":"2024-09-16T15:40:12.879430+0000","last_active":"2024-09-16T15:40:14.145739+0000","last_peered":"2024-09-16T15:40:14.145739+0000","last_clean":"2024-09-16T15:40:14.145739+0000","last_became_active":"2024-09-16T15:40:12.879172+0000","last_became_peered":"2024-09-16T15:40:12.879172+0000","last_unstale":"2024-09-16T15:40:14.145739+0000","last_undegraded":"2024-09-16T15:40:14.145739+0000","last_fullsized":"2024-09-16T15:40:14.145739+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:39.620263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855117+0000","last_change":"2024-09-16T15:40:11.863062+0000","last_active":"2024-09-16T15:40:13.855117+0000","last_peered":"2024-09-16T15:40:13.855117+0000","last_clean":"2024-09-16T15:40:13.855117+0000","last_became_active":"2024-09-16T15:40:11.860714+0000","last_became_peered":"2024-09-16T15:40:11.860714+0000","last_unstale":"2024-09-16T15:40:13.855117+0000","last_undegraded":"2024-09-16T15:40:13.855117+0000","last_fullsized":"2024-09-16T15:40:13.855117+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:58.991736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.856391+0000","last_change":"2024-09-16T15:40:12.883082+0000","last_active":"2024-09-16T15:40:13.856391+0000","last_peered":"2024-09-16T15:40:13.856391+0000","last_clean":"2024-09-16T15:40:13.856391+0000","last_became_active":"2024-09-16T15:40:12.882695+0000","last_became_peered":"2024-09-16T15:40:12.882695+0000","last_unstale":"2024-09-16T15:40:13.856391+0000","last_undegraded":"2024-09-16T15:40:13.856391+0000","last_fullsized":"2024-09-16T15:40:13.856391+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:01.507433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854332+0000","last_change":"2024-09-16T15:40:11.867238+0000","last_active":"2024-09-16T15:40:13.854332+0000","last_peered":"2024-09-16T15:40:13.854332+0000","last_clean":"2024-09-16T15:40:13.854332+0000","last_became_active":"2024-09-16T15:40:11.865996+0000","last_became_peered":"2024-09-16T15:40:11.865996+0000","last_unstale":"2024-09-16T15:40:13.854332+0000","last_undegraded":"2024-09-16T15:40:13.854332+0000","last_fullsized":"2024-09-16T15:40:13.854332+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:23.953426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146190+0000","last_change":"2024-09-16T15:40:12.872931+0000","last_active":"2024-09-16T15:40:14.146190+0000","last_peered":"2024-09-16T15:40:14.146190+0000","last_clean":"2024-09-16T15:40:14.146190+0000","last_became_active":"2024-09-16T15:40:12.872344+0000","last_became_peered":"2024-09-16T15:40:12.872344+0000","last_unstale":"2024-09-16T15:40:14.146190+0000","last_undegraded":"2024-09-16T15:40:14.146190+0000","last_fullsized":"2024-09-16T15:40:14.146190+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:15.043783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855773+0000","last_change":"2024-09-16T15:40:11.865216+0000","last_active":"2024-09-16T15:40:13.855773+0000","last_peered":"2024-09-16T15:40:13.855773+0000","last_clean":"2024-09-16T15:40:13.855773+0000","last_became_active":"2024-09-16T15:40:11.864945+0000","last_became_peered":"2024-09-16T15:40:11.864945+0000","last_unstale":"2024-09-16T15:40:13.855773+0000","last_undegraded":"2024-09-16T15:40:13.855773+0000","last_fullsized":"2024-09-16T15:40:13.855773+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:13.850144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855692+0000","last_change":"2024-09-16T15:40:12.868439+0000","last_active":"2024-09-16T15:40:13.855692+0000","last_peered":"2024-09-16T15:40:13.855692+0000","last_clean":"2024-09-16T15:40:13.855692+0000","last_became_active":"2024-09-16T15:40:12.867651+0000","last_became_peered":"2024-09-16T15:40:12.867651+0000","last_unstale":"2024-09-16T15:40:13.855692+0000","last_undegraded":"2024-09-16T15:40:13.855692+0000","last_fullsized":"2024-09-16T15:40:13.855692+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:27.010957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146873+0000","last_change":"2024-09-16T15:40:11.869272+0000","last_active":"2024-09-16T15:40:14.146873+0000","last_peered":"2024-09-16T15:40:14.146873+0000","last_clean":"2024-09-16T15:40:14.146873+0000","last_became_active":"2024-09-16T15:40:11.868909+0000","last_became_peered":"2024-09-16T15:40:11.868909+0000","last_unstale":"2024-09-16T15:40:14.146873+0000","last_undegraded":"2024-09-16T15:40:14.146873+0000","last_fullsized":"2024-09-16T15:40:14.146873+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:57.684406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854549+0000","last_change":"2024-09-16T15:40:12.862103+0000","last_active":"2024-09-16T15:40:13.854549+0000","last_peered":"2024-09-16T15:40:13.854549+0000","last_clean":"2024-09-16T15:40:13.854549+0000","last_became_active":"2024-09-16T15:40:12.861788+0000","last_became_peered":"2024-09-16T15:40:12.861788+0000","last_unstale":"2024-09-16T15:40:13.854549+0000","last_undegraded":"2024-09-16T15:40:13.854549+0000","last_fullsized":"2024-09-16T15:40:13.854549+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:57.800962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855507+0000","last_change":"2024-09-16T15:40:11.867570+0000","last_active":"2024-09-16T15:40:13.855507+0000","last_peered":"2024-09-16T15:40:13.855507+0000","last_clean":"2024-09-16T15:40:13.855507+0000","last_became_active":"2024-09-16T15:40:11.867301+0000","last_became_peered":"2024-09-16T15:40:11.867301+0000","last_unstale":"2024-09-16T15:40:13.855507+0000","last_undegraded":"2024-09-16T15:40:13.855507+0000","last_fullsized":"2024-09-16T15:40:13.855507+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.854250+0000","last_change":"2024-09-16T15:40:12.875892+0000","last_active":"2024-09-16T15:40:13.854250+0000","last_peered":"2024-09-16T15:40:13.854250+0000","last_clean":"2024-09-16T15:40:13.854250+0000","last_became_active":"2024-09-16T15:40:12.875619+0000","last_became_peered":"2024-09-16T15:40:12.875619+0000","last_unstale":"2024-09-16T15:40:13.854250+0000","last_undegraded":"2024-09-16T15:40:13.854250+0000","last_fullsized":"2024-09-16T15:40:13.854250+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:41.967846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861420+0000","last_change":"2024-09-16T15:40:11.867392+0000","last_active":"2024-09-16T15:40:13.861420+0000","last_peered":"2024-09-16T15:40:13.861420+0000","last_clean":"2024-09-16T15:40:13.861420+0000","last_became_active":"2024-09-16T15:40:11.867003+0000","last_became_peered":"2024-09-16T15:40:11.867003+0000","last_unstale":"2024-09-16T15:40:13.861420+0000","last_undegraded":"2024-09-16T15:40:13.861420+0000","last_fullsized":"2024-09-16T15:40:13.861420+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:00.706403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145864+0000","last_change":"2024-09-16T15:40:12.850142+0000","last_active":"2024-09-16T15:40:14.145864+0000","last_peered":"2024-09-16T15:40:14.145864+0000","last_clean":"2024-09-16T15:40:14.145864+0000","last_became_active":"2024-09-16T15:40:12.849859+0000","last_became_peered":"2024-09-16T15:40:12.849859+0000","last_unstale":"2024-09-16T15:40:14.145864+0000","last_undegraded":"2024-09-16T15:40:14.145864+0000","last_fullsized":"2024-09-16T15:40:14.145864+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:36.820991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146222+0000","last_change":"2024-09-16T15:40:11.860188+0000","last_active":"2024-09-16T15:40:14.146222+0000","last_peered":"2024-09-16T15:40:14.146222+0000","last_clean":"2024-09-16T15:40:14.146222+0000","last_became_active":"2024-09-16T15:40:11.859924+0000","last_became_peered":"2024-09-16T15:40:11.859924+0000","last_unstale":"2024-09-16T15:40:14.146222+0000","last_undegraded":"2024-09-16T15:40:14.146222+0000","last_fullsized":"2024-09-16T15:40:14.146222+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:47.670989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.861165+0000","last_change":"2024-09-16T15:40:12.862284+0000","last_active":"2024-09-16T15:40:13.861165+0000","last_peered":"2024-09-16T15:40:13.861165+0000","last_clean":"2024-09-16T15:40:13.861165+0000","last_became_active":"2024-09-16T15:40:12.861890+0000","last_became_peered":"2024-09-16T15:40:12.861890+0000","last_unstale":"2024-09-16T15:40:13.861165+0000","last_undegraded":"2024-09-16T15:40:13.861165+0000","last_fullsized":"2024-09-16T15:40:13.861165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:14:27.928917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.145969+0000","last_change":"2024-09-16T15:40:11.869696+0000","last_active":"2024-09-16T15:40:14.145969+0000","last_peered":"2024-09-16T15:40:14.145969+0000","last_clean":"2024-09-16T15:40:14.145969+0000","last_became_active":"2024-09-16T15:40:11.869403+0000","last_became_peered":"2024-09-16T15:40:11.869403+0000","last_unstale":"2024-09-16T15:40:14.145969+0000","last_undegraded":"2024-09-16T15:40:14.145969+0000","last_fullsized":"2024-09-16T15:40:14.145969+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:53.785910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.147072+0000","last_change":"2024-09-16T15:40:12.859022+0000","last_active":"2024-09-16T15:40:14.147072+0000","last_peered":"2024-09-16T15:40:14.147072+0000","last_clean":"2024-09-16T15:40:14.147072+0000","last_became_active":"2024-09-16T15:40:12.858683+0000","last_became_peered":"2024-09-16T15:40:12.858683+0000","last_unstale":"2024-09-16T15:40:14.147072+0000","last_undegraded":"2024-09-16T15:40:14.147072+0000","last_fullsized":"2024-09-16T15:40:14.147072+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.835623+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.835623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:36.820991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.146315+0000","last_change":"2024-09-16T15:40:11.872583+0000","last_active":"2024-09-16T15:40:14.146315+0000","last_peered":"2024-09-16T15:40:14.146315+0000","last_clean":"2024-09-16T15:40:14.146315+0000","last_became_active":"2024-09-16T15:40:11.872290+0000","last_became_peered":"2024-09-16T15:40:11.872290+0000","last_unstale":"2024-09-16T15:40:14.146315+0000","last_undegraded":"2024-09-16T15:40:14.146315+0000","last_fullsized":"2024-09-16T15:40:14.146315+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.828200+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.828200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:38.612840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":157,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.855321+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:40:13.855321+0000","last_peered":"2024-09-16T15:40:13.855321+0000","last_clean":"2024-09-16T15:40:13.855321+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:40:13.855321+0000","last_undegraded":"2024-09-16T15:40:13.855321+0000","last_fullsized":"2024-09-16T15:40:13.855321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1155072,"data_stored":816038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509467,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":565248,"data_stored":226252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":561152,"data_stored":225650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":227192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":569344,"data_stored":226218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":573440,"data_stored":227214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1155072,"data_stored":815982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":561152,"data_stored":225650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:22.281 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:40:22.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:40:22.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.485+0000 7fd136c00640 1 -- 172.21.15.60:0/2603450258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 msgr2=0x7fd130073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.485+0000 7fd136c00640 1 --2- 172.21.15.60:0/2603450258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 0x7fd130073440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd120005fd0 tx=0x7fd12002a540 comp rx=0 tx=0).stop 2024-09-16T15:40:22.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.485+0000 7fd136c00640 1 -- 172.21.15.60:0/2603450258 shutdown_connections 2024-09-16T15:40:22.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.485+0000 7fd136c00640 1 --2- 172.21.15.60:0/2603450258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13006efa0 0x7fd1300cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.485+0000 7fd136c00640 1 --2- 172.21.15.60:0/2603450258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 0x7fd130073440 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.485+0000 7fd136c00640 1 -- 172.21.15.60:0/2603450258 >> 172.21.15.60:0/2603450258 conn(0x7fd1300b1840 msgr2=0x7fd1300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:22.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.485+0000 7fd136c00640 1 -- 172.21.15.60:0/2603450258 shutdown_connections 2024-09-16T15:40:22.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.485+0000 7fd136c00640 1 -- 172.21.15.60:0/2603450258 wait complete. 2024-09-16T15:40:22.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.487+0000 7fd136c00640 1 Processor -- start 2024-09-16T15:40:22.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.487+0000 7fd136c00640 1 -- start start 2024-09-16T15:40:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.487+0000 7fd136c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13006efa0 0x7fd1301f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd136c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300ccb00 0x7fd13015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd136c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 0x7fd1300aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd136c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd1300b26b0 con 0x7fd130145750 2024-09-16T15:40:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd136c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1300cedd0 con 0x7fd1300ccb00 2024-09-16T15:40:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd136c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd130160100 con 0x7fd13006efa0 2024-09-16T15:40:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd134c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 0x7fd1300aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd134c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 0x7fd1300aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40424/0 (socket says 172.21.15.60:40424) 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd134c00640 1 -- 172.21.15.60:0/4230520905 learned_addr learned my addr 172.21.15.60:0/4230520905 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd12fe00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13006efa0 0x7fd1301f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd127e00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300ccb00 0x7fd13015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd134c00640 1 -- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13006efa0 msgr2=0x7fd1301f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd134c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13006efa0 0x7fd1301f8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd134c00640 1 -- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300ccb00 msgr2=0x7fd13015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd134c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300ccb00 0x7fd13015e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd134c00640 1 -- 172.21.15.60:0/4230520905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1300af280 con 0x7fd130145750 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.488+0000 7fd127e00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300ccb00 0x7fd13015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.489+0000 7fd134c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 0x7fd1300aeb10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd12800c7a0 tx=0x7fd12800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.489+0000 7fd12cc00640 1 -- 172.21.15.60:0/4230520905 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128003b70 con 0x7fd130145750 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.489+0000 7fd12cc00640 1 -- 172.21.15.60:0/4230520905 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd128003d10 con 0x7fd130145750 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.489+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1300acf80 con 0x7fd130145750 2024-09-16T15:40:22.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.489+0000 7fd12cc00640 1 -- 172.21.15.60:0/4230520905 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12800aab0 con 0x7fd130145750 2024-09-16T15:40:22.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.489+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1300ad500 con 0x7fd130145750 2024-09-16T15:40:22.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.490+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1301605b0 con 0x7fd130145750 2024-09-16T15:40:22.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.491+0000 7fd12cc00640 1 -- 172.21.15.60:0/4230520905 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fd12800ac50 con 0x7fd130145750 2024-09-16T15:40:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.492+0000 7fd12cc00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd10c078a00 0x7fd10c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.492+0000 7fd12cc00640 1 -- 172.21.15.60:0/4230520905 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fd12809ab80 con 0x7fd130145750 2024-09-16T15:40:22.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.492+0000 7fd12fe00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd10c078a00 0x7fd10c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.493+0000 7fd12fe00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd10c078a00 0x7fd10c07aec0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd120033000 tx=0x7fd120004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.496+0000 7fd12cc00640 1 -- 172.21.15.60:0/4230520905 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd128062e50 con 0x7fd130145750 2024-09-16T15:40:22.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.765+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd1300b1e30 con 0x7fd130145750 2024-09-16T15:40:22.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.767+0000 7fd12cc00640 1 -- 172.21.15.60:0/4230520905 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7fd128062750 con 0x7fd130145750 2024-09-16T15:40:22.768 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:40:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:40:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:40:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:40:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:40:22.769 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:40:22.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd10c078a00 msgr2=0x7fd10c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd10c078a00 0x7fd10c07aec0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd120033000 tx=0x7fd120004730 comp rx=0 tx=0).stop 2024-09-16T15:40:22.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 msgr2=0x7fd1300aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 0x7fd1300aeb10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd12800c7a0 tx=0x7fd12800cc60 comp rx=0 tx=0).stop 2024-09-16T15:40:22.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 shutdown_connections 2024-09-16T15:40:22.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd10c078a00 0x7fd10c07aec0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd130145750 0x7fd1300aeb10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1300ccb00 0x7fd13015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 --2- 172.21.15.60:0/4230520905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd13006efa0 0x7fd1301f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 >> 172.21.15.60:0/4230520905 conn(0x7fd1300b1840 msgr2=0x7fd1300cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:22.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 shutdown_connections 2024-09-16T15:40:22.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.770+0000 7fd136c00640 1 -- 172.21.15.60:0/4230520905 wait complete. 2024-09-16T15:40:22.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:22.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.946+0000 7f2be3a00640 1 -- 172.21.15.60:0/2240986377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc15f5a0 msgr2=0x7f2bdc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.946+0000 7f2be3a00640 1 --2- 172.21.15.60:0/2240986377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc15f5a0 0x7f2bdc0b2440 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc06dac0 tx=0x7f2bcc02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.946+0000 7f2be3a00640 1 -- 172.21.15.60:0/2240986377 shutdown_connections 2024-09-16T15:40:22.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.946+0000 7f2be3a00640 1 --2- 172.21.15.60:0/2240986377 >> v2:172.21.15.100:3300/0 conn(0x7f2bdc073fb0 0x7f2bdc074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.946+0000 7f2be3a00640 1 --2- 172.21.15.60:0/2240986377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc15f5a0 0x7f2bdc0b2440 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.946+0000 7f2be3a00640 1 -- 172.21.15.60:0/2240986377 >> 172.21.15.60:0/2240986377 conn(0x7f2bdc0b1950 msgr2=0x7f2bdc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:22.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.946+0000 7f2be3a00640 1 -- 172.21.15.60:0/2240986377 shutdown_connections 2024-09-16T15:40:22.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.946+0000 7f2be3a00640 1 -- 172.21.15.60:0/2240986377 wait complete. 2024-09-16T15:40:22.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.948+0000 7f2be3a00640 1 Processor -- start 2024-09-16T15:40:22.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.948+0000 7f2be3a00640 1 -- start start 2024-09-16T15:40:22.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be3a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc073fb0 0x7f2bdc14e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be3a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bdc0ccf00 0x7f2bdc156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be3a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bdc15f5a0 0x7f2bdc156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be3a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc06f2e0 con 0x7f2bdc073fb0 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be3a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc06f160 con 0x7f2bdc0ccf00 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be3a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2bdc06f460 con 0x7f2bdc15f5a0 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc073fb0 0x7f2bdc14e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc073fb0 0x7f2bdc14e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40448/0 (socket says 172.21.15.60:40448) 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be1000640 1 -- 172.21.15.60:0/4115769580 learned_addr learned my addr 172.21.15.60:0/4115769580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2be1a00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bdc15f5a0 0x7f2bdc156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.949+0000 7f2bd3e00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bdc0ccf00 0x7f2bdc156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2be1000640 1 -- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bdc15f5a0 msgr2=0x7f2bdc156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2be1000640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bdc15f5a0 0x7f2bdc156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2be1000640 1 -- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bdc0ccf00 msgr2=0x7f2bdc156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2be1000640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bdc0ccf00 0x7f2bdc156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2be1000640 1 -- 172.21.15.60:0/4115769580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc0c9c20 con 0x7f2bdc073fb0 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2be1000640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc073fb0 0x7f2bdc14e870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc034040 tx=0x7f2bcc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:22.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2bd9600640 1 -- 172.21.15.60:0/4115769580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc03e030 con 0x7f2bdc073fb0 2024-09-16T15:40:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2bd9600640 1 -- 172.21.15.60:0/4115769580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bcc0047a0 con 0x7f2bdc073fb0 2024-09-16T15:40:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2bd9600640 1 -- 172.21.15.60:0/4115769580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc035040 con 0x7f2bdc073fb0 2024-09-16T15:40:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc0c9f00 con 0x7f2bdc073fb0 2024-09-16T15:40:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.950+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc0c8fd0 con 0x7f2bdc073fb0 2024-09-16T15:40:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.951+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc0ca140 con 0x7f2bdc073fb0 2024-09-16T15:40:22.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.952+0000 7f2bd9600640 1 -- 172.21.15.60:0/4115769580 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f2bcc041030 con 0x7f2bdc073fb0 2024-09-16T15:40:22.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.953+0000 7f2bd9600640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2bb8078a00 0x7f2bb807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.953+0000 7f2bd9600640 1 -- 172.21.15.60:0/4115769580 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f2bcc0bb3f0 con 0x7f2bdc073fb0 2024-09-16T15:40:22.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.954+0000 7f2bd3e00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2bb8078a00 0x7f2bb807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.954+0000 7f2bd3e00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2bb8078a00 0x7f2bb807aec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4007fd0 tx=0x7f2bc4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:22.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:22.957+0000 7f2bd9600640 1 -- 172.21.15.60:0/4115769580 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2bcc083760 con 0x7f2bdc073fb0 2024-09-16T15:40:23.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.188+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bdc06f9f0 con 0x7f2bdc073fb0 2024-09-16T15:40:23.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.188+0000 7f2bd9600640 1 -- 172.21.15.60:0/4115769580 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7f2bcc083060 con 0x7f2bdc073fb0 2024-09-16T15:40:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:23.190 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"btime":"2024-09-16T15:40:15:858549+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:23.190 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-09-16T15:40:23.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2bb8078a00 msgr2=0x7f2bb807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2bb8078a00 0x7f2bb807aec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4007fd0 tx=0x7f2bc4009040 comp rx=0 tx=0).stop 2024-09-16T15:40:23.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc073fb0 msgr2=0x7f2bdc14e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc073fb0 0x7f2bdc14e870 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc034040 tx=0x7f2bcc0039e0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 shutdown_connections 2024-09-16T15:40:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2bb8078a00 0x7f2bb807aec0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bdc15f5a0 0x7f2bdc156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bdc0ccf00 0x7f2bdc156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 --2- 172.21.15.60:0/4115769580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bdc073fb0 0x7f2bdc14e870 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 >> 172.21.15.60:0/4115769580 conn(0x7f2bdc0b1950 msgr2=0x7f2bdc0ad680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 shutdown_connections 2024-09-16T15:40:23.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.191+0000 7f2be3a00640 1 -- 172.21.15.60:0/4115769580 wait complete. 2024-09-16T15:40:23.611 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:40:12.294606+0000', 'modified': '2024-09-16T15:40:15.298172+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5310}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5310': {'gid': 5310, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/2823237251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 2823237251}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2823237251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5310, 'qdb_cluster': [5310]} 2024-09-16T15:40:23.611 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:40:23.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:40:23.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.773+0000 7fee61000640 1 -- 172.21.15.60:0/3663537627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c073fb0 msgr2=0x7fee5c074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.773+0000 7fee61000640 1 --2- 172.21.15.60:0/3663537627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c073fb0 0x7fee5c074460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fee54005fd0 tx=0x7fee5402a980 comp rx=0 tx=0).stop 2024-09-16T15:40:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.773+0000 7fee61000640 1 -- 172.21.15.60:0/3663537627 shutdown_connections 2024-09-16T15:40:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.773+0000 7fee61000640 1 --2- 172.21.15.60:0/3663537627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c073fb0 0x7fee5c074460 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.773+0000 7fee61000640 1 --2- 172.21.15.60:0/3663537627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee5c15f5a0 0x7fee5c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.773+0000 7fee61000640 1 -- 172.21.15.60:0/3663537627 >> 172.21.15.60:0/3663537627 conn(0x7fee5c0b1950 msgr2=0x7fee5c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.773+0000 7fee61000640 1 -- 172.21.15.60:0/3663537627 shutdown_connections 2024-09-16T15:40:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.773+0000 7fee61000640 1 -- 172.21.15.60:0/3663537627 wait complete. 2024-09-16T15:40:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.775+0000 7fee61000640 1 Processor -- start 2024-09-16T15:40:23.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.775+0000 7fee61000640 1 -- start start 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee61000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c073fb0 0x7fee5c147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee61000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee5c0ccf00 0x7fee5c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee61000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c15f5a0 0x7fee5c09cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee61000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c06f2e0 con 0x7fee5c15f5a0 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee61000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c06f160 con 0x7fee5c073fb0 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee61000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c06f460 con 0x7fee5c0ccf00 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee5a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c15f5a0 0x7fee5c09cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee5a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c15f5a0 0x7fee5c09cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40474/0 (socket says 172.21.15.60:40474) 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee5a800640 1 -- 172.21.15.60:0/2148645818 learned_addr learned my addr 172.21.15.60:0/2148645818 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee51400640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee5c0ccf00 0x7fee5c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee59e00640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c073fb0 0x7fee5c147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee5a800640 1 -- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee5c0ccf00 msgr2=0x7fee5c14ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee5a800640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee5c0ccf00 0x7fee5c14ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.776+0000 7fee5a800640 1 -- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c073fb0 msgr2=0x7fee5c147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.777+0000 7fee5a800640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c073fb0 0x7fee5c147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.777+0000 7fee5a800640 1 -- 172.21.15.60:0/2148645818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c14b270 con 0x7fee5c15f5a0 2024-09-16T15:40:23.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.777+0000 7fee5a800640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c15f5a0 0x7fee5c09cc50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fee54009d90 tx=0x7fee54003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.777+0000 7fee52a00640 1 -- 172.21.15.60:0/2148645818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54034050 con 0x7fee5c15f5a0 2024-09-16T15:40:23.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.777+0000 7fee52a00640 1 -- 172.21.15.60:0/2148645818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee540044d0 con 0x7fee5c15f5a0 2024-09-16T15:40:23.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.777+0000 7fee61000640 1 -- 172.21.15.60:0/2148645818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c1f8260 con 0x7fee5c15f5a0 2024-09-16T15:40:23.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.777+0000 7fee52a00640 1 -- 172.21.15.60:0/2148645818 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54043440 con 0x7fee5c15f5a0 2024-09-16T15:40:23.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.777+0000 7fee61000640 1 -- 172.21.15.60:0/2148645818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c1f8820 con 0x7fee5c15f5a0 2024-09-16T15:40:23.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.780+0000 7fee52a00640 1 -- 172.21.15.60:0/2148645818 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fee54037020 con 0x7fee5c15f5a0 2024-09-16T15:40:23.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.780+0000 7fee52a00640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee38078a00 0x7fee3807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.781+0000 7fee52a00640 1 -- 172.21.15.60:0/2148645818 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fee540bc080 con 0x7fee5c15f5a0 2024-09-16T15:40:23.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.781+0000 7fee63e70740 1 Processor -- start 2024-09-16T15:40:23.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.781+0000 7fee63e70740 1 -- start start 2024-09-16T15:40:23.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.781+0000 7fee59e00640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee38078a00 0x7fee3807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee59e00640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee38078a00 0x7fee3807aec0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0098e0 tx=0x7fee4c009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee63e70740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 0x55c7f9e6ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee63e70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29a20 0x55c7f9e6d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee63e70740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee63e70740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c7f9dac430 con 0x55c7f9e29fa0 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee63e70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c7f9d9c5d0 con 0x55c7f9e29a20 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee63e70740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c7f9d8ba00 con 0x55c7f9e6e8b0 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee5a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee5a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40490/0 (socket says 172.21.15.60:40490) 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee59e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 0x55c7f9e6ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee5a800640 1 -- 172.21.15.60:0/3193923045 learned_addr learned my addr 172.21.15.60:0/3193923045 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.782+0000 7fee51400640 1 --2- 172.21.15.60:0/3193923045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29a20 0x55c7f9e6d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee5a800640 1 -- 172.21.15.60:0/3193923045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 msgr2=0x55c7f9e6ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee5a800640 1 --2- 172.21.15.60:0/3193923045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 0x55c7f9e6ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee5a800640 1 -- 172.21.15.60:0/3193923045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29a20 msgr2=0x55c7f9e6d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee5a800640 1 --2- 172.21.15.60:0/3193923045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29a20 0x55c7f9e6d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee5a800640 1 -- 172.21.15.60:0/3193923045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f9d54520 con 0x55c7f9e29fa0 2024-09-16T15:40:23.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee5a800640 1 --2- 172.21.15.60:0/3193923045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6d8d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fee540ba2e0 tx=0x7fee5407fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f2e0e14b497369 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee4aa00640 1 -- 172.21.15.60:0/3193923045 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5406f020 con 0x55c7f9e29fa0 2024-09-16T15:40:23.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee4aa00640 1 -- 172.21.15.60:0/3193923045 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee54083810 con 0x55c7f9e29fa0 2024-09-16T15:40:23.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.783+0000 7fee4aa00640 1 -- 172.21.15.60:0/3193923045 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540ba670 con 0x55c7f9e29fa0 2024-09-16T15:40:23.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 -- 172.21.15.60:0/3193923045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 msgr2=0x55c7f9e6d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 --2- 172.21.15.60:0/3193923045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6d8d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fee540ba2e0 tx=0x7fee5407fc60 comp rx=0 tx=0).stop 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 -- 172.21.15.60:0/3193923045 shutdown_connections 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 --2- 172.21.15.60:0/3193923045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6d8d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 --2- 172.21.15.60:0/3193923045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29a20 0x55c7f9e6d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 --2- 172.21.15.60:0/3193923045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 0x55c7f9e6ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 -- 172.21.15.60:0/3193923045 >> 172.21.15.60:0/3193923045 conn(0x55c7f9e24ff0 msgr2=0x55c7f9e28a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 -- 172.21.15.60:0/3193923045 shutdown_connections 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.784+0000 7fee63e70740 1 -- 172.21.15.60:0/3193923045 wait complete. 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.785+0000 7fee63e70740 1 -- start start 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.785+0000 7fee63e70740 1 Processor -- start 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee63e70740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee63e70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e4cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee63e70740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 0x55c7f9e690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee63e70740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c7f9d32330 con 0x55c7f9e29fa0 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee63e70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c7f9d8f810 con 0x55c7f9e6e8b0 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee63e70740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c7f9afc620 con 0x55c7f9e67230 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee63e70740 10 client.0 _ll_register_callbacks cb 0x55c7f9d37420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee59e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee59e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40498/0 (socket says 172.21.15.60:40498) 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee59e00640 1 -- 172.21.15.60:0/1162466374 learned_addr learned my addr 172.21.15.60:0/1162466374 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee5a800640 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 0x55c7f9e690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.786+0000 7fee51400640 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e4cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee59e00640 1 -- 172.21.15.60:0/1162466374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 msgr2=0x55c7f9e690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee59e00640 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 0x55c7f9e690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee59e00640 1 -- 172.21.15.60:0/1162466374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e6e8b0 msgr2=0x55c7f9e4cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee59e00640 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e4cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee59e00640 1 -- 172.21.15.60:0/1162466374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f9aed850 con 0x55c7f9e29fa0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee59e00640 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6af50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fee4c015730 tx=0x7fee4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642bb5615274c008 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee3f400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee3f400640 1 -- 172.21.15.60:0/1162466374 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c025070 con 0x55c7f9e29fa0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee63e70740 1 -- 172.21.15.60:0/1162466374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee54083810 con 0x55c7f9e29fa0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.787+0000 7fee3f400640 1 -- 172.21.15.60:0/1162466374 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fee4c0137d0 con 0x55c7f9e29fa0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee3f400640 1 -- 172.21.15.60:0/1162466374 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee4c030050 con 0x55c7f9e29fa0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee63e70740 10 client.5442 fetch_fsmap learned FSMap version 26 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee63e70740 1 -- 172.21.15.60:0/1162466374 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55c7f9b646b0 con 0x55c7f9e29fa0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee3f400640 1 -- 172.21.15.60:0/1162466374 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee54083810 con 0x55c7f9e29fa0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee3f400640 1 -- 172.21.15.60:0/1162466374 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fee4c033020 con 0x55c7f9e29fa0 2024-09-16T15:40:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee63e70740 10 client.5442 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T15:40:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee63e70740 10 client.5442 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:40:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee63e70740 20 client.5442 populate_metadata read hostname 'smithi060' 2024-09-16T15:40:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee63e70740 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] conn(0x55c7f9e52220 0x55c7f9e64c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee63e70740 4 client.5442 mds_command: new command op to 5310 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:40:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee63e70740 1 -- 172.21.15.60:0/1162466374 --> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7f9d9e520 con 0x55c7f9e52220 2024-09-16T15:40:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.788+0000 7fee51400640 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] conn(0x55c7f9e52220 0x55c7f9e64c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.789+0000 7fee51400640 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] conn(0x55c7f9e52220 0x55c7f9e64c80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8fdeec6ea99a578 server_cookie=6321f3160736344a in_seq=0 out_seq=0 2024-09-16T15:40:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.789+0000 7fee3f400640 10 client.5442 ms_handle_connect on v2:172.21.15.60:6834/2823237251 2024-09-16T15:40:23.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee3f400640 1 -- 172.21.15.60:0/1162466374 <== mds.0 v2:172.21.15.60:6834/2823237251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c7f9d9e520 con 0x55c7f9e52220 2024-09-16T15:40:23.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee3f400640 10 client.5442 handle_command_reply: tid=0 2024-09-16T15:40:23.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 client.5442 shutdown 2024-09-16T15:40:23.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 -- 172.21.15.60:0/1162466374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 msgr2=0x55c7f9e6af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6af50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fee4c015730 tx=0x7fee4c004730 comp rx=0 tx=0).stop 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 -- 172.21.15.60:0/1162466374 shutdown_connections 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] conn(0x55c7f9e52220 0x55c7f9e64c80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 0x55c7f9e690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e4cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 --2- 172.21.15.60:0/1162466374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e6af50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee3f400640 0 client.5442 ms_handle_reset on v2:172.21.15.60:6834/2823237251 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 -- 172.21.15.60:0/1162466374 >> 172.21.15.60:0/1162466374 conn(0x55c7f9e29a20 msgr2=0x55c7f9e4f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 -- 172.21.15.60:0/1162466374 shutdown_connections 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 1 -- 172.21.15.60:0/1162466374 wait complete. 2024-09-16T15:40:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.790+0000 7fee63e70740 20 client.5442 trim_cache size 0 max 16384 2024-09-16T15:40:23.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.809+0000 7fee63e70740 1 Processor -- start 2024-09-16T15:40:23.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.810+0000 7fee63e70740 1 -- start start 2024-09-16T15:40:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.810+0000 7fee63e70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e70ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.810+0000 7fee63e70740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 0x55c7f9e71020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.810+0000 7fee63e70740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e69b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.810+0000 7fee63e70740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c2004c0 con 0x55c7f9e6e8b0 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.810+0000 7fee63e70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c148ca0 con 0x55c7f9e29fa0 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.810+0000 7fee63e70740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c1492f0 con 0x55c7f9e67230 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e69b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40500/0 (socket says 172.21.15.60:40500) 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 -- 172.21.15.60:0/133615980 learned_addr learned my addr 172.21.15.60:0/133615980 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 -- 172.21.15.60:0/133615980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 msgr2=0x55c7f9e71020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 --2- 172.21.15.60:0/133615980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 0x55c7f9e71020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 -- 172.21.15.60:0/133615980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 msgr2=0x55c7f9e70ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 --2- 172.21.15.60:0/133615980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e70ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 -- 172.21.15.60:0/133615980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f9e51b40 con 0x55c7f9e6e8b0 2024-09-16T15:40:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee5a800640 1 --2- 172.21.15.60:0/133615980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e69b80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fee54009a60 tx=0x7fee5402a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128582caac85c2af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee4a000640 1 -- 172.21.15.60:0/133615980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5409d070 con 0x55c7f9e6e8b0 2024-09-16T15:40:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee4a000640 1 -- 172.21.15.60:0/133615980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee54083810 con 0x55c7f9e6e8b0 2024-09-16T15:40:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.811+0000 7fee4a000640 1 -- 172.21.15.60:0/133615980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5406f040 con 0x55c7f9e6e8b0 2024-09-16T15:40:23.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 -- 172.21.15.60:0/133615980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e6e8b0 msgr2=0x55c7f9e69b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 --2- 172.21.15.60:0/133615980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e69b80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fee54009a60 tx=0x7fee5402a980 comp rx=0 tx=0).stop 2024-09-16T15:40:23.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 -- 172.21.15.60:0/133615980 shutdown_connections 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 --2- 172.21.15.60:0/133615980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e6e8b0 0x55c7f9e69b80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 --2- 172.21.15.60:0/133615980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e67230 0x55c7f9e71020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 --2- 172.21.15.60:0/133615980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 0x55c7f9e70ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 -- 172.21.15.60:0/133615980 >> 172.21.15.60:0/133615980 conn(0x55c7f9e29a20 msgr2=0x55c7f9e64c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 -- 172.21.15.60:0/133615980 shutdown_connections 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.812+0000 7fee63e70740 1 -- 172.21.15.60:0/133615980 wait complete. 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.813+0000 7fee63e70740 1 -- start start 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.813+0000 7fee63e70740 1 Processor -- start 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee63e70740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 0x55c7f9eedb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee63e70740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e67230 0x55c7f9e8fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee63e70740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 0x55c7f9e8ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee63e70740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c7f9e4d280 con 0x55c7f9e67230 2024-09-16T15:40:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee63e70740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c7f9e68c50 con 0x55c7f9e29fa0 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee63e70740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c7f9e52640 con 0x55c7f9e6e8b0 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee63e70740 10 client.0 _ll_register_callbacks cb 0x7fee4c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee51400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e67230 0x55c7f9e8fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee51400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e67230 0x55c7f9e8fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40514/0 (socket says 172.21.15.60:40514) 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee51400640 1 -- 172.21.15.60:0/3903818655 learned_addr learned my addr 172.21.15.60:0/3903818655 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee59e00640 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 0x55c7f9eedb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.814+0000 7fee5a800640 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 0x55c7f9e8ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee51400640 1 -- 172.21.15.60:0/3903818655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 msgr2=0x55c7f9e8ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee51400640 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 0x55c7f9e8ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee51400640 1 -- 172.21.15.60:0/3903818655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 msgr2=0x55c7f9eedb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee51400640 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 0x55c7f9eedb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee51400640 1 -- 172.21.15.60:0/3903818655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f9e27d70 con 0x55c7f9e67230 2024-09-16T15:40:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee51400640 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e67230 0x55c7f9e8fa50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fee44001930 tx=0x7fee440167f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8484b195353ce78b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee3f400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee3f400640 1 -- 172.21.15.60:0/3903818655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440050d0 con 0x55c7f9e67230 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee3f400640 1 -- 172.21.15.60:0/3903818655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fee44005270 con 0x55c7f9e67230 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee63e70740 1 -- 172.21.15.60:0/3903818655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee54083810 con 0x55c7f9e67230 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee3f400640 1 -- 172.21.15.60:0/3903818655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee44005470 con 0x55c7f9e67230 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee63e70740 10 client.5448 fetch_fsmap learned FSMap version 26 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.815+0000 7fee63e70740 1 -- 172.21.15.60:0/3903818655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55c7f9e4a5d0 con 0x55c7f9e67230 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee3f400640 1 -- 172.21.15.60:0/3903818655 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee54083810 con 0x55c7f9e67230 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee3f400640 1 -- 172.21.15.60:0/3903818655 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fee44003800 con 0x55c7f9e67230 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee63e70740 10 client.5448 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee63e70740 10 client.5448 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee63e70740 20 client.5448 populate_metadata read hostname 'smithi060' 2024-09-16T15:40:23.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee63e70740 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] conn(0x55c7f9e4c530 0x55c7f9ef19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee63e70740 4 client.5448 mds_command: new command op to 5310 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee63e70740 1 -- 172.21.15.60:0/3903818655 --> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] -- command(tid 0: {"prefix": "status"}) -- 0x55c7f9d99fd0 con 0x55c7f9e4c530 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.816+0000 7fee59e00640 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] conn(0x55c7f9e4c530 0x55c7f9ef19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee59e00640 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] conn(0x55c7f9e4c530 0x55c7f9ef19b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc09415b975865d server_cookie=5886bc7c0967513f in_seq=0 out_seq=0 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee3f400640 10 client.5448 ms_handle_connect on v2:172.21.15.60:6834/2823237251 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee3f400640 1 -- 172.21.15.60:0/3903818655 <== mds.0 v2:172.21.15.60:6834/2823237251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55c7f9d99fd0 con 0x55c7f9e4c530 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee3f400640 10 client.5448 handle_command_reply: tid=0 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 client.5448 shutdown 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 -- 172.21.15.60:0/3903818655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e67230 msgr2=0x55c7f9e8fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e67230 0x55c7f9e8fa50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fee44001930 tx=0x7fee440167f0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 -- 172.21.15.60:0/3903818655 shutdown_connections 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.60:6834/2823237251,v1:172.21.15.60:6835/2823237251] conn(0x55c7f9e4c530 0x55c7f9ef19b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c7f9e6e8b0 0x55c7f9e8ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c7f9e67230 0x55c7f9e8fa50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 --2- 172.21.15.60:0/3903818655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7f9e29fa0 0x55c7f9eedb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee3f400640 0 client.5448 ms_handle_reset on v2:172.21.15.60:6834/2823237251 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 -- 172.21.15.60:0/3903818655 >> 172.21.15.60:0/3903818655 conn(0x55c7f9e29a20 msgr2=0x55c7f9e70ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 -- 172.21.15.60:0/3903818655 shutdown_connections 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 1 -- 172.21.15.60:0/3903818655 wait complete. 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.817+0000 7fee63e70740 20 client.5448 trim_cache size 0 max 16384 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:40:23.823 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5310, 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 10.967651769, 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 26, 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 48, 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 48, 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 16.254044110999999 2024-09-16T15:40:23.824 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 -- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee38078a00 msgr2=0x7fee3807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee38078a00 0x7fee3807aec0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0098e0 tx=0x7fee4c009060 comp rx=0 tx=0).stop 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 -- 172.21.15.60:0/2148645818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c15f5a0 msgr2=0x7fee5c09cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c15f5a0 0x7fee5c09cc50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fee54009d90 tx=0x7fee54003ef0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 -- 172.21.15.60:0/2148645818 shutdown_connections 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fee38078a00 0x7fee3807aec0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee5c15f5a0 0x7fee5c09cc50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee5c0ccf00 0x7fee5c14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 --2- 172.21.15.60:0/2148645818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee5c073fb0 0x7fee5c147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 -- 172.21.15.60:0/2148645818 >> 172.21.15.60:0/2148645818 conn(0x7fee5c0b1950 msgr2=0x7fee5c15e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 -- 172.21.15.60:0/2148645818 shutdown_connections 2024-09-16T15:40:23.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.818+0000 7fee61000640 1 -- 172.21.15.60:0/2148645818 wait complete. 2024-09-16T15:40:23.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:40:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.994+0000 7fcf6f600640 1 -- 172.21.15.60:0/1481030582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 msgr2=0x7fcf6806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.994+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1481030582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 0x7fcf6806ef50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009830 tx=0x7fcf5802a480 comp rx=0 tx=0).stop 2024-09-16T15:40:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.994+0000 7fcf6f600640 1 -- 172.21.15.60:0/1481030582 shutdown_connections 2024-09-16T15:40:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.994+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1481030582 >> v2:172.21.15.100:3300/0 conn(0x7fcf68072df0 0x7fcf680732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.994+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1481030582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 0x7fcf6806ef50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.994+0000 7fcf6f600640 1 -- 172.21.15.60:0/1481030582 >> 172.21.15.60:0/1481030582 conn(0x7fcf680b1950 msgr2=0x7fcf680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.994+0000 7fcf6f600640 1 -- 172.21.15.60:0/1481030582 shutdown_connections 2024-09-16T15:40:23.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.994+0000 7fcf6f600640 1 -- 172.21.15.60:0/1481030582 wait complete. 2024-09-16T15:40:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.996+0000 7fcf6f600640 1 Processor -- start 2024-09-16T15:40:23.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.996+0000 7fcf6f600640 1 -- start start 2024-09-16T15:40:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6f600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf68072df0 0x7fcf6815e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680af3d0 0x7fcf680c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6f600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 0x7fcf680c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6f600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680737e0 con 0x7fcf68072df0 2024-09-16T15:40:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6806fdb0 con 0x7fcf680af3d0 2024-09-16T15:40:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6f600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcf68073960 con 0x7fcf68161550 2024-09-16T15:40:23.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf68072df0 0x7fcf6815e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf68072df0 0x7fcf6815e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40526/0 (socket says 172.21.15.60:40526) 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf6cc00640 1 -- 172.21.15.60:0/1773803484 learned_addr learned my addr 172.21.15.60:0/1773803484 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.997+0000 7fcf5fe00640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680af3d0 0x7fcf680c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6d600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 0x7fcf680c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6cc00640 1 -- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 msgr2=0x7fcf680c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6cc00640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 0x7fcf680c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6cc00640 1 -- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680af3d0 msgr2=0x7fcf680c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6cc00640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680af3d0 0x7fcf680c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6cc00640 1 -- 172.21.15.60:0/1773803484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf681485a0 con 0x7fcf68072df0 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6d600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 0x7fcf680c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:24.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6cc00640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf68072df0 0x7fcf6815e220 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcf58000c00 tx=0x7fcf58033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:24.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf65600640 1 -- 172.21.15.60:0/1773803484 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58031d60 con 0x7fcf68072df0 2024-09-16T15:40:24.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf65600640 1 -- 172.21.15.60:0/1773803484 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf58033e10 con 0x7fcf68072df0 2024-09-16T15:40:24.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf65600640 1 -- 172.21.15.60:0/1773803484 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf580332b0 con 0x7fcf68072df0 2024-09-16T15:40:24.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.998+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf68148880 con 0x7fcf68072df0 2024-09-16T15:40:24.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:23.999+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf68148e40 con 0x7fcf68072df0 2024-09-16T15:40:24.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.000+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf68073ec0 con 0x7fcf68072df0 2024-09-16T15:40:24.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.001+0000 7fcf65600640 1 -- 172.21.15.60:0/1773803484 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fcf58033450 con 0x7fcf68072df0 2024-09-16T15:40:24.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.002+0000 7fcf65600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcf44078a00 0x7fcf4407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:24.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.002+0000 7fcf65600640 1 -- 172.21.15.60:0/1773803484 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fcf580bbfe0 con 0x7fcf68072df0 2024-09-16T15:40:24.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.002+0000 7fcf5fe00640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcf44078a00 0x7fcf4407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:24.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.003+0000 7fcf5fe00640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcf44078a00 0x7fcf4407aec0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf68149470 tx=0x7fcf50007e90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:24.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.006+0000 7fcf65600640 1 -- 172.21.15.60:0/1773803484 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcf5803f030 con 0x7fcf68072df0 2024-09-16T15:40:24.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.243+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcf680b09f0 con 0x7fcf68072df0 2024-09-16T15:40:24.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.244+0000 7fcf65600640 1 -- 172.21.15.60:0/1773803484 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fcf58084260 con 0x7fcf68072df0 2024-09-16T15:40:24.245 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcf44078a00 msgr2=0x7fcf4407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcf44078a00 0x7fcf4407aec0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf68149470 tx=0x7fcf50007e90 comp rx=0 tx=0).stop 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf68072df0 msgr2=0x7fcf6815e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf68072df0 0x7fcf6815e220 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcf58000c00 tx=0x7fcf58033c20 comp rx=0 tx=0).stop 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 shutdown_connections 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcf44078a00 0x7fcf4407aec0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf68161550 0x7fcf680c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf680af3d0 0x7fcf680c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 --2- 172.21.15.60:0/1773803484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf68072df0 0x7fcf6815e220 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 >> 172.21.15.60:0/1773803484 conn(0x7fcf680b1950 msgr2=0x7fcf6815f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 shutdown_connections 2024-09-16T15:40:24.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24.246+0000 7fcf6f600640 1 -- 172.21.15.60:0/1773803484 wait complete. 2024-09-16T15:40:24.263 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:40:24.263 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:40:24.264 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:40:24.264 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:40:24.264 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:24.264 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:40:24.264 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:40:24.264 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:40:24.264 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:40:24.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.264 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:40:24.300 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:40:24.300 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:40:24.300 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:40:24.300 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:24.300 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42625sec preferred_lft 42625sec 2024-09-16T15:40:24.301 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:24.302 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:40:24.303 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:40:24.303 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:40:24.303 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:24.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.303 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:40:24.395 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:40:24.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.398 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:40:24.434 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:40:24.434 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:40:24.434 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:40:24.434 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:40:24.434 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:40:24.434 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:40:24.434 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:40:24.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:40:24.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:40:24.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:40:24.435 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:40:24.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:40:24.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:40:24.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.632 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:40:24.667 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:40:24.667 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:40:24.667 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:40:24.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:40:24.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:40:24.763 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:24.798 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:40:24.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:40:24.798 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:24.872 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:40:24.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:40:25.002 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:40:25.002 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:40:25.031 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:40:25.031 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:40:25.031 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:40:25.031 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:40:25.031 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:40:25.032 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:40:25.032 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:40:25.033 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:40:25.033 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:40:25.033 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:40:25.036 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:40:25.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:25.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:25.092 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:40:25.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:40:25.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:40:25.188 INFO:teuthology.orchestra.run.smithi060.stdout:client5431 2024-09-16T15:40:25.188 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:40:25.189 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:25.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:40:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.423+0000 7f3917e00640 1 -- 172.21.15.60:0/3300165674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39180725d0 msgr2=0x7f39180b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.423+0000 7f3917e00640 1 --2- 172.21.15.60:0/3300165674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39180725d0 0x7f39180b28f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3904009830 tx=0x7f390402a510 comp rx=0 tx=0).stop 2024-09-16T15:40:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.424+0000 7f3917e00640 1 -- 172.21.15.60:0/3300165674 shutdown_connections 2024-09-16T15:40:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.424+0000 7f3917e00640 1 --2- 172.21.15.60:0/3300165674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39180d0540 0x7f391806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.424+0000 7f3917e00640 1 --2- 172.21.15.60:0/3300165674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39180725d0 0x7f39180b28f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.424+0000 7f3917e00640 1 -- 172.21.15.60:0/3300165674 >> 172.21.15.60:0/3300165674 conn(0x7f39180cfa20 msgr2=0x7f39180cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.424+0000 7f3917e00640 1 -- 172.21.15.60:0/3300165674 shutdown_connections 2024-09-16T15:40:25.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.424+0000 7f3917e00640 1 -- 172.21.15.60:0/3300165674 wait complete. 2024-09-16T15:40:25.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.425+0000 7f3917e00640 1 Processor -- start 2024-09-16T15:40:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.425+0000 7f3917e00640 1 -- start start 2024-09-16T15:40:25.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.426+0000 7f3917e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39180d0540 0x7f39181f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:25.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.426+0000 7f3917e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f391815bc30 0x7f39180cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:25.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.426+0000 7f3917e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39181f4bc0 0x7f39180cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.426+0000 7f3917e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3918074660 con 0x7f391815bc30 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.426+0000 7f3917e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f391806f860 con 0x7f39181f4bc0 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.426+0000 7f3917e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f39181f5620 con 0x7f39180d0540 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f391815bc30 0x7f39180cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f391815bc30 0x7f39180cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40548/0 (socket says 172.21.15.60:40548) 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 -- 172.21.15.60:0/2432976627 learned_addr learned my addr 172.21.15.60:0/2432976627 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916a00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39180d0540 0x7f39181f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3917400640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39181f4bc0 0x7f39180cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 -- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39180d0540 msgr2=0x7f39181f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39180d0540 0x7f39181f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 -- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39181f4bc0 msgr2=0x7f39180cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39181f4bc0 0x7f39180cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 -- 172.21.15.60:0/2432976627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39181f5060 con 0x7f391815bc30 2024-09-16T15:40:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.427+0000 7f3916000640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f391815bc30 0x7f39180cc940 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3904000c00 tx=0x7f3904032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.428+0000 7f390ea00640 1 -- 172.21.15.60:0/2432976627 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904040070 con 0x7f391815bc30 2024-09-16T15:40:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.428+0000 7f390ea00640 1 -- 172.21.15.60:0/2432976627 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3904032e70 con 0x7f391815bc30 2024-09-16T15:40:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.428+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39181f5340 con 0x7f391815bc30 2024-09-16T15:40:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.428+0000 7f390ea00640 1 -- 172.21.15.60:0/2432976627 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904003d90 con 0x7f391815bc30 2024-09-16T15:40:25.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.428+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39180c6f00 con 0x7f391815bc30 2024-09-16T15:40:25.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.429+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391806fdf0 con 0x7f391815bc30 2024-09-16T15:40:25.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.430+0000 7f390ea00640 1 -- 172.21.15.60:0/2432976627 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f3904039040 con 0x7f391815bc30 2024-09-16T15:40:25.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.431+0000 7f390ea00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f38f8078a00 0x7f38f807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:25.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.431+0000 7f390ea00640 1 -- 172.21.15.60:0/2432976627 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f39040bc2b0 con 0x7f391815bc30 2024-09-16T15:40:25.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.431+0000 7f3916a00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f38f8078a00 0x7f38f807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:25.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.432+0000 7f3916a00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f38f8078a00 0x7f38f807aec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3908001720 tx=0x7f3908007e90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.435+0000 7f390ea00640 1 -- 172.21.15.60:0/2432976627 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3904084580 con 0x7f391815bc30 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.672+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f391815b2f0 con 0x7f391815bc30 2024-09-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.672+0000 7f390ea00640 1 -- 172.21.15.60:0/2432976627 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f3904083e80 con 0x7f391815bc30 2024-09-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:40:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.674+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f38f8078a00 msgr2=0x7f38f807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f38f8078a00 0x7f38f807aec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3908001720 tx=0x7f3908007e90 comp rx=0 tx=0).stop 2024-09-16T15:40:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f391815bc30 msgr2=0x7f39180cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f391815bc30 0x7f39180cc940 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3904000c00 tx=0x7f3904032c80 comp rx=0 tx=0).stop 2024-09-16T15:40:25.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 shutdown_connections 2024-09-16T15:40:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f38f8078a00 0x7f38f807aec0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39181f4bc0 0x7f39180cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f391815bc30 0x7f39180cc940 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 --2- 172.21.15.60:0/2432976627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39180d0540 0x7f39181f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 >> 172.21.15.60:0/2432976627 conn(0x7f39180cfa20 msgr2=0x7f391815a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 shutdown_connections 2024-09-16T15:40:25.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:25.675+0000 7f3917e00640 1 -- 172.21.15.60:0/2432976627 wait complete. 2024-09-16T15:40:25.691 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:40:25.691 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:40:25.691 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:40:25.691 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:40:25.691 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:40:25.691 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:40:25.692 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:40:25.692 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:40:25.692 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:40:25.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:25.692 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:40:25.733 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:40:25.733 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:40:25.733 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:40:25.733 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:25.733 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:40:25.733 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:25.733 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:25.733 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42623sec preferred_lft 42623sec 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:40:25.734 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:25.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:25.736 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:40:25.830 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:40:25.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:25.833 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:40:25.870 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:40:25.870 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:40:25.870 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:40:25.870 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:40:25.870 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:40:25.870 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:40:25.870 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:40:25.871 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:40:25.871 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:40:25.871 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:40:25.871 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:40:25.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:40:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:40:26.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:40:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:40:26.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:26.067 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:40:26.102 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:40:26.102 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:40:26.102 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:40:26.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:26.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:40:26.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:40:26.198 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:40:26.232 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:40:26.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:40:26.233 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:40:26.307 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:40:26.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:26.308 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:40:26.437 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:40:26.437 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:40:26.464 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:40:26.464 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:40:26.465 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:40:26.465 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:40:26.465 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:40:26.466 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:40:26.466 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:40:26.466 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:40:26.466 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:40:26.467 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:40:26.469 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:40:26.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:26.470 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:40:26.525 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:40:26.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:40:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:40:26.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:40:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:40:26.614 INFO:teuthology.orchestra.run.smithi100.stdout:client5117 2024-09-16T15:40:26.614 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:40:26.615 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:26.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:40:26.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.826+0000 7f46fe400640 1 -- 172.21.15.60:0/1250387713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f806ee00 msgr2=0x7f46f806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.826+0000 7f46fe400640 1 --2- 172.21.15.60:0/1250387713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f806ee00 0x7f46f806f300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46e0005fd0 tx=0x7f46e002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.826+0000 7f46fe400640 1 -- 172.21.15.60:0/1250387713 shutdown_connections 2024-09-16T15:40:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.826+0000 7f46fe400640 1 --2- 172.21.15.60:0/1250387713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f806ee00 0x7f46f806f300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.826+0000 7f46fe400640 1 --2- 172.21.15.60:0/1250387713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f80b4770 0x7f46f80ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.826+0000 7f46fe400640 1 -- 172.21.15.60:0/1250387713 >> 172.21.15.60:0/1250387713 conn(0x7f46f80b18e0 msgr2=0x7f46f80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.826+0000 7f46fe400640 1 -- 172.21.15.60:0/1250387713 shutdown_connections 2024-09-16T15:40:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.826+0000 7f46fe400640 1 -- 172.21.15.60:0/1250387713 wait complete. 2024-09-16T15:40:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.827+0000 7f46fe400640 1 Processor -- start 2024-09-16T15:40:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.828+0000 7f46fe400640 1 -- start start 2024-09-16T15:40:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.828+0000 7f46fe400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f806ee00 0x7f46f81f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.828+0000 7f46fe400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f80b4770 0x7f46f814ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.828+0000 7f46fe400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f815dfc0 0x7f46f814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.828+0000 7f46fe400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f46f80adfa0 con 0x7f46f806ee00 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.828+0000 7f46fe400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46f80ade20 con 0x7f46f80b4770 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.828+0000 7f46fe400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f46f80ae120 con 0x7f46f815dfc0 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f806ee00 0x7f46f81f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f806ee00 0x7f46f81f8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40552/0 (socket says 172.21.15.60:40552) 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7400640 1 -- 172.21.15.60:0/3185997802 learned_addr learned my addr 172.21.15.60:0/3185997802 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46eea00640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f80b4770 0x7f46f814ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7e00640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f815dfc0 0x7f46f814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7400640 1 -- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f815dfc0 msgr2=0x7f46f814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f815dfc0 0x7f46f814c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7400640 1 -- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f80b4770 msgr2=0x7f46f814ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f80b4770 0x7f46f814ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.829+0000 7f46f7400640 1 -- 172.21.15.60:0/3185997802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f814f6b0 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.830+0000 7f46f7400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f806ee00 0x7f46f81f8ee0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c3a0 tx=0x7f46e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.830+0000 7f46efe00640 1 -- 172.21.15.60:0/3185997802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8017070 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.830+0000 7f46efe00640 1 -- 172.21.15.60:0/3185997802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46e80053c0 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.830+0000 7f46efe00640 1 -- 172.21.15.60:0/3185997802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8003850 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.830+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f814f990 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.830+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f809b8e0 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.831+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f8069360 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.832+0000 7f46efe00640 1 -- 172.21.15.60:0/3185997802 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f46e801a070 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.833+0000 7f46efe00640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f46d4078a30 0x7f46d407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.833+0000 7f46efe00640 1 -- 172.21.15.60:0/3185997802 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f46e8099ff0 con 0x7f46f806ee00 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.834+0000 7f46eea00640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f46d4078a30 0x7f46d407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.834+0000 7f46eea00640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f46d4078a30 0x7f46d407aef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46e0033000 tx=0x7f46e0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:26.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:26.837+0000 7f46efe00640 1 -- 172.21.15.60:0/3185997802 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f46e8062300 con 0x7f46f806ee00 2024-09-16T15:40:27.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.078+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f46f806f300 con 0x7f46f806ee00 2024-09-16T15:40:27.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.901+0000 7f46efe00640 1 -- 172.21.15.60:0/3185997802 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7f46e8061c00 con 0x7f46f806ee00 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f46d4078a30 msgr2=0x7f46d407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f46d4078a30 0x7f46d407aef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46e0033000 tx=0x7f46e0004730 comp rx=0 tx=0).stop 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f806ee00 msgr2=0x7f46f81f8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f806ee00 0x7f46f81f8ee0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c3a0 tx=0x7f46e800c860 comp rx=0 tx=0).stop 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 shutdown_connections 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f46d4078a30 0x7f46d407aef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46f815dfc0 0x7f46f814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f80b4770 0x7f46f814ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 --2- 172.21.15.60:0/3185997802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46f806ee00 0x7f46f81f8ee0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.904+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 >> 172.21.15.60:0/3185997802 conn(0x7f46f80b18e0 msgr2=0x7f46f815ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:27.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.905+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 shutdown_connections 2024-09-16T15:40:27.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:27.905+0000 7f46fe400640 1 -- 172.21.15.60:0/3185997802 wait complete. 2024-09-16T15:40:27.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:40:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.089+0000 7f2815e00640 1 -- 172.21.15.60:0/2332023460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 msgr2=0x7f2810074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.089+0000 7f2815e00640 1 --2- 172.21.15.60:0/2332023460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 0x7f2810074440 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2808005fd0 tx=0x7f280802aa10 comp rx=0 tx=0).stop 2024-09-16T15:40:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.089+0000 7f2815e00640 1 -- 172.21.15.60:0/2332023460 shutdown_connections 2024-09-16T15:40:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.089+0000 7f2815e00640 1 --2- 172.21.15.60:0/2332023460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 0x7f2810074440 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.089+0000 7f2815e00640 1 --2- 172.21.15.60:0/2332023460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28100ccf00 0x7f28100b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.089+0000 7f2815e00640 1 -- 172.21.15.60:0/2332023460 >> 172.21.15.60:0/2332023460 conn(0x7f28100b1950 msgr2=0x7f28100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.089+0000 7f2815e00640 1 -- 172.21.15.60:0/2332023460 shutdown_connections 2024-09-16T15:40:28.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.089+0000 7f2815e00640 1 -- 172.21.15.60:0/2332023460 wait complete. 2024-09-16T15:40:28.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.091+0000 7f2815e00640 1 Processor -- start 2024-09-16T15:40:28.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.091+0000 7f2815e00640 1 -- start start 2024-09-16T15:40:28.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f2815e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 0x7f28100b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f2815e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28100ccf00 0x7f28101f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f2815e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f281015f5a0 0x7f28100c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f2815e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f281006f2e0 con 0x7f2810073fb0 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f2815e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f281006f160 con 0x7f28100ccf00 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f2815e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f281006f460 con 0x7f281015f5a0 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f280ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 0x7f28100b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f280ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 0x7f28100b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40574/0 (socket says 172.21.15.60:40574) 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.092+0000 7f280ea00640 1 -- 172.21.15.60:0/3420641274 learned_addr learned my addr 172.21.15.60:0/3420641274 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f280f400640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f281015f5a0 0x7f28100c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f2806000640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28100ccf00 0x7f28101f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f280ea00640 1 -- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f281015f5a0 msgr2=0x7f28100c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f280ea00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f281015f5a0 0x7f28100c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f280ea00640 1 -- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28100ccf00 msgr2=0x7f28101f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f280ea00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28100ccf00 0x7f28101f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f280ea00640 1 -- 172.21.15.60:0/3420641274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28100b30f0 con 0x7f2810073fb0 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f280ea00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 0x7f28100b4190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f280000c3a0 tx=0x7f280000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f2807e00640 1 -- 172.21.15.60:0/3420641274 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800017070 con 0x7f2810073fb0 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f2807e00640 1 -- 172.21.15.60:0/3420641274 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28000053c0 con 0x7f2810073fb0 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f2807e00640 1 -- 172.21.15.60:0/3420641274 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800003850 con 0x7f2810073fb0 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.093+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28100b33d0 con 0x7f2810073fb0 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.094+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28100b3990 con 0x7f2810073fb0 2024-09-16T15:40:28.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.095+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281006f9f0 con 0x7f2810073fb0 2024-09-16T15:40:28.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.096+0000 7f2807e00640 1 -- 172.21.15.60:0/3420641274 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f280001a070 con 0x7f2810073fb0 2024-09-16T15:40:28.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.097+0000 7f2807e00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f27ec078a00 0x7f27ec07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.097+0000 7f2807e00640 1 -- 172.21.15.60:0/3420641274 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f2800099f60 con 0x7f2810073fb0 2024-09-16T15:40:28.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.097+0000 7f2806000640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f27ec078a00 0x7f27ec07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.098+0000 7f2806000640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f27ec078a00 0x7f27ec07aec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f281015a630 tx=0x7f27f800b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:28.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.101+0000 7f2807e00640 1 -- 172.21.15.60:0/3420641274 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2800061ca0 con 0x7f2810073fb0 2024-09-16T15:40:28.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.323+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f28100b0bf0 con 0x7f27ec078a00 2024-09-16T15:40:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.325+0000 7f2807e00640 1 -- 172.21.15.60:0/3420641274 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f28100b0bf0 con 0x7f27ec078a00 2024-09-16T15:40:28.326 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:40:28.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:40:28.327 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:40:28.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:40:28.327 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:40:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f27ec078a00 msgr2=0x7f27ec07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f27ec078a00 0x7f27ec07aec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f281015a630 tx=0x7f27f800b390 comp rx=0 tx=0).stop 2024-09-16T15:40:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 msgr2=0x7f28100b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 0x7f28100b4190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f280000c3a0 tx=0x7f280000c860 comp rx=0 tx=0).stop 2024-09-16T15:40:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 shutdown_connections 2024-09-16T15:40:28.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f27ec078a00 0x7f27ec07aec0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f281015f5a0 0x7f28100c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28100ccf00 0x7f28101f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 --2- 172.21.15.60:0/3420641274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2810073fb0 0x7f28100b4190 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 >> 172.21.15.60:0/3420641274 conn(0x7f28100b1950 msgr2=0x7f28100ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 shutdown_connections 2024-09-16T15:40:28.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.328+0000 7f2815e00640 1 -- 172.21.15.60:0/3420641274 wait complete. 2024-09-16T15:40:28.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:40:28.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.507+0000 7f6480c00640 1 -- 172.21.15.60:0/3405510385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c0cccd0 msgr2=0x7f647c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.507+0000 7f6480c00640 1 --2- 172.21.15.60:0/3405510385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c0cccd0 0x7f647c073440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6468009830 tx=0x7f646802a480 comp rx=0 tx=0).stop 2024-09-16T15:40:28.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.507+0000 7f6480c00640 1 -- 172.21.15.60:0/3405510385 shutdown_connections 2024-09-16T15:40:28.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.507+0000 7f6480c00640 1 --2- 172.21.15.60:0/3405510385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c0cccd0 0x7f647c073440 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.507+0000 7f6480c00640 1 -- 172.21.15.60:0/3405510385 >> 172.21.15.60:0/3405510385 conn(0x7f647c0b1840 msgr2=0x7f647c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:28.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.507+0000 7f6480c00640 1 -- 172.21.15.60:0/3405510385 shutdown_connections 2024-09-16T15:40:28.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.507+0000 7f6480c00640 1 -- 172.21.15.60:0/3405510385 wait complete. 2024-09-16T15:40:28.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.508+0000 7f6480c00640 1 Processor -- start 2024-09-16T15:40:28.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.509+0000 7f6480c00640 1 -- start start 2024-09-16T15:40:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.509+0000 7f6480c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f647c06efa0 0x7f647c1f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f6480c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f647c0cccd0 0x7f647c15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f6480c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c145750 0x7f647c1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f6480c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f647c0ce6a0 con 0x7f647c145750 2024-09-16T15:40:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f6480c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f647c0ce520 con 0x7f647c06efa0 2024-09-16T15:40:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f6480c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f647c0ce820 con 0x7f647c0cccd0 2024-09-16T15:40:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f647a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c145750 0x7f647c1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f647a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c145750 0x7f647c1399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40590/0 (socket says 172.21.15.60:40590) 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f647a800640 1 -- 172.21.15.60:0/3021980650 learned_addr learned my addr 172.21.15.60:0/3021980650 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f6479e00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f647c06efa0 0x7f647c1f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f6479400640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f647c0cccd0 0x7f647c15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f647a800640 1 -- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f647c0cccd0 msgr2=0x7f647c15e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f647a800640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f647c0cccd0 0x7f647c15e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f647a800640 1 -- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f647c06efa0 msgr2=0x7f647c1f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f647a800640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f647c06efa0 0x7f647c1f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f647a800640 1 -- 172.21.15.60:0/3021980650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c1f8460 con 0x7f647c145750 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.510+0000 7f6479e00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f647c06efa0 0x7f647c1f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.511+0000 7f647a800640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c145750 0x7f647c1399b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f647400c780 tx=0x7f647400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.511+0000 7f6472000640 1 -- 172.21.15.60:0/3021980650 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474004b60 con 0x7f647c145750 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.511+0000 7f6472000640 1 -- 172.21.15.60:0/3021980650 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6474004df0 con 0x7f647c145750 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.511+0000 7f6472000640 1 -- 172.21.15.60:0/3021980650 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647400aa70 con 0x7f647c145750 2024-09-16T15:40:28.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.511+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c1f8700 con 0x7f647c145750 2024-09-16T15:40:28.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.511+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c1f8cc0 con 0x7f647c145750 2024-09-16T15:40:28.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.512+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647c0cf490 con 0x7f647c145750 2024-09-16T15:40:28.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.513+0000 7f6472000640 1 -- 172.21.15.60:0/3021980650 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f647400ac10 con 0x7f647c145750 2024-09-16T15:40:28.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.514+0000 7f6472000640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f645c078a00 0x7f645c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.514+0000 7f6472000640 1 -- 172.21.15.60:0/3021980650 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f647409af50 con 0x7f647c145750 2024-09-16T15:40:28.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.515+0000 7f6479e00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f645c078a00 0x7f645c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.515+0000 7f6479e00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f645c078a00 0x7f645c07aec0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f646c0016a0 tx=0x7f646c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:28.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.518+0000 7f6472000640 1 -- 172.21.15.60:0/3021980650 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6474020070 con 0x7f647c145750 2024-09-16T15:40:28.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.712+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f647c0b03c0 con 0x7f647c145750 2024-09-16T15:40:28.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.713+0000 7f6472000640 1 -- 172.21.15.60:0/3021980650 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f6474063220 con 0x7f647c145750 2024-09-16T15:40:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f645c078a00 msgr2=0x7f645c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f645c078a00 0x7f645c07aec0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f646c0016a0 tx=0x7f646c009040 comp rx=0 tx=0).stop 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c145750 msgr2=0x7f647c1399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c145750 0x7f647c1399b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f647400c780 tx=0x7f647400cc40 comp rx=0 tx=0).stop 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 shutdown_connections 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f645c078a00 0x7f645c07aec0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f647c145750 0x7f647c1399b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f647c0cccd0 0x7f647c15e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 --2- 172.21.15.60:0/3021980650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f647c06efa0 0x7f647c1f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 >> 172.21.15.60:0/3021980650 conn(0x7f647c0b1840 msgr2=0x7f647c0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 shutdown_connections 2024-09-16T15:40:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:28.715+0000 7f6480c00640 1 -- 172.21.15.60:0/3021980650 wait complete. 2024-09-16T15:40:28.732 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:40:28.732 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:28.768 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:40:28.768 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:28.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:40:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.043+0000 7f4308800640 1 -- 172.21.15.60:0/2517498565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f430015f5a0 msgr2=0x7f43000b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.043+0000 7f4308800640 1 --2- 172.21.15.60:0/2517498565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f430015f5a0 0x7f43000b2440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009830 tx=0x7f42f402a480 comp rx=0 tx=0).stop 2024-09-16T15:40:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.044+0000 7f4308800640 1 -- 172.21.15.60:0/2517498565 shutdown_connections 2024-09-16T15:40:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.044+0000 7f4308800640 1 --2- 172.21.15.60:0/2517498565 >> v2:172.21.15.100:3300/0 conn(0x7f43000ccf00 0x7f43000acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.044+0000 7f4308800640 1 --2- 172.21.15.60:0/2517498565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f430015f5a0 0x7f43000b2440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.044+0000 7f4308800640 1 -- 172.21.15.60:0/2517498565 >> 172.21.15.60:0/2517498565 conn(0x7f43000b1950 msgr2=0x7f43000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.044+0000 7f4308800640 1 -- 172.21.15.60:0/2517498565 shutdown_connections 2024-09-16T15:40:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.044+0000 7f4308800640 1 -- 172.21.15.60:0/2517498565 wait complete. 2024-09-16T15:40:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.045+0000 7f4308800640 1 Processor -- start 2024-09-16T15:40:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.045+0000 7f4308800640 1 -- start start 2024-09-16T15:40:29.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.046+0000 7f4308800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43000ad270 0x7f4300154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:29.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.046+0000 7f4308800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43000ccf00 0x7f43001550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:29.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.046+0000 7f4308800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f430015f5a0 0x7f430009c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:29.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.046+0000 7f4308800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f430006f2e0 con 0x7f43000ccf00 2024-09-16T15:40:29.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.046+0000 7f4308800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f430006f160 con 0x7f43000ad270 2024-09-16T15:40:29.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.046+0000 7f4308800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f430006f460 con 0x7f430015f5a0 2024-09-16T15:40:29.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43000ccf00 0x7f43001550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43000ccf00 0x7f43001550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46204/0 (socket says 172.21.15.60:46204) 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305400640 1 -- 172.21.15.60:0/2143787770 learned_addr learned my addr 172.21.15.60:0/2143787770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4306800640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f430015f5a0 0x7f430009c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305e00640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43000ad270 0x7f4300154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305400640 1 -- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f430015f5a0 msgr2=0x7f430009c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305400640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f430015f5a0 0x7f430009c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305400640 1 -- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43000ad270 msgr2=0x7f4300154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305400640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43000ad270 0x7f4300154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.047+0000 7f4305400640 1 -- 172.21.15.60:0/2143787770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43001e8920 con 0x7f43000ccf00 2024-09-16T15:40:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.048+0000 7f4305400640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43000ccf00 0x7f43001550d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42f000e9c0 tx=0x7f42f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.048+0000 7f42fa000640 1 -- 172.21.15.60:0/2143787770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f000cd80 con 0x7f43000ccf00 2024-09-16T15:40:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.048+0000 7f42fa000640 1 -- 172.21.15.60:0/2143787770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42f0004020 con 0x7f43000ccf00 2024-09-16T15:40:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.048+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43001e95f0 con 0x7f43000ccf00 2024-09-16T15:40:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.048+0000 7f42fa000640 1 -- 172.21.15.60:0/2143787770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0004450 con 0x7f43000ccf00 2024-09-16T15:40:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.048+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43001e9bb0 con 0x7f43000ccf00 2024-09-16T15:40:29.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.049+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f430006f980 con 0x7f43000ccf00 2024-09-16T15:40:29.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.050+0000 7f42fa000640 1 -- 172.21.15.60:0/2143787770 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f42f00035f0 con 0x7f43000ccf00 2024-09-16T15:40:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.051+0000 7f42fa000640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f42e4078a00 0x7f42e407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.051+0000 7f42fa000640 1 -- 172.21.15.60:0/2143787770 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f42f009b670 con 0x7f43000ccf00 2024-09-16T15:40:29.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.052+0000 7f4305e00640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f42e4078a00 0x7f42e407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:29.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.052+0000 7f4305e00640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f42e4078a00 0x7f42e407aec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42f4034000 tx=0x7f42f4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:29.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.055+0000 7f42fa000640 1 -- 172.21.15.60:0/2143787770 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f42f00632f0 con 0x7f43000ccf00 2024-09-16T15:40:29.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.263+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43001e85f0 con 0x7f43000ccf00 2024-09-16T15:40:29.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.263+0000 7f42fa000640 1 -- 172.21.15.60:0/2143787770 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7f42f0067d20 con 0x7f43000ccf00 2024-09-16T15:40:29.264 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f42e4078a00 msgr2=0x7f42e407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f42e4078a00 0x7f42e407aec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42f4034000 tx=0x7f42f4004730 comp rx=0 tx=0).stop 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43000ccf00 msgr2=0x7f43001550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43000ccf00 0x7f43001550d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42f000e9c0 tx=0x7f42f000ee80 comp rx=0 tx=0).stop 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 shutdown_connections 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f42e4078a00 0x7f42e407aec0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f430015f5a0 0x7f430009c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43000ccf00 0x7f43001550d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 --2- 172.21.15.60:0/2143787770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43000ad270 0x7f4300154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 >> 172.21.15.60:0/2143787770 conn(0x7f43000b1950 msgr2=0x7f43000b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:29.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.265+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 shutdown_connections 2024-09-16T15:40:29.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.266+0000 7f4308800640 1 -- 172.21.15.60:0/2143787770 wait complete. 2024-09-16T15:40:29.282 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:40:29.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:29.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:29.345 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:40:29.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:29.345 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:40:29.381 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:40:29.381 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:29.421 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:40:29.421 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:29.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:40:29.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.659+0000 7f43c6400640 1 -- 172.21.15.60:0/351953085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43c015f5a0 msgr2=0x7f43c00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.659+0000 7f43c6400640 1 --2- 172.21.15.60:0/351953085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43c015f5a0 0x7f43c00b2440 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f43b0005fd0 tx=0x7f43b002a540 comp rx=0 tx=0).stop 2024-09-16T15:40:29.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.660+0000 7f43c6400640 1 -- 172.21.15.60:0/351953085 shutdown_connections 2024-09-16T15:40:29.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.660+0000 7f43c6400640 1 --2- 172.21.15.60:0/351953085 >> v2:172.21.15.100:3300/0 conn(0x7f43c0073fb0 0x7f43c0074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.660+0000 7f43c6400640 1 --2- 172.21.15.60:0/351953085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43c015f5a0 0x7f43c00b2440 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.660+0000 7f43c6400640 1 -- 172.21.15.60:0/351953085 >> 172.21.15.60:0/351953085 conn(0x7f43c00b1950 msgr2=0x7f43c00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:29.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.660+0000 7f43c6400640 1 -- 172.21.15.60:0/351953085 shutdown_connections 2024-09-16T15:40:29.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.660+0000 7f43c6400640 1 -- 172.21.15.60:0/351953085 wait complete. 2024-09-16T15:40:29.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.661+0000 7f43c6400640 1 Processor -- start 2024-09-16T15:40:29.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.661+0000 7f43c6400640 1 -- start start 2024-09-16T15:40:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.662+0000 7f43c6400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c0073fb0 0x7f43c0144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.662+0000 7f43c6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00ccf00 0x7f43c014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.662+0000 7f43c6400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43c015f5a0 0x7f43c014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.662+0000 7f43c6400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f43c006f2e0 con 0x7f43c0073fb0 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.662+0000 7f43c6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43c006f160 con 0x7f43c00ccf00 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.662+0000 7f43c6400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f43c006f460 con 0x7f43c015f5a0 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.662+0000 7f43bf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c0073fb0 0x7f43c0144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c0073fb0 0x7f43c0144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46220/0 (socket says 172.21.15.60:46220) 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bf400640 1 -- 172.21.15.60:0/3554791925 learned_addr learned my addr 172.21.15.60:0/3554791925 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bfe00640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43c015f5a0 0x7f43c014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43b6a00640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00ccf00 0x7f43c014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bf400640 1 -- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43c015f5a0 msgr2=0x7f43c014fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bf400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43c015f5a0 0x7f43c014fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bf400640 1 -- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00ccf00 msgr2=0x7f43c014c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bf400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00ccf00 0x7f43c014c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bf400640 1 -- 172.21.15.60:0/3554791925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c0144b80 con 0x7f43c0073fb0 2024-09-16T15:40:29.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.663+0000 7f43bf400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c0073fb0 0x7f43c0144640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f43b0034040 tx=0x7f43b00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:29.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.664+0000 7f43b7e00640 1 -- 172.21.15.60:0/3554791925 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b003e030 con 0x7f43c0073fb0 2024-09-16T15:40:29.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.664+0000 7f43b7e00640 1 -- 172.21.15.60:0/3554791925 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43b0003db0 con 0x7f43c0073fb0 2024-09-16T15:40:29.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.664+0000 7f43b7e00640 1 -- 172.21.15.60:0/3554791925 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0035040 con 0x7f43c0073fb0 2024-09-16T15:40:29.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.664+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c0151900 con 0x7f43c0073fb0 2024-09-16T15:40:29.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.664+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c014bf80 con 0x7f43c0073fb0 2024-09-16T15:40:29.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.665+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c006f9f0 con 0x7f43c0073fb0 2024-09-16T15:40:29.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.666+0000 7f43b7e00640 1 -- 172.21.15.60:0/3554791925 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f43b0041030 con 0x7f43c0073fb0 2024-09-16T15:40:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.667+0000 7f43b7e00640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f439c078a00 0x7f439c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.667+0000 7f43b7e00640 1 -- 172.21.15.60:0/3554791925 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f43b00bb290 con 0x7f43c0073fb0 2024-09-16T15:40:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.667+0000 7f43b6a00640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f439c078a00 0x7f439c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.668+0000 7f43b6a00640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f439c078a00 0x7f439c07aec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43c014bab0 tx=0x7f43a8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:29.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.671+0000 7f43b7e00640 1 -- 172.21.15.60:0/3554791925 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f43c0144b80 con 0x7f43c0073fb0 2024-09-16T15:40:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.883+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43c00aff70 con 0x7f43c0073fb0 2024-09-16T15:40:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.883+0000 7f43b7e00640 1 -- 172.21.15.60:0/3554791925 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) ==== 80+0+0 (secure 0 0 0) 0x7f43b0082f30 con 0x7f43c0073fb0 2024-09-16T15:40:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:40:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f439c078a00 msgr2=0x7f439c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f439c078a00 0x7f439c07aec0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f43c014bab0 tx=0x7f43a8009040 comp rx=0 tx=0).stop 2024-09-16T15:40:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c0073fb0 msgr2=0x7f43c0144640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c0073fb0 0x7f43c0144640 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f43b0034040 tx=0x7f43b00038a0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 shutdown_connections 2024-09-16T15:40:29.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f439c078a00 0x7f439c07aec0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43c015f5a0 0x7f43c014fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00ccf00 0x7f43c014c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 --2- 172.21.15.60:0/3554791925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43c0073fb0 0x7f43c0144640 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 >> 172.21.15.60:0/3554791925 conn(0x7f43c00b1950 msgr2=0x7f43c009c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 shutdown_connections 2024-09-16T15:40:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:29.886+0000 7f43c6400640 1 -- 172.21.15.60:0/3554791925 wait complete. 2024-09-16T15:40:29.902 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:40:29.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:29.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:40:29.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:40:29.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:29.970 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:40:30.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.168+0000 7fa5df400640 1 -- 172.21.15.60:0/3996416145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5d80ccf00 msgr2=0x7fa5d80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.168+0000 7fa5df400640 1 --2- 172.21.15.60:0/3996416145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5d80ccf00 0x7fa5d80b2440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009860 tx=0x7fa5c402a510 comp rx=0 tx=0).stop 2024-09-16T15:40:30.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.168+0000 7fa5df400640 1 -- 172.21.15.60:0/3996416145 shutdown_connections 2024-09-16T15:40:30.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.168+0000 7fa5df400640 1 --2- 172.21.15.60:0/3996416145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5d80ccf00 0x7fa5d80b2440 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.168+0000 7fa5df400640 1 -- 172.21.15.60:0/3996416145 >> 172.21.15.60:0/3996416145 conn(0x7fa5d80b1950 msgr2=0x7fa5d80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:30.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.168+0000 7fa5df400640 1 -- 172.21.15.60:0/3996416145 shutdown_connections 2024-09-16T15:40:30.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.168+0000 7fa5df400640 1 -- 172.21.15.60:0/3996416145 wait complete. 2024-09-16T15:40:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.170+0000 7fa5df400640 1 Processor -- start 2024-09-16T15:40:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.170+0000 7fa5df400640 1 -- start start 2024-09-16T15:40:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5df400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5d8073fb0 0x7fa5d815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5df400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5d80ccf00 0x7fa5d809cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5df400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5d815f5a0 0x7fa5d809d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5df400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d806ef30 con 0x7fa5d80ccf00 2024-09-16T15:40:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5df400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d806fa80 con 0x7fa5d815f5a0 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5df400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d8074440 con 0x7fa5d8073fb0 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5d7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5d80ccf00 0x7fa5d809cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5d7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5d80ccf00 0x7fa5d809cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46236/0 (socket says 172.21.15.60:46236) 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5d7e00640 1 -- 172.21.15.60:0/3867389919 learned_addr learned my addr 172.21.15.60:0/3867389919 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5dca00640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5d8073fb0 0x7fa5d815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5dd400640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5d815f5a0 0x7fa5d809d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5d7e00640 1 -- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5d8073fb0 msgr2=0x7fa5d815da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5d7e00640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5d8073fb0 0x7fa5d815da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5d7e00640 1 -- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5d815f5a0 msgr2=0x7fa5d809d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.171+0000 7fa5d7e00640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5d815f5a0 0x7fa5d809d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.172+0000 7fa5d7e00640 1 -- 172.21.15.60:0/3867389919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d81f5370 con 0x7fa5d80ccf00 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.172+0000 7fa5d7e00640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5d80ccf00 0x7fa5d809cc80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009980 tx=0x7fa5c4032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.172+0000 7fa5d4c00640 1 -- 172.21.15.60:0/3867389919 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4040070 con 0x7fa5d80ccf00 2024-09-16T15:40:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.172+0000 7fa5d4c00640 1 -- 172.21.15.60:0/3867389919 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5c40038e0 con 0x7fa5d80ccf00 2024-09-16T15:40:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.172+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d814a670 con 0x7fa5d80ccf00 2024-09-16T15:40:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.172+0000 7fa5d4c00640 1 -- 172.21.15.60:0/3867389919 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4003d30 con 0x7fa5d80ccf00 2024-09-16T15:40:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.172+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d814ac30 con 0x7fa5d80ccf00 2024-09-16T15:40:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.173+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d80b2440 con 0x7fa5d80ccf00 2024-09-16T15:40:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.174+0000 7fa5d4c00640 1 -- 172.21.15.60:0/3867389919 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7fa5c4039040 con 0x7fa5d80ccf00 2024-09-16T15:40:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.175+0000 7fa5d4c00640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5b8078a00 0x7fa5b807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.175+0000 7fa5d4c00640 1 -- 172.21.15.60:0/3867389919 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa5c40bbfe0 con 0x7fa5d80ccf00 2024-09-16T15:40:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.176+0000 7fa5dca00640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5b8078a00 0x7fa5b807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:30.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.177+0000 7fa5dca00640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5b8078a00 0x7fa5b807aec0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc007fd0 tx=0x7fa5cc001370 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.180+0000 7fa5d4c00640 1 -- 172.21.15.60:0/3867389919 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa5c4083c80 con 0x7fa5d80ccf00 2024-09-16T15:40:30.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:40:30.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:40:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.419+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5d815df40 con 0x7fa5d80ccf00 2024-09-16T15:40:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.419+0000 7fa5d4c00640 1 -- 172.21.15.60:0/3867389919 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fa5c40886b0 con 0x7fa5d80ccf00 2024-09-16T15:40:30.421 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:30.421 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:40:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5b8078a00 msgr2=0x7fa5b807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5b8078a00 0x7fa5b807aec0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc007fd0 tx=0x7fa5cc001370 comp rx=0 tx=0).stop 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5d80ccf00 msgr2=0x7fa5d809cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5d80ccf00 0x7fa5d809cc80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009980 tx=0x7fa5c4032b90 comp rx=0 tx=0).stop 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 shutdown_connections 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa5b8078a00 0x7fa5b807aec0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5d815f5a0 0x7fa5d809d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5d80ccf00 0x7fa5d809cc80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 --2- 172.21.15.60:0/3867389919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5d8073fb0 0x7fa5d815da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 >> 172.21.15.60:0/3867389919 conn(0x7fa5d80b1950 msgr2=0x7fa5d809b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 shutdown_connections 2024-09-16T15:40:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.422+0000 7fa5df400640 1 -- 172.21.15.60:0/3867389919 wait complete. 2024-09-16T15:40:30.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:40:30.619 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-09-16T15:40:30.619 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-16T15:40:30.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol70600 2024-09-16T15:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.781+0000 7f5195400640 1 -- 172.21.15.60:0/497892155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51900cec60 msgr2=0x7f51900cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.781+0000 7f5195400640 1 --2- 172.21.15.60:0/497892155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51900cec60 0x7f51900cf070 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5188005fd0 tx=0x7f518802ab40 comp rx=0 tx=0).stop 2024-09-16T15:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.781+0000 7f5195400640 1 -- 172.21.15.60:0/497892155 shutdown_connections 2024-09-16T15:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.781+0000 7f5195400640 1 --2- 172.21.15.60:0/497892155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51900cec60 0x7f51900cf070 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.781+0000 7f5195400640 1 -- 172.21.15.60:0/497892155 >> 172.21.15.60:0/497892155 conn(0x7f51900b1840 msgr2=0x7f51900b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.782+0000 7f5195400640 1 -- 172.21.15.60:0/497892155 shutdown_connections 2024-09-16T15:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.782+0000 7f5195400640 1 -- 172.21.15.60:0/497892155 wait complete. 2024-09-16T15:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.783+0000 7f5195400640 1 Processor -- start 2024-09-16T15:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.783+0000 7f5195400640 1 -- start start 2024-09-16T15:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.784+0000 7f5195400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51900af190 0x7f5190151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.784+0000 7f5195400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51900cec60 0x7f51900aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.784+0000 7f5195400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5190161760 0x7f51900ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.784+0000 7f5195400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5190162520 con 0x7f5190161760 2024-09-16T15:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.784+0000 7f5195400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51901623a0 con 0x7f51900cec60 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.784+0000 7f5195400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f51901626a0 con 0x7f51900af190 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5190161760 0x7f51900ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5190161760 0x7f51900ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46260/0 (socket says 172.21.15.60:46260) 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 -- 172.21.15.60:0/1424903712 learned_addr learned my addr 172.21.15.60:0/1424903712 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518e800640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51900af190 0x7f5190151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518de00640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51900cec60 0x7f51900aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 -- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51900af190 msgr2=0x7f5190151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51900af190 0x7f5190151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 -- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51900cec60 msgr2=0x7f51900aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51900cec60 0x7f51900aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 -- 172.21.15.60:0/1424903712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51900a9960 con 0x7f5190161760 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518de00640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51900cec60 0x7f51900aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.785+0000 7f518f200640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5190161760 0x7f51900ab8e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5188009d90 tx=0x7f518802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.786+0000 7f5186a00640 1 -- 172.21.15.60:0/1424903712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188044070 con 0x7f5190161760 2024-09-16T15:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.786+0000 7f5186a00640 1 -- 172.21.15.60:0/1424903712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5188004250 con 0x7f5190161760 2024-09-16T15:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.786+0000 7f5186a00640 1 -- 172.21.15.60:0/1424903712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51880046a0 con 0x7f5190161760 2024-09-16T15:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.786+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51900a9c40 con 0x7f5190161760 2024-09-16T15:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.786+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51900aa200 con 0x7f5190161760 2024-09-16T15:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.787+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51901612c0 con 0x7f5190161760 2024-09-16T15:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.788+0000 7f5186a00640 1 -- 172.21.15.60:0/1424903712 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f518802c9a0 con 0x7f5190161760 2024-09-16T15:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.789+0000 7f5186a00640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5170078a00 0x7f517007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.789+0000 7f5186a00640 1 -- 172.21.15.60:0/1424903712 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f51880bb4f0 con 0x7f5190161760 2024-09-16T15:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.789+0000 7f518e800640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5170078a00 0x7f517007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.790+0000 7f518e800640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5170078a00 0x7f517007aec0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5180001720 tx=0x7f5180009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:30.793+0000 7f5186a00640 1 -- 172.21.15.60:0/1424903712 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51901612c0 con 0x7f5190161760 2024-09-16T15:40:31.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.017+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol70600", "target": ["mon-mgr", ""]}) -- 0x7f5190162b50 con 0x7f5170078a00 2024-09-16T15:40:31.025 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:40:31.024+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:31.025 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:40:31.024+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:31.025 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:40:31.024+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:31.026 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:40:31.024+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:31.026 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:40:31.024+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:31.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.061+0000 7f5186a00640 1 -- 172.21.15.60:0/1424903712 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5190162b50 con 0x7f5170078a00 2024-09-16T15:40:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5170078a00 msgr2=0x7f517007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5170078a00 0x7f517007aec0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5180001720 tx=0x7f5180009040 comp rx=0 tx=0).stop 2024-09-16T15:40:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5190161760 msgr2=0x7f51900ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5190161760 0x7f51900ab8e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5188009d90 tx=0x7f518802c8b0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 shutdown_connections 2024-09-16T15:40:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5170078a00 0x7f517007aec0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5190161760 0x7f51900ab8e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51900cec60 0x7f51900aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 --2- 172.21.15.60:0/1424903712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51900af190 0x7f5190151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 >> 172.21.15.60:0/1424903712 conn(0x7f51900b1840 msgr2=0x7f51900b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 shutdown_connections 2024-09-16T15:40:31.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.064+0000 7f5195400640 1 -- 172.21.15.60:0/1424903712 wait complete. 2024-09-16T15:40:31.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:31.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.240+0000 7f0f94800640 1 -- 172.21.15.60:0/3094454185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 msgr2=0x7f0f8c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.240+0000 7f0f94800640 1 --2- 172.21.15.60:0/3094454185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 0x7f0f8c0b2440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009860 tx=0x7f0f8002a510 comp rx=0 tx=0).stop 2024-09-16T15:40:31.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.240+0000 7f0f94800640 1 -- 172.21.15.60:0/3094454185 shutdown_connections 2024-09-16T15:40:31.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.240+0000 7f0f94800640 1 --2- 172.21.15.60:0/3094454185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 0x7f0f8c0b2440 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.240+0000 7f0f94800640 1 -- 172.21.15.60:0/3094454185 >> 172.21.15.60:0/3094454185 conn(0x7f0f8c0b1950 msgr2=0x7f0f8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.240+0000 7f0f94800640 1 -- 172.21.15.60:0/3094454185 shutdown_connections 2024-09-16T15:40:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.240+0000 7f0f94800640 1 -- 172.21.15.60:0/3094454185 wait complete. 2024-09-16T15:40:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.241+0000 7f0f94800640 1 Processor -- start 2024-09-16T15:40:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.241+0000 7f0f94800640 1 -- start start 2024-09-16T15:40:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.242+0000 7f0f94800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f8c073fb0 0x7f0f8c149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.242+0000 7f0f94800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f8c0ccf00 0x7f0f8c1f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.242+0000 7f0f94800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 0x7f0f8c146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.242+0000 7f0f94800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c06ef30 con 0x7f0f8c073fb0 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.242+0000 7f0f94800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c06fa80 con 0x7f0f8c0ccf00 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.242+0000 7f0f94800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8c074460 con 0x7f0f8c15f5a0 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f8c073fb0 0x7f0f8c149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f8c073fb0 0x7f0f8c149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46276/0 (socket says 172.21.15.60:46276) 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 -- 172.21.15.60:0/350115891 learned_addr learned my addr 172.21.15.60:0/350115891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f92800640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 0x7f0f8c146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91400640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f8c0ccf00 0x7f0f8c1f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 -- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 msgr2=0x7f0f8c146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 0x7f0f8c146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 -- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f8c0ccf00 msgr2=0x7f0f8c1f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f8c0ccf00 0x7f0f8c1f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 -- 172.21.15.60:0/350115891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c1f8490 con 0x7f0f8c073fb0 2024-09-16T15:40:31.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f92800640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 0x7f0f8c146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:31.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91400640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f8c0ccf00 0x7f0f8c1f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:31.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.243+0000 7f0f91e00640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f8c073fb0 0x7f0f8c149e90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0f8002ab00 tx=0x7f0f80033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.244+0000 7f0f86000640 1 -- 172.21.15.60:0/350115891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80005c90 con 0x7f0f8c073fb0 2024-09-16T15:40:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.244+0000 7f0f86000640 1 -- 172.21.15.60:0/350115891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f80031e50 con 0x7f0f8c073fb0 2024-09-16T15:40:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.244+0000 7f0f86000640 1 -- 172.21.15.60:0/350115891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80005260 con 0x7f0f8c073fb0 2024-09-16T15:40:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.244+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8c1f8770 con 0x7f0f8c073fb0 2024-09-16T15:40:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.244+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8c1474b0 con 0x7f0f8c073fb0 2024-09-16T15:40:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.245+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8c0b2440 con 0x7f0f8c073fb0 2024-09-16T15:40:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.246+0000 7f0f86000640 1 -- 172.21.15.60:0/350115891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f0f80005400 con 0x7f0f8c073fb0 2024-09-16T15:40:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.247+0000 7f0f86000640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f70078a00 0x7f0f7007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.247+0000 7f0f86000640 1 -- 172.21.15.60:0/350115891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f0f800bbc30 con 0x7f0f8c073fb0 2024-09-16T15:40:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.247+0000 7f0f91400640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f70078a00 0x7f0f7007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.248+0000 7f0f91400640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f70078a00 0x7f0f7007aec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c157600 tx=0x7f0f7c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.251+0000 7f0f86000640 1 -- 172.21.15.60:0/350115891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0f8003f030 con 0x7f0f8c073fb0 2024-09-16T15:40:31.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.489+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f8c06aaf0 con 0x7f0f8c073fb0 2024-09-16T15:40:31.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.489+0000 7f0f86000640 1 -- 172.21.15.60:0/350115891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f0f80083f00 con 0x7f0f8c073fb0 2024-09-16T15:40:31.490 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:31.490 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:31.491 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:40:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f70078a00 msgr2=0x7f0f7007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f70078a00 0x7f0f7007aec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c157600 tx=0x7f0f7c009040 comp rx=0 tx=0).stop 2024-09-16T15:40:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f8c073fb0 msgr2=0x7f0f8c149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f8c073fb0 0x7f0f8c149e90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0f8002ab00 tx=0x7f0f80033c20 comp rx=0 tx=0).stop 2024-09-16T15:40:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 shutdown_connections 2024-09-16T15:40:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f70078a00 0x7f0f7007aec0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8c15f5a0 0x7f0f8c146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f8c0ccf00 0x7f0f8c1f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 --2- 172.21.15.60:0/350115891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f8c073fb0 0x7f0f8c149e90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.491+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 >> 172.21.15.60:0/350115891 conn(0x7f0f8c0b1950 msgr2=0x7f0f8c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:31.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.492+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 shutdown_connections 2024-09-16T15:40:31.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.492+0000 7f0f94800640 1 -- 172.21.15.60:0/350115891 wait complete. 2024-09-16T15:40:31.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:40:31.686 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":0.64499386299999995,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.967894077899786,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:40:31.687 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.967894077899786 }, "state": "open", "uptime": 0.644993863 } ] 2024-09-16T15:40:31.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:31.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.846+0000 7f0656400640 1 -- 172.21.15.60:0/3881297767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06500725d0 msgr2=0x7f06500b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.846+0000 7f0656400640 1 --2- 172.21.15.60:0/3881297767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06500725d0 0x7f06500b28f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f063c009830 tx=0x7f063c02a5c0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.846+0000 7f0656400640 1 -- 172.21.15.60:0/3881297767 shutdown_connections 2024-09-16T15:40:31.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.846+0000 7f0656400640 1 --2- 172.21.15.60:0/3881297767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06500725d0 0x7f06500b28f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.846+0000 7f0656400640 1 -- 172.21.15.60:0/3881297767 >> 172.21.15.60:0/3881297767 conn(0x7f06500cfa20 msgr2=0x7f06500cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:31.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.846+0000 7f0656400640 1 -- 172.21.15.60:0/3881297767 shutdown_connections 2024-09-16T15:40:31.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.846+0000 7f0656400640 1 -- 172.21.15.60:0/3881297767 wait complete. 2024-09-16T15:40:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.848+0000 7f0656400640 1 Processor -- start 2024-09-16T15:40:31.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.848+0000 7f0656400640 1 -- start start 2024-09-16T15:40:31.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0656400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06500d0540 0x7f06500c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:31.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0656400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f065015bc30 0x7f06500ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0656400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06501f8440 0x7f06501e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0656400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0650074660 con 0x7f06500d0540 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0656400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f065006fa30 con 0x7f06501f8440 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0656400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f06500ce530 con 0x7f065015bc30 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0655000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06500d0540 0x7f06500c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0655000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06500d0540 0x7f06500c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46278/0 (socket says 172.21.15.60:46278) 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0655000640 1 -- 172.21.15.60:0/803879273 learned_addr learned my addr 172.21.15.60:0/803879273 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f064fe00640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f065015bc30 0x7f06500ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:31.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.849+0000 7f0655a00640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06501f8440 0x7f06501e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f0655000640 1 -- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f065015bc30 msgr2=0x7f06500ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f0655000640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f065015bc30 0x7f06500ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f0655000640 1 -- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06501f8440 msgr2=0x7f06501e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f0655000640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06501f8440 0x7f06501e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f0655000640 1 -- 172.21.15.60:0/803879273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06500cef00 con 0x7f06500d0540 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f0655000640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06500d0540 0x7f06500c5ef0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f064400c430 tx=0x7f064400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f064cc00640 1 -- 172.21.15.60:0/803879273 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644017070 con 0x7f06500d0540 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f064cc00640 1 -- 172.21.15.60:0/803879273 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06440053c0 con 0x7f06500d0540 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f064cc00640 1 -- 172.21.15.60:0/803879273 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644003850 con 0x7f06500d0540 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.850+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06500cd920 con 0x7f06500d0540 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.851+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06500cdee0 con 0x7f06500d0540 2024-09-16T15:40:31.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.852+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0650050ee0 con 0x7f06500d0540 2024-09-16T15:40:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.853+0000 7f064cc00640 1 -- 172.21.15.60:0/803879273 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 7) ==== 101440+0+0 (secure 0 0 0) 0x7f064401a070 con 0x7f06500d0540 2024-09-16T15:40:31.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.854+0000 7f064cc00640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0630078a00 0x7f063007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:31.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.854+0000 7f064cc00640 1 -- 172.21.15.60:0/803879273 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f064409a000 con 0x7f06500d0540 2024-09-16T15:40:31.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.854+0000 7f064fe00640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0630078a00 0x7f063007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:31.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.855+0000 7f064fe00640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0630078a00 0x7f063007aec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f063c034000 tx=0x7f063c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.858+0000 7f064cc00640 1 -- 172.21.15.60:0/803879273 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0650050ee0 con 0x7f06500d0540 2024-09-16T15:40:31.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:31.923+0000 7f064cc00640 1 -- 172.21.15.60:0/803879273 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f064405df10 con 0x7f06500d0540 2024-09-16T15:40:32.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.095+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f065006a180 con 0x7f06500d0540 2024-09-16T15:40:32.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.096+0000 7f064cc00640 1 -- 172.21.15.60:0/803879273 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f065006a180 con 0x7f06500d0540 2024-09-16T15:40:32.097 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:32.097 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:32.098 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:40:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.098+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0630078a00 msgr2=0x7f063007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.098+0000 7f0656400640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0630078a00 0x7f063007aec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f063c034000 tx=0x7f063c004730 comp rx=0 tx=0).stop 2024-09-16T15:40:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.098+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06500d0540 msgr2=0x7f06500c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.098+0000 7f0656400640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06500d0540 0x7f06500c5ef0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f064400c430 tx=0x7f064400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.099+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 shutdown_connections 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.099+0000 7f0656400640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0630078a00 0x7f063007aec0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.099+0000 7f0656400640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06501f8440 0x7f06501e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.099+0000 7f0656400640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f065015bc30 0x7f06500ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.099+0000 7f0656400640 1 --2- 172.21.15.60:0/803879273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06500d0540 0x7f06500c5ef0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.099+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 >> 172.21.15.60:0/803879273 conn(0x7f06500cfa20 msgr2=0x7f06500cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.099+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 shutdown_connections 2024-09-16T15:40:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:32.099+0000 7f0656400640 1 -- 172.21.15.60:0/803879273 wait complete. 2024-09-16T15:40:32.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:40:32.294 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":1.252988078,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.937056016123357,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:40:32.294 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.937056016123357 }, "state": "open", "uptime": 1.252988078 } ] 2024-09-16T15:40:32.295 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:40:37.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:37.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.466+0000 7fe431000640 1 -- 172.21.15.60:0/1171809180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe42c0af3d0 msgr2=0x7fe42c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.466+0000 7fe431000640 1 --2- 172.21.15.60:0/1171809180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe42c0af3d0 0x7fe42c06f490 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe414009830 tx=0x7fe41402a480 comp rx=0 tx=0).stop 2024-09-16T15:40:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.466+0000 7fe431000640 1 -- 172.21.15.60:0/1171809180 shutdown_connections 2024-09-16T15:40:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.466+0000 7fe431000640 1 --2- 172.21.15.60:0/1171809180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe42c0af3d0 0x7fe42c06f490 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.466+0000 7fe431000640 1 --2- 172.21.15.60:0/1171809180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe42c161550 0x7fe42c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.466+0000 7fe431000640 1 -- 172.21.15.60:0/1171809180 >> 172.21.15.60:0/1171809180 conn(0x7fe42c0b1950 msgr2=0x7fe42c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.466+0000 7fe431000640 1 -- 172.21.15.60:0/1171809180 shutdown_connections 2024-09-16T15:40:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.466+0000 7fe431000640 1 -- 172.21.15.60:0/1171809180 wait complete. 2024-09-16T15:40:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.468+0000 7fe431000640 1 Processor -- start 2024-09-16T15:40:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.468+0000 7fe431000640 1 -- start start 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe431000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe42c06f9d0 0x7fe42c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe431000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe42c0af3d0 0x7fe42c0ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe431000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe42c161550 0x7fe42c0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe431000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c1623a0 con 0x7fe42c06f9d0 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe431000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c074010 con 0x7fe42c0af3d0 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe431000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe42c162520 con 0x7fe42c161550 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe429e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe42c06f9d0 0x7fe42c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe429e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe42c06f9d0 0x7fe42c0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46280/0 (socket says 172.21.15.60:46280) 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe429e00640 1 -- 172.21.15.60:0/1060399607 learned_addr learned my addr 172.21.15.60:0/1060399607 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe423e00640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe42c0af3d0 0x7fe42c0ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.469+0000 7fe42a800640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe42c161550 0x7fe42c0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe429e00640 1 -- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe42c161550 msgr2=0x7fe42c0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe429e00640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe42c161550 0x7fe42c0affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe429e00640 1 -- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe42c0af3d0 msgr2=0x7fe42c0ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe429e00640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe42c0af3d0 0x7fe42c0ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe429e00640 1 -- 172.21.15.60:0/1060399607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c159e20 con 0x7fe42c06f9d0 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe429e00640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe42c06f9d0 0x7fe42c0cdf80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00ea20 tx=0x7fe41c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe422000640 1 -- 172.21.15.60:0/1060399607 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c00ce40 con 0x7fe42c06f9d0 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe422000640 1 -- 172.21.15.60:0/1060399607 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe41c004020 con 0x7fe42c06f9d0 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe422000640 1 -- 172.21.15.60:0/1060399607 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c004470 con 0x7fe42c06f9d0 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c158140 con 0x7fe42c06f9d0 2024-09-16T15:40:37.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.470+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c158700 con 0x7fe42c06f9d0 2024-09-16T15:40:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.471+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42c0760b0 con 0x7fe42c06f9d0 2024-09-16T15:40:37.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.473+0000 7fe422000640 1 -- 172.21.15.60:0/1060399607 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7fe41c0035f0 con 0x7fe42c06f9d0 2024-09-16T15:40:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.473+0000 7fe422000640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe408078a90 0x7fe40807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.474+0000 7fe422000640 1 -- 172.21.15.60:0/1060399607 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fe41c09b6a0 con 0x7fe42c06f9d0 2024-09-16T15:40:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.474+0000 7fe423e00640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe408078a90 0x7fe40807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:37.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.475+0000 7fe423e00640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe408078a90 0x7fe40807af50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe414034000 tx=0x7fe414004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:37.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.478+0000 7fe422000640 1 -- 172.21.15.60:0/1060399607 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe42c159e20 con 0x7fe42c06f9d0 2024-09-16T15:40:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.715+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe42c162ab0 con 0x7fe42c06f9d0 2024-09-16T15:40:37.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.715+0000 7fe422000640 1 -- 172.21.15.60:0/1060399607 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fe41c0631c0 con 0x7fe42c06f9d0 2024-09-16T15:40:37.716 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:37.717 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:37.717 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:40:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe408078a90 msgr2=0x7fe40807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe408078a90 0x7fe40807af50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe414034000 tx=0x7fe414004730 comp rx=0 tx=0).stop 2024-09-16T15:40:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe42c06f9d0 msgr2=0x7fe42c0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe42c06f9d0 0x7fe42c0cdf80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00ea20 tx=0x7fe41c00eee0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 shutdown_connections 2024-09-16T15:40:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe408078a90 0x7fe40807af50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe42c161550 0x7fe42c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe42c0af3d0 0x7fe42c0ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 --2- 172.21.15.60:0/1060399607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe42c06f9d0 0x7fe42c0cdf80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 >> 172.21.15.60:0/1060399607 conn(0x7fe42c0b1950 msgr2=0x7fe42c0af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 shutdown_connections 2024-09-16T15:40:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:37.718+0000 7fe431000640 1 -- 172.21.15.60:0/1060399607 wait complete. 2024-09-16T15:40:37.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:40:37.914 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":6.8719346180000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.65409840419008,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:40:37.915 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.65409840419008 }, "state": "open", "uptime": 6.871934618 } ] 2024-09-16T15:40:37.915 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:40:42.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.086+0000 7fe8d5e00640 1 -- 172.21.15.60:0/3309990601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 msgr2=0x7fe8d0074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.086+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/3309990601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 0x7fe8d0074420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0075130 tx=0x7fe8c802a980 comp rx=0 tx=0).stop 2024-09-16T15:40:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.087+0000 7fe8d5e00640 1 -- 172.21.15.60:0/3309990601 shutdown_connections 2024-09-16T15:40:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.087+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/3309990601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 0x7fe8d0074420 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.087+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/3309990601 >> v2:172.21.15.100:3300/0 conn(0x7fe8d015f5a0 0x7fe8d00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.087+0000 7fe8d5e00640 1 -- 172.21.15.60:0/3309990601 >> 172.21.15.60:0/3309990601 conn(0x7fe8d00b1950 msgr2=0x7fe8d00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.087+0000 7fe8d5e00640 1 -- 172.21.15.60:0/3309990601 shutdown_connections 2024-09-16T15:40:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.087+0000 7fe8d5e00640 1 -- 172.21.15.60:0/3309990601 wait complete. 2024-09-16T15:40:43.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.088+0000 7fe8d5e00640 1 Processor -- start 2024-09-16T15:40:43.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.088+0000 7fe8d5e00640 1 -- start start 2024-09-16T15:40:43.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.089+0000 7fe8d5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 0x7fe8d0144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.089+0000 7fe8d5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d00ccf00 0x7fe8d015a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.089+0000 7fe8d5e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d015f5a0 0x7fe8d015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.089+0000 7fe8d5e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d006f2e0 con 0x7fe8d0073fb0 2024-09-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.089+0000 7fe8d5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d006f160 con 0x7fe8d00ccf00 2024-09-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.089+0000 7fe8d5e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d006f460 con 0x7fe8d015f5a0 2024-09-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 0x7fe8d0144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 0x7fe8d0144d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37132/0 (socket says 172.21.15.60:37132) 2024-09-16T15:40:43.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 -- 172.21.15.60:0/996159275 learned_addr learned my addr 172.21.15.60:0/996159275 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:43.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 -- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d015f5a0 msgr2=0x7fe8d015a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:43.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d015f5a0 0x7fe8d015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:43.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 -- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d00ccf00 msgr2=0x7fe8d015a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:43.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8ce000640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d00ccf00 0x7fe8d015a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:43.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d00ccf00 0x7fe8d015a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 -- 172.21.15.60:0/996159275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d009d2a0 con 0x7fe8d0073fb0 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8ce000640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d00ccf00 0x7fe8d015a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.090+0000 7fe8cea00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 0x7fe8d0144d20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000c970 tx=0x7fe8c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.091+0000 7fe8c7400640 1 -- 172.21.15.60:0/996159275 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0007bb0 con 0x7fe8d0073fb0 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.091+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d009d550 con 0x7fe8d0073fb0 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.091+0000 7fe8c7400640 1 -- 172.21.15.60:0/996159275 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8c0005020 con 0x7fe8d0073fb0 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.091+0000 7fe8c7400640 1 -- 172.21.15.60:0/996159275 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0007e60 con 0x7fe8d0073fb0 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.091+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d0145710 con 0x7fe8d0073fb0 2024-09-16T15:40:43.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.092+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d0145e20 con 0x7fe8d0073fb0 2024-09-16T15:40:43.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.093+0000 7fe8c7400640 1 -- 172.21.15.60:0/996159275 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7fe8c0005470 con 0x7fe8d0073fb0 2024-09-16T15:40:43.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.094+0000 7fe8c7400640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe8b0078a90 0x7fe8b007af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:43.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.094+0000 7fe8c7400640 1 -- 172.21.15.60:0/996159275 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fe8c009b3e0 con 0x7fe8d0073fb0 2024-09-16T15:40:43.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.094+0000 7fe8ce000640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe8b0078a90 0x7fe8b007af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:43.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.095+0000 7fe8ce000640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe8b0078a90 0x7fe8b007af50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc007fd0 tx=0x7fe8bc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:43.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.098+0000 7fe8c7400640 1 -- 172.21.15.60:0/996159275 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe8c0014030 con 0x7fe8d0073fb0 2024-09-16T15:40:43.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.334+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8d006f9f0 con 0x7fe8d0073fb0 2024-09-16T15:40:43.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.335+0000 7fe8c7400640 1 -- 172.21.15.60:0/996159275 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fe8c00634c0 con 0x7fe8d0073fb0 2024-09-16T15:40:43.337 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:43.337 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:43.337 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe8b0078a90 msgr2=0x7fe8b007af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe8b0078a90 0x7fe8b007af50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc007fd0 tx=0x7fe8bc009040 comp rx=0 tx=0).stop 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 msgr2=0x7fe8d0144d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 0x7fe8d0144d20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000c970 tx=0x7fe8c000ce30 comp rx=0 tx=0).stop 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 shutdown_connections 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe8b0078a90 0x7fe8b007af50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8d015f5a0 0x7fe8d015a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8d00ccf00 0x7fe8d015a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:43.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 --2- 172.21.15.60:0/996159275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0073fb0 0x7fe8d0144d20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:43.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 >> 172.21.15.60:0/996159275 conn(0x7fe8d00b1950 msgr2=0x7fe8d00af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:43.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 shutdown_connections 2024-09-16T15:40:43.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:43.337+0000 7fe8d5e00640 1 -- 172.21.15.60:0/996159275 wait complete. 2024-09-16T15:40:43.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:40:43.534 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.492881133999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.374691782103294,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:40:43.535 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.374691782103294 }, "state": "open", "uptime": 12.492881134 } ] 2024-09-16T15:40:43.535 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:40:48.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:48.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.707+0000 7fdfe6000640 1 -- 172.21.15.60:0/3223873257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfe015bc30 msgr2=0x7fdfe00b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.707+0000 7fdfe6000640 1 --2- 172.21.15.60:0/3223873257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfe015bc30 0x7fdfe00b28f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009830 tx=0x7fdfd402a510 comp rx=0 tx=0).stop 2024-09-16T15:40:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.707+0000 7fdfe6000640 1 -- 172.21.15.60:0/3223873257 shutdown_connections 2024-09-16T15:40:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.707+0000 7fdfe6000640 1 --2- 172.21.15.60:0/3223873257 >> v2:172.21.15.100:3300/0 conn(0x7fdfe00d0540 0x7fdfe006f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.707+0000 7fdfe6000640 1 --2- 172.21.15.60:0/3223873257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfe015bc30 0x7fdfe00b28f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.707+0000 7fdfe6000640 1 -- 172.21.15.60:0/3223873257 >> 172.21.15.60:0/3223873257 conn(0x7fdfe00cfa20 msgr2=0x7fdfe00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.707+0000 7fdfe6000640 1 -- 172.21.15.60:0/3223873257 shutdown_connections 2024-09-16T15:40:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.707+0000 7fdfe6000640 1 -- 172.21.15.60:0/3223873257 wait complete. 2024-09-16T15:40:48.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.709+0000 7fdfe6000640 1 Processor -- start 2024-09-16T15:40:48.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.709+0000 7fdfe6000640 1 -- start start 2024-09-16T15:40:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe00d0540 0x7fdfe00c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe6000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfe015bc30 0x7fdfe015e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfe015eeb0 0x7fdfe00b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe6000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0074660 con 0x7fdfe015eeb0 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe015c200 con 0x7fdfe00d0540 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe6000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe0153a80 con 0x7fdfe015bc30 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfe015eeb0 0x7fdfe00b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfe015eeb0 0x7fdfe00b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37144/0 (socket says 172.21.15.60:37144) 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe5600640 1 -- 172.21.15.60:0/2951901907 learned_addr learned my addr 172.21.15.60:0/2951901907 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfe4c00640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe00d0540 0x7fdfe00c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.710+0000 7fdfdfe00640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfe015bc30 0x7fdfe015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfe5600640 1 -- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfe015bc30 msgr2=0x7fdfe015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfe5600640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfe015bc30 0x7fdfe015e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfe5600640 1 -- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe00d0540 msgr2=0x7fdfe00c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfe5600640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe00d0540 0x7fdfe00c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfe5600640 1 -- 172.21.15.60:0/2951901907 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe0153d60 con 0x7fdfe015eeb0 2024-09-16T15:40:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfe5600640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfe015eeb0 0x7fdfe00b30c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800efc0 tx=0x7fdfd800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfdcc00640 1 -- 172.21.15.60:0/2951901907 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8003970 con 0x7fdfe015eeb0 2024-09-16T15:40:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfdcc00640 1 -- 172.21.15.60:0/2951901907 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd8003c20 con 0x7fdfe015eeb0 2024-09-16T15:40:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfdcc00640 1 -- 172.21.15.60:0/2951901907 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8004280 con 0x7fdfe015eeb0 2024-09-16T15:40:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe0154040 con 0x7fdfe015eeb0 2024-09-16T15:40:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.711+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe0154600 con 0x7fdfe015eeb0 2024-09-16T15:40:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.712+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe006fc40 con 0x7fdfe015eeb0 2024-09-16T15:40:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.714+0000 7fdfdcc00640 1 -- 172.21.15.60:0/2951901907 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7fdfd8004420 con 0x7fdfe015eeb0 2024-09-16T15:40:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.715+0000 7fdfdcc00640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdfc0078a90 0x7fdfc007af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.715+0000 7fdfdcc00640 1 -- 172.21.15.60:0/2951901907 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fdfd809b6b0 con 0x7fdfe015eeb0 2024-09-16T15:40:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.715+0000 7fdfe4c00640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdfc0078a90 0x7fdfc007af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:48.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.716+0000 7fdfe4c00640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdfc0078a90 0x7fdfc007af50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4034000 tx=0x7fdfd4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:48.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.719+0000 7fdfdcc00640 1 -- 172.21.15.60:0/2951901907 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdfd8063250 con 0x7fdfe015eeb0 2024-09-16T15:40:48.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.957+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfe01547d0 con 0x7fdfe015eeb0 2024-09-16T15:40:48.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.958+0000 7fdfdcc00640 1 -- 172.21.15.60:0/2951901907 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fdfd8067c80 con 0x7fdfe015eeb0 2024-09-16T15:40:48.961 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:48.961 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:48.961 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:40:48.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdfc0078a90 msgr2=0x7fdfc007af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:48.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdfc0078a90 0x7fdfc007af50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4034000 tx=0x7fdfd4004730 comp rx=0 tx=0).stop 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfe015eeb0 msgr2=0x7fdfe00b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfe015eeb0 0x7fdfe00b30c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800efc0 tx=0x7fdfd800c800 comp rx=0 tx=0).stop 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 shutdown_connections 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdfc0078a90 0x7fdfc007af50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfe015eeb0 0x7fdfe00b30c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfe015bc30 0x7fdfe015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 --2- 172.21.15.60:0/2951901907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfe00d0540 0x7fdfe00c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 >> 172.21.15.60:0/2951901907 conn(0x7fdfe00cfa20 msgr2=0x7fdfe0159100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 shutdown_connections 2024-09-16T15:40:48.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:48.961+0000 7fdfe6000640 1 -- 172.21.15.60:0/2951901907 wait complete. 2024-09-16T15:40:48.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:40:49.158 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":18.115827628000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.098792896291013,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:40:49.158 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.098792896291013 }, "state": "open", "uptime": 18.115827628 } ] 2024-09-16T15:40:49.158 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:40:54.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.329+0000 7f5ece200640 1 -- 172.21.15.60:0/3896871656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 msgr2=0x7f5ec80cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.329+0000 7f5ece200640 1 --2- 172.21.15.60:0/3896871656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 0x7f5ec80cc860 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8074110 tx=0x7f5ebc02a540 comp rx=0 tx=0).stop 2024-09-16T15:40:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.329+0000 7f5ece200640 1 -- 172.21.15.60:0/3896871656 shutdown_connections 2024-09-16T15:40:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.329+0000 7f5ece200640 1 --2- 172.21.15.60:0/3896871656 >> v2:172.21.15.100:3300/0 conn(0x7f5ec80afde0 0x7f5ec80b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.329+0000 7f5ece200640 1 --2- 172.21.15.60:0/3896871656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 0x7f5ec80cc860 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.329+0000 7f5ece200640 1 -- 172.21.15.60:0/3896871656 >> 172.21.15.60:0/3896871656 conn(0x7f5ec80b1e90 msgr2=0x7f5ec80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.329+0000 7f5ece200640 1 -- 172.21.15.60:0/3896871656 shutdown_connections 2024-09-16T15:40:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.329+0000 7f5ece200640 1 -- 172.21.15.60:0/3896871656 wait complete. 2024-09-16T15:40:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.330+0000 7f5ece200640 1 Processor -- start 2024-09-16T15:40:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.330+0000 7f5ece200640 1 -- start start 2024-09-16T15:40:54.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.331+0000 7f5ece200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec8074ca0 0x7f5ec81f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:54.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.331+0000 7f5ece200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80afde0 0x7f5ec80b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.331+0000 7f5ece200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 0x7f5ec808bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.331+0000 7f5ece200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec80b0cf0 con 0x7f5ec80ce100 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.331+0000 7f5ece200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec80b0b70 con 0x7f5ec80afde0 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.331+0000 7f5ece200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec80b0e70 con 0x7f5ec8074ca0 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecd800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 0x7f5ec808bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecd800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 0x7f5ec808bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33296/0 (socket says 172.21.15.60:33296) 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecd800640 1 -- 172.21.15.60:0/4118351203 learned_addr learned my addr 172.21.15.60:0/4118351203 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ec7e00640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80afde0 0x7f5ec80b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecce00640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec8074ca0 0x7f5ec81f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecd800640 1 -- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec8074ca0 msgr2=0x7f5ec81f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecd800640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec8074ca0 0x7f5ec81f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecd800640 1 -- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80afde0 msgr2=0x7f5ec80b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecd800640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80afde0 0x7f5ec80b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.332+0000 7f5ecd800640 1 -- 172.21.15.60:0/4118351203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec80b5140 con 0x7f5ec80ce100 2024-09-16T15:40:54.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.333+0000 7f5ecd800640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 0x7f5ec808bf90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000cce0 tx=0x7f5ec000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:54.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.333+0000 7f5ec4c00640 1 -- 172.21.15.60:0/4118351203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000f450 con 0x7f5ec80ce100 2024-09-16T15:40:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.333+0000 7f5ec4c00640 1 -- 172.21.15.60:0/4118351203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec000f700 con 0x7f5ec80ce100 2024-09-16T15:40:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.333+0000 7f5ec4c00640 1 -- 172.21.15.60:0/4118351203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000fb50 con 0x7f5ec80ce100 2024-09-16T15:40:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.333+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec81f86a0 con 0x7f5ec80ce100 2024-09-16T15:40:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.333+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec81f8c60 con 0x7f5ec80ce100 2024-09-16T15:40:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.334+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec80b0250 con 0x7f5ec80ce100 2024-09-16T15:40:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.335+0000 7f5ec4c00640 1 -- 172.21.15.60:0/4118351203 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f5ec000fcf0 con 0x7f5ec80ce100 2024-09-16T15:40:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.336+0000 7f5ec4c00640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5ea8078a90 0x7f5ea807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.336+0000 7f5ec4c00640 1 -- 172.21.15.60:0/4118351203 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f5ec009bbe0 con 0x7f5ec80ce100 2024-09-16T15:40:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.337+0000 7f5ecce00640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5ea8078a90 0x7f5ea807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.337+0000 7f5ecce00640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5ea8078a90 0x7f5ea807af50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc034000 tx=0x7f5ebc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.340+0000 7f5ec4c00640 1 -- 172.21.15.60:0/4118351203 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5ec0014030 con 0x7f5ec80ce100 2024-09-16T15:40:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.578+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ec80ca290 con 0x7f5ec80ce100 2024-09-16T15:40:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.578+0000 7f5ec4c00640 1 -- 172.21.15.60:0/4118351203 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f5ec00636f0 con 0x7f5ec80ce100 2024-09-16T15:40:54.581 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:40:54.581 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:40:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:40:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5ea8078a90 msgr2=0x7f5ea807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5ea8078a90 0x7f5ea807af50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc034000 tx=0x7f5ebc004730 comp rx=0 tx=0).stop 2024-09-16T15:40:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 msgr2=0x7f5ec808bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 0x7f5ec808bf90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000cce0 tx=0x7f5ec000a620 comp rx=0 tx=0).stop 2024-09-16T15:40:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 shutdown_connections 2024-09-16T15:40:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5ea8078a90 0x7f5ea807af50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80ce100 0x7f5ec808bf90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80afde0 0x7f5ec80b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 --2- 172.21.15.60:0/4118351203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec8074ca0 0x7f5ec81f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 >> 172.21.15.60:0/4118351203 conn(0x7f5ec80b1e90 msgr2=0x7f5ec8156020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 shutdown_connections 2024-09-16T15:40:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:54.581+0000 7f5ece200640 1 -- 172.21.15.60:0/4118351203 wait complete. 2024-09-16T15:40:54.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:40:54.777 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":23.735774155000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.826599597452191,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:40:54.778 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.82659959745219 }, "state": "open", "uptime": 23.735774155 } ] 2024-09-16T15:40:54.778 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:40:59.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 -- 172.21.15.60:0/521296170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 msgr2=0x7f162c15dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 --2- 172.21.15.60:0/521296170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 0x7f162c15dbf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f162c0b2560 tx=0x7f162402a6f0 comp rx=0 tx=0).stop 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 -- 172.21.15.60:0/521296170 shutdown_connections 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 --2- 172.21.15.60:0/521296170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 0x7f162c15dbf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 --2- 172.21.15.60:0/521296170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f162c0ccb30 0x7f162c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 --2- 172.21.15.60:0/521296170 >> v2:172.21.15.100:3300/0 conn(0x7f162c1438b0 0x7f162c0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 -- 172.21.15.60:0/521296170 >> 172.21.15.60:0/521296170 conn(0x7f162c0b1950 msgr2=0x7f162c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 -- 172.21.15.60:0/521296170 shutdown_connections 2024-09-16T15:40:59.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.951+0000 7f1630c00640 1 -- 172.21.15.60:0/521296170 wait complete. 2024-09-16T15:40:59.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.953+0000 7f1630c00640 1 Processor -- start 2024-09-16T15:40:59.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.953+0000 7f1630c00640 1 -- start start 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1630c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 0x7f162c09d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1630c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f162c0ccb30 0x7f162c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1630c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f162c1438b0 0x7f162c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1630c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f162c070aa0 con 0x7f162c06fd90 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1630c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f162c070920 con 0x7f162c0ccb30 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1630c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f162c15e130 con 0x7f162c1438b0 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1629e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 0x7f162c09d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1629e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 0x7f162c09d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56058/0 (socket says 172.21.15.60:56058) 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.954+0000 7f1629e00640 1 -- 172.21.15.60:0/1533003123 learned_addr learned my addr 172.21.15.60:0/1533003123 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:40:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f162a800640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f162c1438b0 0x7f162c0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1629e00640 1 -- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f162c1438b0 msgr2=0x7f162c0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1629e00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f162c1438b0 0x7f162c0b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1621400640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f162c0ccb30 0x7f162c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1629e00640 1 -- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f162c0ccb30 msgr2=0x7f162c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1629e00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f162c0ccb30 0x7f162c09aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1629e00640 1 -- 172.21.15.60:0/1533003123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c09af40 con 0x7f162c06fd90 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f162a800640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f162c1438b0 0x7f162c0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1621400640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f162c0ccb30 0x7f162c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1629e00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 0x7f162c09d4a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f161c00e790 tx=0x7f161c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1622a00640 1 -- 172.21.15.60:0/1533003123 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c00eea0 con 0x7f162c06fd90 2024-09-16T15:40:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1622a00640 1 -- 172.21.15.60:0/1533003123 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f161c00cdb0 con 0x7f162c06fd90 2024-09-16T15:40:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1f4f10 con 0x7f162c06fd90 2024-09-16T15:40:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1622a00640 1 -- 172.21.15.60:0/1533003123 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c010690 con 0x7f162c06fd90 2024-09-16T15:40:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.955+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1f54d0 con 0x7f162c06fd90 2024-09-16T15:40:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.957+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c06f850 con 0x7f162c06fd90 2024-09-16T15:40:59.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.958+0000 7f1622a00640 1 -- 172.21.15.60:0/1533003123 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f161c010830 con 0x7f162c06fd90 2024-09-16T15:40:59.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.958+0000 7f1622a00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1608078a90 0x7f160807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:59.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.959+0000 7f1622a00640 1 -- 172.21.15.60:0/1533003123 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f161c09b8f0 con 0x7f162c06fd90 2024-09-16T15:40:59.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.959+0000 7f1621400640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1608078a90 0x7f160807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:59.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.960+0000 7f1621400640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1608078a90 0x7f160807af50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1614005e00 tx=0x7f1614005c70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:59.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:40:59.963+0000 7f1622a00640 1 -- 172.21.15.60:0/1533003123 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f161c063a50 con 0x7f162c06fd90 2024-09-16T15:41:00.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.200+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162c0cc470 con 0x7f162c06fd90 2024-09-16T15:41:00.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.200+0000 7f1622a00640 1 -- 172.21.15.60:0/1533003123 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f161c063350 con 0x7f162c06fd90 2024-09-16T15:41:00.203 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:00.203 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:00.203 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:00.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1608078a90 msgr2=0x7f160807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1608078a90 0x7f160807af50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1614005e00 tx=0x7f1614005c70 comp rx=0 tx=0).stop 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 msgr2=0x7f162c09d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 0x7f162c09d4a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f161c00e790 tx=0x7f161c00ec50 comp rx=0 tx=0).stop 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 shutdown_connections 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1608078a90 0x7f160807af50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f162c1438b0 0x7f162c0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f162c0ccb30 0x7f162c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 --2- 172.21.15.60:0/1533003123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f162c06fd90 0x7f162c09d4a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 >> 172.21.15.60:0/1533003123 conn(0x7f162c0b1950 msgr2=0x7f162c0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 shutdown_connections 2024-09-16T15:41:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:00.203+0000 7f1630c00640 1 -- 172.21.15.60:0/1533003123 wait complete. 2024-09-16T15:41:00.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:00.399 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":29.356720673000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.55787033722082,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:41:00.399 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.55787033722082 }, "state": "open", "uptime": 29.356720673 } ] 2024-09-16T15:41:00.399 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:41:00.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:41:00.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:41:05.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.571+0000 7f25cbe00640 1 -- 172.21.15.60:0/453031558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25cc0725d0 msgr2=0x7f25cc073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.571+0000 7f25cbe00640 1 --2- 172.21.15.60:0/453031558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25cc0725d0 0x7f25cc073d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25b8005fd0 tx=0x7f25b802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.571+0000 7f25cbe00640 1 -- 172.21.15.60:0/453031558 shutdown_connections 2024-09-16T15:41:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.571+0000 7f25cbe00640 1 --2- 172.21.15.60:0/453031558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25cc0725d0 0x7f25cc073d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.571+0000 7f25cbe00640 1 --2- 172.21.15.60:0/453031558 >> v2:172.21.15.100:3300/0 conn(0x7f25cc15bc30 0x7f25cc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.571+0000 7f25cbe00640 1 -- 172.21.15.60:0/453031558 >> 172.21.15.60:0/453031558 conn(0x7f25cc0cfa20 msgr2=0x7f25cc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.571+0000 7f25cbe00640 1 -- 172.21.15.60:0/453031558 shutdown_connections 2024-09-16T15:41:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.571+0000 7f25cbe00640 1 -- 172.21.15.60:0/453031558 wait complete. 2024-09-16T15:41:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.573+0000 7f25cbe00640 1 Processor -- start 2024-09-16T15:41:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.573+0000 7f25cbe00640 1 -- start start 2024-09-16T15:41:05.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc0d0540 0x7f25cc0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25cc15bc30 0x7f25cc1f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25cc14e1f0 0x7f25cc157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25cbe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc074610 con 0x7f25cc0d0540 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25cbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc074490 con 0x7f25cc15bc30 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25cbe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f25cc074790 con 0x7f25cc14e1f0 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25caa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc0d0540 0x7f25cc0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25ca000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25cc15bc30 0x7f25cc1f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25caa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc0d0540 0x7f25cc0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56070/0 (socket says 172.21.15.60:56070) 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25caa00640 1 -- 172.21.15.60:0/1341495140 learned_addr learned my addr 172.21.15.60:0/1341495140 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.574+0000 7f25cb400640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25cc14e1f0 0x7f25cc157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25caa00640 1 -- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25cc14e1f0 msgr2=0x7f25cc157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25caa00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25cc14e1f0 0x7f25cc157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25caa00640 1 -- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25cc15bc30 msgr2=0x7f25cc1f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25caa00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25cc15bc30 0x7f25cc1f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25caa00640 1 -- 172.21.15.60:0/1341495140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc0cd9d0 con 0x7f25cc0d0540 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25caa00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc0d0540 0x7f25cc0cf280 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00e9f0 tx=0x7f25bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25c3400640 1 -- 172.21.15.60:0/1341495140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00cd80 con 0x7f25cc0d0540 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25c3400640 1 -- 172.21.15.60:0/1341495140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25bc004020 con 0x7f25cc0d0540 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25c3400640 1 -- 172.21.15.60:0/1341495140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc004470 con 0x7f25cc0d0540 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25cc0cdcb0 con 0x7f25cc0d0540 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.575+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25cc0ce270 con 0x7f25cc0d0540 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.576+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25cc06fc60 con 0x7f25cc0d0540 2024-09-16T15:41:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.578+0000 7f25c3400640 1 -- 172.21.15.60:0/1341495140 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f25bc0035f0 con 0x7f25cc0d0540 2024-09-16T15:41:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.578+0000 7f25c3400640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25ac078a90 0x7f25ac07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.579+0000 7f25c3400640 1 -- 172.21.15.60:0/1341495140 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f25bc09b8e0 con 0x7f25cc0d0540 2024-09-16T15:41:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.579+0000 7f25ca000640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25ac078a90 0x7f25ac07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.580+0000 7f25ca000640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25ac078a90 0x7f25ac07af50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f25b8033000 tx=0x7f25b8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.583+0000 7f25c3400640 1 -- 172.21.15.60:0/1341495140 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f25bc0633f0 con 0x7f25cc0d0540 2024-09-16T15:41:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.817+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25cc15ace0 con 0x7f25cc0d0540 2024-09-16T15:41:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.818+0000 7f25c3400640 1 -- 172.21.15.60:0/1341495140 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f25bc067e20 con 0x7f25cc0d0540 2024-09-16T15:41:05.820 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:05.820 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25ac078a90 msgr2=0x7f25ac07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25ac078a90 0x7f25ac07af50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f25b8033000 tx=0x7f25b8004730 comp rx=0 tx=0).stop 2024-09-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc0d0540 msgr2=0x7f25cc0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc0d0540 0x7f25cc0cf280 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00e9f0 tx=0x7f25bc00eeb0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 shutdown_connections 2024-09-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25ac078a90 0x7f25ac07af50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25cc14e1f0 0x7f25cc157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25cc15bc30 0x7f25cc1f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 --2- 172.21.15.60:0/1341495140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25cc0d0540 0x7f25cc0cf280 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 >> 172.21.15.60:0/1341495140 conn(0x7f25cc0cfa20 msgr2=0x7f25cc0b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 shutdown_connections 2024-09-16T15:41:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:05.820+0000 7f25cbe00640 1 -- 172.21.15.60:0/1341495140 wait complete. 2024-09-16T15:41:05.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:06.019 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.976667200000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.292655423187863,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:41:06.019 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.292655423187863 }, "state": "open", "uptime": 34.9766672 } ] 2024-09-16T15:41:06.019 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:41:11.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:11.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.190+0000 7ff27b600640 1 -- 172.21.15.60:0/798794516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258004d40 msgr2=0x7ff2580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:11.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.190+0000 7ff27b600640 1 --2- 172.21.15.60:0/798794516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258004d40 0x7ff2580051c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff26c001ed0 tx=0x7ff258005830 comp rx=0 tx=0).stop 2024-09-16T15:41:11.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.190+0000 7ff279600640 1 -- 172.21.15.60:0/798794516 reap_dead start 2024-09-16T15:41:11.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.190+0000 7ff27b600640 1 -- 172.21.15.60:0/798794516 shutdown_connections 2024-09-16T15:41:11.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.190+0000 7ff27b600640 1 -- 172.21.15.60:0/798794516 >> 172.21.15.60:0/798794516 conn(0x7ff2740b1950 msgr2=0x7ff2740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:11.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.190+0000 7ff27b600640 1 -- 172.21.15.60:0/798794516 shutdown_connections 2024-09-16T15:41:11.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.190+0000 7ff27b600640 1 -- 172.21.15.60:0/798794516 wait complete. 2024-09-16T15:41:11.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.192+0000 7ff27b600640 1 Processor -- start 2024-09-16T15:41:11.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.192+0000 7ff27b600640 1 -- start start 2024-09-16T15:41:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff27b600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258004d40 0x7ff274154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff27b600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2740cb690 0x7ff2740cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff27b600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2740b0b70 0x7ff2740b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff27b600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff27406f9f0 con 0x7ff2740b0b70 2024-09-16T15:41:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff27b600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff27406f870 con 0x7ff258004d40 2024-09-16T15:41:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff27b600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff2740b2440 con 0x7ff2740cb690 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff279600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2740b0b70 0x7ff2740b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff279600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2740b0b70 0x7ff2740b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36646/0 (socket says 172.21.15.60:36646) 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.193+0000 7ff279600640 1 -- 172.21.15.60:0/4049025924 learned_addr learned my addr 172.21.15.60:0/4049025924 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff278c00640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258004d40 0x7ff274154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff26be00640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2740cb690 0x7ff2740cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff279600640 1 -- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2740cb690 msgr2=0x7ff2740cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff279600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2740cb690 0x7ff2740cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff279600640 1 -- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258004d40 msgr2=0x7ff274154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff279600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258004d40 0x7ff274154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff279600640 1 -- 172.21.15.60:0/4049025924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740cbd90 con 0x7ff2740b0b70 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff278c00640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258004d40 0x7ff274154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff26be00640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2740cb690 0x7ff2740cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:11.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff279600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2740b0b70 0x7ff2740b0fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff26c001ff0 tx=0x7ff26c006880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:11.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff271600640 1 -- 172.21.15.60:0/4049025924 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c006c40 con 0x7ff2740b0b70 2024-09-16T15:41:11.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff271600640 1 -- 172.21.15.60:0/4049025924 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff26c003d40 con 0x7ff2740b0b70 2024-09-16T15:41:11.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff271600640 1 -- 172.21.15.60:0/4049025924 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0346c0 con 0x7ff2740b0b70 2024-09-16T15:41:11.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.194+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2741f7f40 con 0x7ff2740b0b70 2024-09-16T15:41:11.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.195+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2740cbab0 con 0x7ff2740b0b70 2024-09-16T15:41:11.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.196+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff274074460 con 0x7ff2740b0b70 2024-09-16T15:41:11.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.197+0000 7ff271600640 1 -- 172.21.15.60:0/4049025924 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7ff26c034a70 con 0x7ff2740b0b70 2024-09-16T15:41:11.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.198+0000 7ff271600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff250078ac0 0x7ff25007af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:11.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.198+0000 7ff271600640 1 -- 172.21.15.60:0/4049025924 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7ff26c0ba7f0 con 0x7ff2740b0b70 2024-09-16T15:41:11.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.198+0000 7ff278c00640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff250078ac0 0x7ff25007af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:11.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.199+0000 7ff278c00640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff250078ac0 0x7ff25007af80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff264009460 tx=0x7ff264005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:11.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.202+0000 7ff271600640 1 -- 172.21.15.60:0/4049025924 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff26c046070 con 0x7ff2740b0b70 2024-09-16T15:41:11.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.436+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2740b0650 con 0x7ff2740b0b70 2024-09-16T15:41:11.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.436+0000 7ff271600640 1 -- 172.21.15.60:0/4049025924 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7ff26c082950 con 0x7ff2740b0b70 2024-09-16T15:41:11.438 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:11.439 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:11.439 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.438+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff250078ac0 msgr2=0x7ff25007af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.438+0000 7ff27b600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff250078ac0 0x7ff25007af80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff264009460 tx=0x7ff264005f90 comp rx=0 tx=0).stop 2024-09-16T15:41:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.438+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2740b0b70 msgr2=0x7ff2740b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.438+0000 7ff27b600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2740b0b70 0x7ff2740b0fb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff26c001ff0 tx=0x7ff26c006880 comp rx=0 tx=0).stop 2024-09-16T15:41:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.439+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 shutdown_connections 2024-09-16T15:41:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.439+0000 7ff27b600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff250078ac0 0x7ff25007af80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.439+0000 7ff27b600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2740b0b70 0x7ff2740b0fb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.439+0000 7ff27b600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2740cb690 0x7ff2740cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:11.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.439+0000 7ff27b600640 1 --2- 172.21.15.60:0/4049025924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff258004d40 0x7ff274154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:11.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.439+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 >> 172.21.15.60:0/4049025924 conn(0x7ff2740b1950 msgr2=0x7ff274075ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:11.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.439+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 shutdown_connections 2024-09-16T15:41:11.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:11.439+0000 7ff27b600640 1 -- 172.21.15.60:0/4049025924 wait complete. 2024-09-16T15:41:11.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:11.636 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":40.592613620999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.03104714898565,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:41:11.637 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.03104714898565 }, "state": "open", "uptime": 40.592613621 } ] 2024-09-16T15:41:11.637 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:41:16.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.810+0000 7f0f7b200640 1 -- 172.21.15.60:0/4216008974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ccf00 msgr2=0x7f0f740acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.810+0000 7f0f7b200640 1 --2- 172.21.15.60:0/4216008974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ccf00 0x7f0f740acd30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005fd0 tx=0x7f0f6002a540 comp rx=0 tx=0).stop 2024-09-16T15:41:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.810+0000 7f0f7b200640 1 -- 172.21.15.60:0/4216008974 shutdown_connections 2024-09-16T15:41:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.810+0000 7f0f7b200640 1 --2- 172.21.15.60:0/4216008974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ccf00 0x7f0f740acd30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.810+0000 7f0f7b200640 1 --2- 172.21.15.60:0/4216008974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f7415f5a0 0x7f0f740b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.810+0000 7f0f7b200640 1 -- 172.21.15.60:0/4216008974 >> 172.21.15.60:0/4216008974 conn(0x7f0f740b1950 msgr2=0x7f0f740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.810+0000 7f0f7b200640 1 -- 172.21.15.60:0/4216008974 shutdown_connections 2024-09-16T15:41:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.810+0000 7f0f7b200640 1 -- 172.21.15.60:0/4216008974 wait complete. 2024-09-16T15:41:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.812+0000 7f0f7b200640 1 Processor -- start 2024-09-16T15:41:16.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.812+0000 7f0f7b200640 1 -- start start 2024-09-16T15:41:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f7b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ad270 0x7f0f740b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f7b200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f740ccf00 0x7f0f7414ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f7b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f7415f5a0 0x7f0f7414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f7b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7406f2e0 con 0x7f0f740ad270 2024-09-16T15:41:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f7b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7406f160 con 0x7f0f7415f5a0 2024-09-16T15:41:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f7b200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7406f460 con 0x7f0f740ccf00 2024-09-16T15:41:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f73e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ad270 0x7f0f740b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f73e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ad270 0x7f0f740b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36662/0 (socket says 172.21.15.60:36662) 2024-09-16T15:41:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f73e00640 1 -- 172.21.15.60:0/282563513 learned_addr learned my addr 172.21.15.60:0/282563513 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f73400640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f740ccf00 0x7f0f7414ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.813+0000 7f0f79200640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f7415f5a0 0x7f0f7414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f73e00640 1 -- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f740ccf00 msgr2=0x7f0f7414ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f73e00640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f740ccf00 0x7f0f7414ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f73e00640 1 -- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f7415f5a0 msgr2=0x7f0f7414bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f73e00640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f7415f5a0 0x7f0f7414bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f73e00640 1 -- 172.21.15.60:0/282563513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f740b44c0 con 0x7f0f740ad270 2024-09-16T15:41:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f73e00640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ad270 0x7f0f740b1fa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400d6d0 tx=0x7f0f6400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f6be00640 1 -- 172.21.15.60:0/282563513 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64017070 con 0x7f0f740ad270 2024-09-16T15:41:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f6be00640 1 -- 172.21.15.60:0/282563513 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f64005330 con 0x7f0f740ad270 2024-09-16T15:41:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f741f5280 con 0x7f0f740ad270 2024-09-16T15:41:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f6be00640 1 -- 172.21.15.60:0/282563513 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64009820 con 0x7f0f740ad270 2024-09-16T15:41:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.814+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f741f5840 con 0x7f0f740ad270 2024-09-16T15:41:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.815+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7406f980 con 0x7f0f740ad270 2024-09-16T15:41:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.816+0000 7f0f6be00640 1 -- 172.21.15.60:0/282563513 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f0f64009bd0 con 0x7f0f740ad270 2024-09-16T15:41:16.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.817+0000 7f0f6be00640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f54078a90 0x7f0f5407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.817+0000 7f0f6be00640 1 -- 172.21.15.60:0/282563513 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f0f6409b100 con 0x7f0f740ad270 2024-09-16T15:41:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.818+0000 7f0f73400640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f54078a90 0x7f0f5407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.818+0000 7f0f73400640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f54078a90 0x7f0f5407af50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0f60033000 tx=0x7f0f60004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:16.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:16.822+0000 7f0f6be00640 1 -- 172.21.15.60:0/282563513 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0f64062c60 con 0x7f0f740ad270 2024-09-16T15:41:17.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.058+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f74154430 con 0x7f0f740ad270 2024-09-16T15:41:17.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.059+0000 7f0f6be00640 1 -- 172.21.15.60:0/282563513 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f0f64067690 con 0x7f0f740ad270 2024-09-16T15:41:17.061 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:17.061 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:17.062 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.061+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f54078a90 msgr2=0x7f0f5407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.061+0000 7f0f7b200640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f54078a90 0x7f0f5407af50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0f60033000 tx=0x7f0f60004730 comp rx=0 tx=0).stop 2024-09-16T15:41:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.061+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ad270 msgr2=0x7f0f740b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.061+0000 7f0f7b200640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ad270 0x7f0f740b1fa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400d6d0 tx=0x7f0f6400db90 comp rx=0 tx=0).stop 2024-09-16T15:41:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.062+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 shutdown_connections 2024-09-16T15:41:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.062+0000 7f0f7b200640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f54078a90 0x7f0f5407af50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.062+0000 7f0f7b200640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f7415f5a0 0x7f0f7414bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.062+0000 7f0f7b200640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f740ccf00 0x7f0f7414ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.062+0000 7f0f7b200640 1 --2- 172.21.15.60:0/282563513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f740ad270 0x7f0f740b1fa0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.062+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 >> 172.21.15.60:0/282563513 conn(0x7f0f740b1950 msgr2=0x7f0f74073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.062+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 shutdown_connections 2024-09-16T15:41:17.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:17.062+0000 7f0f7b200640 1 -- 172.21.15.60:0/282563513 wait complete. 2024-09-16T15:41:17.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:17.259 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":46.216559779999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.772446005029277,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:41:17.259 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.772446005029277 }, "state": "open", "uptime": 46.21655978 } ] 2024-09-16T15:41:17.259 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:41:22.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:22.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.431+0000 7f7ee3800640 1 -- 172.21.15.60:0/974880910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 msgr2=0x7f7edc0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:22.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.431+0000 7f7ee3800640 1 --2- 172.21.15.60:0/974880910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 0x7f7edc0acd30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7edc075130 tx=0x7f7ec802a580 comp rx=0 tx=0).stop 2024-09-16T15:41:22.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.431+0000 7f7ee3800640 1 -- 172.21.15.60:0/974880910 shutdown_connections 2024-09-16T15:41:22.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.431+0000 7f7ee3800640 1 --2- 172.21.15.60:0/974880910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 0x7f7edc0acd30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.431+0000 7f7ee3800640 1 --2- 172.21.15.60:0/974880910 >> v2:172.21.15.100:3300/0 conn(0x7f7edc15f5a0 0x7f7edc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.431+0000 7f7ee3800640 1 -- 172.21.15.60:0/974880910 >> 172.21.15.60:0/974880910 conn(0x7f7edc0b1950 msgr2=0x7f7edc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:22.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.431+0000 7f7ee3800640 1 -- 172.21.15.60:0/974880910 shutdown_connections 2024-09-16T15:41:22.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.431+0000 7f7ee3800640 1 -- 172.21.15.60:0/974880910 wait complete. 2024-09-16T15:41:22.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.433+0000 7f7ee3800640 1 Processor -- start 2024-09-16T15:41:22.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.433+0000 7f7ee3800640 1 -- start start 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7ee3800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7edc0ad270 0x7f7edc1f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7ee3800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 0x7f7edc0ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7ee3800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7edc15f5a0 0x7f7edc1f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7ee3800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc06f160 con 0x7f7edc0ccf00 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7ee3800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc06f2e0 con 0x7f7edc15f5a0 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7ee3800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc06f460 con 0x7f7edc0ad270 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7edbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 0x7f7edc0ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7edbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 0x7f7edc0ced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32938/0 (socket says 172.21.15.60:32938) 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7edbe00640 1 -- 172.21.15.60:0/3637395647 learned_addr learned my addr 172.21.15.60:0/3637395647 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7ee0e00640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7edc0ad270 0x7f7edc1f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.434+0000 7f7ee1800640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7edc15f5a0 0x7f7edc1f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:22.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7edbe00640 1 -- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7edc0ad270 msgr2=0x7f7edc1f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:22.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7edbe00640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7edc0ad270 0x7f7edc1f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7edbe00640 1 -- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7edc15f5a0 msgr2=0x7f7edc1f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:22.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7edbe00640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7edc15f5a0 0x7f7edc1f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7edbe00640 1 -- 172.21.15.60:0/3637395647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc0b3d10 con 0x7f7edc0ccf00 2024-09-16T15:41:22.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7ee1800640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7edc15f5a0 0x7f7edc1f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:22.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7edbe00640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 0x7f7edc0ced90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009850 tx=0x7f7ec8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:22.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7ed8c00640 1 -- 172.21.15.60:0/3637395647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec803f070 con 0x7f7edc0ccf00 2024-09-16T15:41:22.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7ed8c00640 1 -- 172.21.15.60:0/3637395647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ec80047a0 con 0x7f7edc0ccf00 2024-09-16T15:41:22.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc0cf400 con 0x7f7edc0ccf00 2024-09-16T15:41:22.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7ed8c00640 1 -- 172.21.15.60:0/3637395647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8035030 con 0x7f7edc0ccf00 2024-09-16T15:41:22.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.435+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc0cf9c0 con 0x7f7edc0ccf00 2024-09-16T15:41:22.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.436+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7edc06f9f0 con 0x7f7edc0ccf00 2024-09-16T15:41:22.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.439+0000 7f7ed8c00640 1 -- 172.21.15.60:0/3637395647 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f7ec8042070 con 0x7f7edc0ccf00 2024-09-16T15:41:22.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.440+0000 7f7ed8c00640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7ebc078ac0 0x7f7ebc07af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:22.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.440+0000 7f7ed8c00640 1 -- 172.21.15.60:0/3637395647 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f7ec80bc0c0 con 0x7f7edc0ccf00 2024-09-16T15:41:22.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.440+0000 7f7ee0e00640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7ebc078ac0 0x7f7ebc07af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:22.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.441+0000 7f7ee0e00640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7ebc078ac0 0x7f7ebc07af80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0b03c0 tx=0x7f7ed000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:22.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.443+0000 7f7ed8c00640 1 -- 172.21.15.60:0/3637395647 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7ec8083d70 con 0x7f7edc0ccf00 2024-09-16T15:41:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.677+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7edc0b0110 con 0x7f7edc0ccf00 2024-09-16T15:41:22.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.677+0000 7f7ed8c00640 1 -- 172.21.15.60:0/3637395647 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f7ec80887a0 con 0x7f7edc0ccf00 2024-09-16T15:41:22.678 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:22.678 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:22.679 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7ebc078ac0 msgr2=0x7f7ebc07af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7ebc078ac0 0x7f7ebc07af80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0b03c0 tx=0x7f7ed000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 msgr2=0x7f7edc0ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 0x7f7edc0ced90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009850 tx=0x7f7ec8003910 comp rx=0 tx=0).stop 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 shutdown_connections 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7ebc078ac0 0x7f7ebc07af80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7edc15f5a0 0x7f7edc1f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7edc0ccf00 0x7f7edc0ced90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 --2- 172.21.15.60:0/3637395647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7edc0ad270 0x7f7edc1f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.679+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 >> 172.21.15.60:0/3637395647 conn(0x7f7edc0b1950 msgr2=0x7f7edc09cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.680+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 shutdown_connections 2024-09-16T15:41:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:22.680+0000 7f7ee3800640 1 -- 172.21.15.60:0/3637395647 wait complete. 2024-09-16T15:41:22.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:22.876 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":51.835505988000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.517408876477447,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:41:22.877 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.517408876477447 }, "state": "open", "uptime": 51.835505988 } ] 2024-09-16T15:41:22.877 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:41:27.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.051+0000 7ff013400640 1 -- 172.21.15.60:0/3484469573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c15f5a0 msgr2=0x7ff00c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.051+0000 7ff013400640 1 --2- 172.21.15.60:0/3484469573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c15f5a0 0x7ff00c0b2440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff000009830 tx=0x7ff00002a490 comp rx=0 tx=0).stop 2024-09-16T15:41:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.051+0000 7ff013400640 1 -- 172.21.15.60:0/3484469573 shutdown_connections 2024-09-16T15:41:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.051+0000 7ff013400640 1 --2- 172.21.15.60:0/3484469573 >> v2:172.21.15.100:3300/0 conn(0x7ff00c073fb0 0x7ff00c074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.051+0000 7ff013400640 1 --2- 172.21.15.60:0/3484469573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c15f5a0 0x7ff00c0b2440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.051+0000 7ff013400640 1 -- 172.21.15.60:0/3484469573 >> 172.21.15.60:0/3484469573 conn(0x7ff00c0b1950 msgr2=0x7ff00c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.051+0000 7ff013400640 1 -- 172.21.15.60:0/3484469573 shutdown_connections 2024-09-16T15:41:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.051+0000 7ff013400640 1 -- 172.21.15.60:0/3484469573 wait complete. 2024-09-16T15:41:28.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.053+0000 7ff013400640 1 Processor -- start 2024-09-16T15:41:28.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.053+0000 7ff013400640 1 -- start start 2024-09-16T15:41:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.054+0000 7ff013400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00c073fb0 0x7ff00c1f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.054+0000 7ff013400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c0ccf00 0x7ff00c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.054+0000 7ff013400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff00c15f5a0 0x7ff00c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.054+0000 7ff013400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c06f2e0 con 0x7ff00c15f5a0 2024-09-16T15:41:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.054+0000 7ff013400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c06f160 con 0x7ff00c073fb0 2024-09-16T15:41:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.054+0000 7ff013400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff00c06f460 con 0x7ff00c0ccf00 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.054+0000 7ff011400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff00c15f5a0 0x7ff00c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff011400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff00c15f5a0 0x7ff00c15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32942/0 (socket says 172.21.15.60:32942) 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff011400640 1 -- 172.21.15.60:0/548356883 learned_addr learned my addr 172.21.15.60:0/548356883 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff010a00640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00c073fb0 0x7ff00c1f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff00be00640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c0ccf00 0x7ff00c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff011400640 1 -- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c0ccf00 msgr2=0x7ff00c0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff011400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c0ccf00 0x7ff00c0ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff011400640 1 -- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00c073fb0 msgr2=0x7ff00c1f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff011400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00c073fb0 0x7ff00c1f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff011400640 1 -- 172.21.15.60:0/548356883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c0ca990 con 0x7ff00c15f5a0 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff00be00640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c0ccf00 0x7ff00c0ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff010a00640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00c073fb0 0x7ff00c1f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:28.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.055+0000 7ff011400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff00c15f5a0 0x7ff00c15e640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff00400efc0 tx=0x7ff00400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:28.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.056+0000 7ff008c00640 1 -- 172.21.15.60:0/548356883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004003970 con 0x7ff00c15f5a0 2024-09-16T15:41:28.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.056+0000 7ff008c00640 1 -- 172.21.15.60:0/548356883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff004003c20 con 0x7ff00c15f5a0 2024-09-16T15:41:28.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.056+0000 7ff008c00640 1 -- 172.21.15.60:0/548356883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004004280 con 0x7ff00c15f5a0 2024-09-16T15:41:28.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.056+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c0cac70 con 0x7ff00c15f5a0 2024-09-16T15:41:28.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.056+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c0cb230 con 0x7ff00c15f5a0 2024-09-16T15:41:28.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.057+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c06f9f0 con 0x7ff00c15f5a0 2024-09-16T15:41:28.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.058+0000 7ff008c00640 1 -- 172.21.15.60:0/548356883 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7ff004004420 con 0x7ff00c15f5a0 2024-09-16T15:41:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.059+0000 7ff008c00640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fefec078a90 0x7fefec07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.059+0000 7ff008c00640 1 -- 172.21.15.60:0/548356883 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7ff00409b810 con 0x7ff00c15f5a0 2024-09-16T15:41:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.059+0000 7ff010a00640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fefec078a90 0x7fefec07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.060+0000 7ff010a00640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fefec078a90 0x7fefec07af50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff000034000 tx=0x7ff000004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:28.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.063+0000 7ff008c00640 1 -- 172.21.15.60:0/548356883 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff0040639c0 con 0x7ff00c15f5a0 2024-09-16T15:41:28.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.295+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff00c15eb80 con 0x7ff00c15f5a0 2024-09-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.296+0000 7ff008c00640 1 -- 172.21.15.60:0/548356883 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7ff0040632c0 con 0x7ff00c15f5a0 2024-09-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:28.297 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:28.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fefec078a90 msgr2=0x7fefec07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fefec078a90 0x7fefec07af50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff000034000 tx=0x7ff000004730 comp rx=0 tx=0).stop 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff00c15f5a0 msgr2=0x7ff00c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff00c15f5a0 0x7ff00c15e640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff00400efc0 tx=0x7ff00400c800 comp rx=0 tx=0).stop 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 shutdown_connections 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fefec078a90 0x7fefec07af50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff00c15f5a0 0x7ff00c15e640 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff00c0ccf00 0x7ff00c0ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 --2- 172.21.15.60:0/548356883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff00c073fb0 0x7ff00c1f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 >> 172.21.15.60:0/548356883 conn(0x7ff00c0b1950 msgr2=0x7ff00c09d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 shutdown_connections 2024-09-16T15:41:28.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:28.298+0000 7ff013400640 1 -- 172.21.15.60:0/548356883 wait complete. 2024-09-16T15:41:28.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:28.492 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":57.451452224999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.265794911701136,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:41:28.493 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.265794911701136 }, "state": "open", "uptime": 57.451452225 } ] 2024-09-16T15:41:28.493 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:41:30.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:41:30.554 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:41:33.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:33.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.667+0000 7f79e2000640 1 -- 172.21.15.60:0/2939454695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc073fb0 msgr2=0x7f79dc074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:33.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.667+0000 7f79e2000640 1 --2- 172.21.15.60:0/2939454695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc073fb0 0x7f79dc074440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f79d4005fd0 tx=0x7f79d402aa10 comp rx=0 tx=0).stop 2024-09-16T15:41:33.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.667+0000 7f79e2000640 1 -- 172.21.15.60:0/2939454695 shutdown_connections 2024-09-16T15:41:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.667+0000 7f79e2000640 1 --2- 172.21.15.60:0/2939454695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc073fb0 0x7f79dc074440 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.667+0000 7f79e2000640 1 --2- 172.21.15.60:0/2939454695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79dc0ccf00 0x7f79dc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.667+0000 7f79e2000640 1 -- 172.21.15.60:0/2939454695 >> 172.21.15.60:0/2939454695 conn(0x7f79dc0b1950 msgr2=0x7f79dc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.667+0000 7f79e2000640 1 -- 172.21.15.60:0/2939454695 shutdown_connections 2024-09-16T15:41:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.667+0000 7f79e2000640 1 -- 172.21.15.60:0/2939454695 wait complete. 2024-09-16T15:41:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.669+0000 7f79e2000640 1 Processor -- start 2024-09-16T15:41:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.669+0000 7f79e2000640 1 -- start start 2024-09-16T15:41:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79e2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79dc073fb0 0x7f79dc148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79e2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc0ccf00 0x7f79dc0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79e2000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79dc15f5a0 0x7f79dc1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79e2000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc06f2e0 con 0x7f79dc0ccf00 2024-09-16T15:41:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79e2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc06f160 con 0x7f79dc073fb0 2024-09-16T15:41:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79e2000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f79dc06f460 con 0x7f79dc15f5a0 2024-09-16T15:41:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79d3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc0ccf00 0x7f79dc0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79d3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc0ccf00 0x7f79dc0cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54026/0 (socket says 172.21.15.60:54026) 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79d3e00640 1 -- 172.21.15.60:0/2823304528 learned_addr learned my addr 172.21.15.60:0/2823304528 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79db400640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79dc073fb0 0x7f79dc148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79dbe00640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79dc15f5a0 0x7f79dc1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79d3e00640 1 -- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79dc15f5a0 msgr2=0x7f79dc1f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79d3e00640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79dc15f5a0 0x7f79dc1f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79d3e00640 1 -- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79dc073fb0 msgr2=0x7f79dc148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79d3e00640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79dc073fb0 0x7f79dc148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.670+0000 7f79d3e00640 1 -- 172.21.15.60:0/2823304528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc14f920 con 0x7f79dc0ccf00 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.671+0000 7f79d3e00640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc0ccf00 0x7f79dc0cfd60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f79c400ea20 tx=0x7f79c400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.671+0000 7f79d3400640 1 -- 172.21.15.60:0/2823304528 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c400cd80 con 0x7f79dc0ccf00 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.671+0000 7f79d3400640 1 -- 172.21.15.60:0/2823304528 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79c4010430 con 0x7f79dc0ccf00 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.671+0000 7f79d3400640 1 -- 172.21.15.60:0/2823304528 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4010840 con 0x7f79dc0ccf00 2024-09-16T15:41:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.671+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79dc14fc00 con 0x7f79dc0ccf00 2024-09-16T15:41:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.671+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79dc1501c0 con 0x7f79dc0ccf00 2024-09-16T15:41:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.672+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79dc06f9f0 con 0x7f79dc0ccf00 2024-09-16T15:41:33.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.673+0000 7f79d3400640 1 -- 172.21.15.60:0/2823304528 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f79c40109e0 con 0x7f79dc0ccf00 2024-09-16T15:41:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.674+0000 7f79d3400640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f79b8078a90 0x7f79b807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.674+0000 7f79d3400640 1 -- 172.21.15.60:0/2823304528 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f79c409b9b0 con 0x7f79dc0ccf00 2024-09-16T15:41:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.675+0000 7f79db400640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f79b8078a90 0x7f79b807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.675+0000 7f79db400640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f79b8078a90 0x7f79b807af50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f79cc001720 tx=0x7f79cc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:33.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.679+0000 7f79d3400640 1 -- 172.21.15.60:0/2823304528 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f79c4063480 con 0x7f79dc0ccf00 2024-09-16T15:41:33.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.910+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79dc0b0870 con 0x7f79dc0ccf00 2024-09-16T15:41:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.911+0000 7f79d3400640 1 -- 172.21.15.60:0/2823304528 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f79c4067eb0 con 0x7f79dc0ccf00 2024-09-16T15:41:33.912 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:33.912 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:33.912 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.913+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f79b8078a90 msgr2=0x7f79b807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.913+0000 7f79e2000640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f79b8078a90 0x7f79b807af50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f79cc001720 tx=0x7f79cc009040 comp rx=0 tx=0).stop 2024-09-16T15:41:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.913+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc0ccf00 msgr2=0x7f79dc0cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.913+0000 7f79e2000640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc0ccf00 0x7f79dc0cfd60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f79c400ea20 tx=0x7f79c400eee0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.913+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 shutdown_connections 2024-09-16T15:41:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.913+0000 7f79e2000640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f79b8078a90 0x7f79b807af50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.913+0000 7f79e2000640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79dc15f5a0 0x7f79dc1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.913+0000 7f79e2000640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79dc0ccf00 0x7f79dc0cfd60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.914+0000 7f79e2000640 1 --2- 172.21.15.60:0/2823304528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79dc073fb0 0x7f79dc148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.914+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 >> 172.21.15.60:0/2823304528 conn(0x7f79dc0b1950 msgr2=0x7f79dc0afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.914+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 shutdown_connections 2024-09-16T15:41:33.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:33.914+0000 7f79e2000640 1 -- 172.21.15.60:0/2823304528 wait complete. 2024-09-16T15:41:33.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:34.109 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":63.06739846,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.017424696614466,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:41:34.109 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.017424696614466 }, "state": "open", "uptime": 63.06739846 } ] 2024-09-16T15:41:34.109 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:41:39.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.280+0000 7f026e000640 1 -- 172.21.15.60:0/2101551212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f026815f5a0 msgr2=0x7f02680b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.280+0000 7f026e000640 1 --2- 172.21.15.60:0/2101551212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f026815f5a0 0x7f02680b2440 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f026806dac0 tx=0x7f025802a560 comp rx=0 tx=0).stop 2024-09-16T15:41:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.280+0000 7f026e000640 1 -- 172.21.15.60:0/2101551212 shutdown_connections 2024-09-16T15:41:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.280+0000 7f026e000640 1 --2- 172.21.15.60:0/2101551212 >> v2:172.21.15.100:3300/0 conn(0x7f0268073fb0 0x7f0268074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.280+0000 7f026e000640 1 --2- 172.21.15.60:0/2101551212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f026815f5a0 0x7f02680b2440 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.280+0000 7f026e000640 1 -- 172.21.15.60:0/2101551212 >> 172.21.15.60:0/2101551212 conn(0x7f02680b1950 msgr2=0x7f02680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.280+0000 7f026e000640 1 -- 172.21.15.60:0/2101551212 shutdown_connections 2024-09-16T15:41:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.280+0000 7f026e000640 1 -- 172.21.15.60:0/2101551212 wait complete. 2024-09-16T15:41:39.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.282+0000 7f026e000640 1 Processor -- start 2024-09-16T15:41:39.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.282+0000 7f026e000640 1 -- start start 2024-09-16T15:41:39.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f026e000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0268073fb0 0x7f0268146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f026e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02680ccf00 0x7f02680b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f026e000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f026815f5a0 0x7f0268150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f026e000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f026806f2e0 con 0x7f02680ccf00 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f026e000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f026806f160 con 0x7f026815f5a0 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f026e000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f026806f460 con 0x7f0268073fb0 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f025ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02680ccf00 0x7f02680b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f025ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02680ccf00 0x7f02680b2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41172/0 (socket says 172.21.15.60:41172) 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f025ea00640 1 -- 172.21.15.60:0/2403014119 learned_addr learned my addr 172.21.15.60:0/2403014119 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f0267400640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0268073fb0 0x7f0268146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.283+0000 7f0267e00640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f026815f5a0 0x7f0268150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025ea00640 1 -- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0268073fb0 msgr2=0x7f0268146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:39.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025ea00640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0268073fb0 0x7f0268146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025ea00640 1 -- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f026815f5a0 msgr2=0x7f0268150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025ea00640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f026815f5a0 0x7f0268150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025ea00640 1 -- 172.21.15.60:0/2403014119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02680b3840 con 0x7f02680ccf00 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025ea00640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02680ccf00 0x7f02680b2d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f025000c450 tx=0x7f025000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025fe00640 1 -- 172.21.15.60:0/2403014119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02500052d0 con 0x7f02680ccf00 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025fe00640 1 -- 172.21.15.60:0/2403014119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0250005470 con 0x7f02680ccf00 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02680b3b20 con 0x7f02680ccf00 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f025fe00640 1 -- 172.21.15.60:0/2403014119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0250012870 con 0x7f02680ccf00 2024-09-16T15:41:39.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.284+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02680b43c0 con 0x7f02680ccf00 2024-09-16T15:41:39.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.286+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02680b45c0 con 0x7f02680ccf00 2024-09-16T15:41:39.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.287+0000 7f025fe00640 1 -- 172.21.15.60:0/2403014119 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f02500035f0 con 0x7f02680ccf00 2024-09-16T15:41:39.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.288+0000 7f025fe00640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0244078ac0 0x7f024407af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:39.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.288+0000 7f025fe00640 1 -- 172.21.15.60:0/2403014119 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f025009a860 con 0x7f02680ccf00 2024-09-16T15:41:39.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.288+0000 7f0267400640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0244078ac0 0x7f024407af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:39.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.289+0000 7f0267400640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0244078ac0 0x7f024407af80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0258033000 tx=0x7f0258004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:39.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.292+0000 7f025fe00640 1 -- 172.21.15.60:0/2403014119 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f02680b45c0 con 0x7f02680ccf00 2024-09-16T15:41:39.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.523+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f026806f9f0 con 0x7f02680ccf00 2024-09-16T15:41:39.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.524+0000 7f025fe00640 1 -- 172.21.15.60:0/2403014119 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f02500623d0 con 0x7f02680ccf00 2024-09-16T15:41:39.525 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:39.525 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:39.525 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0244078ac0 msgr2=0x7f024407af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0244078ac0 0x7f024407af80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0258033000 tx=0x7f0258004730 comp rx=0 tx=0).stop 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02680ccf00 msgr2=0x7f02680b2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02680ccf00 0x7f02680b2d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f025000c450 tx=0x7f025000c910 comp rx=0 tx=0).stop 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 shutdown_connections 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0244078ac0 0x7f024407af80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f026815f5a0 0x7f0268150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02680ccf00 0x7f02680b2d80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 --2- 172.21.15.60:0/2403014119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0268073fb0 0x7f0268146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:39.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.526+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 >> 172.21.15.60:0/2403014119 conn(0x7f02680b1950 msgr2=0x7f0268153f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:39.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.527+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 shutdown_connections 2024-09-16T15:41:39.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:39.527+0000 7f026e000640 1 -- 172.21.15.60:0/2403014119 wait complete. 2024-09-16T15:41:39.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:39.724 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5159,"entity":{"name":{"type":"client","num":5159},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":65748822}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":68.681344718999995,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.772343158853907,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5159 172.21.15.100:0/65748822","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:41:39.724 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 65748822, "type": "any" }, "name": { "num": 5159, "type": "client" } }, "id": 5159, "inst": "client.5159 172.21.15.100:0/65748822", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.772343158853907 }, "state": "open", "uptime": 68.681344719 } ] 2024-09-16T15:41:39.724 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-16T15:41:44.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:44.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.897+0000 7f748c000640 1 -- 172.21.15.60:0/3369584330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748406f850 msgr2=0x7f748406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:44.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.897+0000 7f748c000640 1 --2- 172.21.15.60:0/3369584330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748406f850 0x7f748406fce0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7484143ee0 tx=0x7f747002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:41:44.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.897+0000 7f748c000640 1 -- 172.21.15.60:0/3369584330 shutdown_connections 2024-09-16T15:41:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.897+0000 7f748c000640 1 --2- 172.21.15.60:0/3369584330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f748406f850 0x7f748406fce0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.897+0000 7f748c000640 1 --2- 172.21.15.60:0/3369584330 >> v2:172.21.15.100:3300/0 conn(0x7f7484070920 0x7f74841438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.897+0000 7f748c000640 1 -- 172.21.15.60:0/3369584330 >> 172.21.15.60:0/3369584330 conn(0x7f74840b1950 msgr2=0x7f74840b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.897+0000 7f748c000640 1 -- 172.21.15.60:0/3369584330 shutdown_connections 2024-09-16T15:41:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.897+0000 7f748c000640 1 -- 172.21.15.60:0/3369584330 wait complete. 2024-09-16T15:41:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.898+0000 7f748c000640 1 Processor -- start 2024-09-16T15:41:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.899+0000 7f748c000640 1 -- start start 2024-09-16T15:41:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.899+0000 7f748c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748406f850 0x7f74840c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.899+0000 7f748c000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484070920 0x7f748415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f748c000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f748415dbf0 0x7f74840cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f748c000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f748415e910 con 0x7f7484070920 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f748c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f748415e790 con 0x7f748406f850 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f748c000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74840cc5f0 con 0x7f748415dbf0 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7488c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484070920 0x7f748415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7488c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484070920 0x7f748415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41188/0 (socket says 172.21.15.60:41188) 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7488c00640 1 -- 172.21.15.60:0/2145132879 learned_addr learned my addr 172.21.15.60:0/2145132879 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7489600640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748406f850 0x7f74840c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f748a000640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f748415dbf0 0x7f74840cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7488c00640 1 -- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f748415dbf0 msgr2=0x7f74840cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7488c00640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f748415dbf0 0x7f74840cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7488c00640 1 -- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748406f850 msgr2=0x7f74840c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7488c00640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748406f850 0x7f74840c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:44.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f7488c00640 1 -- 172.21.15.60:0/2145132879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748415b2c0 con 0x7f7484070920 2024-09-16T15:41:44.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.900+0000 7f748a000640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f748415dbf0 0x7f74840cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:44.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.901+0000 7f7488c00640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484070920 0x7f748415ad80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7470009850 tx=0x7f7470003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:44.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.901+0000 7f7481600640 1 -- 172.21.15.60:0/2145132879 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747003f070 con 0x7f7484070920 2024-09-16T15:41:44.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.901+0000 7f7481600640 1 -- 172.21.15.60:0/2145132879 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74700047a0 con 0x7f7484070920 2024-09-16T15:41:44.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.901+0000 7f7481600640 1 -- 172.21.15.60:0/2145132879 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470035030 con 0x7f7484070920 2024-09-16T15:41:44.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.901+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748409ac50 con 0x7f7484070920 2024-09-16T15:41:44.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.901+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748409b210 con 0x7f7484070920 2024-09-16T15:41:44.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.902+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748406fce0 con 0x7f7484070920 2024-09-16T15:41:44.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.903+0000 7f7481600640 1 -- 172.21.15.60:0/2145132879 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f7470042070 con 0x7f7484070920 2024-09-16T15:41:44.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.904+0000 7f7481600640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7464078a90 0x7f746407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:44.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.904+0000 7f7481600640 1 -- 172.21.15.60:0/2145132879 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f74700bbcf0 con 0x7f7484070920 2024-09-16T15:41:44.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.905+0000 7f7489600640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7464078a90 0x7f746407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:44.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.905+0000 7f7489600640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7464078a90 0x7f746407af50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7478005d80 tx=0x7f7478005c50 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:44.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:44.909+0000 7f7481600640 1 -- 172.21.15.60:0/2145132879 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f748415b2c0 con 0x7f7484070920 2024-09-16T15:41:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.140+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74840cd430 con 0x7f7484070920 2024-09-16T15:41:45.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.141+0000 7f7481600640 1 -- 172.21.15.60:0/2145132879 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f7470083ee0 con 0x7f7484070920 2024-09-16T15:41:45.142 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:45.142 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:45.142 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:45.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7464078a90 msgr2=0x7f746407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:45.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7464078a90 0x7f746407af50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7478005d80 tx=0x7f7478005c50 comp rx=0 tx=0).stop 2024-09-16T15:41:45.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484070920 msgr2=0x7f748415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:45.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484070920 0x7f748415ad80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7470009850 tx=0x7f7470003910 comp rx=0 tx=0).stop 2024-09-16T15:41:45.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 shutdown_connections 2024-09-16T15:41:45.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7464078a90 0x7f746407af50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:45.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f748415dbf0 0x7f74840cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:45.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484070920 0x7f748415ad80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:45.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 --2- 172.21.15.60:0/2145132879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f748406f850 0x7f74840c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:45.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 >> 172.21.15.60:0/2145132879 conn(0x7f74840b1950 msgr2=0x7f74840acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:45.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 shutdown_connections 2024-09-16T15:41:45.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.144+0000 7f748c000640 1 -- 172.21.15.60:0/2145132879 wait complete. 2024-09-16T15:41:45.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-16T15:41:45.339 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-09-16T15:41:45.339 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-16T15:41:45.339 DEBUG:tasks.ceph_test_case:wait_until_true: success in 65s and 0 retries 2024-09-16T15:41:45.340 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:41:45.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:41:45.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:41:45.529 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:41:45.530 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:41:45.530 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:41:45.530 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:45.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:41:45.565 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:41:45.566 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:41:45.566 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:45.602 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:41:45.602 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:41:45.602 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:41:45.602 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:45.641 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:41:45.641 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:41:45.641 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:41:45.641 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:45.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:41:45.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.840+0000 7f0c03600640 1 -- 172.21.15.60:0/3518493724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc073fb0 msgr2=0x7f0bfc074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.840+0000 7f0c03600640 1 --2- 172.21.15.60:0/3518493724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc073fb0 0x7f0bfc074440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005fd0 tx=0x7f0bf402aa10 comp rx=0 tx=0).stop 2024-09-16T15:41:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.840+0000 7f0c03600640 1 -- 172.21.15.60:0/3518493724 shutdown_connections 2024-09-16T15:41:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.840+0000 7f0c03600640 1 --2- 172.21.15.60:0/3518493724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc073fb0 0x7f0bfc074440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.840+0000 7f0c03600640 1 --2- 172.21.15.60:0/3518493724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ccf00 0x7f0bfc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.840+0000 7f0c03600640 1 -- 172.21.15.60:0/3518493724 >> 172.21.15.60:0/3518493724 conn(0x7f0bfc0b1950 msgr2=0x7f0bfc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.840+0000 7f0c03600640 1 -- 172.21.15.60:0/3518493724 shutdown_connections 2024-09-16T15:41:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.840+0000 7f0c03600640 1 -- 172.21.15.60:0/3518493724 wait complete. 2024-09-16T15:41:45.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.842+0000 7f0c03600640 1 Processor -- start 2024-09-16T15:41:45.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.842+0000 7f0c03600640 1 -- start start 2024-09-16T15:41:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c03600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc073fb0 0x7f0bfc14bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c03600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ccf00 0x7f0bfc1f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c03600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15f5a0 0x7f0bfc1f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c03600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc06f2e0 con 0x7f0bfc073fb0 2024-09-16T15:41:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c03600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc06f160 con 0x7f0bfc15f5a0 2024-09-16T15:41:45.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c03600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc06f460 con 0x7f0bfc0ccf00 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c00c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc073fb0 0x7f0bfc14bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c00c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc073fb0 0x7f0bfc14bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41202/0 (socket says 172.21.15.60:41202) 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c00c00640 1 -- 172.21.15.60:0/2022502303 learned_addr learned my addr 172.21.15.60:0/2022502303 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0bfbe00640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ccf00 0x7f0bfc1f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c01600640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15f5a0 0x7f0bfc1f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c00c00640 1 -- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ccf00 msgr2=0x7f0bfc1f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.843+0000 7f0c00c00640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ccf00 0x7f0bfc1f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0c00c00640 1 -- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15f5a0 msgr2=0x7f0bfc1f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0c00c00640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15f5a0 0x7f0bfc1f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0c00c00640 1 -- 172.21.15.60:0/2022502303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc0cdb80 con 0x7f0bfc073fb0 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0c01600640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15f5a0 0x7f0bfc1f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0c00c00640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc073fb0 0x7f0bfc14bd10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c430 tx=0x7f0bf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:45.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0bf8c00640 1 -- 172.21.15.60:0/2022502303 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0017070 con 0x7f0bfc073fb0 2024-09-16T15:41:45.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0bf8c00640 1 -- 172.21.15.60:0/2022502303 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bf00053e0 con 0x7f0bfc073fb0 2024-09-16T15:41:45.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0bf8c00640 1 -- 172.21.15.60:0/2022502303 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0003850 con 0x7f0bfc073fb0 2024-09-16T15:41:45.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc0cde60 con 0x7f0bfc073fb0 2024-09-16T15:41:45.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.844+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc0d0560 con 0x7f0bfc073fb0 2024-09-16T15:41:45.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.845+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bfc0ce0a0 con 0x7f0bfc073fb0 2024-09-16T15:41:45.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.847+0000 7f0bf8c00640 1 -- 172.21.15.60:0/2022502303 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 8) ==== 101491+0+0 (secure 0 0 0) 0x7f0bf001a070 con 0x7f0bfc073fb0 2024-09-16T15:41:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.848+0000 7f0bf8c00640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0bdc078a90 0x7f0bdc07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.848+0000 7f0bf8c00640 1 -- 172.21.15.60:0/2022502303 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f0bf009a1f0 con 0x7f0bfc073fb0 2024-09-16T15:41:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.848+0000 7f0bfbe00640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0bdc078a90 0x7f0bdc07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:45.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.849+0000 7f0bfbe00640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0bdc078a90 0x7f0bdc07af50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0be80095f0 tx=0x7f0be8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:45.853+0000 7f0bf8c00640 1 -- 172.21.15.60:0/2022502303 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0bfc0ce0a0 con 0x7f0bfc073fb0 2024-09-16T15:41:46.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.088+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bfc0d0a60 con 0x7f0bfc073fb0 2024-09-16T15:41:46.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.089+0000 7f0bf8c00640 1 -- 172.21.15.60:0/2022502303 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f0bf00623e0 con 0x7f0bfc073fb0 2024-09-16T15:41:46.090 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:46.090 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:46.090 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0bdc078a90 msgr2=0x7f0bdc07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0bdc078a90 0x7f0bdc07af50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0be80095f0 tx=0x7f0be8005f90 comp rx=0 tx=0).stop 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc073fb0 msgr2=0x7f0bfc14bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc073fb0 0x7f0bfc14bd10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000c430 tx=0x7f0bf000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 shutdown_connections 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0bdc078a90 0x7f0bdc07af50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bfc15f5a0 0x7f0bfc1f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bfc0ccf00 0x7f0bfc1f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.091+0000 7f0c03600640 1 --2- 172.21.15.60:0/2022502303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bfc073fb0 0x7f0bfc14bd10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.092+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 >> 172.21.15.60:0/2022502303 conn(0x7f0bfc0b1950 msgr2=0x7f0bfc0b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:46.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.092+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 shutdown_connections 2024-09-16T15:41:46.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.092+0000 7f0c03600640 1 -- 172.21.15.60:0/2022502303 wait complete. 2024-09-16T15:41:46.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.269+0000 7fbc93800640 1 -- 172.21.15.60:0/503585415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 msgr2=0x7fbc8c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.269+0000 7fbc93800640 1 --2- 172.21.15.60:0/503585415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 0x7fbc8c0732a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c06d9e0 tx=0x7fbc8402a820 comp rx=0 tx=0).stop 2024-09-16T15:41:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.270+0000 7fbc93800640 1 -- 172.21.15.60:0/503585415 shutdown_connections 2024-09-16T15:41:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.270+0000 7fbc93800640 1 --2- 172.21.15.60:0/503585415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 0x7fbc8c0732a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.270+0000 7fbc93800640 1 --2- 172.21.15.60:0/503585415 >> v2:172.21.15.100:3300/0 conn(0x7fbc8c161550 0x7fbc8c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.270+0000 7fbc93800640 1 -- 172.21.15.60:0/503585415 >> 172.21.15.60:0/503585415 conn(0x7fbc8c0b1950 msgr2=0x7fbc8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.270+0000 7fbc93800640 1 -- 172.21.15.60:0/503585415 shutdown_connections 2024-09-16T15:41:46.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.270+0000 7fbc93800640 1 -- 172.21.15.60:0/503585415 wait complete. 2024-09-16T15:41:46.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.271+0000 7fbc93800640 1 Processor -- start 2024-09-16T15:41:46.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.271+0000 7fbc93800640 1 -- start start 2024-09-16T15:41:46.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.272+0000 7fbc93800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 0x7fbc8c14f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:46.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.272+0000 7fbc93800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c0af3d0 0x7fbc8c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.272+0000 7fbc93800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc8c161550 0x7fbc8c1568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.272+0000 7fbc93800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c0737e0 con 0x7fbc8c072df0 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.272+0000 7fbc93800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c06fdb0 con 0x7fbc8c0af3d0 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.272+0000 7fbc93800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8c073960 con 0x7fbc8c161550 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.272+0000 7fbc90e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 0x7fbc8c14f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc90e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 0x7fbc8c14f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41218/0 (socket says 172.21.15.60:41218) 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc90e00640 1 -- 172.21.15.60:0/330728243 learned_addr learned my addr 172.21.15.60:0/330728243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc90e00640 1 -- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc8c161550 msgr2=0x7fbc8c1568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc90e00640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc8c161550 0x7fbc8c1568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc90e00640 1 -- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c0af3d0 msgr2=0x7fbc8c158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc90e00640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c0af3d0 0x7fbc8c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc90e00640 1 -- 172.21.15.60:0/330728243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c1f6030 con 0x7fbc8c072df0 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc90e00640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 0x7fbc8c14f6d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00d770 tx=0x7fbc7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:46.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc89600640 1 -- 172.21.15.60:0/330728243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c017070 con 0x7fbc8c072df0 2024-09-16T15:41:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc89600640 1 -- 172.21.15.60:0/330728243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc7c009d20 con 0x7fbc8c072df0 2024-09-16T15:41:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.273+0000 7fbc89600640 1 -- 172.21.15.60:0/330728243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0106c0 con 0x7fbc8c072df0 2024-09-16T15:41:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.274+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c155b10 con 0x7fbc8c072df0 2024-09-16T15:41:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.274+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1560d0 con 0x7fbc8c072df0 2024-09-16T15:41:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.275+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c073de0 con 0x7fbc8c072df0 2024-09-16T15:41:46.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.276+0000 7fbc89600640 1 -- 172.21.15.60:0/330728243 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fbc7c01a070 con 0x7fbc8c072df0 2024-09-16T15:41:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.277+0000 7fbc89600640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbc68078a00 0x7fbc6807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.277+0000 7fbc89600640 1 -- 172.21.15.60:0/330728243 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fbc7c09ae10 con 0x7fbc8c072df0 2024-09-16T15:41:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.277+0000 7fbc83e00640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbc68078a00 0x7fbc6807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.278+0000 7fbc83e00640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbc68078a00 0x7fbc6807aec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0d0670 tx=0x7fbc74007a30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:46.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.281+0000 7fbc89600640 1 -- 172.21.15.60:0/330728243 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbc7c063140 con 0x7fbc8c072df0 2024-09-16T15:41:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.513+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc8c0aec30 con 0x7fbc8c072df0 2024-09-16T15:41:46.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.514+0000 7fbc89600640 1 -- 172.21.15.60:0/330728243 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fbc7c062a40 con 0x7fbc8c072df0 2024-09-16T15:41:46.515 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:46.515 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:46.515 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:46.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbc68078a00 msgr2=0x7fbc6807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbc68078a00 0x7fbc6807aec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0d0670 tx=0x7fbc74007a30 comp rx=0 tx=0).stop 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 msgr2=0x7fbc8c14f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 0x7fbc8c14f6d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00d770 tx=0x7fbc7c00dc30 comp rx=0 tx=0).stop 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 shutdown_connections 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fbc68078a00 0x7fbc6807aec0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc8c161550 0x7fbc8c1568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc8c0af3d0 0x7fbc8c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 --2- 172.21.15.60:0/330728243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8c072df0 0x7fbc8c14f6d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 >> 172.21.15.60:0/330728243 conn(0x7fbc8c0b1950 msgr2=0x7fbc8c0adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 shutdown_connections 2024-09-16T15:41:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.516+0000 7fbc93800640 1 -- 172.21.15.60:0/330728243 wait complete. 2024-09-16T15:41:46.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:41:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.692+0000 7fed1d000640 1 -- 172.21.15.60:0/455351315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18161550 msgr2=0x7fed1806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.692+0000 7fed1d000640 1 --2- 172.21.15.60:0/455351315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18161550 0x7fed1806ef50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fed180752e0 tx=0x7fed0802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.692+0000 7fed1d000640 1 -- 172.21.15.60:0/455351315 shutdown_connections 2024-09-16T15:41:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.692+0000 7fed1d000640 1 --2- 172.21.15.60:0/455351315 >> v2:172.21.15.100:3300/0 conn(0x7fed180af3d0 0x7fed1806f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.692+0000 7fed1d000640 1 --2- 172.21.15.60:0/455351315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed18161550 0x7fed1806ef50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.692+0000 7fed1d000640 1 -- 172.21.15.60:0/455351315 >> 172.21.15.60:0/455351315 conn(0x7fed180b1950 msgr2=0x7fed180b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.693+0000 7fed1d000640 1 -- 172.21.15.60:0/455351315 shutdown_connections 2024-09-16T15:41:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.693+0000 7fed1d000640 1 -- 172.21.15.60:0/455351315 wait complete. 2024-09-16T15:41:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.694+0000 7fed1d000640 1 Processor -- start 2024-09-16T15:41:46.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.694+0000 7fed1d000640 1 -- start start 2024-09-16T15:41:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed1d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806f9d0 0x7fed18149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:46.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed1d000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed180af3d0 0x7fed180c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed1d000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed18161550 0x7fed180c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed1d000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fed18073f80 con 0x7fed180af3d0 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed1d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed18073e00 con 0x7fed1806f9d0 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed1d000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fed18074100 con 0x7fed18161550 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed0d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed180af3d0 0x7fed180c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed0d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed180af3d0 0x7fed180c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41224/0 (socket says 172.21.15.60:41224) 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.695+0000 7fed0d400640 1 -- 172.21.15.60:0/557568098 learned_addr learned my addr 172.21.15.60:0/557568098 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed15e00640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806f9d0 0x7fed18149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed16800640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed18161550 0x7fed180c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0d400640 1 -- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed18161550 msgr2=0x7fed180c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0d400640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed18161550 0x7fed180c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0d400640 1 -- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806f9d0 msgr2=0x7fed18149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0d400640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806f9d0 0x7fed18149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0d400640 1 -- 172.21.15.60:0/557568098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed180d06b0 con 0x7fed180af3d0 2024-09-16T15:41:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0d400640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed180af3d0 0x7fed180c8fd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fed0000d980 tx=0x7fed0000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0ea00640 1 -- 172.21.15.60:0/557568098 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00017070 con 0x7fed180af3d0 2024-09-16T15:41:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0ea00640 1 -- 172.21.15.60:0/557568098 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed00009d20 con 0x7fed180af3d0 2024-09-16T15:41:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed0ea00640 1 -- 172.21.15.60:0/557568098 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed000106c0 con 0x7fed180af3d0 2024-09-16T15:41:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.696+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed180cb0b0 con 0x7fed180af3d0 2024-09-16T15:41:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.697+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed180cb670 con 0x7fed180af3d0 2024-09-16T15:41:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.698+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed180760b0 con 0x7fed180af3d0 2024-09-16T15:41:46.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.699+0000 7fed0ea00640 1 -- 172.21.15.60:0/557568098 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fed0001a070 con 0x7fed180af3d0 2024-09-16T15:41:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.700+0000 7fed0ea00640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf4078a00 0x7fecf407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.700+0000 7fed0ea00640 1 -- 172.21.15.60:0/557568098 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7fed0009aeb0 con 0x7fed180af3d0 2024-09-16T15:41:46.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.700+0000 7fed15e00640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf4078a00 0x7fecf407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:46.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.701+0000 7fed15e00640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf4078a00 0x7fecf407aec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fed08034000 tx=0x7fed08004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:46.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.704+0000 7fed0ea00640 1 -- 172.21.15.60:0/557568098 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fed180760b0 con 0x7fed180af3d0 2024-09-16T15:41:46.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.901+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed1806ea00 con 0x7fed180af3d0 2024-09-16T15:41:46.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.902+0000 7fed0ea00640 1 -- 172.21.15.60:0/557568098 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16824 (secure 0 0 0) 0x7fed1806ea00 con 0x7fed180af3d0 2024-09-16T15:41:46.904 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:46.904 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":50,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:41:16.030282+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.089921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:11.255279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.12","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.904+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf4078a00 msgr2=0x7fecf407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.904+0000 7fed1d000640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf4078a00 0x7fecf407aec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fed08034000 tx=0x7fed08004730 comp rx=0 tx=0).stop 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.904+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed180af3d0 msgr2=0x7fed180c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.904+0000 7fed1d000640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed180af3d0 0x7fed180c8fd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fed0000d980 tx=0x7fed0000de40 comp rx=0 tx=0).stop 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.905+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 shutdown_connections 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.905+0000 7fed1d000640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fecf4078a00 0x7fecf407aec0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.905+0000 7fed1d000640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed18161550 0x7fed180c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.905+0000 7fed1d000640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed180af3d0 0x7fed180c8fd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.905+0000 7fed1d000640 1 --2- 172.21.15.60:0/557568098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1806f9d0 0x7fed18149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.905+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 >> 172.21.15.60:0/557568098 conn(0x7fed180b1950 msgr2=0x7fed180ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.905+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 shutdown_connections 2024-09-16T15:41:46.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:46.905+0000 7fed1d000640 1 -- 172.21.15.60:0/557568098 wait complete. 2024-09-16T15:41:46.922 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:41:46.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.082+0000 7f8d96800640 1 -- 172.21.15.60:0/364520144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9015f5a0 msgr2=0x7f8d900b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.082+0000 7f8d96800640 1 --2- 172.21.15.60:0/364520144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9015f5a0 0x7f8d900b2440 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005fd0 tx=0x7f8d8002a540 comp rx=0 tx=0).stop 2024-09-16T15:41:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.082+0000 7f8d96800640 1 -- 172.21.15.60:0/364520144 shutdown_connections 2024-09-16T15:41:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.082+0000 7f8d96800640 1 --2- 172.21.15.60:0/364520144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d90073fb0 0x7f8d90074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.082+0000 7f8d96800640 1 --2- 172.21.15.60:0/364520144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d9015f5a0 0x7f8d900b2440 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.082+0000 7f8d96800640 1 -- 172.21.15.60:0/364520144 >> 172.21.15.60:0/364520144 conn(0x7f8d900b1950 msgr2=0x7f8d900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.082+0000 7f8d96800640 1 -- 172.21.15.60:0/364520144 shutdown_connections 2024-09-16T15:41:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.082+0000 7f8d96800640 1 -- 172.21.15.60:0/364520144 wait complete. 2024-09-16T15:41:47.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.084+0000 7f8d96800640 1 Processor -- start 2024-09-16T15:41:47.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.084+0000 7f8d96800640 1 -- start start 2024-09-16T15:41:47.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d96800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d90073fb0 0x7f8d9014e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d96800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d900ccf00 0x7f8d900cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d96800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9015f5a0 0x7f8d900cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d96800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9006f2e0 con 0x7f8d90073fb0 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d96800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9006f160 con 0x7f8d9015f5a0 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d96800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9006f460 con 0x7f8d900ccf00 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d90073fb0 0x7f8d9014e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d90073fb0 0x7f8d9014e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41254/0 (socket says 172.21.15.60:41254) 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8f400640 1 -- 172.21.15.60:0/2882973747 learned_addr learned my addr 172.21.15.60:0/2882973747 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8ea00640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d900ccf00 0x7f8d900cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8fe00640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9015f5a0 0x7f8d900cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8f400640 1 -- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d900ccf00 msgr2=0x7f8d900cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8f400640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d900ccf00 0x7f8d900cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8f400640 1 -- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9015f5a0 msgr2=0x7f8d900cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8f400640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9015f5a0 0x7f8d900cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8f400640 1 -- 172.21.15.60:0/2882973747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9009ac10 con 0x7f8d90073fb0 2024-09-16T15:41:47.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.085+0000 7f8d8fe00640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9015f5a0 0x7f8d900cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.086+0000 7f8d8f400640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d90073fb0 0x7f8d9014e4b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8d80006010 tx=0x7f8d800038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.086+0000 7f8d87400640 1 -- 172.21.15.60:0/2882973747 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8003e030 con 0x7f8d90073fb0 2024-09-16T15:41:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.086+0000 7f8d87400640 1 -- 172.21.15.60:0/2882973747 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d80003d90 con 0x7f8d90073fb0 2024-09-16T15:41:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.086+0000 7f8d87400640 1 -- 172.21.15.60:0/2882973747 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80035040 con 0x7f8d90073fb0 2024-09-16T15:41:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.086+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9009aef0 con 0x7f8d90073fb0 2024-09-16T15:41:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.086+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9014e9f0 con 0x7f8d90073fb0 2024-09-16T15:41:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.087+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9006f980 con 0x7f8d90073fb0 2024-09-16T15:41:47.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.088+0000 7f8d87400640 1 -- 172.21.15.60:0/2882973747 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f8d80041030 con 0x7f8d90073fb0 2024-09-16T15:41:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.090+0000 7f8d87400640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8d70078a00 0x7f8d7007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.090+0000 7f8d87400640 1 -- 172.21.15.60:0/2882973747 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f8d800bb300 con 0x7f8d90073fb0 2024-09-16T15:41:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.091+0000 7f8d8ea00640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8d70078a00 0x7f8d7007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.091+0000 7f8d8ea00640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8d70078a00 0x7f8d7007aec0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d90155790 tx=0x7f8d7c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:47.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.094+0000 7f8d87400640 1 -- 172.21.15.60:0/2882973747 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8d80083630 con 0x7f8d90073fb0 2024-09-16T15:41:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.330+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d900ae990 con 0x7f8d90073fb0 2024-09-16T15:41:47.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.330+0000 7f8d87400640 1 -- 172.21.15.60:0/2882973747 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f8d80082f30 con 0x7f8d90073fb0 2024-09-16T15:41:47.331 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:47.332 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:47.332 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8d70078a00 msgr2=0x7f8d7007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8d70078a00 0x7f8d7007aec0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d90155790 tx=0x7f8d7c009040 comp rx=0 tx=0).stop 2024-09-16T15:41:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d90073fb0 msgr2=0x7f8d9014e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d90073fb0 0x7f8d9014e4b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8d80006010 tx=0x7f8d800038a0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 shutdown_connections 2024-09-16T15:41:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8d70078a00 0x7f8d7007aec0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d9015f5a0 0x7f8d900cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d900ccf00 0x7f8d900cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 --2- 172.21.15.60:0/2882973747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d90073fb0 0x7f8d9014e4b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 >> 172.21.15.60:0/2882973747 conn(0x7f8d900b1950 msgr2=0x7f8d900b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 shutdown_connections 2024-09-16T15:41:47.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.333+0000 7f8d96800640 1 -- 172.21.15.60:0/2882973747 wait complete. 2024-09-16T15:41:47.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.510+0000 7f8a10a00640 1 -- 172.21.15.60:0/3334886754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a0c15f5a0 msgr2=0x7f8a0c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.510+0000 7f8a10a00640 1 --2- 172.21.15.60:0/3334886754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a0c15f5a0 0x7f8a0c0b2440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f89fc005fd0 tx=0x7f89fc02a540 comp rx=0 tx=0).stop 2024-09-16T15:41:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.510+0000 7f8a10a00640 1 -- 172.21.15.60:0/3334886754 shutdown_connections 2024-09-16T15:41:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.510+0000 7f8a10a00640 1 --2- 172.21.15.60:0/3334886754 >> v2:172.21.15.100:3300/0 conn(0x7f8a0c0ccf00 0x7f8a0c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.510+0000 7f8a10a00640 1 --2- 172.21.15.60:0/3334886754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a0c15f5a0 0x7f8a0c0b2440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.510+0000 7f8a10a00640 1 -- 172.21.15.60:0/3334886754 >> 172.21.15.60:0/3334886754 conn(0x7f8a0c0b1950 msgr2=0x7f8a0c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.510+0000 7f8a10a00640 1 -- 172.21.15.60:0/3334886754 shutdown_connections 2024-09-16T15:41:47.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.510+0000 7f8a10a00640 1 -- 172.21.15.60:0/3334886754 wait complete. 2024-09-16T15:41:47.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.511+0000 7f8a10a00640 1 Processor -- start 2024-09-16T15:41:47.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.512+0000 7f8a10a00640 1 -- start start 2024-09-16T15:41:47.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.512+0000 7f8a10a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a0c0ad270 0x7f8a0c0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a10a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a0c0ccf00 0x7f8a0c1f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a10a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a0c15f5a0 0x7f8a0c15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a10a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c06f2e0 con 0x7f8a0c0ad270 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a10a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c06f160 con 0x7f8a0c0ccf00 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a10a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c06f460 con 0x7f8a0c15f5a0 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a09e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a0c0ad270 0x7f8a0c0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a09e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a0c0ad270 0x7f8a0c0cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41262/0 (socket says 172.21.15.60:41262) 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a09e00640 1 -- 172.21.15.60:0/886297985 learned_addr learned my addr 172.21.15.60:0/886297985 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a01400640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a0c0ccf00 0x7f8a0c1f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a0a800640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a0c15f5a0 0x7f8a0c15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a09e00640 1 -- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a0c15f5a0 msgr2=0x7f8a0c15e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a09e00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a0c15f5a0 0x7f8a0c15e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a09e00640 1 -- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a0c0ccf00 msgr2=0x7f8a0c1f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a09e00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a0c0ccf00 0x7f8a0c1f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.513+0000 7f8a09e00640 1 -- 172.21.15.60:0/886297985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c1f5460 con 0x7f8a0c0ad270 2024-09-16T15:41:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.514+0000 7f8a09e00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a0c0ad270 0x7f8a0c0cb190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f89fc034040 tx=0x7f89fc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.514+0000 7f8a02a00640 1 -- 172.21.15.60:0/886297985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc03e030 con 0x7f8a0c0ad270 2024-09-16T15:41:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.514+0000 7f8a02a00640 1 -- 172.21.15.60:0/886297985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89fc003d90 con 0x7f8a0c0ad270 2024-09-16T15:41:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.514+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c1f5740 con 0x7f8a0c0ad270 2024-09-16T15:41:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.514+0000 7f8a02a00640 1 -- 172.21.15.60:0/886297985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc035040 con 0x7f8a0c0ad270 2024-09-16T15:41:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.514+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c0c91e0 con 0x7f8a0c0ad270 2024-09-16T15:41:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.515+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c0c8fd0 con 0x7f8a0c0ad270 2024-09-16T15:41:47.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.516+0000 7f8a02a00640 1 -- 172.21.15.60:0/886297985 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f89fc041030 con 0x7f8a0c0ad270 2024-09-16T15:41:47.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.517+0000 7f8a02a00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f89e8078a00 0x7f89e807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.517+0000 7f8a02a00640 1 -- 172.21.15.60:0/886297985 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f89fc0bb650 con 0x7f8a0c0ad270 2024-09-16T15:41:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.518+0000 7f8a01400640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f89e8078a00 0x7f89e807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.519+0000 7f8a01400640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f89e8078a00 0x7f89e807aec0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c1555b0 tx=0x7f89f4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:47.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.521+0000 7f8a02a00640 1 -- 172.21.15.60:0/886297985 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f89fc083340 con 0x7f8a0c0ad270 2024-09-16T15:41:47.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.753+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a0c06f980 con 0x7f8a0c0ad270 2024-09-16T15:41:47.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.753+0000 7f8a02a00640 1 -- 172.21.15.60:0/886297985 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f89fc087d70 con 0x7f8a0c0ad270 2024-09-16T15:41:47.755 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:47.755 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"btime":"2024-09-16T15:40:27:896343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:12.294606+0000","modified":"2024-09-16T15:40:15.298172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2823237251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2823237251},{"type":"v1","addr":"172.21.15.60:6835","nonce":2823237251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-16T15:41:47.755 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 27 2024-09-16T15:41:47.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f89e8078a00 msgr2=0x7f89e807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f89e8078a00 0x7f89e807aec0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c1555b0 tx=0x7f89f4005f90 comp rx=0 tx=0).stop 2024-09-16T15:41:47.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a0c0ad270 msgr2=0x7f8a0c0cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a0c0ad270 0x7f8a0c0cb190 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f89fc034040 tx=0x7f89fc0038a0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 shutdown_connections 2024-09-16T15:41:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f89e8078a00 0x7f89e807aec0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a0c15f5a0 0x7f8a0c15e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a0c0ccf00 0x7f8a0c1f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 --2- 172.21.15.60:0/886297985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a0c0ad270 0x7f8a0c0cb190 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 >> 172.21.15.60:0/886297985 conn(0x7f8a0c0b1950 msgr2=0x7f8a0c0b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 shutdown_connections 2024-09-16T15:41:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.756+0000 7f8a10a00640 1 -- 172.21.15.60:0/886297985 wait complete. 2024-09-16T15:41:47.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:41:47.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.934+0000 7f9ad6600640 1 -- 172.21.15.60:0/615026214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ad0161520 msgr2=0x7f9ad00d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.934+0000 7f9ad6600640 1 --2- 172.21.15.60:0/615026214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ad0161520 0x7f9ad00d4940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009860 tx=0x7f9ac402a5b0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.934+0000 7f9ad6600640 1 -- 172.21.15.60:0/615026214 shutdown_connections 2024-09-16T15:41:47.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.934+0000 7f9ad6600640 1 --2- 172.21.15.60:0/615026214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ad0161520 0x7f9ad00d4940 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.934+0000 7f9ad6600640 1 -- 172.21.15.60:0/615026214 >> 172.21.15.60:0/615026214 conn(0x7f9ad00d3e20 msgr2=0x7f9ad00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:47.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.934+0000 7f9ad6600640 1 -- 172.21.15.60:0/615026214 shutdown_connections 2024-09-16T15:41:47.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.934+0000 7f9ad6600640 1 -- 172.21.15.60:0/615026214 wait complete. 2024-09-16T15:41:47.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.936+0000 7f9ad6600640 1 Processor -- start 2024-09-16T15:41:47.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.936+0000 7f9ad6600640 1 -- start start 2024-09-16T15:41:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9ad6600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ad00cf440 0x7f9ad0153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9ad6600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ad00d25b0 0x7f9ad00ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9ad6600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad0161520 0x7f9ad015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9ad6600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad00b8680 con 0x7f9ad00d25b0 2024-09-16T15:41:47.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9ad6600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad00b8500 con 0x7f9ad0161520 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9ad6600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad00b8800 con 0x7f9ad00cf440 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ad00d25b0 0x7f9ad00ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ad00d25b0 0x7f9ad00ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41272/0 (socket says 172.21.15.60:41272) 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9acfe00640 1 -- 172.21.15.60:0/1690957740 learned_addr learned my addr 172.21.15.60:0/1690957740 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9ad5200640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ad00cf440 0x7f9ad0153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9ad5c00640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad0161520 0x7f9ad015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9acfe00640 1 -- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ad00cf440 msgr2=0x7f9ad0153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9acfe00640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ad00cf440 0x7f9ad0153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9acfe00640 1 -- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad0161520 msgr2=0x7f9ad015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9acfe00640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad0161520 0x7f9ad015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.937+0000 7f9acfe00640 1 -- 172.21.15.60:0/1690957740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad00cdb00 con 0x7f9ad00d25b0 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.938+0000 7f9acfe00640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ad00d25b0 0x7f9ad00ca550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c460 tx=0x7f9abc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:47.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.938+0000 7f9accc00640 1 -- 172.21.15.60:0/1690957740 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc016020 con 0x7f9ad00d25b0 2024-09-16T15:41:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.938+0000 7f9accc00640 1 -- 172.21.15.60:0/1690957740 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9abc012920 con 0x7f9ad00d25b0 2024-09-16T15:41:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.938+0000 7f9accc00640 1 -- 172.21.15.60:0/1690957740 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc012d50 con 0x7f9ad00d25b0 2024-09-16T15:41:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.938+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad00cdde0 con 0x7f9ad00d25b0 2024-09-16T15:41:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.938+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad015f170 con 0x7f9ad00d25b0 2024-09-16T15:41:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.939+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad015ef60 con 0x7f9ad00d25b0 2024-09-16T15:41:47.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.940+0000 7f9accc00640 1 -- 172.21.15.60:0/1690957740 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f9abc019020 con 0x7f9ad00d25b0 2024-09-16T15:41:47.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.941+0000 7f9accc00640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9ab0078a00 0x7f9ab007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:47.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.941+0000 7f9accc00640 1 -- 172.21.15.60:0/1690957740 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f9abc09aa00 con 0x7f9ad00d25b0 2024-09-16T15:41:47.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.942+0000 7f9ad5200640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9ab0078a00 0x7f9ab007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:47.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.942+0000 7f9ad5200640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9ab0078a00 0x7f9ab007aec0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4034000 tx=0x7f9ac4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:47.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:47.945+0000 7f9accc00640 1 -- 172.21.15.60:0/1690957740 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9abc020030 con 0x7f9ad00d25b0 2024-09-16T15:41:48.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.144+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ad00d4940 con 0x7f9ad00d25b0 2024-09-16T15:41:48.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.145+0000 7f9accc00640 1 -- 172.21.15.60:0/1690957740 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16824 (secure 0 0 0) 0x7f9abc062d50 con 0x7f9ad00d25b0 2024-09-16T15:41:48.146 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:48.147 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":50,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:41:16.030282+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.089921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:11.255279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":5}]},{"pgid":"7.12","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:41:48.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9ab0078a00 msgr2=0x7f9ab007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9ab0078a00 0x7f9ab007aec0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4034000 tx=0x7f9ac4004730 comp rx=0 tx=0).stop 2024-09-16T15:41:48.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ad00d25b0 msgr2=0x7f9ad00ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ad00d25b0 0x7f9ad00ca550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00c460 tx=0x7f9abc00c920 comp rx=0 tx=0).stop 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 shutdown_connections 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9ab0078a00 0x7f9ab007aec0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad0161520 0x7f9ad015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ad00d25b0 0x7f9ad00ca550 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 --2- 172.21.15.60:0/1690957740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ad00cf440 0x7f9ad0153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 >> 172.21.15.60:0/1690957740 conn(0x7f9ad00d3e20 msgr2=0x7f9ad00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 shutdown_connections 2024-09-16T15:41:48.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.148+0000 7f9ad6600640 1 -- 172.21.15.60:0/1690957740 wait complete. 2024-09-16T15:41:48.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:41:48.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.327+0000 7f441b000640 1 -- 172.21.15.60:0/4104051864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44140b5370 msgr2=0x7f44140b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.327+0000 7f441b000640 1 --2- 172.21.15.60:0/4104051864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44140b5370 0x7f44140b3bc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f44140d4ea0 tx=0x7f440002a540 comp rx=0 tx=0).stop 2024-09-16T15:41:48.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.327+0000 7f441b000640 1 -- 172.21.15.60:0/4104051864 shutdown_connections 2024-09-16T15:41:48.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.327+0000 7f441b000640 1 --2- 172.21.15.60:0/4104051864 >> v2:172.21.15.100:3300/0 conn(0x7f44140b4100 0x7f44140b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.327+0000 7f441b000640 1 --2- 172.21.15.60:0/4104051864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44140b5370 0x7f44140b3bc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.327+0000 7f441b000640 1 -- 172.21.15.60:0/4104051864 >> 172.21.15.60:0/4104051864 conn(0x7f44140d3df0 msgr2=0x7f44140d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:48.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.327+0000 7f441b000640 1 -- 172.21.15.60:0/4104051864 shutdown_connections 2024-09-16T15:41:48.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.327+0000 7f441b000640 1 -- 172.21.15.60:0/4104051864 wait complete. 2024-09-16T15:41:48.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.329+0000 7f441b000640 1 Processor -- start 2024-09-16T15:41:48.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.329+0000 7f441b000640 1 -- start start 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44140b4100 0x7f44140b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441b000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44140b5370 0x7f44140d1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441b000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4414161440 0x7f44140d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441b000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f44140b2040 con 0x7f4414161440 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f44140b1ec0 con 0x7f44140b4100 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441b000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f44140b21c0 con 0x7f44140b5370 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4414161440 0x7f44140d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4414161440 0x7f44140d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41288/0 (socket says 172.21.15.60:41288) 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.330+0000 7f441a600640 1 -- 172.21.15.60:0/3728097600 learned_addr learned my addr 172.21.15.60:0/3728097600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f4419200640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44140b5370 0x7f44140d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:48.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f4419c00640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44140b4100 0x7f44140b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:48.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f441a600640 1 -- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44140b5370 msgr2=0x7f44140d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f441a600640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44140b5370 0x7f44140d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f441a600640 1 -- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44140b4100 msgr2=0x7f44140b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f441a600640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44140b4100 0x7f44140b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f441a600640 1 -- 172.21.15.60:0/3728097600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44141f7a30 con 0x7f4414161440 2024-09-16T15:41:48.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f441a600640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4414161440 0x7f44140d21b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f440c009cf0 tx=0x7f440c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:48.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f4411600640 1 -- 172.21.15.60:0/3728097600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c003b70 con 0x7f4414161440 2024-09-16T15:41:48.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f4411600640 1 -- 172.21.15.60:0/3728097600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f440c003e00 con 0x7f4414161440 2024-09-16T15:41:48.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.331+0000 7f4411600640 1 -- 172.21.15.60:0/3728097600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c004280 con 0x7f4414161440 2024-09-16T15:41:48.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.332+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44141f7d10 con 0x7f4414161440 2024-09-16T15:41:48.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.332+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141f82d0 con 0x7f4414161440 2024-09-16T15:41:48.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.333+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44140b1770 con 0x7f4414161440 2024-09-16T15:41:48.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.334+0000 7f4411600640 1 -- 172.21.15.60:0/3728097600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f440c004420 con 0x7f4414161440 2024-09-16T15:41:48.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.335+0000 7f4411600640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f43f4078a00 0x7f43f407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:48.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.335+0000 7f4411600640 1 -- 172.21.15.60:0/3728097600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5767+0+0 (secure 0 0 0) 0x7f440c09bb10 con 0x7f4414161440 2024-09-16T15:41:48.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.335+0000 7f4419c00640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f43f4078a00 0x7f43f407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:48.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.336+0000 7f4419c00640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f43f4078a00 0x7f43f407aec0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4408007fd0 tx=0x7f4408009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:48.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.339+0000 7f4411600640 1 -- 172.21.15.60:0/3728097600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f440c0637f0 con 0x7f4414161440 2024-09-16T15:41:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.573+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f44140d4270 con 0x7f4414161440 2024-09-16T15:41:48.575 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:41:48.574+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:41:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.576+0000 7f4411600640 1 -- 172.21.15.60:0/3728097600 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f440c068220 con 0x7f4414161440 2024-09-16T15:41:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:41:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f43f4078a00 msgr2=0x7f43f407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f43f4078a00 0x7f43f407aec0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4408007fd0 tx=0x7f4408009040 comp rx=0 tx=0).stop 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4414161440 msgr2=0x7f44140d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4414161440 0x7f44140d21b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f440c009cf0 tx=0x7f440c00c6a0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 shutdown_connections 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f43f4078a00 0x7f43f407aec0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4414161440 0x7f44140d21b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44140b5370 0x7f44140d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 --2- 172.21.15.60:0/3728097600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44140b4100 0x7f44140b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 >> 172.21.15.60:0/3728097600 conn(0x7f44140d3df0 msgr2=0x7f441406ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 shutdown_connections 2024-09-16T15:41:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.579+0000 7f441b000640 1 -- 172.21.15.60:0/3728097600 wait complete. 2024-09-16T15:41:48.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:41:48.649 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:41:48.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.764+0000 7f49aa800640 1 -- 172.21.15.60:0/3245068014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988004d40 msgr2=0x7f49880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.764+0000 7f49aa800640 1 --2- 172.21.15.60:0/3245068014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4988004d40 0x7f49880051c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f499c001f30 tx=0x7f4988005830 comp rx=0 tx=0).stop 2024-09-16T15:41:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.764+0000 7f49a3e00640 1 -- 172.21.15.60:0/3245068014 reap_dead start 2024-09-16T15:41:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.764+0000 7f49aa800640 1 -- 172.21.15.60:0/3245068014 shutdown_connections 2024-09-16T15:41:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.764+0000 7f49aa800640 1 -- 172.21.15.60:0/3245068014 >> 172.21.15.60:0/3245068014 conn(0x7f49a40b1840 msgr2=0x7f49a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.764+0000 7f49aa800640 1 -- 172.21.15.60:0/3245068014 shutdown_connections 2024-09-16T15:41:48.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.764+0000 7f49aa800640 1 -- 172.21.15.60:0/3245068014 wait complete. 2024-09-16T15:41:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.766+0000 7f49aa800640 1 Processor -- start 2024-09-16T15:41:48.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.766+0000 7f49aa800640 1 -- start start 2024-09-16T15:41:48.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49aa800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4988004d40 0x7f49a41e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:48.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49aa800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49a41e6980 0x7f49a4131090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:48.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49aa800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49a41315d0 0x7f49a4131a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:48.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49aa800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f49a40737d0 con 0x7f49a41e6980 2024-09-16T15:41:48.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49aa800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f49a4073650 con 0x7f4988004d40 2024-09-16T15:41:48.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49aa800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f49a40ce2c0 con 0x7f49a41315d0 2024-09-16T15:41:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f499aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49a41e6980 0x7f49a4131090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f499aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49a41e6980 0x7f49a4131090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41304/0 (socket says 172.21.15.60:41304) 2024-09-16T15:41:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f499aa00640 1 -- 172.21.15.60:0/2091712089 learned_addr learned my addr 172.21.15.60:0/2091712089 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49a3400640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4988004d40 0x7f49a41e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49a3e00640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49a41315d0 0x7f49a4131a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f499aa00640 1 -- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49a41315d0 msgr2=0x7f49a4131a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f499aa00640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49a41315d0 0x7f49a4131a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f499aa00640 1 -- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4988004d40 msgr2=0x7f49a41e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f499aa00640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4988004d40 0x7f49a41e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f499aa00640 1 -- 172.21.15.60:0/2091712089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a41e6e00 con 0x7f49a41e6980 2024-09-16T15:41:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49a3400640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4988004d40 0x7f49a41e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.767+0000 7f49a3e00640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49a41315d0 0x7f49a4131a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.768+0000 7f499aa00640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49a41e6980 0x7f49a4131090 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f498c011b40 tx=0x7f498c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:48.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.768+0000 7f499be00640 1 -- 172.21.15.60:0/2091712089 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c01e070 con 0x7f49a41e6980 2024-09-16T15:41:48.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.768+0000 7f499be00640 1 -- 172.21.15.60:0/2091712089 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f498c004050 con 0x7f49a41e6980 2024-09-16T15:41:48.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.768+0000 7f499be00640 1 -- 172.21.15.60:0/2091712089 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0044a0 con 0x7f49a41e6980 2024-09-16T15:41:48.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.768+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a41e70a0 con 0x7f49a41e6980 2024-09-16T15:41:48.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.768+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a41e7660 con 0x7f49a41e6980 2024-09-16T15:41:48.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.769+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a40cdbb0 con 0x7f49a41e6980 2024-09-16T15:41:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.770+0000 7f499be00640 1 -- 172.21.15.60:0/2091712089 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f498c021070 con 0x7f49a41e6980 2024-09-16T15:41:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.772+0000 7f499be00640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4980078a00 0x7f498007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.772+0000 7f499be00640 1 -- 172.21.15.60:0/2091712089 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5853+0+0 (secure 0 0 0) 0x7f498c09b7b0 con 0x7f49a41e6980 2024-09-16T15:41:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.772+0000 7f49a3400640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4980078a00 0x7f498007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:48.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.773+0000 7f49a3400640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4980078a00 0x7f498007aec0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4994001860 tx=0x7f4994000ed0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:48.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:48.777+0000 7f499be00640 1 -- 172.21.15.60:0/2091712089 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f498c063470 con 0x7f49a41e6980 2024-09-16T15:41:49.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.016+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f49a412f050 con 0x7f49a41e6980 2024-09-16T15:41:49.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.577+0000 7f499be00640 1 -- 172.21.15.60:0/2091712089 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f498c067ea0 con 0x7f49a41e6980 2024-09-16T15:41:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4980078a00 msgr2=0x7f498007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4980078a00 0x7f498007aec0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4994001860 tx=0x7f4994000ed0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49a41e6980 msgr2=0x7f49a4131090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49a41e6980 0x7f49a4131090 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f498c011b40 tx=0x7f498c009e90 comp rx=0 tx=0).stop 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 shutdown_connections 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f4980078a00 0x7f498007aec0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49a41315d0 0x7f49a4131a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49a41e6980 0x7f49a4131090 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 --2- 172.21.15.60:0/2091712089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4988004d40 0x7f49a41e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 >> 172.21.15.60:0/2091712089 conn(0x7f49a40b1840 msgr2=0x7f49a40aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 shutdown_connections 2024-09-16T15:41:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.580+0000 7f49aa800640 1 -- 172.21.15.60:0/2091712089 wait complete. 2024-09-16T15:41:49.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:41:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.766+0000 7fb605c00640 1 -- 172.21.15.60:0/2118973785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb60015bd60 msgr2=0x7fb6000cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.766+0000 7fb605c00640 1 --2- 172.21.15.60:0/2118973785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb60015bd60 0x7fb6000cdad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009830 tx=0x7fb5f002a520 comp rx=0 tx=0).stop 2024-09-16T15:41:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.766+0000 7fb605c00640 1 -- 172.21.15.60:0/2118973785 shutdown_connections 2024-09-16T15:41:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.766+0000 7fb605c00640 1 --2- 172.21.15.60:0/2118973785 >> v2:172.21.15.100:3300/0 conn(0x7fb60006e910 0x7fb600073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.766+0000 7fb605c00640 1 --2- 172.21.15.60:0/2118973785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb60015bd60 0x7fb6000cdad0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.766+0000 7fb605c00640 1 -- 172.21.15.60:0/2118973785 >> 172.21.15.60:0/2118973785 conn(0x7fb6000b18e0 msgr2=0x7fb6000b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.766+0000 7fb605c00640 1 -- 172.21.15.60:0/2118973785 shutdown_connections 2024-09-16T15:41:49.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.767+0000 7fb605c00640 1 -- 172.21.15.60:0/2118973785 wait complete. 2024-09-16T15:41:49.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.768+0000 7fb605c00640 1 Processor -- start 2024-09-16T15:41:49.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.768+0000 7fb605c00640 1 -- start start 2024-09-16T15:41:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb605c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb60006e910 0x7fb6000b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb605c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb600074290 0x7fb6000cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:49.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb605c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb60015bd60 0x7fb60009cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb605c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb60015ff70 con 0x7fb600074290 2024-09-16T15:41:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb605c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb600074a10 con 0x7fb60006e910 2024-09-16T15:41:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb605c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb6001600f0 con 0x7fb60015bd60 2024-09-16T15:41:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb5f6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb600074290 0x7fb6000cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb5f6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb600074290 0x7fb6000cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39948/0 (socket says 172.21.15.60:39948) 2024-09-16T15:41:49.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.769+0000 7fb5f6000640 1 -- 172.21.15.60:0/2108186523 learned_addr learned my addr 172.21.15.60:0/2108186523 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f6000640 1 -- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb60015bd60 msgr2=0x7fb60009cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f6000640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb60015bd60 0x7fb60009cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f6000640 1 -- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb60006e910 msgr2=0x7fb6000b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5fea00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb60006e910 0x7fb6000b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f6000640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb60006e910 0x7fb6000b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f6000640 1 -- 172.21.15.60:0/2108186523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6001f61a0 con 0x7fb600074290 2024-09-16T15:41:49.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5fea00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb60006e910 0x7fb6000b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f6000640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb600074290 0x7fb6000cb640 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9f0 tx=0x7fb5e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f7400640 1 -- 172.21.15.60:0/2108186523 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800cd80 con 0x7fb600074290 2024-09-16T15:41:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f7400640 1 -- 172.21.15.60:0/2108186523 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5e8010430 con 0x7fb600074290 2024-09-16T15:41:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb5f7400640 1 -- 172.21.15.60:0/2108186523 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8010840 con 0x7fb600074290 2024-09-16T15:41:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6001f5620 con 0x7fb600074290 2024-09-16T15:41:49.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.770+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6001f5be0 con 0x7fb600074290 2024-09-16T15:41:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.772+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60009c8b0 con 0x7fb600074290 2024-09-16T15:41:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.773+0000 7fb5f7400640 1 -- 172.21.15.60:0/2108186523 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fb5e80109e0 con 0x7fb600074290 2024-09-16T15:41:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.774+0000 7fb5f7400640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb5dc078a00 0x7fb5dc07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:49.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.774+0000 7fb5f7400640 1 -- 172.21.15.60:0/2108186523 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5853+0+0 (secure 0 0 0) 0x7fb5e809b8e0 con 0x7fb600074290 2024-09-16T15:41:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.774+0000 7fb5fea00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb5dc078a00 0x7fb5dc07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.775+0000 7fb5fea00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb5dc078a00 0x7fb5dc07aec0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0034000 tx=0x7fb5f0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:49.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.778+0000 7fb5f7400640 1 -- 172.21.15.60:0/2108186523 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5e8063bf0 con 0x7fb600074290 2024-09-16T15:41:49.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:49.973+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb600069360 con 0x7fb600074290 2024-09-16T15:41:50.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.584+0000 7fb5f7400640 1 -- 172.21.15.60:0/2108186523 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) ==== 181+0+0 (secure 0 0 0) 0x7fb5e80634f0 con 0x7fb600074290 2024-09-16T15:41:50.586 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:41:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb5dc078a00 msgr2=0x7fb5dc07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb5dc078a00 0x7fb5dc07aec0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0034000 tx=0x7fb5f0004730 comp rx=0 tx=0).stop 2024-09-16T15:41:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb600074290 msgr2=0x7fb6000cb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb600074290 0x7fb6000cb640 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e9f0 tx=0x7fb5e800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 shutdown_connections 2024-09-16T15:41:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb5dc078a00 0x7fb5dc07aec0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb60015bd60 0x7fb60009cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb600074290 0x7fb6000cb640 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 --2- 172.21.15.60:0/2108186523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb60006e910 0x7fb6000b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 >> 172.21.15.60:0/2108186523 conn(0x7fb6000b18e0 msgr2=0x7fb60015a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 shutdown_connections 2024-09-16T15:41:50.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.588+0000 7fb605c00640 1 -- 172.21.15.60:0/2108186523 wait complete. 2024-09-16T15:41:50.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:41:50.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 -- 172.21.15.60:0/3480984323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2608073de0 msgr2=0x7f2608074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 --2- 172.21.15.60:0/3480984323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2608073de0 0x7f2608074240 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f26080702a0 tx=0x7f25f402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 -- 172.21.15.60:0/3480984323 shutdown_connections 2024-09-16T15:41:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 --2- 172.21.15.60:0/3480984323 >> v2:172.21.15.100:3300/0 conn(0x7f260815c3f0 0x7f260815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 --2- 172.21.15.60:0/3480984323 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2608073de0 0x7f2608074240 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 --2- 172.21.15.60:0/3480984323 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f260815f9a0 0x7f260815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 -- 172.21.15.60:0/3480984323 >> 172.21.15.60:0/3480984323 conn(0x7f26080b18e0 msgr2=0x7f26080b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 -- 172.21.15.60:0/3480984323 shutdown_connections 2024-09-16T15:41:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.779+0000 7f260f400640 1 -- 172.21.15.60:0/3480984323 wait complete. 2024-09-16T15:41:50.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.781+0000 7f260f400640 1 Processor -- start 2024-09-16T15:41:50.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.781+0000 7f260f400640 1 -- start start 2024-09-16T15:41:50.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2608073de0 0x7f26081599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260f400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f260815c3f0 0x7f2608150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f260815f9a0 0x7f26080d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260f400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f26080cd550 con 0x7f260815f9a0 2024-09-16T15:41:50.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f26080cd3d0 con 0x7f2608073de0 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260f400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f260815cde0 con 0x7f260815c3f0 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f260815f9a0 0x7f26080d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f260815f9a0 0x7f26080d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39968/0 (socket says 172.21.15.60:39968) 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260d400640 1 -- 172.21.15.60:0/2964002880 learned_addr learned my addr 172.21.15.60:0/2964002880 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f2607e00640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f260815c3f0 0x7f2608150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260ca00640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2608073de0 0x7f26081599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260d400640 1 -- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f260815c3f0 msgr2=0x7f2608150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260d400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f260815c3f0 0x7f2608150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260d400640 1 -- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2608073de0 msgr2=0x7f26081599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260d400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2608073de0 0x7f26081599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260d400640 1 -- 172.21.15.60:0/2964002880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081585a0 con 0x7f260815f9a0 2024-09-16T15:41:50.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f2607e00640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f260815c3f0 0x7f2608150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.782+0000 7f260ca00640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2608073de0 0x7f26081599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:41:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.783+0000 7f260d400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f260815f9a0 0x7f26080d0420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f260000eaa0 tx=0x7f260000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.783+0000 7f2604c00640 1 -- 172.21.15.60:0/2964002880 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600004b20 con 0x7f260815f9a0 2024-09-16T15:41:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.783+0000 7f2604c00640 1 -- 172.21.15.60:0/2964002880 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2600004db0 con 0x7f260815f9a0 2024-09-16T15:41:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.783+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2608158880 con 0x7f260815f9a0 2024-09-16T15:41:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.783+0000 7f2604c00640 1 -- 172.21.15.60:0/2964002880 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600005280 con 0x7f260815f9a0 2024-09-16T15:41:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.783+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2608158e40 con 0x7f260815f9a0 2024-09-16T15:41:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.784+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2608069360 con 0x7f260815f9a0 2024-09-16T15:41:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.785+0000 7f2604c00640 1 -- 172.21.15.60:0/2964002880 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f2600005420 con 0x7f260815f9a0 2024-09-16T15:41:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.786+0000 7f2604c00640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25e8078a00 0x7f25e807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.786+0000 7f2604c00640 1 -- 172.21.15.60:0/2964002880 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5415+0+0 (secure 0 0 0) 0x7f260009bb80 con 0x7f260815f9a0 2024-09-16T15:41:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.786+0000 7f260ca00640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25e8078a00 0x7f25e807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.787+0000 7f260ca00640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25e8078a00 0x7f25e807aec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ab0 tx=0x7f25fc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:50.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.790+0000 7f2604c00640 1 -- 172.21.15.60:0/2964002880 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2600063ee0 con 0x7f260815f9a0 2024-09-16T15:41:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:50.988+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f260815c8a0 con 0x7f260815f9a0 2024-09-16T15:41:51.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.586+0000 7f2604c00640 1 -- 172.21.15.60:0/2964002880 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) ==== 169+0+0 (secure 0 0 0) 0x7f26000637e0 con 0x7f260815f9a0 2024-09-16T15:41:51.588 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:41:51.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.590+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25e8078a00 msgr2=0x7f25e807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:51.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.590+0000 7f260f400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25e8078a00 0x7f25e807aec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ab0 tx=0x7f25fc005f90 comp rx=0 tx=0).stop 2024-09-16T15:41:51.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.590+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f260815f9a0 msgr2=0x7f26080d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:51.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.590+0000 7f260f400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f260815f9a0 0x7f26080d0420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f260000eaa0 tx=0x7f260000ef60 comp rx=0 tx=0).stop 2024-09-16T15:41:51.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.591+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 shutdown_connections 2024-09-16T15:41:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.591+0000 7f260f400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f25e8078a00 0x7f25e807aec0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.591+0000 7f260f400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f260815f9a0 0x7f26080d0420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.591+0000 7f260f400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f260815c3f0 0x7f2608150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.591+0000 7f260f400640 1 --2- 172.21.15.60:0/2964002880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2608073de0 0x7f26081599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.591+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 >> 172.21.15.60:0/2964002880 conn(0x7f26080b18e0 msgr2=0x7f260815b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.591+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 shutdown_connections 2024-09-16T15:41:51.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.591+0000 7f260f400640 1 -- 172.21.15.60:0/2964002880 wait complete. 2024-09-16T15:41:51.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:41:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.779+0000 7fa2b1000640 1 -- 172.21.15.60:0/360953905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac0cff50 msgr2=0x7fa2ac0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.779+0000 7fa2b1000640 1 --2- 172.21.15.60:0/360953905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac0cff50 0x7fa2ac0b3410 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac06a1b0 tx=0x7fa2a402a7a0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.779+0000 7fa2b1000640 1 -- 172.21.15.60:0/360953905 shutdown_connections 2024-09-16T15:41:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.779+0000 7fa2b1000640 1 --2- 172.21.15.60:0/360953905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac0cff50 0x7fa2ac0b3410 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.779+0000 7fa2b1000640 1 --2- 172.21.15.60:0/360953905 >> v2:172.21.15.100:3300/0 conn(0x7fa2ac161430 0x7fa2ac0b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.780+0000 7fa2b1000640 1 -- 172.21.15.60:0/360953905 >> 172.21.15.60:0/360953905 conn(0x7fa2ac0d3dc0 msgr2=0x7fa2ac0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.780+0000 7fa2b1000640 1 -- 172.21.15.60:0/360953905 shutdown_connections 2024-09-16T15:41:51.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.780+0000 7fa2b1000640 1 -- 172.21.15.60:0/360953905 wait complete. 2024-09-16T15:41:51.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.781+0000 7fa2b1000640 1 Processor -- start 2024-09-16T15:41:51.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.781+0000 7fa2b1000640 1 -- start start 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2b1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac0b4aa0 0x7fa2ac15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2b1000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0cff50 0x7fa2ac0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2b1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac161430 0x7fa2ac0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2b1000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac0b3d10 con 0x7fa2ac161430 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2b1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac0b3b90 con 0x7fa2ac0b4aa0 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2b1000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa2ac0b3e90 con 0x7fa2ac0cff50 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac161430 0x7fa2ac0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac161430 0x7fa2ac0cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39976/0 (socket says 172.21.15.60:39976) 2024-09-16T15:41:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.782+0000 7fa2abe00640 1 -- 172.21.15.60:0/2472716796 learned_addr learned my addr 172.21.15.60:0/2472716796 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.783+0000 7fa2abe00640 1 -- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0cff50 msgr2=0x7fa2ac0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.783+0000 7fa2abe00640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0cff50 0x7fa2ac0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.783+0000 7fa2abe00640 1 -- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac0b4aa0 msgr2=0x7fa2ac15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.783+0000 7fa2ab400640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac0b4aa0 0x7fa2ac15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.783+0000 7fa2abe00640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac0b4aa0 0x7fa2ac15b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.783+0000 7fa2abe00640 1 -- 172.21.15.60:0/2472716796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac0cc6d0 con 0x7fa2ac161430 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.783+0000 7fa2abe00640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac161430 0x7fa2ac0cae00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4034040 tx=0x7fa2a4003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.783+0000 7fa2a3e00640 1 -- 172.21.15.60:0/2472716796 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4041020 con 0x7fa2ac161430 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.784+0000 7fa2a3e00640 1 -- 172.21.15.60:0/2472716796 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2a4004350 con 0x7fa2ac161430 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.784+0000 7fa2a3e00640 1 -- 172.21.15.60:0/2472716796 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4036040 con 0x7fa2ac161430 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.784+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac0cb630 con 0x7fa2ac161430 2024-09-16T15:41:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.784+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac0cbbf0 con 0x7fa2ac161430 2024-09-16T15:41:51.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.785+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ac0b3410 con 0x7fa2ac161430 2024-09-16T15:41:51.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.786+0000 7fa2a3e00640 1 -- 172.21.15.60:0/2472716796 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fa2a4044020 con 0x7fa2ac161430 2024-09-16T15:41:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.787+0000 7fa2a3e00640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa288078a00 0x7fa28807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.787+0000 7fa2a3e00640 1 -- 172.21.15.60:0/2472716796 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7fa2a40bbbf0 con 0x7fa2ac161430 2024-09-16T15:41:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.787+0000 7fa2ab400640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa288078a00 0x7fa28807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.788+0000 7fa2ab400640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa288078a00 0x7fa28807aec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0058c0 tx=0x7fa29c005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.791+0000 7fa2a3e00640 1 -- 172.21.15.60:0/2472716796 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa2a404b030 con 0x7fa2ac161430 2024-09-16T15:41:51.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:51.998+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fa2ac0d2700 con 0x7fa2ac161430 2024-09-16T15:41:52.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.001+0000 7fa2a3e00640 1 -- 172.21.15.60:0/2472716796 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v31) ==== 71+0+0 (secure 0 0 0) 0x7fa2a4083ae0 con 0x7fa2ac161430 2024-09-16T15:41:52.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa288078a00 msgr2=0x7fa28807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa288078a00 0x7fa28807aec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0058c0 tx=0x7fa29c005850 comp rx=0 tx=0).stop 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac161430 msgr2=0x7fa2ac0cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac161430 0x7fa2ac0cae00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4034040 tx=0x7fa2a4003ea0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 shutdown_connections 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa288078a00 0x7fa28807aec0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2ac161430 0x7fa2ac0cae00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2ac0cff50 0x7fa2ac0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 --2- 172.21.15.60:0/2472716796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2ac0b4aa0 0x7fa2ac15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 >> 172.21.15.60:0/2472716796 conn(0x7fa2ac0d3dc0 msgr2=0x7fa2ac0ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 shutdown_connections 2024-09-16T15:41:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.004+0000 7fa2b1000640 1 -- 172.21.15.60:0/2472716796 wait complete. 2024-09-16T15:41:52.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-16T15:41:52.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.181+0000 7f32ce600640 1 -- 172.21.15.60:0/255751615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c806e910 msgr2=0x7f32c80cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.181+0000 7f32ce600640 1 --2- 172.21.15.60:0/255751615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c806e910 0x7f32c80cdad0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f32b0005fd0 tx=0x7f32b002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.181+0000 7f32ce600640 1 -- 172.21.15.60:0/255751615 shutdown_connections 2024-09-16T15:41:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.181+0000 7f32ce600640 1 --2- 172.21.15.60:0/255751615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c806e910 0x7f32c80cdad0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.181+0000 7f32ce600640 1 -- 172.21.15.60:0/255751615 >> 172.21.15.60:0/255751615 conn(0x7f32c80b18e0 msgr2=0x7f32c80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.181+0000 7f32ce600640 1 -- 172.21.15.60:0/255751615 shutdown_connections 2024-09-16T15:41:52.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.181+0000 7f32ce600640 1 -- 172.21.15.60:0/255751615 wait complete. 2024-09-16T15:41:52.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.183+0000 7f32ce600640 1 Processor -- start 2024-09-16T15:41:52.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.183+0000 7f32ce600640 1 -- start start 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32ce600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32c806e910 0x7f32c81e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32ce600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32c8074460 0x7f32c81f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32ce600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c815bd60 0x7f32c809aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32ce600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80cd050 con 0x7f32c815bd60 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32ce600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80cced0 con 0x7f32c8074460 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32ce600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f32c80cd1d0 con 0x7f32c806e910 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c815bd60 0x7f32c809aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32c7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c815bd60 0x7f32c809aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39996/0 (socket says 172.21.15.60:39996) 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32c7e00640 1 -- 172.21.15.60:0/4027606834 learned_addr learned my addr 172.21.15.60:0/4027606834 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32bfe00640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32c8074460 0x7f32c81f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.184+0000 7f32c7400640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32c806e910 0x7f32c81e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32c7e00640 1 -- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32c806e910 msgr2=0x7f32c81e6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32c7e00640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32c806e910 0x7f32c81e6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32c7e00640 1 -- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32c8074460 msgr2=0x7f32c81f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32c7e00640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32c8074460 0x7f32c81f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32c7e00640 1 -- 172.21.15.60:0/4027606834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c81f3f70 con 0x7f32c815bd60 2024-09-16T15:41:52.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32c7e00640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c815bd60 0x7f32c809aed0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c7a0 tx=0x7f32c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32bf400640 1 -- 172.21.15.60:0/4027606834 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0003b70 con 0x7f32c815bd60 2024-09-16T15:41:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32bf400640 1 -- 172.21.15.60:0/4027606834 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32c0003d10 con 0x7f32c815bd60 2024-09-16T15:41:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c809ca20 con 0x7f32c815bd60 2024-09-16T15:41:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32bf400640 1 -- 172.21.15.60:0/4027606834 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c000aab0 con 0x7f32c815bd60 2024-09-16T15:41:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.185+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c809cfe0 con 0x7f32c815bd60 2024-09-16T15:41:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.186+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c8069360 con 0x7f32c815bd60 2024-09-16T15:41:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.187+0000 7f32bf400640 1 -- 172.21.15.60:0/4027606834 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f32c000ac50 con 0x7f32c815bd60 2024-09-16T15:41:52.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.188+0000 7f32bf400640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f32a4078a00 0x7f32a407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:52.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.188+0000 7f32bf400640 1 -- 172.21.15.60:0/4027606834 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f32c009a7b0 con 0x7f32c815bd60 2024-09-16T15:41:52.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.189+0000 7f32c7400640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f32a4078a00 0x7f32a407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:52.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.190+0000 7f32c7400640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f32a4078a00 0x7f32a407aec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f32c81e7cb0 tx=0x7f32b8009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.193+0000 7f32bf400640 1 -- 172.21.15.60:0/4027606834 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f32c815c140 con 0x7f32c815bd60 2024-09-16T15:41:52.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.464+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f32c809acb0 con 0x7f32c815bd60 2024-09-16T15:41:52.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.604+0000 7f32bf400640 1 -- 172.21.15.60:0/4027606834 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v185) ==== 140+0+0 (secure 0 0 0) 0x7f32c00626a0 con 0x7f32c815bd60 2024-09-16T15:41:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f32a4078a00 msgr2=0x7f32a407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f32a4078a00 0x7f32a407aec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f32c81e7cb0 tx=0x7f32b8009060 comp rx=0 tx=0).stop 2024-09-16T15:41:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c815bd60 msgr2=0x7f32c809aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c815bd60 0x7f32c809aed0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c7a0 tx=0x7f32c000cc60 comp rx=0 tx=0).stop 2024-09-16T15:41:52.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 shutdown_connections 2024-09-16T15:41:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f32a4078a00 0x7f32a407aec0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32c815bd60 0x7f32c809aed0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32c8074460 0x7f32c81f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 --2- 172.21.15.60:0/4027606834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32c806e910 0x7f32c81e6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 >> 172.21.15.60:0/4027606834 conn(0x7f32c80b18e0 msgr2=0x7f32c80b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 shutdown_connections 2024-09-16T15:41:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.607+0000 7f32ce600640 1 -- 172.21.15.60:0/4027606834 wait complete. 2024-09-16T15:41:52.625 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-16T15:41:52.625 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-16T15:41:52.625 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:41:52.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:41:52.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.785+0000 7f6e09000640 1 -- 172.21.15.60:0/1877409128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 msgr2=0x7f6de800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.785+0000 7f6e09000640 1 --2- 172.21.15.60:0/1877409128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 0x7f6de800b9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc009ed0 tx=0x7f6de8005830 comp rx=0 tx=0).stop 2024-09-16T15:41:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.785+0000 7f6e02800640 1 -- 172.21.15.60:0/1877409128 reap_dead start 2024-09-16T15:41:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.785+0000 7f6e09000640 1 -- 172.21.15.60:0/1877409128 shutdown_connections 2024-09-16T15:41:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.785+0000 7f6e09000640 1 -- 172.21.15.60:0/1877409128 >> 172.21.15.60:0/1877409128 conn(0x7f6e040b1950 msgr2=0x7f6e040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.785+0000 7f6e09000640 1 -- 172.21.15.60:0/1877409128 shutdown_connections 2024-09-16T15:41:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.785+0000 7f6e09000640 1 -- 172.21.15.60:0/1877409128 wait complete. 2024-09-16T15:41:52.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.787+0000 7f6e09000640 1 Processor -- start 2024-09-16T15:41:52.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.787+0000 7f6e09000640 1 -- start start 2024-09-16T15:41:52.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e09000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 0x7f6e04150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e09000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e04156ff0 0x7f6e040b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e09000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e041fcc60 0x7f6e040b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e09000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6e040b25c0 con 0x7f6de80051c0 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e09000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6e040b2440 con 0x7f6e041fcc60 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e09000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6e040b2740 con 0x7f6e04156ff0 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e01e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 0x7f6e04150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e01e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 0x7f6e04150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40004/0 (socket says 172.21.15.60:40004) 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e01e00640 1 -- 172.21.15.60:0/1643037867 learned_addr learned my addr 172.21.15.60:0/1643037867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6e02800640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e041fcc60 0x7f6e040b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.788+0000 7f6df9400640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e04156ff0 0x7f6e040b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e01e00640 1 -- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e04156ff0 msgr2=0x7f6e040b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e01e00640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e04156ff0 0x7f6e040b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e01e00640 1 -- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e041fcc60 msgr2=0x7f6e040b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e01e00640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e041fcc60 0x7f6e040b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e01e00640 1 -- 172.21.15.60:0/1643037867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e040b3650 con 0x7f6de80051c0 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e02800640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e041fcc60 0x7f6e040b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e01e00640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 0x7f6e04150110 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6df4010940 tx=0x7f6df4010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6dfaa00640 1 -- 172.21.15.60:0/1643037867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df400ed40 con 0x7f6de80051c0 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6dfaa00640 1 -- 172.21.15.60:0/1643037867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6df4005020 con 0x7f6de80051c0 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e040b3930 con 0x7f6de80051c0 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6dfaa00640 1 -- 172.21.15.60:0/1643037867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4005430 con 0x7f6de80051c0 2024-09-16T15:41:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.789+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e041fc460 con 0x7f6de80051c0 2024-09-16T15:41:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.790+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e04074460 con 0x7f6de80051c0 2024-09-16T15:41:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.791+0000 7f6dfaa00640 1 -- 172.21.15.60:0/1643037867 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f6df40047a0 con 0x7f6de80051c0 2024-09-16T15:41:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.792+0000 7f6dfaa00640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6de0078a00 0x7f6de007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.792+0000 7f6dfaa00640 1 -- 172.21.15.60:0/1643037867 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f6df409b8b0 con 0x7f6de80051c0 2024-09-16T15:41:52.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.793+0000 7f6df9400640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6de0078a00 0x7f6de007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.794+0000 7f6df9400640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6de0078a00 0x7f6de007aec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6dec001030 tx=0x7f6dec001740 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:52.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.797+0000 7f6dfaa00640 1 -- 172.21.15.60:0/1643037867 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6df4063df0 con 0x7f6de80051c0 2024-09-16T15:41:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.995+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e0409af30 con 0x7f6de80051c0 2024-09-16T15:41:52.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.996+0000 7f6dfaa00640 1 -- 172.21.15.60:0/1643037867 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+13032 (secure 0 0 0) 0x7f6df40636f0 con 0x7f6de80051c0 2024-09-16T15:41:52.997 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:52.997 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":53,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:41:51.585167+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2823237251":"2024-09-17T15:41:48.574843+0000","172.21.15.60:6834/2823237251":"2024-09-17T15:41:48.574843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6de0078a00 msgr2=0x7f6de007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6de0078a00 0x7f6de007aec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6dec001030 tx=0x7f6dec001740 comp rx=0 tx=0).stop 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 msgr2=0x7f6e04150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 0x7f6e04150110 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6df4010940 tx=0x7f6df4010e00 comp rx=0 tx=0).stop 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 shutdown_connections 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6de0078a00 0x7f6de007aec0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e041fcc60 0x7f6e040b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e04156ff0 0x7f6e040b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 --2- 172.21.15.60:0/1643037867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6de80051c0 0x7f6e04150110 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 >> 172.21.15.60:0/1643037867 conn(0x7f6e040b1950 msgr2=0x7f6e04075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.998+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 shutdown_connections 2024-09-16T15:41:53.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:52.999+0000 7f6e09000640 1 -- 172.21.15.60:0/1643037867 wait complete. 2024-09-16T15:41:53.015 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:41:53.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.178+0000 7fc4d2200640 1 -- 172.21.15.60:0/684899429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 msgr2=0x7fc4cc15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.178+0000 7fc4d2200640 1 --2- 172.21.15.60:0/684899429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 0x7fc4cc15f9f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0d0cd0 tx=0x7fc4c402a980 comp rx=0 tx=0).stop 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.178+0000 7fc4d2200640 1 -- 172.21.15.60:0/684899429 shutdown_connections 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.178+0000 7fc4d2200640 1 --2- 172.21.15.60:0/684899429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 0x7fc4cc15f9f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.178+0000 7fc4d2200640 1 --2- 172.21.15.60:0/684899429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4cc06efa0 0x7fc4cc0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.178+0000 7fc4d2200640 1 --2- 172.21.15.60:0/684899429 >> v2:172.21.15.100:3300/0 conn(0x7fc4cc145750 0x7fc4cc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.178+0000 7fc4d2200640 1 -- 172.21.15.60:0/684899429 >> 172.21.15.60:0/684899429 conn(0x7fc4cc0b1840 msgr2=0x7fc4cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.178+0000 7fc4d2200640 1 -- 172.21.15.60:0/684899429 shutdown_connections 2024-09-16T15:41:53.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.179+0000 7fc4d2200640 1 -- 172.21.15.60:0/684899429 wait complete. 2024-09-16T15:41:53.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.180+0000 7fc4d2200640 1 Processor -- start 2024-09-16T15:41:53.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.180+0000 7fc4d2200640 1 -- start start 2024-09-16T15:41:53.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.181+0000 7fc4d2200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4cc06efa0 0x7fc4cc1f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:53.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.181+0000 7fc4d2200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 0x7fc4cc1f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:53.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.181+0000 7fc4d2200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4cc145750 0x7fc4cc1f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:53.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.181+0000 7fc4d2200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1600b0 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.181+0000 7fc4d2200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc15ff30 con 0x7fc4cc06efa0 2024-09-16T15:41:53.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.181+0000 7fc4d2200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc4cc160230 con 0x7fc4cc145750 2024-09-16T15:41:53.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.181+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 0x7fc4cc1f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 0x7fc4cc1f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40030/0 (socket says 172.21.15.60:40030) 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3e00640 1 -- 172.21.15.60:0/1928161362 learned_addr learned my addr 172.21.15.60:0/1928161362 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4cbe00640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4cc145750 0x7fc4cc1f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4cb400640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4cc06efa0 0x7fc4cc1f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3e00640 1 -- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4cc145750 msgr2=0x7fc4cc1f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3e00640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4cc145750 0x7fc4cc1f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3e00640 1 -- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4cc06efa0 msgr2=0x7fc4cc1f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3e00640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4cc06efa0 0x7fc4cc1f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3e00640 1 -- 172.21.15.60:0/1928161362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc1f5aa0 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3e00640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 0x7fc4cc1f81c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d770 tx=0x7fc4b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:53.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.182+0000 7fc4c3400640 1 -- 172.21.15.60:0/1928161362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4017070 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.183+0000 7fc4c3400640 1 -- 172.21.15.60:0/1928161362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b4009d20 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.183+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc1f5d80 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.183+0000 7fc4c3400640 1 -- 172.21.15.60:0/1928161362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40106c0 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.183+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc1f57a0 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.184+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4cc160710 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.185+0000 7fc4c3400640 1 -- 172.21.15.60:0/1928161362 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fc4b401a070 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.186+0000 7fc4c3400640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4a8078a00 0x7fc4a807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:53.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.186+0000 7fc4c3400640 1 -- 172.21.15.60:0/1928161362 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7fc4b409aad0 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.186+0000 7fc4cb400640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4a8078a00 0x7fc4a807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:53.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.187+0000 7fc4cb400640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4a8078a00 0x7fc4a807aec0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0aeae0 tx=0x7fc4bc00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:53.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.190+0000 7fc4c3400640 1 -- 172.21.15.60:0/1928161362 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc4b4062a70 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.383+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4cc15f650 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.384+0000 7fc4c3400640 1 -- 172.21.15.60:0/1928161362 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) ==== 93+0+10 (secure 0 0 0) 0x7fc4b40674a0 con 0x7fc4cc0ccb00 2024-09-16T15:41:53.385 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:41:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4a8078a00 msgr2=0x7fc4a807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4a8078a00 0x7fc4a807aec0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0aeae0 tx=0x7fc4bc00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 msgr2=0x7fc4cc1f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 0x7fc4cc1f81c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d770 tx=0x7fc4b400dc30 comp rx=0 tx=0).stop 2024-09-16T15:41:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 shutdown_connections 2024-09-16T15:41:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4a8078a00 0x7fc4a807aec0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4cc145750 0x7fc4cc1f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4cc0ccb00 0x7fc4cc1f81c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 --2- 172.21.15.60:0/1928161362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4cc06efa0 0x7fc4cc1f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 >> 172.21.15.60:0/1928161362 conn(0x7fc4cc0b1840 msgr2=0x7fc4cc0af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 shutdown_connections 2024-09-16T15:41:53.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.386+0000 7fc4d2200640 1 -- 172.21.15.60:0/1928161362 wait complete. 2024-09-16T15:41:53.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-16T15:41:53.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 -- 172.21.15.60:0/711083019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ccf00 msgr2=0x7fabd40acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 --2- 172.21.15.60:0/711083019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ccf00 0x7fabd40acd30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fabd4076720 tx=0x7fabc002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 -- 172.21.15.60:0/711083019 shutdown_connections 2024-09-16T15:41:53.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 --2- 172.21.15.60:0/711083019 >> v2:172.21.15.100:3300/0 conn(0x7fabd40ad270 0x7fabd4073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 --2- 172.21.15.60:0/711083019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ccf00 0x7fabd40acd30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 --2- 172.21.15.60:0/711083019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415f5a0 0x7fabd40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 -- 172.21.15.60:0/711083019 >> 172.21.15.60:0/711083019 conn(0x7fabd40b1950 msgr2=0x7fabd40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:53.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 -- 172.21.15.60:0/711083019 shutdown_connections 2024-09-16T15:41:53.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.563+0000 7fabd9800640 1 -- 172.21.15.60:0/711083019 wait complete. 2024-09-16T15:41:53.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.565+0000 7fabd9800640 1 Processor -- start 2024-09-16T15:41:53.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.565+0000 7fabd9800640 1 -- start start 2024-09-16T15:41:53.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ad270 0x7fabd40cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:53.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd40ccf00 0x7fabd41446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:53.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd9800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415f5a0 0x7fabd4156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:53.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd9800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fabd406f160 con 0x7fabd40ad270 2024-09-16T15:41:53.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4074780 con 0x7fabd40ccf00 2024-09-16T15:41:53.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd9800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fabd406f2e0 con 0x7fabd415f5a0 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ad270 0x7fabd40cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ad270 0x7fabd40cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40054/0 (socket says 172.21.15.60:40054) 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2a00640 1 -- 172.21.15.60:0/4165042071 learned_addr learned my addr 172.21.15.60:0/4165042071 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2000640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd40ccf00 0x7fabd41446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd3400640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415f5a0 0x7fabd4156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2a00640 1 -- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415f5a0 msgr2=0x7fabd4156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2a00640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415f5a0 0x7fabd4156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2a00640 1 -- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd40ccf00 msgr2=0x7fabd41446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2a00640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd40ccf00 0x7fabd41446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.566+0000 7fabd2a00640 1 -- 172.21.15.60:0/4165042071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd40cfc40 con 0x7fabd40ad270 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.567+0000 7fabd2a00640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ad270 0x7fabd40cf560 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fabc400d6d0 tx=0x7fabc400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.567+0000 7fabcaa00640 1 -- 172.21.15.60:0/4165042071 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4017070 con 0x7fabd40ad270 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.567+0000 7fabcaa00640 1 -- 172.21.15.60:0/4165042071 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabc4005330 con 0x7fabd40ad270 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.567+0000 7fabcaa00640 1 -- 172.21.15.60:0/4165042071 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4009820 con 0x7fabd40ad270 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.567+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd40cff20 con 0x7fabd40ad270 2024-09-16T15:41:53.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.567+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd40d04e0 con 0x7fabd40ad270 2024-09-16T15:41:53.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.568+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd406f870 con 0x7fabd40ad270 2024-09-16T15:41:53.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.569+0000 7fabcaa00640 1 -- 172.21.15.60:0/4165042071 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fabc4009bd0 con 0x7fabd40ad270 2024-09-16T15:41:53.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.570+0000 7fabcaa00640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabb4078a00 0x7fabb407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:53.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.570+0000 7fabcaa00640 1 -- 172.21.15.60:0/4165042071 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7fabc409ace0 con 0x7fabd40ad270 2024-09-16T15:41:53.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.571+0000 7fabd2000640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabb4078a00 0x7fabb407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:53.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.571+0000 7fabd2000640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabb4078a00 0x7fabb407aec0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fabc0033000 tx=0x7fabc0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:53.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.575+0000 7fabcaa00640 1 -- 172.21.15.60:0/4165042071 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fabc40632b0 con 0x7fabd40ad270 2024-09-16T15:41:53.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:53.844+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fabd40aeaf0 con 0x7fabd40ad270 2024-09-16T15:41:54.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.612+0000 7fabcaa00640 1 -- 172.21.15.60:0/4165042071 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v187) ==== 164+0+0 (secure 0 0 0) 0x7fabc4062bb0 con 0x7fabd40ad270 2024-09-16T15:41:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabb4078a00 msgr2=0x7fabb407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabb4078a00 0x7fabb407aec0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fabc0033000 tx=0x7fabc0004730 comp rx=0 tx=0).stop 2024-09-16T15:41:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ad270 msgr2=0x7fabd40cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ad270 0x7fabd40cf560 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fabc400d6d0 tx=0x7fabc400db90 comp rx=0 tx=0).stop 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 shutdown_connections 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabb4078a00 0x7fabb407aec0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415f5a0 0x7fabd4156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabd40ccf00 0x7fabd41446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 --2- 172.21.15.60:0/4165042071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd40ad270 0x7fabd40cf560 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 >> 172.21.15.60:0/4165042071 conn(0x7fabd40b1950 msgr2=0x7fabd4154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 shutdown_connections 2024-09-16T15:41:54.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.615+0000 7fabd9800640 1 -- 172.21.15.60:0/4165042071 wait complete. 2024-09-16T15:41:54.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:41:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.798+0000 7f2906c00640 1 -- 172.21.15.60:0/1909075477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29000725d0 msgr2=0x7f2900073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.798+0000 7f2906c00640 1 --2- 172.21.15.60:0/1909075477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29000725d0 0x7f2900073d40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009830 tx=0x7f28ec02a410 comp rx=0 tx=0).stop 2024-09-16T15:41:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.799+0000 7f2906c00640 1 -- 172.21.15.60:0/1909075477 shutdown_connections 2024-09-16T15:41:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.799+0000 7f2906c00640 1 --2- 172.21.15.60:0/1909075477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29000725d0 0x7f2900073d40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.799+0000 7f2906c00640 1 --2- 172.21.15.60:0/1909075477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f290015bc30 0x7f29000b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.799+0000 7f2906c00640 1 -- 172.21.15.60:0/1909075477 >> 172.21.15.60:0/1909075477 conn(0x7f29000cfa20 msgr2=0x7f29000cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.799+0000 7f2906c00640 1 -- 172.21.15.60:0/1909075477 shutdown_connections 2024-09-16T15:41:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.799+0000 7f2906c00640 1 -- 172.21.15.60:0/1909075477 wait complete. 2024-09-16T15:41:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.800+0000 7f2906c00640 1 Processor -- start 2024-09-16T15:41:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.800+0000 7f2906c00640 1 -- start start 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.801+0000 7f2906c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29000d0540 0x7f290014ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2906c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f290015bc30 0x7f29001f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2906c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2900151ed0 0x7f2900152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2906c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2900074610 con 0x7f290015bc30 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2906c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2900074490 con 0x7f2900151ed0 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2906c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2900074790 con 0x7f29000d0540 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2904e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f290015bc30 0x7f29001f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2904e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f290015bc30 0x7f29001f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40062/0 (socket says 172.21.15.60:40062) 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2904e00640 1 -- 172.21.15.60:0/1494661033 learned_addr learned my addr 172.21.15.60:0/1494661033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2905800640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29000d0540 0x7f290014ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2906200640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2900151ed0 0x7f2900152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2904e00640 1 -- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29000d0540 msgr2=0x7f290014ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2904e00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29000d0540 0x7f290014ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2904e00640 1 -- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2900151ed0 msgr2=0x7f2900152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2904e00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2900151ed0 0x7f2900152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.802+0000 7f2904e00640 1 -- 172.21.15.60:0/1494661033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29000af290 con 0x7f290015bc30 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.803+0000 7f2904e00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f290015bc30 0x7f29001f83c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009950 tx=0x7f28ec033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.803+0000 7f28fd600640 1 -- 172.21.15.60:0/1494661033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec03f020 con 0x7f290015bc30 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.803+0000 7f28fd600640 1 -- 172.21.15.60:0/1494661033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28ec033d90 con 0x7f290015bc30 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.803+0000 7f28fd600640 1 -- 172.21.15.60:0/1494661033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec037040 con 0x7f290015bc30 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.803+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29000aecd0 con 0x7f290015bc30 2024-09-16T15:41:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.803+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29001f7bc0 con 0x7f290015bc30 2024-09-16T15:41:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.804+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f290006a180 con 0x7f290015bc30 2024-09-16T15:41:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.805+0000 7f28fd600640 1 -- 172.21.15.60:0/1494661033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f28ec042020 con 0x7f290015bc30 2024-09-16T15:41:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.806+0000 7f28fd600640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f28e0078a00 0x7f28e007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.806+0000 7f28fd600640 1 -- 172.21.15.60:0/1494661033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f28ec0ba8b0 con 0x7f290015bc30 2024-09-16T15:41:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.807+0000 7f2905800640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f28e0078a00 0x7f28e007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.808+0000 7f2905800640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f28e0078a00 0x7f28e007aec0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f28f4005fd0 tx=0x7f28f4005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:54.810+0000 7f28fd600640 1 -- 172.21.15.60:0/1494661033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f290006a180 con 0x7f290015bc30 2024-09-16T15:41:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.004+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f290006fcb0 con 0x7f290015bc30 2024-09-16T15:41:55.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.005+0000 7f28fd600640 1 -- 172.21.15.60:0/1494661033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+1998 (secure 0 0 0) 0x7f28ec049030 con 0x7f290015bc30 2024-09-16T15:41:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:41:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:41:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:41:55.006 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:41:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:41:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:41:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:41:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:41:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:41:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:41:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:41:55.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:41:55.008 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:41:55.009 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:41:55.009 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:41:55.009 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:41:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f28e0078a00 msgr2=0x7f28e007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f28e0078a00 0x7f28e007aec0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f28f4005fd0 tx=0x7f28f4005850 comp rx=0 tx=0).stop 2024-09-16T15:41:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f290015bc30 msgr2=0x7f29001f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f290015bc30 0x7f29001f83c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009950 tx=0x7f28ec033ba0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 shutdown_connections 2024-09-16T15:41:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f28e0078a00 0x7f28e007aec0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2900151ed0 0x7f2900152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f290015bc30 0x7f29001f83c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 --2- 172.21.15.60:0/1494661033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29000d0540 0x7f290014ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 >> 172.21.15.60:0/1494661033 conn(0x7f29000cfa20 msgr2=0x7f29000cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 shutdown_connections 2024-09-16T15:41:55.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.007+0000 7f2906c00640 1 -- 172.21.15.60:0/1494661033 wait complete. 2024-09-16T15:41:55.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:41:55.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.187+0000 7f8171600640 1 -- 172.21.15.60:0/3893305171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c0cbc00 msgr2=0x7f816c073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.187+0000 7f8171600640 1 --2- 172.21.15.60:0/3893305171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c0cbc00 0x7f816c073ef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f816c0c9ff0 tx=0x7f816402aac0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.187+0000 7f8171600640 1 -- 172.21.15.60:0/3893305171 shutdown_connections 2024-09-16T15:41:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.187+0000 7f8171600640 1 --2- 172.21.15.60:0/3893305171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c0cbc00 0x7f816c073ef0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.187+0000 7f8171600640 1 --2- 172.21.15.60:0/3893305171 >> v2:172.21.15.100:3300/0 conn(0x7f816c15bec0 0x7f816c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.187+0000 7f8171600640 1 -- 172.21.15.60:0/3893305171 >> 172.21.15.60:0/3893305171 conn(0x7f816c0cf9f0 msgr2=0x7f816c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.187+0000 7f8171600640 1 -- 172.21.15.60:0/3893305171 shutdown_connections 2024-09-16T15:41:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.187+0000 7f8171600640 1 -- 172.21.15.60:0/3893305171 wait complete. 2024-09-16T15:41:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.189+0000 7f8171600640 1 Processor -- start 2024-09-16T15:41:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.189+0000 7f8171600640 1 -- start start 2024-09-16T15:41:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8171600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c072560 0x7f816c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8171600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c0cbc00 0x7f816c0ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8171600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c15bec0 0x7f816c150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8171600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0ae990 con 0x7f816c15bec0 2024-09-16T15:41:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8171600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0ae810 con 0x7f816c072560 2024-09-16T15:41:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8171600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f816c0aeb10 con 0x7f816c0cbc00 2024-09-16T15:41:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8170c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c15bec0 0x7f816c150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8170c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c15bec0 0x7f816c150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40088/0 (socket says 172.21.15.60:40088) 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8170c00640 1 -- 172.21.15.60:0/1772501641 learned_addr learned my addr 172.21.15.60:0/1772501641 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f816be00640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c072560 0x7f816c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f816b400640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c0cbc00 0x7f816c0ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8170c00640 1 -- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c0cbc00 msgr2=0x7f816c0ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8170c00640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c0cbc00 0x7f816c0ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8170c00640 1 -- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c072560 msgr2=0x7f816c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8170c00640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c072560 0x7f816c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f8170c00640 1 -- 172.21.15.60:0/1772501641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c1f5750 con 0x7f816c15bec0 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.190+0000 7f816b400640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c0cbc00 0x7f816c0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.191+0000 7f8170c00640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c15bec0 0x7f816c150930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f816402b0b0 tx=0x7f8164003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.191+0000 7f8163e00640 1 -- 172.21.15.60:0/1772501641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164044070 con 0x7f816c15bec0 2024-09-16T15:41:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.191+0000 7f8163e00640 1 -- 172.21.15.60:0/1772501641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8164004250 con 0x7f816c15bec0 2024-09-16T15:41:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.191+0000 7f8163e00640 1 -- 172.21.15.60:0/1772501641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164004680 con 0x7f816c15bec0 2024-09-16T15:41:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.191+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c150130 con 0x7f816c15bec0 2024-09-16T15:41:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.191+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c1f5460 con 0x7f816c15bec0 2024-09-16T15:41:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.192+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c0aefc0 con 0x7f816c15bec0 2024-09-16T15:41:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.193+0000 7f8163e00640 1 -- 172.21.15.60:0/1772501641 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f816402c840 con 0x7f816c15bec0 2024-09-16T15:41:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.194+0000 7f8163e00640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f814c078a00 0x7f814c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.194+0000 7f8163e00640 1 -- 172.21.15.60:0/1772501641 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f81640baf70 con 0x7f816c15bec0 2024-09-16T15:41:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.195+0000 7f816be00640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f814c078a00 0x7f814c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.195+0000 7f816be00640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f814c078a00 0x7f814c07aec0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f815c005fd0 tx=0x7f815c00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.198+0000 7f8163e00640 1 -- 172.21.15.60:0/1772501641 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8164032030 con 0x7f816c15bec0 2024-09-16T15:41:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.394+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f816c0cd5c0 con 0x7f816c15bec0 2024-09-16T15:41:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.398+0000 7f8163e00640 1 -- 172.21.15.60:0/1772501641 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f8164004820 con 0x7f816c15bec0 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f814c078a00 msgr2=0x7f814c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f814c078a00 0x7f814c07aec0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f815c005fd0 tx=0x7f815c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c15bec0 msgr2=0x7f816c150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c15bec0 0x7f816c150930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f816402b0b0 tx=0x7f8164003f10 comp rx=0 tx=0).stop 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 shutdown_connections 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f814c078a00 0x7f814c07aec0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c15bec0 0x7f816c150930 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c0cbc00 0x7f816c0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 --2- 172.21.15.60:0/1772501641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c072560 0x7f816c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 >> 172.21.15.60:0/1772501641 conn(0x7f816c0cf9f0 msgr2=0x7f816c15e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 shutdown_connections 2024-09-16T15:41:55.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.400+0000 7f8171600640 1 -- 172.21.15.60:0/1772501641 wait complete. 2024-09-16T15:41:55.417 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:41:55.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:41:55.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:41:55.588 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:41:55.588 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:41:55.588 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:55.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:41:55.624 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:41:55.624 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:55.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:41:55.661 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:41:55.661 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:55.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:41:55.699 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:41:55.699 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:55.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:41:55.738 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:41:55.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:41:55.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.898+0000 7f0614a00640 1 -- 172.21.15.60:0/906688167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0610074ca0 msgr2=0x7f06100cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.898+0000 7f0614a00640 1 --2- 172.21.15.60:0/906688167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0610074ca0 0x7f06100cc860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0610074110 tx=0x7f05f802a540 comp rx=0 tx=0).stop 2024-09-16T15:41:55.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.899+0000 7f0614a00640 1 -- 172.21.15.60:0/906688167 shutdown_connections 2024-09-16T15:41:55.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.899+0000 7f0614a00640 1 --2- 172.21.15.60:0/906688167 >> v2:172.21.15.100:3300/0 conn(0x7f06100afde0 0x7f06100b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.899+0000 7f0614a00640 1 --2- 172.21.15.60:0/906688167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0610074ca0 0x7f06100cc860 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.899+0000 7f0614a00640 1 -- 172.21.15.60:0/906688167 >> 172.21.15.60:0/906688167 conn(0x7f06100b1e90 msgr2=0x7f06100b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.899+0000 7f0614a00640 1 -- 172.21.15.60:0/906688167 shutdown_connections 2024-09-16T15:41:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.899+0000 7f0614a00640 1 -- 172.21.15.60:0/906688167 wait complete. 2024-09-16T15:41:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.900+0000 7f0614a00640 1 Processor -- start 2024-09-16T15:41:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.900+0000 7f0614a00640 1 -- start start 2024-09-16T15:41:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.901+0000 7f0614a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0610074ca0 0x7f061009da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.901+0000 7f0614a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06100afde0 0x7f06100c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.901+0000 7f0614a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06100ce100 0x7f06100c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.901+0000 7f0614a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f06100b0cf0 con 0x7f06100afde0 2024-09-16T15:41:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.901+0000 7f0614a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06100b0b70 con 0x7f0610074ca0 2024-09-16T15:41:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.901+0000 7f0614a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f06100b0e70 con 0x7f06100ce100 2024-09-16T15:41:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.901+0000 7f0607e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06100afde0 0x7f06100c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06100afde0 0x7f06100c4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40116/0 (socket says 172.21.15.60:40116) 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607e00640 1 -- 172.21.15.60:0/1719820380 learned_addr learned my addr 172.21.15.60:0/1719820380 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f060fe00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06100ce100 0x7f06100c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f060f400640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0610074ca0 0x7f061009da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607e00640 1 -- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06100ce100 msgr2=0x7f06100c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607e00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06100ce100 0x7f06100c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607e00640 1 -- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0610074ca0 msgr2=0x7f061009da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607e00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0610074ca0 0x7f061009da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607e00640 1 -- 172.21.15.60:0/1719820380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06100b25c0 con 0x7f06100afde0 2024-09-16T15:41:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607e00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06100afde0 0x7f06100c4540 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009780 tx=0x7f05f8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.902+0000 7f0607400640 1 -- 172.21.15.60:0/1719820380 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8031d60 con 0x7f06100afde0 2024-09-16T15:41:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.903+0000 7f0607400640 1 -- 172.21.15.60:0/1719820380 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05f8033e10 con 0x7f06100afde0 2024-09-16T15:41:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.903+0000 7f0607400640 1 -- 172.21.15.60:0/1719820380 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8005260 con 0x7f06100afde0 2024-09-16T15:41:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.903+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0610146a10 con 0x7f06100afde0 2024-09-16T15:41:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.903+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06100b2270 con 0x7f06100afde0 2024-09-16T15:41:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.904+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06100cc860 con 0x7f06100afde0 2024-09-16T15:41:55.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.905+0000 7f0607400640 1 -- 172.21.15.60:0/1719820380 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f05f8005400 con 0x7f06100afde0 2024-09-16T15:41:55.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.906+0000 7f0607400640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f05ec078a30 0x7f05ec07aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:55.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.906+0000 7f0607400640 1 -- 172.21.15.60:0/1719820380 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f05f80bbb20 con 0x7f06100afde0 2024-09-16T15:41:55.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.906+0000 7f060f400640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f05ec078a30 0x7f05ec07aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:55.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.907+0000 7f060f400640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f05ec078a30 0x7f05ec07aef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0600001700 tx=0x7f0600009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:55.910+0000 7f0607400640 1 -- 172.21.15.60:0/1719820380 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f05f803f030 con 0x7f06100afde0 2024-09-16T15:41:56.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.140+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0610159000 con 0x7f06100afde0 2024-09-16T15:41:56.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.141+0000 7f0607400640 1 -- 172.21.15.60:0/1719820380 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3914 (secure 0 0 0) 0x7f05f8084070 con 0x7f06100afde0 2024-09-16T15:41:56.142 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:56.142 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":29,"btime":"2024-09-16T15:41:49:576617+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4991,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/1386527013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1386527013},{"type":"v1","addr":"172.21.15.100:6836","nonce":1386527013}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":4994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4203331462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":4203331462},{"type":"v1","addr":"172.21.15.100:6837","nonce":4203331462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/618545844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":618545844},{"type":"v1","addr":"172.21.15.60:6833","nonce":618545844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2035503754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2035503754},{"type":"v1","addr":"172.21.15.60:6835","nonce":2035503754}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-16T15:41:56.143 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 29 2024-09-16T15:41:56.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f05ec078a30 msgr2=0x7f05ec07aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f05ec078a30 0x7f05ec07aef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0600001700 tx=0x7f0600009040 comp rx=0 tx=0).stop 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06100afde0 msgr2=0x7f06100c4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06100afde0 0x7f06100c4540 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009780 tx=0x7f05f8033c20 comp rx=0 tx=0).stop 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 shutdown_connections 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f05ec078a30 0x7f05ec07aef0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06100ce100 0x7f06100c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06100afde0 0x7f06100c4540 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 --2- 172.21.15.60:0/1719820380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0610074ca0 0x7f061009da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 >> 172.21.15.60:0/1719820380 conn(0x7f06100b1e90 msgr2=0x7f06100ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 shutdown_connections 2024-09-16T15:41:56.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.143+0000 7f0614a00640 1 -- 172.21.15.60:0/1719820380 wait complete. 2024-09-16T15:41:56.160 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:41:56.160 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:41:56.161 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:41:56.161 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:41:56.161 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:41:56.161 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:41:56.161 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:41:56.161 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:41:56.161 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:41:56.162 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:41:56.162 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:41:56.162 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:41:56.162 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:41:56.162 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:41:56.162 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:41:56.163 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:41:56.178 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:41:56.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:41:56.179 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:41:56.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:41:56.180 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:41:56.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:41:56.181 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:41:56.181 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:41:56.222 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:41:56.222 DEBUG:teuthology.parallel:result is None 2024-09-16T15:41:56.224 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:41:56.224 DEBUG:teuthology.parallel:result is None 2024-09-16T15:41:56.226 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:41:56.226 DEBUG:teuthology.parallel:result is None 2024-09-16T15:41:56.228 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:41:56.228 DEBUG:teuthology.parallel:result is None 2024-09-16T15:41:56.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:41:56.354 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:41:56.355 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:41:56.358 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:41:56.359 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:41:56.361 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-09-16T15:41:56.359+0000 7fed80400640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-16T15:41:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.396+0000 7f65d0200640 1 -- 172.21.15.60:0/596446551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c80cf300 msgr2=0x7f65c80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.396+0000 7f65d0200640 1 --2- 172.21.15.60:0/596446551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c80cf300 0x7f65c80729a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009860 tx=0x7f65b402a5f0 comp rx=0 tx=0).stop 2024-09-16T15:41:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.396+0000 7f65d0200640 1 -- 172.21.15.60:0/596446551 shutdown_connections 2024-09-16T15:41:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.396+0000 7f65d0200640 1 --2- 172.21.15.60:0/596446551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c80cf300 0x7f65c80729a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.396+0000 7f65d0200640 1 -- 172.21.15.60:0/596446551 >> 172.21.15.60:0/596446551 conn(0x7f65c80b1950 msgr2=0x7f65c80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.396+0000 7f65d0200640 1 -- 172.21.15.60:0/596446551 shutdown_connections 2024-09-16T15:41:56.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.396+0000 7f65d0200640 1 -- 172.21.15.60:0/596446551 wait complete. 2024-09-16T15:41:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.398+0000 7f65d0200640 1 Processor -- start 2024-09-16T15:41:56.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.398+0000 7f65d0200640 1 -- start start 2024-09-16T15:41:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65d0200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65c80ccfd0 0x7f65c813b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65d0200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65c80cf300 0x7f65c80cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65d0200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c81445d0 0x7f65c81f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65d0200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80735f0 con 0x7f65c81445d0 2024-09-16T15:41:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65d0200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80b4f60 con 0x7f65c80cf300 2024-09-16T15:41:56.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65d0200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f65c8073770 con 0x7f65c80ccfd0 2024-09-16T15:41:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65cd800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65c80ccfd0 0x7f65c813b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65ce200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c81445d0 0x7f65c81f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65cd800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65c80ccfd0 0x7f65c813b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56856/0 (socket says 172.21.15.60:56856) 2024-09-16T15:41:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65cce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65c80cf300 0x7f65c80cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.399+0000 7f65cd800640 1 -- 172.21.15.60:0/3307175120 learned_addr learned my addr 172.21.15.60:0/3307175120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65cd800640 1 -- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65c80cf300 msgr2=0x7f65c80cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65cd800640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65c80cf300 0x7f65c80cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65cd800640 1 -- 172.21.15.60:0/3307175120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c81445d0 msgr2=0x7f65c81f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65cd800640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c81445d0 0x7f65c81f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65cd800640 1 -- 172.21.15.60:0/3307175120 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c813bb10 con 0x7f65c80ccfd0 2024-09-16T15:41:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65ce200640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c81445d0 0x7f65c81f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:41:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65cd800640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65c80ccfd0 0x7f65c813b5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c400 tx=0x7f65bc00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65c5600640 1 -- 172.21.15.60:0/3307175120 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc017070 con 0x7f65c80ccfd0 2024-09-16T15:41:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65c5600640 1 -- 172.21.15.60:0/3307175120 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65bc0053c0 con 0x7f65c80ccfd0 2024-09-16T15:41:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65c5600640 1 -- 172.21.15.60:0/3307175120 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc003850 con 0x7f65c80ccfd0 2024-09-16T15:41:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c813e120 con 0x7f65c80ccfd0 2024-09-16T15:41:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.400+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c813e6e0 con 0x7f65c80ccfd0 2024-09-16T15:41:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.401+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c813e920 con 0x7f65c80ccfd0 2024-09-16T15:41:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.403+0000 7f65c5600640 1 -- 172.21.15.60:0/3307175120 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f65bc01a070 con 0x7f65c80ccfd0 2024-09-16T15:41:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.404+0000 7f65c5600640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65a8078a00 0x7f65a807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.404+0000 7f65c5600640 1 -- 172.21.15.60:0/3307175120 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 4943+0+0 (secure 0 0 0) 0x7f65bc038e00 con 0x7f65c80ccfd0 2024-09-16T15:41:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.404+0000 7f65cce00640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65a8078a00 0x7f65a807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.405+0000 7f65cce00640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65a8078a00 0x7f65a807aec0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f65b4034000 tx=0x7f65b4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.407+0000 7f65c5600640 1 -- 172.21.15.60:0/3307175120 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f65c813e920 con 0x7f65c80ccfd0 2024-09-16T15:41:56.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:56.618+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f65c8073c20 con 0x7f65c80ccfd0 2024-09-16T15:41:57.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.649+0000 7f65c5600640 1 -- 172.21.15.60:0/3307175120 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) ==== 97+0+0 (secure 0 0 0) 0x7f65bc061f80 con 0x7f65c80ccfd0 2024-09-16T15:41:57.650 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:41:57.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65a8078a00 msgr2=0x7f65a807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65a8078a00 0x7f65a807aec0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f65b4034000 tx=0x7f65b4004730 comp rx=0 tx=0).stop 2024-09-16T15:41:57.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65c80ccfd0 msgr2=0x7f65c813b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65c80ccfd0 0x7f65c813b5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c400 tx=0x7f65bc00c8c0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 shutdown_connections 2024-09-16T15:41:57.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f65a8078a00 0x7f65a807aec0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65c81445d0 0x7f65c81f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65c80cf300 0x7f65c80cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 --2- 172.21.15.60:0/3307175120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65c80ccfd0 0x7f65c813b5d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 >> 172.21.15.60:0/3307175120 conn(0x7f65c80b1950 msgr2=0x7f65c80ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:57.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 shutdown_connections 2024-09-16T15:41:57.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.652+0000 7f65d0200640 1 -- 172.21.15.60:0/3307175120 wait complete. 2024-09-16T15:41:57.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:41:57.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.835+0000 7f51f3600640 1 -- 172.21.15.60:0/2771687676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51ec0af3d0 msgr2=0x7f51ec06f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.835+0000 7f51f3600640 1 --2- 172.21.15.60:0/2771687676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51ec0af3d0 0x7f51ec06f660 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009830 tx=0x7f51d802a480 comp rx=0 tx=0).stop 2024-09-16T15:41:57.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.835+0000 7f51f3600640 1 -- 172.21.15.60:0/2771687676 shutdown_connections 2024-09-16T15:41:57.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.835+0000 7f51f3600640 1 --2- 172.21.15.60:0/2771687676 >> v2:172.21.15.100:3300/0 conn(0x7f51ec072df0 0x7f51ec0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.835+0000 7f51f3600640 1 --2- 172.21.15.60:0/2771687676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51ec0af3d0 0x7f51ec06f660 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.835+0000 7f51f3600640 1 -- 172.21.15.60:0/2771687676 >> 172.21.15.60:0/2771687676 conn(0x7f51ec0b1950 msgr2=0x7f51ec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:57.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.835+0000 7f51f3600640 1 -- 172.21.15.60:0/2771687676 shutdown_connections 2024-09-16T15:41:57.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.835+0000 7f51f3600640 1 -- 172.21.15.60:0/2771687676 wait complete. 2024-09-16T15:41:57.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.837+0000 7f51f3600640 1 Processor -- start 2024-09-16T15:41:57.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.837+0000 7f51f3600640 1 -- start start 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f3600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51ec072df0 0x7f51ec1f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f3600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51ec0af3d0 0x7f51ec1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f3600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51ec161550 0x7f51ec09c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f3600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec0737e0 con 0x7f51ec161550 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f3600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec06fdb0 con 0x7f51ec072df0 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f3600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec073960 con 0x7f51ec0af3d0 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51ec161550 0x7f51ec09c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51ec161550 0x7f51ec09c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40170/0 (socket says 172.21.15.60:40170) 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f1600640 1 -- 172.21.15.60:0/2405969821 learned_addr learned my addr 172.21.15.60:0/2405969821 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51ebe00640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51ec0af3d0 0x7f51ec1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.838+0000 7f51f0c00640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51ec072df0 0x7f51ec1f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51f1600640 1 -- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51ec0af3d0 msgr2=0x7f51ec1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51f1600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51ec0af3d0 0x7f51ec1547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51f1600640 1 -- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51ec072df0 msgr2=0x7f51ec1f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51f1600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51ec072df0 0x7f51ec1f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51f1600640 1 -- 172.21.15.60:0/2405969821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec0cdb80 con 0x7f51ec161550 2024-09-16T15:41:57.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51f1600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51ec161550 0x7f51ec09c4c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f51e400efd0 tx=0x7f51e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:57.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51e8c00640 1 -- 172.21.15.60:0/2405969821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4003b70 con 0x7f51ec161550 2024-09-16T15:41:57.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51e8c00640 1 -- 172.21.15.60:0/2405969821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51e4003e00 con 0x7f51ec161550 2024-09-16T15:41:57.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51e8c00640 1 -- 172.21.15.60:0/2405969821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4004280 con 0x7f51ec161550 2024-09-16T15:41:57.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec1f8e00 con 0x7f51ec161550 2024-09-16T15:41:57.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.839+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec0cddf0 con 0x7f51ec161550 2024-09-16T15:41:57.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.840+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec073ec0 con 0x7f51ec161550 2024-09-16T15:41:57.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.842+0000 7f51e8c00640 1 -- 172.21.15.60:0/2405969821 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f51e4004420 con 0x7f51ec161550 2024-09-16T15:41:57.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.843+0000 7f51e8c00640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f51cc078a00 0x7f51cc07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.843+0000 7f51e8c00640 1 -- 172.21.15.60:0/2405969821 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4857+0+0 (secure 0 0 0) 0x7f51e409ba30 con 0x7f51ec161550 2024-09-16T15:41:57.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.843+0000 7f51f0c00640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f51cc078a00 0x7f51cc07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.844+0000 7f51f0c00640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f51cc078a00 0x7f51cc07aec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51e0001610 tx=0x7f51e0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:57.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:57.847+0000 7f51e8c00640 1 -- 172.21.15.60:0/2405969821 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51ec1f8e00 con 0x7f51ec161550 2024-09-16T15:41:58.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.111+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f51ec157c90 con 0x7f51ec161550 2024-09-16T15:41:58.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.112+0000 7f51e8c00640 1 -- 172.21.15.60:0/2405969821 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f51e4063fc0 con 0x7f51ec161550 2024-09-16T15:41:58.113 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:58.113 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:41:58.113 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:41:58.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:41:58.114 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:41:58.115 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:41:58.117 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.118 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.119 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:41:58.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.121 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:41:58.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:41:58.123 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:41:58.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.125 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.127 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:41:58.127 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:41:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f51cc078a00 msgr2=0x7f51cc07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f51cc078a00 0x7f51cc07aec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51e0001610 tx=0x7f51e0009040 comp rx=0 tx=0).stop 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51ec161550 msgr2=0x7f51ec09c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51ec161550 0x7f51ec09c4c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f51e400efd0 tx=0x7f51e400c6a0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 shutdown_connections 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f51cc078a00 0x7f51cc07aec0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51ec161550 0x7f51ec09c4c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51ec0af3d0 0x7f51ec1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 --2- 172.21.15.60:0/2405969821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51ec072df0 0x7f51ec1f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.114+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 >> 172.21.15.60:0/2405969821 conn(0x7f51ec0b1950 msgr2=0x7f51ec0740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.115+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 shutdown_connections 2024-09-16T15:41:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.115+0000 7f51f3600640 1 -- 172.21.15.60:0/2405969821 wait complete. 2024-09-16T15:41:58.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:41:58.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.291+0000 7fabf7e00640 1 -- 172.21.15.60:0/4006575948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf815bc30 msgr2=0x7fabf80b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.291+0000 7fabf7e00640 1 --2- 172.21.15.60:0/4006575948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf815bc30 0x7fabf80b28f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fabf8073b00 tx=0x7fabe802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.291+0000 7fabf7e00640 1 -- 172.21.15.60:0/4006575948 shutdown_connections 2024-09-16T15:41:58.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.291+0000 7fabf7e00640 1 --2- 172.21.15.60:0/4006575948 >> v2:172.21.15.100:3300/0 conn(0x7fabf80725d0 0x7fabf8073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.291+0000 7fabf7e00640 1 --2- 172.21.15.60:0/4006575948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf815bc30 0x7fabf80b28f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.291+0000 7fabf7e00640 1 -- 172.21.15.60:0/4006575948 >> 172.21.15.60:0/4006575948 conn(0x7fabf80cfa20 msgr2=0x7fabf80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.291+0000 7fabf7e00640 1 -- 172.21.15.60:0/4006575948 shutdown_connections 2024-09-16T15:41:58.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.291+0000 7fabf7e00640 1 -- 172.21.15.60:0/4006575948 wait complete. 2024-09-16T15:41:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.293+0000 7fabf7e00640 1 Processor -- start 2024-09-16T15:41:58.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.293+0000 7fabf7e00640 1 -- start start 2024-09-16T15:41:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf80d0540 0x7fabf814f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabf815bc30 0x7fabf814ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf814dcf0 0x7fabf8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:58.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8074610 con 0x7fabf814dcf0 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8074490 con 0x7fabf80d0540 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fabf8074790 con 0x7fabf815bc30 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf814dcf0 0x7fabf8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf814dcf0 0x7fabf8150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40184/0 (socket says 172.21.15.60:40184) 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf7400640 1 -- 172.21.15.60:0/43264983 learned_addr learned my addr 172.21.15.60:0/43264983 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabf6a00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf80d0540 0x7fabf814f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.294+0000 7fabee000640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabf815bc30 0x7fabf814ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabf7400640 1 -- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabf815bc30 msgr2=0x7fabf814ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabf7400640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabf815bc30 0x7fabf814ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabf7400640 1 -- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf80d0540 msgr2=0x7fabf814f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabf7400640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf80d0540 0x7fabf814f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabf7400640 1 -- 172.21.15.60:0/43264983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf8150b80 con 0x7fabf814dcf0 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabee000640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabf815bc30 0x7fabf814ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabf7400640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf814dcf0 0x7fabf8150280 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c7a0 tx=0x7fabf000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabef400640 1 -- 172.21.15.60:0/43264983 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0003b70 con 0x7fabf814dcf0 2024-09-16T15:41:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabef400640 1 -- 172.21.15.60:0/43264983 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabf0003d10 con 0x7fabf814dcf0 2024-09-16T15:41:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabef400640 1 -- 172.21.15.60:0/43264983 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf000aab0 con 0x7fabf814dcf0 2024-09-16T15:41:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf814e7d0 con 0x7fabf814dcf0 2024-09-16T15:41:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.295+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf814ed90 con 0x7fabf814dcf0 2024-09-16T15:41:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.296+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf806fd20 con 0x7fabf814dcf0 2024-09-16T15:41:58.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.298+0000 7fabef400640 1 -- 172.21.15.60:0/43264983 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fabf000ac50 con 0x7fabf814dcf0 2024-09-16T15:41:58.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.299+0000 7fabef400640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabd4078a00 0x7fabd407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:58.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.299+0000 7fabef400640 1 -- 172.21.15.60:0/43264983 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4857+0+0 (secure 0 0 0) 0x7fabf009a860 con 0x7fabf814dcf0 2024-09-16T15:41:58.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.299+0000 7fabf6a00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabd4078a00 0x7fabd407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:58.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.300+0000 7fabf6a00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabd4078a00 0x7fabd407aec0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabe8033000 tx=0x7fabe8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:58.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.303+0000 7fabef400640 1 -- 172.21.15.60:0/43264983 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fabf0062df0 con 0x7fabf814dcf0 2024-09-16T15:41:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.498+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fabf8158d80 con 0x7fabf814dcf0 2024-09-16T15:41:58.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.498+0000 7fabef400640 1 -- 172.21.15.60:0/43264983 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) ==== 84+0+65 (secure 0 0 0) 0x7fabf00626f0 con 0x7fabf814dcf0 2024-09-16T15:41:58.500 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:41:58.500 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:41:58.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:41:58.500 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:41:58.500 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:41:58.500 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:41:58.500 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:41:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabd4078a00 msgr2=0x7fabd407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabd4078a00 0x7fabd407aec0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fabe8033000 tx=0x7fabe8004730 comp rx=0 tx=0).stop 2024-09-16T15:41:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf814dcf0 msgr2=0x7fabf8150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf814dcf0 0x7fabf8150280 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c7a0 tx=0x7fabf000cc60 comp rx=0 tx=0).stop 2024-09-16T15:41:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 shutdown_connections 2024-09-16T15:41:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fabd4078a00 0x7fabd407aec0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabf814dcf0 0x7fabf8150280 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabf815bc30 0x7fabf814ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 --2- 172.21.15.60:0/43264983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabf80d0540 0x7fabf814f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 >> 172.21.15.60:0/43264983 conn(0x7fabf80cfa20 msgr2=0x7fabf80b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:58.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 shutdown_connections 2024-09-16T15:41:58.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.501+0000 7fabf7e00640 1 -- 172.21.15.60:0/43264983 wait complete. 2024-09-16T15:41:58.518 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:41:58.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:41:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.684+0000 7f894fe00640 1 -- 172.21.15.60:0/678976228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f895015dfb0 msgr2=0x7f89500d4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.684+0000 7f894fe00640 1 --2- 172.21.15.60:0/678976228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f895015dfb0 0x7f89500d4790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8940009830 tx=0x7f894002a510 comp rx=0 tx=0).stop 2024-09-16T15:41:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.684+0000 7f894fe00640 1 -- 172.21.15.60:0/678976228 shutdown_connections 2024-09-16T15:41:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.684+0000 7f894fe00640 1 --2- 172.21.15.60:0/678976228 >> v2:172.21.15.100:3300/0 conn(0x7f89500d0500 0x7f89500d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.684+0000 7f894fe00640 1 --2- 172.21.15.60:0/678976228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f895015dfb0 0x7f89500d4790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.684+0000 7f894fe00640 1 -- 172.21.15.60:0/678976228 >> 172.21.15.60:0/678976228 conn(0x7f895009d180 msgr2=0x7f89500d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.684+0000 7f894fe00640 1 -- 172.21.15.60:0/678976228 shutdown_connections 2024-09-16T15:41:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.684+0000 7f894fe00640 1 -- 172.21.15.60:0/678976228 wait complete. 2024-09-16T15:41:58.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.686+0000 7f894fe00640 1 Processor -- start 2024-09-16T15:41:58.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.686+0000 7f894fe00640 1 -- start start 2024-09-16T15:41:58.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89500cf470 0x7f89500a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:58.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89500d0500 0x7f89500caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:58.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f895015dfb0 0x7f89500c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:58.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f89500b6e90 con 0x7f895015dfb0 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f89500b6d10 con 0x7f89500d0500 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f89500b7010 con 0x7f89500cf470 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f895015dfb0 0x7f89500c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f895015dfb0 0x7f89500c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40200/0 (socket says 172.21.15.60:40200) 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894f400640 1 -- 172.21.15.60:0/417815854 learned_addr learned my addr 172.21.15.60:0/417815854 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894ea00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89500cf470 0x7f89500a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894e000640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89500d0500 0x7f89500caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894f400640 1 -- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89500cf470 msgr2=0x7f89500a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894f400640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89500cf470 0x7f89500a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894f400640 1 -- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89500d0500 msgr2=0x7f89500caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894f400640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89500d0500 0x7f89500caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894f400640 1 -- 172.21.15.60:0/417815854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89500cb4d0 con 0x7f895015dfb0 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894ea00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89500cf470 0x7f89500a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:58.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.687+0000 7f894e000640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89500d0500 0x7f89500caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.688+0000 7f894f400640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f895015dfb0 0x7f89500c80b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f894800eb10 tx=0x7f894800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.688+0000 7f8946a00640 1 -- 172.21.15.60:0/417815854 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948004b20 con 0x7f895015dfb0 2024-09-16T15:41:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.688+0000 7f8946a00640 1 -- 172.21.15.60:0/417815854 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8948004ce0 con 0x7f895015dfb0 2024-09-16T15:41:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.688+0000 7f8946a00640 1 -- 172.21.15.60:0/417815854 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948005280 con 0x7f895015dfb0 2024-09-16T15:41:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.688+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89501f5e00 con 0x7f895015dfb0 2024-09-16T15:41:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.688+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89501f63c0 con 0x7f895015dfb0 2024-09-16T15:41:58.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.689+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89500b7550 con 0x7f895015dfb0 2024-09-16T15:41:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.690+0000 7f8946a00640 1 -- 172.21.15.60:0/417815854 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f8948005420 con 0x7f895015dfb0 2024-09-16T15:41:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.691+0000 7f8946a00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8930078a00 0x7f893007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.692+0000 7f8946a00640 1 -- 172.21.15.60:0/417815854 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4857+0+0 (secure 0 0 0) 0x7f894809b670 con 0x7f895015dfb0 2024-09-16T15:41:58.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.692+0000 7f894ea00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8930078a00 0x7f893007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:58.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.693+0000 7f894ea00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8930078a00 0x7f893007aec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8940034000 tx=0x7f8940004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:58.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.696+0000 7f8946a00640 1 -- 172.21.15.60:0/417815854 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8948063d30 con 0x7f895015dfb0 2024-09-16T15:41:58.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:58.895+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8950075e70 con 0x7f895015dfb0 2024-09-16T15:41:59.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.653+0000 7f8946a00640 1 -- 172.21.15.60:0/417815854 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) ==== 138+0+0 (secure 0 0 0) 0x7f8948063630 con 0x7f895015dfb0 2024-09-16T15:41:59.655 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:41:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8930078a00 msgr2=0x7f893007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8930078a00 0x7f893007aec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8940034000 tx=0x7f8940004730 comp rx=0 tx=0).stop 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f895015dfb0 msgr2=0x7f89500c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f895015dfb0 0x7f89500c80b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f894800eb10 tx=0x7f894800c2d0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 shutdown_connections 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8930078a00 0x7f893007aec0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f895015dfb0 0x7f89500c80b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89500d0500 0x7f89500caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 --2- 172.21.15.60:0/417815854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89500cf470 0x7f89500a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 >> 172.21.15.60:0/417815854 conn(0x7f895009d180 msgr2=0x7f89500b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 shutdown_connections 2024-09-16T15:41:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.657+0000 7f894fe00640 1 -- 172.21.15.60:0/417815854 wait complete. 2024-09-16T15:41:59.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:41:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 -- 172.21.15.60:0/220997718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e80c5730 msgr2=0x7f98e815fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 --2- 172.21.15.60:0/220997718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e80c5730 0x7f98e815fa50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f98e80752c0 tx=0x7f98d402a560 comp rx=0 tx=0).stop 2024-09-16T15:41:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 -- 172.21.15.60:0/220997718 shutdown_connections 2024-09-16T15:41:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 --2- 172.21.15.60:0/220997718 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98e815c0f0 0x7f98e815c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 --2- 172.21.15.60:0/220997718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e80c5730 0x7f98e815fa50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 --2- 172.21.15.60:0/220997718 >> v2:172.21.15.100:3300/0 conn(0x7f98e815f670 0x7f98e80c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 -- 172.21.15.60:0/220997718 >> 172.21.15.60:0/220997718 conn(0x7f98e80cc490 msgr2=0x7f98e80cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:59.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 -- 172.21.15.60:0/220997718 shutdown_connections 2024-09-16T15:41:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.846+0000 7f98eee00640 1 -- 172.21.15.60:0/220997718 wait complete. 2024-09-16T15:41:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.848+0000 7f98eee00640 1 Processor -- start 2024-09-16T15:41:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.848+0000 7f98eee00640 1 -- start start 2024-09-16T15:41:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98eee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e80c5730 0x7f98e81f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98eee00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98e815c0f0 0x7f98e80c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98eee00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e815f670 0x7f98e80c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98eee00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f98e80cd2c0 con 0x7f98e815f670 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98eee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98e80cd140 con 0x7f98e80c5730 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98eee00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f98e80ca370 con 0x7f98e815c0f0 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98ee400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e815f670 0x7f98e80c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98ee400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e815f670 0x7f98e80c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41884/0 (socket says 172.21.15.60:41884) 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.849+0000 7f98ee400640 1 -- 172.21.15.60:0/339628347 learned_addr learned my addr 172.21.15.60:0/339628347 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98eda00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e80c5730 0x7f98e81f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98ed000640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98e815c0f0 0x7f98e80c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98ee400640 1 -- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98e815c0f0 msgr2=0x7f98e80c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98ee400640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98e815c0f0 0x7f98e80c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98ee400640 1 -- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e80c5730 msgr2=0x7f98e81f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98ee400640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e80c5730 0x7f98e81f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98ee400640 1 -- 172.21.15.60:0/339628347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e81f8320 con 0x7f98e815f670 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98eda00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e80c5730 0x7f98e81f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98ed000640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98e815c0f0 0x7f98e80c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98ee400640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e815f670 0x7f98e80c7b30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f98e000eaa0 tx=0x7f98e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98e5600640 1 -- 172.21.15.60:0/339628347 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0004b20 con 0x7f98e815f670 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98e5600640 1 -- 172.21.15.60:0/339628347 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98e0004db0 con 0x7f98e815f670 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98e5600640 1 -- 172.21.15.60:0/339628347 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0005280 con 0x7f98e815f670 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.850+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e81f8600 con 0x7f98e815f670 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.851+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e80c8070 con 0x7f98e815f670 2024-09-16T15:41:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.852+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e80736a0 con 0x7f98e815f670 2024-09-16T15:41:59.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.853+0000 7f98e5600640 1 -- 172.21.15.60:0/339628347 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f98e0005420 con 0x7f98e815f670 2024-09-16T15:41:59.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.854+0000 7f98e5600640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98c8078a00 0x7f98c807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:59.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.854+0000 7f98e5600640 1 -- 172.21.15.60:0/339628347 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5227+0+0 (secure 0 0 0) 0x7f98e009b9f0 con 0x7f98e815f670 2024-09-16T15:41:59.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.854+0000 7f98eda00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98c8078a00 0x7f98c807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.855+0000 7f98eda00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98c8078a00 0x7f98c807aec0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009b80 tx=0x7f98dc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:59.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:41:59.859+0000 7f98e5600640 1 -- 172.21.15.60:0/339628347 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f98e80736a0 con 0x7f98e815f670 2024-09-16T15:42:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.055+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f98e81f4630 con 0x7f98e815f670 2024-09-16T15:42:00.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:42:00.658 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:42:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.660+0000 7f98e5600640 1 -- 172.21.15.60:0/339628347 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) ==== 170+0+0 (secure 0 0 0) 0x7f98e0063e10 con 0x7f98e815f670 2024-09-16T15:42:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.663+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98c8078a00 msgr2=0x7f98c807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.663+0000 7f98eee00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98c8078a00 0x7f98c807aec0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009b80 tx=0x7f98dc005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.663+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e815f670 msgr2=0x7f98e80c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.663+0000 7f98eee00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e815f670 0x7f98e80c7b30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f98e000eaa0 tx=0x7f98e000ef60 comp rx=0 tx=0).stop 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.664+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 shutdown_connections 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.664+0000 7f98eee00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f98c8078a00 0x7f98c807aec0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.664+0000 7f98eee00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98e815f670 0x7f98e80c7b30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.664+0000 7f98eee00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f98e815c0f0 0x7f98e80c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.664+0000 7f98eee00640 1 --2- 172.21.15.60:0/339628347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98e80c5730 0x7f98e81f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.664+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 >> 172.21.15.60:0/339628347 conn(0x7f98e80cc490 msgr2=0x7f98e80743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.664+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 shutdown_connections 2024-09-16T15:42:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.664+0000 7f98eee00640 1 -- 172.21.15.60:0/339628347 wait complete. 2024-09-16T15:42:00.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:42:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.907+0000 7f9db2800640 1 -- 172.21.15.60:0/4198861980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0cccd0 msgr2=0x7f9dac0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.907+0000 7f9db2800640 1 --2- 172.21.15.60:0/4198861980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0cccd0 0x7f9dac0cdbb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9da4005fd0 tx=0x7f9da402aa10 comp rx=0 tx=0).stop 2024-09-16T15:42:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.907+0000 7f9db2800640 1 -- 172.21.15.60:0/4198861980 shutdown_connections 2024-09-16T15:42:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.907+0000 7f9db2800640 1 --2- 172.21.15.60:0/4198861980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0cccd0 0x7f9dac0cdbb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.907+0000 7f9db2800640 1 --2- 172.21.15.60:0/4198861980 >> v2:172.21.15.100:3300/0 conn(0x7f9dac145750 0x7f9dac073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.907+0000 7f9db2800640 1 -- 172.21.15.60:0/4198861980 >> 172.21.15.60:0/4198861980 conn(0x7f9dac0b1840 msgr2=0x7f9dac0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:00.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.907+0000 7f9db2800640 1 -- 172.21.15.60:0/4198861980 shutdown_connections 2024-09-16T15:42:00.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.908+0000 7f9db2800640 1 -- 172.21.15.60:0/4198861980 wait complete. 2024-09-16T15:42:00.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.909+0000 7f9db2800640 1 Processor -- start 2024-09-16T15:42:00.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.909+0000 7f9db2800640 1 -- start start 2024-09-16T15:42:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9db2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac06efa0 0x7f9dac15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9db2800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0cccd0 0x7f9dac0ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9db2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac145750 0x7f9dac0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9db2800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac0ce480 con 0x7f9dac06efa0 2024-09-16T15:42:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9db2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac0ce300 con 0x7f9dac145750 2024-09-16T15:42:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9db2800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9dac0ce600 con 0x7f9dac0cccd0 2024-09-16T15:42:00.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9dab400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac06efa0 0x7f9dac15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9dab400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac06efa0 0x7f9dac15e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41900/0 (socket says 172.21.15.60:41900) 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.910+0000 7f9dab400640 1 -- 172.21.15.60:0/2112177291 learned_addr learned my addr 172.21.15.60:0/2112177291 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9dabe00640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac145750 0x7f9dac0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9da2a00640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0cccd0 0x7f9dac0ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9dab400640 1 -- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0cccd0 msgr2=0x7f9dac0ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9dab400640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0cccd0 0x7f9dac0ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9dab400640 1 -- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac145750 msgr2=0x7f9dac0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9dab400640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac145750 0x7f9dac0adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9dab400640 1 -- 172.21.15.60:0/2112177291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac1f7d60 con 0x7f9dac06efa0 2024-09-16T15:42:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9dab400640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac06efa0 0x7f9dac15e0e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00e790 tx=0x7f9d9c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:00.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9da3e00640 1 -- 172.21.15.60:0/2112177291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00eea0 con 0x7f9dac06efa0 2024-09-16T15:42:00.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9da3e00640 1 -- 172.21.15.60:0/2112177291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d9c00cdb0 con 0x7f9dac06efa0 2024-09-16T15:42:00.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac1f8000 con 0x7f9dac06efa0 2024-09-16T15:42:00.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.911+0000 7f9da3e00640 1 -- 172.21.15.60:0/2112177291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c010690 con 0x7f9dac06efa0 2024-09-16T15:42:00.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.912+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1f85c0 con 0x7f9dac06efa0 2024-09-16T15:42:00.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.913+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac0cdbb0 con 0x7f9dac06efa0 2024-09-16T15:42:00.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.914+0000 7f9da3e00640 1 -- 172.21.15.60:0/2112177291 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f9d9c010830 con 0x7f9dac06efa0 2024-09-16T15:42:00.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.915+0000 7f9da3e00640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d88078a00 0x7f9d8807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.915+0000 7f9da3e00640 1 -- 172.21.15.60:0/2112177291 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5609+0+0 (secure 0 0 0) 0x7f9d9c09b490 con 0x7f9dac06efa0 2024-09-16T15:42:00.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.915+0000 7f9da2a00640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d88078a00 0x7f9d8807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.916+0000 7f9da2a00640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d88078a00 0x7f9d8807aec0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d94007fd0 tx=0x7f9d94009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:00.920+0000 7f9da3e00640 1 -- 172.21.15.60:0/2112177291 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d9c0632c0 con 0x7f9dac06efa0 2024-09-16T15:42:01.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.153+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9dac0afb00 con 0x7f9dac06efa0 2024-09-16T15:42:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.662+0000 7f9da3e00640 1 -- 172.21.15.60:0/2112177291 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f9d9c067cf0 con 0x7f9dac06efa0 2024-09-16T15:42:01.664 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 8 and data pool 9 2024-09-16T15:42:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d88078a00 msgr2=0x7f9d8807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d88078a00 0x7f9d8807aec0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d94007fd0 tx=0x7f9d94009040 comp rx=0 tx=0).stop 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac06efa0 msgr2=0x7f9dac15e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac06efa0 0x7f9dac15e0e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00e790 tx=0x7f9d9c00ec50 comp rx=0 tx=0).stop 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 shutdown_connections 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d88078a00 0x7f9d8807aec0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dac145750 0x7f9dac0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dac0cccd0 0x7f9dac0ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 --2- 172.21.15.60:0/2112177291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dac06efa0 0x7f9dac15e0e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.665+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 >> 172.21.15.60:0/2112177291 conn(0x7f9dac0b1840 msgr2=0x7f9dac0a9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:01.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.666+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 shutdown_connections 2024-09-16T15:42:01.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.667+0000 7f9db2800640 1 -- 172.21.15.60:0/2112177291 wait complete. 2024-09-16T15:42:01.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.867+0000 7facaa400640 1 -- 172.21.15.60:0/3717242279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40701e0 msgr2=0x7faca415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.867+0000 7facaa400640 1 --2- 172.21.15.60:0/3717242279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40701e0 0x7faca415e170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009c10 tx=0x7fac9c02a900 comp rx=0 tx=0).stop 2024-09-16T15:42:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.867+0000 7facaa400640 1 -- 172.21.15.60:0/3717242279 shutdown_connections 2024-09-16T15:42:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.867+0000 7facaa400640 1 --2- 172.21.15.60:0/3717242279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40701e0 0x7faca415e170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.867+0000 7facaa400640 1 --2- 172.21.15.60:0/3717242279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca40aecc0 0x7faca415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.867+0000 7facaa400640 1 -- 172.21.15.60:0/3717242279 >> 172.21.15.60:0/3717242279 conn(0x7faca40cfa20 msgr2=0x7faca40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.867+0000 7facaa400640 1 -- 172.21.15.60:0/3717242279 shutdown_connections 2024-09-16T15:42:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.867+0000 7facaa400640 1 -- 172.21.15.60:0/3717242279 wait complete. 2024-09-16T15:42:01.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.869+0000 7facaa400640 1 Processor -- start 2024-09-16T15:42:01.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.869+0000 7facaa400640 1 -- start start 2024-09-16T15:42:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7facaa400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca40701e0 0x7faca41597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7facaa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40aecc0 0x7faca415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7facaa400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca40caa50 0x7faca40b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7facaa400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faca4073410 con 0x7faca40701e0 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7facaa400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faca4073290 con 0x7faca40aecc0 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7facaa400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7faca4073590 con 0x7faca40caa50 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7faca9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca40701e0 0x7faca41597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7faca9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca40701e0 0x7faca41597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41912/0 (socket says 172.21.15.60:41912) 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7faca9000640 1 -- 172.21.15.60:0/1838142972 learned_addr learned my addr 172.21.15.60:0/1838142972 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7faca9a00640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca40caa50 0x7faca40b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.870+0000 7fac9be00640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40aecc0 0x7faca415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca9000640 1 -- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca40caa50 msgr2=0x7faca40b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca9000640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca40caa50 0x7faca40b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca9000640 1 -- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40aecc0 msgr2=0x7faca415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca9000640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40aecc0 0x7faca415a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca9000640 1 -- 172.21.15.60:0/1838142972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca40b3c10 con 0x7faca40701e0 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7fac9be00640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40aecc0 0x7faca415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca9a00640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca40caa50 0x7faca40b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca9000640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca40701e0 0x7faca41597c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c430 tx=0x7fac9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca1600640 1 -- 172.21.15.60:0/1838142972 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94011790 con 0x7faca40701e0 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca1600640 1 -- 172.21.15.60:0/1838142972 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac94017070 con 0x7faca40701e0 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7faca1600640 1 -- 172.21.15.60:0/1838142972 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94011ce0 con 0x7faca40701e0 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca415aa30 con 0x7faca40701e0 2024-09-16T15:42:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.871+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca415aff0 con 0x7faca40701e0 2024-09-16T15:42:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.872+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca4073ab0 con 0x7faca40701e0 2024-09-16T15:42:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.873+0000 7faca1600640 1 -- 172.21.15.60:0/1838142972 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fac9401c460 con 0x7faca40701e0 2024-09-16T15:42:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.875+0000 7faca1600640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fac80078a00 0x7fac8007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.875+0000 7faca1600640 1 -- 172.21.15.60:0/1838142972 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7fac9409ab50 con 0x7faca40701e0 2024-09-16T15:42:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.876+0000 7fac9be00640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fac80078a00 0x7fac8007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.876+0000 7fac9be00640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fac80078a00 0x7fac8007aec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faca41f7830 tx=0x7fac8c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:01.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:01.879+0000 7faca1600640 1 -- 172.21.15.60:0/1838142972 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fac94062d90 con 0x7faca40701e0 2024-09-16T15:42:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.076+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faca40b3de0 con 0x7faca40701e0 2024-09-16T15:42:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.077+0000 7faca1600640 1 -- 172.21.15.60:0/1838142972 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16738 (secure 0 0 0) 0x7fac94062690 con 0x7faca40701e0 2024-09-16T15:42:02.079 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:02.079 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":61,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:01.154445+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:41:58.905230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:00.065450+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fac80078a00 msgr2=0x7fac8007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fac80078a00 0x7fac8007aec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faca41f7830 tx=0x7fac8c005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca40701e0 msgr2=0x7faca41597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca40701e0 0x7faca41597c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c430 tx=0x7fac9400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 shutdown_connections 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fac80078a00 0x7fac8007aec0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca40caa50 0x7faca40b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca40aecc0 0x7faca415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 --2- 172.21.15.60:0/1838142972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca40701e0 0x7faca41597c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 >> 172.21.15.60:0/1838142972 conn(0x7faca40cfa20 msgr2=0x7faca40af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 shutdown_connections 2024-09-16T15:42:02.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.080+0000 7facaa400640 1 -- 172.21.15.60:0/1838142972 wait complete. 2024-09-16T15:42:02.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.257+0000 7f825d800640 1 -- 172.21.15.60:0/1191653014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8258073fb0 msgr2=0x7f8258074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.257+0000 7f825d800640 1 --2- 172.21.15.60:0/1191653014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8258073fb0 0x7f8258074460 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8250005fd0 tx=0x7f825002a820 comp rx=0 tx=0).stop 2024-09-16T15:42:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.257+0000 7f825d800640 1 -- 172.21.15.60:0/1191653014 shutdown_connections 2024-09-16T15:42:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.257+0000 7f825d800640 1 --2- 172.21.15.60:0/1191653014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8258073fb0 0x7f8258074460 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.257+0000 7f825d800640 1 --2- 172.21.15.60:0/1191653014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f825815f5a0 0x7f82580b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.257+0000 7f825d800640 1 -- 172.21.15.60:0/1191653014 >> 172.21.15.60:0/1191653014 conn(0x7f82580b1950 msgr2=0x7f82580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.257+0000 7f825d800640 1 -- 172.21.15.60:0/1191653014 shutdown_connections 2024-09-16T15:42:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.257+0000 7f825d800640 1 -- 172.21.15.60:0/1191653014 wait complete. 2024-09-16T15:42:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.259+0000 7f825d800640 1 Processor -- start 2024-09-16T15:42:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.259+0000 7f825d800640 1 -- start start 2024-09-16T15:42:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f825d800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8258073fb0 0x7f82581f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f825d800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82580ccf00 0x7f8258151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f825d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825815f5a0 0x7f8258150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f825d800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f825806f2e0 con 0x7f825815f5a0 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f825d800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f825806f160 con 0x7f82580ccf00 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f825d800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f825806f460 con 0x7f8258073fb0 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8257400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825815f5a0 0x7f8258150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8257400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825815f5a0 0x7f8258150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41938/0 (socket says 172.21.15.60:41938) 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8257400640 1 -- 172.21.15.60:0/2216416210 learned_addr learned my addr 172.21.15.60:0/2216416210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8256a00640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8258073fb0 0x7f82581f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f824e000640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82580ccf00 0x7f8258151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8257400640 1 -- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8258073fb0 msgr2=0x7f82581f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8257400640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8258073fb0 0x7f82581f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8257400640 1 -- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82580ccf00 msgr2=0x7f8258151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8257400640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82580ccf00 0x7f8258151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8257400640 1 -- 172.21.15.60:0/2216416210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82581520a0 con 0x7f825815f5a0 2024-09-16T15:42:02.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f8256a00640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8258073fb0 0x7f82581f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:02.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.260+0000 7f824e000640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82580ccf00 0x7f8258151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:02.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.261+0000 7f8257400640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825815f5a0 0x7f8258150890 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8250006010 tx=0x7f8250032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.261+0000 7f824f400640 1 -- 172.21.15.60:0/2216416210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250037040 con 0x7f825815f5a0 2024-09-16T15:42:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.261+0000 7f824f400640 1 -- 172.21.15.60:0/2216416210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8250042070 con 0x7f825815f5a0 2024-09-16T15:42:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.261+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8258152380 con 0x7f825815f5a0 2024-09-16T15:42:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.261+0000 7f824f400640 1 -- 172.21.15.60:0/2216416210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82500042a0 con 0x7f825815f5a0 2024-09-16T15:42:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.261+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8258151140 con 0x7f825815f5a0 2024-09-16T15:42:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.262+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825806f990 con 0x7f825815f5a0 2024-09-16T15:42:02.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.264+0000 7f824f400640 1 -- 172.21.15.60:0/2216416210 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f8250004440 con 0x7f825815f5a0 2024-09-16T15:42:02.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.265+0000 7f824f400640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8234078a00 0x7f823407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:02.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.265+0000 7f824f400640 1 -- 172.21.15.60:0/2216416210 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f82500bb930 con 0x7f825815f5a0 2024-09-16T15:42:02.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.265+0000 7f8256a00640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8234078a00 0x7f823407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:02.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.266+0000 7f8256a00640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8234078a00 0x7f823407aec0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8248009b80 tx=0x7f8248005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:02.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.269+0000 7f824f400640 1 -- 172.21.15.60:0/2216416210 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8250083c00 con 0x7f825815f5a0 2024-09-16T15:42:02.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.465+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82580acf80 con 0x7f825815f5a0 2024-09-16T15:42:02.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.466+0000 7f824f400640 1 -- 172.21.15.60:0/2216416210 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16738 (secure 0 0 0) 0x7f8250083500 con 0x7f825815f5a0 2024-09-16T15:42:02.468 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:02.468 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":61,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:01.154445+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:41:58.905230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:00.065450+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8234078a00 msgr2=0x7f823407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8234078a00 0x7f823407aec0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8248009b80 tx=0x7f8248005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825815f5a0 msgr2=0x7f8258150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825815f5a0 0x7f8258150890 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8250006010 tx=0x7f8250032c90 comp rx=0 tx=0).stop 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 shutdown_connections 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8234078a00 0x7f823407aec0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f825815f5a0 0x7f8258150890 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82580ccf00 0x7f8258151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 --2- 172.21.15.60:0/2216416210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8258073fb0 0x7f82581f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 >> 172.21.15.60:0/2216416210 conn(0x7f82580b1950 msgr2=0x7f825809d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 shutdown_connections 2024-09-16T15:42:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.469+0000 7f825d800640 1 -- 172.21.15.60:0/2216416210 wait complete. 2024-09-16T15:42:02.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:42:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.650+0000 7fa002000640 1 -- 172.21.15.60:0/544266529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 msgr2=0x7f9ffc0d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.650+0000 7fa002000640 1 --2- 172.21.15.60:0/544266529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 0x7f9ffc0d2830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009830 tx=0x7f9fe402a480 comp rx=0 tx=0).stop 2024-09-16T15:42:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.650+0000 7fa002000640 1 -- 172.21.15.60:0/544266529 shutdown_connections 2024-09-16T15:42:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.650+0000 7fa002000640 1 --2- 172.21.15.60:0/544266529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 0x7f9ffc0d2830 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.650+0000 7fa002000640 1 --2- 172.21.15.60:0/544266529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ffc161460 0x7f9ffc0d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.650+0000 7fa002000640 1 -- 172.21.15.60:0/544266529 >> 172.21.15.60:0/544266529 conn(0x7f9ffc09d0b0 msgr2=0x7f9ffc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.650+0000 7fa002000640 1 -- 172.21.15.60:0/544266529 shutdown_connections 2024-09-16T15:42:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.650+0000 7fa002000640 1 -- 172.21.15.60:0/544266529 wait complete. 2024-09-16T15:42:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.652+0000 7fa002000640 1 Processor -- start 2024-09-16T15:42:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.652+0000 7fa002000640 1 -- start start 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7fa002000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ffc0b4f40 0x7f9ffc15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7fa002000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 0x7f9ffc09ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7fa002000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc161460 0x7f9ffc09f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7fa002000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc0cf0b0 con 0x7f9ffc0b5480 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7fa002000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc0cef30 con 0x7f9ffc161460 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7fa002000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9ffc0cf230 con 0x7f9ffc0b4f40 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7f9ff3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 0x7f9ffc09ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7f9ff3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 0x7f9ffc09ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41968/0 (socket says 172.21.15.60:41968) 2024-09-16T15:42:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7f9ff3e00640 1 -- 172.21.15.60:0/2538643946 learned_addr learned my addr 172.21.15.60:0/2538643946 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7fa000c00640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ffc0b4f40 0x7f9ffc15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7fa001600640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc161460 0x7f9ffc09f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7f9ff3e00640 1 -- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ffc0b4f40 msgr2=0x7f9ffc15ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7f9ff3e00640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ffc0b4f40 0x7f9ffc15ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7f9ff3e00640 1 -- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc161460 msgr2=0x7f9ffc09f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7f9ff3e00640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc161460 0x7f9ffc09f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.653+0000 7f9ff3e00640 1 -- 172.21.15.60:0/2538643946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc09f1a0 con 0x7f9ffc0b5480 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.654+0000 7f9ff3e00640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 0x7f9ffc09ec60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4000c00 tx=0x7f9fe4032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.654+0000 7f9ff9600640 1 -- 172.21.15.60:0/2538643946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4040070 con 0x7f9ffc0b5480 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.654+0000 7f9ff9600640 1 -- 172.21.15.60:0/2538643946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fe4032e70 con 0x7f9ffc0b5480 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.654+0000 7f9ff9600640 1 -- 172.21.15.60:0/2538643946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4003d90 con 0x7f9ffc0b5480 2024-09-16T15:42:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.654+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc09dbc0 con 0x7f9ffc0b5480 2024-09-16T15:42:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.654+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc09e180 con 0x7f9ffc0b5480 2024-09-16T15:42:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.655+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc0cf750 con 0x7f9ffc0b5480 2024-09-16T15:42:02.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.656+0000 7f9ff9600640 1 -- 172.21.15.60:0/2538643946 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f9fe4039040 con 0x7f9ffc0b5480 2024-09-16T15:42:02.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.657+0000 7f9ff9600640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9fd8078a00 0x7f9fd807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:02.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.657+0000 7f9ff9600640 1 -- 172.21.15.60:0/2538643946 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5709+0+0 (secure 0 0 0) 0x7f9fe40bc260 con 0x7f9ffc0b5480 2024-09-16T15:42:02.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.658+0000 7fa000c00640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9fd8078a00 0x7f9fd807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:02.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.659+0000 7fa000c00640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9fd8078a00 0x7f9fd807aec0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9fec005fd0 tx=0x7f9fec005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:02.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.662+0000 7f9ff9600640 1 -- 172.21.15.60:0/2538643946 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9fe4084530 con 0x7f9ffc0b5480 2024-09-16T15:42:02.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:02.898+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9ffc075ac0 con 0x7f9ffc0b5480 2024-09-16T15:42:03.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.689+0000 7f9ff9600640 1 -- 172.21.15.60:0/2538643946 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f9fe4083e30 con 0x7f9ffc0b5480 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9fd8078a00 msgr2=0x7f9fd807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9fd8078a00 0x7f9fd807aec0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9fec005fd0 tx=0x7f9fec005850 comp rx=0 tx=0).stop 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 msgr2=0x7f9ffc09ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 0x7f9ffc09ec60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4000c00 tx=0x7f9fe4032c80 comp rx=0 tx=0).stop 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 shutdown_connections 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9fd8078a00 0x7f9fd807aec0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc161460 0x7f9ffc09f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc0b5480 0x7f9ffc09ec60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 --2- 172.21.15.60:0/2538643946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ffc0b4f40 0x7f9ffc15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.692+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 >> 172.21.15.60:0/2538643946 conn(0x7f9ffc09d0b0 msgr2=0x7f9ffc0cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.693+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 shutdown_connections 2024-09-16T15:42:03.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.693+0000 7fa002000640 1 -- 172.21.15.60:0/2538643946 wait complete. 2024-09-16T15:42:03.712 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:42:03.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:42:03.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.880+0000 7f517ae00640 1 -- 172.21.15.60:0/2423580847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f515c003a50 msgr2=0x7f515c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:03.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.880+0000 7f517ae00640 1 --2- 172.21.15.60:0/2423580847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f515c003a50 0x7f515c003e70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f516800c310 tx=0x7f515c005830 comp rx=0 tx=0).stop 2024-09-16T15:42:03.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.880+0000 7f517ae00640 1 -- 172.21.15.60:0/2423580847 shutdown_connections 2024-09-16T15:42:03.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.880+0000 7f517ae00640 1 --2- 172.21.15.60:0/2423580847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f515c003a50 0x7f515c003e70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:03.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.880+0000 7f517ae00640 1 -- 172.21.15.60:0/2423580847 >> 172.21.15.60:0/2423580847 conn(0x7f517409d0b0 msgr2=0x7f51740d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:03.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.880+0000 7f517ae00640 1 -- 172.21.15.60:0/2423580847 shutdown_connections 2024-09-16T15:42:03.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.880+0000 7f517ae00640 1 -- 172.21.15.60:0/2423580847 wait complete. 2024-09-16T15:42:03.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.881+0000 7f517ae00640 1 Processor -- start 2024-09-16T15:42:03.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.882+0000 7f517ae00640 1 -- start start 2024-09-16T15:42:03.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.882+0000 7f517ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f515c003a50 0x7f517415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:03.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.882+0000 7f517ae00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51740b1de0 0x7f51740b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:03.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.882+0000 7f517ae00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51740b2260 0x7f51740b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:03.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.882+0000 7f517ae00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51740b50c0 con 0x7f51740b1de0 2024-09-16T15:42:03.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.882+0000 7f517ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51740b4f40 con 0x7f515c003a50 2024-09-16T15:42:03.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.882+0000 7f517ae00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f51740d22f0 con 0x7f51740b2260 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51740b1de0 0x7f51740b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51740b1de0 0x7f51740b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41986/0 (socket says 172.21.15.60:41986) 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 -- 172.21.15.60:0/1939034832 learned_addr learned my addr 172.21.15.60:0/1939034832 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179a00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f515c003a50 0x7f517415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f517a400640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51740b2260 0x7f51740b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 -- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51740b2260 msgr2=0x7f51740b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51740b2260 0x7f51740b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 -- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f515c003a50 msgr2=0x7f517415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f515c003a50 0x7f517415dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 -- 172.21.15.60:0/1939034832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517415e2b0 con 0x7f51740b1de0 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f517a400640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51740b2260 0x7f51740b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179a00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f515c003a50 0x7f517415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:03.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.883+0000 7f5179000640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51740b1de0 0x7f51740b2f40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f516000f980 tx=0x7f516000fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:03.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.884+0000 7f5171600640 1 -- 172.21.15.60:0/1939034832 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160015070 con 0x7f51740b1de0 2024-09-16T15:42:03.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.884+0000 7f5171600640 1 -- 172.21.15.60:0/1939034832 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f516000b8a0 con 0x7f51740b1de0 2024-09-16T15:42:03.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.884+0000 7f5171600640 1 -- 172.21.15.60:0/1939034832 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516000bc70 con 0x7f51740b1de0 2024-09-16T15:42:03.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.884+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517415d2b0 con 0x7f51740b1de0 2024-09-16T15:42:03.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.884+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51741f56e0 con 0x7f51740b1de0 2024-09-16T15:42:03.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.885+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51740ce820 con 0x7f51740b1de0 2024-09-16T15:42:03.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.886+0000 7f5171600640 1 -- 172.21.15.60:0/1939034832 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f5160005020 con 0x7f51740b1de0 2024-09-16T15:42:03.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.887+0000 7f5171600640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5154078a00 0x7f515407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:03.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.887+0000 7f5171600640 1 -- 172.21.15.60:0/1939034832 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f516009b5f0 con 0x7f51740b1de0 2024-09-16T15:42:03.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.888+0000 7f5179a00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5154078a00 0x7f515407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:03.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.888+0000 7f5179a00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5154078a00 0x7f515407aec0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5168033000 tx=0x7f5168001a60 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:03.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:03.892+0000 7f5171600640 1 -- 172.21.15.60:0/1939034832 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5160063830 con 0x7f51740b1de0 2024-09-16T15:42:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.122+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f517406d3c0 con 0x7f51740b1de0 2024-09-16T15:42:04.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.695+0000 7f5171600640 1 -- 172.21.15.60:0/1939034832 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f5160063130 con 0x7f51740b1de0 2024-09-16T15:42:04.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5154078a00 msgr2=0x7f515407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:04.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5154078a00 0x7f515407aec0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5168033000 tx=0x7f5168001a60 comp rx=0 tx=0).stop 2024-09-16T15:42:04.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51740b1de0 msgr2=0x7f51740b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:04.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51740b1de0 0x7f51740b2f40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f516000f980 tx=0x7f516000fe40 comp rx=0 tx=0).stop 2024-09-16T15:42:04.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 shutdown_connections 2024-09-16T15:42:04.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5154078a00 0x7f515407aec0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:04.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51740b2260 0x7f51740b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:04.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51740b1de0 0x7f51740b2f40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:04.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 --2- 172.21.15.60:0/1939034832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f515c003a50 0x7f517415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:04.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 >> 172.21.15.60:0/1939034832 conn(0x7f517409d0b0 msgr2=0x7f51740d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:04.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 shutdown_connections 2024-09-16T15:42:04.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.698+0000 7f517ae00640 1 -- 172.21.15.60:0/1939034832 wait complete. 2024-09-16T15:42:04.717 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:42:04.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.880+0000 7f127fe00640 1 -- 172.21.15.60:0/1476567887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 msgr2=0x7f1260004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.880+0000 7f127fe00640 1 --2- 172.21.15.60:0/1476567887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 0x7f1260004c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f12640016c0 tx=0x7f1260005830 comp rx=0 tx=0).stop 2024-09-16T15:42:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.880+0000 7f127fe00640 1 -- 172.21.15.60:0/1476567887 shutdown_connections 2024-09-16T15:42:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.880+0000 7f127fe00640 1 --2- 172.21.15.60:0/1476567887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 0x7f1260004c80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.880+0000 7f127fe00640 1 -- 172.21.15.60:0/1476567887 >> 172.21.15.60:0/1476567887 conn(0x7f127806d230 msgr2=0x7f12780688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.880+0000 7f127fe00640 1 -- 172.21.15.60:0/1476567887 shutdown_connections 2024-09-16T15:42:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.880+0000 7f127fe00640 1 -- 172.21.15.60:0/1476567887 wait complete. 2024-09-16T15:42:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.882+0000 7f127fe00640 1 Processor -- start 2024-09-16T15:42:04.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.882+0000 7f127fe00640 1 -- start start 2024-09-16T15:42:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 0x7f12780d2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f127815e6d0 0x7f127815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:04.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12780cfbc0 0x7f12780c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1278071d70 con 0x7f1260004800 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1278071bf0 con 0x7f12780cfbc0 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1278071ef0 con 0x7f127815e6d0 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 0x7f12780d2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 0x7f12780d2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42000/0 (socket says 172.21.15.60:42000) 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.883+0000 7f127d400640 1 -- 172.21.15.60:0/3576407641 learned_addr learned my addr 172.21.15.60:0/3576407641 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f127d400640 1 -- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f127815e6d0 msgr2=0x7f127815f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f127d400640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f127815e6d0 0x7f127815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f127d400640 1 -- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12780cfbc0 msgr2=0x7f12780c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f127d400640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12780cfbc0 0x7f12780c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f127d400640 1 -- 172.21.15.60:0/3576407641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12781f5f50 con 0x7f1260004800 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f127d400640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 0x7f12780d2600 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f126c011a10 tx=0x7f126c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f1275600640 1 -- 172.21.15.60:0/3576407641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c01e070 con 0x7f1260004800 2024-09-16T15:42:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f1275600640 1 -- 172.21.15.60:0/3576407641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f126c004460 con 0x7f1260004800 2024-09-16T15:42:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f1275600640 1 -- 172.21.15.60:0/3576407641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c00fb80 con 0x7f1260004800 2024-09-16T15:42:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12781f6230 con 0x7f1260004800 2024-09-16T15:42:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.884+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12781f67f0 con 0x7f1260004800 2024-09-16T15:42:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.885+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12780ced50 con 0x7f1260004800 2024-09-16T15:42:04.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.886+0000 7f1275600640 1 -- 172.21.15.60:0/3576407641 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f126c021070 con 0x7f1260004800 2024-09-16T15:42:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.887+0000 7f1275600640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1258078a00 0x7f125807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:04.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.887+0000 7f1275600640 1 -- 172.21.15.60:0/3576407641 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f126c09b650 con 0x7f1260004800 2024-09-16T15:42:04.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.888+0000 7f127ca00640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1258078a00 0x7f125807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:04.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.889+0000 7f127ca00640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1258078a00 0x7f125807aec0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1264032000 tx=0x7f1264001120 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:04.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:04.892+0000 7f1275600640 1 -- 172.21.15.60:0/3576407641 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f126c0632d0 con 0x7f1260004800 2024-09-16T15:42:05.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.124+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12780b18b0 con 0x7f1260004800 2024-09-16T15:42:05.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.124+0000 7f1275600640 1 -- 172.21.15.60:0/3576407641 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5226 (secure 0 0 0) 0x7f126c067d00 con 0x7f1260004800 2024-09-16T15:42:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:05.126 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":36,"btime":"2024-09-16T15:42:04:690732+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5671,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3625257481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3625257481},{"type":"v1","addr":"172.21.15.60:6834","nonce":3625257481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2519251073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2519251073},{"type":"v1","addr":"172.21.15.100:6837","nonce":2519251073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/333738926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":333738926},{"type":"v1","addr":"172.21.15.60:6835","nonce":333738926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:01.154412+0000","modified":"2024-09-16T15:42:04.123997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2580251790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2580251790},{"type":"v1","addr":"172.21.15.100:6836","nonce":2580251790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5757,"qdb_cluster":[5757]},"id":4}]} 2024-09-16T15:42:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 36 2024-09-16T15:42:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1258078a00 msgr2=0x7f125807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1258078a00 0x7f125807aec0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1264032000 tx=0x7f1264001120 comp rx=0 tx=0).stop 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 msgr2=0x7f12780d2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 0x7f12780d2600 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f126c011a10 tx=0x7f126c011ed0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 shutdown_connections 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1258078a00 0x7f125807aec0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12780cfbc0 0x7f12780c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f127815e6d0 0x7f127815f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 --2- 172.21.15.60:0/3576407641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1260004800 0x7f12780d2600 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 >> 172.21.15.60:0/3576407641 conn(0x7f127806d230 msgr2=0x7f12780cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 shutdown_connections 2024-09-16T15:42:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.127+0000 7f127fe00640 1 -- 172.21.15.60:0/3576407641 wait complete. 2024-09-16T15:42:05.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:05.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.304+0000 7f714cc00640 1 -- 172.21.15.60:0/516401350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b4050 msgr2=0x7f71480b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.304+0000 7f714cc00640 1 --2- 172.21.15.60:0/516401350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b4050 0x7f71480b1770 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f71480b6930 tx=0x7f714002a820 comp rx=0 tx=0).stop 2024-09-16T15:42:05.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.304+0000 7f714cc00640 1 -- 172.21.15.60:0/516401350 shutdown_connections 2024-09-16T15:42:05.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.304+0000 7f714cc00640 1 --2- 172.21.15.60:0/516401350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b4050 0x7f71480b1770 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.304+0000 7f714cc00640 1 --2- 172.21.15.60:0/516401350 >> v2:172.21.15.100:3300/0 conn(0x7f7148161440 0x7f71480b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.304+0000 7f714cc00640 1 -- 172.21.15.60:0/516401350 >> 172.21.15.60:0/516401350 conn(0x7f71480d3df0 msgr2=0x7f71480d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.304+0000 7f714cc00640 1 -- 172.21.15.60:0/516401350 shutdown_connections 2024-09-16T15:42:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.304+0000 7f714cc00640 1 -- 172.21.15.60:0/516401350 wait complete. 2024-09-16T15:42:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.305+0000 7f714cc00640 1 Processor -- start 2024-09-16T15:42:05.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.305+0000 7f714cc00640 1 -- start start 2024-09-16T15:42:05.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.306+0000 7f714cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b34b0 0x7f71481f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.306+0000 7f714cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71480b4050 0x7f71480d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.306+0000 7f714cc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7148161440 0x7f71480d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.306+0000 7f714cc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f71480b2040 con 0x7f71480b34b0 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.306+0000 7f714cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71480b1ec0 con 0x7f71480b4050 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.306+0000 7f714cc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f71480b21c0 con 0x7f7148161440 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b34b0 0x7f71481f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b34b0 0x7f71481f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42026/0 (socket says 172.21.15.60:42026) 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 -- 172.21.15.60:0/4064428282 learned_addr learned my addr 172.21.15.60:0/4064428282 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 -- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7148161440 msgr2=0x7f71480d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7148161440 0x7f71480d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 -- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71480b4050 msgr2=0x7f71480d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71480b4050 0x7f71480d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 -- 172.21.15.60:0/4064428282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71480cfa80 con 0x7f71480b34b0 2024-09-16T15:42:05.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.307+0000 7f7147400640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b34b0 0x7f71481f4bd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f713800d770 tx=0x7f713800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.308+0000 7f713f400640 1 -- 172.21.15.60:0/4064428282 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138017070 con 0x7f71480b34b0 2024-09-16T15:42:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.308+0000 7f713f400640 1 -- 172.21.15.60:0/4064428282 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7138009d20 con 0x7f71480b34b0 2024-09-16T15:42:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.308+0000 7f713f400640 1 -- 172.21.15.60:0/4064428282 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71380106c0 con 0x7f71480b34b0 2024-09-16T15:42:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.308+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71480cfd20 con 0x7f71480b34b0 2024-09-16T15:42:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.308+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71480d02e0 con 0x7f71480b34b0 2024-09-16T15:42:05.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.309+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71480b2f60 con 0x7f71480b34b0 2024-09-16T15:42:05.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.310+0000 7f713f400640 1 -- 172.21.15.60:0/4064428282 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f713801a070 con 0x7f71480b34b0 2024-09-16T15:42:05.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.311+0000 7f713f400640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7128078a00 0x7f712807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.311+0000 7f713f400640 1 -- 172.21.15.60:0/4064428282 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f713809af00 con 0x7f71480b34b0 2024-09-16T15:42:05.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.311+0000 7f7146a00640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7128078a00 0x7f712807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.312+0000 7f7146a00640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7128078a00 0x7f712807aec0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f71480cdd10 tx=0x7f7134007a30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:05.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.315+0000 7f713f400640 1 -- 172.21.15.60:0/4064428282 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f71480b2f60 con 0x7f71480b34b0 2024-09-16T15:42:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.515+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71480cdd40 con 0x7f71480b34b0 2024-09-16T15:42:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.516+0000 7f713f400640 1 -- 172.21.15.60:0/4064428282 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16725 (secure 0 0 0) 0x7f7138063140 con 0x7f71480b34b0 2024-09-16T15:42:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:05.518 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:02.674798+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:41:58.905230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:00.065450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7128078a00 msgr2=0x7f712807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7128078a00 0x7f712807aec0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f71480cdd10 tx=0x7f7134007a30 comp rx=0 tx=0).stop 2024-09-16T15:42:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b34b0 msgr2=0x7f71481f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b34b0 0x7f71481f4bd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f713800d770 tx=0x7f713800dc30 comp rx=0 tx=0).stop 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 shutdown_connections 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7128078a00 0x7f712807aec0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7148161440 0x7f71480d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71480b4050 0x7f71480d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 --2- 172.21.15.60:0/4064428282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71480b34b0 0x7f71481f4bd0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 >> 172.21.15.60:0/4064428282 conn(0x7f71480d3df0 msgr2=0x7f714815bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 shutdown_connections 2024-09-16T15:42:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.519+0000 7f714cc00640 1 -- 172.21.15.60:0/4064428282 wait complete. 2024-09-16T15:42:05.536 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:42:05.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.694+0000 7f56b5a00640 1 -- 172.21.15.60:0/3014131803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b0161550 msgr2=0x7f56b006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.694+0000 7f56b5a00640 1 --2- 172.21.15.60:0/3014131803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b0161550 0x7f56b006ef50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f56b00752e0 tx=0x7f56a002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.694+0000 7f56b5a00640 1 -- 172.21.15.60:0/3014131803 shutdown_connections 2024-09-16T15:42:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.694+0000 7f56b5a00640 1 --2- 172.21.15.60:0/3014131803 >> v2:172.21.15.100:3300/0 conn(0x7f56b00af3d0 0x7f56b006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.694+0000 7f56b5a00640 1 --2- 172.21.15.60:0/3014131803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b0161550 0x7f56b006ef50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.694+0000 7f56b5a00640 1 -- 172.21.15.60:0/3014131803 >> 172.21.15.60:0/3014131803 conn(0x7f56b00b1950 msgr2=0x7f56b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.694+0000 7f56b5a00640 1 -- 172.21.15.60:0/3014131803 shutdown_connections 2024-09-16T15:42:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.695+0000 7f56b5a00640 1 -- 172.21.15.60:0/3014131803 wait complete. 2024-09-16T15:42:05.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.696+0000 7f56b5a00640 1 Processor -- start 2024-09-16T15:42:05.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.696+0000 7f56b5a00640 1 -- start start 2024-09-16T15:42:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.697+0000 7f56b5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b006f9d0 0x7f56b00cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.697+0000 7f56b5a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56b00af3d0 0x7f56b01f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.697+0000 7f56b5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b0161550 0x7f56b01f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.697+0000 7f56b5a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f56b0161a40 con 0x7f56b006f9d0 2024-09-16T15:42:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.697+0000 7f56b5a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56b0074010 con 0x7f56b0161550 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.697+0000 7f56b5a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f56b01623a0 con 0x7f56b00af3d0 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b006f9d0 0x7f56b00cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b006f9d0 0x7f56b00cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42046/0 (socket says 172.21.15.60:42046) 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 -- 172.21.15.60:0/261327565 learned_addr learned my addr 172.21.15.60:0/261327565 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56a6000640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56b00af3d0 0x7f56b01f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56af400640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b0161550 0x7f56b01f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 -- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56b00af3d0 msgr2=0x7f56b01f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56b00af3d0 0x7f56b01f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 -- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b0161550 msgr2=0x7f56b01f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b0161550 0x7f56b01f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 -- 172.21.15.60:0/261327565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b01f5830 con 0x7f56b006f9d0 2024-09-16T15:42:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56af400640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b0161550 0x7f56b01f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:05.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.698+0000 7f56aea00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b006f9d0 0x7f56b00cd050 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f56a0034040 tx=0x7f56a00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:05.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.699+0000 7f56a7400640 1 -- 172.21.15.60:0/261327565 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a003e030 con 0x7f56b006f9d0 2024-09-16T15:42:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.699+0000 7f56a7400640 1 -- 172.21.15.60:0/261327565 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56a00047a0 con 0x7f56b006f9d0 2024-09-16T15:42:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.699+0000 7f56a7400640 1 -- 172.21.15.60:0/261327565 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0035040 con 0x7f56b006f9d0 2024-09-16T15:42:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.699+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b00ad2d0 con 0x7f56b006f9d0 2024-09-16T15:42:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.700+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b00ad890 con 0x7f56b006f9d0 2024-09-16T15:42:05.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.701+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b0162850 con 0x7f56b006f9d0 2024-09-16T15:42:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.702+0000 7f56a7400640 1 -- 172.21.15.60:0/261327565 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f56a0041030 con 0x7f56b006f9d0 2024-09-16T15:42:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.703+0000 7f56a7400640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f568c078a00 0x7f568c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.703+0000 7f56a7400640 1 -- 172.21.15.60:0/261327565 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f56a00bb390 con 0x7f56b006f9d0 2024-09-16T15:42:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.703+0000 7f56a6000640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f568c078a00 0x7f568c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.704+0000 7f56a6000640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f568c078a00 0x7f568c07aec0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56b00d0720 tx=0x7f5698009110 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:05.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.707+0000 7f56a7400640 1 -- 172.21.15.60:0/261327565 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f56b01f5830 con 0x7f56b006f9d0 2024-09-16T15:42:05.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.898+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56b0074420 con 0x7f568c078a00 2024-09-16T15:42:05.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.903+0000 7f56a7400640 1 -- 172.21.15.60:0/261327565 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f56b0074420 con 0x7f568c078a00 2024-09-16T15:42:05.907 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:42:05.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.907+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f568c078a00 msgr2=0x7f568c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.907+0000 7f56b5a00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f568c078a00 0x7f568c07aec0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56b00d0720 tx=0x7f5698009110 comp rx=0 tx=0).stop 2024-09-16T15:42:05.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.907+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b006f9d0 msgr2=0x7f56b00cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.907+0000 7f56b5a00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b006f9d0 0x7f56b00cd050 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f56a0034040 tx=0x7f56a00039e0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.908+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 shutdown_connections 2024-09-16T15:42:05.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.908+0000 7f56b5a00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f568c078a00 0x7f568c07aec0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.908+0000 7f56b5a00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b0161550 0x7f56b01f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.908+0000 7f56b5a00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56b00af3d0 0x7f56b01f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.908+0000 7f56b5a00640 1 --2- 172.21.15.60:0/261327565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56b006f9d0 0x7f56b00cd050 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.908+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 >> 172.21.15.60:0/261327565 conn(0x7f56b00b1950 msgr2=0x7f56b015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:05.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.908+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 shutdown_connections 2024-09-16T15:42:05.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:05.908+0000 7f56b5a00640 1 -- 172.21.15.60:0/261327565 wait complete. 2024-09-16T15:42:05.924 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-16T15:42:05.764436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243936,"kb_used_data":6884,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730784,"statfs":{"total":773094113280,"available":772844322816,"internally_reserved":0,"allocated":7049216,"data_stored":3584551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001522"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684673+0000","last_change":"2024-09-16T15:42:01.701872+0000","last_active":"2024-09-16T15:42:02.684673+0000","last_peered":"2024-09-16T15:42:02.684673+0000","last_clean":"2024-09-16T15:42:02.684673+0000","last_became_active":"2024-09-16T15:42:01.701561+0000","last_became_peered":"2024-09-16T15:42:01.701561+0000","last_unstale":"2024-09-16T15:42:02.684673+0000","last_undegraded":"2024-09-16T15:42:02.684673+0000","last_fullsized":"2024-09-16T15:42:02.684673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:42:59.032177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.696246+0000","last_change":"2024-09-16T15:42:01.685789+0000","last_active":"2024-09-16T15:42:01.696246+0000","last_peered":"2024-09-16T15:42:01.696246+0000","last_clean":"2024-09-16T15:42:01.696246+0000","last_became_active":"2024-09-16T15:42:01.685531+0000","last_became_peered":"2024-09-16T15:42:01.685531+0000","last_unstale":"2024-09-16T15:42:01.696246+0000","last_undegraded":"2024-09-16T15:42:01.696246+0000","last_fullsized":"2024-09-16T15:42:01.696246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:06.436205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.690537+0000","last_change":"2024-09-16T15:42:01.685980+0000","last_active":"2024-09-16T15:42:01.690537+0000","last_peered":"2024-09-16T15:42:01.690537+0000","last_clean":"2024-09-16T15:42:01.690537+0000","last_became_active":"2024-09-16T15:42:01.685681+0000","last_became_peered":"2024-09-16T15:42:01.685681+0000","last_unstale":"2024-09-16T15:42:01.690537+0000","last_undegraded":"2024-09-16T15:42:01.690537+0000","last_fullsized":"2024-09-16T15:42:01.690537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:43.243954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697584+0000","last_change":"2024-09-16T15:42:01.693833+0000","last_active":"2024-09-16T15:42:01.697584+0000","last_peered":"2024-09-16T15:42:01.697584+0000","last_clean":"2024-09-16T15:42:01.697584+0000","last_became_active":"2024-09-16T15:42:01.693552+0000","last_became_peered":"2024-09-16T15:42:01.693552+0000","last_unstale":"2024-09-16T15:42:01.697584+0000","last_undegraded":"2024-09-16T15:42:01.697584+0000","last_fullsized":"2024-09-16T15:42:01.697584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:37:59.303591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684501+0000","last_change":"2024-09-16T15:42:01.699017+0000","last_active":"2024-09-16T15:42:02.684501+0000","last_peered":"2024-09-16T15:42:02.684501+0000","last_clean":"2024-09-16T15:42:02.684501+0000","last_became_active":"2024-09-16T15:42:01.698745+0000","last_became_peered":"2024-09-16T15:42:01.698745+0000","last_unstale":"2024-09-16T15:42:02.684501+0000","last_undegraded":"2024-09-16T15:42:02.684501+0000","last_fullsized":"2024-09-16T15:42:02.684501+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:40.196318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857682+0000","last_change":"2024-09-16T15:42:01.705189+0000","last_active":"2024-09-16T15:42:02.857682+0000","last_peered":"2024-09-16T15:42:02.857682+0000","last_clean":"2024-09-16T15:42:02.857682+0000","last_became_active":"2024-09-16T15:42:01.702343+0000","last_became_peered":"2024-09-16T15:42:01.702343+0000","last_unstale":"2024-09-16T15:42:02.857682+0000","last_undegraded":"2024-09-16T15:42:02.857682+0000","last_fullsized":"2024-09-16T15:42:02.857682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:29.517420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.685980+0000","last_change":"2024-09-16T15:42:01.691969+0000","last_active":"2024-09-16T15:42:02.685980+0000","last_peered":"2024-09-16T15:42:02.685980+0000","last_clean":"2024-09-16T15:42:02.685980+0000","last_became_active":"2024-09-16T15:42:01.691680+0000","last_became_peered":"2024-09-16T15:42:01.691680+0000","last_unstale":"2024-09-16T15:42:02.685980+0000","last_undegraded":"2024-09-16T15:42:02.685980+0000","last_fullsized":"2024-09-16T15:42:02.685980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:09.922958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.689343+0000","last_change":"2024-09-16T15:42:01.675850+0000","last_active":"2024-09-16T15:42:01.689343+0000","last_peered":"2024-09-16T15:42:01.689343+0000","last_clean":"2024-09-16T15:42:01.689343+0000","last_became_active":"2024-09-16T15:42:01.675573+0000","last_became_peered":"2024-09-16T15:42:01.675573+0000","last_unstale":"2024-09-16T15:42:01.689343+0000","last_undegraded":"2024-09-16T15:42:01.689343+0000","last_fullsized":"2024-09-16T15:42:01.689343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:23.865937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.687239+0000","last_change":"2024-09-16T15:42:01.686826+0000","last_active":"2024-09-16T15:42:02.687239+0000","last_peered":"2024-09-16T15:42:02.687239+0000","last_clean":"2024-09-16T15:42:02.687239+0000","last_became_active":"2024-09-16T15:42:01.686500+0000","last_became_peered":"2024-09-16T15:42:01.686500+0000","last_unstale":"2024-09-16T15:42:02.687239+0000","last_undegraded":"2024-09-16T15:42:02.687239+0000","last_fullsized":"2024-09-16T15:42:02.687239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:39.983556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.693125+0000","last_change":"2024-09-16T15:42:01.686295+0000","last_active":"2024-09-16T15:42:01.693125+0000","last_peered":"2024-09-16T15:42:01.693125+0000","last_clean":"2024-09-16T15:42:01.693125+0000","last_became_active":"2024-09-16T15:42:01.685807+0000","last_became_peered":"2024-09-16T15:42:01.685807+0000","last_unstale":"2024-09-16T15:42:01.693125+0000","last_undegraded":"2024-09-16T15:42:01.693125+0000","last_fullsized":"2024-09-16T15:42:01.693125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:40.542270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.696354+0000","last_change":"2024-09-16T15:42:01.691986+0000","last_active":"2024-09-16T15:42:01.696354+0000","last_peered":"2024-09-16T15:42:01.696354+0000","last_clean":"2024-09-16T15:42:01.696354+0000","last_became_active":"2024-09-16T15:42:01.691695+0000","last_became_peered":"2024-09-16T15:42:01.691695+0000","last_unstale":"2024-09-16T15:42:01.696354+0000","last_undegraded":"2024-09-16T15:42:01.696354+0000","last_fullsized":"2024-09-16T15:42:01.696354+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:21.369098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697560+0000","last_change":"2024-09-16T15:42:01.693507+0000","last_active":"2024-09-16T15:42:01.697560+0000","last_peered":"2024-09-16T15:42:01.697560+0000","last_clean":"2024-09-16T15:42:01.697560+0000","last_became_active":"2024-09-16T15:42:01.693231+0000","last_became_peered":"2024-09-16T15:42:01.693231+0000","last_unstale":"2024-09-16T15:42:01.697560+0000","last_undegraded":"2024-09-16T15:42:01.697560+0000","last_fullsized":"2024-09-16T15:42:01.697560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:57.412150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686913+0000","last_change":"2024-09-16T15:42:01.700345+0000","last_active":"2024-09-16T15:42:02.686913+0000","last_peered":"2024-09-16T15:42:02.686913+0000","last_clean":"2024-09-16T15:42:02.686913+0000","last_became_active":"2024-09-16T15:42:01.699257+0000","last_became_peered":"2024-09-16T15:42:01.699257+0000","last_unstale":"2024-09-16T15:42:02.686913+0000","last_undegraded":"2024-09-16T15:42:02.686913+0000","last_fullsized":"2024-09-16T15:42:02.686913+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:00.125483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686280+0000","last_change":"2024-09-16T15:42:01.693048+0000","last_active":"2024-09-16T15:42:02.686280+0000","last_peered":"2024-09-16T15:42:02.686280+0000","last_clean":"2024-09-16T15:42:02.686280+0000","last_became_active":"2024-09-16T15:42:01.692780+0000","last_became_peered":"2024-09-16T15:42:01.692780+0000","last_unstale":"2024-09-16T15:42:02.686280+0000","last_undegraded":"2024-09-16T15:42:02.686280+0000","last_fullsized":"2024-09-16T15:42:02.686280+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:57.412150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.685914+0000","last_change":"2024-09-16T15:42:01.677211+0000","last_active":"2024-09-16T15:42:02.685914+0000","last_peered":"2024-09-16T15:42:02.685914+0000","last_clean":"2024-09-16T15:42:02.685914+0000","last_became_active":"2024-09-16T15:42:01.676878+0000","last_became_peered":"2024-09-16T15:42:01.676878+0000","last_unstale":"2024-09-16T15:42:02.685914+0000","last_undegraded":"2024-09-16T15:42:02.685914+0000","last_fullsized":"2024-09-16T15:42:02.685914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:07.992031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857543+0000","last_change":"2024-09-16T15:42:01.689082+0000","last_active":"2024-09-16T15:42:02.857543+0000","last_peered":"2024-09-16T15:42:02.857543+0000","last_clean":"2024-09-16T15:42:02.857543+0000","last_became_active":"2024-09-16T15:42:01.688801+0000","last_became_peered":"2024-09-16T15:42:01.688801+0000","last_unstale":"2024-09-16T15:42:02.857543+0000","last_undegraded":"2024-09-16T15:42:02.857543+0000","last_fullsized":"2024-09-16T15:42:02.857543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:00.087598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.694570+0000","last_change":"2024-09-16T15:42:01.684771+0000","last_active":"2024-09-16T15:42:01.694570+0000","last_peered":"2024-09-16T15:42:01.694570+0000","last_clean":"2024-09-16T15:42:01.694570+0000","last_became_active":"2024-09-16T15:42:01.684420+0000","last_became_peered":"2024-09-16T15:42:01.684420+0000","last_unstale":"2024-09-16T15:42:01.694570+0000","last_undegraded":"2024-09-16T15:42:01.694570+0000","last_fullsized":"2024-09-16T15:42:01.694570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:01.092167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.702431+0000","last_change":"2024-09-16T15:42:01.700407+0000","last_active":"2024-09-16T15:42:01.702431+0000","last_peered":"2024-09-16T15:42:01.702431+0000","last_clean":"2024-09-16T15:42:01.702431+0000","last_became_active":"2024-09-16T15:42:01.700026+0000","last_became_peered":"2024-09-16T15:42:01.700026+0000","last_unstale":"2024-09-16T15:42:01.702431+0000","last_undegraded":"2024-09-16T15:42:01.702431+0000","last_fullsized":"2024-09-16T15:42:01.702431+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:50.367222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684720+0000","last_change":"2024-09-16T15:42:01.699059+0000","last_active":"2024-09-16T15:42:02.684720+0000","last_peered":"2024-09-16T15:42:02.684720+0000","last_clean":"2024-09-16T15:42:02.684720+0000","last_became_active":"2024-09-16T15:42:01.698740+0000","last_became_peered":"2024-09-16T15:42:01.698740+0000","last_unstale":"2024-09-16T15:42:02.684720+0000","last_undegraded":"2024-09-16T15:42:02.684720+0000","last_fullsized":"2024-09-16T15:42:02.684720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:44.991672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.703429+0000","last_change":"2024-09-16T15:42:01.700188+0000","last_active":"2024-09-16T15:42:01.703429+0000","last_peered":"2024-09-16T15:42:01.703429+0000","last_clean":"2024-09-16T15:42:01.703429+0000","last_became_active":"2024-09-16T15:42:01.699918+0000","last_became_peered":"2024-09-16T15:42:01.699918+0000","last_unstale":"2024-09-16T15:42:01.703429+0000","last_undegraded":"2024-09-16T15:42:01.703429+0000","last_fullsized":"2024-09-16T15:42:01.703429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:55.757473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701921+0000","last_change":"2024-09-16T15:42:01.698181+0000","last_active":"2024-09-16T15:42:01.701921+0000","last_peered":"2024-09-16T15:42:01.701921+0000","last_clean":"2024-09-16T15:42:01.701921+0000","last_became_active":"2024-09-16T15:42:01.697744+0000","last_became_peered":"2024-09-16T15:42:01.697744+0000","last_unstale":"2024-09-16T15:42:01.701921+0000","last_undegraded":"2024-09-16T15:42:01.701921+0000","last_fullsized":"2024-09-16T15:42:01.701921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:31.877426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.704453+0000","last_change":"2024-09-16T15:42:01.702253+0000","last_active":"2024-09-16T15:42:01.704453+0000","last_peered":"2024-09-16T15:42:01.704453+0000","last_clean":"2024-09-16T15:42:01.704453+0000","last_became_active":"2024-09-16T15:42:01.701998+0000","last_became_peered":"2024-09-16T15:42:01.701998+0000","last_unstale":"2024-09-16T15:42:01.704453+0000","last_undegraded":"2024-09-16T15:42:01.704453+0000","last_fullsized":"2024-09-16T15:42:01.704453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:21.679693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697517+0000","last_change":"2024-09-16T15:42:01.692774+0000","last_active":"2024-09-16T15:42:01.697517+0000","last_peered":"2024-09-16T15:42:01.697517+0000","last_clean":"2024-09-16T15:42:01.697517+0000","last_became_active":"2024-09-16T15:42:01.692195+0000","last_became_peered":"2024-09-16T15:42:01.692195+0000","last_unstale":"2024-09-16T15:42:01.697517+0000","last_undegraded":"2024-09-16T15:42:01.697517+0000","last_fullsized":"2024-09-16T15:42:01.697517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:14.724788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686558+0000","last_change":"2024-09-16T15:42:01.692650+0000","last_active":"2024-09-16T15:42:02.686558+0000","last_peered":"2024-09-16T15:42:02.686558+0000","last_clean":"2024-09-16T15:42:02.686558+0000","last_became_active":"2024-09-16T15:42:01.691521+0000","last_became_peered":"2024-09-16T15:42:01.691521+0000","last_unstale":"2024-09-16T15:42:02.686558+0000","last_undegraded":"2024-09-16T15:42:02.686558+0000","last_fullsized":"2024-09-16T15:42:02.686558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:35.752581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684773+0000","last_change":"2024-09-16T15:42:01.695241+0000","last_active":"2024-09-16T15:42:02.684773+0000","last_peered":"2024-09-16T15:42:02.684773+0000","last_clean":"2024-09-16T15:42:02.684773+0000","last_became_active":"2024-09-16T15:42:01.694978+0000","last_became_peered":"2024-09-16T15:42:01.694978+0000","last_unstale":"2024-09-16T15:42:02.684773+0000","last_undegraded":"2024-09-16T15:42:02.684773+0000","last_fullsized":"2024-09-16T15:42:02.684773+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:51:48.045488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.700637+0000","last_change":"2024-09-16T15:42:01.696260+0000","last_active":"2024-09-16T15:42:01.700637+0000","last_peered":"2024-09-16T15:42:01.700637+0000","last_clean":"2024-09-16T15:42:01.700637+0000","last_became_active":"2024-09-16T15:42:01.695884+0000","last_became_peered":"2024-09-16T15:42:01.695884+0000","last_unstale":"2024-09-16T15:42:01.700637+0000","last_undegraded":"2024-09-16T15:42:01.700637+0000","last_fullsized":"2024-09-16T15:42:01.700637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:58:24.956096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857836+0000","last_change":"2024-09-16T15:42:01.696919+0000","last_active":"2024-09-16T15:42:02.857836+0000","last_peered":"2024-09-16T15:42:02.857836+0000","last_clean":"2024-09-16T15:42:02.857836+0000","last_became_active":"2024-09-16T15:42:01.695287+0000","last_became_peered":"2024-09-16T15:42:01.695287+0000","last_unstale":"2024-09-16T15:42:02.857836+0000","last_undegraded":"2024-09-16T15:42:02.857836+0000","last_fullsized":"2024-09-16T15:42:02.857836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:06.436205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684698+0000","last_change":"2024-09-16T15:42:01.699269+0000","last_active":"2024-09-16T15:42:02.684698+0000","last_peered":"2024-09-16T15:42:02.684698+0000","last_clean":"2024-09-16T15:42:02.684698+0000","last_became_active":"2024-09-16T15:42:01.698989+0000","last_became_peered":"2024-09-16T15:42:01.698989+0000","last_unstale":"2024-09-16T15:42:02.684698+0000","last_undegraded":"2024-09-16T15:42:02.684698+0000","last_fullsized":"2024-09-16T15:42:02.684698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:24.687399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.695016+0000","last_change":"2024-09-16T15:42:01.687787+0000","last_active":"2024-09-16T15:42:01.695016+0000","last_peered":"2024-09-16T15:42:01.695016+0000","last_clean":"2024-09-16T15:42:01.695016+0000","last_became_active":"2024-09-16T15:42:01.687458+0000","last_became_peered":"2024-09-16T15:42:01.687458+0000","last_unstale":"2024-09-16T15:42:01.695016+0000","last_undegraded":"2024-09-16T15:42:01.695016+0000","last_fullsized":"2024-09-16T15:42:01.695016+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:45.764966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.691333+0000","last_change":"2024-09-16T15:42:01.686550+0000","last_active":"2024-09-16T15:42:01.691333+0000","last_peered":"2024-09-16T15:42:01.691333+0000","last_clean":"2024-09-16T15:42:01.691333+0000","last_became_active":"2024-09-16T15:42:01.685725+0000","last_became_peered":"2024-09-16T15:42:01.685725+0000","last_unstale":"2024-09-16T15:42:01.691333+0000","last_undegraded":"2024-09-16T15:42:01.691333+0000","last_fullsized":"2024-09-16T15:42:01.691333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:07.737557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684407+0000","last_change":"2024-09-16T15:42:01.696865+0000","last_active":"2024-09-16T15:42:02.684407+0000","last_peered":"2024-09-16T15:42:02.684407+0000","last_clean":"2024-09-16T15:42:02.684407+0000","last_became_active":"2024-09-16T15:42:01.696586+0000","last_became_peered":"2024-09-16T15:42:01.696586+0000","last_unstale":"2024-09-16T15:42:02.684407+0000","last_undegraded":"2024-09-16T15:42:02.684407+0000","last_fullsized":"2024-09-16T15:42:02.684407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:18.489011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857581+0000","last_change":"2024-09-16T15:42:01.695603+0000","last_active":"2024-09-16T15:42:02.857581+0000","last_peered":"2024-09-16T15:42:02.857581+0000","last_clean":"2024-09-16T15:42:02.857581+0000","last_became_active":"2024-09-16T15:42:01.695243+0000","last_became_peered":"2024-09-16T15:42:01.695243+0000","last_unstale":"2024-09-16T15:42:02.857581+0000","last_undegraded":"2024-09-16T15:42:02.857581+0000","last_fullsized":"2024-09-16T15:42:02.857581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:52.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857798+0000","last_change":"2024-09-16T15:42:01.704509+0000","last_active":"2024-09-16T15:42:02.857798+0000","last_peered":"2024-09-16T15:42:02.857798+0000","last_clean":"2024-09-16T15:42:02.857798+0000","last_became_active":"2024-09-16T15:42:01.697495+0000","last_became_peered":"2024-09-16T15:42:01.697495+0000","last_unstale":"2024-09-16T15:42:02.857798+0000","last_undegraded":"2024-09-16T15:42:02.857798+0000","last_fullsized":"2024-09-16T15:42:02.857798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:01.092167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.858065+0000","last_change":"2024-09-16T15:42:00.688890+0000","last_active":"2024-09-16T15:42:02.858065+0000","last_peered":"2024-09-16T15:42:02.858065+0000","last_clean":"2024-09-16T15:42:02.858065+0000","last_became_active":"2024-09-16T15:42:00.688622+0000","last_became_peered":"2024-09-16T15:42:00.688622+0000","last_unstale":"2024-09-16T15:42:02.858065+0000","last_undegraded":"2024-09-16T15:42:02.858065+0000","last_fullsized":"2024-09-16T15:42:02.858065+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:30.365719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688701+0000","last_change":"2024-09-16T15:42:01.681732+0000","last_active":"2024-09-16T15:42:01.688701+0000","last_peered":"2024-09-16T15:42:01.688701+0000","last_clean":"2024-09-16T15:42:01.688701+0000","last_became_active":"2024-09-16T15:42:01.681460+0000","last_became_peered":"2024-09-16T15:42:01.681460+0000","last_unstale":"2024-09-16T15:42:01.688701+0000","last_undegraded":"2024-09-16T15:42:01.688701+0000","last_fullsized":"2024-09-16T15:42:01.688701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:36.579086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.703106+0000","last_change":"2024-09-16T15:42:00.695966+0000","last_active":"2024-09-16T15:42:01.703106+0000","last_peered":"2024-09-16T15:42:01.703106+0000","last_clean":"2024-09-16T15:42:01.703106+0000","last_became_active":"2024-09-16T15:42:00.695723+0000","last_became_peered":"2024-09-16T15:42:00.695723+0000","last_unstale":"2024-09-16T15:42:01.703106+0000","last_undegraded":"2024-09-16T15:42:01.703106+0000","last_fullsized":"2024-09-16T15:42:01.703106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:01.922450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.858035+0000","last_change":"2024-09-16T15:42:01.694117+0000","last_active":"2024-09-16T15:42:02.858035+0000","last_peered":"2024-09-16T15:42:02.858035+0000","last_clean":"2024-09-16T15:42:02.858035+0000","last_became_active":"2024-09-16T15:42:01.693875+0000","last_became_peered":"2024-09-16T15:42:01.693875+0000","last_unstale":"2024-09-16T15:42:02.858035+0000","last_undegraded":"2024-09-16T15:42:02.858035+0000","last_fullsized":"2024-09-16T15:42:02.858035+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:45.933455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.858037+0000","last_change":"2024-09-16T15:42:00.676919+0000","last_active":"2024-09-16T15:42:02.858037+0000","last_peered":"2024-09-16T15:42:02.858037+0000","last_clean":"2024-09-16T15:42:02.858037+0000","last_became_active":"2024-09-16T15:42:00.675876+0000","last_became_peered":"2024-09-16T15:42:00.675876+0000","last_unstale":"2024-09-16T15:42:02.858037+0000","last_undegraded":"2024-09-16T15:42:02.858037+0000","last_fullsized":"2024-09-16T15:42:02.858037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:05.651155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857955+0000","last_change":"2024-09-16T15:42:01.701658+0000","last_active":"2024-09-16T15:42:02.857955+0000","last_peered":"2024-09-16T15:42:02.857955+0000","last_clean":"2024-09-16T15:42:02.857955+0000","last_became_active":"2024-09-16T15:42:01.701056+0000","last_became_peered":"2024-09-16T15:42:01.701056+0000","last_unstale":"2024-09-16T15:42:02.857955+0000","last_undegraded":"2024-09-16T15:42:02.857955+0000","last_fullsized":"2024-09-16T15:42:02.857955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:11.686124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701671+0000","last_change":"2024-09-16T15:42:00.693520+0000","last_active":"2024-09-16T15:42:01.701671+0000","last_peered":"2024-09-16T15:42:01.701671+0000","last_clean":"2024-09-16T15:42:01.701671+0000","last_became_active":"2024-09-16T15:42:00.693191+0000","last_became_peered":"2024-09-16T15:42:00.693191+0000","last_unstale":"2024-09-16T15:42:01.701671+0000","last_undegraded":"2024-09-16T15:42:01.701671+0000","last_fullsized":"2024-09-16T15:42:01.701671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:53.239092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.704430+0000","last_change":"2024-09-16T15:42:01.701397+0000","last_active":"2024-09-16T15:42:01.704430+0000","last_peered":"2024-09-16T15:42:01.704430+0000","last_clean":"2024-09-16T15:42:01.704430+0000","last_became_active":"2024-09-16T15:42:01.701143+0000","last_became_peered":"2024-09-16T15:42:01.701143+0000","last_unstale":"2024-09-16T15:42:01.704430+0000","last_undegraded":"2024-09-16T15:42:01.704430+0000","last_fullsized":"2024-09-16T15:42:01.704430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:18.024432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686343+0000","last_change":"2024-09-16T15:42:00.687567+0000","last_active":"2024-09-16T15:42:02.686343+0000","last_peered":"2024-09-16T15:42:02.686343+0000","last_clean":"2024-09-16T15:42:02.686343+0000","last_became_active":"2024-09-16T15:42:00.687251+0000","last_became_peered":"2024-09-16T15:42:00.687251+0000","last_unstale":"2024-09-16T15:42:02.686343+0000","last_undegraded":"2024-09-16T15:42:02.686343+0000","last_fullsized":"2024-09-16T15:42:02.686343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:52.890387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.703672+0000","last_change":"2024-09-16T15:42:01.700699+0000","last_active":"2024-09-16T15:42:01.703672+0000","last_peered":"2024-09-16T15:42:01.703672+0000","last_clean":"2024-09-16T15:42:01.703672+0000","last_became_active":"2024-09-16T15:42:01.700424+0000","last_became_peered":"2024-09-16T15:42:01.700424+0000","last_unstale":"2024-09-16T15:42:01.703672+0000","last_undegraded":"2024-09-16T15:42:01.703672+0000","last_fullsized":"2024-09-16T15:42:01.703672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:32.834931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857660+0000","last_change":"2024-09-16T15:42:00.684677+0000","last_active":"2024-09-16T15:42:02.857660+0000","last_peered":"2024-09-16T15:42:02.857660+0000","last_clean":"2024-09-16T15:42:02.857660+0000","last_became_active":"2024-09-16T15:42:00.684403+0000","last_became_peered":"2024-09-16T15:42:00.684403+0000","last_unstale":"2024-09-16T15:42:02.857660+0000","last_undegraded":"2024-09-16T15:42:02.857660+0000","last_fullsized":"2024-09-16T15:42:02.857660+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:34.325907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.687651+0000","last_change":"2024-09-16T15:42:01.690130+0000","last_active":"2024-09-16T15:42:02.687651+0000","last_peered":"2024-09-16T15:42:02.687651+0000","last_clean":"2024-09-16T15:42:02.687651+0000","last_became_active":"2024-09-16T15:42:01.689685+0000","last_became_peered":"2024-09-16T15:42:01.689685+0000","last_unstale":"2024-09-16T15:42:02.687651+0000","last_undegraded":"2024-09-16T15:42:02.687651+0000","last_fullsized":"2024-09-16T15:42:02.687651+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:10.053017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.679955+0000","last_change":"2024-09-16T15:42:00.684375+0000","last_active":"2024-09-16T15:42:01.679955+0000","last_peered":"2024-09-16T15:42:01.679955+0000","last_clean":"2024-09-16T15:42:01.679955+0000","last_became_active":"2024-09-16T15:42:00.684068+0000","last_became_peered":"2024-09-16T15:42:00.684068+0000","last_unstale":"2024-09-16T15:42:01.679955+0000","last_undegraded":"2024-09-16T15:42:01.679955+0000","last_fullsized":"2024-09-16T15:42:01.679955+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:35.900780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688681+0000","last_change":"2024-09-16T15:42:01.682887+0000","last_active":"2024-09-16T15:42:01.688681+0000","last_peered":"2024-09-16T15:42:01.688681+0000","last_clean":"2024-09-16T15:42:01.688681+0000","last_became_active":"2024-09-16T15:42:01.678636+0000","last_became_peered":"2024-09-16T15:42:01.678636+0000","last_unstale":"2024-09-16T15:42:01.688681+0000","last_undegraded":"2024-09-16T15:42:01.688681+0000","last_fullsized":"2024-09-16T15:42:01.688681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:52.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.685764+0000","last_change":"2024-09-16T15:42:00.691237+0000","last_active":"2024-09-16T15:42:02.685764+0000","last_peered":"2024-09-16T15:42:02.685764+0000","last_clean":"2024-09-16T15:42:02.685764+0000","last_became_active":"2024-09-16T15:42:00.690963+0000","last_became_peered":"2024-09-16T15:42:00.690963+0000","last_unstale":"2024-09-16T15:42:02.685764+0000","last_undegraded":"2024-09-16T15:42:02.685764+0000","last_fullsized":"2024-09-16T15:42:02.685764+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:31.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.687261+0000","last_change":"2024-09-16T15:42:01.684034+0000","last_active":"2024-09-16T15:42:02.687261+0000","last_peered":"2024-09-16T15:42:02.687261+0000","last_clean":"2024-09-16T15:42:02.687261+0000","last_became_active":"2024-09-16T15:42:01.683360+0000","last_became_peered":"2024-09-16T15:42:01.683360+0000","last_unstale":"2024-09-16T15:42:02.687261+0000","last_undegraded":"2024-09-16T15:42:02.687261+0000","last_fullsized":"2024-09-16T15:42:02.687261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:37:59.303591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.676145+0000","last_change":"2024-09-16T15:42:00.692094+0000","last_active":"2024-09-16T15:42:01.676145+0000","last_peered":"2024-09-16T15:42:01.676145+0000","last_clean":"2024-09-16T15:42:01.676145+0000","last_became_active":"2024-09-16T15:42:00.691880+0000","last_became_peered":"2024-09-16T15:42:00.691880+0000","last_unstale":"2024-09-16T15:42:01.676145+0000","last_undegraded":"2024-09-16T15:42:01.676145+0000","last_fullsized":"2024-09-16T15:42:01.676145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:30.365719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697876+0000","last_change":"2024-09-16T15:42:01.694070+0000","last_active":"2024-09-16T15:42:01.697876+0000","last_peered":"2024-09-16T15:42:01.697876+0000","last_clean":"2024-09-16T15:42:01.697876+0000","last_became_active":"2024-09-16T15:42:01.693772+0000","last_became_peered":"2024-09-16T15:42:01.693772+0000","last_unstale":"2024-09-16T15:42:01.697876+0000","last_undegraded":"2024-09-16T15:42:01.697876+0000","last_fullsized":"2024-09-16T15:42:01.697876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:17.273841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.669868+0000","last_change":"2024-09-16T15:42:00.688298+0000","last_active":"2024-09-16T15:42:01.669868+0000","last_peered":"2024-09-16T15:42:01.669868+0000","last_clean":"2024-09-16T15:42:01.669868+0000","last_became_active":"2024-09-16T15:42:00.688008+0000","last_became_peered":"2024-09-16T15:42:00.688008+0000","last_unstale":"2024-09-16T15:42:01.669868+0000","last_undegraded":"2024-09-16T15:42:01.669868+0000","last_fullsized":"2024-09-16T15:42:01.669868+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:56:05.548578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.693217+0000","last_change":"2024-09-16T15:42:01.686295+0000","last_active":"2024-09-16T15:42:01.693217+0000","last_peered":"2024-09-16T15:42:01.693217+0000","last_clean":"2024-09-16T15:42:01.693217+0000","last_became_active":"2024-09-16T15:42:01.685996+0000","last_became_peered":"2024-09-16T15:42:01.685996+0000","last_unstale":"2024-09-16T15:42:01.693217+0000","last_undegraded":"2024-09-16T15:42:01.693217+0000","last_fullsized":"2024-09-16T15:42:01.693217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:39:36.988064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.690285+0000","last_change":"2024-09-16T15:42:00.693398+0000","last_active":"2024-09-16T15:42:01.690285+0000","last_peered":"2024-09-16T15:42:01.690285+0000","last_clean":"2024-09-16T15:42:01.690285+0000","last_became_active":"2024-09-16T15:42:00.693093+0000","last_became_peered":"2024-09-16T15:42:00.693093+0000","last_unstale":"2024-09-16T15:42:01.690285+0000","last_undegraded":"2024-09-16T15:42:01.690285+0000","last_fullsized":"2024-09-16T15:42:01.690285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:05.651155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.695829+0000","last_change":"2024-09-16T15:42:01.689009+0000","last_active":"2024-09-16T15:42:01.695829+0000","last_peered":"2024-09-16T15:42:01.695829+0000","last_clean":"2024-09-16T15:42:01.695829+0000","last_became_active":"2024-09-16T15:42:01.687659+0000","last_became_peered":"2024-09-16T15:42:01.687659+0000","last_unstale":"2024-09-16T15:42:01.695829+0000","last_undegraded":"2024-09-16T15:42:01.695829+0000","last_fullsized":"2024-09-16T15:42:01.695829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:27.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":21,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686839+0000","last_change":"2024-09-16T15:42:00.682861+0000","last_active":"2024-09-16T15:42:02.686839+0000","last_peered":"2024-09-16T15:42:02.686839+0000","last_clean":"2024-09-16T15:42:02.686839+0000","last_became_active":"2024-09-16T15:42:00.679507+0000","last_became_peered":"2024-09-16T15:42:00.679507+0000","last_unstale":"2024-09-16T15:42:02.686839+0000","last_undegraded":"2024-09-16T15:42:02.686839+0000","last_fullsized":"2024-09-16T15:42:02.686839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:16.999055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701390+0000","last_change":"2024-09-16T15:42:01.697957+0000","last_active":"2024-09-16T15:42:01.701390+0000","last_peered":"2024-09-16T15:42:01.701390+0000","last_clean":"2024-09-16T15:42:01.701390+0000","last_became_active":"2024-09-16T15:42:01.697277+0000","last_became_peered":"2024-09-16T15:42:01.697277+0000","last_unstale":"2024-09-16T15:42:01.701390+0000","last_undegraded":"2024-09-16T15:42:01.701390+0000","last_fullsized":"2024-09-16T15:42:01.701390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:09.957927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.707987+0000","last_change":"2024-09-16T15:42:00.693171+0000","last_active":"2024-09-16T15:42:01.707987+0000","last_peered":"2024-09-16T15:42:01.707987+0000","last_clean":"2024-09-16T15:42:01.707987+0000","last_became_active":"2024-09-16T15:42:00.692922+0000","last_became_peered":"2024-09-16T15:42:00.692922+0000","last_unstale":"2024-09-16T15:42:01.707987+0000","last_undegraded":"2024-09-16T15:42:01.707987+0000","last_fullsized":"2024-09-16T15:42:01.707987+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:13.489823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688535+0000","last_change":"2024-09-16T15:42:01.681261+0000","last_active":"2024-09-16T15:42:01.688535+0000","last_peered":"2024-09-16T15:42:01.688535+0000","last_clean":"2024-09-16T15:42:01.688535+0000","last_became_active":"2024-09-16T15:42:01.681005+0000","last_became_peered":"2024-09-16T15:42:01.681005+0000","last_unstale":"2024-09-16T15:42:01.688535+0000","last_undegraded":"2024-09-16T15:42:01.688535+0000","last_fullsized":"2024-09-16T15:42:01.688535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:40.211032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686596+0000","last_change":"2024-09-16T15:42:00.686556+0000","last_active":"2024-09-16T15:42:02.686596+0000","last_peered":"2024-09-16T15:42:02.686596+0000","last_clean":"2024-09-16T15:42:02.686596+0000","last_became_active":"2024-09-16T15:42:00.685727+0000","last_became_peered":"2024-09-16T15:42:00.685727+0000","last_unstale":"2024-09-16T15:42:02.686596+0000","last_undegraded":"2024-09-16T15:42:02.686596+0000","last_fullsized":"2024-09-16T15:42:02.686596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:22.460009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.685816+0000","last_change":"2024-09-16T15:42:01.691027+0000","last_active":"2024-09-16T15:42:02.685816+0000","last_peered":"2024-09-16T15:42:02.685816+0000","last_clean":"2024-09-16T15:42:02.685816+0000","last_became_active":"2024-09-16T15:42:01.690674+0000","last_became_peered":"2024-09-16T15:42:01.690674+0000","last_unstale":"2024-09-16T15:42:02.685816+0000","last_undegraded":"2024-09-16T15:42:02.685816+0000","last_fullsized":"2024-09-16T15:42:02.685816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:55.291825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.680208+0000","last_change":"2024-09-16T15:42:00.684742+0000","last_active":"2024-09-16T15:42:01.680208+0000","last_peered":"2024-09-16T15:42:01.680208+0000","last_clean":"2024-09-16T15:42:01.680208+0000","last_became_active":"2024-09-16T15:42:00.682868+0000","last_became_peered":"2024-09-16T15:42:00.682868+0000","last_unstale":"2024-09-16T15:42:01.680208+0000","last_undegraded":"2024-09-16T15:42:01.680208+0000","last_fullsized":"2024-09-16T15:42:01.680208+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:53.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697544+0000","last_change":"2024-09-16T15:42:01.693049+0000","last_active":"2024-09-16T15:42:01.697544+0000","last_peered":"2024-09-16T15:42:01.697544+0000","last_clean":"2024-09-16T15:42:01.697544+0000","last_became_active":"2024-09-16T15:42:01.692530+0000","last_became_peered":"2024-09-16T15:42:01.692530+0000","last_unstale":"2024-09-16T15:42:01.697544+0000","last_undegraded":"2024-09-16T15:42:01.697544+0000","last_fullsized":"2024-09-16T15:42:01.697544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:08.444450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.698813+0000","last_change":"2024-09-16T15:42:00.689701+0000","last_active":"2024-09-16T15:42:01.698813+0000","last_peered":"2024-09-16T15:42:01.698813+0000","last_clean":"2024-09-16T15:42:01.698813+0000","last_became_active":"2024-09-16T15:42:00.689416+0000","last_became_peered":"2024-09-16T15:42:00.689416+0000","last_unstale":"2024-09-16T15:42:01.698813+0000","last_undegraded":"2024-09-16T15:42:01.698813+0000","last_fullsized":"2024-09-16T15:42:01.698813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:51.576762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.698228+0000","last_change":"2024-09-16T15:42:01.690293+0000","last_active":"2024-09-16T15:42:01.698228+0000","last_peered":"2024-09-16T15:42:01.698228+0000","last_clean":"2024-09-16T15:42:01.698228+0000","last_became_active":"2024-09-16T15:42:01.689998+0000","last_became_peered":"2024-09-16T15:42:01.689998+0000","last_unstale":"2024-09-16T15:42:01.698228+0000","last_undegraded":"2024-09-16T15:42:01.698228+0000","last_fullsized":"2024-09-16T15:42:01.698228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:48:00.823263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.707908+0000","last_change":"2024-09-16T15:42:00.688827+0000","last_active":"2024-09-16T15:42:01.707908+0000","last_peered":"2024-09-16T15:42:01.707908+0000","last_clean":"2024-09-16T15:42:01.707908+0000","last_became_active":"2024-09-16T15:42:00.688211+0000","last_became_peered":"2024-09-16T15:42:00.688211+0000","last_unstale":"2024-09-16T15:42:01.707908+0000","last_undegraded":"2024-09-16T15:42:01.707908+0000","last_fullsized":"2024-09-16T15:42:01.707908+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:08.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701419+0000","last_change":"2024-09-16T15:42:01.698337+0000","last_active":"2024-09-16T15:42:01.701419+0000","last_peered":"2024-09-16T15:42:01.701419+0000","last_clean":"2024-09-16T15:42:01.701419+0000","last_became_active":"2024-09-16T15:42:01.698034+0000","last_became_peered":"2024-09-16T15:42:01.698034+0000","last_unstale":"2024-09-16T15:42:01.701419+0000","last_undegraded":"2024-09-16T15:42:01.701419+0000","last_fullsized":"2024-09-16T15:42:01.701419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.929071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857803+0000","last_change":"2024-09-16T15:42:00.683923+0000","last_active":"2024-09-16T15:42:02.857803+0000","last_peered":"2024-09-16T15:42:02.857803+0000","last_clean":"2024-09-16T15:42:02.857803+0000","last_became_active":"2024-09-16T15:42:00.683535+0000","last_became_peered":"2024-09-16T15:42:00.683535+0000","last_unstale":"2024-09-16T15:42:02.857803+0000","last_undegraded":"2024-09-16T15:42:02.857803+0000","last_fullsized":"2024-09-16T15:42:02.857803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:39.204411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688551+0000","last_change":"2024-09-16T15:42:01.683286+0000","last_active":"2024-09-16T15:42:01.688551+0000","last_peered":"2024-09-16T15:42:01.688551+0000","last_clean":"2024-09-16T15:42:01.688551+0000","last_became_active":"2024-09-16T15:42:01.680838+0000","last_became_peered":"2024-09-16T15:42:01.680838+0000","last_unstale":"2024-09-16T15:42:01.688551+0000","last_undegraded":"2024-09-16T15:42:01.688551+0000","last_fullsized":"2024-09-16T15:42:01.688551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:29.517420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701671+0000","last_change":"2024-09-16T15:42:00.689508+0000","last_active":"2024-09-16T15:42:01.701671+0000","last_peered":"2024-09-16T15:42:01.701671+0000","last_clean":"2024-09-16T15:42:01.701671+0000","last_became_active":"2024-09-16T15:42:00.689241+0000","last_became_peered":"2024-09-16T15:42:00.689241+0000","last_unstale":"2024-09-16T15:42:01.701671+0000","last_undegraded":"2024-09-16T15:42:01.701671+0000","last_fullsized":"2024-09-16T15:42:01.701671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:08.916337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697836+0000","last_change":"2024-09-16T15:42:01.689979+0000","last_active":"2024-09-16T15:42:01.697836+0000","last_peered":"2024-09-16T15:42:01.697836+0000","last_clean":"2024-09-16T15:42:01.697836+0000","last_became_active":"2024-09-16T15:42:01.689512+0000","last_became_peered":"2024-09-16T15:42:01.689512+0000","last_unstale":"2024-09-16T15:42:01.697836+0000","last_undegraded":"2024-09-16T15:42:01.697836+0000","last_fullsized":"2024-09-16T15:42:01.697836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:14.491336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857953+0000","last_change":"2024-09-16T15:42:00.688739+0000","last_active":"2024-09-16T15:42:02.857953+0000","last_peered":"2024-09-16T15:42:02.857953+0000","last_clean":"2024-09-16T15:42:02.857953+0000","last_became_active":"2024-09-16T15:42:00.688462+0000","last_became_peered":"2024-09-16T15:42:00.688462+0000","last_unstale":"2024-09-16T15:42:02.857953+0000","last_undegraded":"2024-09-16T15:42:02.857953+0000","last_fullsized":"2024-09-16T15:42:02.857953+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:51.576762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701498+0000","last_change":"2024-09-16T15:42:01.699023+0000","last_active":"2024-09-16T15:42:01.701498+0000","last_peered":"2024-09-16T15:42:01.701498+0000","last_clean":"2024-09-16T15:42:01.701498+0000","last_became_active":"2024-09-16T15:42:01.698505+0000","last_became_peered":"2024-09-16T15:42:01.698505+0000","last_unstale":"2024-09-16T15:42:01.701498+0000","last_undegraded":"2024-09-16T15:42:01.701498+0000","last_fullsized":"2024-09-16T15:42:01.701498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:14.496444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.703125+0000","last_change":"2024-09-16T15:42:00.684907+0000","last_active":"2024-09-16T15:42:01.703125+0000","last_peered":"2024-09-16T15:42:01.703125+0000","last_clean":"2024-09-16T15:42:01.703125+0000","last_became_active":"2024-09-16T15:42:00.684593+0000","last_became_peered":"2024-09-16T15:42:00.684593+0000","last_unstale":"2024-09-16T15:42:01.703125+0000","last_undegraded":"2024-09-16T15:42:01.703125+0000","last_fullsized":"2024-09-16T15:42:01.703125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:15.889144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857925+0000","last_change":"2024-09-16T15:42:01.694493+0000","last_active":"2024-09-16T15:42:02.857925+0000","last_peered":"2024-09-16T15:42:02.857925+0000","last_clean":"2024-09-16T15:42:02.857925+0000","last_became_active":"2024-09-16T15:42:01.694267+0000","last_became_peered":"2024-09-16T15:42:01.694267+0000","last_unstale":"2024-09-16T15:42:02.857925+0000","last_undegraded":"2024-09-16T15:42:02.857925+0000","last_fullsized":"2024-09-16T15:42:02.857925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:05.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.677237+0000","last_change":"2024-09-16T15:42:00.682457+0000","last_active":"2024-09-16T15:42:01.677237+0000","last_peered":"2024-09-16T15:42:01.677237+0000","last_clean":"2024-09-16T15:42:01.677237+0000","last_became_active":"2024-09-16T15:42:00.682181+0000","last_became_peered":"2024-09-16T15:42:00.682181+0000","last_unstale":"2024-09-16T15:42:01.677237+0000","last_undegraded":"2024-09-16T15:42:01.677237+0000","last_fullsized":"2024-09-16T15:42:01.677237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:38.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857590+0000","last_change":"2024-09-16T15:42:01.691412+0000","last_active":"2024-09-16T15:42:02.857590+0000","last_peered":"2024-09-16T15:42:02.857590+0000","last_clean":"2024-09-16T15:42:02.857590+0000","last_became_active":"2024-09-16T15:42:01.691147+0000","last_became_peered":"2024-09-16T15:42:01.691147+0000","last_unstale":"2024-09-16T15:42:02.857590+0000","last_undegraded":"2024-09-16T15:42:02.857590+0000","last_fullsized":"2024-09-16T15:42:02.857590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:56.370244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.683181+0000","last_change":"2024-09-16T15:42:00.694921+0000","last_active":"2024-09-16T15:42:01.683181+0000","last_peered":"2024-09-16T15:42:01.683181+0000","last_clean":"2024-09-16T15:42:01.683181+0000","last_became_active":"2024-09-16T15:42:00.694618+0000","last_became_peered":"2024-09-16T15:42:00.694618+0000","last_unstale":"2024-09-16T15:42:01.683181+0000","last_undegraded":"2024-09-16T15:42:01.683181+0000","last_fullsized":"2024-09-16T15:42:01.683181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:06.985410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697702+0000","last_change":"2024-09-16T15:42:01.693761+0000","last_active":"2024-09-16T15:42:01.697702+0000","last_peered":"2024-09-16T15:42:01.697702+0000","last_clean":"2024-09-16T15:42:01.697702+0000","last_became_active":"2024-09-16T15:42:01.693471+0000","last_became_peered":"2024-09-16T15:42:01.693471+0000","last_unstale":"2024-09-16T15:42:01.697702+0000","last_undegraded":"2024-09-16T15:42:01.697702+0000","last_fullsized":"2024-09-16T15:42:01.697702+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:39.983556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.705007+0000","last_change":"2024-09-16T15:42:00.693187+0000","last_active":"2024-09-16T15:42:01.705007+0000","last_peered":"2024-09-16T15:42:01.705007+0000","last_clean":"2024-09-16T15:42:01.705007+0000","last_became_active":"2024-09-16T15:42:00.692900+0000","last_became_peered":"2024-09-16T15:42:00.692900+0000","last_unstale":"2024-09-16T15:42:01.705007+0000","last_undegraded":"2024-09-16T15:42:01.705007+0000","last_fullsized":"2024-09-16T15:42:01.705007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:29.797553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.700059+0000","last_change":"2024-09-16T15:42:01.695785+0000","last_active":"2024-09-16T15:42:01.700059+0000","last_peered":"2024-09-16T15:42:01.700059+0000","last_clean":"2024-09-16T15:42:01.700059+0000","last_became_active":"2024-09-16T15:42:01.695489+0000","last_became_peered":"2024-09-16T15:42:01.695489+0000","last_unstale":"2024-09-16T15:42:01.700059+0000","last_undegraded":"2024-09-16T15:42:01.700059+0000","last_fullsized":"2024-09-16T15:42:01.700059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:07.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857905+0000","last_change":"2024-09-16T15:42:00.679378+0000","last_active":"2024-09-16T15:42:02.857905+0000","last_peered":"2024-09-16T15:42:02.857905+0000","last_clean":"2024-09-16T15:42:02.857905+0000","last_became_active":"2024-09-16T15:42:00.678869+0000","last_became_peered":"2024-09-16T15:42:00.678869+0000","last_unstale":"2024-09-16T15:42:02.857905+0000","last_undegraded":"2024-09-16T15:42:02.857905+0000","last_fullsized":"2024-09-16T15:42:02.857905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:42.237333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.682856+0000","last_change":"2024-09-16T15:42:01.678730+0000","last_active":"2024-09-16T15:42:01.682856+0000","last_peered":"2024-09-16T15:42:01.682856+0000","last_clean":"2024-09-16T15:42:01.682856+0000","last_became_active":"2024-09-16T15:42:01.677673+0000","last_became_peered":"2024-09-16T15:42:01.677673+0000","last_unstale":"2024-09-16T15:42:01.682856+0000","last_undegraded":"2024-09-16T15:42:01.682856+0000","last_fullsized":"2024-09-16T15:42:01.682856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:59:43.943868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686801+0000","last_change":"2024-09-16T15:42:00.684446+0000","last_active":"2024-09-16T15:42:02.686801+0000","last_peered":"2024-09-16T15:42:02.686801+0000","last_clean":"2024-09-16T15:42:02.686801+0000","last_became_active":"2024-09-16T15:42:00.684199+0000","last_became_peered":"2024-09-16T15:42:00.684199+0000","last_unstale":"2024-09-16T15:42:02.686801+0000","last_undegraded":"2024-09-16T15:42:02.686801+0000","last_fullsized":"2024-09-16T15:42:02.686801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:32.522922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.858078+0000","last_change":"2024-09-16T15:42:01.701309+0000","last_active":"2024-09-16T15:42:02.858078+0000","last_peered":"2024-09-16T15:42:02.858078+0000","last_clean":"2024-09-16T15:42:02.858078+0000","last_became_active":"2024-09-16T15:42:01.700700+0000","last_became_peered":"2024-09-16T15:42:01.700700+0000","last_unstale":"2024-09-16T15:42:02.858078+0000","last_undegraded":"2024-09-16T15:42:02.858078+0000","last_fullsized":"2024-09-16T15:42:02.858078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:55.757473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.680488+0000","last_change":"2024-09-16T15:42:00.688239+0000","last_active":"2024-09-16T15:42:01.680488+0000","last_peered":"2024-09-16T15:42:01.680488+0000","last_clean":"2024-09-16T15:42:01.680488+0000","last_became_active":"2024-09-16T15:42:00.687145+0000","last_became_peered":"2024-09-16T15:42:00.687145+0000","last_unstale":"2024-09-16T15:42:01.680488+0000","last_undegraded":"2024-09-16T15:42:01.680488+0000","last_fullsized":"2024-09-16T15:42:01.680488+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:54.285204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688392+0000","last_change":"2024-09-16T15:42:01.674464+0000","last_active":"2024-09-16T15:42:01.688392+0000","last_peered":"2024-09-16T15:42:01.688392+0000","last_clean":"2024-09-16T15:42:01.688392+0000","last_became_active":"2024-09-16T15:42:01.674192+0000","last_became_peered":"2024-09-16T15:42:01.674192+0000","last_unstale":"2024-09-16T15:42:01.688392+0000","last_undegraded":"2024-09-16T15:42:01.688392+0000","last_fullsized":"2024-09-16T15:42:01.688392+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:56.370244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.677517+0000","last_change":"2024-09-16T15:42:00.685535+0000","last_active":"2024-09-16T15:42:01.677517+0000","last_peered":"2024-09-16T15:42:01.677517+0000","last_clean":"2024-09-16T15:42:01.677517+0000","last_became_active":"2024-09-16T15:42:00.684505+0000","last_became_peered":"2024-09-16T15:42:00.684505+0000","last_unstale":"2024-09-16T15:42:01.677517+0000","last_undegraded":"2024-09-16T15:42:01.677517+0000","last_fullsized":"2024-09-16T15:42:01.677517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:48.491290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697794+0000","last_change":"2024-09-16T15:42:01.692576+0000","last_active":"2024-09-16T15:42:01.697794+0000","last_peered":"2024-09-16T15:42:01.697794+0000","last_clean":"2024-09-16T15:42:01.697794+0000","last_became_active":"2024-09-16T15:42:01.691812+0000","last_became_peered":"2024-09-16T15:42:01.691812+0000","last_unstale":"2024-09-16T15:42:01.697794+0000","last_undegraded":"2024-09-16T15:42:01.697794+0000","last_fullsized":"2024-09-16T15:42:01.697794+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:52.607046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686195+0000","last_change":"2024-09-16T15:42:00.679352+0000","last_active":"2024-09-16T15:42:02.686195+0000","last_peered":"2024-09-16T15:42:02.686195+0000","last_clean":"2024-09-16T15:42:02.686195+0000","last_became_active":"2024-09-16T15:42:00.678234+0000","last_became_peered":"2024-09-16T15:42:00.678234+0000","last_unstale":"2024-09-16T15:42:02.686195+0000","last_undegraded":"2024-09-16T15:42:02.686195+0000","last_fullsized":"2024-09-16T15:42:02.686195+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:23.780987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.693574+0000","last_change":"2024-09-16T15:42:01.688820+0000","last_active":"2024-09-16T15:42:01.693574+0000","last_peered":"2024-09-16T15:42:01.693574+0000","last_clean":"2024-09-16T15:42:01.693574+0000","last_became_active":"2024-09-16T15:42:01.684663+0000","last_became_peered":"2024-09-16T15:42:01.684663+0000","last_unstale":"2024-09-16T15:42:01.693574+0000","last_undegraded":"2024-09-16T15:42:01.693574+0000","last_fullsized":"2024-09-16T15:42:01.693574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:32.409713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.676070+0000","last_change":"2024-09-16T15:42:00.681709+0000","last_active":"2024-09-16T15:42:01.676070+0000","last_peered":"2024-09-16T15:42:01.676070+0000","last_clean":"2024-09-16T15:42:01.676070+0000","last_became_active":"2024-09-16T15:42:00.681299+0000","last_became_peered":"2024-09-16T15:42:00.681299+0000","last_unstale":"2024-09-16T15:42:01.676070+0000","last_undegraded":"2024-09-16T15:42:01.676070+0000","last_fullsized":"2024-09-16T15:42:01.676070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:50.385571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.694969+0000","last_change":"2024-09-16T15:42:01.686534+0000","last_active":"2024-09-16T15:42:01.694969+0000","last_peered":"2024-09-16T15:42:01.694969+0000","last_clean":"2024-09-16T15:42:01.694969+0000","last_became_active":"2024-09-16T15:42:01.677771+0000","last_became_peered":"2024-09-16T15:42:01.677771+0000","last_unstale":"2024-09-16T15:42:01.694969+0000","last_undegraded":"2024-09-16T15:42:01.694969+0000","last_fullsized":"2024-09-16T15:42:01.694969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:51:57.849542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.668210+0000","last_change":"2024-09-16T15:42:00.694425+0000","last_active":"2024-09-16T15:42:01.668210+0000","last_peered":"2024-09-16T15:42:01.668210+0000","last_clean":"2024-09-16T15:42:01.668210+0000","last_became_active":"2024-09-16T15:42:00.691062+0000","last_became_peered":"2024-09-16T15:42:00.691062+0000","last_unstale":"2024-09-16T15:42:01.668210+0000","last_undegraded":"2024-09-16T15:42:01.668210+0000","last_fullsized":"2024-09-16T15:42:01.668210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:58:23.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.695056+0000","last_change":"2024-09-16T15:42:01.690752+0000","last_active":"2024-09-16T15:42:01.695056+0000","last_peered":"2024-09-16T15:42:01.695056+0000","last_clean":"2024-09-16T15:42:01.695056+0000","last_became_active":"2024-09-16T15:42:01.690490+0000","last_became_peered":"2024-09-16T15:42:01.690490+0000","last_unstale":"2024-09-16T15:42:01.695056+0000","last_undegraded":"2024-09-16T15:42:01.695056+0000","last_fullsized":"2024-09-16T15:42:01.695056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:45.933455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686777+0000","last_change":"2024-09-16T15:42:00.680219+0000","last_active":"2024-09-16T15:42:02.686777+0000","last_peered":"2024-09-16T15:42:02.686777+0000","last_clean":"2024-09-16T15:42:02.686777+0000","last_became_active":"2024-09-16T15:42:00.679901+0000","last_became_peered":"2024-09-16T15:42:00.679901+0000","last_unstale":"2024-09-16T15:42:02.686777+0000","last_undegraded":"2024-09-16T15:42:02.686777+0000","last_fullsized":"2024-09-16T15:42:02.686777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:05.431535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.677524+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:42:01.677524+0000","last_peered":"2024-09-16T15:42:01.677524+0000","last_clean":"2024-09-16T15:42:01.677524+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:42:01.677524+0000","last_undegraded":"2024-09-16T15:42:01.677524+0000","last_fullsized":"2024-09-16T15:42:01.677524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":1327104,"data_stored":890473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":724992,"data_stored":299945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":733184,"data_stored":300597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":301557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":733184,"data_stored":300001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":745472,"data_stored":306301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":892507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":712704,"data_stored":293170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:05.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.123+0000 7faa6e600640 1 -- 172.21.15.60:0/4181142256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa680cf300 msgr2=0x7faa680729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.123+0000 7faa6e600640 1 --2- 172.21.15.60:0/4181142256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa680cf300 0x7faa680729a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa50009900 tx=0x7faa5002a690 comp rx=0 tx=0).stop 2024-09-16T15:42:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.124+0000 7faa6e600640 1 -- 172.21.15.60:0/4181142256 shutdown_connections 2024-09-16T15:42:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.124+0000 7faa6e600640 1 --2- 172.21.15.60:0/4181142256 >> v2:172.21.15.100:3300/0 conn(0x7faa680ccfd0 0x7faa68072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.124+0000 7faa6e600640 1 --2- 172.21.15.60:0/4181142256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa680cf300 0x7faa680729a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.124+0000 7faa6e600640 1 -- 172.21.15.60:0/4181142256 >> 172.21.15.60:0/4181142256 conn(0x7faa680b1950 msgr2=0x7faa680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:06.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.124+0000 7faa6e600640 1 -- 172.21.15.60:0/4181142256 shutdown_connections 2024-09-16T15:42:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.124+0000 7faa6e600640 1 -- 172.21.15.60:0/4181142256 wait complete. 2024-09-16T15:42:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.125+0000 7faa6e600640 1 Processor -- start 2024-09-16T15:42:06.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.125+0000 7faa6e600640 1 -- start start 2024-09-16T15:42:06.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.126+0000 7faa6e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa680ccfd0 0x7faa680b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.126+0000 7faa6e600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa680cf300 0x7faa681f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.126+0000 7faa6e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa681445d0 0x7faa680c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.126+0000 7faa6e600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faa68073420 con 0x7faa681445d0 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.126+0000 7faa6e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa680b4eb0 con 0x7faa680ccfd0 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.126+0000 7faa6e600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7faa680735a0 con 0x7faa680cf300 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa681445d0 0x7faa680c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa681445d0 0x7faa680c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42048/0 (socket says 172.21.15.60:42048) 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 -- 172.21.15.60:0/903842013 learned_addr learned my addr 172.21.15.60:0/903842013 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa5ea00640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa680cf300 0x7faa681f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67400640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa680ccfd0 0x7faa680b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 -- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa680cf300 msgr2=0x7faa681f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa680cf300 0x7faa681f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 -- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa680ccfd0 msgr2=0x7faa680b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa680ccfd0 0x7faa680b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 -- 172.21.15.60:0/903842013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa681f9240 con 0x7faa681445d0 2024-09-16T15:42:06.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.127+0000 7faa67e00640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa681445d0 0x7faa680c8fd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faa6000eb10 tx=0x7faa6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:06.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.128+0000 7faa5fe00640 1 -- 172.21.15.60:0/903842013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60004b20 con 0x7faa681445d0 2024-09-16T15:42:06.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.128+0000 7faa5fe00640 1 -- 172.21.15.60:0/903842013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa60004ce0 con 0x7faa681445d0 2024-09-16T15:42:06.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.128+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6806e7b0 con 0x7faa681445d0 2024-09-16T15:42:06.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.128+0000 7faa5fe00640 1 -- 172.21.15.60:0/903842013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60005280 con 0x7faa681445d0 2024-09-16T15:42:06.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.128+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6806ed70 con 0x7faa681445d0 2024-09-16T15:42:06.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.129+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6809dcf0 con 0x7faa681445d0 2024-09-16T15:42:06.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.130+0000 7faa5fe00640 1 -- 172.21.15.60:0/903842013 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7faa60005420 con 0x7faa681445d0 2024-09-16T15:42:06.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.131+0000 7faa5fe00640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faa44078a00 0x7faa4407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.131+0000 7faa5fe00640 1 -- 172.21.15.60:0/903842013 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7faa6009b900 con 0x7faa681445d0 2024-09-16T15:42:06.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.131+0000 7faa67400640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faa44078a00 0x7faa4407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.132+0000 7faa67400640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faa44078a00 0x7faa4407aec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa68154ee0 tx=0x7faa58009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.135+0000 7faa5fe00640 1 -- 172.21.15.60:0/903842013 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faa6809dcf0 con 0x7faa681445d0 2024-09-16T15:42:06.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.326+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa68073a30 con 0x7faa44078a00 2024-09-16T15:42:06.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.331+0000 7faa5fe00640 1 -- 172.21.15.60:0/903842013 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7faa68073a30 con 0x7faa44078a00 2024-09-16T15:42:06.335 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:06.336 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:42:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faa44078a00 msgr2=0x7faa4407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faa44078a00 0x7faa4407aec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7faa68154ee0 tx=0x7faa58009040 comp rx=0 tx=0).stop 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa681445d0 msgr2=0x7faa680c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa681445d0 0x7faa680c8fd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faa6000eb10 tx=0x7faa6000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 shutdown_connections 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7faa44078a00 0x7faa4407aec0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa681445d0 0x7faa680c8fd0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa680cf300 0x7faa681f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 --2- 172.21.15.60:0/903842013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa680ccfd0 0x7faa680b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 >> 172.21.15.60:0/903842013 conn(0x7faa680b1950 msgr2=0x7faa6809ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 shutdown_connections 2024-09-16T15:42:06.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.335+0000 7faa6e600640 1 -- 172.21.15.60:0/903842013 wait complete. 2024-09-16T15:42:06.351 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-16T15:42:05.764436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243936,"kb_used_data":6884,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730784,"statfs":{"total":773094113280,"available":772844322816,"internally_reserved":0,"allocated":7049216,"data_stored":3584551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001522"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684673+0000","last_change":"2024-09-16T15:42:01.701872+0000","last_active":"2024-09-16T15:42:02.684673+0000","last_peered":"2024-09-16T15:42:02.684673+0000","last_clean":"2024-09-16T15:42:02.684673+0000","last_became_active":"2024-09-16T15:42:01.701561+0000","last_became_peered":"2024-09-16T15:42:01.701561+0000","last_unstale":"2024-09-16T15:42:02.684673+0000","last_undegraded":"2024-09-16T15:42:02.684673+0000","last_fullsized":"2024-09-16T15:42:02.684673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:42:59.032177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.696246+0000","last_change":"2024-09-16T15:42:01.685789+0000","last_active":"2024-09-16T15:42:01.696246+0000","last_peered":"2024-09-16T15:42:01.696246+0000","last_clean":"2024-09-16T15:42:01.696246+0000","last_became_active":"2024-09-16T15:42:01.685531+0000","last_became_peered":"2024-09-16T15:42:01.685531+0000","last_unstale":"2024-09-16T15:42:01.696246+0000","last_undegraded":"2024-09-16T15:42:01.696246+0000","last_fullsized":"2024-09-16T15:42:01.696246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:06.436205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.690537+0000","last_change":"2024-09-16T15:42:01.685980+0000","last_active":"2024-09-16T15:42:01.690537+0000","last_peered":"2024-09-16T15:42:01.690537+0000","last_clean":"2024-09-16T15:42:01.690537+0000","last_became_active":"2024-09-16T15:42:01.685681+0000","last_became_peered":"2024-09-16T15:42:01.685681+0000","last_unstale":"2024-09-16T15:42:01.690537+0000","last_undegraded":"2024-09-16T15:42:01.690537+0000","last_fullsized":"2024-09-16T15:42:01.690537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:43.243954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697584+0000","last_change":"2024-09-16T15:42:01.693833+0000","last_active":"2024-09-16T15:42:01.697584+0000","last_peered":"2024-09-16T15:42:01.697584+0000","last_clean":"2024-09-16T15:42:01.697584+0000","last_became_active":"2024-09-16T15:42:01.693552+0000","last_became_peered":"2024-09-16T15:42:01.693552+0000","last_unstale":"2024-09-16T15:42:01.697584+0000","last_undegraded":"2024-09-16T15:42:01.697584+0000","last_fullsized":"2024-09-16T15:42:01.697584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:37:59.303591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684501+0000","last_change":"2024-09-16T15:42:01.699017+0000","last_active":"2024-09-16T15:42:02.684501+0000","last_peered":"2024-09-16T15:42:02.684501+0000","last_clean":"2024-09-16T15:42:02.684501+0000","last_became_active":"2024-09-16T15:42:01.698745+0000","last_became_peered":"2024-09-16T15:42:01.698745+0000","last_unstale":"2024-09-16T15:42:02.684501+0000","last_undegraded":"2024-09-16T15:42:02.684501+0000","last_fullsized":"2024-09-16T15:42:02.684501+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:40.196318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857682+0000","last_change":"2024-09-16T15:42:01.705189+0000","last_active":"2024-09-16T15:42:02.857682+0000","last_peered":"2024-09-16T15:42:02.857682+0000","last_clean":"2024-09-16T15:42:02.857682+0000","last_became_active":"2024-09-16T15:42:01.702343+0000","last_became_peered":"2024-09-16T15:42:01.702343+0000","last_unstale":"2024-09-16T15:42:02.857682+0000","last_undegraded":"2024-09-16T15:42:02.857682+0000","last_fullsized":"2024-09-16T15:42:02.857682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:29.517420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.685980+0000","last_change":"2024-09-16T15:42:01.691969+0000","last_active":"2024-09-16T15:42:02.685980+0000","last_peered":"2024-09-16T15:42:02.685980+0000","last_clean":"2024-09-16T15:42:02.685980+0000","last_became_active":"2024-09-16T15:42:01.691680+0000","last_became_peered":"2024-09-16T15:42:01.691680+0000","last_unstale":"2024-09-16T15:42:02.685980+0000","last_undegraded":"2024-09-16T15:42:02.685980+0000","last_fullsized":"2024-09-16T15:42:02.685980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:09.922958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.689343+0000","last_change":"2024-09-16T15:42:01.675850+0000","last_active":"2024-09-16T15:42:01.689343+0000","last_peered":"2024-09-16T15:42:01.689343+0000","last_clean":"2024-09-16T15:42:01.689343+0000","last_became_active":"2024-09-16T15:42:01.675573+0000","last_became_peered":"2024-09-16T15:42:01.675573+0000","last_unstale":"2024-09-16T15:42:01.689343+0000","last_undegraded":"2024-09-16T15:42:01.689343+0000","last_fullsized":"2024-09-16T15:42:01.689343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:23.865937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.687239+0000","last_change":"2024-09-16T15:42:01.686826+0000","last_active":"2024-09-16T15:42:02.687239+0000","last_peered":"2024-09-16T15:42:02.687239+0000","last_clean":"2024-09-16T15:42:02.687239+0000","last_became_active":"2024-09-16T15:42:01.686500+0000","last_became_peered":"2024-09-16T15:42:01.686500+0000","last_unstale":"2024-09-16T15:42:02.687239+0000","last_undegraded":"2024-09-16T15:42:02.687239+0000","last_fullsized":"2024-09-16T15:42:02.687239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:39.983556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.693125+0000","last_change":"2024-09-16T15:42:01.686295+0000","last_active":"2024-09-16T15:42:01.693125+0000","last_peered":"2024-09-16T15:42:01.693125+0000","last_clean":"2024-09-16T15:42:01.693125+0000","last_became_active":"2024-09-16T15:42:01.685807+0000","last_became_peered":"2024-09-16T15:42:01.685807+0000","last_unstale":"2024-09-16T15:42:01.693125+0000","last_undegraded":"2024-09-16T15:42:01.693125+0000","last_fullsized":"2024-09-16T15:42:01.693125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:40.542270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.696354+0000","last_change":"2024-09-16T15:42:01.691986+0000","last_active":"2024-09-16T15:42:01.696354+0000","last_peered":"2024-09-16T15:42:01.696354+0000","last_clean":"2024-09-16T15:42:01.696354+0000","last_became_active":"2024-09-16T15:42:01.691695+0000","last_became_peered":"2024-09-16T15:42:01.691695+0000","last_unstale":"2024-09-16T15:42:01.696354+0000","last_undegraded":"2024-09-16T15:42:01.696354+0000","last_fullsized":"2024-09-16T15:42:01.696354+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:21.369098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697560+0000","last_change":"2024-09-16T15:42:01.693507+0000","last_active":"2024-09-16T15:42:01.697560+0000","last_peered":"2024-09-16T15:42:01.697560+0000","last_clean":"2024-09-16T15:42:01.697560+0000","last_became_active":"2024-09-16T15:42:01.693231+0000","last_became_peered":"2024-09-16T15:42:01.693231+0000","last_unstale":"2024-09-16T15:42:01.697560+0000","last_undegraded":"2024-09-16T15:42:01.697560+0000","last_fullsized":"2024-09-16T15:42:01.697560+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:57.412150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686913+0000","last_change":"2024-09-16T15:42:01.700345+0000","last_active":"2024-09-16T15:42:02.686913+0000","last_peered":"2024-09-16T15:42:02.686913+0000","last_clean":"2024-09-16T15:42:02.686913+0000","last_became_active":"2024-09-16T15:42:01.699257+0000","last_became_peered":"2024-09-16T15:42:01.699257+0000","last_unstale":"2024-09-16T15:42:02.686913+0000","last_undegraded":"2024-09-16T15:42:02.686913+0000","last_fullsized":"2024-09-16T15:42:02.686913+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:00.125483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686280+0000","last_change":"2024-09-16T15:42:01.693048+0000","last_active":"2024-09-16T15:42:02.686280+0000","last_peered":"2024-09-16T15:42:02.686280+0000","last_clean":"2024-09-16T15:42:02.686280+0000","last_became_active":"2024-09-16T15:42:01.692780+0000","last_became_peered":"2024-09-16T15:42:01.692780+0000","last_unstale":"2024-09-16T15:42:02.686280+0000","last_undegraded":"2024-09-16T15:42:02.686280+0000","last_fullsized":"2024-09-16T15:42:02.686280+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:57.412150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.685914+0000","last_change":"2024-09-16T15:42:01.677211+0000","last_active":"2024-09-16T15:42:02.685914+0000","last_peered":"2024-09-16T15:42:02.685914+0000","last_clean":"2024-09-16T15:42:02.685914+0000","last_became_active":"2024-09-16T15:42:01.676878+0000","last_became_peered":"2024-09-16T15:42:01.676878+0000","last_unstale":"2024-09-16T15:42:02.685914+0000","last_undegraded":"2024-09-16T15:42:02.685914+0000","last_fullsized":"2024-09-16T15:42:02.685914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:07.992031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857543+0000","last_change":"2024-09-16T15:42:01.689082+0000","last_active":"2024-09-16T15:42:02.857543+0000","last_peered":"2024-09-16T15:42:02.857543+0000","last_clean":"2024-09-16T15:42:02.857543+0000","last_became_active":"2024-09-16T15:42:01.688801+0000","last_became_peered":"2024-09-16T15:42:01.688801+0000","last_unstale":"2024-09-16T15:42:02.857543+0000","last_undegraded":"2024-09-16T15:42:02.857543+0000","last_fullsized":"2024-09-16T15:42:02.857543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:00.087598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.694570+0000","last_change":"2024-09-16T15:42:01.684771+0000","last_active":"2024-09-16T15:42:01.694570+0000","last_peered":"2024-09-16T15:42:01.694570+0000","last_clean":"2024-09-16T15:42:01.694570+0000","last_became_active":"2024-09-16T15:42:01.684420+0000","last_became_peered":"2024-09-16T15:42:01.684420+0000","last_unstale":"2024-09-16T15:42:01.694570+0000","last_undegraded":"2024-09-16T15:42:01.694570+0000","last_fullsized":"2024-09-16T15:42:01.694570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:01.092167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.702431+0000","last_change":"2024-09-16T15:42:01.700407+0000","last_active":"2024-09-16T15:42:01.702431+0000","last_peered":"2024-09-16T15:42:01.702431+0000","last_clean":"2024-09-16T15:42:01.702431+0000","last_became_active":"2024-09-16T15:42:01.700026+0000","last_became_peered":"2024-09-16T15:42:01.700026+0000","last_unstale":"2024-09-16T15:42:01.702431+0000","last_undegraded":"2024-09-16T15:42:01.702431+0000","last_fullsized":"2024-09-16T15:42:01.702431+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:06:50.367222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684720+0000","last_change":"2024-09-16T15:42:01.699059+0000","last_active":"2024-09-16T15:42:02.684720+0000","last_peered":"2024-09-16T15:42:02.684720+0000","last_clean":"2024-09-16T15:42:02.684720+0000","last_became_active":"2024-09-16T15:42:01.698740+0000","last_became_peered":"2024-09-16T15:42:01.698740+0000","last_unstale":"2024-09-16T15:42:02.684720+0000","last_undegraded":"2024-09-16T15:42:02.684720+0000","last_fullsized":"2024-09-16T15:42:02.684720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:44.991672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.703429+0000","last_change":"2024-09-16T15:42:01.700188+0000","last_active":"2024-09-16T15:42:01.703429+0000","last_peered":"2024-09-16T15:42:01.703429+0000","last_clean":"2024-09-16T15:42:01.703429+0000","last_became_active":"2024-09-16T15:42:01.699918+0000","last_became_peered":"2024-09-16T15:42:01.699918+0000","last_unstale":"2024-09-16T15:42:01.703429+0000","last_undegraded":"2024-09-16T15:42:01.703429+0000","last_fullsized":"2024-09-16T15:42:01.703429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:55.757473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701921+0000","last_change":"2024-09-16T15:42:01.698181+0000","last_active":"2024-09-16T15:42:01.701921+0000","last_peered":"2024-09-16T15:42:01.701921+0000","last_clean":"2024-09-16T15:42:01.701921+0000","last_became_active":"2024-09-16T15:42:01.697744+0000","last_became_peered":"2024-09-16T15:42:01.697744+0000","last_unstale":"2024-09-16T15:42:01.701921+0000","last_undegraded":"2024-09-16T15:42:01.701921+0000","last_fullsized":"2024-09-16T15:42:01.701921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:18:31.877426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.704453+0000","last_change":"2024-09-16T15:42:01.702253+0000","last_active":"2024-09-16T15:42:01.704453+0000","last_peered":"2024-09-16T15:42:01.704453+0000","last_clean":"2024-09-16T15:42:01.704453+0000","last_became_active":"2024-09-16T15:42:01.701998+0000","last_became_peered":"2024-09-16T15:42:01.701998+0000","last_unstale":"2024-09-16T15:42:01.704453+0000","last_undegraded":"2024-09-16T15:42:01.704453+0000","last_fullsized":"2024-09-16T15:42:01.704453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:21.679693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697517+0000","last_change":"2024-09-16T15:42:01.692774+0000","last_active":"2024-09-16T15:42:01.697517+0000","last_peered":"2024-09-16T15:42:01.697517+0000","last_clean":"2024-09-16T15:42:01.697517+0000","last_became_active":"2024-09-16T15:42:01.692195+0000","last_became_peered":"2024-09-16T15:42:01.692195+0000","last_unstale":"2024-09-16T15:42:01.697517+0000","last_undegraded":"2024-09-16T15:42:01.697517+0000","last_fullsized":"2024-09-16T15:42:01.697517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:14.724788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686558+0000","last_change":"2024-09-16T15:42:01.692650+0000","last_active":"2024-09-16T15:42:02.686558+0000","last_peered":"2024-09-16T15:42:02.686558+0000","last_clean":"2024-09-16T15:42:02.686558+0000","last_became_active":"2024-09-16T15:42:01.691521+0000","last_became_peered":"2024-09-16T15:42:01.691521+0000","last_unstale":"2024-09-16T15:42:02.686558+0000","last_undegraded":"2024-09-16T15:42:02.686558+0000","last_fullsized":"2024-09-16T15:42:02.686558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:35.752581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684773+0000","last_change":"2024-09-16T15:42:01.695241+0000","last_active":"2024-09-16T15:42:02.684773+0000","last_peered":"2024-09-16T15:42:02.684773+0000","last_clean":"2024-09-16T15:42:02.684773+0000","last_became_active":"2024-09-16T15:42:01.694978+0000","last_became_peered":"2024-09-16T15:42:01.694978+0000","last_unstale":"2024-09-16T15:42:02.684773+0000","last_undegraded":"2024-09-16T15:42:02.684773+0000","last_fullsized":"2024-09-16T15:42:02.684773+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:51:48.045488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.700637+0000","last_change":"2024-09-16T15:42:01.696260+0000","last_active":"2024-09-16T15:42:01.700637+0000","last_peered":"2024-09-16T15:42:01.700637+0000","last_clean":"2024-09-16T15:42:01.700637+0000","last_became_active":"2024-09-16T15:42:01.695884+0000","last_became_peered":"2024-09-16T15:42:01.695884+0000","last_unstale":"2024-09-16T15:42:01.700637+0000","last_undegraded":"2024-09-16T15:42:01.700637+0000","last_fullsized":"2024-09-16T15:42:01.700637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:58:24.956096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857836+0000","last_change":"2024-09-16T15:42:01.696919+0000","last_active":"2024-09-16T15:42:02.857836+0000","last_peered":"2024-09-16T15:42:02.857836+0000","last_clean":"2024-09-16T15:42:02.857836+0000","last_became_active":"2024-09-16T15:42:01.695287+0000","last_became_peered":"2024-09-16T15:42:01.695287+0000","last_unstale":"2024-09-16T15:42:02.857836+0000","last_undegraded":"2024-09-16T15:42:02.857836+0000","last_fullsized":"2024-09-16T15:42:02.857836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:06.436205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684698+0000","last_change":"2024-09-16T15:42:01.699269+0000","last_active":"2024-09-16T15:42:02.684698+0000","last_peered":"2024-09-16T15:42:02.684698+0000","last_clean":"2024-09-16T15:42:02.684698+0000","last_became_active":"2024-09-16T15:42:01.698989+0000","last_became_peered":"2024-09-16T15:42:01.698989+0000","last_unstale":"2024-09-16T15:42:02.684698+0000","last_undegraded":"2024-09-16T15:42:02.684698+0000","last_fullsized":"2024-09-16T15:42:02.684698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:24.687399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.695016+0000","last_change":"2024-09-16T15:42:01.687787+0000","last_active":"2024-09-16T15:42:01.695016+0000","last_peered":"2024-09-16T15:42:01.695016+0000","last_clean":"2024-09-16T15:42:01.695016+0000","last_became_active":"2024-09-16T15:42:01.687458+0000","last_became_peered":"2024-09-16T15:42:01.687458+0000","last_unstale":"2024-09-16T15:42:01.695016+0000","last_undegraded":"2024-09-16T15:42:01.695016+0000","last_fullsized":"2024-09-16T15:42:01.695016+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:45.764966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.691333+0000","last_change":"2024-09-16T15:42:01.686550+0000","last_active":"2024-09-16T15:42:01.691333+0000","last_peered":"2024-09-16T15:42:01.691333+0000","last_clean":"2024-09-16T15:42:01.691333+0000","last_became_active":"2024-09-16T15:42:01.685725+0000","last_became_peered":"2024-09-16T15:42:01.685725+0000","last_unstale":"2024-09-16T15:42:01.691333+0000","last_undegraded":"2024-09-16T15:42:01.691333+0000","last_fullsized":"2024-09-16T15:42:01.691333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:07.737557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.684407+0000","last_change":"2024-09-16T15:42:01.696865+0000","last_active":"2024-09-16T15:42:02.684407+0000","last_peered":"2024-09-16T15:42:02.684407+0000","last_clean":"2024-09-16T15:42:02.684407+0000","last_became_active":"2024-09-16T15:42:01.696586+0000","last_became_peered":"2024-09-16T15:42:01.696586+0000","last_unstale":"2024-09-16T15:42:02.684407+0000","last_undegraded":"2024-09-16T15:42:02.684407+0000","last_fullsized":"2024-09-16T15:42:02.684407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:18.489011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857581+0000","last_change":"2024-09-16T15:42:01.695603+0000","last_active":"2024-09-16T15:42:02.857581+0000","last_peered":"2024-09-16T15:42:02.857581+0000","last_clean":"2024-09-16T15:42:02.857581+0000","last_became_active":"2024-09-16T15:42:01.695243+0000","last_became_peered":"2024-09-16T15:42:01.695243+0000","last_unstale":"2024-09-16T15:42:02.857581+0000","last_undegraded":"2024-09-16T15:42:02.857581+0000","last_fullsized":"2024-09-16T15:42:02.857581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:52.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857798+0000","last_change":"2024-09-16T15:42:01.704509+0000","last_active":"2024-09-16T15:42:02.857798+0000","last_peered":"2024-09-16T15:42:02.857798+0000","last_clean":"2024-09-16T15:42:02.857798+0000","last_became_active":"2024-09-16T15:42:01.697495+0000","last_became_peered":"2024-09-16T15:42:01.697495+0000","last_unstale":"2024-09-16T15:42:02.857798+0000","last_undegraded":"2024-09-16T15:42:02.857798+0000","last_fullsized":"2024-09-16T15:42:02.857798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:01.092167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.858065+0000","last_change":"2024-09-16T15:42:00.688890+0000","last_active":"2024-09-16T15:42:02.858065+0000","last_peered":"2024-09-16T15:42:02.858065+0000","last_clean":"2024-09-16T15:42:02.858065+0000","last_became_active":"2024-09-16T15:42:00.688622+0000","last_became_peered":"2024-09-16T15:42:00.688622+0000","last_unstale":"2024-09-16T15:42:02.858065+0000","last_undegraded":"2024-09-16T15:42:02.858065+0000","last_fullsized":"2024-09-16T15:42:02.858065+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:30.365719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688701+0000","last_change":"2024-09-16T15:42:01.681732+0000","last_active":"2024-09-16T15:42:01.688701+0000","last_peered":"2024-09-16T15:42:01.688701+0000","last_clean":"2024-09-16T15:42:01.688701+0000","last_became_active":"2024-09-16T15:42:01.681460+0000","last_became_peered":"2024-09-16T15:42:01.681460+0000","last_unstale":"2024-09-16T15:42:01.688701+0000","last_undegraded":"2024-09-16T15:42:01.688701+0000","last_fullsized":"2024-09-16T15:42:01.688701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:36.579086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.703106+0000","last_change":"2024-09-16T15:42:00.695966+0000","last_active":"2024-09-16T15:42:01.703106+0000","last_peered":"2024-09-16T15:42:01.703106+0000","last_clean":"2024-09-16T15:42:01.703106+0000","last_became_active":"2024-09-16T15:42:00.695723+0000","last_became_peered":"2024-09-16T15:42:00.695723+0000","last_unstale":"2024-09-16T15:42:01.703106+0000","last_undegraded":"2024-09-16T15:42:01.703106+0000","last_fullsized":"2024-09-16T15:42:01.703106+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:01.922450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.858035+0000","last_change":"2024-09-16T15:42:01.694117+0000","last_active":"2024-09-16T15:42:02.858035+0000","last_peered":"2024-09-16T15:42:02.858035+0000","last_clean":"2024-09-16T15:42:02.858035+0000","last_became_active":"2024-09-16T15:42:01.693875+0000","last_became_peered":"2024-09-16T15:42:01.693875+0000","last_unstale":"2024-09-16T15:42:02.858035+0000","last_undegraded":"2024-09-16T15:42:02.858035+0000","last_fullsized":"2024-09-16T15:42:02.858035+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:45.933455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.858037+0000","last_change":"2024-09-16T15:42:00.676919+0000","last_active":"2024-09-16T15:42:02.858037+0000","last_peered":"2024-09-16T15:42:02.858037+0000","last_clean":"2024-09-16T15:42:02.858037+0000","last_became_active":"2024-09-16T15:42:00.675876+0000","last_became_peered":"2024-09-16T15:42:00.675876+0000","last_unstale":"2024-09-16T15:42:02.858037+0000","last_undegraded":"2024-09-16T15:42:02.858037+0000","last_fullsized":"2024-09-16T15:42:02.858037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:05.651155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857955+0000","last_change":"2024-09-16T15:42:01.701658+0000","last_active":"2024-09-16T15:42:02.857955+0000","last_peered":"2024-09-16T15:42:02.857955+0000","last_clean":"2024-09-16T15:42:02.857955+0000","last_became_active":"2024-09-16T15:42:01.701056+0000","last_became_peered":"2024-09-16T15:42:01.701056+0000","last_unstale":"2024-09-16T15:42:02.857955+0000","last_undegraded":"2024-09-16T15:42:02.857955+0000","last_fullsized":"2024-09-16T15:42:02.857955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:11.686124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701671+0000","last_change":"2024-09-16T15:42:00.693520+0000","last_active":"2024-09-16T15:42:01.701671+0000","last_peered":"2024-09-16T15:42:01.701671+0000","last_clean":"2024-09-16T15:42:01.701671+0000","last_became_active":"2024-09-16T15:42:00.693191+0000","last_became_peered":"2024-09-16T15:42:00.693191+0000","last_unstale":"2024-09-16T15:42:01.701671+0000","last_undegraded":"2024-09-16T15:42:01.701671+0000","last_fullsized":"2024-09-16T15:42:01.701671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:53.239092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.704430+0000","last_change":"2024-09-16T15:42:01.701397+0000","last_active":"2024-09-16T15:42:01.704430+0000","last_peered":"2024-09-16T15:42:01.704430+0000","last_clean":"2024-09-16T15:42:01.704430+0000","last_became_active":"2024-09-16T15:42:01.701143+0000","last_became_peered":"2024-09-16T15:42:01.701143+0000","last_unstale":"2024-09-16T15:42:01.704430+0000","last_undegraded":"2024-09-16T15:42:01.704430+0000","last_fullsized":"2024-09-16T15:42:01.704430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:18.024432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686343+0000","last_change":"2024-09-16T15:42:00.687567+0000","last_active":"2024-09-16T15:42:02.686343+0000","last_peered":"2024-09-16T15:42:02.686343+0000","last_clean":"2024-09-16T15:42:02.686343+0000","last_became_active":"2024-09-16T15:42:00.687251+0000","last_became_peered":"2024-09-16T15:42:00.687251+0000","last_unstale":"2024-09-16T15:42:02.686343+0000","last_undegraded":"2024-09-16T15:42:02.686343+0000","last_fullsized":"2024-09-16T15:42:02.686343+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:52.890387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.703672+0000","last_change":"2024-09-16T15:42:01.700699+0000","last_active":"2024-09-16T15:42:01.703672+0000","last_peered":"2024-09-16T15:42:01.703672+0000","last_clean":"2024-09-16T15:42:01.703672+0000","last_became_active":"2024-09-16T15:42:01.700424+0000","last_became_peered":"2024-09-16T15:42:01.700424+0000","last_unstale":"2024-09-16T15:42:01.703672+0000","last_undegraded":"2024-09-16T15:42:01.703672+0000","last_fullsized":"2024-09-16T15:42:01.703672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:32.834931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857660+0000","last_change":"2024-09-16T15:42:00.684677+0000","last_active":"2024-09-16T15:42:02.857660+0000","last_peered":"2024-09-16T15:42:02.857660+0000","last_clean":"2024-09-16T15:42:02.857660+0000","last_became_active":"2024-09-16T15:42:00.684403+0000","last_became_peered":"2024-09-16T15:42:00.684403+0000","last_unstale":"2024-09-16T15:42:02.857660+0000","last_undegraded":"2024-09-16T15:42:02.857660+0000","last_fullsized":"2024-09-16T15:42:02.857660+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:34.325907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.687651+0000","last_change":"2024-09-16T15:42:01.690130+0000","last_active":"2024-09-16T15:42:02.687651+0000","last_peered":"2024-09-16T15:42:02.687651+0000","last_clean":"2024-09-16T15:42:02.687651+0000","last_became_active":"2024-09-16T15:42:01.689685+0000","last_became_peered":"2024-09-16T15:42:01.689685+0000","last_unstale":"2024-09-16T15:42:02.687651+0000","last_undegraded":"2024-09-16T15:42:02.687651+0000","last_fullsized":"2024-09-16T15:42:02.687651+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:10.053017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.679955+0000","last_change":"2024-09-16T15:42:00.684375+0000","last_active":"2024-09-16T15:42:01.679955+0000","last_peered":"2024-09-16T15:42:01.679955+0000","last_clean":"2024-09-16T15:42:01.679955+0000","last_became_active":"2024-09-16T15:42:00.684068+0000","last_became_peered":"2024-09-16T15:42:00.684068+0000","last_unstale":"2024-09-16T15:42:01.679955+0000","last_undegraded":"2024-09-16T15:42:01.679955+0000","last_fullsized":"2024-09-16T15:42:01.679955+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:35.900780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688681+0000","last_change":"2024-09-16T15:42:01.682887+0000","last_active":"2024-09-16T15:42:01.688681+0000","last_peered":"2024-09-16T15:42:01.688681+0000","last_clean":"2024-09-16T15:42:01.688681+0000","last_became_active":"2024-09-16T15:42:01.678636+0000","last_became_peered":"2024-09-16T15:42:01.678636+0000","last_unstale":"2024-09-16T15:42:01.688681+0000","last_undegraded":"2024-09-16T15:42:01.688681+0000","last_fullsized":"2024-09-16T15:42:01.688681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:52.016186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.685764+0000","last_change":"2024-09-16T15:42:00.691237+0000","last_active":"2024-09-16T15:42:02.685764+0000","last_peered":"2024-09-16T15:42:02.685764+0000","last_clean":"2024-09-16T15:42:02.685764+0000","last_became_active":"2024-09-16T15:42:00.690963+0000","last_became_peered":"2024-09-16T15:42:00.690963+0000","last_unstale":"2024-09-16T15:42:02.685764+0000","last_undegraded":"2024-09-16T15:42:02.685764+0000","last_fullsized":"2024-09-16T15:42:02.685764+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:31.926147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.687261+0000","last_change":"2024-09-16T15:42:01.684034+0000","last_active":"2024-09-16T15:42:02.687261+0000","last_peered":"2024-09-16T15:42:02.687261+0000","last_clean":"2024-09-16T15:42:02.687261+0000","last_became_active":"2024-09-16T15:42:01.683360+0000","last_became_peered":"2024-09-16T15:42:01.683360+0000","last_unstale":"2024-09-16T15:42:02.687261+0000","last_undegraded":"2024-09-16T15:42:02.687261+0000","last_fullsized":"2024-09-16T15:42:02.687261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:37:59.303591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.676145+0000","last_change":"2024-09-16T15:42:00.692094+0000","last_active":"2024-09-16T15:42:01.676145+0000","last_peered":"2024-09-16T15:42:01.676145+0000","last_clean":"2024-09-16T15:42:01.676145+0000","last_became_active":"2024-09-16T15:42:00.691880+0000","last_became_peered":"2024-09-16T15:42:00.691880+0000","last_unstale":"2024-09-16T15:42:01.676145+0000","last_undegraded":"2024-09-16T15:42:01.676145+0000","last_fullsized":"2024-09-16T15:42:01.676145+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:30.365719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697876+0000","last_change":"2024-09-16T15:42:01.694070+0000","last_active":"2024-09-16T15:42:01.697876+0000","last_peered":"2024-09-16T15:42:01.697876+0000","last_clean":"2024-09-16T15:42:01.697876+0000","last_became_active":"2024-09-16T15:42:01.693772+0000","last_became_peered":"2024-09-16T15:42:01.693772+0000","last_unstale":"2024-09-16T15:42:01.697876+0000","last_undegraded":"2024-09-16T15:42:01.697876+0000","last_fullsized":"2024-09-16T15:42:01.697876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:17.273841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.669868+0000","last_change":"2024-09-16T15:42:00.688298+0000","last_active":"2024-09-16T15:42:01.669868+0000","last_peered":"2024-09-16T15:42:01.669868+0000","last_clean":"2024-09-16T15:42:01.669868+0000","last_became_active":"2024-09-16T15:42:00.688008+0000","last_became_peered":"2024-09-16T15:42:00.688008+0000","last_unstale":"2024-09-16T15:42:01.669868+0000","last_undegraded":"2024-09-16T15:42:01.669868+0000","last_fullsized":"2024-09-16T15:42:01.669868+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:56:05.548578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.693217+0000","last_change":"2024-09-16T15:42:01.686295+0000","last_active":"2024-09-16T15:42:01.693217+0000","last_peered":"2024-09-16T15:42:01.693217+0000","last_clean":"2024-09-16T15:42:01.693217+0000","last_became_active":"2024-09-16T15:42:01.685996+0000","last_became_peered":"2024-09-16T15:42:01.685996+0000","last_unstale":"2024-09-16T15:42:01.693217+0000","last_undegraded":"2024-09-16T15:42:01.693217+0000","last_fullsized":"2024-09-16T15:42:01.693217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:39:36.988064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.690285+0000","last_change":"2024-09-16T15:42:00.693398+0000","last_active":"2024-09-16T15:42:01.690285+0000","last_peered":"2024-09-16T15:42:01.690285+0000","last_clean":"2024-09-16T15:42:01.690285+0000","last_became_active":"2024-09-16T15:42:00.693093+0000","last_became_peered":"2024-09-16T15:42:00.693093+0000","last_unstale":"2024-09-16T15:42:01.690285+0000","last_undegraded":"2024-09-16T15:42:01.690285+0000","last_fullsized":"2024-09-16T15:42:01.690285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:05.651155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.695829+0000","last_change":"2024-09-16T15:42:01.689009+0000","last_active":"2024-09-16T15:42:01.695829+0000","last_peered":"2024-09-16T15:42:01.695829+0000","last_clean":"2024-09-16T15:42:01.695829+0000","last_became_active":"2024-09-16T15:42:01.687659+0000","last_became_peered":"2024-09-16T15:42:01.687659+0000","last_unstale":"2024-09-16T15:42:01.695829+0000","last_undegraded":"2024-09-16T15:42:01.695829+0000","last_fullsized":"2024-09-16T15:42:01.695829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:27.025617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":21,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686839+0000","last_change":"2024-09-16T15:42:00.682861+0000","last_active":"2024-09-16T15:42:02.686839+0000","last_peered":"2024-09-16T15:42:02.686839+0000","last_clean":"2024-09-16T15:42:02.686839+0000","last_became_active":"2024-09-16T15:42:00.679507+0000","last_became_peered":"2024-09-16T15:42:00.679507+0000","last_unstale":"2024-09-16T15:42:02.686839+0000","last_undegraded":"2024-09-16T15:42:02.686839+0000","last_fullsized":"2024-09-16T15:42:02.686839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:16.999055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701390+0000","last_change":"2024-09-16T15:42:01.697957+0000","last_active":"2024-09-16T15:42:01.701390+0000","last_peered":"2024-09-16T15:42:01.701390+0000","last_clean":"2024-09-16T15:42:01.701390+0000","last_became_active":"2024-09-16T15:42:01.697277+0000","last_became_peered":"2024-09-16T15:42:01.697277+0000","last_unstale":"2024-09-16T15:42:01.701390+0000","last_undegraded":"2024-09-16T15:42:01.701390+0000","last_fullsized":"2024-09-16T15:42:01.701390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:09.957927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.707987+0000","last_change":"2024-09-16T15:42:00.693171+0000","last_active":"2024-09-16T15:42:01.707987+0000","last_peered":"2024-09-16T15:42:01.707987+0000","last_clean":"2024-09-16T15:42:01.707987+0000","last_became_active":"2024-09-16T15:42:00.692922+0000","last_became_peered":"2024-09-16T15:42:00.692922+0000","last_unstale":"2024-09-16T15:42:01.707987+0000","last_undegraded":"2024-09-16T15:42:01.707987+0000","last_fullsized":"2024-09-16T15:42:01.707987+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:13.489823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688535+0000","last_change":"2024-09-16T15:42:01.681261+0000","last_active":"2024-09-16T15:42:01.688535+0000","last_peered":"2024-09-16T15:42:01.688535+0000","last_clean":"2024-09-16T15:42:01.688535+0000","last_became_active":"2024-09-16T15:42:01.681005+0000","last_became_peered":"2024-09-16T15:42:01.681005+0000","last_unstale":"2024-09-16T15:42:01.688535+0000","last_undegraded":"2024-09-16T15:42:01.688535+0000","last_fullsized":"2024-09-16T15:42:01.688535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:40.211032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686596+0000","last_change":"2024-09-16T15:42:00.686556+0000","last_active":"2024-09-16T15:42:02.686596+0000","last_peered":"2024-09-16T15:42:02.686596+0000","last_clean":"2024-09-16T15:42:02.686596+0000","last_became_active":"2024-09-16T15:42:00.685727+0000","last_became_peered":"2024-09-16T15:42:00.685727+0000","last_unstale":"2024-09-16T15:42:02.686596+0000","last_undegraded":"2024-09-16T15:42:02.686596+0000","last_fullsized":"2024-09-16T15:42:02.686596+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:22.460009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.685816+0000","last_change":"2024-09-16T15:42:01.691027+0000","last_active":"2024-09-16T15:42:02.685816+0000","last_peered":"2024-09-16T15:42:02.685816+0000","last_clean":"2024-09-16T15:42:02.685816+0000","last_became_active":"2024-09-16T15:42:01.690674+0000","last_became_peered":"2024-09-16T15:42:01.690674+0000","last_unstale":"2024-09-16T15:42:02.685816+0000","last_undegraded":"2024-09-16T15:42:02.685816+0000","last_fullsized":"2024-09-16T15:42:02.685816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:55.291825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.680208+0000","last_change":"2024-09-16T15:42:00.684742+0000","last_active":"2024-09-16T15:42:01.680208+0000","last_peered":"2024-09-16T15:42:01.680208+0000","last_clean":"2024-09-16T15:42:01.680208+0000","last_became_active":"2024-09-16T15:42:00.682868+0000","last_became_peered":"2024-09-16T15:42:00.682868+0000","last_unstale":"2024-09-16T15:42:01.680208+0000","last_undegraded":"2024-09-16T15:42:01.680208+0000","last_fullsized":"2024-09-16T15:42:01.680208+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:53.180573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697544+0000","last_change":"2024-09-16T15:42:01.693049+0000","last_active":"2024-09-16T15:42:01.697544+0000","last_peered":"2024-09-16T15:42:01.697544+0000","last_clean":"2024-09-16T15:42:01.697544+0000","last_became_active":"2024-09-16T15:42:01.692530+0000","last_became_peered":"2024-09-16T15:42:01.692530+0000","last_unstale":"2024-09-16T15:42:01.697544+0000","last_undegraded":"2024-09-16T15:42:01.697544+0000","last_fullsized":"2024-09-16T15:42:01.697544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:08.444450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.698813+0000","last_change":"2024-09-16T15:42:00.689701+0000","last_active":"2024-09-16T15:42:01.698813+0000","last_peered":"2024-09-16T15:42:01.698813+0000","last_clean":"2024-09-16T15:42:01.698813+0000","last_became_active":"2024-09-16T15:42:00.689416+0000","last_became_peered":"2024-09-16T15:42:00.689416+0000","last_unstale":"2024-09-16T15:42:01.698813+0000","last_undegraded":"2024-09-16T15:42:01.698813+0000","last_fullsized":"2024-09-16T15:42:01.698813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:51.576762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.698228+0000","last_change":"2024-09-16T15:42:01.690293+0000","last_active":"2024-09-16T15:42:01.698228+0000","last_peered":"2024-09-16T15:42:01.698228+0000","last_clean":"2024-09-16T15:42:01.698228+0000","last_became_active":"2024-09-16T15:42:01.689998+0000","last_became_peered":"2024-09-16T15:42:01.689998+0000","last_unstale":"2024-09-16T15:42:01.698228+0000","last_undegraded":"2024-09-16T15:42:01.698228+0000","last_fullsized":"2024-09-16T15:42:01.698228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:48:00.823263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.707908+0000","last_change":"2024-09-16T15:42:00.688827+0000","last_active":"2024-09-16T15:42:01.707908+0000","last_peered":"2024-09-16T15:42:01.707908+0000","last_clean":"2024-09-16T15:42:01.707908+0000","last_became_active":"2024-09-16T15:42:00.688211+0000","last_became_peered":"2024-09-16T15:42:00.688211+0000","last_unstale":"2024-09-16T15:42:01.707908+0000","last_undegraded":"2024-09-16T15:42:01.707908+0000","last_fullsized":"2024-09-16T15:42:01.707908+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:08.951306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701419+0000","last_change":"2024-09-16T15:42:01.698337+0000","last_active":"2024-09-16T15:42:01.701419+0000","last_peered":"2024-09-16T15:42:01.701419+0000","last_clean":"2024-09-16T15:42:01.701419+0000","last_became_active":"2024-09-16T15:42:01.698034+0000","last_became_peered":"2024-09-16T15:42:01.698034+0000","last_unstale":"2024-09-16T15:42:01.701419+0000","last_undegraded":"2024-09-16T15:42:01.701419+0000","last_fullsized":"2024-09-16T15:42:01.701419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.929071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857803+0000","last_change":"2024-09-16T15:42:00.683923+0000","last_active":"2024-09-16T15:42:02.857803+0000","last_peered":"2024-09-16T15:42:02.857803+0000","last_clean":"2024-09-16T15:42:02.857803+0000","last_became_active":"2024-09-16T15:42:00.683535+0000","last_became_peered":"2024-09-16T15:42:00.683535+0000","last_unstale":"2024-09-16T15:42:02.857803+0000","last_undegraded":"2024-09-16T15:42:02.857803+0000","last_fullsized":"2024-09-16T15:42:02.857803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:39.204411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688551+0000","last_change":"2024-09-16T15:42:01.683286+0000","last_active":"2024-09-16T15:42:01.688551+0000","last_peered":"2024-09-16T15:42:01.688551+0000","last_clean":"2024-09-16T15:42:01.688551+0000","last_became_active":"2024-09-16T15:42:01.680838+0000","last_became_peered":"2024-09-16T15:42:01.680838+0000","last_unstale":"2024-09-16T15:42:01.688551+0000","last_undegraded":"2024-09-16T15:42:01.688551+0000","last_fullsized":"2024-09-16T15:42:01.688551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:29.517420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701671+0000","last_change":"2024-09-16T15:42:00.689508+0000","last_active":"2024-09-16T15:42:01.701671+0000","last_peered":"2024-09-16T15:42:01.701671+0000","last_clean":"2024-09-16T15:42:01.701671+0000","last_became_active":"2024-09-16T15:42:00.689241+0000","last_became_peered":"2024-09-16T15:42:00.689241+0000","last_unstale":"2024-09-16T15:42:01.701671+0000","last_undegraded":"2024-09-16T15:42:01.701671+0000","last_fullsized":"2024-09-16T15:42:01.701671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:08.916337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697836+0000","last_change":"2024-09-16T15:42:01.689979+0000","last_active":"2024-09-16T15:42:01.697836+0000","last_peered":"2024-09-16T15:42:01.697836+0000","last_clean":"2024-09-16T15:42:01.697836+0000","last_became_active":"2024-09-16T15:42:01.689512+0000","last_became_peered":"2024-09-16T15:42:01.689512+0000","last_unstale":"2024-09-16T15:42:01.697836+0000","last_undegraded":"2024-09-16T15:42:01.697836+0000","last_fullsized":"2024-09-16T15:42:01.697836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:14.491336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857953+0000","last_change":"2024-09-16T15:42:00.688739+0000","last_active":"2024-09-16T15:42:02.857953+0000","last_peered":"2024-09-16T15:42:02.857953+0000","last_clean":"2024-09-16T15:42:02.857953+0000","last_became_active":"2024-09-16T15:42:00.688462+0000","last_became_peered":"2024-09-16T15:42:00.688462+0000","last_unstale":"2024-09-16T15:42:02.857953+0000","last_undegraded":"2024-09-16T15:42:02.857953+0000","last_fullsized":"2024-09-16T15:42:02.857953+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:51.576762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.701498+0000","last_change":"2024-09-16T15:42:01.699023+0000","last_active":"2024-09-16T15:42:01.701498+0000","last_peered":"2024-09-16T15:42:01.701498+0000","last_clean":"2024-09-16T15:42:01.701498+0000","last_became_active":"2024-09-16T15:42:01.698505+0000","last_became_peered":"2024-09-16T15:42:01.698505+0000","last_unstale":"2024-09-16T15:42:01.701498+0000","last_undegraded":"2024-09-16T15:42:01.701498+0000","last_fullsized":"2024-09-16T15:42:01.701498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:14.496444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.703125+0000","last_change":"2024-09-16T15:42:00.684907+0000","last_active":"2024-09-16T15:42:01.703125+0000","last_peered":"2024-09-16T15:42:01.703125+0000","last_clean":"2024-09-16T15:42:01.703125+0000","last_became_active":"2024-09-16T15:42:00.684593+0000","last_became_peered":"2024-09-16T15:42:00.684593+0000","last_unstale":"2024-09-16T15:42:01.703125+0000","last_undegraded":"2024-09-16T15:42:01.703125+0000","last_fullsized":"2024-09-16T15:42:01.703125+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:15.889144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857925+0000","last_change":"2024-09-16T15:42:01.694493+0000","last_active":"2024-09-16T15:42:02.857925+0000","last_peered":"2024-09-16T15:42:02.857925+0000","last_clean":"2024-09-16T15:42:02.857925+0000","last_became_active":"2024-09-16T15:42:01.694267+0000","last_became_peered":"2024-09-16T15:42:01.694267+0000","last_unstale":"2024-09-16T15:42:02.857925+0000","last_undegraded":"2024-09-16T15:42:02.857925+0000","last_fullsized":"2024-09-16T15:42:02.857925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:05.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.677237+0000","last_change":"2024-09-16T15:42:00.682457+0000","last_active":"2024-09-16T15:42:01.677237+0000","last_peered":"2024-09-16T15:42:01.677237+0000","last_clean":"2024-09-16T15:42:01.677237+0000","last_became_active":"2024-09-16T15:42:00.682181+0000","last_became_peered":"2024-09-16T15:42:00.682181+0000","last_unstale":"2024-09-16T15:42:01.677237+0000","last_undegraded":"2024-09-16T15:42:01.677237+0000","last_fullsized":"2024-09-16T15:42:01.677237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:38.868310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857590+0000","last_change":"2024-09-16T15:42:01.691412+0000","last_active":"2024-09-16T15:42:02.857590+0000","last_peered":"2024-09-16T15:42:02.857590+0000","last_clean":"2024-09-16T15:42:02.857590+0000","last_became_active":"2024-09-16T15:42:01.691147+0000","last_became_peered":"2024-09-16T15:42:01.691147+0000","last_unstale":"2024-09-16T15:42:02.857590+0000","last_undegraded":"2024-09-16T15:42:02.857590+0000","last_fullsized":"2024-09-16T15:42:02.857590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:56.370244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.683181+0000","last_change":"2024-09-16T15:42:00.694921+0000","last_active":"2024-09-16T15:42:01.683181+0000","last_peered":"2024-09-16T15:42:01.683181+0000","last_clean":"2024-09-16T15:42:01.683181+0000","last_became_active":"2024-09-16T15:42:00.694618+0000","last_became_peered":"2024-09-16T15:42:00.694618+0000","last_unstale":"2024-09-16T15:42:01.683181+0000","last_undegraded":"2024-09-16T15:42:01.683181+0000","last_fullsized":"2024-09-16T15:42:01.683181+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:06.985410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697702+0000","last_change":"2024-09-16T15:42:01.693761+0000","last_active":"2024-09-16T15:42:01.697702+0000","last_peered":"2024-09-16T15:42:01.697702+0000","last_clean":"2024-09-16T15:42:01.697702+0000","last_became_active":"2024-09-16T15:42:01.693471+0000","last_became_peered":"2024-09-16T15:42:01.693471+0000","last_unstale":"2024-09-16T15:42:01.697702+0000","last_undegraded":"2024-09-16T15:42:01.697702+0000","last_fullsized":"2024-09-16T15:42:01.697702+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:39.983556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.705007+0000","last_change":"2024-09-16T15:42:00.693187+0000","last_active":"2024-09-16T15:42:01.705007+0000","last_peered":"2024-09-16T15:42:01.705007+0000","last_clean":"2024-09-16T15:42:01.705007+0000","last_became_active":"2024-09-16T15:42:00.692900+0000","last_became_peered":"2024-09-16T15:42:00.692900+0000","last_unstale":"2024-09-16T15:42:01.705007+0000","last_undegraded":"2024-09-16T15:42:01.705007+0000","last_fullsized":"2024-09-16T15:42:01.705007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:29.797553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.700059+0000","last_change":"2024-09-16T15:42:01.695785+0000","last_active":"2024-09-16T15:42:01.700059+0000","last_peered":"2024-09-16T15:42:01.700059+0000","last_clean":"2024-09-16T15:42:01.700059+0000","last_became_active":"2024-09-16T15:42:01.695489+0000","last_became_peered":"2024-09-16T15:42:01.695489+0000","last_unstale":"2024-09-16T15:42:01.700059+0000","last_undegraded":"2024-09-16T15:42:01.700059+0000","last_fullsized":"2024-09-16T15:42:01.700059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:07.733589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.857905+0000","last_change":"2024-09-16T15:42:00.679378+0000","last_active":"2024-09-16T15:42:02.857905+0000","last_peered":"2024-09-16T15:42:02.857905+0000","last_clean":"2024-09-16T15:42:02.857905+0000","last_became_active":"2024-09-16T15:42:00.678869+0000","last_became_peered":"2024-09-16T15:42:00.678869+0000","last_unstale":"2024-09-16T15:42:02.857905+0000","last_undegraded":"2024-09-16T15:42:02.857905+0000","last_fullsized":"2024-09-16T15:42:02.857905+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:42.237333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.682856+0000","last_change":"2024-09-16T15:42:01.678730+0000","last_active":"2024-09-16T15:42:01.682856+0000","last_peered":"2024-09-16T15:42:01.682856+0000","last_clean":"2024-09-16T15:42:01.682856+0000","last_became_active":"2024-09-16T15:42:01.677673+0000","last_became_peered":"2024-09-16T15:42:01.677673+0000","last_unstale":"2024-09-16T15:42:01.682856+0000","last_undegraded":"2024-09-16T15:42:01.682856+0000","last_fullsized":"2024-09-16T15:42:01.682856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:59:43.943868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686801+0000","last_change":"2024-09-16T15:42:00.684446+0000","last_active":"2024-09-16T15:42:02.686801+0000","last_peered":"2024-09-16T15:42:02.686801+0000","last_clean":"2024-09-16T15:42:02.686801+0000","last_became_active":"2024-09-16T15:42:00.684199+0000","last_became_peered":"2024-09-16T15:42:00.684199+0000","last_unstale":"2024-09-16T15:42:02.686801+0000","last_undegraded":"2024-09-16T15:42:02.686801+0000","last_fullsized":"2024-09-16T15:42:02.686801+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:32.522922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.858078+0000","last_change":"2024-09-16T15:42:01.701309+0000","last_active":"2024-09-16T15:42:02.858078+0000","last_peered":"2024-09-16T15:42:02.858078+0000","last_clean":"2024-09-16T15:42:02.858078+0000","last_became_active":"2024-09-16T15:42:01.700700+0000","last_became_peered":"2024-09-16T15:42:01.700700+0000","last_unstale":"2024-09-16T15:42:02.858078+0000","last_undegraded":"2024-09-16T15:42:02.858078+0000","last_fullsized":"2024-09-16T15:42:02.858078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:55.757473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.680488+0000","last_change":"2024-09-16T15:42:00.688239+0000","last_active":"2024-09-16T15:42:01.680488+0000","last_peered":"2024-09-16T15:42:01.680488+0000","last_clean":"2024-09-16T15:42:01.680488+0000","last_became_active":"2024-09-16T15:42:00.687145+0000","last_became_peered":"2024-09-16T15:42:00.687145+0000","last_unstale":"2024-09-16T15:42:01.680488+0000","last_undegraded":"2024-09-16T15:42:01.680488+0000","last_fullsized":"2024-09-16T15:42:01.680488+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:54.285204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.688392+0000","last_change":"2024-09-16T15:42:01.674464+0000","last_active":"2024-09-16T15:42:01.688392+0000","last_peered":"2024-09-16T15:42:01.688392+0000","last_clean":"2024-09-16T15:42:01.688392+0000","last_became_active":"2024-09-16T15:42:01.674192+0000","last_became_peered":"2024-09-16T15:42:01.674192+0000","last_unstale":"2024-09-16T15:42:01.688392+0000","last_undegraded":"2024-09-16T15:42:01.688392+0000","last_fullsized":"2024-09-16T15:42:01.688392+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:56.370244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.677517+0000","last_change":"2024-09-16T15:42:00.685535+0000","last_active":"2024-09-16T15:42:01.677517+0000","last_peered":"2024-09-16T15:42:01.677517+0000","last_clean":"2024-09-16T15:42:01.677517+0000","last_became_active":"2024-09-16T15:42:00.684505+0000","last_became_peered":"2024-09-16T15:42:00.684505+0000","last_unstale":"2024-09-16T15:42:01.677517+0000","last_undegraded":"2024-09-16T15:42:01.677517+0000","last_fullsized":"2024-09-16T15:42:01.677517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:48.491290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.697794+0000","last_change":"2024-09-16T15:42:01.692576+0000","last_active":"2024-09-16T15:42:01.697794+0000","last_peered":"2024-09-16T15:42:01.697794+0000","last_clean":"2024-09-16T15:42:01.697794+0000","last_became_active":"2024-09-16T15:42:01.691812+0000","last_became_peered":"2024-09-16T15:42:01.691812+0000","last_unstale":"2024-09-16T15:42:01.697794+0000","last_undegraded":"2024-09-16T15:42:01.697794+0000","last_fullsized":"2024-09-16T15:42:01.697794+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:52.607046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686195+0000","last_change":"2024-09-16T15:42:00.679352+0000","last_active":"2024-09-16T15:42:02.686195+0000","last_peered":"2024-09-16T15:42:02.686195+0000","last_clean":"2024-09-16T15:42:02.686195+0000","last_became_active":"2024-09-16T15:42:00.678234+0000","last_became_peered":"2024-09-16T15:42:00.678234+0000","last_unstale":"2024-09-16T15:42:02.686195+0000","last_undegraded":"2024-09-16T15:42:02.686195+0000","last_fullsized":"2024-09-16T15:42:02.686195+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:23.780987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.693574+0000","last_change":"2024-09-16T15:42:01.688820+0000","last_active":"2024-09-16T15:42:01.693574+0000","last_peered":"2024-09-16T15:42:01.693574+0000","last_clean":"2024-09-16T15:42:01.693574+0000","last_became_active":"2024-09-16T15:42:01.684663+0000","last_became_peered":"2024-09-16T15:42:01.684663+0000","last_unstale":"2024-09-16T15:42:01.693574+0000","last_undegraded":"2024-09-16T15:42:01.693574+0000","last_fullsized":"2024-09-16T15:42:01.693574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:32.409713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.676070+0000","last_change":"2024-09-16T15:42:00.681709+0000","last_active":"2024-09-16T15:42:01.676070+0000","last_peered":"2024-09-16T15:42:01.676070+0000","last_clean":"2024-09-16T15:42:01.676070+0000","last_became_active":"2024-09-16T15:42:00.681299+0000","last_became_peered":"2024-09-16T15:42:00.681299+0000","last_unstale":"2024-09-16T15:42:01.676070+0000","last_undegraded":"2024-09-16T15:42:01.676070+0000","last_fullsized":"2024-09-16T15:42:01.676070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:50.385571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.694969+0000","last_change":"2024-09-16T15:42:01.686534+0000","last_active":"2024-09-16T15:42:01.694969+0000","last_peered":"2024-09-16T15:42:01.694969+0000","last_clean":"2024-09-16T15:42:01.694969+0000","last_became_active":"2024-09-16T15:42:01.677771+0000","last_became_peered":"2024-09-16T15:42:01.677771+0000","last_unstale":"2024-09-16T15:42:01.694969+0000","last_undegraded":"2024-09-16T15:42:01.694969+0000","last_fullsized":"2024-09-16T15:42:01.694969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:51:57.849542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.668210+0000","last_change":"2024-09-16T15:42:00.694425+0000","last_active":"2024-09-16T15:42:01.668210+0000","last_peered":"2024-09-16T15:42:01.668210+0000","last_clean":"2024-09-16T15:42:01.668210+0000","last_became_active":"2024-09-16T15:42:00.691062+0000","last_became_peered":"2024-09-16T15:42:00.691062+0000","last_unstale":"2024-09-16T15:42:01.668210+0000","last_undegraded":"2024-09-16T15:42:01.668210+0000","last_fullsized":"2024-09-16T15:42:01.668210+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:58:23.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.695056+0000","last_change":"2024-09-16T15:42:01.690752+0000","last_active":"2024-09-16T15:42:01.695056+0000","last_peered":"2024-09-16T15:42:01.695056+0000","last_clean":"2024-09-16T15:42:01.695056+0000","last_became_active":"2024-09-16T15:42:01.690490+0000","last_became_peered":"2024-09-16T15:42:01.690490+0000","last_unstale":"2024-09-16T15:42:01.695056+0000","last_undegraded":"2024-09-16T15:42:01.695056+0000","last_fullsized":"2024-09-16T15:42:01.695056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:00.654774+0000","last_clean_scrub_stamp":"2024-09-16T15:42:00.654774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:45.933455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:42:02.686777+0000","last_change":"2024-09-16T15:42:00.680219+0000","last_active":"2024-09-16T15:42:02.686777+0000","last_peered":"2024-09-16T15:42:02.686777+0000","last_clean":"2024-09-16T15:42:02.686777+0000","last_became_active":"2024-09-16T15:42:00.679901+0000","last_became_peered":"2024-09-16T15:42:00.679901+0000","last_unstale":"2024-09-16T15:42:02.686777+0000","last_undegraded":"2024-09-16T15:42:02.686777+0000","last_fullsized":"2024-09-16T15:42:02.686777+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:41:59.648154+0000","last_clean_scrub_stamp":"2024-09-16T15:41:59.648154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:05.431535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":182,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:42:01.677524+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:42:01.677524+0000","last_peered":"2024-09-16T15:42:01.677524+0000","last_clean":"2024-09-16T15:42:01.677524+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:42:01.677524+0000","last_undegraded":"2024-09-16T15:42:01.677524+0000","last_fullsized":"2024-09-16T15:42:01.677524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28328,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343512,"statfs":{"total":96636764160,"available":96607756288,"internally_reserved":0,"allocated":1327104,"data_stored":890473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":724992,"data_stored":299945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":733184,"data_stored":300597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":737280,"data_stored":301557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":733184,"data_stored":300001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":745472,"data_stored":306301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509488,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":892507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":712704,"data_stored":293170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:06.355 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:42:06.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:42:06.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.558+0000 7f7083200640 1 -- 172.21.15.60:0/892055413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708406e660 msgr2=0x7f708406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.558+0000 7f7083200640 1 --2- 172.21.15.60:0/892055413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708406e660 0x7f708406eac0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7084159c50 tx=0x7f707c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:42:06.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.558+0000 7f7083200640 1 -- 172.21.15.60:0/892055413 shutdown_connections 2024-09-16T15:42:06.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.558+0000 7f7083200640 1 --2- 172.21.15.60:0/892055413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708406e660 0x7f708406eac0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.558+0000 7f7083200640 1 --2- 172.21.15.60:0/892055413 >> v2:172.21.15.100:3300/0 conn(0x7f70840c5730 0x7f70840c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.558+0000 7f7083200640 1 -- 172.21.15.60:0/892055413 >> 172.21.15.60:0/892055413 conn(0x7f70840cc490 msgr2=0x7f70840cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:06.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.558+0000 7f7083200640 1 -- 172.21.15.60:0/892055413 shutdown_connections 2024-09-16T15:42:06.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.558+0000 7f7083200640 1 -- 172.21.15.60:0/892055413 wait complete. 2024-09-16T15:42:06.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.560+0000 7f7083200640 1 Processor -- start 2024-09-16T15:42:06.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.560+0000 7f7083200640 1 -- start start 2024-09-16T15:42:06.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7083200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708406e660 0x7f70841505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7083200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840c5730 0x7f7084073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7083200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708415f670 0x7f7084073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7083200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f70840ca4f0 con 0x7f70840c5730 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7083200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70840ca370 con 0x7f708406e660 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7083200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f708406f000 con 0x7f708415f670 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7079400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840c5730 0x7f7084073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7079400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840c5730 0x7f7084073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42078/0 (socket says 172.21.15.60:42078) 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7079400640 1 -- 172.21.15.60:0/1049845632 learned_addr learned my addr 172.21.15.60:0/1049845632 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7081e00640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708406e660 0x7f70841505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7082800640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708415f670 0x7f7084073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7079400640 1 -- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708415f670 msgr2=0x7f7084073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7079400640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708415f670 0x7f7084073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7079400640 1 -- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708406e660 msgr2=0x7f70841505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7079400640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708406e660 0x7f70841505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.561+0000 7f7079400640 1 -- 172.21.15.60:0/1049845632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70841f67f0 con 0x7f70840c5730 2024-09-16T15:42:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.562+0000 7f7079400640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840c5730 0x7f7084073590 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c950 tx=0x7f706c00ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.562+0000 7f707aa00640 1 -- 172.21.15.60:0/1049845632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c007c70 con 0x7f70840c5730 2024-09-16T15:42:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.562+0000 7f707aa00640 1 -- 172.21.15.60:0/1049845632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f706c00f450 con 0x7f70840c5730 2024-09-16T15:42:06.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.562+0000 7f707aa00640 1 -- 172.21.15.60:0/1049845632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706c00f8a0 con 0x7f70840c5730 2024-09-16T15:42:06.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.562+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70841f6270 con 0x7f70840c5730 2024-09-16T15:42:06.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.562+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70841532b0 con 0x7f70840c5730 2024-09-16T15:42:06.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.563+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708406eac0 con 0x7f70840c5730 2024-09-16T15:42:06.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.565+0000 7f707aa00640 1 -- 172.21.15.60:0/1049845632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f706c00fa40 con 0x7f70840c5730 2024-09-16T15:42:06.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.566+0000 7f707aa00640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7060078a30 0x7f706007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.566+0000 7f707aa00640 1 -- 172.21.15.60:0/1049845632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f706c09b4a0 con 0x7f70840c5730 2024-09-16T15:42:06.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.566+0000 7f7081e00640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7060078a30 0x7f706007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.567+0000 7f7081e00640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7060078a30 0x7f706007aef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7074006fd0 tx=0x7f7074009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:06.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.570+0000 7f707aa00640 1 -- 172.21.15.60:0/1049845632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f706c014030 con 0x7f70840c5730 2024-09-16T15:42:06.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.837+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f708414e630 con 0x7f70840c5730 2024-09-16T15:42:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.840+0000 7f707aa00640 1 -- 172.21.15.60:0/1049845632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f706c063090 con 0x7f70840c5730 2024-09-16T15:42:06.841 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:42:06.841 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:42:06.841 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:42:06.841 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:42:06.842 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:42:06.842 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:42:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.842+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7060078a30 msgr2=0x7f706007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7060078a30 0x7f706007aef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7074006fd0 tx=0x7f7074009060 comp rx=0 tx=0).stop 2024-09-16T15:42:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840c5730 msgr2=0x7f7084073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840c5730 0x7f7084073590 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c950 tx=0x7f706c00ce10 comp rx=0 tx=0).stop 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 shutdown_connections 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7060078a30 0x7f706007aef0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708415f670 0x7f7084073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70840c5730 0x7f7084073590 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 --2- 172.21.15.60:0/1049845632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708406e660 0x7f70841505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 >> 172.21.15.60:0/1049845632 conn(0x7f70840cc490 msgr2=0x7f708415fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 shutdown_connections 2024-09-16T15:42:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:06.843+0000 7f7083200640 1 -- 172.21.15.60:0/1049845632 wait complete. 2024-09-16T15:42:06.860 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:07.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.021+0000 7ff268400640 1 -- 172.21.15.60:0/2441511642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600ccf00 msgr2=0x7ff2600acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.021+0000 7ff268400640 1 --2- 172.21.15.60:0/2441511642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600ccf00 0x7ff2600acd30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff260075130 tx=0x7ff24c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.021+0000 7ff268400640 1 -- 172.21.15.60:0/2441511642 shutdown_connections 2024-09-16T15:42:07.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.021+0000 7ff268400640 1 --2- 172.21.15.60:0/2441511642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2600ad270 0x7ff260073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.021+0000 7ff268400640 1 --2- 172.21.15.60:0/2441511642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2600ccf00 0x7ff2600acd30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.021+0000 7ff268400640 1 --2- 172.21.15.60:0/2441511642 >> v2:172.21.15.100:3300/0 conn(0x7ff26015f5a0 0x7ff2600b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.021+0000 7ff268400640 1 -- 172.21.15.60:0/2441511642 >> 172.21.15.60:0/2441511642 conn(0x7ff2600b1950 msgr2=0x7ff2600b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:07.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.022+0000 7ff268400640 1 -- 172.21.15.60:0/2441511642 shutdown_connections 2024-09-16T15:42:07.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.022+0000 7ff268400640 1 -- 172.21.15.60:0/2441511642 wait complete. 2024-09-16T15:42:07.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.023+0000 7ff268400640 1 Processor -- start 2024-09-16T15:42:07.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.023+0000 7ff268400640 1 -- start start 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff268400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2600ad270 0x7ff26015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff268400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2600ccf00 0x7ff2600c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff268400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff26015f5a0 0x7ff26014f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff268400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff260074880 con 0x7ff26015f5a0 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff268400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff260074700 con 0x7ff2600ccf00 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff268400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff26006f160 con 0x7ff2600ad270 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff266400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff26015f5a0 0x7ff26014f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff266400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff26015f5a0 0x7ff26014f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42104/0 (socket says 172.21.15.60:42104) 2024-09-16T15:42:07.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff266400640 1 -- 172.21.15.60:0/3403065960 learned_addr learned my addr 172.21.15.60:0/3403065960 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.024+0000 7ff265000640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2600ccf00 0x7ff2600c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff265a00640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2600ad270 0x7ff26015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff266400640 1 -- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2600ad270 msgr2=0x7ff26015d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff266400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2600ad270 0x7ff26015d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff266400640 1 -- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2600ccf00 msgr2=0x7ff2600c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff266400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2600ccf00 0x7ff2600c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff266400640 1 -- 172.21.15.60:0/3403065960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2601e6aa0 con 0x7ff26015f5a0 2024-09-16T15:42:07.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff266400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff26015f5a0 0x7ff26014f5c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff25800e980 tx=0x7ff25800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff25d600640 1 -- 172.21.15.60:0/3403065960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258004d20 con 0x7ff26015f5a0 2024-09-16T15:42:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff25d600640 1 -- 172.21.15.60:0/3403065960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff25800cb20 con 0x7ff26015f5a0 2024-09-16T15:42:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff25d600640 1 -- 172.21.15.60:0/3403065960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258005020 con 0x7ff26015f5a0 2024-09-16T15:42:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.025+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2601e6d80 con 0x7ff26015f5a0 2024-09-16T15:42:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.026+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2601e7340 con 0x7ff26015f5a0 2024-09-16T15:42:07.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.027+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff260073fb0 con 0x7ff26015f5a0 2024-09-16T15:42:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.028+0000 7ff25d600640 1 -- 172.21.15.60:0/3403065960 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7ff2580051c0 con 0x7ff26015f5a0 2024-09-16T15:42:07.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.029+0000 7ff25d600640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff240078a00 0x7ff24007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.029+0000 7ff25d600640 1 -- 172.21.15.60:0/3403065960 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7ff25809b4e0 con 0x7ff26015f5a0 2024-09-16T15:42:07.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.029+0000 7ff265a00640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff240078a00 0x7ff24007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.030+0000 7ff265a00640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff240078a00 0x7ff24007aec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff25400bb50 tx=0x7ff25400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:07.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.033+0000 7ff25d600640 1 -- 172.21.15.60:0/3403065960 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff258016030 con 0x7ff26015f5a0 2024-09-16T15:42:07.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.268+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff26015edc0 con 0x7ff26015f5a0 2024-09-16T15:42:07.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.269+0000 7ff25d600640 1 -- 172.21.15.60:0/3403065960 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5226 (secure 0 0 0) 0x7ff258063210 con 0x7ff26015f5a0 2024-09-16T15:42:07.270 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:07.270 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":36,"btime":"2024-09-16T15:42:04:690732+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5671,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3625257481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3625257481},{"type":"v1","addr":"172.21.15.60:6834","nonce":3625257481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2519251073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2519251073},{"type":"v1","addr":"172.21.15.100:6837","nonce":2519251073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/333738926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":333738926},{"type":"v1","addr":"172.21.15.60:6835","nonce":333738926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:01.154412+0000","modified":"2024-09-16T15:42:04.123997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2580251790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2580251790},{"type":"v1","addr":"172.21.15.100:6836","nonce":2580251790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5757,"qdb_cluster":[5757]},"id":4}]} 2024-09-16T15:42:07.270 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 36 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff240078a00 msgr2=0x7ff24007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff240078a00 0x7ff24007aec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff25400bb50 tx=0x7ff25400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff26015f5a0 msgr2=0x7ff26014f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff26015f5a0 0x7ff26014f5c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff25800e980 tx=0x7ff25800ee40 comp rx=0 tx=0).stop 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 shutdown_connections 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff240078a00 0x7ff24007aec0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff26015f5a0 0x7ff26014f5c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2600ccf00 0x7ff2600c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 --2- 172.21.15.60:0/3403065960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2600ad270 0x7ff26015d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 >> 172.21.15.60:0/3403065960 conn(0x7ff2600b1950 msgr2=0x7ff2600aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 shutdown_connections 2024-09-16T15:42:07.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.271+0000 7ff268400640 1 -- 172.21.15.60:0/3403065960 wait complete. 2024-09-16T15:42:07.638 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-09-16T15:42:07.637+0000 7fed78c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:42:07.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:42:01.154412+0000', 'modified': '2024-09-16T15:42:04.123997+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5757}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5757': {'gid': 5757, 'name': 'b', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6836/2580251790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6835', 'nonce': 2580251790}, {'type': 'v1', 'addr': '172.21.15.100:6836', 'nonce': 2580251790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5757, 'qdb_cluster': [5757]} 2024-09-16T15:42:07.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:42:07.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-16T15:42:07.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.854+0000 7fa333400640 1 -- 172.21.15.60:0/2404501141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c0ccf00 msgr2=0x7fa32c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.854+0000 7fa333400640 1 --2- 172.21.15.60:0/2404501141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c0ccf00 0x7fa32c0acd30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa32c075130 tx=0x7fa31802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.854+0000 7fa333400640 1 -- 172.21.15.60:0/2404501141 shutdown_connections 2024-09-16T15:42:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.854+0000 7fa333400640 1 --2- 172.21.15.60:0/2404501141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c0ccf00 0x7fa32c0acd30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.854+0000 7fa333400640 1 --2- 172.21.15.60:0/2404501141 >> v2:172.21.15.100:3300/0 conn(0x7fa32c15f5a0 0x7fa32c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.854+0000 7fa333400640 1 -- 172.21.15.60:0/2404501141 >> 172.21.15.60:0/2404501141 conn(0x7fa32c0b1950 msgr2=0x7fa32c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.854+0000 7fa333400640 1 -- 172.21.15.60:0/2404501141 shutdown_connections 2024-09-16T15:42:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.854+0000 7fa333400640 1 -- 172.21.15.60:0/2404501141 wait complete. 2024-09-16T15:42:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.856+0000 7fa333400640 1 Processor -- start 2024-09-16T15:42:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.856+0000 7fa333400640 1 -- start start 2024-09-16T15:42:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa333400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa32c0ad270 0x7fa32c14c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa333400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c0ccf00 0x7fa32c0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa333400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c15f5a0 0x7fa32c150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa333400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c06f160 con 0x7fa32c15f5a0 2024-09-16T15:42:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa333400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c06f2e0 con 0x7fa32c0ccf00 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa333400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa32c06f460 con 0x7fa32c0ad270 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa331400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c15f5a0 0x7fa32c150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa331400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c15f5a0 0x7fa32c150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42110/0 (socket says 172.21.15.60:42110) 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa331400640 1 -- 172.21.15.60:0/732740178 learned_addr learned my addr 172.21.15.60:0/732740178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa32be00640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c0ccf00 0x7fa32c0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.857+0000 7fa330a00640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa32c0ad270 0x7fa32c14c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa331400640 1 -- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa32c0ad270 msgr2=0x7fa32c14c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa331400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa32c0ad270 0x7fa32c14c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa331400640 1 -- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c0ccf00 msgr2=0x7fa32c0b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa331400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c0ccf00 0x7fa32c0b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa331400640 1 -- 172.21.15.60:0/732740178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32c0b3df0 con 0x7fa32c15f5a0 2024-09-16T15:42:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa331400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c15f5a0 0x7fa32c150f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c780 tx=0x7fa32400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:07.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa328c00640 1 -- 172.21.15.60:0/732740178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324004b60 con 0x7fa32c15f5a0 2024-09-16T15:42:07.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa328c00640 1 -- 172.21.15.60:0/732740178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa324004df0 con 0x7fa32c15f5a0 2024-09-16T15:42:07.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa328c00640 1 -- 172.21.15.60:0/732740178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32400aa70 con 0x7fa32c15f5a0 2024-09-16T15:42:07.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa333400640 1 -- 172.21.15.60:0/732740178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32c0b40d0 con 0x7fa32c15f5a0 2024-09-16T15:42:07.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.858+0000 7fa333400640 1 -- 172.21.15.60:0/732740178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32c14e5e0 con 0x7fa32c15f5a0 2024-09-16T15:42:07.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.860+0000 7fa328c00640 1 -- 172.21.15.60:0/732740178 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fa32400ac10 con 0x7fa32c15f5a0 2024-09-16T15:42:07.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.861+0000 7fa328c00640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa30c078a00 0x7fa30c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.861+0000 7fa328c00640 1 -- 172.21.15.60:0/732740178 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa32409aed0 con 0x7fa32c15f5a0 2024-09-16T15:42:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.862+0000 7fa3361f9740 1 Processor -- start 2024-09-16T15:42:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.862+0000 7fa330a00640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa30c078a00 0x7fa30c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.862+0000 7fa3361f9740 1 -- start start 2024-09-16T15:42:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa330a00640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa30c078a00 0x7fa30c07aec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa320005fd0 tx=0x7fa32000a250 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88ef068b0 0x55f88ef06c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88eec1a20 0x55f88ef05390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88ef058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa3361f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f88ee44430 con 0x55f88eec1a20 2024-09-16T15:42:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa3361f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f88ee345d0 con 0x55f88ef068b0 2024-09-16T15:42:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa3361f9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55f88ee23a00 con 0x55f88eec1fa0 2024-09-16T15:42:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa32be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88eec1a20 0x55f88ef05390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa32be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88eec1a20 0x55f88ef05390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42112/0 (socket says 172.21.15.60:42112) 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa32be00640 1 -- 172.21.15.60:0/4090316435 learned_addr learned my addr 172.21.15.60:0/4090316435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa331400640 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88ef058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.863+0000 7fa330a00640 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88ef068b0 0x55f88ef06c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa32be00640 1 -- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 msgr2=0x55f88ef058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa32be00640 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88ef058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa32be00640 1 -- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88ef068b0 msgr2=0x55f88ef06c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa32be00640 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88ef068b0 0x55f88ef06c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa32be00640 1 -- 172.21.15.60:0/4090316435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88edec520 con 0x55f88eec1a20 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa331400640 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88ef058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa32be00640 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88eec1a20 0x55f88ef05390 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa318009230 tx=0x7fa3180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3dee7450ee51d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa31d200640 1 -- 172.21.15.60:0/4090316435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318040070 con 0x55f88eec1a20 2024-09-16T15:42:07.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa31d200640 1 -- 172.21.15.60:0/4090316435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa318005310 con 0x55f88eec1a20 2024-09-16T15:42:07.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.864+0000 7fa31d200640 1 -- 172.21.15.60:0/4090316435 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318035040 con 0x55f88eec1a20 2024-09-16T15:42:07.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 -- 172.21.15.60:0/4090316435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88eec1a20 msgr2=0x55f88ef05390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88eec1a20 0x55f88ef05390 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa318009230 tx=0x7fa3180046c0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 -- 172.21.15.60:0/4090316435 shutdown_connections 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88ef058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88eec1a20 0x55f88ef05390 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 --2- 172.21.15.60:0/4090316435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88ef068b0 0x55f88ef06c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 -- 172.21.15.60:0/4090316435 >> 172.21.15.60:0/4090316435 conn(0x55f88eebcff0 msgr2=0x55f88eec0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 -- 172.21.15.60:0/4090316435 shutdown_connections 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.865+0000 7fa3361f9740 1 -- 172.21.15.60:0/4090316435 wait complete. 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.866+0000 7fa3361f9740 1 -- start start 2024-09-16T15:42:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.866+0000 7fa3361f9740 1 Processor -- start 2024-09-16T15:42:07.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eeea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eed8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88eed5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa3361f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f88edca330 con 0x55f88ef068b0 2024-09-16T15:42:07.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa3361f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f88ee27810 con 0x55f88eeea930 2024-09-16T15:42:07.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa3361f9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55f88eb94620 con 0x55f88eec1fa0 2024-09-16T15:42:07.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa3361f9740 10 client.0 _ll_register_callbacks cb 0x55f88edcf420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa32be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eed8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa32be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eed8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42122/0 (socket says 172.21.15.60:42122) 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.867+0000 7fa32be00640 1 -- 172.21.15.60:0/1413322408 learned_addr learned my addr 172.21.15.60:0/1413322408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa331400640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88eed5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa330a00640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eeea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa32be00640 1 -- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 msgr2=0x55f88eeea3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa32be00640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eeea3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa32be00640 1 -- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 msgr2=0x55f88eed5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa32be00640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88eed5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa32be00640 1 -- 172.21.15.60:0/1413322408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88eb85850 con 0x55f88ef068b0 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa331400640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88eed5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa330a00640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eeea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:07.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa32be00640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eed8ad0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa31802aac0 tx=0x7fa318005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb39c8a9b065cc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa311600640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa311600640 1 -- 172.21.15.60:0/1413322408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31804c060 con 0x55f88ef068b0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa3361f9740 1 -- 172.21.15.60:0/1413322408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa318005310 con 0x55f88ef068b0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa311600640 1 -- 172.21.15.60:0/1413322408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa318004bb0 con 0x55f88ef068b0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa311600640 1 -- 172.21.15.60:0/1413322408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa318052050 con 0x55f88ef068b0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa3361f9740 10 client.5874 fetch_fsmap learned FSMap version 36 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.868+0000 7fa3361f9740 1 -- 172.21.15.60:0/1413322408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55f88ebfc6b0 con 0x55f88ef068b0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa311600640 1 -- 172.21.15.60:0/1413322408 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa318005310 con 0x55f88ef068b0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa311600640 1 -- 172.21.15.60:0/1413322408 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7fa318004db0 con 0x55f88ef068b0 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa3361f9740 10 client.5874 fetch_fsmap finished waiting for FSMap version 36 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa3361f9740 10 client.5874 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa3361f9740 20 client.5874 populate_metadata read hostname 'smithi060' 2024-09-16T15:42:07.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] conn(0x55f88ef05620 0x55f88ef05a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa3361f9740 4 client.5874 mds_command: new command op to 5757 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:42:07.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa3361f9740 1 -- 172.21.15.60:0/1413322408 --> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88ee36520 con 0x55f88ef05620 2024-09-16T15:42:07.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.869+0000 7fa330a00640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] conn(0x55f88ef05620 0x55f88ef05a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.870+0000 7fa330a00640 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] conn(0x55f88ef05620 0x55f88ef05a00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff8174db1a408fe server_cookie=933dae51a00a7aa2 in_seq=0 out_seq=0 2024-09-16T15:42:07.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.870+0000 7fa311600640 10 client.5874 ms_handle_connect on v2:172.21.15.100:6835/2580251790 2024-09-16T15:42:07.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa311600640 1 -- 172.21.15.60:0/1413322408 <== mds.0 v2:172.21.15.100:6835/2580251790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f88ee36520 con 0x55f88ef05620 2024-09-16T15:42:07.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa311600640 10 client.5874 handle_command_reply: tid=0 2024-09-16T15:42:07.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 client.5874 shutdown 2024-09-16T15:42:07.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 -- 172.21.15.60:0/1413322408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 msgr2=0x55f88eed8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eed8ad0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa31802aac0 tx=0x7fa318005900 comp rx=0 tx=0).stop 2024-09-16T15:42:07.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 -- 172.21.15.60:0/1413322408 shutdown_connections 2024-09-16T15:42:07.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] conn(0x55f88ef05620 0x55f88ef05a00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88eed5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eed8ad0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1413322408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eeea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa311600640 0 client.5874 ms_handle_reset on v2:172.21.15.100:6835/2580251790 2024-09-16T15:42:07.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 -- 172.21.15.60:0/1413322408 >> 172.21.15.60:0/1413322408 conn(0x55f88eec1a20 msgr2=0x55f88ef08cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:07.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.871+0000 7fa3361f9740 1 -- 172.21.15.60:0/1413322408 shutdown_connections 2024-09-16T15:42:07.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.872+0000 7fa3361f9740 1 -- 172.21.15.60:0/1413322408 wait complete. 2024-09-16T15:42:07.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.872+0000 7fa3361f9740 20 client.5874 trim_cache size 0 max 16384 2024-09-16T15:42:07.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.890+0000 7fa3361f9740 1 Processor -- start 2024-09-16T15:42:07.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.891+0000 7fa3361f9740 1 -- start start 2024-09-16T15:42:07.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.891+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eec1fa0 0x55f88eee72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.891+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eeea930 0x55f88eee9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.891+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eeea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa3361f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c200200 con 0x55f88ef068b0 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa3361f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa32c1466e0 con 0x55f88eec1fa0 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa3361f9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa32c153920 con 0x55f88eeea930 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eeea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eeea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42126/0 (socket says 172.21.15.60:42126) 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 -- 172.21.15.60:0/1994299941 learned_addr learned my addr 172.21.15.60:0/1994299941 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa32be00640 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eeea930 0x55f88eee9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa330a00640 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eec1fa0 0x55f88eee72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 -- 172.21.15.60:0/1994299941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eeea930 msgr2=0x55f88eee9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:07.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eeea930 0x55f88eee9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 -- 172.21.15.60:0/1994299941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eec1fa0 msgr2=0x55f88eee72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eec1fa0 0x55f88eee72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 -- 172.21.15.60:0/1994299941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88eec0ec0 con 0x55f88ef068b0 2024-09-16T15:42:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.892+0000 7fa331400640 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eeea290 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa324099010 tx=0x7fa3240623d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec940a48a411d743 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa313e00640 1 -- 172.21.15.60:0/1994299941 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3240994c0 con 0x55f88ef068b0 2024-09-16T15:42:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa313e00640 1 -- 172.21.15.60:0/1994299941 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa324067580 con 0x55f88ef068b0 2024-09-16T15:42:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa313e00640 1 -- 172.21.15.60:0/1994299941 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32406c430 con 0x55f88ef068b0 2024-09-16T15:42:07.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 -- 172.21.15.60:0/1994299941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 msgr2=0x55f88eeea290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eeea290 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa324099010 tx=0x7fa3240623d0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 -- 172.21.15.60:0/1994299941 shutdown_connections 2024-09-16T15:42:07.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88eeea290 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eeea930 0x55f88eee9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 --2- 172.21.15.60:0/1994299941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eec1fa0 0x55f88eee72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 -- 172.21.15.60:0/1994299941 >> 172.21.15.60:0/1994299941 conn(0x55f88eec1a20 msgr2=0x55f88eed5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 -- 172.21.15.60:0/1994299941 shutdown_connections 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.893+0000 7fa3361f9740 1 -- 172.21.15.60:0/1994299941 wait complete. 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.894+0000 7fa3361f9740 1 -- start start 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.894+0000 7fa3361f9740 1 Processor -- start 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eefbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88ef7cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa3361f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88ef898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa3361f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55f88eed66c0 con 0x55f88ef068b0 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa3361f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f88eed62c0 con 0x55f88eeea930 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa3361f9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55f88eed6d50 con 0x55f88eec1fa0 2024-09-16T15:42:07.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa3361f9740 10 client.0 _ll_register_callbacks cb 0x7fa318004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa331400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88ef898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa331400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88ef898e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42140/0 (socket says 172.21.15.60:42140) 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa331400640 1 -- 172.21.15.60:0/2493619816 learned_addr learned my addr 172.21.15.60:0/2493619816 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa32be00640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88ef7cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.895+0000 7fa330a00640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eefbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa331400640 1 -- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 msgr2=0x55f88eefbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa331400640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eefbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa331400640 1 -- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 msgr2=0x55f88ef7cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa331400640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88ef7cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa331400640 1 -- 172.21.15.60:0/2493619816 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88eee6710 con 0x55f88ef068b0 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa32be00640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88ef7cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa330a00640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eefbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa331400640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88ef898e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa324062850 tx=0x7fa324099570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0f867aa2503e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:07.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa311600640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa311600640 1 -- 172.21.15.60:0/2493619816 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324081060 con 0x55f88ef068b0 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa311600640 1 -- 172.21.15.60:0/2493619816 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa32406c1a0 con 0x55f88ef068b0 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa3361f9740 1 -- 172.21.15.60:0/2493619816 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa324067580 con 0x55f88ef068b0 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa311600640 1 -- 172.21.15.60:0/2493619816 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa324067340 con 0x55f88ef068b0 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa3361f9740 10 client.5880 fetch_fsmap learned FSMap version 36 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.896+0000 7fa3361f9740 1 -- 172.21.15.60:0/2493619816 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55f88ef8a470 con 0x55f88ef068b0 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa311600640 1 -- 172.21.15.60:0/2493619816 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa324067580 con 0x55f88ef068b0 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa311600640 1 -- 172.21.15.60:0/2493619816 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7fa324006b50 con 0x55f88ef068b0 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa3361f9740 10 client.5880 fetch_fsmap finished waiting for FSMap version 36 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa3361f9740 10 client.5880 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa3361f9740 20 client.5880 populate_metadata read hostname 'smithi060' 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa3361f9740 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] conn(0x55f88eee9f60 0x55f88eeea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa3361f9740 4 client.5880 mds_command: new command op to 5757 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:42:07.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa3361f9740 1 -- 172.21.15.60:0/2493619816 --> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] -- command(tid 0: {"prefix": "status"}) -- 0x55f88ee31fd0 con 0x55f88eee9f60 2024-09-16T15:42:07.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.897+0000 7fa330a00640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] conn(0x55f88eee9f60 0x55f88eeea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa330a00640 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] conn(0x55f88eee9f60 0x55f88eeea340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b3aa8576356941 server_cookie=5da83f792b1eec4a in_seq=0 out_seq=0 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa311600640 10 client.5880 ms_handle_connect on v2:172.21.15.100:6835/2580251790 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa311600640 1 -- 172.21.15.60:0/2493619816 <== mds.0 v2:172.21.15.100:6835/2580251790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55f88ee31fd0 con 0x55f88eee9f60 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa311600640 10 client.5880 handle_command_reply: tid=0 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 client.5880 shutdown 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 -- 172.21.15.60:0/2493619816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 msgr2=0x55f88ef898e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88ef898e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa324062850 tx=0x7fa324099570 comp rx=0 tx=0).stop 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 -- 172.21.15.60:0/2493619816 shutdown_connections 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:6835/2580251790,v1:172.21.15.100:6836/2580251790] conn(0x55f88eee9f60 0x55f88eeea340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f88ef068b0 0x55f88ef898e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f88eeea930 0x55f88ef7cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 --2- 172.21.15.60:0/2493619816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f88eec1fa0 0x55f88eefbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa3361f9740 1 -- 172.21.15.60:0/2493619816 >> 172.21.15.60:0/2493619816 conn(0x55f88eec1a20 msgr2=0x55f88eefe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.898+0000 7fa311600640 0 client.5880 ms_handle_reset on v2:172.21.15.100:6835/2580251790 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa3361f9740 1 -- 172.21.15.60:0/2493619816 shutdown_connections 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa3361f9740 1 -- 172.21.15.60:0/2493619816 wait complete. 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa3361f9740 20 client.5880 trim_cache size 0 max 16384 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:42:07.904 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5757, 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2345542949999997, 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 36, 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 62, 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 62, 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.536334532 2024-09-16T15:42:07.905 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 -- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa30c078a00 msgr2=0x7fa30c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa30c078a00 0x7fa30c07aec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa320005fd0 tx=0x7fa32000a250 comp rx=0 tx=0).stop 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 -- 172.21.15.60:0/732740178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c15f5a0 msgr2=0x7fa32c150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c15f5a0 0x7fa32c150f50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c780 tx=0x7fa32400cc40 comp rx=0 tx=0).stop 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 -- 172.21.15.60:0/732740178 shutdown_connections 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa30c078a00 0x7fa30c07aec0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa32c15f5a0 0x7fa32c150f50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa32c0ccf00 0x7fa32c0b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 --2- 172.21.15.60:0/732740178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa32c0ad270 0x7fa32c14c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:07.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.899+0000 7fa333400640 1 -- 172.21.15.60:0/732740178 >> 172.21.15.60:0/732740178 conn(0x7fa32c0b1950 msgr2=0x7fa32c09aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:07.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.900+0000 7fa333400640 1 -- 172.21.15.60:0/732740178 shutdown_connections 2024-09-16T15:42:07.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:07.900+0000 7fa333400640 1 -- 172.21.15.60:0/732740178 wait complete. 2024-09-16T15:42:07.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:42:08.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.075+0000 7f8e77400640 1 -- 172.21.15.60:0/3035257993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7015f5a0 msgr2=0x7f8e700b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:08.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.075+0000 7f8e77400640 1 --2- 172.21.15.60:0/3035257993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7015f5a0 0x7f8e700b2440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009830 tx=0x7f8e6402a480 comp rx=0 tx=0).stop 2024-09-16T15:42:08.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.075+0000 7f8e77400640 1 -- 172.21.15.60:0/3035257993 shutdown_connections 2024-09-16T15:42:08.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.075+0000 7f8e77400640 1 --2- 172.21.15.60:0/3035257993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e70073fb0 0x7f8e70074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.075+0000 7f8e77400640 1 --2- 172.21.15.60:0/3035257993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7015f5a0 0x7f8e700b2440 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.075+0000 7f8e77400640 1 -- 172.21.15.60:0/3035257993 >> 172.21.15.60:0/3035257993 conn(0x7f8e700b1950 msgr2=0x7f8e700b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:08.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.076+0000 7f8e77400640 1 -- 172.21.15.60:0/3035257993 shutdown_connections 2024-09-16T15:42:08.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.076+0000 7f8e77400640 1 -- 172.21.15.60:0/3035257993 wait complete. 2024-09-16T15:42:08.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.077+0000 7f8e77400640 1 Processor -- start 2024-09-16T15:42:08.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.077+0000 7f8e77400640 1 -- start start 2024-09-16T15:42:08.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.078+0000 7f8e77400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e70073fb0 0x7f8e701f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:08.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.078+0000 7f8e77400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e700ccf00 0x7f8e70154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:08.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.078+0000 7f8e77400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7015f5a0 0x7f8e700ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:08.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.078+0000 7f8e77400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7006f2e0 con 0x7f8e700ccf00 2024-09-16T15:42:08.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.078+0000 7f8e77400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7006f160 con 0x7f8e70073fb0 2024-09-16T15:42:08.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.078+0000 7f8e77400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7006f460 con 0x7f8e7015f5a0 2024-09-16T15:42:08.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.078+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e700ccf00 0x7f8e70154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:08.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e700ccf00 0x7f8e70154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42162/0 (socket says 172.21.15.60:42162) 2024-09-16T15:42:08.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e6fe00640 1 -- 172.21.15.60:0/1553475977 learned_addr learned my addr 172.21.15.60:0/1553475977 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:08.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e75400640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7015f5a0 0x7f8e700ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:08.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e74a00640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e70073fb0 0x7f8e701f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:08.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e6fe00640 1 -- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7015f5a0 msgr2=0x7f8e700ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:08.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e6fe00640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7015f5a0 0x7f8e700ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e6fe00640 1 -- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e70073fb0 msgr2=0x7f8e701f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:08.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e6fe00640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e70073fb0 0x7f8e701f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e6fe00640 1 -- 172.21.15.60:0/1553475977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e701f8e50 con 0x7f8e700ccf00 2024-09-16T15:42:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.079+0000 7f8e6fe00640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e700ccf00 0x7f8e70154840 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00c450 tx=0x7f8e5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.080+0000 7f8e6cc00640 1 -- 172.21.15.60:0/1553475977 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c016020 con 0x7f8e700ccf00 2024-09-16T15:42:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.080+0000 7f8e6cc00640 1 -- 172.21.15.60:0/1553475977 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e5c012920 con 0x7f8e700ccf00 2024-09-16T15:42:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.080+0000 7f8e6cc00640 1 -- 172.21.15.60:0/1553475977 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c012d70 con 0x7f8e700ccf00 2024-09-16T15:42:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.080+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e701f4a20 con 0x7f8e700ccf00 2024-09-16T15:42:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.080+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e701f4fe0 con 0x7f8e700ccf00 2024-09-16T15:42:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.081+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7006f990 con 0x7f8e700ccf00 2024-09-16T15:42:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.082+0000 7f8e6cc00640 1 -- 172.21.15.60:0/1553475977 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f8e5c019020 con 0x7f8e700ccf00 2024-09-16T15:42:08.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.083+0000 7f8e6cc00640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e50078a00 0x7f8e5007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:08.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.084+0000 7f8e6cc00640 1 -- 172.21.15.60:0/1553475977 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f8e5c09a910 con 0x7f8e700ccf00 2024-09-16T15:42:08.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.084+0000 7f8e74a00640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e50078a00 0x7f8e5007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:08.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.085+0000 7f8e74a00640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e50078a00 0x7f8e5007aec0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e64034000 tx=0x7f8e64004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:08.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.088+0000 7f8e6cc00640 1 -- 172.21.15.60:0/1553475977 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8e5c020030 con 0x7f8e700ccf00 2024-09-16T15:42:08.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.326+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8e700b48b0 con 0x7f8e700ccf00 2024-09-16T15:42:08.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.327+0000 7f8e6cc00640 1 -- 172.21.15.60:0/1553475977 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f8e701f8e50 con 0x7f8e700ccf00 2024-09-16T15:42:08.328 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:42:08.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.329+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e50078a00 msgr2=0x7f8e5007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:08.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.329+0000 7f8e77400640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e50078a00 0x7f8e5007aec0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e64034000 tx=0x7f8e64004730 comp rx=0 tx=0).stop 2024-09-16T15:42:08.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e700ccf00 msgr2=0x7f8e70154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:08.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e700ccf00 0x7f8e70154840 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00c450 tx=0x7f8e5c00c910 comp rx=0 tx=0).stop 2024-09-16T15:42:08.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 shutdown_connections 2024-09-16T15:42:08.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e50078a00 0x7f8e5007aec0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7015f5a0 0x7f8e700ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e700ccf00 0x7f8e70154840 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 --2- 172.21.15.60:0/1553475977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e70073fb0 0x7f8e701f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 >> 172.21.15.60:0/1553475977 conn(0x7f8e700b1950 msgr2=0x7f8e700b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 shutdown_connections 2024-09-16T15:42:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08.330+0000 7f8e77400640 1 -- 172.21.15.60:0/1553475977 wait complete. 2024-09-16T15:42:08.346 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:42:08.346 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:42:08.346 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:42:08.346 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:42:08.346 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:08.346 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:42:08.347 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:42:08.347 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:42:08.347 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:08.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:08.347 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:42:08.382 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:42:08.383 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42521sec preferred_lft 42521sec 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:42:08.384 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:08.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:08.385 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:42:08.477 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:42:08.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:08.480 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:42:08.516 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:42:08.516 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:42:08.516 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:42:08.516 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:42:08.516 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:42:08.516 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:42:08.516 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:42:08.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:42:08.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:42:08.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:42:08.517 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:42:08.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:42:08.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:08.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:08.716 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:42:08.751 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:42:08.751 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:42:08.751 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:08.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:08.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:42:08.848 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:08.849 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:08.883 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:08.883 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:08.883 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:08.961 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:42:08.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:08.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:09.089 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:09.089 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:42:09.118 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:09.118 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:42:09.118 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:42:09.118 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:42:09.118 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:42:09.119 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:42:09.120 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:42:09.120 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:42:09.120 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:42:09.120 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:42:09.123 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:42:09.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:09.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:09.179 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:42:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:42:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:09.279 INFO:teuthology.orchestra.run.smithi060.stdout:client5408 2024-09-16T15:42:09.279 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:42:09.280 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:09.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:42:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.514+0000 7f60bcc00640 1 -- 172.21.15.60:0/3962725184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b815f5a0 msgr2=0x7f60b80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.514+0000 7f60bcc00640 1 --2- 172.21.15.60:0/3962725184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b815f5a0 0x7f60b80b2440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009830 tx=0x7f60a802a480 comp rx=0 tx=0).stop 2024-09-16T15:42:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.515+0000 7f60bcc00640 1 -- 172.21.15.60:0/3962725184 shutdown_connections 2024-09-16T15:42:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.515+0000 7f60bcc00640 1 --2- 172.21.15.60:0/3962725184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60b8073fb0 0x7f60b8074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.515+0000 7f60bcc00640 1 --2- 172.21.15.60:0/3962725184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b815f5a0 0x7f60b80b2440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.515+0000 7f60bcc00640 1 -- 172.21.15.60:0/3962725184 >> 172.21.15.60:0/3962725184 conn(0x7f60b80b1950 msgr2=0x7f60b80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.515+0000 7f60bcc00640 1 -- 172.21.15.60:0/3962725184 shutdown_connections 2024-09-16T15:42:09.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.515+0000 7f60bcc00640 1 -- 172.21.15.60:0/3962725184 wait complete. 2024-09-16T15:42:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.516+0000 7f60bcc00640 1 Processor -- start 2024-09-16T15:42:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.516+0000 7f60bcc00640 1 -- start start 2024-09-16T15:42:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60bcc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60b8073fb0 0x7f60b80c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60bcc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b80ccf00 0x7f60b81e9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60bcc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60b815f5a0 0x7f60b809d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60bcc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f60b806f2e0 con 0x7f60b8073fb0 2024-09-16T15:42:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60bcc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60b806f160 con 0x7f60b80ccf00 2024-09-16T15:42:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60bcc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f60b806f460 con 0x7f60b815f5a0 2024-09-16T15:42:09.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60b5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60b8073fb0 0x7f60b80c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60b5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60b8073fb0 0x7f60b80c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56400/0 (socket says 172.21.15.60:56400) 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.517+0000 7f60b5e00640 1 -- 172.21.15.60:0/295182529 learned_addr learned my addr 172.21.15.60:0/295182529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60ad400640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b80ccf00 0x7f60b81e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60b6800640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60b815f5a0 0x7f60b809d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60b5e00640 1 -- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60b815f5a0 msgr2=0x7f60b809d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60b5e00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60b815f5a0 0x7f60b809d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60b5e00640 1 -- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b80ccf00 msgr2=0x7f60b81e9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60b5e00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b80ccf00 0x7f60b81e9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60b5e00640 1 -- 172.21.15.60:0/295182529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b80aff90 con 0x7f60b8073fb0 2024-09-16T15:42:09.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60b5e00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60b8073fb0 0x7f60b80c8fd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009950 tx=0x7f60a8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60aea00640 1 -- 172.21.15.60:0/295182529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8005c90 con 0x7f60b8073fb0 2024-09-16T15:42:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60aea00640 1 -- 172.21.15.60:0/295182529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60a8031e50 con 0x7f60b8073fb0 2024-09-16T15:42:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60aea00640 1 -- 172.21.15.60:0/295182529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8005260 con 0x7f60b8073fb0 2024-09-16T15:42:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.518+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b81e8ce0 con 0x7f60b8073fb0 2024-09-16T15:42:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.519+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b81e92a0 con 0x7f60b8073fb0 2024-09-16T15:42:09.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.520+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b80ad200 con 0x7f60b8073fb0 2024-09-16T15:42:09.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.521+0000 7f60aea00640 1 -- 172.21.15.60:0/295182529 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f60a8005400 con 0x7f60b8073fb0 2024-09-16T15:42:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.522+0000 7f60aea00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6094078a00 0x7f609407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.522+0000 7f60aea00640 1 -- 172.21.15.60:0/295182529 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f60a80bbc80 con 0x7f60b8073fb0 2024-09-16T15:42:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.523+0000 7f60ad400640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6094078a00 0x7f609407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.523+0000 7f60ad400640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6094078a00 0x7f609407aec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f60b809da90 tx=0x7f60a0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:09.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.527+0000 7f60aea00640 1 -- 172.21.15.60:0/295182529 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f60a803f030 con 0x7f60b8073fb0 2024-09-16T15:42:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.767+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f60b806f980 con 0x7f60b8073fb0 2024-09-16T15:42:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.768+0000 7f60aea00640 1 -- 172.21.15.60:0/295182529 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f60a8083f50 con 0x7f60b8073fb0 2024-09-16T15:42:09.769 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:42:09.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6094078a00 msgr2=0x7f609407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6094078a00 0x7f609407aec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f60b809da90 tx=0x7f60a0009040 comp rx=0 tx=0).stop 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60b8073fb0 msgr2=0x7f60b80c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60b8073fb0 0x7f60b80c8fd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009950 tx=0x7f60a8033c20 comp rx=0 tx=0).stop 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 shutdown_connections 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f6094078a00 0x7f609407aec0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60b815f5a0 0x7f60b809d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60b80ccf00 0x7f60b81e9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 --2- 172.21.15.60:0/295182529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60b8073fb0 0x7f60b80c8fd0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 >> 172.21.15.60:0/295182529 conn(0x7f60b80b1950 msgr2=0x7f60b815e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 shutdown_connections 2024-09-16T15:42:09.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:09.770+0000 7f60bcc00640 1 -- 172.21.15.60:0/295182529 wait complete. 2024-09-16T15:42:09.788 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:42:09.788 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:42:09.788 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:42:09.788 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:42:09.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:09.789 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:42:09.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:42:09.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:42:09.789 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:42:09.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:09.789 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:42:09.829 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:42:09.829 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:42:09.829 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:42:09.829 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:09.829 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:42:09.829 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:09.829 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:09.829 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42519sec preferred_lft 42519sec 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:09.830 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:42:09.831 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:09.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:09.832 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:42:09.929 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:42:09.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:09.932 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:42:09.969 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:42:09.969 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:42:09.969 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:42:09.969 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:42:09.969 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:42:09.969 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:42:09.970 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:42:09.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:42:09.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:42:09.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:42:09.970 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:42:10.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:42:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:42:10.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:42:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:10.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:10.164 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:42:10.199 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:42:10.199 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:42:10.199 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:42:10.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:10.200 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:42:10.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:10.295 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:10.330 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:42:10.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:10.330 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:10.404 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:42:10.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:10.404 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:10.531 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:10.531 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:42:10.558 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:10.559 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:42:10.559 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:42:10.559 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:42:10.559 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:42:10.560 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:42:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:42:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:42:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:42:10.560 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:42:10.563 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:42:10.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:10.563 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:10.618 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:42:10.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:42:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:42:10.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:42:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:10.707 INFO:teuthology.orchestra.run.smithi100.stdout:client5417 2024-09-16T15:42:10.707 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:42:10.707 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:10.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:42:10.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.917+0000 7f0f48a00640 1 -- 172.21.15.60:0/534317358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 msgr2=0x7f0f28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.917+0000 7f0f48a00640 1 --2- 172.21.15.60:0/534317358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 0x7f0f28004c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c006f70 tx=0x7f0f28005830 comp rx=0 tx=0).stop 2024-09-16T15:42:10.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.917+0000 7f0f48a00640 1 -- 172.21.15.60:0/534317358 shutdown_connections 2024-09-16T15:42:10.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.917+0000 7f0f48a00640 1 --2- 172.21.15.60:0/534317358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 0x7f0f28004c80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.917+0000 7f0f48a00640 1 -- 172.21.15.60:0/534317358 >> 172.21.15.60:0/534317358 conn(0x7f0f440b18e0 msgr2=0x7f0f440b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:10.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.918+0000 7f0f48a00640 1 -- 172.21.15.60:0/534317358 shutdown_connections 2024-09-16T15:42:10.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.918+0000 7f0f48a00640 1 -- 172.21.15.60:0/534317358 wait complete. 2024-09-16T15:42:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.919+0000 7f0f48a00640 1 Processor -- start 2024-09-16T15:42:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.919+0000 7f0f48a00640 1 -- start start 2024-09-16T15:42:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f48a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 0x7f0f441f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f48a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f441f4c70 0x7f0f440cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f48a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f440b0760 0x7f0f4415d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f48a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0f440cd550 con 0x7f0f28004800 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f48a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f440cd3d0 con 0x7f0f440b0760 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f48a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4415fd80 con 0x7f0f441f4c70 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f41e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 0x7f0f441f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f41e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 0x7f0f441f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56404/0 (socket says 172.21.15.60:56404) 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.920+0000 7f0f41e00640 1 -- 172.21.15.60:0/3843428723 learned_addr learned my addr 172.21.15.60:0/3843428723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f42800640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f440b0760 0x7f0f4415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f39400640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f441f4c70 0x7f0f440cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f41e00640 1 -- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f441f4c70 msgr2=0x7f0f440cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f41e00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f441f4c70 0x7f0f440cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f41e00640 1 -- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f440b0760 msgr2=0x7f0f4415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f41e00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f440b0760 0x7f0f4415d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f41e00640 1 -- 172.21.15.60:0/3843428723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f441f5230 con 0x7f0f28004800 2024-09-16T15:42:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f41e00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 0x7f0f441f8c70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0f34011a40 tx=0x7f0f34011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.921+0000 7f0f3aa00640 1 -- 172.21.15.60:0/3843428723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3401e070 con 0x7f0f28004800 2024-09-16T15:42:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.922+0000 7f0f3aa00640 1 -- 172.21.15.60:0/3843428723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f34004460 con 0x7f0f28004800 2024-09-16T15:42:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.922+0000 7f0f3aa00640 1 -- 172.21.15.60:0/3843428723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3400fc00 con 0x7f0f28004800 2024-09-16T15:42:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.922+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f441f5510 con 0x7f0f28004800 2024-09-16T15:42:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.922+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f441f5ad0 con 0x7f0f28004800 2024-09-16T15:42:10.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.923+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f44069360 con 0x7f0f28004800 2024-09-16T15:42:10.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.924+0000 7f0f3aa00640 1 -- 172.21.15.60:0/3843428723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f0f34021070 con 0x7f0f28004800 2024-09-16T15:42:10.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.925+0000 7f0f3aa00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f20078a00 0x7f0f2007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.925+0000 7f0f3aa00640 1 -- 172.21.15.60:0/3843428723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f0f3409b640 con 0x7f0f28004800 2024-09-16T15:42:10.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.926+0000 7f0f39400640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f20078a00 0x7f0f2007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.926+0000 7f0f39400640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f20078a00 0x7f0f2007aec0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c033000 tx=0x7f0f440af430 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:10.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:10.929+0000 7f0f3aa00640 1 -- 172.21.15.60:0/3843428723 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0f340632e0 con 0x7f0f28004800 2024-09-16T15:42:11.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.163+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f0f440cd7b0 con 0x7f0f28004800 2024-09-16T15:42:11.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.726+0000 7f0f3aa00640 1 -- 172.21.15.60:0/3843428723 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f0f34067d10 con 0x7f0f28004800 2024-09-16T15:42:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.728+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f20078a00 msgr2=0x7f0f2007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.728+0000 7f0f48a00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f20078a00 0x7f0f2007aec0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c033000 tx=0x7f0f440af430 comp rx=0 tx=0).stop 2024-09-16T15:42:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 msgr2=0x7f0f441f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 0x7f0f441f8c70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0f34011a40 tx=0x7f0f34011f00 comp rx=0 tx=0).stop 2024-09-16T15:42:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 shutdown_connections 2024-09-16T15:42:11.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f0f20078a00 0x7f0f2007aec0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f440b0760 0x7f0f4415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f441f4c70 0x7f0f440cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 --2- 172.21.15.60:0/3843428723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f28004800 0x7f0f441f8c70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 >> 172.21.15.60:0/3843428723 conn(0x7f0f440b18e0 msgr2=0x7f0f440b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:11.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 shutdown_connections 2024-09-16T15:42:11.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.729+0000 7f0f48a00640 1 -- 172.21.15.60:0/3843428723 wait complete. 2024-09-16T15:42:11.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:42:11.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 -- 172.21.15.60:0/4022402968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc0ccf00 msgr2=0x7fefbc0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 --2- 172.21.15.60:0/4022402968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc0ccf00 0x7fefbc0acd30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fefbc075130 tx=0x7fefa802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 -- 172.21.15.60:0/4022402968 shutdown_connections 2024-09-16T15:42:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 --2- 172.21.15.60:0/4022402968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefbc0ad270 0x7fefbc073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 --2- 172.21.15.60:0/4022402968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc0ccf00 0x7fefbc0acd30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 --2- 172.21.15.60:0/4022402968 >> v2:172.21.15.100:3300/0 conn(0x7fefbc15f5a0 0x7fefbc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 -- 172.21.15.60:0/4022402968 >> 172.21.15.60:0/4022402968 conn(0x7fefbc0b1950 msgr2=0x7fefbc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 -- 172.21.15.60:0/4022402968 shutdown_connections 2024-09-16T15:42:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.913+0000 7fefc3600640 1 -- 172.21.15.60:0/4022402968 wait complete. 2024-09-16T15:42:11.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.915+0000 7fefc3600640 1 Processor -- start 2024-09-16T15:42:11.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.915+0000 7fefc3600640 1 -- start start 2024-09-16T15:42:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc3600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefbc0ad270 0x7fefbc09ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc3600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefbc0ccf00 0x7fefbc152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc3600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc15f5a0 0x7fefbc15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc3600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc074880 con 0x7fefbc15f5a0 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc3600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc074700 con 0x7fefbc0ccf00 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc3600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fefbc06f160 con 0x7fefbc0ad270 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc15f5a0 0x7fefbc15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc15f5a0 0x7fefbc15a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56424/0 (socket says 172.21.15.60:56424) 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc1600640 1 -- 172.21.15.60:0/1168688044 learned_addr learned my addr 172.21.15.60:0/1168688044 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefc0c00640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefbc0ad270 0x7fefbc09ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.916+0000 7fefbbe00640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefbc0ccf00 0x7fefbc152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefc1600640 1 -- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefbc0ad270 msgr2=0x7fefbc09ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefc1600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefbc0ad270 0x7fefbc09ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefc1600640 1 -- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefbc0ccf00 msgr2=0x7fefbc152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefc1600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefbc0ccf00 0x7fefbc152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefc1600640 1 -- 172.21.15.60:0/1168688044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc09be30 con 0x7fefbc15f5a0 2024-09-16T15:42:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefc1600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc15f5a0 0x7fefbc15a220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fefb400cce0 tx=0x7fefb400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefb8c00640 1 -- 172.21.15.60:0/1168688044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb400f450 con 0x7fefbc15f5a0 2024-09-16T15:42:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefb8c00640 1 -- 172.21.15.60:0/1168688044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefb400f6e0 con 0x7fefbc15f5a0 2024-09-16T15:42:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefbc09c110 con 0x7fefbc15f5a0 2024-09-16T15:42:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefb8c00640 1 -- 172.21.15.60:0/1168688044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb400fb30 con 0x7fefbc15f5a0 2024-09-16T15:42:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.917+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefbc09c6d0 con 0x7fefbc15f5a0 2024-09-16T15:42:11.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.918+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefbc073fb0 con 0x7fefbc15f5a0 2024-09-16T15:42:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.920+0000 7fefb8c00640 1 -- 172.21.15.60:0/1168688044 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7fefb400fcd0 con 0x7fefbc15f5a0 2024-09-16T15:42:11.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.920+0000 7fefb8c00640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fef9c078a00 0x7fef9c07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.921+0000 7fefb8c00640 1 -- 172.21.15.60:0/1168688044 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fefb409b790 con 0x7fefbc15f5a0 2024-09-16T15:42:11.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.921+0000 7fefc0c00640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fef9c078a00 0x7fef9c07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.922+0000 7fefc0c00640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fef9c078a00 0x7fef9c07aec0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fefac00bb50 tx=0x7fefac00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:11.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:11.925+0000 7fefb8c00640 1 -- 172.21.15.60:0/1168688044 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fefb4014030 con 0x7fefbc15f5a0 2024-09-16T15:42:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.146+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fefbc15b6e0 con 0x7fef9c078a00 2024-09-16T15:42:12.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.148+0000 7fefb8c00640 1 -- 172.21.15.60:0/1168688044 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fefbc15b6e0 con 0x7fef9c078a00 2024-09-16T15:42:12.149 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:42:12.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:12.149 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:42:12.149 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:12.149 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:42:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fef9c078a00 msgr2=0x7fef9c07aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fef9c078a00 0x7fef9c07aec0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fefac00bb50 tx=0x7fefac00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc15f5a0 msgr2=0x7fefbc15a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc15f5a0 0x7fefbc15a220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fefb400cce0 tx=0x7fefb400a620 comp rx=0 tx=0).stop 2024-09-16T15:42:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 shutdown_connections 2024-09-16T15:42:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fef9c078a00 0x7fef9c07aec0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fefbc15f5a0 0x7fefbc15a220 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fefbc0ccf00 0x7fefbc152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 --2- 172.21.15.60:0/1168688044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fefbc0ad270 0x7fefbc09ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 >> 172.21.15.60:0/1168688044 conn(0x7fefbc0b1950 msgr2=0x7fefbc0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 shutdown_connections 2024-09-16T15:42:12.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.150+0000 7fefc3600640 1 -- 172.21.15.60:0/1168688044 wait complete. 2024-09-16T15:42:12.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:42:12.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.330+0000 7f954d000640 1 -- 172.21.15.60:0/991144714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95480d0400 msgr2=0x7f95480ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.330+0000 7f954d000640 1 --2- 172.21.15.60:0/991144714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95480d0400 0x7f95480ae9a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9540005fd0 tx=0x7f954002aa10 comp rx=0 tx=0).stop 2024-09-16T15:42:12.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.330+0000 7f954d000640 1 -- 172.21.15.60:0/991144714 shutdown_connections 2024-09-16T15:42:12.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.330+0000 7f954d000640 1 --2- 172.21.15.60:0/991144714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95480d0400 0x7f95480ae9a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.330+0000 7f954d000640 1 --2- 172.21.15.60:0/991144714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f954815f260 0x7f9548072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.330+0000 7f954d000640 1 -- 172.21.15.60:0/991144714 >> 172.21.15.60:0/991144714 conn(0x7f95480d1bf0 msgr2=0x7f95480d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:12.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.330+0000 7f954d000640 1 -- 172.21.15.60:0/991144714 shutdown_connections 2024-09-16T15:42:12.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.330+0000 7f954d000640 1 -- 172.21.15.60:0/991144714 wait complete. 2024-09-16T15:42:12.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.331+0000 7f954d000640 1 Processor -- start 2024-09-16T15:42:12.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.332+0000 7f954d000640 1 -- start start 2024-09-16T15:42:12.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f954d000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95480d0400 0x7f9548152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f954d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954815f260 0x7f95480b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f954d000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f954815f640 0x7f95480b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f954d000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f95480af060 con 0x7f954815f640 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f954d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95480aeee0 con 0x7f954815f260 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f954d000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f95480af1e0 con 0x7f95480d0400 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f954815f640 0x7f95480b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f954815f640 0x7f95480b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56444/0 (socket says 172.21.15.60:56444) 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547e00640 1 -- 172.21.15.60:0/2975122883 learned_addr learned my addr 172.21.15.60:0/2975122883 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547400640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95480d0400 0x7f9548152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9546a00640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954815f260 0x7f95480b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547e00640 1 -- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95480d0400 msgr2=0x7f9548152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547e00640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95480d0400 0x7f9548152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547e00640 1 -- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954815f260 msgr2=0x7f95480b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547e00640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954815f260 0x7f95480b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547e00640 1 -- 172.21.15.60:0/2975122883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95480c9230 con 0x7f954815f640 2024-09-16T15:42:12.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9547400640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95480d0400 0x7f9548152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:12.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.333+0000 7f9546a00640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954815f260 0x7f95480b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:12.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.334+0000 7f9547e00640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f954815f640 0x7f95480b4050 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9540006010 tx=0x7f9540003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:12.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.334+0000 7f953f400640 1 -- 172.21.15.60:0/2975122883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540047070 con 0x7f954815f640 2024-09-16T15:42:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.334+0000 7f953f400640 1 -- 172.21.15.60:0/2975122883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9540004250 con 0x7f954815f640 2024-09-16T15:42:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.334+0000 7f953f400640 1 -- 172.21.15.60:0/2975122883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540033050 con 0x7f954815f640 2024-09-16T15:42:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.334+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95480c9510 con 0x7f954815f640 2024-09-16T15:42:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.334+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95480c9ad0 con 0x7f954815f640 2024-09-16T15:42:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.335+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95480ae9a0 con 0x7f954815f640 2024-09-16T15:42:12.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.336+0000 7f953f400640 1 -- 172.21.15.60:0/2975122883 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f954004a070 con 0x7f954815f640 2024-09-16T15:42:12.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.338+0000 7f953f400640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9528078a00 0x7f952807aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.338+0000 7f953f400640 1 -- 172.21.15.60:0/2975122883 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f95400bbff0 con 0x7f954815f640 2024-09-16T15:42:12.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.338+0000 7f9547400640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9528078a00 0x7f952807aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.339+0000 7f9547400640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9528078a00 0x7f952807aec0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f95380058c0 tx=0x7f9538005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:12.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.342+0000 7f953f400640 1 -- 172.21.15.60:0/2975122883 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9540084270 con 0x7f954815f640 2024-09-16T15:42:12.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.537+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f95480cf8c0 con 0x7f954815f640 2024-09-16T15:42:12.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.537+0000 7f953f400640 1 -- 172.21.15.60:0/2975122883 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f9540083b70 con 0x7f954815f640 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.539+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9528078a00 msgr2=0x7f952807aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.539+0000 7f954d000640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9528078a00 0x7f952807aec0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f95380058c0 tx=0x7f9538005850 comp rx=0 tx=0).stop 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.539+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f954815f640 msgr2=0x7f95480b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.539+0000 7f954d000640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f954815f640 0x7f95480b4050 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9540006010 tx=0x7f9540003f10 comp rx=0 tx=0).stop 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.540+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 shutdown_connections 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.540+0000 7f954d000640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9528078a00 0x7f952807aec0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.540+0000 7f954d000640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f954815f640 0x7f95480b4050 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.540+0000 7f954d000640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f954815f260 0x7f95480b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.540+0000 7f954d000640 1 --2- 172.21.15.60:0/2975122883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95480d0400 0x7f9548152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.540+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 >> 172.21.15.60:0/2975122883 conn(0x7f95480d1bf0 msgr2=0x7f95480cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:12.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.540+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 shutdown_connections 2024-09-16T15:42:12.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.540+0000 7f954d000640 1 -- 172.21.15.60:0/2975122883 wait complete. 2024-09-16T15:42:12.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:12.556 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol75450) 2024-09-16T15:42:12.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:12.621 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:42:12.658 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:42:12.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:12.658 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol75450) 2024-09-16T15:42:12.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol97901 2024-09-16T15:42:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.923+0000 7f1a66600640 1 -- 172.21.15.60:0/1872444705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a600c8de0 msgr2=0x7f1a60168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.923+0000 7f1a66600640 1 --2- 172.21.15.60:0/1872444705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a600c8de0 0x7f1a60168a00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1a58005fd0 tx=0x7f1a5802a940 comp rx=0 tx=0).stop 2024-09-16T15:42:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.923+0000 7f1a66600640 1 -- 172.21.15.60:0/1872444705 shutdown_connections 2024-09-16T15:42:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.923+0000 7f1a66600640 1 --2- 172.21.15.60:0/1872444705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a600c8de0 0x7f1a60168a00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.923+0000 7f1a66600640 1 -- 172.21.15.60:0/1872444705 >> 172.21.15.60:0/1872444705 conn(0x7f1a600d3e60 msgr2=0x7f1a600d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.923+0000 7f1a66600640 1 -- 172.21.15.60:0/1872444705 shutdown_connections 2024-09-16T15:42:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.923+0000 7f1a66600640 1 -- 172.21.15.60:0/1872444705 wait complete. 2024-09-16T15:42:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.924+0000 7f1a66600640 1 Processor -- start 2024-09-16T15:42:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.925+0000 7f1a66600640 1 -- start start 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.925+0000 7f1a66600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a600c8de0 0x7f1a6014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.925+0000 7f1a66600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a600d4980 0x7f1a6015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.925+0000 7f1a66600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6015df50 0x7f1a6014f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a66600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601690c0 con 0x7f1a6015df50 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a66600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60168f40 con 0x7f1a600d4980 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a66600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1a60169240 con 0x7f1a600c8de0 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6015df50 0x7f1a6014f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6015df50 0x7f1a6014f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56454/0 (socket says 172.21.15.60:56454) 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65c00640 1 -- 172.21.15.60:0/2892860150 learned_addr learned my addr 172.21.15.60:0/2892860150 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a5fe00640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a600d4980 0x7f1a6015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65200640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a600c8de0 0x7f1a6014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65c00640 1 -- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a600c8de0 msgr2=0x7f1a6014e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65c00640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a600c8de0 0x7f1a6014e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65c00640 1 -- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a600d4980 msgr2=0x7f1a6015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65c00640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a600d4980 0x7f1a6015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65c00640 1 -- 172.21.15.60:0/2892860150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6015daa0 con 0x7f1a6015df50 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.926+0000 7f1a65200640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a600c8de0 0x7f1a6014e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.927+0000 7f1a65c00640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6015df50 0x7f1a6014f7d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009cd0 tx=0x7f1a58031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.927+0000 7f1a5cc00640 1 -- 172.21.15.60:0/2892860150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58004240 con 0x7f1a6015df50 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.927+0000 7f1a5cc00640 1 -- 172.21.15.60:0/2892860150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a58004400 con 0x7f1a6015df50 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.927+0000 7f1a5cc00640 1 -- 172.21.15.60:0/2892860150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58004850 con 0x7f1a6015df50 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.927+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6015bea0 con 0x7f1a6015df50 2024-09-16T15:42:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.927+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6014efd0 con 0x7f1a6015df50 2024-09-16T15:42:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.928+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a600ce930 con 0x7f1a6015df50 2024-09-16T15:42:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.929+0000 7f1a5cc00640 1 -- 172.21.15.60:0/2892860150 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 9) ==== 101440+0+0 (secure 0 0 0) 0x7f1a58039440 con 0x7f1a6015df50 2024-09-16T15:42:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.930+0000 7f1a5cc00640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1a40078ba0 0x7f1a4007b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.930+0000 7f1a5cc00640 1 -- 172.21.15.60:0/2892860150 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f1a580bc6f0 con 0x7f1a6015df50 2024-09-16T15:42:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.931+0000 7f1a65200640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1a40078ba0 0x7f1a4007b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.931+0000 7f1a65200640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1a40078ba0 0x7f1a4007b060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1a60150500 tx=0x7f1a54009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:12.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:12.935+0000 7f1a5cc00640 1 -- 172.21.15.60:0/2892860150 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1a580849c0 con 0x7f1a6015df50 2024-09-16T15:42:13.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.155+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol97901", "target": ["mon-mgr", ""]}) -- 0x7f1a6006ecb0 con 0x7f1a40078ba0 2024-09-16T15:42:13.163 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:13.161+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:13.163 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:13.161+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:13.163 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:13.161+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:13.163 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:13.161+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:13.163 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:13.161+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:13.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.203+0000 7f1a5cc00640 1 -- 172.21.15.60:0/2892860150 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1a6006ecb0 con 0x7f1a40078ba0 2024-09-16T15:42:13.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1a40078ba0 msgr2=0x7f1a4007b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:13.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1a40078ba0 0x7f1a4007b060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1a60150500 tx=0x7f1a54009040 comp rx=0 tx=0).stop 2024-09-16T15:42:13.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6015df50 msgr2=0x7f1a6014f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:13.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6015df50 0x7f1a6014f7d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009cd0 tx=0x7f1a58031ea0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 shutdown_connections 2024-09-16T15:42:13.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1a40078ba0 0x7f1a4007b060 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a6015df50 0x7f1a6014f7d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a600d4980 0x7f1a6015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 --2- 172.21.15.60:0/2892860150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a600c8de0 0x7f1a6014e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.205+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 >> 172.21.15.60:0/2892860150 conn(0x7f1a600d3e60 msgr2=0x7f1a6015a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:13.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.206+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 shutdown_connections 2024-09-16T15:42:13.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:13.206+0000 7f1a66600640 1 -- 172.21.15.60:0/2892860150 wait complete. 2024-09-16T15:42:13.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:13.223 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol97901/.meta ./volumes/_nogroup/subvol75450) 2024-09-16T15:42:16.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol75450 2024-09-16T15:42:16.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.555+0000 7f350d600640 1 -- 172.21.15.60:0/1167896437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35080cbc00 msgr2=0x7f34ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:16.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.555+0000 7f350d600640 1 --2- 172.21.15.60:0/1167896437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35080cbc00 0x7f34ec004800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3508072b20 tx=0x7f34ec005830 comp rx=0 tx=0).stop 2024-09-16T15:42:16.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.555+0000 7f350cc00640 1 -- 172.21.15.60:0/1167896437 reap_dead start 2024-09-16T15:42:16.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.555+0000 7f350d600640 1 -- 172.21.15.60:0/1167896437 shutdown_connections 2024-09-16T15:42:16.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.555+0000 7f350d600640 1 -- 172.21.15.60:0/1167896437 >> 172.21.15.60:0/1167896437 conn(0x7f35080cf9f0 msgr2=0x7f35080cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:16.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.555+0000 7f350d600640 1 -- 172.21.15.60:0/1167896437 shutdown_connections 2024-09-16T15:42:16.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.555+0000 7f350d600640 1 -- 172.21.15.60:0/1167896437 wait complete. 2024-09-16T15:42:16.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.557+0000 7f350d600640 1 Processor -- start 2024-09-16T15:42:16.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.557+0000 7f350d600640 1 -- start start 2024-09-16T15:42:16.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f350d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35080cbc00 0x7f3508159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:16.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f350d600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35080c5a80 0x7f35080c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:16.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f350d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35080c85f0 0x7f35080ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:16.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f350d600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3508074280 con 0x7f35080cbc00 2024-09-16T15:42:16.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f350d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3508074100 con 0x7f35080c85f0 2024-09-16T15:42:16.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f350d600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3508074400 con 0x7f35080c5a80 2024-09-16T15:42:16.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f3507e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35080cbc00 0x7f3508159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f3507e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35080cbc00 0x7f3508159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56472/0 (socket says 172.21.15.60:56472) 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f3507e00640 1 -- 172.21.15.60:0/650688328 learned_addr learned my addr 172.21.15.60:0/650688328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f3507400640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35080c5a80 0x7f35080c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f350cc00640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35080c85f0 0x7f35080ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.558+0000 7f3507e00640 1 -- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35080c5a80 msgr2=0x7f35080c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f3507e00640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35080c5a80 0x7f35080c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f3507e00640 1 -- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35080c85f0 msgr2=0x7f35080ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f3507e00640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35080c85f0 0x7f35080ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f3507e00640 1 -- 172.21.15.60:0/650688328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35080c8a70 con 0x7f35080cbc00 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f350cc00640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35080c85f0 0x7f35080ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f3507e00640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35080cbc00 0x7f3508159dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f34f80066b0 tx=0x7f34f80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f34ffe00640 1 -- 172.21.15.60:0/650688328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8015070 con 0x7f35080cbc00 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f34ffe00640 1 -- 172.21.15.60:0/650688328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f8011920 con 0x7f35080cbc00 2024-09-16T15:42:16.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f34ffe00640 1 -- 172.21.15.60:0/650688328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8011d70 con 0x7f35080cbc00 2024-09-16T15:42:16.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35080c8d50 con 0x7f35080cbc00 2024-09-16T15:42:16.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.559+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35080c9310 con 0x7f35080cbc00 2024-09-16T15:42:16.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.560+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35080c96b0 con 0x7f35080cbc00 2024-09-16T15:42:16.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.561+0000 7f34ffe00640 1 -- 172.21.15.60:0/650688328 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 10) ==== 101491+0+0 (secure 0 0 0) 0x7f34f800f430 con 0x7f35080cbc00 2024-09-16T15:42:16.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.562+0000 7f34ffe00640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34e8078a90 0x7f34e807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:16.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.562+0000 7f34ffe00640 1 -- 172.21.15.60:0/650688328 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f34f809bb40 con 0x7f35080cbc00 2024-09-16T15:42:16.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.563+0000 7f3507400640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34e8078a90 0x7f34e807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:16.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.564+0000 7f3507400640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34e8078a90 0x7f34e807af50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f34f4035000 tx=0x7f34f402d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:16.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.567+0000 7f34ffe00640 1 -- 172.21.15.60:0/650688328 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f34f80636e0 con 0x7f35080cbc00 2024-09-16T15:42:16.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.796+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol75450", "target": ["mon-mgr", ""]}) -- 0x7f35080cd5c0 con 0x7f34e8078a90 2024-09-16T15:42:16.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.835+0000 7f34ffe00640 1 -- 172.21.15.60:0/650688328 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f35080cd5c0 con 0x7f34e8078a90 2024-09-16T15:42:16.837 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol75450 2024-09-16T15:42:16.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34e8078a90 msgr2=0x7f34e807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:16.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34e8078a90 0x7f34e807af50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f34f4035000 tx=0x7f34f402d040 comp rx=0 tx=0).stop 2024-09-16T15:42:16.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35080cbc00 msgr2=0x7f3508159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:16.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35080cbc00 0x7f3508159dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f34f80066b0 tx=0x7f34f80039c0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 shutdown_connections 2024-09-16T15:42:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f34e8078a90 0x7f34e807af50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35080c85f0 0x7f35080ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35080c5a80 0x7f35080c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 --2- 172.21.15.60:0/650688328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35080cbc00 0x7f3508159dc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 >> 172.21.15.60:0/650688328 conn(0x7f35080cf9f0 msgr2=0x7f350815f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 shutdown_connections 2024-09-16T15:42:16.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:16.838+0000 7f350d600640 1 -- 172.21.15.60:0/650688328 wait complete. 2024-09-16T15:42:16.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol75450 2024-09-16T15:42:17.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.018+0000 7f9d7de00640 1 -- 172.21.15.60:0/2099688551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d78145750 msgr2=0x7f9d78073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.018+0000 7f9d7de00640 1 --2- 172.21.15.60:0/2099688551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d78145750 0x7f9d78073440 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009830 tx=0x7f9d6802a480 comp rx=0 tx=0).stop 2024-09-16T15:42:17.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.018+0000 7f9d7de00640 1 -- 172.21.15.60:0/2099688551 shutdown_connections 2024-09-16T15:42:17.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.018+0000 7f9d7de00640 1 --2- 172.21.15.60:0/2099688551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d78145750 0x7f9d78073440 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.018+0000 7f9d7de00640 1 -- 172.21.15.60:0/2099688551 >> 172.21.15.60:0/2099688551 conn(0x7f9d780b1840 msgr2=0x7f9d780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.018+0000 7f9d7de00640 1 -- 172.21.15.60:0/2099688551 shutdown_connections 2024-09-16T15:42:17.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.018+0000 7f9d7de00640 1 -- 172.21.15.60:0/2099688551 wait complete. 2024-09-16T15:42:17.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.020+0000 7f9d7de00640 1 Processor -- start 2024-09-16T15:42:17.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.020+0000 7f9d7de00640 1 -- start start 2024-09-16T15:42:17.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d7de00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7806efa0 0x7f9d780cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d7de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d780cccd0 0x7f9d780c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d7de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d78145750 0x7f9d7813e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d7de00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9d780ce650 con 0x7f9d780cccd0 2024-09-16T15:42:17.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d7de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d780ce4d0 con 0x7f9d78145750 2024-09-16T15:42:17.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d7de00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9d780ce7d0 con 0x7f9d7806efa0 2024-09-16T15:42:17.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d76000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d780cccd0 0x7f9d780c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d76000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d780cccd0 0x7f9d780c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56488/0 (socket says 172.21.15.60:56488) 2024-09-16T15:42:17.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d76000640 1 -- 172.21.15.60:0/3813756833 learned_addr learned my addr 172.21.15.60:0/3813756833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:17.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d76a00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7806efa0 0x7f9d780cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.021+0000 7f9d77400640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d78145750 0x7f9d7813e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d76000640 1 -- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7806efa0 msgr2=0x7f9d780cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d76000640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7806efa0 0x7f9d780cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d76000640 1 -- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d78145750 msgr2=0x7f9d7813e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d76000640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d78145750 0x7f9d7813e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d76000640 1 -- 172.21.15.60:0/3813756833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d780c4ff0 con 0x7f9d780cccd0 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d76000640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d780cccd0 0x7f9d780c4ab0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c500 tx=0x7f9d6400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d6f400640 1 -- 172.21.15.60:0/3813756833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64018070 con 0x7f9d780cccd0 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d6f400640 1 -- 172.21.15.60:0/3813756833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d640035f0 con 0x7f9d780cccd0 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d780c52c0 con 0x7f9d780cccd0 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d6f400640 1 -- 172.21.15.60:0/3813756833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64003ac0 con 0x7f9d780cccd0 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.022+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d781f4e60 con 0x7f9d780cccd0 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.023+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d780ceca0 con 0x7f9d780cccd0 2024-09-16T15:42:17.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.024+0000 7f9d6f400640 1 -- 172.21.15.60:0/3813756833 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 10) ==== 101491+0+0 (secure 0 0 0) 0x7f9d6401b070 con 0x7f9d780cccd0 2024-09-16T15:42:17.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.025+0000 7f9d6f400640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a90 0x7f9d5807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.026+0000 7f9d6f400640 1 -- 172.21.15.60:0/3813756833 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f9d6409a160 con 0x7f9d780cccd0 2024-09-16T15:42:17.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.026+0000 7f9d76a00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a90 0x7f9d5807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.027+0000 7f9d76a00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a90 0x7f9d5807af50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9d68034000 tx=0x7f9d68004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.030+0000 7f9d6f400640 1 -- 172.21.15.60:0/3813756833 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d64061c80 con 0x7f9d780cccd0 2024-09-16T15:42:17.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.252+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol75450", "target": ["mon-mgr", ""]}) -- 0x7f9d780b4c40 con 0x7f9d58078a90 2024-09-16T15:42:17.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.259+0000 7f9d6f400640 1 -- 172.21.15.60:0/3813756833 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f9d68004420 con 0x7f9d58078a90 2024-09-16T15:42:17.260 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol75450 2024-09-16T15:42:17.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a90 msgr2=0x7f9d5807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a90 0x7f9d5807af50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9d68034000 tx=0x7f9d68004730 comp rx=0 tx=0).stop 2024-09-16T15:42:17.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d780cccd0 msgr2=0x7f9d780c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d780cccd0 0x7f9d780c4ab0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c500 tx=0x7f9d6400c9c0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 shutdown_connections 2024-09-16T15:42:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d58078a90 0x7f9d5807af50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d78145750 0x7f9d7813e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d780cccd0 0x7f9d780c4ab0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 --2- 172.21.15.60:0/3813756833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7806efa0 0x7f9d780cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 >> 172.21.15.60:0/3813756833 conn(0x7f9d780b1840 msgr2=0x7f9d780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 shutdown_connections 2024-09-16T15:42:17.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.261+0000 7f9d7de00640 1 -- 172.21.15.60:0/3813756833 wait complete. 2024-09-16T15:42:17.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:17.279 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/0a2c8bee6abaa9e838a08b5453ccbf25.meta) 2024-09-16T15:42:17.339 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:42:17.340 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:42:17.340 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:42:17.340 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/_nogroup/subvol75450 2024-09-16T15:42:17.340 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:42:17.340 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:17.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol75450 alice 2024-09-16T15:42:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.506+0000 7f873b400640 1 -- 172.21.15.60:0/810493933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8734159aa0 msgr2=0x7f87340741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.506+0000 7f873b400640 1 --2- 172.21.15.60:0/810493933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8734159aa0 0x7f87340741a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8728009830 tx=0x7f872802a520 comp rx=0 tx=0).stop 2024-09-16T15:42:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.506+0000 7f873b400640 1 -- 172.21.15.60:0/810493933 shutdown_connections 2024-09-16T15:42:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.506+0000 7f873b400640 1 --2- 172.21.15.60:0/810493933 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873406f220 0x7f873406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.506+0000 7f873b400640 1 --2- 172.21.15.60:0/810493933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8734159aa0 0x7f87340741a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.506+0000 7f873b400640 1 -- 172.21.15.60:0/810493933 >> 172.21.15.60:0/810493933 conn(0x7f87340b18e0 msgr2=0x7f87340b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.506+0000 7f873b400640 1 -- 172.21.15.60:0/810493933 shutdown_connections 2024-09-16T15:42:17.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.506+0000 7f873b400640 1 -- 172.21.15.60:0/810493933 wait complete. 2024-09-16T15:42:17.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.508+0000 7f873b400640 1 Processor -- start 2024-09-16T15:42:17.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.508+0000 7f873b400640 1 -- start start 2024-09-16T15:42:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f873b400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873406f220 0x7f87340b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f873b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8734159aa0 0x7f87340c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f873b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873415cff0 0x7f87340adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f873b400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f87340cc520 con 0x7f873415cff0 2024-09-16T15:42:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f873b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87340cc3a0 con 0x7f8734159aa0 2024-09-16T15:42:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f873b400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f87340cc6a0 con 0x7f873406f220 2024-09-16T15:42:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f8739400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873415cff0 0x7f87340adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f8739400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873415cff0 0x7f87340adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56508/0 (socket says 172.21.15.60:56508) 2024-09-16T15:42:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f8739400640 1 -- 172.21.15.60:0/1869113996 learned_addr learned my addr 172.21.15.60:0/1869113996 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f8738a00640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873406f220 0x7f87340b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.509+0000 7f8733e00640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8734159aa0 0x7f87340c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8739400640 1 -- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873406f220 msgr2=0x7f87340b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8739400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873406f220 0x7f87340b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8739400640 1 -- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8734159aa0 msgr2=0x7f87340c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8739400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8734159aa0 0x7f87340c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8739400640 1 -- 172.21.15.60:0/1869113996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87340cb4e0 con 0x7f873415cff0 2024-09-16T15:42:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8739400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873415cff0 0x7f87340adfe0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f872c00eaa0 tx=0x7f872c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8730c00640 1 -- 172.21.15.60:0/1869113996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c00cdb0 con 0x7f873415cff0 2024-09-16T15:42:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8730c00640 1 -- 172.21.15.60:0/1869113996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f872c004c10 con 0x7f873415cff0 2024-09-16T15:42:17.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f8730c00640 1 -- 172.21.15.60:0/1869113996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c005020 con 0x7f873415cff0 2024-09-16T15:42:17.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.510+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87340cb7c0 con 0x7f873415cff0 2024-09-16T15:42:17.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.511+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8734153280 con 0x7f873415cff0 2024-09-16T15:42:17.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.512+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8734153480 con 0x7f873415cff0 2024-09-16T15:42:17.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.513+0000 7f8730c00640 1 -- 172.21.15.60:0/1869113996 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 10) ==== 101491+0+0 (secure 0 0 0) 0x7f872c0051c0 con 0x7f873415cff0 2024-09-16T15:42:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.514+0000 7f8730c00640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8714078a90 0x7f871407af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.514+0000 7f8730c00640 1 -- 172.21.15.60:0/1869113996 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f872c09b740 con 0x7f873415cff0 2024-09-16T15:42:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.514+0000 7f8738a00640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8714078a90 0x7f871407af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.515+0000 7f8738a00640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8714078a90 0x7f871407af50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8728034000 tx=0x7f8728004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.518+0000 7f8730c00640 1 -- 172.21.15.60:0/1869113996 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8734153480 con 0x7f873415cff0 2024-09-16T15:42:17.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.746+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol75450", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7f8734070980 con 0x7f8714078a90 2024-09-16T15:42:17.758 INFO:tasks.ceph.mgr.x.smithi100.stderr:Creating meta for ID alice with tenant None 2024-09-16T15:42:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.793+0000 7f8730c00640 1 -- 172.21.15.60:0/1869113996 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8734070980 con 0x7f8714078a90 2024-09-16T15:42:17.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8714078a90 msgr2=0x7f871407af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8714078a90 0x7f871407af50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8728034000 tx=0x7f8728004730 comp rx=0 tx=0).stop 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873415cff0 msgr2=0x7f87340adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873415cff0 0x7f87340adfe0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f872c00eaa0 tx=0x7f872c00ef60 comp rx=0 tx=0).stop 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 shutdown_connections 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8714078a90 0x7f871407af50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873415cff0 0x7f87340adfe0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8734159aa0 0x7f87340c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 --2- 172.21.15.60:0/1869113996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873406f220 0x7f87340b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 >> 172.21.15.60:0/1869113996 conn(0x7f87340b18e0 msgr2=0x7f8734159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 shutdown_connections 2024-09-16T15:42:17.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.795+0000 7f873b400640 1 -- 172.21.15.60:0/1869113996 wait complete. 2024-09-16T15:42:17.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-16T15:42:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.971+0000 7f81ee400640 1 -- 172.21.15.60:0/2608224701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e815f5a0 msgr2=0x7f81e80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.971+0000 7f81ee400640 1 --2- 172.21.15.60:0/2608224701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e815f5a0 0x7f81e80b2440 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f81e80cfbb0 tx=0x7f81d802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.972+0000 7f81ee400640 1 -- 172.21.15.60:0/2608224701 shutdown_connections 2024-09-16T15:42:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.972+0000 7f81ee400640 1 --2- 172.21.15.60:0/2608224701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e80ad270 0x7f81e8073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.972+0000 7f81ee400640 1 --2- 172.21.15.60:0/2608224701 >> v2:172.21.15.100:3300/0 conn(0x7f81e80ccf00 0x7f81e80acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.972+0000 7f81ee400640 1 --2- 172.21.15.60:0/2608224701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e815f5a0 0x7f81e80b2440 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.972+0000 7f81ee400640 1 -- 172.21.15.60:0/2608224701 >> 172.21.15.60:0/2608224701 conn(0x7f81e80b1950 msgr2=0x7f81e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.972+0000 7f81ee400640 1 -- 172.21.15.60:0/2608224701 shutdown_connections 2024-09-16T15:42:17.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.972+0000 7f81ee400640 1 -- 172.21.15.60:0/2608224701 wait complete. 2024-09-16T15:42:17.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.973+0000 7f81ee400640 1 Processor -- start 2024-09-16T15:42:17.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.973+0000 7f81ee400640 1 -- start start 2024-09-16T15:42:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.974+0000 7f81ee400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80ad270 0x7f81e81e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.974+0000 7f81ee400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80ccf00 0x7f81e81f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.974+0000 7f81ee400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815f5a0 0x7f81e81f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.974+0000 7f81ee400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8074880 con 0x7f81e80ccf00 2024-09-16T15:42:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.974+0000 7f81ee400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8074700 con 0x7f81e80ad270 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.974+0000 7f81ee400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f81e806f160 con 0x7f81e815f5a0 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80ccf00 0x7f81e81f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80ccf00 0x7f81e81f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56530/0 (socket says 172.21.15.60:56530) 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 -- 172.21.15.60:0/3243650618 learned_addr learned my addr 172.21.15.60:0/3243650618 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81e7400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80ad270 0x7f81e81e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81e7e00640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815f5a0 0x7f81e81f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 -- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815f5a0 msgr2=0x7f81e81f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815f5a0 0x7f81e81f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 -- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80ad270 msgr2=0x7f81e81e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80ad270 0x7f81e81e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 -- 172.21.15.60:0/3243650618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e80c91b0 con 0x7f81e80ccf00 2024-09-16T15:42:17.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.975+0000 7f81dea00640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80ccf00 0x7f81e81f4ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f81d000d950 tx=0x7f81d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.976+0000 7f81dfe00640 1 -- 172.21.15.60:0/3243650618 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0017070 con 0x7f81e80ccf00 2024-09-16T15:42:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.976+0000 7f81dfe00640 1 -- 172.21.15.60:0/3243650618 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81d0009d20 con 0x7f81e80ccf00 2024-09-16T15:42:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.976+0000 7f81dfe00640 1 -- 172.21.15.60:0/3243650618 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d00106c0 con 0x7f81e80ccf00 2024-09-16T15:42:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.976+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e80c9490 con 0x7f81e80ccf00 2024-09-16T15:42:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.976+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e81e5620 con 0x7f81e80ccf00 2024-09-16T15:42:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.977+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e81e58e0 con 0x7f81e80ccf00 2024-09-16T15:42:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.978+0000 7f81dfe00640 1 -- 172.21.15.60:0/3243650618 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 10) ==== 101491+0+0 (secure 0 0 0) 0x7f81d0010860 con 0x7f81e80ccf00 2024-09-16T15:42:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.979+0000 7f81dfe00640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81c4078ac0 0x7f81c407af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.979+0000 7f81dfe00640 1 -- 172.21.15.60:0/3243650618 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f81d009b1d0 con 0x7f81e80ccf00 2024-09-16T15:42:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.979+0000 7f81e7400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81c4078ac0 0x7f81c407af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.980+0000 7f81e7400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81c4078ac0 0x7f81c407af80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f81d8033000 tx=0x7f81d8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:17.983+0000 7f81dfe00640 1 -- 172.21.15.60:0/3243650618 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f81d0062ce0 con 0x7f81e80ccf00 2024-09-16T15:42:18.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.248+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7f81e8073fb0 con 0x7f81e80ccf00 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.249+0000 7f81dfe00640 1 -- 172.21.15.60:0/3243650618 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v12) ==== 107+0+280 (secure 0 0 0) 0x7f81d0067710 con 0x7f81e80ccf00 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.alice", 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUehmgYz4LRAA9MddHC7F/dmFVD0at+o5Ww==", 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol75450", 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-16T15:42:18.250 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:18.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:18.251 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:42:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81c4078ac0 msgr2=0x7f81c407af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81c4078ac0 0x7f81c407af80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f81d8033000 tx=0x7f81d8004730 comp rx=0 tx=0).stop 2024-09-16T15:42:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80ccf00 msgr2=0x7f81e81f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80ccf00 0x7f81e81f4ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f81d000d950 tx=0x7f81d000de10 comp rx=0 tx=0).stop 2024-09-16T15:42:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 shutdown_connections 2024-09-16T15:42:18.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81c4078ac0 0x7f81c407af80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81e815f5a0 0x7f81e81f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81e80ccf00 0x7f81e81f4ee0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 --2- 172.21.15.60:0/3243650618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81e80ad270 0x7f81e81e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 >> 172.21.15.60:0/3243650618 conn(0x7f81e80b1950 msgr2=0x7f81e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:18.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 shutdown_connections 2024-09-16T15:42:18.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.251+0000 7f81ee400640 1 -- 172.21.15.60:0/3243650618 wait complete. 2024-09-16T15:42:18.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol75450 2024-09-16T15:42:18.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.430+0000 7fed9e400640 1 -- 172.21.15.60:0/43300227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed98145750 msgr2=0x7fed98073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.430+0000 7fed9e400640 1 --2- 172.21.15.60:0/43300227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed98145750 0x7fed98073440 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fed88009830 tx=0x7fed8802a530 comp rx=0 tx=0).stop 2024-09-16T15:42:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.430+0000 7fed9e400640 1 -- 172.21.15.60:0/43300227 shutdown_connections 2024-09-16T15:42:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.430+0000 7fed9e400640 1 --2- 172.21.15.60:0/43300227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed98145750 0x7fed98073440 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.430+0000 7fed9e400640 1 -- 172.21.15.60:0/43300227 >> 172.21.15.60:0/43300227 conn(0x7fed980b1840 msgr2=0x7fed980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.430+0000 7fed9e400640 1 -- 172.21.15.60:0/43300227 shutdown_connections 2024-09-16T15:42:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.430+0000 7fed9e400640 1 -- 172.21.15.60:0/43300227 wait complete. 2024-09-16T15:42:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.432+0000 7fed9e400640 1 Processor -- start 2024-09-16T15:42:18.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.432+0000 7fed9e400640 1 -- start start 2024-09-16T15:42:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed9e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed9806efa0 0x7fed980ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed9e400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed980cccd0 0x7fed9813c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed9e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed98145750 0x7fed9815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed9e400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fed980ce6a0 con 0x7fed9806efa0 2024-09-16T15:42:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed9e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed980ce520 con 0x7fed98145750 2024-09-16T15:42:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed9e400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fed980ce820 con 0x7fed980cccd0 2024-09-16T15:42:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed97400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed9806efa0 0x7fed980ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed97400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed9806efa0 0x7fed980ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56548/0 (socket says 172.21.15.60:56548) 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed97400640 1 -- 172.21.15.60:0/896359373 learned_addr learned my addr 172.21.15.60:0/896359373 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed96a00640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed980cccd0 0x7fed9813c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.433+0000 7fed97e00640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed98145750 0x7fed9815e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.434+0000 7fed97400640 1 -- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed980cccd0 msgr2=0x7fed9813c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.434+0000 7fed97400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed980cccd0 0x7fed9813c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.434+0000 7fed97400640 1 -- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed98145750 msgr2=0x7fed9815e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.434+0000 7fed97400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed98145750 0x7fed9815e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.434+0000 7fed97400640 1 -- 172.21.15.60:0/896359373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9813c8f0 con 0x7fed9806efa0 2024-09-16T15:42:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.434+0000 7fed97400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed9806efa0 0x7fed980ae2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fed88000c00 tx=0x7fed88033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:18.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.434+0000 7fed8f400640 1 -- 172.21.15.60:0/896359373 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88031d60 con 0x7fed9806efa0 2024-09-16T15:42:18.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.435+0000 7fed8f400640 1 -- 172.21.15.60:0/896359373 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed880330d0 con 0x7fed9806efa0 2024-09-16T15:42:18.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.435+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed981f31b0 con 0x7fed9806efa0 2024-09-16T15:42:18.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.435+0000 7fed8f400640 1 -- 172.21.15.60:0/896359373 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88033520 con 0x7fed9806efa0 2024-09-16T15:42:18.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.435+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed980aa9e0 con 0x7fed9806efa0 2024-09-16T15:42:18.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.436+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed980cecd0 con 0x7fed9806efa0 2024-09-16T15:42:18.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.437+0000 7fed8f400640 1 -- 172.21.15.60:0/896359373 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 10) ==== 101491+0+0 (secure 0 0 0) 0x7fed880035f0 con 0x7fed9806efa0 2024-09-16T15:42:18.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.438+0000 7fed8f400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fed78078a90 0x7fed7807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:18.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.438+0000 7fed8f400640 1 -- 172.21.15.60:0/896359373 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fed880bc100 con 0x7fed9806efa0 2024-09-16T15:42:18.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.438+0000 7fed96a00640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fed78078a90 0x7fed7807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:18.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.439+0000 7fed96a00640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fed78078a90 0x7fed7807af50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fed84007fd0 tx=0x7fed84009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:18.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.442+0000 7fed8f400640 1 -- 172.21.15.60:0/896359373 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fed8803f030 con 0x7fed9806efa0 2024-09-16T15:42:18.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.670+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol75450", "target": ["mon-mgr", ""]}) -- 0x7fed98144c00 con 0x7fed78078a90 2024-09-16T15:42:18.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.701+0000 7fed8f400640 1 -- 172.21.15.60:0/896359373 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fed98144c00 con 0x7fed78078a90 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fed78078a90 msgr2=0x7fed7807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fed78078a90 0x7fed7807af50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fed84007fd0 tx=0x7fed84009040 comp rx=0 tx=0).stop 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed9806efa0 msgr2=0x7fed980ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed9806efa0 0x7fed980ae2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fed88000c00 tx=0x7fed88033d60 comp rx=0 tx=0).stop 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 shutdown_connections 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fed78078a90 0x7fed7807af50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed98145750 0x7fed9815e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed980cccd0 0x7fed9813c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 --2- 172.21.15.60:0/896359373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed9806efa0 0x7fed980ae2f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.703+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 >> 172.21.15.60:0/896359373 conn(0x7fed980b1840 msgr2=0x7fed980b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.704+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 shutdown_connections 2024-09-16T15:42:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.704+0000 7fed9e400640 1 -- 172.21.15.60:0/896359373 wait complete. 2024-09-16T15:42:18.707 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.706+0000 7fa50ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.707 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.706+0000 7fa50ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.707 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.706+0000 7fa50ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.707 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.706+0000 7fa50ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.707 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.706+0000 7fa50ca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol97901 2024-09-16T15:42:18.725 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.723+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.725 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.723+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.725 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.723+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.725 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.723+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.725 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:42:18.723+0000 7fa50e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.887+0000 7f9255a00640 1 -- 172.21.15.60:0/1171784028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9234004d40 msgr2=0x7f92340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.887+0000 7f9255a00640 1 --2- 172.21.15.60:0/1171784028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9234004d40 0x7f92340051c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9248009ed0 tx=0x7f9234005830 comp rx=0 tx=0).stop 2024-09-16T15:42:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.887+0000 7f924f400640 1 -- 172.21.15.60:0/1171784028 reap_dead start 2024-09-16T15:42:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.887+0000 7f9255a00640 1 -- 172.21.15.60:0/1171784028 shutdown_connections 2024-09-16T15:42:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.887+0000 7f9255a00640 1 -- 172.21.15.60:0/1171784028 >> 172.21.15.60:0/1171784028 conn(0x7f92500b1840 msgr2=0x7f92500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:18.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.887+0000 7f9255a00640 1 -- 172.21.15.60:0/1171784028 shutdown_connections 2024-09-16T15:42:18.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.887+0000 7f9255a00640 1 -- 172.21.15.60:0/1171784028 wait complete. 2024-09-16T15:42:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.888+0000 7f9255a00640 1 Processor -- start 2024-09-16T15:42:18.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.889+0000 7f9255a00640 1 -- start start 2024-09-16T15:42:18.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.889+0000 7f9255a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925013e4b0 0x7f92501e41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:18.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.889+0000 7f9255a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92501f5080 0x7f92501419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:18.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f9255a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f925013e890 0x7f925015e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:18.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f9255a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f92500737d0 con 0x7f925013e4b0 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f9255a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9250073650 con 0x7f92501f5080 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f9255a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f92500ce2c0 con 0x7f925013e890 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925013e4b0 0x7f92501e41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925013e4b0 0x7f92501e41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32806/0 (socket says 172.21.15.60:32806) 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 -- 172.21.15.60:0/1521009619 learned_addr learned my addr 172.21.15.60:0/1521009619 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f9246000640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92501f5080 0x7f92501419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924f400640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f925013e890 0x7f925015e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 -- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f925013e890 msgr2=0x7f925015e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f925013e890 0x7f925015e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 -- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92501f5080 msgr2=0x7f92501419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92501f5080 0x7f92501419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 -- 172.21.15.60:0/1521009619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92501f5580 con 0x7f925013e4b0 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.890+0000 7f924ea00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925013e4b0 0x7f92501e41c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9240006450 tx=0x7f92400039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.891+0000 7f9247400640 1 -- 172.21.15.60:0/1521009619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240015070 con 0x7f925013e4b0 2024-09-16T15:42:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.891+0000 7f9247400640 1 -- 172.21.15.60:0/1521009619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9240011900 con 0x7f925013e4b0 2024-09-16T15:42:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.891+0000 7f9247400640 1 -- 172.21.15.60:0/1521009619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240011d50 con 0x7f925013e4b0 2024-09-16T15:42:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.891+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92501f5860 con 0x7f925013e4b0 2024-09-16T15:42:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.891+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92501f5e20 con 0x7f925013e4b0 2024-09-16T15:42:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.892+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92500cdbb0 con 0x7f925013e4b0 2024-09-16T15:42:18.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.893+0000 7f9247400640 1 -- 172.21.15.60:0/1521009619 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 10) ==== 101491+0+0 (secure 0 0 0) 0x7f9240018070 con 0x7f925013e4b0 2024-09-16T15:42:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.894+0000 7f9247400640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f922c078a90 0x7f922c07af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.894+0000 7f9247400640 1 -- 172.21.15.60:0/1521009619 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f924009bbf0 con 0x7f925013e4b0 2024-09-16T15:42:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.895+0000 7f9246000640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f922c078a90 0x7f922c07af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:18.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.895+0000 7f9246000640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f922c078a90 0x7f922c07af50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9238000c00 tx=0x7f9238006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:18.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:18.899+0000 7f9247400640 1 -- 172.21.15.60:0/1521009619 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9240063d60 con 0x7f925013e4b0 2024-09-16T15:42:19.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:19.118+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol97901", "target": ["mon-mgr", ""]}) -- 0x7f925006ebe0 con 0x7f922c078a90 2024-09-16T15:42:20.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:20.769+0000 7f9247400640 1 -- 172.21.15.60:0/1521009619 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f924005f8d0 con 0x7f925013e4b0 2024-09-16T15:42:26.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.379+0000 7f9247400640 1 -- 172.21.15.60:0/1521009619 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f925006ebe0 con 0x7f922c078a90 2024-09-16T15:42:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f922c078a90 msgr2=0x7f922c07af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f922c078a90 0x7f922c07af50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9238000c00 tx=0x7f9238006b10 comp rx=0 tx=0).stop 2024-09-16T15:42:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925013e4b0 msgr2=0x7f92501e41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925013e4b0 0x7f92501e41c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9240006450 tx=0x7f92400039c0 comp rx=0 tx=0).stop 2024-09-16T15:42:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 shutdown_connections 2024-09-16T15:42:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f922c078a90 0x7f922c07af50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f925013e890 0x7f925015e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:26.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92501f5080 0x7f92501419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 --2- 172.21.15.60:0/1521009619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f925013e4b0 0x7f92501e41c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 >> 172.21.15.60:0/1521009619 conn(0x7f92500b1840 msgr2=0x7f92500ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.382+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 shutdown_connections 2024-09-16T15:42:26.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26.383+0000 7f9255a00640 1 -- 172.21.15.60:0/1521009619 wait complete. 2024-09-16T15:42:26.404 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:42:26.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:42:26.496 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-16T15:42:26.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:26.505 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:42:26.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:42:26.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:42:26.695 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:42:26.695 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:42:26.695 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:42:26.695 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:26.730 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:42:26.730 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:26.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:42:27.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.009+0000 7f9d60200640 1 -- 172.21.15.60:0/3983380574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58161550 msgr2=0x7f9d5806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.009+0000 7f9d60200640 1 --2- 172.21.15.60:0/3983380574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58161550 0x7f9d5806ef50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009830 tx=0x7f9d4c02a480 comp rx=0 tx=0).stop 2024-09-16T15:42:27.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.009+0000 7f9d60200640 1 -- 172.21.15.60:0/3983380574 shutdown_connections 2024-09-16T15:42:27.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.009+0000 7f9d60200640 1 --2- 172.21.15.60:0/3983380574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d58072df0 0x7f9d580732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.009+0000 7f9d60200640 1 --2- 172.21.15.60:0/3983380574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58161550 0x7f9d5806ef50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.009+0000 7f9d60200640 1 -- 172.21.15.60:0/3983380574 >> 172.21.15.60:0/3983380574 conn(0x7f9d580b1950 msgr2=0x7f9d580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:27.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.009+0000 7f9d60200640 1 -- 172.21.15.60:0/3983380574 shutdown_connections 2024-09-16T15:42:27.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.009+0000 7f9d60200640 1 -- 172.21.15.60:0/3983380574 wait complete. 2024-09-16T15:42:27.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.010+0000 7f9d60200640 1 Processor -- start 2024-09-16T15:42:27.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.011+0000 7f9d60200640 1 -- start start 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.011+0000 7f9d60200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d58072df0 0x7f9d5809ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.011+0000 7f9d60200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d580af3d0 0x7f9d581f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.011+0000 7f9d60200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58161550 0x7f9d580d0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.011+0000 7f9d60200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9d580737e0 con 0x7f9d58072df0 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.011+0000 7f9d60200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5806fdb0 con 0x7f9d58161550 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.011+0000 7f9d60200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9d58073960 con 0x7f9d580af3d0 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d58072df0 0x7f9d5809ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d58072df0 0x7f9d5809ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32814/0 (socket says 172.21.15.60:32814) 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5d800640 1 -- 172.21.15.60:0/4242230540 learned_addr learned my addr 172.21.15.60:0/4242230540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5ce00640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d580af3d0 0x7f9d581f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5e200640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58161550 0x7f9d580d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5d800640 1 -- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d580af3d0 msgr2=0x7f9d581f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5d800640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d580af3d0 0x7f9d581f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5d800640 1 -- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58161550 msgr2=0x7f9d580d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5d800640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58161550 0x7f9d580d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.012+0000 7f9d5d800640 1 -- 172.21.15.60:0/4242230540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5809d290 con 0x7f9d58072df0 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.013+0000 7f9d5d800640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d58072df0 0x7f9d5809ccb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c000c00 tx=0x7f9d4c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.013+0000 7f9d55600640 1 -- 172.21.15.60:0/4242230540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c005c90 con 0x7f9d58072df0 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.013+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5809d570 con 0x7f9d58072df0 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.013+0000 7f9d55600640 1 -- 172.21.15.60:0/4242230540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d4c031e50 con 0x7f9d58072df0 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.013+0000 7f9d55600640 1 -- 172.21.15.60:0/4242230540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0332f0 con 0x7f9d58072df0 2024-09-16T15:42:27.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.013+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5809db30 con 0x7f9d58072df0 2024-09-16T15:42:27.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.014+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5809dd70 con 0x7f9d58072df0 2024-09-16T15:42:27.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.015+0000 7f9d55600640 1 -- 172.21.15.60:0/4242230540 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f9d4c033490 con 0x7f9d58072df0 2024-09-16T15:42:27.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.016+0000 7f9d55600640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d38078bb0 0x7f9d3807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.016+0000 7f9d55600640 1 -- 172.21.15.60:0/4242230540 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f9d4c0bc0d0 con 0x7f9d58072df0 2024-09-16T15:42:27.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.016+0000 7f9d5ce00640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d38078bb0 0x7f9d3807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.017+0000 7f9d5ce00640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d38078bb0 0x7f9d3807b070 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d44007fd0 tx=0x7f9d44001410 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:27.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.020+0000 7f9d55600640 1 -- 172.21.15.60:0/4242230540 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d5809dd70 con 0x7f9d58072df0 2024-09-16T15:42:27.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.228+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9d581f4670 con 0x7f9d58072df0 2024-09-16T15:42:27.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.228+0000 7f9d55600640 1 -- 172.21.15.60:0/4242230540 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) ==== 80+0+0 (secure 0 0 0) 0x7f9d4c03f030 con 0x7f9d58072df0 2024-09-16T15:42:27.231 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:42:27.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d38078bb0 msgr2=0x7f9d3807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d38078bb0 0x7f9d3807b070 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d44007fd0 tx=0x7f9d44001410 comp rx=0 tx=0).stop 2024-09-16T15:42:27.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d58072df0 msgr2=0x7f9d5809ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d58072df0 0x7f9d5809ccb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c000c00 tx=0x7f9d4c033c20 comp rx=0 tx=0).stop 2024-09-16T15:42:27.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 shutdown_connections 2024-09-16T15:42:27.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9d38078bb0 0x7f9d3807b070 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d58161550 0x7f9d580d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d580af3d0 0x7f9d581f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 --2- 172.21.15.60:0/4242230540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d58072df0 0x7f9d5809ccb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 >> 172.21.15.60:0/4242230540 conn(0x7f9d580b1950 msgr2=0x7f9d580ad640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:27.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 shutdown_connections 2024-09-16T15:42:27.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.231+0000 7f9d60200640 1 -- 172.21.15.60:0/4242230540 wait complete. 2024-09-16T15:42:27.249 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:42:27.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:27.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:27.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:42:27.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:27.341 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:42:27.386 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:42:27.386 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:42:27.386 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:27.423 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:42:27.423 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:27.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:42:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.659+0000 7f83aca00640 1 -- 172.21.15.60:0/1012201808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 msgr2=0x7f83a80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.659+0000 7f83aca00640 1 --2- 172.21.15.60:0/1012201808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 0x7f83a80b2440 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f83a806dac0 tx=0x7f839802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.659+0000 7f83aca00640 1 -- 172.21.15.60:0/1012201808 shutdown_connections 2024-09-16T15:42:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.659+0000 7f83aca00640 1 --2- 172.21.15.60:0/1012201808 >> v2:172.21.15.100:3300/0 conn(0x7f83a8073fb0 0x7f83a8074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.659+0000 7f83aca00640 1 --2- 172.21.15.60:0/1012201808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 0x7f83a80b2440 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.659+0000 7f83aca00640 1 -- 172.21.15.60:0/1012201808 >> 172.21.15.60:0/1012201808 conn(0x7f83a80b1950 msgr2=0x7f83a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:27.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.659+0000 7f83aca00640 1 -- 172.21.15.60:0/1012201808 shutdown_connections 2024-09-16T15:42:27.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.659+0000 7f83aca00640 1 -- 172.21.15.60:0/1012201808 wait complete. 2024-09-16T15:42:27.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.661+0000 7f83aca00640 1 Processor -- start 2024-09-16T15:42:27.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.661+0000 7f83aca00640 1 -- start start 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.662+0000 7f83aca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a8073fb0 0x7f83a814ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.662+0000 7f83aca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a80ccf00 0x7f83a81f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.662+0000 7f83aca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 0x7f83a809cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.662+0000 7f83aca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f83a806f2e0 con 0x7f83a815f5a0 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.662+0000 7f83aca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83a806f160 con 0x7f83a8073fb0 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.662+0000 7f83aca00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f83a806f460 con 0x7f83a80ccf00 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 0x7f83a809cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 0x7f83a809cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32832/0 (socket says 172.21.15.60:32832) 2024-09-16T15:42:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 -- 172.21.15.60:0/3047270048 learned_addr learned my addr 172.21.15.60:0/3047270048 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a5e00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a8073fb0 0x7f83a814ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f839d400640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a80ccf00 0x7f83a81f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 -- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a80ccf00 msgr2=0x7f83a81f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a80ccf00 0x7f83a81f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 -- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a8073fb0 msgr2=0x7f83a814ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a8073fb0 0x7f83a814ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 -- 172.21.15.60:0/3047270048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a81f4d20 con 0x7f83a815f5a0 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a5e00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a8073fb0 0x7f83a814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.663+0000 7f83a6800640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 0x7f83a809cc80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f83a000eb10 tx=0x7f83a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.664+0000 7f839ea00640 1 -- 172.21.15.60:0/3047270048 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0004b20 con 0x7f83a815f5a0 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.664+0000 7f839ea00640 1 -- 172.21.15.60:0/3047270048 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83a0004ce0 con 0x7f83a815f5a0 2024-09-16T15:42:27.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.664+0000 7f839ea00640 1 -- 172.21.15.60:0/3047270048 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0005280 con 0x7f83a815f5a0 2024-09-16T15:42:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.664+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a81f8890 con 0x7f83a815f5a0 2024-09-16T15:42:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.664+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81f8e50 con 0x7f83a815f5a0 2024-09-16T15:42:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.665+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a806f9f0 con 0x7f83a815f5a0 2024-09-16T15:42:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.666+0000 7f839ea00640 1 -- 172.21.15.60:0/3047270048 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f83a0005420 con 0x7f83a815f5a0 2024-09-16T15:42:27.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.667+0000 7f839ea00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8384078bb0 0x7f838407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.667+0000 7f839ea00640 1 -- 172.21.15.60:0/3047270048 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f83a009bda0 con 0x7f83a815f5a0 2024-09-16T15:42:27.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.667+0000 7f83a5e00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8384078bb0 0x7f838407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.668+0000 7f83a5e00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8384078bb0 0x7f838407b070 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8398034000 tx=0x7f8398004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:27.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.671+0000 7f839ea00640 1 -- 172.21.15.60:0/3047270048 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f83a0063e40 con 0x7f83a815f5a0 2024-09-16T15:42:27.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.883+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f83a80b3000 con 0x7f83a815f5a0 2024-09-16T15:42:27.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.883+0000 7f839ea00640 1 -- 172.21.15.60:0/3047270048 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) ==== 80+0+0 (secure 0 0 0) 0x7f83a0063740 con 0x7f83a815f5a0 2024-09-16T15:42:27.885 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:42:27.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.885+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8384078bb0 msgr2=0x7f838407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.885+0000 7f83aca00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8384078bb0 0x7f838407b070 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8398034000 tx=0x7f8398004730 comp rx=0 tx=0).stop 2024-09-16T15:42:27.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.885+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 msgr2=0x7f83a809cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.885+0000 7f83aca00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 0x7f83a809cc80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f83a000eb10 tx=0x7f83a000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.886+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 shutdown_connections 2024-09-16T15:42:27.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.886+0000 7f83aca00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8384078bb0 0x7f838407b070 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.886+0000 7f83aca00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a815f5a0 0x7f83a809cc80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.886+0000 7f83aca00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a80ccf00 0x7f83a81f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.886+0000 7f83aca00640 1 --2- 172.21.15.60:0/3047270048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a8073fb0 0x7f83a814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.886+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 >> 172.21.15.60:0/3047270048 conn(0x7f83a80b1950 msgr2=0x7f83a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:27.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.886+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 shutdown_connections 2024-09-16T15:42:27.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:27.886+0000 7f83aca00640 1 -- 172.21.15.60:0/3047270048 wait complete. 2024-09-16T15:42:27.904 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:42:27.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:27.904 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:27.968 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:42:27.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:27.968 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:42:28.004 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:42:28.004 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:42:28.004 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:28.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:42:28.040 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:42:28.040 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:42:28.040 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:28.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:42:28.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 -- 172.21.15.60:0/2494283516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ccf00 msgr2=0x7f72900acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 --2- 172.21.15.60:0/2494283516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ccf00 0x7f72900acd30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7290076720 tx=0x7f727c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 -- 172.21.15.60:0/2494283516 shutdown_connections 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 --2- 172.21.15.60:0/2494283516 >> v2:172.21.15.100:3300/0 conn(0x7f72900ad270 0x7f7290073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 --2- 172.21.15.60:0/2494283516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ccf00 0x7f72900acd30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 --2- 172.21.15.60:0/2494283516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f729015f5a0 0x7f72900b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 -- 172.21.15.60:0/2494283516 >> 172.21.15.60:0/2494283516 conn(0x7f72900b1950 msgr2=0x7f72900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 -- 172.21.15.60:0/2494283516 shutdown_connections 2024-09-16T15:42:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.239+0000 7f7295200640 1 -- 172.21.15.60:0/2494283516 wait complete. 2024-09-16T15:42:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.241+0000 7f7295200640 1 Processor -- start 2024-09-16T15:42:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.241+0000 7f7295200640 1 -- start start 2024-09-16T15:42:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f7295200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ad270 0x7f7290156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f7295200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72900ccf00 0x7f729015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f7295200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f729015f5a0 0x7f729015a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f7295200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f729006f160 con 0x7f72900ad270 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f7295200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7290074780 con 0x7f72900ccf00 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f7295200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f729006f2e0 con 0x7f729015f5a0 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ad270 0x7f7290156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ad270 0x7f7290156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32844/0 (socket says 172.21.15.60:32844) 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728de00640 1 -- 172.21.15.60:0/2666594171 learned_addr learned my addr 172.21.15.60:0/2666594171 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728e800640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f729015f5a0 0x7f729015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728d400640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72900ccf00 0x7f729015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728de00640 1 -- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f729015f5a0 msgr2=0x7f729015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728de00640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f729015f5a0 0x7f729015a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728de00640 1 -- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72900ccf00 msgr2=0x7f729015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728de00640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72900ccf00 0x7f729015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.242+0000 7f728de00640 1 -- 172.21.15.60:0/2666594171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7290157380 con 0x7f72900ad270 2024-09-16T15:42:28.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.243+0000 7f728de00640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ad270 0x7f7290156e40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f728000c970 tx=0x7f728000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.243+0000 7f7286000640 1 -- 172.21.15.60:0/2666594171 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280007bb0 con 0x7f72900ad270 2024-09-16T15:42:28.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.243+0000 7f7286000640 1 -- 172.21.15.60:0/2666594171 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7280007e40 con 0x7f72900ad270 2024-09-16T15:42:28.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.243+0000 7f7286000640 1 -- 172.21.15.60:0/2666594171 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280005280 con 0x7f72900ad270 2024-09-16T15:42:28.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.243+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72900b13b0 con 0x7f72900ad270 2024-09-16T15:42:28.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.243+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72901591b0 con 0x7f72900ad270 2024-09-16T15:42:28.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.244+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729006f870 con 0x7f72900ad270 2024-09-16T15:42:28.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.245+0000 7f7286000640 1 -- 172.21.15.60:0/2666594171 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f7280005420 con 0x7f72900ad270 2024-09-16T15:42:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.246+0000 7f7286000640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7270078be0 0x7f727007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.246+0000 7f7286000640 1 -- 172.21.15.60:0/2666594171 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f728009b420 con 0x7f72900ad270 2024-09-16T15:42:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.247+0000 7f728d400640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7270078be0 0x7f727007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.247+0000 7f728d400640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7270078be0 0x7f727007b0a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f727c033000 tx=0x7f727c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.250+0000 7f7286000640 1 -- 172.21.15.60:0/2666594171 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7290157380 con 0x7f72900ad270 2024-09-16T15:42:28.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.484+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72900ceb30 con 0x7f72900ad270 2024-09-16T15:42:28.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.484+0000 7f7286000640 1 -- 172.21.15.60:0/2666594171 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7f7280014030 con 0x7f72900ad270 2024-09-16T15:42:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:28.487 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"btime":"2024-09-16T15:42:11:718145+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5671,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3625257481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3625257481},{"type":"v1","addr":"172.21.15.60:6834","nonce":3625257481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2519251073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2519251073},{"type":"v1","addr":"172.21.15.100:6837","nonce":2519251073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/333738926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":333738926},{"type":"v1","addr":"172.21.15.60:6835","nonce":333738926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:01.154412+0000","modified":"2024-09-16T15:42:04.123997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2580251790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2580251790},{"type":"v1","addr":"172.21.15.100:6836","nonce":2580251790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5757,"qdb_cluster":[5757]},"id":4}]} 2024-09-16T15:42:28.487 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 37 2024-09-16T15:42:28.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7270078be0 msgr2=0x7f727007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7270078be0 0x7f727007b0a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f727c033000 tx=0x7f727c004730 comp rx=0 tx=0).stop 2024-09-16T15:42:28.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ad270 msgr2=0x7f7290156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ad270 0x7f7290156e40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f728000c970 tx=0x7f728000ce30 comp rx=0 tx=0).stop 2024-09-16T15:42:28.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 shutdown_connections 2024-09-16T15:42:28.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7270078be0 0x7f727007b0a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f729015f5a0 0x7f729015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72900ccf00 0x7f729015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 --2- 172.21.15.60:0/2666594171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72900ad270 0x7f7290156e40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 >> 172.21.15.60:0/2666594171 conn(0x7f72900b1950 msgr2=0x7f729009d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 shutdown_connections 2024-09-16T15:42:28.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.487+0000 7f7295200640 1 -- 172.21.15.60:0/2666594171 wait complete. 2024-09-16T15:42:28.504 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.664+0000 7fdbe1600640 1 -- 172.21.15.60:0/366040168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc161440 msgr2=0x7fdbdc0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.664+0000 7fdbe1600640 1 --2- 172.21.15.60:0/366040168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc161440 0x7fdbdc0b5370 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0d4ea0 tx=0x7fdbcc02a470 comp rx=0 tx=0).stop 2024-09-16T15:42:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.664+0000 7fdbe1600640 1 -- 172.21.15.60:0/366040168 shutdown_connections 2024-09-16T15:42:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.664+0000 7fdbe1600640 1 --2- 172.21.15.60:0/366040168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbdc0b34b0 0x7fdbdc0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.664+0000 7fdbe1600640 1 --2- 172.21.15.60:0/366040168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc161440 0x7fdbdc0b5370 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.664+0000 7fdbe1600640 1 -- 172.21.15.60:0/366040168 >> 172.21.15.60:0/366040168 conn(0x7fdbdc0d3df0 msgr2=0x7fdbdc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.664+0000 7fdbe1600640 1 -- 172.21.15.60:0/366040168 shutdown_connections 2024-09-16T15:42:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.664+0000 7fdbe1600640 1 -- 172.21.15.60:0/366040168 wait complete. 2024-09-16T15:42:28.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.665+0000 7fdbe1600640 1 Processor -- start 2024-09-16T15:42:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.666+0000 7fdbe1600640 1 -- start start 2024-09-16T15:42:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.666+0000 7fdbe1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc0b34b0 0x7fdbdc156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.666+0000 7fdbe1600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbdc0b3e50 0x7fdbdc14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbe1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbdc161440 0x7fdbdc0b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbe1600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0b2900 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbe1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc0b2780 con 0x7fdbdc161440 2024-09-16T15:42:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbe1600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc0b2a80 con 0x7fdbdc0b3e50 2024-09-16T15:42:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc0b34b0 0x7fdbdc156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc0b34b0 0x7fdbdc156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:32860/0 (socket says 172.21.15.60:32860) 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdbe00640 1 -- 172.21.15.60:0/2353078901 learned_addr learned my addr 172.21.15.60:0/2353078901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdb400640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbdc0b3e50 0x7fdbdc14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbe0c00640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbdc161440 0x7fdbdc0b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdbe00640 1 -- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbdc0b3e50 msgr2=0x7fdbdc14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdbe00640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbdc0b3e50 0x7fdbdc14e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdbe00640 1 -- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbdc161440 msgr2=0x7fdbdc0b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdbe00640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbdc161440 0x7fdbdc0b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdbe00640 1 -- 172.21.15.60:0/2353078901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc0cd1c0 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbdb400640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbdc0b3e50 0x7fdbdc14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.667+0000 7fdbe0c00640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbdc161440 0x7fdbdc0b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.668+0000 7fdbdbe00640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc0b34b0 0x7fdbdc156250 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009780 tx=0x7fdbcc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.668+0000 7fdbd3e00640 1 -- 172.21.15.60:0/2353078901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc031d60 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.668+0000 7fdbd3e00640 1 -- 172.21.15.60:0/2353078901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbcc033e10 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.668+0000 7fdbd3e00640 1 -- 172.21.15.60:0/2353078901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc005260 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.668+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc0cd4a0 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.668+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc0c7ac0 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.669+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc0b3910 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.670+0000 7fdbd3e00640 1 -- 172.21.15.60:0/2353078901 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fdbcc005400 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.671+0000 7fdbd3e00640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbbc078be0 0x7fdbbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.671+0000 7fdbd3e00640 1 -- 172.21.15.60:0/2353078901 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fdbcc0bc1f0 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.671+0000 7fdbdb400640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbbc078be0 0x7fdbbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.672+0000 7fdbdb400640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbbc078be0 0x7fdbbc07b0a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b98a0 tx=0x7fdbc8005e70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.675+0000 7fdbd3e00640 1 -- 172.21.15.60:0/2353078901 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdbcc03f030 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.915+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbdc0d46d0 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.916+0000 7fdbd3e00640 1 -- 172.21.15.60:0/2353078901 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7fdbcc084200 con 0x7fdbdc0b34b0 2024-09-16T15:42:28.918 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:28.918 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"btime":"2024-09-16T15:42:11:718145+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5671,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3625257481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3625257481},{"type":"v1","addr":"172.21.15.60:6834","nonce":3625257481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2519251073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2519251073},{"type":"v1","addr":"172.21.15.100:6837","nonce":2519251073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/333738926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":333738926},{"type":"v1","addr":"172.21.15.60:6835","nonce":333738926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:01.154412+0000","modified":"2024-09-16T15:42:04.123997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2580251790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2580251790},{"type":"v1","addr":"172.21.15.100:6836","nonce":2580251790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5757,"qdb_cluster":[5757]},"id":4}]} 2024-09-16T15:42:28.919 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 37 2024-09-16T15:42:28.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbbc078be0 msgr2=0x7fdbbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbbc078be0 0x7fdbbc07b0a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b98a0 tx=0x7fdbc8005e70 comp rx=0 tx=0).stop 2024-09-16T15:42:28.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc0b34b0 msgr2=0x7fdbdc156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc0b34b0 0x7fdbdc156250 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009780 tx=0x7fdbcc033c20 comp rx=0 tx=0).stop 2024-09-16T15:42:28.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 shutdown_connections 2024-09-16T15:42:28.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fdbbc078be0 0x7fdbbc07b0a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbdc161440 0x7fdbdc0b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbdc0b3e50 0x7fdbdc14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 --2- 172.21.15.60:0/2353078901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbdc0b34b0 0x7fdbdc156250 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 >> 172.21.15.60:0/2353078901 conn(0x7fdbdc0d3df0 msgr2=0x7fdbdc15e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 shutdown_connections 2024-09-16T15:42:28.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:28.919+0000 7fdbe1600640 1 -- 172.21.15.60:0/2353078901 wait complete. 2024-09-16T15:42:28.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:29.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.096+0000 7f2222200640 1 -- 172.21.15.60:0/1295510471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c15bc30 msgr2=0x7f221c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.096+0000 7f2222200640 1 --2- 172.21.15.60:0/1295510471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c15bc30 0x7f221c0b28f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2210005fd0 tx=0x7f221002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.096+0000 7f2222200640 1 -- 172.21.15.60:0/1295510471 shutdown_connections 2024-09-16T15:42:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.096+0000 7f2222200640 1 --2- 172.21.15.60:0/1295510471 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221c0d0540 0x7f221c06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.096+0000 7f2222200640 1 --2- 172.21.15.60:0/1295510471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c15bc30 0x7f221c0b28f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.096+0000 7f2222200640 1 -- 172.21.15.60:0/1295510471 >> 172.21.15.60:0/1295510471 conn(0x7f221c0cfa20 msgr2=0x7f221c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.096+0000 7f2222200640 1 -- 172.21.15.60:0/1295510471 shutdown_connections 2024-09-16T15:42:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.096+0000 7f2222200640 1 -- 172.21.15.60:0/1295510471 wait complete. 2024-09-16T15:42:29.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.098+0000 7f2222200640 1 Processor -- start 2024-09-16T15:42:29.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.098+0000 7f2222200640 1 -- start start 2024-09-16T15:42:29.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f2222200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0d0540 0x7f221c15e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f2222200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c15bc30 0x7f221c157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f2222200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221c1516e0 0x7f221c151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f2222200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f221c074660 con 0x7f221c15bc30 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f2222200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f221c06f860 con 0x7f221c0d0540 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f2222200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f221c1520a0 con 0x7f221c1516e0 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f221be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c15bc30 0x7f221c157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f221be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c15bc30 0x7f221c157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35444/0 (socket says 172.21.15.60:35444) 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f221be00640 1 -- 172.21.15.60:0/51808184 learned_addr learned my addr 172.21.15.60:0/51808184 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.099+0000 7f2220e00640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0d0540 0x7f221c15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f2221800640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221c1516e0 0x7f221c151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f221be00640 1 -- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221c1516e0 msgr2=0x7f221c151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f221be00640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221c1516e0 0x7f221c151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f221be00640 1 -- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0d0540 msgr2=0x7f221c15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f221be00640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0d0540 0x7f221c15e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f221be00640 1 -- 172.21.15.60:0/51808184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c1f4ae0 con 0x7f221c15bc30 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f221be00640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c15bc30 0x7f221c157db0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f220800c420 tx=0x7f220800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f2218c00640 1 -- 172.21.15.60:0/51808184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208017070 con 0x7f221c15bc30 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f2218c00640 1 -- 172.21.15.60:0/51808184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22080053c0 con 0x7f221c15bc30 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f2218c00640 1 -- 172.21.15.60:0/51808184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208003850 con 0x7f221c15bc30 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.100+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c1f4da0 con 0x7f221c15bc30 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.101+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c1f5360 con 0x7f221c15bc30 2024-09-16T15:42:29.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.102+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221c06fdf0 con 0x7f221c15bc30 2024-09-16T15:42:29.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.103+0000 7f2218c00640 1 -- 172.21.15.60:0/51808184 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f220801a070 con 0x7f221c15bc30 2024-09-16T15:42:29.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.104+0000 7f2218c00640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f21fc078bb0 0x7f21fc07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.104+0000 7f2218c00640 1 -- 172.21.15.60:0/51808184 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f220809a290 con 0x7f221c15bc30 2024-09-16T15:42:29.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.104+0000 7f2220e00640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f21fc078bb0 0x7f21fc07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.105+0000 7f2220e00640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f21fc078bb0 0x7f21fc07b070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2210033000 tx=0x7f2210004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:29.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.108+0000 7f2218c00640 1 -- 172.21.15.60:0/51808184 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2208061d50 con 0x7f221c15bc30 2024-09-16T15:42:29.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.305+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f221c06aaf0 con 0x7f221c15bc30 2024-09-16T15:42:29.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.306+0000 7f2218c00640 1 -- 172.21.15.60:0/51808184 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16725 (secure 0 0 0) 0x7f2208066780 con 0x7f221c15bc30 2024-09-16T15:42:29.313 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:29.313 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:02.674798+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:41:58.905230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:00.065450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f21fc078bb0 msgr2=0x7f21fc07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f21fc078bb0 0x7f21fc07b070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2210033000 tx=0x7f2210004730 comp rx=0 tx=0).stop 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c15bc30 msgr2=0x7f221c157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c15bc30 0x7f221c157db0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f220800c420 tx=0x7f220800c8e0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 shutdown_connections 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f21fc078bb0 0x7f21fc07b070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221c1516e0 0x7f221c151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c15bc30 0x7f221c157db0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 --2- 172.21.15.60:0/51808184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c0d0540 0x7f221c15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 >> 172.21.15.60:0/51808184 conn(0x7f221c0cfa20 msgr2=0x7f221c159100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 shutdown_connections 2024-09-16T15:42:29.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.309+0000 7f2222200640 1 -- 172.21.15.60:0/51808184 wait complete. 2024-09-16T15:42:29.327 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:42:29.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.486+0000 7f76c3200640 1 -- 172.21.15.60:0/2626945965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40ce100 msgr2=0x7f76c40cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.486+0000 7f76c3200640 1 --2- 172.21.15.60:0/2626945965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40ce100 0x7f76c40cc860 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f76b4005fd0 tx=0x7f76b402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.486+0000 7f76c3200640 1 -- 172.21.15.60:0/2626945965 shutdown_connections 2024-09-16T15:42:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.486+0000 7f76c3200640 1 --2- 172.21.15.60:0/2626945965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c40afde0 0x7f76c40b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.486+0000 7f76c3200640 1 --2- 172.21.15.60:0/2626945965 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40ce100 0x7f76c40cc860 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.486+0000 7f76c3200640 1 -- 172.21.15.60:0/2626945965 >> 172.21.15.60:0/2626945965 conn(0x7f76c40b1e90 msgr2=0x7f76c40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:29.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.486+0000 7f76c3200640 1 -- 172.21.15.60:0/2626945965 shutdown_connections 2024-09-16T15:42:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.486+0000 7f76c3200640 1 -- 172.21.15.60:0/2626945965 wait complete. 2024-09-16T15:42:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.487+0000 7f76c3200640 1 Processor -- start 2024-09-16T15:42:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.488+0000 7f76c3200640 1 -- start start 2024-09-16T15:42:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.488+0000 7f76c3200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c4074ca0 0x7f76c41558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.488+0000 7f76c3200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40afde0 0x7f76c415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c40ce100 0x7f76c4099720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c3200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f76c40b0d30 con 0x7f76c40afde0 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76c40b0bb0 con 0x7f76c40ce100 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c3200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f76c40b0eb0 con 0x7f76c4074ca0 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40afde0 0x7f76c415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40afde0 0x7f76c415a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35462/0 (socket says 172.21.15.60:35462) 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1400640 1 -- 172.21.15.60:0/2685098012 learned_addr learned my addr 172.21.15.60:0/2685098012 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1e00640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c4074ca0 0x7f76c41558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c2800640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c40ce100 0x7f76c4099720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1400640 1 -- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c4074ca0 msgr2=0x7f76c41558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1400640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c4074ca0 0x7f76c41558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1400640 1 -- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c40ce100 msgr2=0x7f76c4099720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1400640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c40ce100 0x7f76c4099720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.489+0000 7f76c1400640 1 -- 172.21.15.60:0/2685098012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c4155ea0 con 0x7f76c40afde0 2024-09-16T15:42:29.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.490+0000 7f76c1400640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40afde0 0x7f76c415a590 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f76b000c3a0 tx=0x7f76b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:29.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.490+0000 7f76ba000640 1 -- 172.21.15.60:0/2685098012 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0017070 con 0x7f76c40afde0 2024-09-16T15:42:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.490+0000 7f76ba000640 1 -- 172.21.15.60:0/2685098012 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76b00053c0 con 0x7f76c40afde0 2024-09-16T15:42:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.490+0000 7f76ba000640 1 -- 172.21.15.60:0/2685098012 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0003850 con 0x7f76c40afde0 2024-09-16T15:42:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.490+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c4156180 con 0x7f76c40afde0 2024-09-16T15:42:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.490+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c40c52b0 con 0x7f76c40afde0 2024-09-16T15:42:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.491+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c40b0270 con 0x7f76c40afde0 2024-09-16T15:42:29.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.492+0000 7f76ba000640 1 -- 172.21.15.60:0/2685098012 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f76b001a070 con 0x7f76c40afde0 2024-09-16T15:42:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.493+0000 7f76ba000640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f76a4078bb0 0x7f76a407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.493+0000 7f76ba000640 1 -- 172.21.15.60:0/2685098012 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f76b009a1f0 con 0x7f76c40afde0 2024-09-16T15:42:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.494+0000 7f76c1e00640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f76a4078bb0 0x7f76a407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.494+0000 7f76c1e00640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f76a4078bb0 0x7f76a407b070 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f76b4033000 tx=0x7f76b4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:29.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.498+0000 7f76ba000640 1 -- 172.21.15.60:0/2685098012 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f76b0061d40 con 0x7f76c40afde0 2024-09-16T15:42:29.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.733+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76c40c5480 con 0x7f76c40afde0 2024-09-16T15:42:29.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.733+0000 7f76ba000640 1 -- 172.21.15.60:0/2685098012 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7f76b0066770 con 0x7f76c40afde0 2024-09-16T15:42:29.888 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:29.888 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"btime":"2024-09-16T15:42:11:718145+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5671,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3625257481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3625257481},{"type":"v1","addr":"172.21.15.60:6834","nonce":3625257481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2519251073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2519251073},{"type":"v1","addr":"172.21.15.100:6837","nonce":2519251073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/333738926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":333738926},{"type":"v1","addr":"172.21.15.60:6835","nonce":333738926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:01.154412+0000","modified":"2024-09-16T15:42:04.123997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2580251790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2580251790},{"type":"v1","addr":"172.21.15.100:6836","nonce":2580251790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5757,"qdb_cluster":[5757]},"id":4}]} 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 37 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f76a4078bb0 msgr2=0x7f76a407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f76a4078bb0 0x7f76a407b070 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f76b4033000 tx=0x7f76b4004730 comp rx=0 tx=0).stop 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40afde0 msgr2=0x7f76c415a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40afde0 0x7f76c415a590 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f76b000c3a0 tx=0x7f76b000c860 comp rx=0 tx=0).stop 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 shutdown_connections 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f76a4078bb0 0x7f76a407b070 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76c40ce100 0x7f76c4099720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76c40afde0 0x7f76c415a590 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 --2- 172.21.15.60:0/2685098012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76c4074ca0 0x7f76c41558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 >> 172.21.15.60:0/2685098012 conn(0x7f76c40b1e90 msgr2=0x7f76c40cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.735+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 shutdown_connections 2024-09-16T15:42:29.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:29.736+0000 7f76c3200640 1 -- 172.21.15.60:0/2685098012 wait complete. 2024-09-16T15:42:29.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.051+0000 7fcd96000640 1 -- 172.21.15.60:0/2207277430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd74004800 msgr2=0x7fcd74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.051+0000 7fcd96000640 1 --2- 172.21.15.60:0/2207277430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd74004800 0x7fcd74004c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcd780016c0 tx=0x7fcd74005830 comp rx=0 tx=0).stop 2024-09-16T15:42:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.052+0000 7fcd96000640 1 -- 172.21.15.60:0/2207277430 shutdown_connections 2024-09-16T15:42:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.052+0000 7fcd96000640 1 --2- 172.21.15.60:0/2207277430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd74004800 0x7fcd74004c80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.052+0000 7fcd96000640 1 -- 172.21.15.60:0/2207277430 >> 172.21.15.60:0/2207277430 conn(0x7fcd900b1950 msgr2=0x7fcd900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.052+0000 7fcd96000640 1 -- 172.21.15.60:0/2207277430 shutdown_connections 2024-09-16T15:42:30.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.052+0000 7fcd96000640 1 -- 172.21.15.60:0/2207277430 wait complete. 2024-09-16T15:42:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.053+0000 7fcd96000640 1 Processor -- start 2024-09-16T15:42:30.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.053+0000 7fcd96000640 1 -- start start 2024-09-16T15:42:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd96000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd74004800 0x7fcd901546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd96000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9009bdb0 0x7fcd901f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd96000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900cdae0 0x7fcd900cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd96000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9006f0d0 con 0x7fcd9009bdb0 2024-09-16T15:42:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd96000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9006ef50 con 0x7fcd900cdae0 2024-09-16T15:42:30.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd96000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9006f250 con 0x7fcd74004800 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd86a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9009bdb0 0x7fcd901f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd86a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9009bdb0 0x7fcd901f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35468/0 (socket says 172.21.15.60:35468) 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.054+0000 7fcd86a00640 1 -- 172.21.15.60:0/863629747 learned_addr learned my addr 172.21.15.60:0/863629747 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd8f400640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd74004800 0x7fcd901546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd8fe00640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900cdae0 0x7fcd900cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd86a00640 1 -- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd74004800 msgr2=0x7fcd901546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd86a00640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd74004800 0x7fcd901546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd86a00640 1 -- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900cdae0 msgr2=0x7fcd900cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd86a00640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900cdae0 0x7fcd900cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd86a00640 1 -- 172.21.15.60:0/863629747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd900d0a20 con 0x7fcd9009bdb0 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd8fe00640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900cdae0 0x7fcd900cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:30.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.055+0000 7fcd86a00640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9009bdb0 0x7fcd901f4820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd7802a8e0 tx=0x7fcd78006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.056+0000 7fcd87e00640 1 -- 172.21.15.60:0/863629747 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78003a00 con 0x7fcd9009bdb0 2024-09-16T15:42:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.056+0000 7fcd87e00640 1 -- 172.21.15.60:0/863629747 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd78039070 con 0x7fcd9009bdb0 2024-09-16T15:42:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.056+0000 7fcd87e00640 1 -- 172.21.15.60:0/863629747 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78033420 con 0x7fcd9009bdb0 2024-09-16T15:42:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.056+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd901f4f90 con 0x7fcd9009bdb0 2024-09-16T15:42:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.056+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd901f5550 con 0x7fcd9009bdb0 2024-09-16T15:42:30.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.057+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9006f7b0 con 0x7fcd9009bdb0 2024-09-16T15:42:30.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.058+0000 7fcd87e00640 1 -- 172.21.15.60:0/863629747 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fcd78044020 con 0x7fcd9009bdb0 2024-09-16T15:42:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.059+0000 7fcd87e00640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcd6c078bb0 0x7fcd6c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.059+0000 7fcd87e00640 1 -- 172.21.15.60:0/863629747 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fcd780ba6a0 con 0x7fcd9009bdb0 2024-09-16T15:42:30.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.059+0000 7fcd8f400640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcd6c078bb0 0x7fcd6c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.060+0000 7fcd8f400640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcd6c078bb0 0x7fcd6c07b070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcd9006bf20 tx=0x7fcd80006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.063+0000 7fcd87e00640 1 -- 172.21.15.60:0/863629747 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcd900d0a20 con 0x7fcd9009bdb0 2024-09-16T15:42:30.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.295+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd9006e820 con 0x7fcd9009bdb0 2024-09-16T15:42:30.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.296+0000 7fcd87e00640 1 -- 172.21.15.60:0/863629747 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7fcd780827b0 con 0x7fcd9009bdb0 2024-09-16T15:42:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:30.298 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":37,"btime":"2024-09-16T15:42:11:718145+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5671,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3625257481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3625257481},{"type":"v1","addr":"172.21.15.60:6834","nonce":3625257481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2519251073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2519251073},{"type":"v1","addr":"172.21.15.100:6837","nonce":2519251073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/333738926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":333738926},{"type":"v1","addr":"172.21.15.60:6835","nonce":333738926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:01.154412+0000","modified":"2024-09-16T15:42:04.123997+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"b","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/2580251790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2580251790},{"type":"v1","addr":"172.21.15.100:6836","nonce":2580251790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5757,"qdb_cluster":[5757]},"id":4}]} 2024-09-16T15:42:30.298 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 37 2024-09-16T15:42:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.298+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcd6c078bb0 msgr2=0x7fcd6c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.298+0000 7fcd96000640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcd6c078bb0 0x7fcd6c07b070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcd9006bf20 tx=0x7fcd80006720 comp rx=0 tx=0).stop 2024-09-16T15:42:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.298+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9009bdb0 msgr2=0x7fcd901f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.298+0000 7fcd96000640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9009bdb0 0x7fcd901f4820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd7802a8e0 tx=0x7fcd78006430 comp rx=0 tx=0).stop 2024-09-16T15:42:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.299+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 shutdown_connections 2024-09-16T15:42:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.299+0000 7fcd96000640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fcd6c078bb0 0x7fcd6c07b070 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.299+0000 7fcd96000640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900cdae0 0x7fcd900cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.299+0000 7fcd96000640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9009bdb0 0x7fcd901f4820 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.299+0000 7fcd96000640 1 --2- 172.21.15.60:0/863629747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd74004800 0x7fcd901546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.299+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 >> 172.21.15.60:0/863629747 conn(0x7fcd900b1950 msgr2=0x7fcd9015ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.299+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 shutdown_connections 2024-09-16T15:42:30.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.299+0000 7fcd96000640 1 -- 172.21.15.60:0/863629747 wait complete. 2024-09-16T15:42:30.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.475+0000 7fa347200640 1 -- 172.21.15.60:0/3804856055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa340161440 msgr2=0x7fa3400b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.475+0000 7fa347200640 1 --2- 172.21.15.60:0/3804856055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa340161440 0x7fa3400b5370 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa3400c7470 tx=0x7fa33402a4b0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.475+0000 7fa347200640 1 -- 172.21.15.60:0/3804856055 shutdown_connections 2024-09-16T15:42:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.475+0000 7fa347200640 1 --2- 172.21.15.60:0/3804856055 >> v2:172.21.15.100:3300/0 conn(0x7fa3400b34b0 0x7fa3400b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.475+0000 7fa347200640 1 --2- 172.21.15.60:0/3804856055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa340161440 0x7fa3400b5370 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.475+0000 7fa347200640 1 -- 172.21.15.60:0/3804856055 >> 172.21.15.60:0/3804856055 conn(0x7fa3400d3df0 msgr2=0x7fa3400d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.475+0000 7fa347200640 1 -- 172.21.15.60:0/3804856055 shutdown_connections 2024-09-16T15:42:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.475+0000 7fa347200640 1 -- 172.21.15.60:0/3804856055 wait complete. 2024-09-16T15:42:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.476+0000 7fa347200640 1 Processor -- start 2024-09-16T15:42:30.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.477+0000 7fa347200640 1 -- start start 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.477+0000 7fa347200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3400b34b0 0x7fa3401f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.477+0000 7fa347200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3400b3e80 0x7fa3401f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.477+0000 7fa347200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa340161440 0x7fa3400b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.477+0000 7fa347200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa3400b2900 con 0x7fa3400b3e80 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.477+0000 7fa347200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3400b2780 con 0x7fa3400b34b0 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.477+0000 7fa347200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa3400b2a80 con 0x7fa340161440 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3400b3e80 0x7fa3401f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3400b3e80 0x7fa3401f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35488/0 (socket says 172.21.15.60:35488) 2024-09-16T15:42:30.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345400640 1 -- 172.21.15.60:0/117597946 learned_addr learned my addr 172.21.15.60:0/117597946 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345e00640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3400b34b0 0x7fa3401f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa346800640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa340161440 0x7fa3400b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345400640 1 -- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa340161440 msgr2=0x7fa3400b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345400640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa340161440 0x7fa3400b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345400640 1 -- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3400b34b0 msgr2=0x7fa3401f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345400640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3400b34b0 0x7fa3401f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa345400640 1 -- 172.21.15.60:0/117597946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3401f5af0 con 0x7fa3400b3e80 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.478+0000 7fa346800640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa340161440 0x7fa3400b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.479+0000 7fa345400640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3400b3e80 0x7fa3401f1fc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c920 tx=0x7fa33000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.479+0000 7fa33a000640 1 -- 172.21.15.60:0/117597946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330007c70 con 0x7fa3400b3e80 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.479+0000 7fa33a000640 1 -- 172.21.15.60:0/117597946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa33000f450 con 0x7fa3400b3e80 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.479+0000 7fa33a000640 1 -- 172.21.15.60:0/117597946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33000f8a0 con 0x7fa3400b3e80 2024-09-16T15:42:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.479+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3400caf60 con 0x7fa3400b3e80 2024-09-16T15:42:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.479+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3400cb520 con 0x7fa3400b3e80 2024-09-16T15:42:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.480+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3400b3940 con 0x7fa3400b3e80 2024-09-16T15:42:30.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.481+0000 7fa33a000640 1 -- 172.21.15.60:0/117597946 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fa33000fa40 con 0x7fa3400b3e80 2024-09-16T15:42:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.483+0000 7fa33a000640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa324078bb0 0x7fa32407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.483+0000 7fa33a000640 1 -- 172.21.15.60:0/117597946 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fa33009b780 con 0x7fa3400b3e80 2024-09-16T15:42:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.483+0000 7fa345e00640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa324078bb0 0x7fa32407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.484+0000 7fa345e00640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa324078bb0 0x7fa32407b070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa334034000 tx=0x7fa334004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.487+0000 7fa33a000640 1 -- 172.21.15.60:0/117597946 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa330014030 con 0x7fa3400b3e80 2024-09-16T15:42:30.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.683+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3400d4380 con 0x7fa3400b3e80 2024-09-16T15:42:30.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.684+0000 7fa33a000640 1 -- 172.21.15.60:0/117597946 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16725 (secure 0 0 0) 0x7fa330068e20 con 0x7fa3400b3e80 2024-09-16T15:42:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:30.687 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:02.674798+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:41:58.905230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:00.065450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa324078bb0 msgr2=0x7fa32407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa324078bb0 0x7fa32407b070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa334034000 tx=0x7fa334004730 comp rx=0 tx=0).stop 2024-09-16T15:42:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3400b3e80 msgr2=0x7fa3401f1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3400b3e80 0x7fa3401f1fc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa33000c920 tx=0x7fa33000cde0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 shutdown_connections 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa324078bb0 0x7fa32407b070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa340161440 0x7fa3400b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3400b3e80 0x7fa3401f1fc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 --2- 172.21.15.60:0/117597946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3400b34b0 0x7fa3401f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 >> 172.21.15.60:0/117597946 conn(0x7fa3400d3df0 msgr2=0x7fa3400d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 shutdown_connections 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.687+0000 7fa347200640 1 -- 172.21.15.60:0/117597946 wait complete. 2024-09-16T15:42:30.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:42:30.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:42:30.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:42:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.869+0000 7fb382a00640 1 -- 172.21.15.60:0/1741880195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb360003a00 msgr2=0x7fb360003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.869+0000 7fb382a00640 1 --2- 172.21.15.60:0/1741880195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb360003a00 0x7fb360003e20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009b50 tx=0x7fb360005830 comp rx=0 tx=0).stop 2024-09-16T15:42:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.869+0000 7fb380a00640 1 -- 172.21.15.60:0/1741880195 reap_dead start 2024-09-16T15:42:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.869+0000 7fb382a00640 1 -- 172.21.15.60:0/1741880195 shutdown_connections 2024-09-16T15:42:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.869+0000 7fb382a00640 1 -- 172.21.15.60:0/1741880195 >> 172.21.15.60:0/1741880195 conn(0x7fb37c0b1950 msgr2=0x7fb37c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.869+0000 7fb382a00640 1 -- 172.21.15.60:0/1741880195 shutdown_connections 2024-09-16T15:42:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.869+0000 7fb382a00640 1 -- 172.21.15.60:0/1741880195 wait complete. 2024-09-16T15:42:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.871+0000 7fb382a00640 1 Processor -- start 2024-09-16T15:42:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.871+0000 7fb382a00640 1 -- start start 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb382a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb360003a00 0x7fb37c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb382a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb37c153d30 0x7fb37c14fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb382a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb37c1fc180 0x7fb37c06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb382a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c0745c0 con 0x7fb37c1fc180 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb382a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c074440 con 0x7fb37c153d30 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb382a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c074740 con 0x7fb360003a00 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb380a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb37c1fc180 0x7fb37c06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb37be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb360003a00 0x7fb37c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb380a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb37c1fc180 0x7fb37c06f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35504/0 (socket says 172.21.15.60:35504) 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.872+0000 7fb380a00640 1 -- 172.21.15.60:0/3080544625 learned_addr learned my addr 172.21.15.60:0/3080544625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb373e00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb37c153d30 0x7fb37c14fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb380a00640 1 -- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb360003a00 msgr2=0x7fb37c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb380a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb360003a00 0x7fb37c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb380a00640 1 -- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb37c153d30 msgr2=0x7fb37c14fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb380a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb37c153d30 0x7fb37c14fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb380a00640 1 -- 172.21.15.60:0/3080544625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c1fc680 con 0x7fb37c1fc180 2024-09-16T15:42:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb380a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb37c1fc180 0x7fb37c06f2d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb374011b00 tx=0x7fb374011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb378c00640 1 -- 172.21.15.60:0/3080544625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37400fd70 con 0x7fb37c1fc180 2024-09-16T15:42:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb378c00640 1 -- 172.21.15.60:0/3080544625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb374004b20 con 0x7fb37c1fc180 2024-09-16T15:42:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb378c00640 1 -- 172.21.15.60:0/3080544625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374003970 con 0x7fb37c1fc180 2024-09-16T15:42:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.873+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c1fc960 con 0x7fb37c1fc180 2024-09-16T15:42:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.874+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c1fcf20 con 0x7fb37c1fc180 2024-09-16T15:42:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.875+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37c0b2440 con 0x7fb37c1fc180 2024-09-16T15:42:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.876+0000 7fb378c00640 1 -- 172.21.15.60:0/3080544625 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fb374020020 con 0x7fb37c1fc180 2024-09-16T15:42:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.877+0000 7fb378c00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb358078be0 0x7fb35807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.877+0000 7fb378c00640 1 -- 172.21.15.60:0/3080544625 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7fb37409b5a0 con 0x7fb37c1fc180 2024-09-16T15:42:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.877+0000 7fb37be00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb358078be0 0x7fb35807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.878+0000 7fb37be00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb358078be0 0x7fb35807b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb36c035000 tx=0x7fb36c02d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:30.881+0000 7fb378c00640 1 -- 172.21.15.60:0/3080544625 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb3740636b0 con 0x7fb37c1fc180 2024-09-16T15:42:31.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.118+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb37c0afa50 con 0x7fb37c1fc180 2024-09-16T15:42:31.121 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:42:31.119+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:42:31.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.121+0000 7fb378c00640 1 -- 172.21.15.60:0/3080544625 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7fb374062fb0 con 0x7fb37c1fc180 2024-09-16T15:42:31.123 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:42:31.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb358078be0 msgr2=0x7fb35807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:31.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb358078be0 0x7fb35807b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb36c035000 tx=0x7fb36c02d040 comp rx=0 tx=0).stop 2024-09-16T15:42:31.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb37c1fc180 msgr2=0x7fb37c06f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:31.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb37c1fc180 0x7fb37c06f2d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb374011b00 tx=0x7fb374011f00 comp rx=0 tx=0).stop 2024-09-16T15:42:31.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 shutdown_connections 2024-09-16T15:42:31.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fb358078be0 0x7fb35807b0a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:31.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb37c1fc180 0x7fb37c06f2d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:31.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb37c153d30 0x7fb37c14fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:31.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 --2- 172.21.15.60:0/3080544625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb360003a00 0x7fb37c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:31.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 >> 172.21.15.60:0/3080544625 conn(0x7fb37c0b1950 msgr2=0x7fb37c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:31.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 shutdown_connections 2024-09-16T15:42:31.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.124+0000 7fb382a00640 1 -- 172.21.15.60:0/3080544625 wait complete. 2024-09-16T15:42:31.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:42:31.194 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:42:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.342+0000 7f8839200640 1 -- 172.21.15.60:0/3231685548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 msgr2=0x7f88340cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.342+0000 7f8839200640 1 --2- 172.21.15.60:0/3231685548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 0x7f88340cdad0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8834072fb0 tx=0x7f881c02b8d0 comp rx=0 tx=0).stop 2024-09-16T15:42:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.343+0000 7f8839200640 1 -- 172.21.15.60:0/3231685548 shutdown_connections 2024-09-16T15:42:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.343+0000 7f8839200640 1 --2- 172.21.15.60:0/3231685548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 0x7f88340cdad0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.343+0000 7f8839200640 1 -- 172.21.15.60:0/3231685548 >> 172.21.15.60:0/3231685548 conn(0x7f88340b1840 msgr2=0x7f88340b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.343+0000 7f8839200640 1 -- 172.21.15.60:0/3231685548 shutdown_connections 2024-09-16T15:42:31.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.343+0000 7f8839200640 1 -- 172.21.15.60:0/3231685548 wait complete. 2024-09-16T15:42:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.344+0000 7f8839200640 1 Processor -- start 2024-09-16T15:42:31.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.344+0000 7f8839200640 1 -- start start 2024-09-16T15:42:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.345+0000 7f8839200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88340739d0 0x7f8834151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.345+0000 7f8839200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 0x7f88341435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.345+0000 7f8839200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88341517a0 0x7f88341f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.345+0000 7f8839200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88340cc7d0 con 0x7f88340739d0 2024-09-16T15:42:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.345+0000 7f8839200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88340cc650 con 0x7f88340cd490 2024-09-16T15:42:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.345+0000 7f8839200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88340cc950 con 0x7f88341517a0 2024-09-16T15:42:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.345+0000 7f8831e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88340739d0 0x7f8834151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:31.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8831e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88340739d0 0x7f8834151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35512/0 (socket says 172.21.15.60:35512) 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8831e00640 1 -- 172.21.15.60:0/1374147648 learned_addr learned my addr 172.21.15.60:0/1374147648 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8832800640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88341517a0 0x7f88341f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8829400640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 0x7f88341435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8831e00640 1 -- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88341517a0 msgr2=0x7f88341f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8831e00640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88341517a0 0x7f88341f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8831e00640 1 -- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 msgr2=0x7f88341435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8831e00640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 0x7f88341435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8831e00640 1 -- 172.21.15.60:0/1374147648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88341f2a30 con 0x7f88340739d0 2024-09-16T15:42:31.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8829400640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 0x7f88341435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f8831e00640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88340739d0 0x7f8834151c20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f882400c430 tx=0x7f882400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.346+0000 7f882aa00640 1 -- 172.21.15.60:0/1374147648 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824017070 con 0x7f88340739d0 2024-09-16T15:42:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.347+0000 7f882aa00640 1 -- 172.21.15.60:0/1374147648 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88240053c0 con 0x7f88340739d0 2024-09-16T15:42:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.347+0000 7f882aa00640 1 -- 172.21.15.60:0/1374147648 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824003850 con 0x7f88340739d0 2024-09-16T15:42:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.347+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88340cf430 con 0x7f88340739d0 2024-09-16T15:42:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.347+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8834154900 con 0x7f88340739d0 2024-09-16T15:42:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.348+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8834074210 con 0x7f88340739d0 2024-09-16T15:42:31.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.349+0000 7f882aa00640 1 -- 172.21.15.60:0/1374147648 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f882401a070 con 0x7f88340739d0 2024-09-16T15:42:31.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.350+0000 7f882aa00640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8810078bb0 0x7f881007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:31.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.350+0000 7f882aa00640 1 -- 172.21.15.60:0/1374147648 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f882409a330 con 0x7f88340739d0 2024-09-16T15:42:31.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.350+0000 7f8829400640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8810078bb0 0x7f881007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:31.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.351+0000 7f8829400640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8810078bb0 0x7f881007b070 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f881c036000 tx=0x7f881c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:31.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.354+0000 7f882aa00640 1 -- 172.21.15.60:0/1374147648 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8834074210 con 0x7f88340739d0 2024-09-16T15:42:31.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:31.590+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f88340cdad0 con 0x7f88340739d0 2024-09-16T15:42:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.126+0000 7f882aa00640 1 -- 172.21.15.60:0/1374147648 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f8824062470 con 0x7f88340739d0 2024-09-16T15:42:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8810078bb0 msgr2=0x7f881007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8810078bb0 0x7f881007b070 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f881c036000 tx=0x7f881c004730 comp rx=0 tx=0).stop 2024-09-16T15:42:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88340739d0 msgr2=0x7f8834151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:32.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88340739d0 0x7f8834151c20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f882400c430 tx=0x7f882400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:42:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 shutdown_connections 2024-09-16T15:42:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8810078bb0 0x7f881007b070 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88341517a0 0x7f88341f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88340cd490 0x7f88341435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 --2- 172.21.15.60:0/1374147648 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88340739d0 0x7f8834151c20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 >> 172.21.15.60:0/1374147648 conn(0x7f88340b1840 msgr2=0x7f88340cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 shutdown_connections 2024-09-16T15:42:32.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.129+0000 7f8839200640 1 -- 172.21.15.60:0/1374147648 wait complete. 2024-09-16T15:42:32.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:42:32.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.315+0000 7f7500200640 1 -- 172.21.15.60:0/514900894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e0004800 msgr2=0x7f74e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:32.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.315+0000 7f7500200640 1 --2- 172.21.15.60:0/514900894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74e0004800 0x7f74e0004c80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009b50 tx=0x7f74e0005830 comp rx=0 tx=0).stop 2024-09-16T15:42:32.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.315+0000 7f74fe200640 1 -- 172.21.15.60:0/514900894 reap_dead start 2024-09-16T15:42:32.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.315+0000 7f7500200640 1 -- 172.21.15.60:0/514900894 shutdown_connections 2024-09-16T15:42:32.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.315+0000 7f7500200640 1 -- 172.21.15.60:0/514900894 >> 172.21.15.60:0/514900894 conn(0x7f74f80b18e0 msgr2=0x7f74f80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:32.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.315+0000 7f7500200640 1 -- 172.21.15.60:0/514900894 shutdown_connections 2024-09-16T15:42:32.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.315+0000 7f7500200640 1 -- 172.21.15.60:0/514900894 wait complete. 2024-09-16T15:42:32.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.316+0000 7f7500200640 1 Processor -- start 2024-09-16T15:42:32.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.317+0000 7f7500200640 1 -- start start 2024-09-16T15:42:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.317+0000 7f7500200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e0004800 0x7f74f81eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.317+0000 7f7500200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74f80cf060 0x7f74f80cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.317+0000 7f7500200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f80cf4e0 0x7f74f81f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.317+0000 7f7500200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74f806ed50 con 0x7f74f80cf060 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f7500200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74f806ebd0 con 0x7f74f80cf4e0 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f7500200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74f80cc190 con 0x7f74e0004800 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74f80cf060 0x7f74f80cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74f80cf060 0x7f74f80cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35544/0 (socket says 172.21.15.60:35544) 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fce00640 1 -- 172.21.15.60:0/2333265600 learned_addr learned my addr 172.21.15.60:0/2333265600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fe200640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f80cf4e0 0x7f74f81f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fd800640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e0004800 0x7f74f81eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fce00640 1 -- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e0004800 msgr2=0x7f74f81eeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fce00640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e0004800 0x7f74f81eeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fce00640 1 -- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f80cf4e0 msgr2=0x7f74f81f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fce00640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f80cf4e0 0x7f74f81f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.318+0000 7f74fce00640 1 -- 172.21.15.60:0/2333265600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f81ef1e0 con 0x7f74f80cf060 2024-09-16T15:42:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.319+0000 7f74fce00640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74f80cf060 0x7f74f80cdad0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f74e4036040 tx=0x7f74e40336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.319+0000 7f74f5600640 1 -- 172.21.15.60:0/2333265600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4038040 con 0x7f74f80cf060 2024-09-16T15:42:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.319+0000 7f74f5600640 1 -- 172.21.15.60:0/2333265600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74e4043070 con 0x7f74f80cf060 2024-09-16T15:42:32.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.319+0000 7f74f5600640 1 -- 172.21.15.60:0/2333265600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4033b40 con 0x7f74f80cf060 2024-09-16T15:42:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.319+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f80c9fd0 con 0x7f74f80cf060 2024-09-16T15:42:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.319+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f80ca590 con 0x7f74f80cf060 2024-09-16T15:42:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.320+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f8069360 con 0x7f74f80cf060 2024-09-16T15:42:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.321+0000 7f74f5600640 1 -- 172.21.15.60:0/2333265600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f74e4033ce0 con 0x7f74f80cf060 2024-09-16T15:42:32.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.323+0000 7f74f5600640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f74d8078be0 0x7f74d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:32.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.323+0000 7f74f5600640 1 -- 172.21.15.60:0/2333265600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5795+0+0 (secure 0 0 0) 0x7f74e40bb530 con 0x7f74f80cf060 2024-09-16T15:42:32.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.323+0000 7f74fd800640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f74d8078be0 0x7f74d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:32.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.324+0000 7f74fd800640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f74d8078be0 0x7f74d807b0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f74ec000fb0 tx=0x7f74ec006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:32.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.327+0000 7f74f5600640 1 -- 172.21.15.60:0/2333265600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f74e404c020 con 0x7f74f80cf060 2024-09-16T15:42:32.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:32.522+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f74f8071070 con 0x7f74f80cf060 2024-09-16T15:42:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.128+0000 7f74f5600640 1 -- 172.21.15.60:0/2333265600 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) ==== 181+0+0 (secure 0 0 0) 0x7f74f8071070 con 0x7f74f80cf060 2024-09-16T15:42:33.130 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:42:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f74d8078be0 msgr2=0x7f74d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f74d8078be0 0x7f74d807b0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f74ec000fb0 tx=0x7f74ec006720 comp rx=0 tx=0).stop 2024-09-16T15:42:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74f80cf060 msgr2=0x7f74f80cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74f80cf060 0x7f74f80cdad0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f74e4036040 tx=0x7f74e40336f0 comp rx=0 tx=0).stop 2024-09-16T15:42:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 shutdown_connections 2024-09-16T15:42:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f74d8078be0 0x7f74d807b0a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74f80cf4e0 0x7f74f81f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74f80cf060 0x7f74f80cdad0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 --2- 172.21.15.60:0/2333265600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74e0004800 0x7f74f81eeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 >> 172.21.15.60:0/2333265600 conn(0x7f74f80b18e0 msgr2=0x7f74f80af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 shutdown_connections 2024-09-16T15:42:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.132+0000 7f7500200640 1 -- 172.21.15.60:0/2333265600 wait complete. 2024-09-16T15:42:33.152 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:42:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.321+0000 7f2833e00640 1 -- 172.21.15.60:0/1925632097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 msgr2=0x7f2818003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.321+0000 7f2833e00640 1 --2- 172.21.15.60:0/1925632097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 0x7f2818003e70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f28240016c0 tx=0x7f2818005830 comp rx=0 tx=0).stop 2024-09-16T15:42:33.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.321+0000 7f2833e00640 1 -- 172.21.15.60:0/1925632097 shutdown_connections 2024-09-16T15:42:33.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.321+0000 7f2833e00640 1 --2- 172.21.15.60:0/1925632097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 0x7f2818003e70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:33.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.321+0000 7f2833e00640 1 -- 172.21.15.60:0/1925632097 >> 172.21.15.60:0/1925632097 conn(0x7f28340cfa70 msgr2=0x7f28340cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:33.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.321+0000 7f2833e00640 1 -- 172.21.15.60:0/1925632097 shutdown_connections 2024-09-16T15:42:33.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.321+0000 7f2833e00640 1 -- 172.21.15.60:0/1925632097 wait complete. 2024-09-16T15:42:33.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.323+0000 7f2833e00640 1 Processor -- start 2024-09-16T15:42:33.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.323+0000 7f2833e00640 1 -- start start 2024-09-16T15:42:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2833e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 0x7f28340adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2833e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28340cbdd0 0x7f28340c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:33.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2833e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28340c4f20 0x7f2834151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2833e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2834070280 con 0x7f2818003a50 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2833e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2834070100 con 0x7f28340cbdd0 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2833e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2834073010 con 0x7f28340c4f20 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2832a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 0x7f28340adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2832a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 0x7f28340adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35562/0 (socket says 172.21.15.60:35562) 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2832a00640 1 -- 172.21.15.60:0/1227441172 learned_addr learned my addr 172.21.15.60:0/1227441172 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2833400640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28340c4f20 0x7f2834151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f282a000640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28340cbdd0 0x7f28340c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.324+0000 7f2832a00640 1 -- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28340c4f20 msgr2=0x7f2834151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f2832a00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28340c4f20 0x7f2834151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f2832a00640 1 -- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28340cbdd0 msgr2=0x7f28340c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f2832a00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28340cbdd0 0x7f28340c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f2832a00640 1 -- 172.21.15.60:0/1227441172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28340c3940 con 0x7f2818003a50 2024-09-16T15:42:33.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f2832a00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 0x7f28340adaf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f282402a8e0 tx=0x7f28240063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f282be00640 1 -- 172.21.15.60:0/1227441172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824003a00 con 0x7f2818003a50 2024-09-16T15:42:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f282be00640 1 -- 172.21.15.60:0/1227441172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2824039070 con 0x7f2818003a50 2024-09-16T15:42:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28340c3c20 con 0x7f2818003a50 2024-09-16T15:42:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f282be00640 1 -- 172.21.15.60:0/1227441172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824033420 con 0x7f2818003a50 2024-09-16T15:42:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.325+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28340c41e0 con 0x7f2818003a50 2024-09-16T15:42:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.326+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2834073500 con 0x7f2818003a50 2024-09-16T15:42:33.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.327+0000 7f282be00640 1 -- 172.21.15.60:0/1227441172 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f2824044020 con 0x7f2818003a50 2024-09-16T15:42:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.328+0000 7f282be00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2810078be0 0x7f281007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:33.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.328+0000 7f282be00640 1 -- 172.21.15.60:0/1227441172 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5357+0+0 (secure 0 0 0) 0x7f28240ba420 con 0x7f2818003a50 2024-09-16T15:42:33.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.329+0000 7f282a000640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2810078be0 0x7f281007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:33.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.330+0000 7f282a000640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2810078be0 0x7f281007b0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f28340cbb20 tx=0x7f281c006040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:33.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.332+0000 7f282be00640 1 -- 172.21.15.60:0/1227441172 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2824082040 con 0x7f2818003a50 2024-09-16T15:42:33.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:33.532+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f28340b4880 con 0x7f2818003a50 2024-09-16T15:42:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.134+0000 7f282be00640 1 -- 172.21.15.60:0/1227441172 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) ==== 169+0+0 (secure 0 0 0) 0x7f2824086a70 con 0x7f2818003a50 2024-09-16T15:42:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:42:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2810078be0 msgr2=0x7f281007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2810078be0 0x7f281007b0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f28340cbb20 tx=0x7f281c006040 comp rx=0 tx=0).stop 2024-09-16T15:42:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 msgr2=0x7f28340adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 0x7f28340adaf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f282402a8e0 tx=0x7f28240063a0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 shutdown_connections 2024-09-16T15:42:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2810078be0 0x7f281007b0a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28340c4f20 0x7f2834151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28340cbdd0 0x7f28340c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 --2- 172.21.15.60:0/1227441172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2818003a50 0x7f28340adaf0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 >> 172.21.15.60:0/1227441172 conn(0x7f28340cfa70 msgr2=0x7f28340cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.139+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 shutdown_connections 2024-09-16T15:42:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.140+0000 7f2833e00640 1 -- 172.21.15.60:0/1227441172 wait complete. 2024-09-16T15:42:34.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:42:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.325+0000 7f53e2e00640 1 -- 172.21.15.60:0/49778029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53dc0ccf00 msgr2=0x7f53dc0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.325+0000 7f53e2e00640 1 --2- 172.21.15.60:0/49778029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53dc0ccf00 0x7f53dc0acd30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005fd0 tx=0x7f53c802a540 comp rx=0 tx=0).stop 2024-09-16T15:42:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.325+0000 7f53e2e00640 1 -- 172.21.15.60:0/49778029 shutdown_connections 2024-09-16T15:42:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.325+0000 7f53e2e00640 1 --2- 172.21.15.60:0/49778029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53dc0ccf00 0x7f53dc0acd30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.325+0000 7f53e2e00640 1 --2- 172.21.15.60:0/49778029 >> v2:172.21.15.100:3300/0 conn(0x7f53dc15f5a0 0x7f53dc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.325+0000 7f53e2e00640 1 -- 172.21.15.60:0/49778029 >> 172.21.15.60:0/49778029 conn(0x7f53dc0b1950 msgr2=0x7f53dc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.325+0000 7f53e2e00640 1 -- 172.21.15.60:0/49778029 shutdown_connections 2024-09-16T15:42:34.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.325+0000 7f53e2e00640 1 -- 172.21.15.60:0/49778029 wait complete. 2024-09-16T15:42:34.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.326+0000 7f53e2e00640 1 Processor -- start 2024-09-16T15:42:34.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.327+0000 7f53e2e00640 1 -- start start 2024-09-16T15:42:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.327+0000 7f53e2e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53dc0ad270 0x7f53dc09d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.327+0000 7f53e2e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc0ccf00 0x7f53dc1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.327+0000 7f53e2e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53dc15f5a0 0x7f53dc1f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.327+0000 7f53e2e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc06f2e0 con 0x7f53dc15f5a0 2024-09-16T15:42:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.327+0000 7f53e2e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc06f160 con 0x7f53dc0ccf00 2024-09-16T15:42:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.327+0000 7f53e2e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f53dc06f460 con 0x7f53dc0ad270 2024-09-16T15:42:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53e0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53dc15f5a0 0x7f53dc1f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53e0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53dc15f5a0 0x7f53dc1f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35566/0 (socket says 172.21.15.60:35566) 2024-09-16T15:42:34.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53e0e00640 1 -- 172.21.15.60:0/3952080322 learned_addr learned my addr 172.21.15.60:0/3952080322 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:34.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53db400640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc0ccf00 0x7f53dc1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53dbe00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53dc0ad270 0x7f53dc09d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53e0e00640 1 -- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53dc0ad270 msgr2=0x7f53dc09d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53e0e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53dc0ad270 0x7f53dc09d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53e0e00640 1 -- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc0ccf00 msgr2=0x7f53dc1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53e0e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc0ccf00 0x7f53dc1549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.328+0000 7f53e0e00640 1 -- 172.21.15.60:0/3952080322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc0cdb80 con 0x7f53dc15f5a0 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.329+0000 7f53e0e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53dc15f5a0 0x7f53dc1f52a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f53d400c7a0 tx=0x7f53d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.329+0000 7f53d3e00640 1 -- 172.21.15.60:0/3952080322 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4003b70 con 0x7f53dc15f5a0 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.329+0000 7f53d3e00640 1 -- 172.21.15.60:0/3952080322 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53d4003d10 con 0x7f53dc15f5a0 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.329+0000 7f53d3e00640 1 -- 172.21.15.60:0/3952080322 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d400aab0 con 0x7f53dc15f5a0 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.329+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc0cde60 con 0x7f53dc15f5a0 2024-09-16T15:42:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.329+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc09d7f0 con 0x7f53dc15f5a0 2024-09-16T15:42:34.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.330+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc06f9f0 con 0x7f53dc15f5a0 2024-09-16T15:42:34.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.331+0000 7f53d3e00640 1 -- 172.21.15.60:0/3952080322 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f53d400ac50 con 0x7f53dc15f5a0 2024-09-16T15:42:34.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.332+0000 7f53d3e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f53bc078bb0 0x7f53bc07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.332+0000 7f53d3e00640 1 -- 172.21.15.60:0/3952080322 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f53d409aad0 con 0x7f53dc15f5a0 2024-09-16T15:42:34.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.333+0000 7f53dbe00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f53bc078bb0 0x7f53bc07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.334+0000 7f53dbe00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f53bc078bb0 0x7f53bc07b070 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0095f0 tx=0x7f53cc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.337+0000 7f53d3e00640 1 -- 172.21.15.60:0/3952080322 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f53d4062790 con 0x7f53dc15f5a0 2024-09-16T15:42:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.543+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f53dc0aeb30 con 0x7f53dc15f5a0 2024-09-16T15:42:34.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.547+0000 7f53d3e00640 1 -- 172.21.15.60:0/3952080322 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f53d40671c0 con 0x7f53dc15f5a0 2024-09-16T15:42:34.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f53bc078bb0 msgr2=0x7f53bc07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f53bc078bb0 0x7f53bc07b070 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0095f0 tx=0x7f53cc005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53dc15f5a0 msgr2=0x7f53dc1f52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53dc15f5a0 0x7f53dc1f52a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f53d400c7a0 tx=0x7f53d400cc60 comp rx=0 tx=0).stop 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 shutdown_connections 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f53bc078bb0 0x7f53bc07b070 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53dc15f5a0 0x7f53dc1f52a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53dc0ccf00 0x7f53dc1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 --2- 172.21.15.60:0/3952080322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53dc0ad270 0x7f53dc09d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 >> 172.21.15.60:0/3952080322 conn(0x7f53dc0b1950 msgr2=0x7f53dc15d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 shutdown_connections 2024-09-16T15:42:34.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.549+0000 7f53e2e00640 1 -- 172.21.15.60:0/3952080322 wait complete. 2024-09-16T15:42:34.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-16T15:42:34.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.728+0000 7fca9a600640 1 -- 172.21.15.60:0/2955221696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 msgr2=0x7fca7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.728+0000 7fca9a600640 1 --2- 172.21.15.60:0/2955221696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 0x7fca7800b9c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fca8c006fd0 tx=0x7fca78005830 comp rx=0 tx=0).stop 2024-09-16T15:42:34.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.728+0000 7fca9a600640 1 -- 172.21.15.60:0/2955221696 shutdown_connections 2024-09-16T15:42:34.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.728+0000 7fca9a600640 1 --2- 172.21.15.60:0/2955221696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 0x7fca7800b9c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.728+0000 7fca9a600640 1 -- 172.21.15.60:0/2955221696 >> 172.21.15.60:0/2955221696 conn(0x7fca940b1950 msgr2=0x7fca940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.728+0000 7fca9a600640 1 -- 172.21.15.60:0/2955221696 shutdown_connections 2024-09-16T15:42:34.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.728+0000 7fca9a600640 1 -- 172.21.15.60:0/2955221696 wait complete. 2024-09-16T15:42:34.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.730+0000 7fca9a600640 1 Processor -- start 2024-09-16T15:42:34.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.730+0000 7fca9a600640 1 -- start start 2024-09-16T15:42:34.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca9a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 0x7fca94150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca9a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca94154850 0x7fca94159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca9a600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca940ad8f0 0x7fca940add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca9a600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fca9406f160 con 0x7fca780052e0 2024-09-16T15:42:34.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca9a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca94161aa0 con 0x7fca94154850 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca9a600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fca9406f2e0 con 0x7fca940ad8f0 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca93400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 0x7fca94150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca93400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 0x7fca94150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35582/0 (socket says 172.21.15.60:35582) 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca93400640 1 -- 172.21.15.60:0/3134647641 learned_addr learned my addr 172.21.15.60:0/3134647641 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca93400640 1 -- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca940ad8f0 msgr2=0x7fca940add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca93400640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca940ad8f0 0x7fca940add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca93400640 1 -- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca94154850 msgr2=0x7fca94159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca92a00640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca94154850 0x7fca94159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca93400640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca94154850 0x7fca94159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca93400640 1 -- 172.21.15.60:0/3134647641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca941f92b0 con 0x7fca780052e0 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.731+0000 7fca92a00640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca94154850 0x7fca94159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.732+0000 7fca93400640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 0x7fca94150bc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fca84011b10 tx=0x7fca84009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.732+0000 7fca8b400640 1 -- 172.21.15.60:0/3134647641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8401e070 con 0x7fca780052e0 2024-09-16T15:42:34.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.732+0000 7fca8b400640 1 -- 172.21.15.60:0/3134647641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca84004480 con 0x7fca780052e0 2024-09-16T15:42:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.732+0000 7fca8b400640 1 -- 172.21.15.60:0/3134647641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8400fc30 con 0x7fca780052e0 2024-09-16T15:42:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.732+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca94159980 con 0x7fca780052e0 2024-09-16T15:42:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.732+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca941f8a10 con 0x7fca780052e0 2024-09-16T15:42:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.733+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9406f840 con 0x7fca780052e0 2024-09-16T15:42:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.734+0000 7fca8b400640 1 -- 172.21.15.60:0/3134647641 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fca84021070 con 0x7fca780052e0 2024-09-16T15:42:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.735+0000 7fca8b400640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fca74078bb0 0x7fca7407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.735+0000 7fca8b400640 1 -- 172.21.15.60:0/3134647641 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7fca8409b6f0 con 0x7fca780052e0 2024-09-16T15:42:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.736+0000 7fca92a00640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fca74078bb0 0x7fca7407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.736+0000 7fca92a00640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fca74078bb0 0x7fca7407b070 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca940d0c90 tx=0x7fca80006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:34.739+0000 7fca8b400640 1 -- 172.21.15.60:0/3134647641 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fca840633b0 con 0x7fca780052e0 2024-09-16T15:42:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.011+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fca9415fcd0 con 0x7fca780052e0 2024-09-16T15:42:35.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.136+0000 7fca8b400640 1 -- 172.21.15.60:0/3134647641 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v224) ==== 161+0+0 (secure 0 0 0) 0x7fca84067de0 con 0x7fca780052e0 2024-09-16T15:42:35.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fca74078bb0 msgr2=0x7fca7407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fca74078bb0 0x7fca7407b070 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca940d0c90 tx=0x7fca80006b70 comp rx=0 tx=0).stop 2024-09-16T15:42:35.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 msgr2=0x7fca94150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 0x7fca94150bc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fca84011b10 tx=0x7fca84009e40 comp rx=0 tx=0).stop 2024-09-16T15:42:35.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 shutdown_connections 2024-09-16T15:42:35.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fca74078bb0 0x7fca7407b070 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca940ad8f0 0x7fca940add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca94154850 0x7fca94159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 --2- 172.21.15.60:0/3134647641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca780052e0 0x7fca94150bc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 >> 172.21.15.60:0/3134647641 conn(0x7fca940b1950 msgr2=0x7fca94073240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 shutdown_connections 2024-09-16T15:42:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.139+0000 7fca9a600640 1 -- 172.21.15.60:0/3134647641 wait complete. 2024-09-16T15:42:35.157 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-16T15:42:35.157 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:42:35.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:35.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.315+0000 7f1bc5600640 1 -- 172.21.15.60:0/848952338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 msgr2=0x7f1bc0074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.315+0000 7f1bc5600640 1 --2- 172.21.15.60:0/848952338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 0x7f1bc0074440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8005fd0 tx=0x7f1bb802aa10 comp rx=0 tx=0).stop 2024-09-16T15:42:35.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.316+0000 7f1bc5600640 1 -- 172.21.15.60:0/848952338 shutdown_connections 2024-09-16T15:42:35.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.316+0000 7f1bc5600640 1 --2- 172.21.15.60:0/848952338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 0x7f1bc0074440 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.316+0000 7f1bc5600640 1 --2- 172.21.15.60:0/848952338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00ccf00 0x7f1bc00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.316+0000 7f1bc5600640 1 -- 172.21.15.60:0/848952338 >> 172.21.15.60:0/848952338 conn(0x7f1bc00b1950 msgr2=0x7f1bc00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.316+0000 7f1bc5600640 1 -- 172.21.15.60:0/848952338 shutdown_connections 2024-09-16T15:42:35.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.316+0000 7f1bc5600640 1 -- 172.21.15.60:0/848952338 wait complete. 2024-09-16T15:42:35.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.317+0000 7f1bc5600640 1 Processor -- start 2024-09-16T15:42:35.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.317+0000 7f1bc5600640 1 -- start start 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.318+0000 7f1bc5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 0x7f1bc01f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.318+0000 7f1bc5600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00ccf00 0x7f1bc01f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.318+0000 7f1bc5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015f5a0 0x7f1bc01580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.318+0000 7f1bc5600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc006f2e0 con 0x7f1bc0073fb0 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.318+0000 7f1bc5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc006f160 con 0x7f1bc015f5a0 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.318+0000 7f1bc5600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc006f460 con 0x7f1bc00ccf00 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 0x7f1bc01f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 0x7f1bc01f58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35608/0 (socket says 172.21.15.60:35608) 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 -- 172.21.15.60:0/1178188821 learned_addr learned my addr 172.21.15.60:0/1178188821 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bb6000640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00ccf00 0x7f1bc01f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbf400640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015f5a0 0x7f1bc01580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 -- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00ccf00 msgr2=0x7f1bc01f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00ccf00 0x7f1bc01f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 -- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015f5a0 msgr2=0x7f1bc01580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015f5a0 0x7f1bc01580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 -- 172.21.15.60:0/1178188821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc0157330 con 0x7f1bc0073fb0 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbf400640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015f5a0 0x7f1bc01580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.319+0000 7f1bbea00640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 0x7f1bc01f58a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000c3a0 tx=0x7f1bb000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.320+0000 7f1bb7400640 1 -- 172.21.15.60:0/1178188821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0017070 con 0x7f1bc0073fb0 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.320+0000 7f1bb7400640 1 -- 172.21.15.60:0/1178188821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bb00053c0 con 0x7f1bc0073fb0 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.320+0000 7f1bb7400640 1 -- 172.21.15.60:0/1178188821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0003850 con 0x7f1bc0073fb0 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.320+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc0157610 con 0x7f1bc0073fb0 2024-09-16T15:42:35.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.320+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc0157bd0 con 0x7f1bc0073fb0 2024-09-16T15:42:35.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.321+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc006f9f0 con 0x7f1bc0073fb0 2024-09-16T15:42:35.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.322+0000 7f1bb7400640 1 -- 172.21.15.60:0/1178188821 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f1bb001a070 con 0x7f1bc0073fb0 2024-09-16T15:42:35.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.323+0000 7f1bb7400640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b9c078bb0 0x7f1b9c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.323+0000 7f1bb7400640 1 -- 172.21.15.60:0/1178188821 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f1bb0099fa0 con 0x7f1bc0073fb0 2024-09-16T15:42:35.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.323+0000 7f1bb6000640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b9c078bb0 0x7f1b9c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.324+0000 7f1bb6000640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b9c078bb0 0x7f1b9c07b070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1bc014d390 tx=0x7f1ba8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.327+0000 7f1bb7400640 1 -- 172.21.15.60:0/1178188821 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1bb0062440 con 0x7f1bc0073fb0 2024-09-16T15:42:35.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.521+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bc0157e10 con 0x7f1bc0073fb0 2024-09-16T15:42:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.522+0000 7f1bb7400640 1 -- 172.21.15.60:0/1178188821 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+13034 (secure 0 0 0) 0x7f1bc0157e10 con 0x7f1bc0073fb0 2024-09-16T15:42:35.525 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:35.525 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":65,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:34.130665+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6836/2580251790":"2024-09-17T15:42:31.120591+0000","172.21.15.100:6835/2580251790":"2024-09-17T15:42:31.120591+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b9c078bb0 msgr2=0x7f1b9c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b9c078bb0 0x7f1b9c07b070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1bc014d390 tx=0x7f1ba8005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 msgr2=0x7f1bc01f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 0x7f1bc01f58a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000c3a0 tx=0x7f1bb000c860 comp rx=0 tx=0).stop 2024-09-16T15:42:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 shutdown_connections 2024-09-16T15:42:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f1b9c078bb0 0x7f1b9c07b070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015f5a0 0x7f1bc01580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00ccf00 0x7f1bc01f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 --2- 172.21.15.60:0/1178188821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc0073fb0 0x7f1bc01f58a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 >> 172.21.15.60:0/1178188821 conn(0x7f1bc00b1950 msgr2=0x7f1bc009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 shutdown_connections 2024-09-16T15:42:35.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.525+0000 7f1bc5600640 1 -- 172.21.15.60:0/1178188821 wait complete. 2024-09-16T15:42:35.543 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:42:35.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:42:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.706+0000 7f534aa00640 1 -- 172.21.15.60:0/1655221201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344161760 msgr2=0x7f53440ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.706+0000 7f534aa00640 1 --2- 172.21.15.60:0/1655221201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344161760 0x7f53440ce4a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5334005fd0 tx=0x7f533402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.706+0000 7f534aa00640 1 -- 172.21.15.60:0/1655221201 shutdown_connections 2024-09-16T15:42:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.706+0000 7f534aa00640 1 --2- 172.21.15.60:0/1655221201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53440cec60 0x7f53440cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.706+0000 7f534aa00640 1 --2- 172.21.15.60:0/1655221201 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5344161760 0x7f53440ce4a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.706+0000 7f534aa00640 1 -- 172.21.15.60:0/1655221201 >> 172.21.15.60:0/1655221201 conn(0x7f53440b1840 msgr2=0x7f53440b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.707+0000 7f534aa00640 1 -- 172.21.15.60:0/1655221201 shutdown_connections 2024-09-16T15:42:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.707+0000 7f534aa00640 1 -- 172.21.15.60:0/1655221201 wait complete. 2024-09-16T15:42:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.708+0000 7f534aa00640 1 Processor -- start 2024-09-16T15:42:35.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.708+0000 7f534aa00640 1 -- start start 2024-09-16T15:42:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.709+0000 7f534aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53440af190 0x7f534415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.709+0000 7f534aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53440cec60 0x7f5344151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.709+0000 7f534aa00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5344161760 0x7f53441f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.709+0000 7f534aa00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5344162310 con 0x7f53440cec60 2024-09-16T15:42:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.709+0000 7f534aa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5344162190 con 0x7f53440af190 2024-09-16T15:42:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.709+0000 7f534aa00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5344162490 con 0x7f5344161760 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.709+0000 7f533b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53440cec60 0x7f5344151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f533b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53440cec60 0x7f5344151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35620/0 (socket says 172.21.15.60:35620) 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f533b400640 1 -- 172.21.15.60:0/231129894 learned_addr learned my addr 172.21.15.60:0/231129894 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f5343e00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53440af190 0x7f534415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f5348a00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5344161760 0x7f53441f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f533b400640 1 -- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5344161760 msgr2=0x7f53441f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f533b400640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5344161760 0x7f53441f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f533b400640 1 -- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53440af190 msgr2=0x7f534415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f533b400640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53440af190 0x7f534415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f533b400640 1 -- 172.21.15.60:0/231129894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534415e200 con 0x7f53440cec60 2024-09-16T15:42:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f533b400640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53440cec60 0x7f5344151660 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c450 tx=0x7f532c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.710+0000 7f5340c00640 1 -- 172.21.15.60:0/231129894 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0052d0 con 0x7f53440cec60 2024-09-16T15:42:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.711+0000 7f5340c00640 1 -- 172.21.15.60:0/231129894 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f532c005470 con 0x7f53440cec60 2024-09-16T15:42:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.711+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534415e4e0 con 0x7f53440cec60 2024-09-16T15:42:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.711+0000 7f5340c00640 1 -- 172.21.15.60:0/231129894 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c012870 con 0x7f53440cec60 2024-09-16T15:42:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.711+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534415eaa0 con 0x7f53440cec60 2024-09-16T15:42:35.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.712+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53441629a0 con 0x7f53440cec60 2024-09-16T15:42:35.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.713+0000 7f5340c00640 1 -- 172.21.15.60:0/231129894 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f532c0035f0 con 0x7f53440cec60 2024-09-16T15:42:35.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.714+0000 7f5340c00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5320078bb0 0x7f532007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.714+0000 7f5340c00640 1 -- 172.21.15.60:0/231129894 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f532c09a6f0 con 0x7f53440cec60 2024-09-16T15:42:35.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.714+0000 7f5343e00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5320078bb0 0x7f532007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.715+0000 7f5343e00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5320078bb0 0x7f532007b070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5334033000 tx=0x7f5334004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.719+0000 7f5340c00640 1 -- 172.21.15.60:0/231129894 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f53440cf5b0 con 0x7f53440cec60 2024-09-16T15:42:35.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.913+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f53440cf0e0 con 0x7f53440cec60 2024-09-16T15:42:35.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.914+0000 7f5340c00640 1 -- 172.21.15.60:0/231129894 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f532c062480 con 0x7f53440cec60 2024-09-16T15:42:35.916 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:42:35.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5320078bb0 msgr2=0x7f532007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5320078bb0 0x7f532007b070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5334033000 tx=0x7f5334004730 comp rx=0 tx=0).stop 2024-09-16T15:42:35.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53440cec60 msgr2=0x7f5344151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53440cec60 0x7f5344151660 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c450 tx=0x7f532c00c910 comp rx=0 tx=0).stop 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 shutdown_connections 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5320078bb0 0x7f532007b070 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5344161760 0x7f53441f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53440cec60 0x7f5344151660 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 --2- 172.21.15.60:0/231129894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53440af190 0x7f534415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 >> 172.21.15.60:0/231129894 conn(0x7f53440b1840 msgr2=0x7f53440cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 shutdown_connections 2024-09-16T15:42:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:35.916+0000 7f534aa00640 1 -- 172.21.15.60:0/231129894 wait complete. 2024-09-16T15:42:35.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-16T15:42:36.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.093+0000 7f23a6000640 1 -- 172.21.15.60:0/2122905400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2388003a50 msgr2=0x7f2388003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:36.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.093+0000 7f23a6000640 1 --2- 172.21.15.60:0/2122905400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2388003a50 0x7f2388003e90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f23940016c0 tx=0x7f2388005830 comp rx=0 tx=0).stop 2024-09-16T15:42:36.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.093+0000 7f23a6000640 1 -- 172.21.15.60:0/2122905400 shutdown_connections 2024-09-16T15:42:36.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.093+0000 7f23a6000640 1 --2- 172.21.15.60:0/2122905400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2388003a50 0x7f2388003e90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:36.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.093+0000 7f23a6000640 1 -- 172.21.15.60:0/2122905400 >> 172.21.15.60:0/2122905400 conn(0x7f23a00d3df0 msgr2=0x7f23a00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:36.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.093+0000 7f23a6000640 1 -- 172.21.15.60:0/2122905400 shutdown_connections 2024-09-16T15:42:36.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.093+0000 7f23a6000640 1 -- 172.21.15.60:0/2122905400 wait complete. 2024-09-16T15:42:36.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.095+0000 7f23a6000640 1 Processor -- start 2024-09-16T15:42:36.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.095+0000 7f23a6000640 1 -- start start 2024-09-16T15:42:36.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f23a6000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2388003a50 0x7f23a00c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:36.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f23a6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23a01586f0 0x7f23a00b9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:36.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f23a6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a00b99b0 0x7f23a00d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:36.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f23a6000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f23a00b3840 con 0x7f23a01586f0 2024-09-16T15:42:36.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f23a6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23a00b36c0 con 0x7f23a00b99b0 2024-09-16T15:42:36.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f23a6000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f23a00b39c0 con 0x7f2388003a50 2024-09-16T15:42:36.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f239fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23a01586f0 0x7f23a00b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f239fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23a01586f0 0x7f23a00b9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35650/0 (socket says 172.21.15.60:35650) 2024-09-16T15:42:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f239fe00640 1 -- 172.21.15.60:0/2529953456 learned_addr learned my addr 172.21.15.60:0/2529953456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f23a4c00640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2388003a50 0x7f23a00c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:36.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f23a5600640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a00b99b0 0x7f23a00d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f239fe00640 1 -- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2388003a50 msgr2=0x7f23a00c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f239fe00640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2388003a50 0x7f23a00c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f239fe00640 1 -- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a00b99b0 msgr2=0x7f23a00d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f239fe00640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a00b99b0 0x7f23a00d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.096+0000 7f239fe00640 1 -- 172.21.15.60:0/2529953456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a01f8010 con 0x7f23a01586f0 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.097+0000 7f239fe00640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23a01586f0 0x7f23a00b9470 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f238c011930 tx=0x7f238c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.097+0000 7f239cc00640 1 -- 172.21.15.60:0/2529953456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c00fe10 con 0x7f23a01586f0 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.097+0000 7f239cc00640 1 -- 172.21.15.60:0/2529953456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f238c004020 con 0x7f23a01586f0 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.097+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a01f82f0 con 0x7f23a01586f0 2024-09-16T15:42:36.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.097+0000 7f239cc00640 1 -- 172.21.15.60:0/2529953456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c004470 con 0x7f23a01586f0 2024-09-16T15:42:36.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.097+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a01f88b0 con 0x7f23a01586f0 2024-09-16T15:42:36.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.098+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a00d1d60 con 0x7f23a01586f0 2024-09-16T15:42:36.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.099+0000 7f239cc00640 1 -- 172.21.15.60:0/2529953456 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f238c021020 con 0x7f23a01586f0 2024-09-16T15:42:36.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.100+0000 7f239cc00640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2380078be0 0x7f238007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:36.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.100+0000 7f239cc00640 1 -- 172.21.15.60:0/2529953456 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f238c09b5d0 con 0x7f23a01586f0 2024-09-16T15:42:36.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.101+0000 7f23a4c00640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2380078be0 0x7f238007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:36.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.102+0000 7f23a4c00640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2380078be0 0x7f238007b0a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2394032000 tx=0x7f2394001190 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:36.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.104+0000 7f239cc00640 1 -- 172.21.15.60:0/2529953456 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f238c063390 con 0x7f23a01586f0 2024-09-16T15:42:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:36.373+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f23a006e1c0 con 0x7f23a01586f0 2024-09-16T15:42:37.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.143+0000 7f239cc00640 1 -- 172.21.15.60:0/2529953456 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v226) ==== 134+0+0 (secure 0 0 0) 0x7f238c067dc0 con 0x7f23a01586f0 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.146+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2380078be0 msgr2=0x7f238007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.146+0000 7f23a6000640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2380078be0 0x7f238007b0a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2394032000 tx=0x7f2394001190 comp rx=0 tx=0).stop 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.146+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23a01586f0 msgr2=0x7f23a00b9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23a01586f0 0x7f23a00b9470 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f238c011930 tx=0x7f238c011df0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 shutdown_connections 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f2380078be0 0x7f238007b0a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23a00b99b0 0x7f23a00d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23a01586f0 0x7f23a00b9470 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 --2- 172.21.15.60:0/2529953456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2388003a50 0x7f23a00c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 >> 172.21.15.60:0/2529953456 conn(0x7f23a00d3df0 msgr2=0x7f23a006e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:37.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 shutdown_connections 2024-09-16T15:42:37.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.147+0000 7f23a6000640 1 -- 172.21.15.60:0/2529953456 wait complete. 2024-09-16T15:42:37.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:42:37.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.330+0000 7fc4f8400640 1 -- 172.21.15.60:0/1925023826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 msgr2=0x7fc4f0074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.330+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1925023826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 0x7fc4f0074440 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8005fd0 tx=0x7fc4e802aa10 comp rx=0 tx=0).stop 2024-09-16T15:42:37.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.331+0000 7fc4f8400640 1 -- 172.21.15.60:0/1925023826 shutdown_connections 2024-09-16T15:42:37.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.331+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1925023826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 0x7fc4f0074440 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.331+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1925023826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4f00ccf00 0x7fc4f00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.331+0000 7fc4f8400640 1 -- 172.21.15.60:0/1925023826 >> 172.21.15.60:0/1925023826 conn(0x7fc4f00b1950 msgr2=0x7fc4f00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:37.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.331+0000 7fc4f8400640 1 -- 172.21.15.60:0/1925023826 shutdown_connections 2024-09-16T15:42:37.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.331+0000 7fc4f8400640 1 -- 172.21.15.60:0/1925023826 wait complete. 2024-09-16T15:42:37.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.332+0000 7fc4f8400640 1 Processor -- start 2024-09-16T15:42:37.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.332+0000 7fc4f8400640 1 -- start start 2024-09-16T15:42:37.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f8400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 0x7fc4f00cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f8400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ccf00 0x7fc4f01f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f8400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4f015f5a0 0x7fc4f00ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f8400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f006f2e0 con 0x7fc4f0073fb0 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f8400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f006f160 con 0x7fc4f00ccf00 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f8400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f006f460 con 0x7fc4f015f5a0 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 0x7fc4f00cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 0x7fc4f00cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35670/0 (socket says 172.21.15.60:35670) 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.333+0000 7fc4f5a00640 1 -- 172.21.15.60:0/1646335996 learned_addr learned my addr 172.21.15.60:0/1646335996 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f6400640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4f015f5a0 0x7fc4f00ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f5000640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ccf00 0x7fc4f01f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f5a00640 1 -- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4f015f5a0 msgr2=0x7fc4f00ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f5a00640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4f015f5a0 0x7fc4f00ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f5a00640 1 -- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ccf00 msgr2=0x7fc4f01f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f5a00640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ccf00 0x7fc4f01f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f5a00640 1 -- 172.21.15.60:0/1646335996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f01f8d10 con 0x7fc4f0073fb0 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f5000640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ccf00 0x7fc4f01f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4f5a00640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 0x7fc4f00cdae0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c3a0 tx=0x7fc4e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4ed600640 1 -- 172.21.15.60:0/1646335996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4017070 con 0x7fc4f0073fb0 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4ed600640 1 -- 172.21.15.60:0/1646335996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4e40053c0 con 0x7fc4f0073fb0 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.334+0000 7fc4ed600640 1 -- 172.21.15.60:0/1646335996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4003850 con 0x7fc4f0073fb0 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.335+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f01f4bc0 con 0x7fc4f0073fb0 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.335+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f01f5180 con 0x7fc4f0073fb0 2024-09-16T15:42:37.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.336+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f006f9f0 con 0x7fc4f0073fb0 2024-09-16T15:42:37.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.337+0000 7fc4ed600640 1 -- 172.21.15.60:0/1646335996 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fc4e401a070 con 0x7fc4f0073fb0 2024-09-16T15:42:37.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.338+0000 7fc4ed600640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4d0078bb0 0x7fc4d007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.339+0000 7fc4ed600640 1 -- 172.21.15.60:0/1646335996 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7fc4e409a000 con 0x7fc4f0073fb0 2024-09-16T15:42:37.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.339+0000 7fc4f5000640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4d0078bb0 0x7fc4d007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.340+0000 7fc4f5000640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4d0078bb0 0x7fc4d007b070 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01eb8c0 tx=0x7fc4dc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:37.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.343+0000 7fc4ed600640 1 -- 172.21.15.60:0/1646335996 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc4f01f8d10 con 0x7fc4f0073fb0 2024-09-16T15:42:37.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.537+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc4f00ae830 con 0x7fc4f0073fb0 2024-09-16T15:42:37.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.538+0000 7fc4ed600640 1 -- 172.21.15.60:0/1646335996 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7fc4e4062410 con 0x7fc4f0073fb0 2024-09-16T15:42:37.540 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:42:37.540 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:42:37.540 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:42:37.540 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:42:37.541 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:42:37.542 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:42:37.542 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:42:37.542 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:42:37.542 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:42:37.542 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:42:37.542 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:42:37.542 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:42:37.542 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.540+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4d0078bb0 msgr2=0x7fc4d007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.540+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4d0078bb0 0x7fc4d007b070 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01eb8c0 tx=0x7fc4dc005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.540+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 msgr2=0x7fc4f00cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.540+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 0x7fc4f00cdae0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c3a0 tx=0x7fc4e400c860 comp rx=0 tx=0).stop 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.541+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 shutdown_connections 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.541+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc4d0078bb0 0x7fc4d007b070 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.541+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4f015f5a0 0x7fc4f00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.541+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4f00ccf00 0x7fc4f01f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.541+0000 7fc4f8400640 1 --2- 172.21.15.60:0/1646335996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4f0073fb0 0x7fc4f00cdae0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.541+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 >> 172.21.15.60:0/1646335996 conn(0x7fc4f00b1950 msgr2=0x7fc4f009c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:37.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.541+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 shutdown_connections 2024-09-16T15:42:37.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.541+0000 7fc4f8400640 1 -- 172.21.15.60:0/1646335996 wait complete. 2024-09-16T15:42:37.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:42:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.720+0000 7f10dfa00640 1 -- 172.21.15.60:0/1362633747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d8145750 msgr2=0x7f10d8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.721+0000 7f10dfa00640 1 --2- 172.21.15.60:0/1362633747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d8145750 0x7f10d8073440 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f10c8005fd0 tx=0x7f10c802a540 comp rx=0 tx=0).stop 2024-09-16T15:42:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.721+0000 7f10dfa00640 1 -- 172.21.15.60:0/1362633747 shutdown_connections 2024-09-16T15:42:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.721+0000 7f10dfa00640 1 --2- 172.21.15.60:0/1362633747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 0x7f10d80cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.721+0000 7f10dfa00640 1 --2- 172.21.15.60:0/1362633747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d8145750 0x7f10d8073440 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.721+0000 7f10dfa00640 1 -- 172.21.15.60:0/1362633747 >> 172.21.15.60:0/1362633747 conn(0x7f10d80b1840 msgr2=0x7f10d80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.721+0000 7f10dfa00640 1 -- 172.21.15.60:0/1362633747 shutdown_connections 2024-09-16T15:42:37.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.721+0000 7f10dfa00640 1 -- 172.21.15.60:0/1362633747 wait complete. 2024-09-16T15:42:37.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.722+0000 7f10dfa00640 1 Processor -- start 2024-09-16T15:42:37.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.722+0000 7f10dfa00640 1 -- start start 2024-09-16T15:42:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.723+0000 7f10dfa00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 0x7f10d815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.723+0000 7f10dfa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d80ccb00 0x7f10d8136410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.723+0000 7f10dfa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8145750 0x7f10d80cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.723+0000 7f10dfa00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80b26b0 con 0x7f10d80ccb00 2024-09-16T15:42:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.723+0000 7f10dfa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80cedd0 con 0x7f10d8145750 2024-09-16T15:42:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.723+0000 7f10dfa00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f10d8160100 con 0x7f10d806efa0 2024-09-16T15:42:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d80ccb00 0x7f10d8136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d80ccb00 0x7f10d8136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35702/0 (socket says 172.21.15.60:35702) 2024-09-16T15:42:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 -- 172.21.15.60:0/4132939124 learned_addr learned my addr 172.21.15.60:0/4132939124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10dd000640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 0x7f10d815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10dda00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8145750 0x7f10d80cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 -- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 msgr2=0x7f10d815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 0x7f10d815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 -- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8145750 msgr2=0x7f10d80cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8145750 0x7f10d80cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 -- 172.21.15.60:0/4132939124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d815ee00 con 0x7f10d80ccb00 2024-09-16T15:42:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.724+0000 7f10cfe00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d80ccb00 0x7f10d8136410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f10c000d950 tx=0x7f10c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.725+0000 7f10d5600640 1 -- 172.21.15.60:0/4132939124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0017070 con 0x7f10d80ccb00 2024-09-16T15:42:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.725+0000 7f10d5600640 1 -- 172.21.15.60:0/4132939124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10c0009d20 con 0x7f10d80ccb00 2024-09-16T15:42:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.725+0000 7f10d5600640 1 -- 172.21.15.60:0/4132939124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0004280 con 0x7f10d80ccb00 2024-09-16T15:42:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.725+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d80b2de0 con 0x7f10d80ccb00 2024-09-16T15:42:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.725+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d80b33a0 con 0x7f10d80ccb00 2024-09-16T15:42:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.726+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d81605b0 con 0x7f10d80ccb00 2024-09-16T15:42:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.727+0000 7f10d5600640 1 -- 172.21.15.60:0/4132939124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f10c00035f0 con 0x7f10d80ccb00 2024-09-16T15:42:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.728+0000 7f10d5600640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10b4078be0 0x7f10b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.728+0000 7f10d5600640 1 -- 172.21.15.60:0/4132939124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f10c009ae30 con 0x7f10d80ccb00 2024-09-16T15:42:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.728+0000 7f10dd000640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10b4078be0 0x7f10b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.729+0000 7f10dd000640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10b4078be0 0x7f10b407b0a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f10c8033000 tx=0x7f10c8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.732+0000 7f10d5600640 1 -- 172.21.15.60:0/4132939124 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f10c0062af0 con 0x7f10d80ccb00 2024-09-16T15:42:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.928+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f10d80aa100 con 0x7f10d80ccb00 2024-09-16T15:42:37.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.931+0000 7f10d5600640 1 -- 172.21.15.60:0/4132939124 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f10c0067520 con 0x7f10d80ccb00 2024-09-16T15:42:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10b4078be0 msgr2=0x7f10b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10b4078be0 0x7f10b407b0a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f10c8033000 tx=0x7f10c8004730 comp rx=0 tx=0).stop 2024-09-16T15:42:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d80ccb00 msgr2=0x7f10d8136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d80ccb00 0x7f10d8136410 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f10c000d950 tx=0x7f10c000de10 comp rx=0 tx=0).stop 2024-09-16T15:42:37.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 shutdown_connections 2024-09-16T15:42:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f10b4078be0 0x7f10b407b0a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8145750 0x7f10d80cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d80ccb00 0x7f10d8136410 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 --2- 172.21.15.60:0/4132939124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 0x7f10d815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 >> 172.21.15.60:0/4132939124 conn(0x7f10d80b1840 msgr2=0x7f10d806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 shutdown_connections 2024-09-16T15:42:37.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:37.934+0000 7f10dfa00640 1 -- 172.21.15.60:0/4132939124 wait complete. 2024-09-16T15:42:37.952 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:42:37.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:42:38.043 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:42:38.148 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:42:38.148 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:42:38.148 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:38.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:42:38.184 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:42:38.184 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:38.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:42:38.221 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:42:38.221 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:38.259 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:42:38.259 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:42:38.259 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:38.297 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:42:38.297 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:42:38.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:38.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.457+0000 7f81cea00640 1 -- 172.21.15.60:0/3850309207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c80af3d0 msgr2=0x7f81c806f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.457+0000 7f81cea00640 1 --2- 172.21.15.60:0/3850309207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c80af3d0 0x7f81c806f660 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f81b0005fd0 tx=0x7f81b002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.457+0000 7f81cea00640 1 -- 172.21.15.60:0/3850309207 shutdown_connections 2024-09-16T15:42:38.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.457+0000 7f81cea00640 1 --2- 172.21.15.60:0/3850309207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c80af3d0 0x7f81c806f660 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.457+0000 7f81cea00640 1 -- 172.21.15.60:0/3850309207 >> 172.21.15.60:0/3850309207 conn(0x7f81c80b1950 msgr2=0x7f81c80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:38.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.457+0000 7f81cea00640 1 -- 172.21.15.60:0/3850309207 shutdown_connections 2024-09-16T15:42:38.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.458+0000 7f81cea00640 1 -- 172.21.15.60:0/3850309207 wait complete. 2024-09-16T15:42:38.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.459+0000 7f81cea00640 1 Processor -- start 2024-09-16T15:42:38.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.459+0000 7f81cea00640 1 -- start start 2024-09-16T15:42:38.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.460+0000 7f81cea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81c8072df0 0x7f81c80c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.460+0000 7f81cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c80af3d0 0x7f81c80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.460+0000 7f81cea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c8161550 0x7f81c814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.460+0000 7f81cea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8073950 con 0x7f81c8161550 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.460+0000 7f81cea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81c806fdb0 con 0x7f81c80af3d0 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.460+0000 7f81cea00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f81c8073ad0 con 0x7f81c8072df0 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.460+0000 7f81cca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c8161550 0x7f81c814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81cca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c8161550 0x7f81c814d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35720/0 (socket says 172.21.15.60:35720) 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81bf400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c80af3d0 0x7f81c80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81cca00640 1 -- 172.21.15.60:0/3996242247 learned_addr learned my addr 172.21.15.60:0/3996242247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81c7e00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81c8072df0 0x7f81c80c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81bf400640 1 -- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81c8072df0 msgr2=0x7f81c80c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81bf400640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81c8072df0 0x7f81c80c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81bf400640 1 -- 172.21.15.60:0/3996242247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c8161550 msgr2=0x7f81c814d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81bf400640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c8161550 0x7f81c814d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81bf400640 1 -- 172.21.15.60:0/3996242247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c809aba0 con 0x7f81c80af3d0 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81cca00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c8161550 0x7f81c814d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.461+0000 7f81bf400640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c80af3d0 0x7f81c80cc6e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81b0031000 tx=0x7f81b00039e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.462+0000 7f81c4c00640 1 -- 172.21.15.60:0/3996242247 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b003f070 con 0x7f81c80af3d0 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.462+0000 7f81c4c00640 1 -- 172.21.15.60:0/3996242247 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81b0004890 con 0x7f81c80af3d0 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.462+0000 7f81c4c00640 1 -- 172.21.15.60:0/3996242247 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0035030 con 0x7f81c80af3d0 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.462+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c809ae80 con 0x7f81c80af3d0 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.462+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c80ca2b0 con 0x7f81c80af3d0 2024-09-16T15:42:38.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.463+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c8073fc0 con 0x7f81c80af3d0 2024-09-16T15:42:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.464+0000 7f81c4c00640 1 -- 172.21.15.60:0/3996242247 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f81b0042070 con 0x7f81c80af3d0 2024-09-16T15:42:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.465+0000 7f81c4c00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81a4078be0 0x7f81a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.465+0000 7f81c4c00640 1 -- 172.21.15.60:0/3996242247 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f81b00bbca0 con 0x7f81c80af3d0 2024-09-16T15:42:38.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.466+0000 7f81c7e00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81a4078be0 0x7f81a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.466+0000 7f81c7e00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81a4078be0 0x7f81a407b0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f81c80b4890 tx=0x7f81b8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:38.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.469+0000 7f81c4c00640 1 -- 172.21.15.60:0/3996242247 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f81b0083b70 con 0x7f81c80af3d0 2024-09-16T15:42:38.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.701+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81c81628e0 con 0x7f81c80af3d0 2024-09-16T15:42:38.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.702+0000 7f81c4c00640 1 -- 172.21.15.60:0/3996242247 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3911 (secure 0 0 0) 0x7f81b00885a0 con 0x7f81c80af3d0 2024-09-16T15:42:38.704 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:38.704 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"btime":"2024-09-16T15:42:32:123652+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5522,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/934892309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":934892309},{"type":"v1","addr":"172.21.15.100:6836","nonce":934892309}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5671,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3625257481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3625257481},{"type":"v1","addr":"172.21.15.60:6834","nonce":3625257481}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5680,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2519251073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2519251073},{"type":"v1","addr":"172.21.15.100:6837","nonce":2519251073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5751,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/333738926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":333738926},{"type":"v1","addr":"172.21.15.60:6835","nonce":333738926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-09-16T15:42:38.704 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 39 2024-09-16T15:42:38.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81a4078be0 msgr2=0x7f81a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81a4078be0 0x7f81a407b0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f81c80b4890 tx=0x7f81b8009040 comp rx=0 tx=0).stop 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c80af3d0 msgr2=0x7f81c80cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c80af3d0 0x7f81c80cc6e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f81b0031000 tx=0x7f81b00039e0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 shutdown_connections 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81a4078be0 0x7f81a407b0a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81c8161550 0x7f81c814d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81c80af3d0 0x7f81c80cc6e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 --2- 172.21.15.60:0/3996242247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81c8072df0 0x7f81c80c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 >> 172.21.15.60:0/3996242247 conn(0x7f81c80b1950 msgr2=0x7f81c815ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 shutdown_connections 2024-09-16T15:42:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:38.704+0000 7f81cea00640 1 -- 172.21.15.60:0/3996242247 wait complete. 2024-09-16T15:42:38.723 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:42:38.723 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:42:38.723 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:42:38.723 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:42:38.723 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:42:38.723 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:42:38.723 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:42:38.724 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:42:38.724 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:42:38.724 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:42:38.724 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:42:38.724 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:42:38.724 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:42:38.724 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:42:38.725 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:42:38.725 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:42:38.737 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:42:38.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:42:38.740 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:42:38.741 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:42:38.741 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:42:38.741 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:42:38.782 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:42:38.782 DEBUG:teuthology.parallel:result is None 2024-09-16T15:42:38.785 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:42:38.785 DEBUG:teuthology.parallel:result is None 2024-09-16T15:42:38.788 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:42:38.788 DEBUG:teuthology.parallel:result is None 2024-09-16T15:42:38.909 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:42:38.918 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:42:38.918 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:42:40.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-16T15:42:44.738 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:42:44.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:42:44.743 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:42:44.743 DEBUG:teuthology.parallel:result is None 2024-09-16T15:42:44.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:42:44.883 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:42:44.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.916+0000 7f7b8f400640 1 -- 172.21.15.60:0/4183487376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880ccfd0 msgr2=0x7f7b88072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:44.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.916+0000 7f7b8f400640 1 --2- 172.21.15.60:0/4183487376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880ccfd0 0x7f7b88072ee0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009ca0 tx=0x7f7b8002ab50 comp rx=0 tx=0).stop 2024-09-16T15:42:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.916+0000 7f7b8f400640 1 -- 172.21.15.60:0/4183487376 shutdown_connections 2024-09-16T15:42:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.916+0000 7f7b8f400640 1 --2- 172.21.15.60:0/4183487376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880ccfd0 0x7f7b88072ee0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.916+0000 7f7b8f400640 1 --2- 172.21.15.60:0/4183487376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b880cf300 0x7f7b880729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.916+0000 7f7b8f400640 1 -- 172.21.15.60:0/4183487376 >> 172.21.15.60:0/4183487376 conn(0x7f7b880b1950 msgr2=0x7f7b880b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.916+0000 7f7b8f400640 1 -- 172.21.15.60:0/4183487376 shutdown_connections 2024-09-16T15:42:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.916+0000 7f7b8f400640 1 -- 172.21.15.60:0/4183487376 wait complete. 2024-09-16T15:42:44.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.918+0000 7f7b8f400640 1 Processor -- start 2024-09-16T15:42:44.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.918+0000 7f7b8f400640 1 -- start start 2024-09-16T15:42:44.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b880ccfd0 0x7f7b8813d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:44.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880cf300 0x7f7b881f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:44.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b881445d0 0x7f7b881efc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b8f400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88073420 con 0x7f7b880cf300 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b8f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880b4eb0 con 0x7f7b881445d0 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b8f400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7b880735a0 con 0x7f7b880ccfd0 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b7fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880cf300 0x7f7b881f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b7fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880cf300 0x7f7b881f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37118/0 (socket says 172.21.15.60:37118) 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b7fe00640 1 -- 172.21.15.60:0/3428439474 learned_addr learned my addr 172.21.15.60:0/3428439474 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b8ca00640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b880ccfd0 0x7f7b8813d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.919+0000 7f7b8d400640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b881445d0 0x7f7b881efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b7fe00640 1 -- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b880ccfd0 msgr2=0x7f7b8813d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b7fe00640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b880ccfd0 0x7f7b8813d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b7fe00640 1 -- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b881445d0 msgr2=0x7f7b881efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:44.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b7fe00640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b881445d0 0x7f7b881efc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:44.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b7fe00640 1 -- 172.21.15.60:0/3428439474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b881f6d40 con 0x7f7b880cf300 2024-09-16T15:42:44.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b7fe00640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880cf300 0x7f7b881f6800 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000d950 tx=0x7f7b7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:44.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b85600640 1 -- 172.21.15.60:0/3428439474 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70017070 con 0x7f7b880cf300 2024-09-16T15:42:44.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b85600640 1 -- 172.21.15.60:0/3428439474 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b70009d20 con 0x7f7b880cf300 2024-09-16T15:42:44.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b85600640 1 -- 172.21.15.60:0/3428439474 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b700106c0 con 0x7f7b880cf300 2024-09-16T15:42:44.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b881f29a0 con 0x7f7b880cf300 2024-09-16T15:42:44.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.920+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881f2f60 con 0x7f7b880cf300 2024-09-16T15:42:44.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.921+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b88073a80 con 0x7f7b880cf300 2024-09-16T15:42:44.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.923+0000 7f7b85600640 1 -- 172.21.15.60:0/3428439474 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f7b70010860 con 0x7f7b880cf300 2024-09-16T15:42:44.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.924+0000 7f7b85600640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7b64078bb0 0x7f7b6407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:44.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.924+0000 7f7b85600640 1 -- 172.21.15.60:0/3428439474 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f7b7009b070 con 0x7f7b880cf300 2024-09-16T15:42:44.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.924+0000 7f7b8ca00640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7b64078bb0 0x7f7b6407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:44.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.925+0000 7f7b8ca00640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7b64078bb0 0x7f7b6407b070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7b78000f60 tx=0x7f7b78009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:44.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:44.928+0000 7f7b85600640 1 -- 172.21.15.60:0/3428439474 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7b881f6d40 con 0x7f7b880cf300 2024-09-16T15:42:45.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:45.137+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7b880ce740 con 0x7f7b880cf300 2024-09-16T15:42:46.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.175+0000 7f7b85600640 1 -- 172.21.15.60:0/3428439474 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) ==== 97+0+0 (secure 0 0 0) 0x7f7b70062dc0 con 0x7f7b880cf300 2024-09-16T15:42:46.178 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:42:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7b64078bb0 msgr2=0x7f7b6407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7b64078bb0 0x7f7b6407b070 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7b78000f60 tx=0x7f7b78009040 comp rx=0 tx=0).stop 2024-09-16T15:42:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880cf300 msgr2=0x7f7b881f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880cf300 0x7f7b881f6800 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000d950 tx=0x7f7b7000de10 comp rx=0 tx=0).stop 2024-09-16T15:42:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 shutdown_connections 2024-09-16T15:42:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7b64078bb0 0x7f7b6407b070 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b881445d0 0x7f7b881efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b880cf300 0x7f7b881f6800 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 --2- 172.21.15.60:0/3428439474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b880ccfd0 0x7f7b8813d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 >> 172.21.15.60:0/3428439474 conn(0x7f7b880b1950 msgr2=0x7f7b88072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 shutdown_connections 2024-09-16T15:42:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.180+0000 7f7b8f400640 1 -- 172.21.15.60:0/3428439474 wait complete. 2024-09-16T15:42:46.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:42:46.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.365+0000 7f7043e00640 1 -- 172.21.15.60:0/3935617084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 msgr2=0x7f703c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.365+0000 7f7043e00640 1 --2- 172.21.15.60:0/3935617084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 0x7f703c0b2440 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f703c06dac0 tx=0x7f702802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.365+0000 7f7043e00640 1 -- 172.21.15.60:0/3935617084 shutdown_connections 2024-09-16T15:42:46.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.365+0000 7f7043e00640 1 --2- 172.21.15.60:0/3935617084 >> v2:172.21.15.100:3300/0 conn(0x7f703c073fb0 0x7f703c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.365+0000 7f7043e00640 1 --2- 172.21.15.60:0/3935617084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 0x7f703c0b2440 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.365+0000 7f7043e00640 1 -- 172.21.15.60:0/3935617084 >> 172.21.15.60:0/3935617084 conn(0x7f703c0b1950 msgr2=0x7f703c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:46.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.365+0000 7f7043e00640 1 -- 172.21.15.60:0/3935617084 shutdown_connections 2024-09-16T15:42:46.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.365+0000 7f7043e00640 1 -- 172.21.15.60:0/3935617084 wait complete. 2024-09-16T15:42:46.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.367+0000 7f7043e00640 1 Processor -- start 2024-09-16T15:42:46.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.367+0000 7f7043e00640 1 -- start start 2024-09-16T15:42:46.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7043e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c073fb0 0x7f703c156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:46.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7043e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 0x7f703c14c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:46.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7043e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f703c15f5a0 0x7f703c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:46.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7043e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f703c06f2e0 con 0x7f703c0ccf00 2024-09-16T15:42:46.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7043e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f703c06f160 con 0x7f703c073fb0 2024-09-16T15:42:46.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7043e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f703c06f460 con 0x7f703c15f5a0 2024-09-16T15:42:46.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7040a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 0x7f703c14c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:46.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7040a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 0x7f703c14c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37140/0 (socket says 172.21.15.60:37140) 2024-09-16T15:42:46.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.368+0000 7f7040a00640 1 -- 172.21.15.60:0/3275516407 learned_addr learned my addr 172.21.15.60:0/3275516407 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7040a00640 1 -- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f703c15f5a0 msgr2=0x7f703c0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7040a00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f703c15f5a0 0x7f703c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7040a00640 1 -- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c073fb0 msgr2=0x7f703c156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7041400640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c073fb0 0x7f703c156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7040a00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c073fb0 0x7f703c156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7040a00640 1 -- 172.21.15.60:0/3275516407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c1f6a60 con 0x7f703c0ccf00 2024-09-16T15:42:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7041400640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c073fb0 0x7f703c156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7040a00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 0x7f703c14c210 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f702802aa90 tx=0x7f7028032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7039600640 1 -- 172.21.15.60:0/3275516407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028040070 con 0x7f703c0ccf00 2024-09-16T15:42:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7039600640 1 -- 172.21.15.60:0/3275516407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7028005310 con 0x7f703c0ccf00 2024-09-16T15:42:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7039600640 1 -- 172.21.15.60:0/3275516407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280047a0 con 0x7f703c0ccf00 2024-09-16T15:42:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c1f6d40 con 0x7f703c0ccf00 2024-09-16T15:42:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.369+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c156410 con 0x7f703c0ccf00 2024-09-16T15:42:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.370+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c06f9f0 con 0x7f703c0ccf00 2024-09-16T15:42:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.371+0000 7f7039600640 1 -- 172.21.15.60:0/3275516407 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f7028039040 con 0x7f703c0ccf00 2024-09-16T15:42:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.372+0000 7f7039600640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f701c078bb0 0x7f701c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.373+0000 7f7039600640 1 -- 172.21.15.60:0/3275516407 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4857+0+0 (secure 0 0 0) 0x7f70280bbfe0 con 0x7f703c0ccf00 2024-09-16T15:42:46.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.373+0000 7f7041400640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f701c078bb0 0x7f701c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.374+0000 7f7041400640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f701c078bb0 0x7f701c07b070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7030000f60 tx=0x7f7030009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:46.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.377+0000 7f7039600640 1 -- 172.21.15.60:0/3275516407 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7028084440 con 0x7f703c0ccf00 2024-09-16T15:42:46.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.643+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f703c155ce0 con 0x7f703c0ccf00 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.644+0000 7f7039600640 1 -- 172.21.15.60:0/3275516407 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5617 (secure 0 0 0) 0x7f7028083d40 con 0x7f703c0ccf00 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.646 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:42:46.647 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.649 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:46.650 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.651 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:42:46.652 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:42:46.653 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.654 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:42:46.656 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:42:46.657 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.alice", 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUehmgYz4LRAA9MddHC7F/dmFVD0at+o5Ww==", 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol75450", 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:42:46.658 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:42:46.659 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:42:46.660 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:42:46.660 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:42:46.660 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:42:46.660 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.660 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:46.660 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:42:46.660 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.646+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f701c078bb0 msgr2=0x7f701c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.646+0000 7f7043e00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f701c078bb0 0x7f701c07b070 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7030000f60 tx=0x7f7030009040 comp rx=0 tx=0).stop 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.646+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 msgr2=0x7f703c14c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.646+0000 7f7043e00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 0x7f703c14c210 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f702802aa90 tx=0x7f7028032c10 comp rx=0 tx=0).stop 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.647+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 shutdown_connections 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.647+0000 7f7043e00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f701c078bb0 0x7f701c07b070 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.647+0000 7f7043e00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f703c15f5a0 0x7f703c0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.647+0000 7f7043e00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703c0ccf00 0x7f703c14c210 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.647+0000 7f7043e00640 1 --2- 172.21.15.60:0/3275516407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f703c073fb0 0x7f703c156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.647+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 >> 172.21.15.60:0/3275516407 conn(0x7f703c0b1950 msgr2=0x7f703c0b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.647+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 shutdown_connections 2024-09-16T15:42:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.647+0000 7f7043e00640 1 -- 172.21.15.60:0/3275516407 wait complete. 2024-09-16T15:42:46.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-16T15:42:46.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.825+0000 7f8e9d200640 1 -- 172.21.15.60:0/1488965256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7c004800 msgr2=0x7f8e7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.825+0000 7f8e9d200640 1 --2- 172.21.15.60:0/1488965256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e80001830 tx=0x7f8e7c005830 comp rx=0 tx=0).stop 2024-09-16T15:42:46.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.825+0000 7f8e96800640 1 -- 172.21.15.60:0/1488965256 reap_dead start 2024-09-16T15:42:46.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.825+0000 7f8e9d200640 1 -- 172.21.15.60:0/1488965256 shutdown_connections 2024-09-16T15:42:46.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.825+0000 7f8e9d200640 1 -- 172.21.15.60:0/1488965256 >> 172.21.15.60:0/1488965256 conn(0x7f8e980b1950 msgr2=0x7f8e980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:46.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.825+0000 7f8e9d200640 1 -- 172.21.15.60:0/1488965256 shutdown_connections 2024-09-16T15:42:46.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.825+0000 7f8e9d200640 1 -- 172.21.15.60:0/1488965256 wait complete. 2024-09-16T15:42:46.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.826+0000 7f8e9d200640 1 Processor -- start 2024-09-16T15:42:46.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.827+0000 7f8e9d200640 1 -- start start 2024-09-16T15:42:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e9d200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7c004800 0x7f8e9809c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e9d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e980cc700 0x7f8e9815e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e9d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e9815e640 0x7f8e981f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e9d200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8e98074840 con 0x7f8e9815e640 2024-09-16T15:42:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e9d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e980746c0 con 0x7f8e980cc700 2024-09-16T15:42:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e9d200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9806f160 con 0x7f8e7c004800 2024-09-16T15:42:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e96800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e9815e640 0x7f8e981f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:46.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e96800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e9815e640 0x7f8e981f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37150/0 (socket says 172.21.15.60:37150) 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e96800640 1 -- 172.21.15.60:0/2569769638 learned_addr learned my addr 172.21.15.60:0/2569769638 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e8d400640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e980cc700 0x7f8e9815e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e95e00640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7c004800 0x7f8e9809c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e96800640 1 -- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7c004800 msgr2=0x7f8e9809c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e96800640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7c004800 0x7f8e9809c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e96800640 1 -- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e980cc700 msgr2=0x7f8e9815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e96800640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e980cc700 0x7f8e9815e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.828+0000 7f8e96800640 1 -- 172.21.15.60:0/2569769638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9809cbf0 con 0x7f8e9815e640 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.829+0000 7f8e96800640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e9815e640 0x7f8e981f7cf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8e980b2820 tx=0x7f8e9000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.829+0000 7f8e8ea00640 1 -- 172.21.15.60:0/2569769638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90017070 con 0x7f8e9815e640 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.829+0000 7f8e8ea00640 1 -- 172.21.15.60:0/2569769638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e90015070 con 0x7f8e9815e640 2024-09-16T15:42:46.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.829+0000 7f8e8ea00640 1 -- 172.21.15.60:0/2569769638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9000eea0 con 0x7f8e9815e640 2024-09-16T15:42:46.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.829+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9809ad30 con 0x7f8e9815e640 2024-09-16T15:42:46.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.829+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9809b2f0 con 0x7f8e9815e640 2024-09-16T15:42:46.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.830+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9806f650 con 0x7f8e9815e640 2024-09-16T15:42:46.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.831+0000 7f8e8ea00640 1 -- 172.21.15.60:0/2569769638 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f8e90011440 con 0x7f8e9815e640 2024-09-16T15:42:46.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.832+0000 7f8e8ea00640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e74078bb0 0x7f8e7407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:46.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.832+0000 7f8e8ea00640 1 -- 172.21.15.60:0/2569769638 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4857+0+0 (secure 0 0 0) 0x7f8e9009b2d0 con 0x7f8e9815e640 2024-09-16T15:42:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.833+0000 7f8e95e00640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e74078bb0 0x7f8e7407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:46.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.834+0000 7f8e95e00640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e74078bb0 0x7f8e7407b070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8e88001860 tx=0x7f8e880011a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:46.836+0000 7f8e8ea00640 1 -- 172.21.15.60:0/2569769638 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8e90062fe0 con 0x7f8e9815e640 2024-09-16T15:42:47.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.106+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7f8e9815cb90 con 0x7f8e9815e640 2024-09-16T15:42:47.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.109+0000 7f8e8ea00640 1 -- 172.21.15.60:0/2569769638 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v13) ==== 82+0+0 (secure 0 0 0) 0x7f8e90067a10 con 0x7f8e9815e640 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.111+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e74078bb0 msgr2=0x7f8e7407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.111+0000 7f8e9d200640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e74078bb0 0x7f8e7407b070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8e88001860 tx=0x7f8e880011a0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.111+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e9815e640 msgr2=0x7f8e981f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.111+0000 7f8e9d200640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e9815e640 0x7f8e981f7cf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8e980b2820 tx=0x7f8e9000e720 comp rx=0 tx=0).stop 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.112+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 shutdown_connections 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.112+0000 7f8e9d200640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8e74078bb0 0x7f8e7407b070 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.112+0000 7f8e9d200640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e9815e640 0x7f8e981f7cf0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.112+0000 7f8e9d200640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e980cc700 0x7f8e9815e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.112+0000 7f8e9d200640 1 --2- 172.21.15.60:0/2569769638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e7c004800 0x7f8e9809c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.112+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 >> 172.21.15.60:0/2569769638 conn(0x7f8e980b1950 msgr2=0x7f8e980b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:47.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.112+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 shutdown_connections 2024-09-16T15:42:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.112+0000 7f8e9d200640 1 -- 172.21.15.60:0/2569769638 wait complete. 2024-09-16T15:42:47.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:42:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.287+0000 7f52a6a00640 1 -- 172.21.15.60:0/3070113710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00d0540 msgr2=0x7f52a006f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.287+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3070113710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00d0540 0x7f52a006f320 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5298005fd0 tx=0x7f529802aaa0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.287+0000 7f52a6a00640 1 -- 172.21.15.60:0/3070113710 shutdown_connections 2024-09-16T15:42:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.287+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3070113710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00d0540 0x7f52a006f320 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.288+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3070113710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52a00725d0 0x7f52a00b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.288+0000 7f52a6a00640 1 -- 172.21.15.60:0/3070113710 >> 172.21.15.60:0/3070113710 conn(0x7f52a00cfa20 msgr2=0x7f52a00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.288+0000 7f52a6a00640 1 -- 172.21.15.60:0/3070113710 shutdown_connections 2024-09-16T15:42:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.288+0000 7f52a6a00640 1 -- 172.21.15.60:0/3070113710 wait complete. 2024-09-16T15:42:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.289+0000 7f52a6a00640 1 Processor -- start 2024-09-16T15:42:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.289+0000 7f52a6a00640 1 -- start start 2024-09-16T15:42:47.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.290+0000 7f52a6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52a00d0540 0x7f52a01f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:47.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.290+0000 7f52a6a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52a015bc30 0x7f52a0150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.290+0000 7f52a6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00ab710 0x7f52a00c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.290+0000 7f52a6a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f52a0074660 con 0x7f52a00ab710 2024-09-16T15:42:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.290+0000 7f52a6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52a006f860 con 0x7f52a00d0540 2024-09-16T15:42:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.290+0000 7f52a6a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f52a014e050 con 0x7f52a015bc30 2024-09-16T15:42:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00ab710 0x7f52a00c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00ab710 0x7f52a00c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37168/0 (socket says 172.21.15.60:37168) 2024-09-16T15:42:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 -- 172.21.15.60:0/3191819713 learned_addr learned my addr 172.21.15.60:0/3191819713 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a4c00640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52a015bc30 0x7f52a0150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a5600640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52a00d0540 0x7f52a01f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 -- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52a015bc30 msgr2=0x7f52a0150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52a015bc30 0x7f52a0150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 -- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52a00d0540 msgr2=0x7f52a01f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52a00d0540 0x7f52a01f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 -- 172.21.15.60:0/3191819713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a01f4c00 con 0x7f52a00ab710 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a5600640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52a00d0540 0x7f52a01f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.291+0000 7f52a6000640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00ab710 0x7f52a00c6af0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5298006010 tx=0x7f5298003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.292+0000 7f529d600640 1 -- 172.21.15.60:0/3191819713 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298046070 con 0x7f52a00ab710 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.292+0000 7f529d600640 1 -- 172.21.15.60:0/3191819713 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f529802e800 con 0x7f52a00ab710 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.292+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a01f4e80 con 0x7f52a00ab710 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.292+0000 7f529d600640 1 -- 172.21.15.60:0/3191819713 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980044b0 con 0x7f52a00ab710 2024-09-16T15:42:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.292+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a01f5440 con 0x7f52a00ab710 2024-09-16T15:42:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.293+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a006fdf0 con 0x7f52a00ab710 2024-09-16T15:42:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.294+0000 7f529d600640 1 -- 172.21.15.60:0/3191819713 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f5298049070 con 0x7f52a00ab710 2024-09-16T15:42:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.295+0000 7f529d600640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5280078bb0 0x7f528007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.295+0000 7f529d600640 1 -- 172.21.15.60:0/3191819713 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4857+0+0 (secure 0 0 0) 0x7f52980bbb60 con 0x7f52a00ab710 2024-09-16T15:42:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.295+0000 7f52a5600640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5280078bb0 0x7f528007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:47.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.296+0000 7f52a5600640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5280078bb0 0x7f528007b070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f52940016a0 tx=0x7f5294009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.299+0000 7f529d600640 1 -- 172.21.15.60:0/3191819713 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f52a01f4c00 con 0x7f52a00ab710 2024-09-16T15:42:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.494+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f52a0158be0 con 0x7f52a00ab710 2024-09-16T15:42:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.494+0000 7f529d600640 1 -- 172.21.15.60:0/3191819713 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7f5298032030 con 0x7f52a00ab710 2024-09-16T15:42:47.496 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:47.496 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:42:47.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:42:47.497 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:42:47.497 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:42:47.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:42:47.497 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5280078bb0 msgr2=0x7f528007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5280078bb0 0x7f528007b070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f52940016a0 tx=0x7f5294009040 comp rx=0 tx=0).stop 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00ab710 msgr2=0x7f52a00c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00ab710 0x7f52a00c6af0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5298006010 tx=0x7f5298003f10 comp rx=0 tx=0).stop 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 shutdown_connections 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5280078bb0 0x7f528007b070 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52a00ab710 0x7f52a00c6af0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52a015bc30 0x7f52a0150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 --2- 172.21.15.60:0/3191819713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52a00d0540 0x7f52a01f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 >> 172.21.15.60:0/3191819713 conn(0x7f52a00cfa20 msgr2=0x7f52a00ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 shutdown_connections 2024-09-16T15:42:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.497+0000 7f52a6a00640 1 -- 172.21.15.60:0/3191819713 wait complete. 2024-09-16T15:42:47.514 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:42:47.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:42:47.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-16T15:42:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.677+0000 7f518ca00640 1 -- 172.21.15.60:0/572149358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51880d0fe0 msgr2=0x7f51880ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.677+0000 7f518ca00640 1 --2- 172.21.15.60:0/572149358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51880d0fe0 0x7f51880ad170 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5174009830 tx=0x7f517402a510 comp rx=0 tx=0).stop 2024-09-16T15:42:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.677+0000 7f518ca00640 1 -- 172.21.15.60:0/572149358 shutdown_connections 2024-09-16T15:42:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.677+0000 7f518ca00640 1 --2- 172.21.15.60:0/572149358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51880d0fe0 0x7f51880ad170 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.677+0000 7f518ca00640 1 --2- 172.21.15.60:0/572149358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f518815de90 0x7f51880aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.677+0000 7f518ca00640 1 -- 172.21.15.60:0/572149358 >> 172.21.15.60:0/572149358 conn(0x7f51880d03e0 msgr2=0x7f51880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.677+0000 7f518ca00640 1 -- 172.21.15.60:0/572149358 shutdown_connections 2024-09-16T15:42:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.677+0000 7f518ca00640 1 -- 172.21.15.60:0/572149358 wait complete. 2024-09-16T15:42:47.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.678+0000 7f518ca00640 1 Processor -- start 2024-09-16T15:42:47.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.679+0000 7f518ca00640 1 -- start start 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.679+0000 7f518ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51880d0fe0 0x7f518815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f518ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f518815de90 0x7f518809a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f518ca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f518815e270 0x7f51880b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f518ca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51880cadc0 con 0x7f518815de90 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f518ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51880cac40 con 0x7f51880d0fe0 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f518ca00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f51880caf40 con 0x7f518815e270 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5186a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f518815de90 0x7f518809a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5186a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f518815de90 0x7f518809a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37186/0 (socket says 172.21.15.60:37186) 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5186a00640 1 -- 172.21.15.60:0/514679014 learned_addr learned my addr 172.21.15.60:0/514679014 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:47.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5187400640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51880d0fe0 0x7f518815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5187e00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f518815e270 0x7f51880b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5186a00640 1 -- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f518815e270 msgr2=0x7f51880b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5186a00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f518815e270 0x7f51880b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5186a00640 1 -- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51880d0fe0 msgr2=0x7f518815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5186a00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51880d0fe0 0x7f518815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5186a00640 1 -- 172.21.15.60:0/514679014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51880af3a0 con 0x7f518815de90 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.680+0000 7f5187400640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51880d0fe0 0x7f518815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.681+0000 7f5186a00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f518815de90 0x7f518809a280 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5174000c00 tx=0x7f5174032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.681+0000 7f517f400640 1 -- 172.21.15.60:0/514679014 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174040070 con 0x7f518815de90 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.681+0000 7f517f400640 1 -- 172.21.15.60:0/514679014 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5174032e70 con 0x7f518815de90 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.681+0000 7f517f400640 1 -- 172.21.15.60:0/514679014 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174003d90 con 0x7f518815de90 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.681+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51880af680 con 0x7f518815de90 2024-09-16T15:42:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.681+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51880afc40 con 0x7f518815de90 2024-09-16T15:42:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.682+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51880cb460 con 0x7f518815de90 2024-09-16T15:42:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.683+0000 7f517f400640 1 -- 172.21.15.60:0/514679014 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f5174039040 con 0x7f518815de90 2024-09-16T15:42:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.685+0000 7f517f400640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5168078bb0 0x7f516807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.685+0000 7f517f400640 1 -- 172.21.15.60:0/514679014 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 4857+0+0 (secure 0 0 0) 0x7f51740bc0e0 con 0x7f518815de90 2024-09-16T15:42:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.685+0000 7f5187400640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5168078bb0 0x7f516807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.686+0000 7f5187400640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5168078bb0 0x7f516807b070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f51780058c0 tx=0x7f5178005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.689+0000 7f517f400640 1 -- 172.21.15.60:0/514679014 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51740845d0 con 0x7f518815de90 2024-09-16T15:42:47.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:47.887+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f51880afe10 con 0x7f518815de90 2024-09-16T15:42:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.198+0000 7f517f400640 1 -- 172.21.15.60:0/514679014 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f5174083ed0 con 0x7f518815de90 2024-09-16T15:42:48.200 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5168078bb0 msgr2=0x7f516807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5168078bb0 0x7f516807b070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f51780058c0 tx=0x7f5178005850 comp rx=0 tx=0).stop 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f518815de90 msgr2=0x7f518809a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f518815de90 0x7f518809a280 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5174000c00 tx=0x7f5174032c80 comp rx=0 tx=0).stop 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 shutdown_connections 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5168078bb0 0x7f516807b070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f518815e270 0x7f51880b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f518815de90 0x7f518809a280 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 --2- 172.21.15.60:0/514679014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51880d0fe0 0x7f518815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 >> 172.21.15.60:0/514679014 conn(0x7f51880d03e0 msgr2=0x7f51880b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 shutdown_connections 2024-09-16T15:42:48.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.202+0000 7f518ca00640 1 -- 172.21.15.60:0/514679014 wait complete. 2024-09-16T15:42:48.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:42:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.390+0000 7f7a3fe00640 1 -- 172.21.15.60:0/1090582575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380cec60 msgr2=0x7f7a380cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.390+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/1090582575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380cec60 0x7f7a380cf070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7a380cfba0 tx=0x7f7a3002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.391+0000 7f7a3fe00640 1 -- 172.21.15.60:0/1090582575 shutdown_connections 2024-09-16T15:42:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.391+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/1090582575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380cec60 0x7f7a380cf070 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.391+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/1090582575 >> v2:172.21.15.100:3300/0 conn(0x7f7a38161760 0x7f7a380ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.391+0000 7f7a3fe00640 1 -- 172.21.15.60:0/1090582575 >> 172.21.15.60:0/1090582575 conn(0x7f7a380b1840 msgr2=0x7f7a380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:48.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.391+0000 7f7a3fe00640 1 -- 172.21.15.60:0/1090582575 shutdown_connections 2024-09-16T15:42:48.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.391+0000 7f7a3fe00640 1 -- 172.21.15.60:0/1090582575 wait complete. 2024-09-16T15:42:48.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.392+0000 7f7a3fe00640 1 Processor -- start 2024-09-16T15:42:48.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.392+0000 7f7a3fe00640 1 -- start start 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.393+0000 7f7a3fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380af190 0x7f7a38144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.393+0000 7f7a3fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a380cec60 0x7f7a381443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.393+0000 7f7a3fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a38161760 0x7f7a380cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.393+0000 7f7a3fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38162310 con 0x7f7a380af190 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.393+0000 7f7a3fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38162190 con 0x7f7a380cec60 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.393+0000 7f7a3fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7a38162490 con 0x7f7a38161760 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380af190 0x7f7a38144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380af190 0x7f7a38144b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37208/0 (socket says 172.21.15.60:37208) 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 -- 172.21.15.60:0/3021747365 learned_addr learned my addr 172.21.15.60:0/3021747365 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:48.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3de00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a38161760 0x7f7a380cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 -- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a38161760 msgr2=0x7f7a380cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a38161760 0x7f7a380cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3ca00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a380cec60 0x7f7a381443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 -- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a380cec60 msgr2=0x7f7a381443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a380cec60 0x7f7a381443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 -- 172.21.15.60:0/3021747365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a381f4d30 con 0x7f7a380af190 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3de00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a38161760 0x7f7a380cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3ca00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a380cec60 0x7f7a381443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.394+0000 7f7a3d400640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380af190 0x7f7a38144b40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00e790 tx=0x7f7a2c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.395+0000 7f7a35600640 1 -- 172.21.15.60:0/3021747365 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00eea0 con 0x7f7a380af190 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.395+0000 7f7a35600640 1 -- 172.21.15.60:0/3021747365 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a2c00cdb0 con 0x7f7a380af190 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.395+0000 7f7a35600640 1 -- 172.21.15.60:0/3021747365 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c010690 con 0x7f7a380af190 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.395+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a381f5010 con 0x7f7a380af190 2024-09-16T15:42:48.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.395+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a381f55d0 con 0x7f7a380af190 2024-09-16T15:42:48.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.396+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a38162980 con 0x7f7a380af190 2024-09-16T15:42:48.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.398+0000 7f7a35600640 1 -- 172.21.15.60:0/3021747365 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f7a2c010830 con 0x7f7a380af190 2024-09-16T15:42:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.399+0000 7f7a35600640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a18078bb0 0x7f7a1807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.399+0000 7f7a35600640 1 -- 172.21.15.60:0/3021747365 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5227+0+0 (secure 0 0 0) 0x7f7a2c09bf80 con 0x7f7a380af190 2024-09-16T15:42:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.399+0000 7f7a3ca00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a18078bb0 0x7f7a1807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:48.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.400+0000 7f7a3ca00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a18078bb0 0x7f7a1807b070 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7a3806f410 tx=0x7f7a24009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:48.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.402+0000 7f7a35600640 1 -- 172.21.15.60:0/3021747365 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7a2c0642f0 con 0x7f7a380af190 2024-09-16T15:42:48.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:48.598+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7a38074000 con 0x7f7a380af190 2024-09-16T15:42:49.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.201+0000 7f7a35600640 1 -- 172.21.15.60:0/3021747365 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7f7a2c063bf0 con 0x7f7a380af190 2024-09-16T15:42:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:42:49.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a18078bb0 msgr2=0x7f7a1807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:49.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a18078bb0 0x7f7a1807b070 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7a3806f410 tx=0x7f7a24009040 comp rx=0 tx=0).stop 2024-09-16T15:42:49.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380af190 msgr2=0x7f7a38144b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:49.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380af190 0x7f7a38144b40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00e790 tx=0x7f7a2c00ec50 comp rx=0 tx=0).stop 2024-09-16T15:42:49.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 shutdown_connections 2024-09-16T15:42:49.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f7a18078bb0 0x7f7a1807b070 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:49.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a38161760 0x7f7a380cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:49.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a380cec60 0x7f7a381443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:49.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 --2- 172.21.15.60:0/3021747365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a380af190 0x7f7a38144b40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:49.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 >> 172.21.15.60:0/3021747365 conn(0x7f7a380b1840 msgr2=0x7f7a380b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:49.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 shutdown_connections 2024-09-16T15:42:49.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.205+0000 7f7a3fe00640 1 -- 172.21.15.60:0/3021747365 wait complete. 2024-09-16T15:42:49.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:42:49.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.405+0000 7fee0c400640 1 -- 172.21.15.60:0/1055310365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedec0051c0 msgr2=0x7fedec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:49.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.405+0000 7fee0c400640 1 --2- 172.21.15.60:0/1055310365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedec0051c0 0x7fedec00b9c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fedfc001dc0 tx=0x7fedec005830 comp rx=0 tx=0).stop 2024-09-16T15:42:49.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.405+0000 7fee0c400640 1 -- 172.21.15.60:0/1055310365 shutdown_connections 2024-09-16T15:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.405+0000 7fee0c400640 1 --2- 172.21.15.60:0/1055310365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedec0051c0 0x7fedec00b9c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.405+0000 7fee0c400640 1 -- 172.21.15.60:0/1055310365 >> 172.21.15.60:0/1055310365 conn(0x7fee040b1840 msgr2=0x7fee040b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.405+0000 7fee0c400640 1 -- 172.21.15.60:0/1055310365 shutdown_connections 2024-09-16T15:42:49.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.405+0000 7fee0c400640 1 -- 172.21.15.60:0/1055310365 wait complete. 2024-09-16T15:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.407+0000 7fee0c400640 1 Processor -- start 2024-09-16T15:42:49.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.407+0000 7fee0c400640 1 -- start start 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0c400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedec0051c0 0x7fee0415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee04149740 0x7fee041462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0c400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee04146c60 0x7fee04160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0c400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fee040ce620 con 0x7fee04146c60 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee040ce4a0 con 0x7fee04149740 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0c400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fee040ce7a0 con 0x7fedec0051c0 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee04146c60 0x7fee04160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee04146c60 0x7fee04160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60044/0 (socket says 172.21.15.60:60044) 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee0a400640 1 -- 172.21.15.60:0/1679266490 learned_addr learned my addr 172.21.15.60:0/1679266490 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee09000640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee04149740 0x7fee041462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.408+0000 7fee09a00640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedec0051c0 0x7fee0415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee0a400640 1 -- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedec0051c0 msgr2=0x7fee0415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee0a400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedec0051c0 0x7fee0415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:49.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee0a400640 1 -- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee04149740 msgr2=0x7fee041462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:49.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee0a400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee04149740 0x7fee041462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:49.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee0a400640 1 -- 172.21.15.60:0/1679266490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0415e2c0 con 0x7fee04146c60 2024-09-16T15:42:49.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee0a400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee04146c60 0x7fee04160540 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fedfc001eb0 tx=0x7fedfc0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:49.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee01600640 1 -- 172.21.15.60:0/1679266490 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc003d80 con 0x7fee04146c60 2024-09-16T15:42:49.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee01600640 1 -- 172.21.15.60:0/1679266490 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedfc0060d0 con 0x7fee04146c60 2024-09-16T15:42:49.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee01600640 1 -- 172.21.15.60:0/1679266490 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0326d0 con 0x7fee04146c60 2024-09-16T15:42:49.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee041e7f70 con 0x7fee04146c60 2024-09-16T15:42:49.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.409+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee041e8530 con 0x7fee04146c60 2024-09-16T15:42:49.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.410+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee040ce980 con 0x7fee04146c60 2024-09-16T15:42:49.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.413+0000 7fee01600640 1 -- 172.21.15.60:0/1679266490 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fedfc032870 con 0x7fee04146c60 2024-09-16T15:42:49.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.414+0000 7fee01600640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fede4078bb0 0x7fede407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:49.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.414+0000 7fee01600640 1 -- 172.21.15.60:0/1679266490 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5609+0+0 (secure 0 0 0) 0x7fedfc0bb120 con 0x7fee04146c60 2024-09-16T15:42:49.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.414+0000 7fee09a00640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fede4078bb0 0x7fede407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.415+0000 7fee09a00640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fede4078bb0 0x7fede407b070 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009ad0 tx=0x7fedf8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:49.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.417+0000 7fee01600640 1 -- 172.21.15.60:0/1679266490 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fedfc045020 con 0x7fee04146c60 2024-09-16T15:42:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:49.651+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fee040adfd0 con 0x7fee04146c60 2024-09-16T15:42:50.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.224+0000 7fee01600640 1 -- 172.21.15.60:0/1679266490 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v43) ==== 174+0+0 (secure 0 0 0) 0x7fedfc0833d0 con 0x7fee04146c60 2024-09-16T15:42:50.228 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 10 and data pool 11 2024-09-16T15:42:50.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.231+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fede4078bb0 msgr2=0x7fede407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.231+0000 7fee0c400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fede4078bb0 0x7fede407b070 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009ad0 tx=0x7fedf8005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:50.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.231+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee04146c60 msgr2=0x7fee04160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.231+0000 7fee0c400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee04146c60 0x7fee04160540 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fedfc001eb0 tx=0x7fedfc0019e0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.234+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 shutdown_connections 2024-09-16T15:42:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.234+0000 7fee0c400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fede4078bb0 0x7fede407b070 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.234+0000 7fee0c400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee04146c60 0x7fee04160540 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.234+0000 7fee0c400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee04149740 0x7fee041462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.234+0000 7fee0c400640 1 --2- 172.21.15.60:0/1679266490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedec0051c0 0x7fee0415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.234+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 >> 172.21.15.60:0/1679266490 conn(0x7fee040b1840 msgr2=0x7fee04074020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:50.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.235+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 shutdown_connections 2024-09-16T15:42:50.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.235+0000 7fee0c400640 1 -- 172.21.15.60:0/1679266490 wait complete. 2024-09-16T15:42:50.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.426+0000 7efe27e00640 1 -- 172.21.15.60:0/1875049155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe280b1680 msgr2=0x7efe280b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.426+0000 7efe27e00640 1 --2- 172.21.15.60:0/1875049155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe280b1680 0x7efe280b1b30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7efe280c52c0 tx=0x7efe2002aac0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.427+0000 7efe27e00640 1 -- 172.21.15.60:0/1875049155 shutdown_connections 2024-09-16T15:42:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.427+0000 7efe27e00640 1 --2- 172.21.15.60:0/1875049155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe280b1680 0x7efe280b1b30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.427+0000 7efe27e00640 1 --2- 172.21.15.60:0/1875049155 >> v2:172.21.15.100:3300/0 conn(0x7efe280d0670 0x7efe280d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.427+0000 7efe27e00640 1 -- 172.21.15.60:0/1875049155 >> 172.21.15.60:0/1875049155 conn(0x7efe280d1c40 msgr2=0x7efe280d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.427+0000 7efe27e00640 1 -- 172.21.15.60:0/1875049155 shutdown_connections 2024-09-16T15:42:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.427+0000 7efe27e00640 1 -- 172.21.15.60:0/1875049155 wait complete. 2024-09-16T15:42:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.428+0000 7efe27e00640 1 Processor -- start 2024-09-16T15:42:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.428+0000 7efe27e00640 1 -- start start 2024-09-16T15:42:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.429+0000 7efe27e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe280b1680 0x7efe281f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.429+0000 7efe27e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe280d0670 0x7efe2815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.429+0000 7efe27e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe2815bd30 0x7efe28070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.429+0000 7efe27e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efe280b2070 con 0x7efe2815bd30 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.429+0000 7efe27e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efe280cfa20 con 0x7efe280b1680 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.429+0000 7efe27e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7efe280b21f0 con 0x7efe280d0670 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe27400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe2815bd30 0x7efe28070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe27400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe2815bd30 0x7efe28070e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60062/0 (socket says 172.21.15.60:60062) 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe27400640 1 -- 172.21.15.60:0/1253360062 learned_addr learned my addr 172.21.15.60:0/1253360062 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe26a00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe280b1680 0x7efe281f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe1fe00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe280d0670 0x7efe2815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe27400640 1 -- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe280d0670 msgr2=0x7efe2815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe27400640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe280d0670 0x7efe2815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe27400640 1 -- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe280b1680 msgr2=0x7efe281f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe27400640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe280b1680 0x7efe281f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.430+0000 7efe27400640 1 -- 172.21.15.60:0/1253360062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe281f4c90 con 0x7efe2815bd30 2024-09-16T15:42:50.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.431+0000 7efe27400640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe2815bd30 0x7efe28070e90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efe20009d60 tx=0x7efe20003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:50.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.431+0000 7efe1ea00640 1 -- 172.21.15.60:0/1253360062 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe20044070 con 0x7efe2815bd30 2024-09-16T15:42:50.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.431+0000 7efe1ea00640 1 -- 172.21.15.60:0/1253360062 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe20004250 con 0x7efe2815bd30 2024-09-16T15:42:50.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.431+0000 7efe1ea00640 1 -- 172.21.15.60:0/1253360062 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe200046a0 con 0x7efe2815bd30 2024-09-16T15:42:50.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.431+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe281f4f70 con 0x7efe2815bd30 2024-09-16T15:42:50.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.431+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe281f5530 con 0x7efe2815bd30 2024-09-16T15:42:50.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.432+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe280b2740 con 0x7efe2815bd30 2024-09-16T15:42:50.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.433+0000 7efe1ea00640 1 -- 172.21.15.60:0/1253360062 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7efe20036400 con 0x7efe2815bd30 2024-09-16T15:42:50.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.434+0000 7efe1ea00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efe04078bb0 0x7efe0407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.434+0000 7efe1ea00640 1 -- 172.21.15.60:0/1253360062 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7efe200bb6a0 con 0x7efe2815bd30 2024-09-16T15:42:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.435+0000 7efe26a00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efe04078bb0 0x7efe0407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.436+0000 7efe26a00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efe04078bb0 0x7efe0407b070 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe180015e0 tx=0x7efe18009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:50.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.439+0000 7efe1ea00640 1 -- 172.21.15.60:0/1253360062 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efe20032030 con 0x7efe2815bd30 2024-09-16T15:42:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.636+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe280cde20 con 0x7efe2815bd30 2024-09-16T15:42:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.638+0000 7efe1ea00640 1 -- 172.21.15.60:0/1253360062 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16741 (secure 0 0 0) 0x7efe20083110 con 0x7efe2815bd30 2024-09-16T15:42:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:50.640 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":73,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:49.653953+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:47.898202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:48.609707+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:50.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efe04078bb0 msgr2=0x7efe0407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efe04078bb0 0x7efe0407b070 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe180015e0 tx=0x7efe18009040 comp rx=0 tx=0).stop 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe2815bd30 msgr2=0x7efe28070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe2815bd30 0x7efe28070e90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efe20009d60 tx=0x7efe20003ce0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 shutdown_connections 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7efe04078bb0 0x7efe0407b070 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe2815bd30 0x7efe28070e90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe280d0670 0x7efe2815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 --2- 172.21.15.60:0/1253360062 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe280b1680 0x7efe281f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 >> 172.21.15.60:0/1253360062 conn(0x7efe280d1c40 msgr2=0x7efe280b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 shutdown_connections 2024-09-16T15:42:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.640+0000 7efe27e00640 1 -- 172.21.15.60:0/1253360062 wait complete. 2024-09-16T15:42:50.659 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.816+0000 7f9275000640 1 -- 172.21.15.60:0/2896626056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9270161550 msgr2=0x7f927006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.816+0000 7f9275000640 1 --2- 172.21.15.60:0/2896626056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9270161550 0x7f927006ef50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9260009860 tx=0x7f926002a510 comp rx=0 tx=0).stop 2024-09-16T15:42:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.816+0000 7f9275000640 1 -- 172.21.15.60:0/2896626056 shutdown_connections 2024-09-16T15:42:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.816+0000 7f9275000640 1 --2- 172.21.15.60:0/2896626056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9270161550 0x7f927006ef50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.816+0000 7f9275000640 1 -- 172.21.15.60:0/2896626056 >> 172.21.15.60:0/2896626056 conn(0x7f92700b1950 msgr2=0x7f92700b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.816+0000 7f9275000640 1 -- 172.21.15.60:0/2896626056 shutdown_connections 2024-09-16T15:42:50.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.816+0000 7f9275000640 1 -- 172.21.15.60:0/2896626056 wait complete. 2024-09-16T15:42:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.818+0000 7f9275000640 1 Processor -- start 2024-09-16T15:42:50.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.818+0000 7f9275000640 1 -- start start 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f9275000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9270072df0 0x7f92701f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f9275000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92700af3d0 0x7f927009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f9275000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270161550 0x7f927009af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f9275000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f92700739b0 con 0x7f92700af3d0 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f9275000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f927006fdb0 con 0x7f9270161550 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f9275000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9270073b30 con 0x7f9270072df0 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f926d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92700af3d0 0x7f927009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f926d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92700af3d0 0x7f927009aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60066/0 (socket says 172.21.15.60:60066) 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.819+0000 7f926d400640 1 -- 172.21.15.60:0/119383769 learned_addr learned my addr 172.21.15.60:0/119383769 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f926de00640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9270072df0 0x7f92701f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f926e800640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270161550 0x7f927009af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f926d400640 1 -- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9270072df0 msgr2=0x7f92701f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f926d400640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9270072df0 0x7f92701f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f926d400640 1 -- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270161550 msgr2=0x7f927009af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f926d400640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270161550 0x7f927009af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:50.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f926d400640 1 -- 172.21.15.60:0/119383769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9270143200 con 0x7f92700af3d0 2024-09-16T15:42:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f926d400640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92700af3d0 0x7f927009aa00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c480 tx=0x7f925c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f9266000640 1 -- 172.21.15.60:0/119383769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c0052d0 con 0x7f92700af3d0 2024-09-16T15:42:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f9266000640 1 -- 172.21.15.60:0/119383769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f925c005470 con 0x7f92700af3d0 2024-09-16T15:42:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.820+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927009cb70 con 0x7f92700af3d0 2024-09-16T15:42:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.821+0000 7f9266000640 1 -- 172.21.15.60:0/119383769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c012870 con 0x7f92700af3d0 2024-09-16T15:42:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.821+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927009d130 con 0x7f92700af3d0 2024-09-16T15:42:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.822+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9270073fb0 con 0x7f92700af3d0 2024-09-16T15:42:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.823+0000 7f9266000640 1 -- 172.21.15.60:0/119383769 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f925c012a10 con 0x7f92700af3d0 2024-09-16T15:42:50.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.824+0000 7f9266000640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9250078bb0 0x7f925007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:50.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.824+0000 7f9266000640 1 -- 172.21.15.60:0/119383769 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f925c09ab50 con 0x7f92700af3d0 2024-09-16T15:42:50.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.825+0000 7f926de00640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9250078bb0 0x7f925007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:50.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.825+0000 7f926de00640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9250078bb0 0x7f925007b070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9260034000 tx=0x7f9260004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:50.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:50.828+0000 7f9266000640 1 -- 172.21.15.60:0/119383769 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f925c062bf0 con 0x7f92700af3d0 2024-09-16T15:42:51.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.025+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92700ad990 con 0x7f92700af3d0 2024-09-16T15:42:51.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.027+0000 7f9266000640 1 -- 172.21.15.60:0/119383769 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16741 (secure 0 0 0) 0x7f925c0624f0 con 0x7f92700af3d0 2024-09-16T15:42:51.032 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:51.032 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":73,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:49.653953+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:47.898202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:48.609707+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:51.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9250078bb0 msgr2=0x7f925007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9250078bb0 0x7f925007b070 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9260034000 tx=0x7f9260004730 comp rx=0 tx=0).stop 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92700af3d0 msgr2=0x7f927009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92700af3d0 0x7f927009aa00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c480 tx=0x7f925c00c940 comp rx=0 tx=0).stop 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 shutdown_connections 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9250078bb0 0x7f925007b070 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9270161550 0x7f927009af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92700af3d0 0x7f927009aa00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 --2- 172.21.15.60:0/119383769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9270072df0 0x7f92701f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 >> 172.21.15.60:0/119383769 conn(0x7f92700b1950 msgr2=0x7f927015f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:51.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 shutdown_connections 2024-09-16T15:42:51.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.029+0000 7f9275000640 1 -- 172.21.15.60:0/119383769 wait complete. 2024-09-16T15:42:51.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:42:51.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.209+0000 7fd902400640 1 -- 172.21.15.60:0/3556145749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8fc0b43c0 msgr2=0x7fd8fc0d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:51.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.209+0000 7fd902400640 1 --2- 172.21.15.60:0/3556145749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8fc0b43c0 0x7fd8fc0d2830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4005fd0 tx=0x7fd8f402a820 comp rx=0 tx=0).stop 2024-09-16T15:42:51.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.209+0000 7fd902400640 1 -- 172.21.15.60:0/3556145749 shutdown_connections 2024-09-16T15:42:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.209+0000 7fd902400640 1 --2- 172.21.15.60:0/3556145749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8fc0b43c0 0x7fd8fc0d2830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.209+0000 7fd902400640 1 --2- 172.21.15.60:0/3556145749 >> v2:172.21.15.100:3300/0 conn(0x7fd8fc161460 0x7fd8fc0d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.209+0000 7fd902400640 1 -- 172.21.15.60:0/3556145749 >> 172.21.15.60:0/3556145749 conn(0x7fd8fc09d0b0 msgr2=0x7fd8fc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.209+0000 7fd902400640 1 -- 172.21.15.60:0/3556145749 shutdown_connections 2024-09-16T15:42:51.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.209+0000 7fd902400640 1 -- 172.21.15.60:0/3556145749 wait complete. 2024-09-16T15:42:51.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.211+0000 7fd902400640 1 Processor -- start 2024-09-16T15:42:51.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.211+0000 7fd902400640 1 -- start start 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd902400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8fc0b43c0 0x7fd8fc0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd902400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8fc0b5480 0x7fd8fc15e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd902400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8fc161460 0x7fd8fc1f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd902400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0cef30 con 0x7fd8fc161460 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd902400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0b5180 con 0x7fd8fc0b5480 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd902400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc0cf0b0 con 0x7fd8fc0b43c0 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd901a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8fc161460 0x7fd8fc1f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd901a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8fc161460 0x7fd8fc1f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60080/0 (socket says 172.21.15.60:60080) 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd901a00640 1 -- 172.21.15.60:0/2159259457 learned_addr learned my addr 172.21.15.60:0/2159259457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd901000640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8fc0b43c0 0x7fd8fc0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:51.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.212+0000 7fd8fbe00640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8fc0b5480 0x7fd8fc15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:51.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd901a00640 1 -- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8fc0b43c0 msgr2=0x7fd8fc0b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:51.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd901a00640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8fc0b43c0 0x7fd8fc0b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:51.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd901a00640 1 -- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8fc0b5480 msgr2=0x7fd8fc15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:51.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd901a00640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8fc0b5480 0x7fd8fc15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:51.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd901a00640 1 -- 172.21.15.60:0/2159259457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc0b3580 con 0x7fd8fc161460 2024-09-16T15:42:51.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd901a00640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8fc161460 0x7fd8fc1f48e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4035040 tx=0x7fd8f4032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:51.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd8f8c00640 1 -- 172.21.15.60:0/2159259457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4037040 con 0x7fd8fc161460 2024-09-16T15:42:51.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc0b3860 con 0x7fd8fc161460 2024-09-16T15:42:51.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.213+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc0b3e20 con 0x7fd8fc161460 2024-09-16T15:42:51.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.214+0000 7fd8f8c00640 1 -- 172.21.15.60:0/2159259457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8f4042070 con 0x7fd8fc161460 2024-09-16T15:42:51.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.214+0000 7fd8f8c00640 1 -- 172.21.15.60:0/2159259457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4003e60 con 0x7fd8fc161460 2024-09-16T15:42:51.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.215+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d8005290 con 0x7fd8fc161460 2024-09-16T15:42:51.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.216+0000 7fd8f8c00640 1 -- 172.21.15.60:0/2159259457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fd8f4004040 con 0x7fd8fc161460 2024-09-16T15:42:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.218+0000 7fd8f8c00640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8dc078bb0 0x7fd8dc07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.218+0000 7fd8f8c00640 1 -- 172.21.15.60:0/2159259457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fd8f40bb990 con 0x7fd8fc161460 2024-09-16T15:42:51.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.218+0000 7fd901000640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8dc078bb0 0x7fd8dc07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:51.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.220+0000 7fd901000640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8dc078bb0 0x7fd8dc07b070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00096c0 tx=0x7fd8f0009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:51.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.223+0000 7fd8f8c00640 1 -- 172.21.15.60:0/2159259457 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8f40834e0 con 0x7fd8fc161460 2024-09-16T15:42:51.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:51.463+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd8d8005850 con 0x7fd8fc161460 2024-09-16T15:42:52.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.236+0000 7fd8f8c00640 1 -- 172.21.15.60:0/2159259457 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) ==== 118+0+0 (secure 0 0 0) 0x7fd8f4087f10 con 0x7fd8fc161460 2024-09-16T15:42:52.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.238+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8dc078bb0 msgr2=0x7fd8dc07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:52.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.238+0000 7fd902400640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8dc078bb0 0x7fd8dc07b070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00096c0 tx=0x7fd8f0009260 comp rx=0 tx=0).stop 2024-09-16T15:42:52.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.238+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8fc161460 msgr2=0x7fd8fc1f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:52.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.238+0000 7fd902400640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8fc161460 0x7fd8fc1f48e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4035040 tx=0x7fd8f4032c60 comp rx=0 tx=0).stop 2024-09-16T15:42:52.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.239+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 shutdown_connections 2024-09-16T15:42:52.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.239+0000 7fd902400640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fd8dc078bb0 0x7fd8dc07b070 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.239+0000 7fd902400640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8fc161460 0x7fd8fc1f48e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.239+0000 7fd902400640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8fc0b5480 0x7fd8fc15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.239+0000 7fd902400640 1 --2- 172.21.15.60:0/2159259457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8fc0b43c0 0x7fd8fc0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.239+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 >> 172.21.15.60:0/2159259457 conn(0x7fd8fc09d0b0 msgr2=0x7fd8fc0d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:52.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.239+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 shutdown_connections 2024-09-16T15:42:52.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.239+0000 7fd902400640 1 -- 172.21.15.60:0/2159259457 wait complete. 2024-09-16T15:42:52.260 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:42:52.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:42:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.428+0000 7f3f05600640 1 -- 172.21.15.60:0/1681062092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f0015bb80 msgr2=0x7f3f000b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.428+0000 7f3f05600640 1 --2- 172.21.15.60:0/1681062092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f0015bb80 0x7f3f000b3bc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3f00072750 tx=0x7f3eec02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.428+0000 7f3f05600640 1 -- 172.21.15.60:0/1681062092 shutdown_connections 2024-09-16T15:42:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.428+0000 7f3f05600640 1 --2- 172.21.15.60:0/1681062092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f0015bb80 0x7f3f000b3bc0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.428+0000 7f3f05600640 1 --2- 172.21.15.60:0/1681062092 >> v2:172.21.15.100:3300/0 conn(0x7f3f000b2480 0x7f3f000b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.428+0000 7f3f05600640 1 -- 172.21.15.60:0/1681062092 >> 172.21.15.60:0/1681062092 conn(0x7f3f000b5680 msgr2=0x7f3f000b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:52.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.428+0000 7f3f05600640 1 -- 172.21.15.60:0/1681062092 shutdown_connections 2024-09-16T15:42:52.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.428+0000 7f3f05600640 1 -- 172.21.15.60:0/1681062092 wait complete. 2024-09-16T15:42:52.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.429+0000 7f3f05600640 1 Processor -- start 2024-09-16T15:42:52.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.430+0000 7f3f05600640 1 -- start start 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.430+0000 7f3f05600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f000b2480 0x7f3f00142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.430+0000 7f3f05600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f000b4100 0x7f3f001f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.430+0000 7f3f05600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f0015bb80 0x7f3f00154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.430+0000 7f3f05600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0015e240 con 0x7f3f000b2480 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.430+0000 7f3f05600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0015e0c0 con 0x7f3f0015bb80 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.430+0000 7f3f05600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0015e3c0 con 0x7f3f000b4100 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f000b2480 0x7f3f00142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f000b2480 0x7f3f00142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60106/0 (socket says 172.21.15.60:60106) 2024-09-16T15:42:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 -- 172.21.15.60:0/160760937 learned_addr learned my addr 172.21.15.60:0/160760937 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3eff400640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f000b4100 0x7f3f001f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 -- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f000b4100 msgr2=0x7f3f001f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3f04c00640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f0015bb80 0x7f3f00154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f000b4100 0x7f3f001f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 -- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f0015bb80 msgr2=0x7f3f00154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f0015bb80 0x7f3f00154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 -- 172.21.15.60:0/160760937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f001f8870 con 0x7f3f000b2480 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3eff400640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f000b4100 0x7f3f001f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3f04c00640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f0015bb80 0x7f3f00154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.431+0000 7f3effe00640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f000b2480 0x7f3f00142520 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d980 tx=0x7f3ef000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.432+0000 7f3ef7e00640 1 -- 172.21.15.60:0/160760937 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0017070 con 0x7f3f000b2480 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.432+0000 7f3ef7e00640 1 -- 172.21.15.60:0/160760937 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ef0009d20 con 0x7f3f000b2480 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.432+0000 7f3ef7e00640 1 -- 172.21.15.60:0/160760937 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00106c0 con 0x7f3f000b2480 2024-09-16T15:42:52.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.432+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f001f8b50 con 0x7f3f000b2480 2024-09-16T15:42:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.432+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f001f9110 con 0x7f3f000b2480 2024-09-16T15:42:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.433+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f000b3120 con 0x7f3f000b2480 2024-09-16T15:42:52.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.434+0000 7f3ef7e00640 1 -- 172.21.15.60:0/160760937 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f3ef0010860 con 0x7f3f000b2480 2024-09-16T15:42:52.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.435+0000 7f3ef7e00640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ee0078bb0 0x7f3ee007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:52.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.435+0000 7f3ef7e00640 1 -- 172.21.15.60:0/160760937 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f3ef009b490 con 0x7f3f000b2480 2024-09-16T15:42:52.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.435+0000 7f3eff400640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ee0078bb0 0x7f3ee007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:52.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.436+0000 7f3eff400640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ee0078bb0 0x7f3ee007b070 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3eec033000 tx=0x7f3eec004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:52.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.439+0000 7f3ef7e00640 1 -- 172.21.15.60:0/160760937 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3ef00634a0 con 0x7f3f000b2480 2024-09-16T15:42:52.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:52.677+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3f0006e760 con 0x7f3f000b2480 2024-09-16T15:42:53.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.237+0000 7f3ef7e00640 1 -- 172.21.15.60:0/160760937 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7f3ef0062da0 con 0x7f3f000b2480 2024-09-16T15:42:53.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.240+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ee0078bb0 msgr2=0x7f3ee007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.240+0000 7f3f05600640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ee0078bb0 0x7f3ee007b070 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3eec033000 tx=0x7f3eec004730 comp rx=0 tx=0).stop 2024-09-16T15:42:53.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.240+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f000b2480 msgr2=0x7f3f00142520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.240+0000 7f3f05600640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f000b2480 0x7f3f00142520 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d980 tx=0x7f3ef000de40 comp rx=0 tx=0).stop 2024-09-16T15:42:53.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.241+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 shutdown_connections 2024-09-16T15:42:53.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.241+0000 7f3f05600640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3ee0078bb0 0x7f3ee007b070 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.241+0000 7f3f05600640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f0015bb80 0x7f3f00154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.241+0000 7f3f05600640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f000b4100 0x7f3f001f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.241+0000 7f3f05600640 1 --2- 172.21.15.60:0/160760937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f000b2480 0x7f3f00142520 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.241+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 >> 172.21.15.60:0/160760937 conn(0x7f3f000b5680 msgr2=0x7f3f000b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:53.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.241+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 shutdown_connections 2024-09-16T15:42:53.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.241+0000 7f3f05600640 1 -- 172.21.15.60:0/160760937 wait complete. 2024-09-16T15:42:53.261 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:42:53.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.422+0000 7f820f200640 1 -- 172.21.15.60:0/378484837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82080b5370 msgr2=0x7f82080b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.422+0000 7f820f200640 1 --2- 172.21.15.60:0/378484837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82080b5370 0x7f82080b3bc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f81f8005fd0 tx=0x7f81f802a540 comp rx=0 tx=0).stop 2024-09-16T15:42:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.422+0000 7f820f200640 1 -- 172.21.15.60:0/378484837 shutdown_connections 2024-09-16T15:42:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.422+0000 7f820f200640 1 --2- 172.21.15.60:0/378484837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82080b4100 0x7f82080b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.422+0000 7f820f200640 1 --2- 172.21.15.60:0/378484837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82080b5370 0x7f82080b3bc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.422+0000 7f820f200640 1 -- 172.21.15.60:0/378484837 >> 172.21.15.60:0/378484837 conn(0x7f82080d3df0 msgr2=0x7f82080d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:53.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.422+0000 7f820f200640 1 -- 172.21.15.60:0/378484837 shutdown_connections 2024-09-16T15:42:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.422+0000 7f820f200640 1 -- 172.21.15.60:0/378484837 wait complete. 2024-09-16T15:42:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.423+0000 7f820f200640 1 Processor -- start 2024-09-16T15:42:53.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.423+0000 7f820f200640 1 -- start start 2024-09-16T15:42:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.424+0000 7f820f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82080b4100 0x7f82081f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.424+0000 7f820f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b5370 0x7f82081f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.424+0000 7f820f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8208161440 0x7f82081f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.424+0000 7f820f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f82080b2040 con 0x7f8208161440 2024-09-16T15:42:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.424+0000 7f820f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82080b1ec0 con 0x7f82080b5370 2024-09-16T15:42:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.424+0000 7f820f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f82080b21c0 con 0x7f82080b4100 2024-09-16T15:42:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8208161440 0x7f82081f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8208161440 0x7f82081f7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60124/0 (socket says 172.21.15.60:60124) 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 -- 172.21.15.60:0/1026750446 learned_addr learned my addr 172.21.15.60:0/1026750446 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820de00640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82080b4100 0x7f82081f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820d400640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b5370 0x7f82081f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 -- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82080b4100 msgr2=0x7f82081f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82080b4100 0x7f82081f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 -- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b5370 msgr2=0x7f82081f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b5370 0x7f82081f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 -- 172.21.15.60:0/1026750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82080ca000 con 0x7f8208161440 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.425+0000 7f820e800640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8208161440 0x7f82081f7fe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f820400eaa0 tx=0x7f820400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.426+0000 7f8202000640 1 -- 172.21.15.60:0/1026750446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820400cdb0 con 0x7f8208161440 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.426+0000 7f8202000640 1 -- 172.21.15.60:0/1026750446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8204004c10 con 0x7f8208161440 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.426+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82080ca2e0 con 0x7f8208161440 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.426+0000 7f8202000640 1 -- 172.21.15.60:0/1026750446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204005020 con 0x7f8208161440 2024-09-16T15:42:53.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.426+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8208148cd0 con 0x7f8208161440 2024-09-16T15:42:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.427+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82080b1770 con 0x7f8208161440 2024-09-16T15:42:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.428+0000 7f8202000640 1 -- 172.21.15.60:0/1026750446 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f82040051c0 con 0x7f8208161440 2024-09-16T15:42:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.429+0000 7f8202000640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81ec078bb0 0x7f81ec07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.429+0000 7f8202000640 1 -- 172.21.15.60:0/1026750446 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f820409bad0 con 0x7f8208161440 2024-09-16T15:42:53.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.429+0000 7f820de00640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81ec078bb0 0x7f81ec07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:53.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.430+0000 7f820de00640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81ec078bb0 0x7f81ec07b070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f82080b2f10 tx=0x7f81fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:53.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.433+0000 7f8202000640 1 -- 172.21.15.60:0/1026750446 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8204016030 con 0x7f8208161440 2024-09-16T15:42:53.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.668+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f820806ef00 con 0x7f8208161440 2024-09-16T15:42:53.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.669+0000 7f8202000640 1 -- 172.21.15.60:0/1026750446 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f820406c020 con 0x7f8208161440 2024-09-16T15:42:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:53.671 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"btime":"2024-09-16T15:42:53:237452+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:42:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 47 2024-09-16T15:42:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81ec078bb0 msgr2=0x7f81ec07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81ec078bb0 0x7f81ec07b070 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f82080b2f10 tx=0x7f81fc009040 comp rx=0 tx=0).stop 2024-09-16T15:42:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8208161440 msgr2=0x7f82081f7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8208161440 0x7f82081f7fe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f820400eaa0 tx=0x7f820400ef60 comp rx=0 tx=0).stop 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 shutdown_connections 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f81ec078bb0 0x7f81ec07b070 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8208161440 0x7f82081f7fe0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82080b5370 0x7f82081f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 --2- 172.21.15.60:0/1026750446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82080b4100 0x7f82081f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 >> 172.21.15.60:0/1026750446 conn(0x7f82080d3df0 msgr2=0x7f820815fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 shutdown_connections 2024-09-16T15:42:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.671+0000 7f820f200640 1 -- 172.21.15.60:0/1026750446 wait complete. 2024-09-16T15:42:53.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:53.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.849+0000 7f66d3e00640 1 -- 172.21.15.60:0/336613958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b8004d40 msgr2=0x7f66b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.849+0000 7f66d3e00640 1 --2- 172.21.15.60:0/336613958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b8004d40 0x7f66b80051c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005fd0 tx=0x7f66b8005830 comp rx=0 tx=0).stop 2024-09-16T15:42:53.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.849+0000 7f66d3400640 1 -- 172.21.15.60:0/336613958 reap_dead start 2024-09-16T15:42:53.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.849+0000 7f66d3e00640 1 -- 172.21.15.60:0/336613958 shutdown_connections 2024-09-16T15:42:53.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.849+0000 7f66d3e00640 1 -- 172.21.15.60:0/336613958 >> 172.21.15.60:0/336613958 conn(0x7f66d40d3df0 msgr2=0x7f66d40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:53.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.850+0000 7f66d3e00640 1 -- 172.21.15.60:0/336613958 shutdown_connections 2024-09-16T15:42:53.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.850+0000 7f66d3e00640 1 -- 172.21.15.60:0/336613958 wait complete. 2024-09-16T15:42:53.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.851+0000 7f66d3e00640 1 Processor -- start 2024-09-16T15:42:53.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.851+0000 7f66d3e00640 1 -- start start 2024-09-16T15:42:53.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.852+0000 7f66d3e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b8004d40 0x7f66d4154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:53.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.852+0000 7f66d3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66d4154770 0x7f66d41f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:53.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.852+0000 7f66d3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66d40b94f0 0x7f66d415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:53.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.852+0000 7f66d3e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f66d4166b90 con 0x7f66d4154770 2024-09-16T15:42:53.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.852+0000 7f66d3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66d40d2690 con 0x7f66d40b94f0 2024-09-16T15:42:53.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.852+0000 7f66d3e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f66d4166d10 con 0x7f66b8004d40 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66d4154770 0x7f66d41f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66d4154770 0x7f66d41f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60132/0 (socket says 172.21.15.60:60132) 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 -- 172.21.15.60:0/3556884576 learned_addr learned my addr 172.21.15.60:0/3556884576 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66d3400640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66d40b94f0 0x7f66d415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66d2a00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b8004d40 0x7f66d4154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 -- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b8004d40 msgr2=0x7f66d4154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b8004d40 0x7f66d4154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 -- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66d40b94f0 msgr2=0x7f66d415a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66d40b94f0 0x7f66d415a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 -- 172.21.15.60:0/3556884576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d415f840 con 0x7f66d4154770 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66d2a00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b8004d40 0x7f66d4154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.853+0000 7f66ca000640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66d4154770 0x7f66d41f9090 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f66bc006ee0 tx=0x7f66bc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:53.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.854+0000 7f66cb400640 1 -- 172.21.15.60:0/3556884576 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0107d0 con 0x7f66d4154770 2024-09-16T15:42:53.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.854+0000 7f66cb400640 1 -- 172.21.15.60:0/3556884576 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66bc010a60 con 0x7f66d4154770 2024-09-16T15:42:53.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.854+0000 7f66cb400640 1 -- 172.21.15.60:0/3556884576 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc011460 con 0x7f66d4154770 2024-09-16T15:42:53.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.854+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d415fb20 con 0x7f66d4154770 2024-09-16T15:42:53.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.854+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d40b8fd0 con 0x7f66d4154770 2024-09-16T15:42:53.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.855+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d40d1c60 con 0x7f66d4154770 2024-09-16T15:42:53.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.856+0000 7f66cb400640 1 -- 172.21.15.60:0/3556884576 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f66bc011600 con 0x7f66d4154770 2024-09-16T15:42:53.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.857+0000 7f66cb400640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66b0078be0 0x7f66b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:53.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.857+0000 7f66cb400640 1 -- 172.21.15.60:0/3556884576 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f66bc09b9c0 con 0x7f66d4154770 2024-09-16T15:42:53.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.858+0000 7f66d2a00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66b0078be0 0x7f66b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:53.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.858+0000 7f66d2a00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66b0078be0 0x7f66b007b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f66c40036b0 tx=0x7f66c4006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:53.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:53.862+0000 7f66cb400640 1 -- 172.21.15.60:0/3556884576 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66d40d2cc0 con 0x7f66d4154770 2024-09-16T15:42:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.057+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66d415fcf0 con 0x7f66d4154770 2024-09-16T15:42:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.058+0000 7f66cb400640 1 -- 172.21.15.60:0/3556884576 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16728 (secure 0 0 0) 0x7f66bc0639d0 con 0x7f66d4154770 2024-09-16T15:42:54.060 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:54.060 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":74,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:42:51.215281+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:47.898202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:48.609707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66b0078be0 msgr2=0x7f66b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66b0078be0 0x7f66b007b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f66c40036b0 tx=0x7f66c4006b70 comp rx=0 tx=0).stop 2024-09-16T15:42:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66d4154770 msgr2=0x7f66d41f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66d4154770 0x7f66d41f9090 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f66bc006ee0 tx=0x7f66bc0049d0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 shutdown_connections 2024-09-16T15:42:54.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f66b0078be0 0x7f66b007b0a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66d40b94f0 0x7f66d415a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66d4154770 0x7f66d41f9090 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 --2- 172.21.15.60:0/3556884576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b8004d40 0x7f66d4154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 >> 172.21.15.60:0/3556884576 conn(0x7f66d40d3df0 msgr2=0x7f66d415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:54.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 shutdown_connections 2024-09-16T15:42:54.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.060+0000 7f66d3e00640 1 -- 172.21.15.60:0/3556884576 wait complete. 2024-09-16T15:42:54.078 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:42:54.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.236+0000 7fe195000640 1 -- 172.21.15.60:0/3470832181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900af3d0 msgr2=0x7fe19006f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.236+0000 7fe195000640 1 --2- 172.21.15.60:0/3470832181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900af3d0 0x7fe19006f660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe178009830 tx=0x7fe17802a530 comp rx=0 tx=0).stop 2024-09-16T15:42:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.236+0000 7fe195000640 1 -- 172.21.15.60:0/3470832181 shutdown_connections 2024-09-16T15:42:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.236+0000 7fe195000640 1 --2- 172.21.15.60:0/3470832181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1900af3d0 0x7fe19006f660 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.236+0000 7fe195000640 1 -- 172.21.15.60:0/3470832181 >> 172.21.15.60:0/3470832181 conn(0x7fe1900b1950 msgr2=0x7fe1900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.236+0000 7fe195000640 1 -- 172.21.15.60:0/3470832181 shutdown_connections 2024-09-16T15:42:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.236+0000 7fe195000640 1 -- 172.21.15.60:0/3470832181 wait complete. 2024-09-16T15:42:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.237+0000 7fe195000640 1 Processor -- start 2024-09-16T15:42:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.238+0000 7fe195000640 1 -- start start 2024-09-16T15:42:54.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.238+0000 7fe195000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe190072df0 0x7fe1901f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.238+0000 7fe195000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1900af3d0 0x7fe1900cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.238+0000 7fe195000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe190161550 0x7fe1900afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.238+0000 7fe195000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe190073950 con 0x7fe190072df0 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.238+0000 7fe195000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe19006fdb0 con 0x7fe190161550 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.238+0000 7fe195000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe190073ad0 con 0x7fe1900af3d0 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe190072df0 0x7fe1901f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe190072df0 0x7fe1901f8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60134/0 (socket says 172.21.15.60:60134) 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18de00640 1 -- 172.21.15.60:0/209887195 learned_addr learned my addr 172.21.15.60:0/209887195 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18e800640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe190161550 0x7fe1900afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe185400640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1900af3d0 0x7fe1900cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18de00640 1 -- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1900af3d0 msgr2=0x7fe1900cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18de00640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1900af3d0 0x7fe1900cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18de00640 1 -- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe190161550 msgr2=0x7fe1900afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18de00640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe190161550 0x7fe1900afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.239+0000 7fe18de00640 1 -- 172.21.15.60:0/209887195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1900cba80 con 0x7fe190072df0 2024-09-16T15:42:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.240+0000 7fe18de00640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe190072df0 0x7fe1901f8bc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c3c0 tx=0x7fe18000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.241+0000 7fe186a00640 1 -- 172.21.15.60:0/209887195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180017070 con 0x7fe190072df0 2024-09-16T15:42:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.241+0000 7fe186a00640 1 -- 172.21.15.60:0/209887195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1800053e0 con 0x7fe190072df0 2024-09-16T15:42:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.241+0000 7fe186a00640 1 -- 172.21.15.60:0/209887195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180003870 con 0x7fe190072df0 2024-09-16T15:42:54.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.241+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1901f58b0 con 0x7fe190072df0 2024-09-16T15:42:54.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.241+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1900cbfe0 con 0x7fe190072df0 2024-09-16T15:42:54.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.242+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe190073fc0 con 0x7fe190072df0 2024-09-16T15:42:54.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.243+0000 7fe186a00640 1 -- 172.21.15.60:0/209887195 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fe18001a070 con 0x7fe190072df0 2024-09-16T15:42:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.244+0000 7fe186a00640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe16c078bb0 0x7fe16c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.244+0000 7fe186a00640 1 -- 172.21.15.60:0/209887195 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7fe18009a250 con 0x7fe190072df0 2024-09-16T15:42:54.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.245+0000 7fe185400640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe16c078bb0 0x7fe16c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:54.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.246+0000 7fe185400640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe16c078bb0 0x7fe16c07b070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe178034000 tx=0x7fe178004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:54.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.249+0000 7fe186a00640 1 -- 172.21.15.60:0/209887195 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe1900cba80 con 0x7fe190072df0 2024-09-16T15:42:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.440+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1900ae0b0 con 0x7fe16c078bb0 2024-09-16T15:42:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.446+0000 7fe186a00640 1 -- 172.21.15.60:0/209887195 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252058 (secure 0 0 0) 0x7fe1900ae0b0 con 0x7fe16c078bb0 2024-09-16T15:42:54.451 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:42:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe16c078bb0 msgr2=0x7fe16c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe16c078bb0 0x7fe16c07b070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe178034000 tx=0x7fe178004730 comp rx=0 tx=0).stop 2024-09-16T15:42:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe190072df0 msgr2=0x7fe1901f8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe190072df0 0x7fe1901f8bc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c3c0 tx=0x7fe18000c880 comp rx=0 tx=0).stop 2024-09-16T15:42:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 shutdown_connections 2024-09-16T15:42:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe16c078bb0 0x7fe16c07b070 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe190161550 0x7fe1900afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1900af3d0 0x7fe1900cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 --2- 172.21.15.60:0/209887195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe190072df0 0x7fe1901f8bc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 >> 172.21.15.60:0/209887195 conn(0x7fe1900b1950 msgr2=0x7fe19015ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 shutdown_connections 2024-09-16T15:42:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.450+0000 7fe195000640 1 -- 172.21.15.60:0/209887195 wait complete. 2024-09-16T15:42:54.467 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-16T15:42:53.776252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249412,"kb_used_data":8200,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725308,"statfs":{"total":773094113280,"available":772838715392,"internally_reserved":0,"allocated":8396800,"data_stored":4183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.575010"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632224+0000","last_change":"2024-09-16T15:42:50.246949+0000","last_active":"2024-09-16T15:42:51.632224+0000","last_peered":"2024-09-16T15:42:51.632224+0000","last_clean":"2024-09-16T15:42:51.632224+0000","last_became_active":"2024-09-16T15:42:50.245057+0000","last_became_peered":"2024-09-16T15:42:50.245057+0000","last_unstale":"2024-09-16T15:42:51.632224+0000","last_undegraded":"2024-09-16T15:42:51.632224+0000","last_fullsized":"2024-09-16T15:42:51.632224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:32.199533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222866+0000","last_change":"2024-09-16T15:42:50.241966+0000","last_active":"2024-09-16T15:42:51.222866+0000","last_peered":"2024-09-16T15:42:51.222866+0000","last_clean":"2024-09-16T15:42:51.222866+0000","last_became_active":"2024-09-16T15:42:50.240434+0000","last_became_peered":"2024-09-16T15:42:50.240434+0000","last_unstale":"2024-09-16T15:42:51.222866+0000","last_undegraded":"2024-09-16T15:42:51.222866+0000","last_fullsized":"2024-09-16T15:42:51.222866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:50.405364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886709+0000","last_change":"2024-09-16T15:42:50.232921+0000","last_active":"2024-09-16T15:42:51.886709+0000","last_peered":"2024-09-16T15:42:51.886709+0000","last_clean":"2024-09-16T15:42:51.886709+0000","last_became_active":"2024-09-16T15:42:50.232658+0000","last_became_peered":"2024-09-16T15:42:50.232658+0000","last_unstale":"2024-09-16T15:42:51.886709+0000","last_undegraded":"2024-09-16T15:42:51.886709+0000","last_fullsized":"2024-09-16T15:42:51.886709+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:26.597473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219437+0000","last_change":"2024-09-16T15:42:50.237383+0000","last_active":"2024-09-16T15:42:51.219437+0000","last_peered":"2024-09-16T15:42:51.219437+0000","last_clean":"2024-09-16T15:42:51.219437+0000","last_became_active":"2024-09-16T15:42:50.237112+0000","last_became_peered":"2024-09-16T15:42:50.237112+0000","last_unstale":"2024-09-16T15:42:51.219437+0000","last_undegraded":"2024-09-16T15:42:51.219437+0000","last_fullsized":"2024-09-16T15:42:51.219437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:44.775395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532971+0000","last_change":"2024-09-16T15:42:50.243001+0000","last_active":"2024-09-16T15:42:51.532971+0000","last_peered":"2024-09-16T15:42:51.532971+0000","last_clean":"2024-09-16T15:42:51.532971+0000","last_became_active":"2024-09-16T15:42:50.241449+0000","last_became_peered":"2024-09-16T15:42:50.241449+0000","last_unstale":"2024-09-16T15:42:51.532971+0000","last_undegraded":"2024-09-16T15:42:51.532971+0000","last_fullsized":"2024-09-16T15:42:51.532971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:41.128580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222308+0000","last_change":"2024-09-16T15:42:50.236079+0000","last_active":"2024-09-16T15:42:51.222308+0000","last_peered":"2024-09-16T15:42:51.222308+0000","last_clean":"2024-09-16T15:42:51.222308+0000","last_became_active":"2024-09-16T15:42:50.234626+0000","last_became_peered":"2024-09-16T15:42:50.234626+0000","last_unstale":"2024-09-16T15:42:51.222308+0000","last_undegraded":"2024-09-16T15:42:51.222308+0000","last_fullsized":"2024-09-16T15:42:51.222308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:18.670844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221523+0000","last_change":"2024-09-16T15:42:50.239884+0000","last_active":"2024-09-16T15:42:51.221523+0000","last_peered":"2024-09-16T15:42:51.221523+0000","last_clean":"2024-09-16T15:42:51.221523+0000","last_became_active":"2024-09-16T15:42:50.238583+0000","last_became_peered":"2024-09-16T15:42:50.238583+0000","last_unstale":"2024-09-16T15:42:51.221523+0000","last_undegraded":"2024-09-16T15:42:51.221523+0000","last_fullsized":"2024-09-16T15:42:51.221523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:39.104202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221629+0000","last_change":"2024-09-16T15:42:50.237211+0000","last_active":"2024-09-16T15:42:51.221629+0000","last_peered":"2024-09-16T15:42:51.221629+0000","last_clean":"2024-09-16T15:42:51.221629+0000","last_became_active":"2024-09-16T15:42:50.235538+0000","last_became_peered":"2024-09-16T15:42:50.235538+0000","last_unstale":"2024-09-16T15:42:51.221629+0000","last_undegraded":"2024-09-16T15:42:51.221629+0000","last_fullsized":"2024-09-16T15:42:51.221629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:42.073739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533042+0000","last_change":"2024-09-16T15:42:50.248832+0000","last_active":"2024-09-16T15:42:51.533042+0000","last_peered":"2024-09-16T15:42:51.533042+0000","last_clean":"2024-09-16T15:42:51.533042+0000","last_became_active":"2024-09-16T15:42:50.244745+0000","last_became_peered":"2024-09-16T15:42:50.244745+0000","last_unstale":"2024-09-16T15:42:51.533042+0000","last_undegraded":"2024-09-16T15:42:51.533042+0000","last_fullsized":"2024-09-16T15:42:51.533042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:56.282754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671475+0000","last_change":"2024-09-16T15:42:50.250302+0000","last_active":"2024-09-16T15:42:51.671475+0000","last_peered":"2024-09-16T15:42:51.671475+0000","last_clean":"2024-09-16T15:42:51.671475+0000","last_became_active":"2024-09-16T15:42:50.248474+0000","last_became_peered":"2024-09-16T15:42:50.248474+0000","last_unstale":"2024-09-16T15:42:51.671475+0000","last_undegraded":"2024-09-16T15:42:51.671475+0000","last_fullsized":"2024-09-16T15:42:51.671475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:10:26.440645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631934+0000","last_change":"2024-09-16T15:42:50.242486+0000","last_active":"2024-09-16T15:42:51.631934+0000","last_peered":"2024-09-16T15:42:51.631934+0000","last_clean":"2024-09-16T15:42:51.631934+0000","last_became_active":"2024-09-16T15:42:50.240689+0000","last_became_peered":"2024-09-16T15:42:50.240689+0000","last_unstale":"2024-09-16T15:42:51.631934+0000","last_undegraded":"2024-09-16T15:42:51.631934+0000","last_fullsized":"2024-09-16T15:42:51.631934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:11.985920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219465+0000","last_change":"2024-09-16T15:42:50.237151+0000","last_active":"2024-09-16T15:42:51.219465+0000","last_peered":"2024-09-16T15:42:51.219465+0000","last_clean":"2024-09-16T15:42:51.219465+0000","last_became_active":"2024-09-16T15:42:50.236870+0000","last_became_peered":"2024-09-16T15:42:50.236870+0000","last_unstale":"2024-09-16T15:42:51.219465+0000","last_undegraded":"2024-09-16T15:42:51.219465+0000","last_fullsized":"2024-09-16T15:42:51.219465+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:37.469154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240847+0000","last_change":"2024-09-16T15:42:50.236288+0000","last_active":"2024-09-16T15:42:51.240847+0000","last_peered":"2024-09-16T15:42:51.240847+0000","last_clean":"2024-09-16T15:42:51.240847+0000","last_became_active":"2024-09-16T15:42:50.235987+0000","last_became_peered":"2024-09-16T15:42:50.235987+0000","last_unstale":"2024-09-16T15:42:51.240847+0000","last_undegraded":"2024-09-16T15:42:51.240847+0000","last_fullsized":"2024-09-16T15:42:51.240847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:41.139551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533185+0000","last_change":"2024-09-16T15:42:50.248623+0000","last_active":"2024-09-16T15:42:51.533185+0000","last_peered":"2024-09-16T15:42:51.533185+0000","last_clean":"2024-09-16T15:42:51.533185+0000","last_became_active":"2024-09-16T15:42:50.248296+0000","last_became_peered":"2024-09-16T15:42:50.248296+0000","last_unstale":"2024-09-16T15:42:51.533185+0000","last_undegraded":"2024-09-16T15:42:51.533185+0000","last_fullsized":"2024-09-16T15:42:51.533185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:15.570815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240472+0000","last_change":"2024-09-16T15:42:50.240837+0000","last_active":"2024-09-16T15:42:51.240472+0000","last_peered":"2024-09-16T15:42:51.240472+0000","last_clean":"2024-09-16T15:42:51.240472+0000","last_became_active":"2024-09-16T15:42:50.240516+0000","last_became_peered":"2024-09-16T15:42:50.240516+0000","last_unstale":"2024-09-16T15:42:51.240472+0000","last_undegraded":"2024-09-16T15:42:51.240472+0000","last_fullsized":"2024-09-16T15:42:51.240472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:50.681711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671048+0000","last_change":"2024-09-16T15:42:50.237124+0000","last_active":"2024-09-16T15:42:51.671048+0000","last_peered":"2024-09-16T15:42:51.671048+0000","last_clean":"2024-09-16T15:42:51.671048+0000","last_became_active":"2024-09-16T15:42:50.236795+0000","last_became_peered":"2024-09-16T15:42:50.236795+0000","last_unstale":"2024-09-16T15:42:51.671048+0000","last_undegraded":"2024-09-16T15:42:51.671048+0000","last_fullsized":"2024-09-16T15:42:51.671048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:00.018854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533240+0000","last_change":"2024-09-16T15:42:50.247702+0000","last_active":"2024-09-16T15:42:51.533240+0000","last_peered":"2024-09-16T15:42:51.533240+0000","last_clean":"2024-09-16T15:42:51.533240+0000","last_became_active":"2024-09-16T15:42:50.244483+0000","last_became_peered":"2024-09-16T15:42:50.244483+0000","last_unstale":"2024-09-16T15:42:51.533240+0000","last_undegraded":"2024-09-16T15:42:51.533240+0000","last_fullsized":"2024-09-16T15:42:51.533240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:48:49.368461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219746+0000","last_change":"2024-09-16T15:42:50.241747+0000","last_active":"2024-09-16T15:42:51.219746+0000","last_peered":"2024-09-16T15:42:51.219746+0000","last_clean":"2024-09-16T15:42:51.219746+0000","last_became_active":"2024-09-16T15:42:50.241449+0000","last_became_peered":"2024-09-16T15:42:50.241449+0000","last_unstale":"2024-09-16T15:42:51.219746+0000","last_undegraded":"2024-09-16T15:42:51.219746+0000","last_fullsized":"2024-09-16T15:42:51.219746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:18.883958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222354+0000","last_change":"2024-09-16T15:42:50.244421+0000","last_active":"2024-09-16T15:42:51.222354+0000","last_peered":"2024-09-16T15:42:51.222354+0000","last_clean":"2024-09-16T15:42:51.222354+0000","last_became_active":"2024-09-16T15:42:50.244148+0000","last_became_peered":"2024-09-16T15:42:50.244148+0000","last_unstale":"2024-09-16T15:42:51.222354+0000","last_undegraded":"2024-09-16T15:42:51.222354+0000","last_fullsized":"2024-09-16T15:42:51.222354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:21.839575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240471+0000","last_change":"2024-09-16T15:42:50.238173+0000","last_active":"2024-09-16T15:42:51.240471+0000","last_peered":"2024-09-16T15:42:51.240471+0000","last_clean":"2024-09-16T15:42:51.240471+0000","last_became_active":"2024-09-16T15:42:50.237909+0000","last_became_peered":"2024-09-16T15:42:50.237909+0000","last_unstale":"2024-09-16T15:42:51.240471+0000","last_undegraded":"2024-09-16T15:42:51.240471+0000","last_fullsized":"2024-09-16T15:42:51.240471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:45.525365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671481+0000","last_change":"2024-09-16T15:42:50.235046+0000","last_active":"2024-09-16T15:42:51.671481+0000","last_peered":"2024-09-16T15:42:51.671481+0000","last_clean":"2024-09-16T15:42:51.671481+0000","last_became_active":"2024-09-16T15:42:50.234185+0000","last_became_peered":"2024-09-16T15:42:50.234185+0000","last_unstale":"2024-09-16T15:42:51.671481+0000","last_undegraded":"2024-09-16T15:42:51.671481+0000","last_fullsized":"2024-09-16T15:42:51.671481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:09.791095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671125+0000","last_change":"2024-09-16T15:42:50.248296+0000","last_active":"2024-09-16T15:42:51.671125+0000","last_peered":"2024-09-16T15:42:51.671125+0000","last_clean":"2024-09-16T15:42:51.671125+0000","last_became_active":"2024-09-16T15:42:50.247898+0000","last_became_peered":"2024-09-16T15:42:50.247898+0000","last_unstale":"2024-09-16T15:42:51.671125+0000","last_undegraded":"2024-09-16T15:42:51.671125+0000","last_fullsized":"2024-09-16T15:42:51.671125+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:21:50.270344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631984+0000","last_change":"2024-09-16T15:42:50.236819+0000","last_active":"2024-09-16T15:42:51.631984+0000","last_peered":"2024-09-16T15:42:51.631984+0000","last_clean":"2024-09-16T15:42:51.631984+0000","last_became_active":"2024-09-16T15:42:50.236538+0000","last_became_peered":"2024-09-16T15:42:50.236538+0000","last_unstale":"2024-09-16T15:42:51.631984+0000","last_undegraded":"2024-09-16T15:42:51.631984+0000","last_fullsized":"2024-09-16T15:42:51.631984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:13.298137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632075+0000","last_change":"2024-09-16T15:42:50.244718+0000","last_active":"2024-09-16T15:42:51.632075+0000","last_peered":"2024-09-16T15:42:51.632075+0000","last_clean":"2024-09-16T15:42:51.632075+0000","last_became_active":"2024-09-16T15:42:50.244439+0000","last_became_peered":"2024-09-16T15:42:50.244439+0000","last_unstale":"2024-09-16T15:42:51.632075+0000","last_undegraded":"2024-09-16T15:42:51.632075+0000","last_fullsized":"2024-09-16T15:42:51.632075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:41.188566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671430+0000","last_change":"2024-09-16T15:42:50.239034+0000","last_active":"2024-09-16T15:42:51.671430+0000","last_peered":"2024-09-16T15:42:51.671430+0000","last_clean":"2024-09-16T15:42:51.671430+0000","last_became_active":"2024-09-16T15:42:50.236895+0000","last_became_peered":"2024-09-16T15:42:50.236895+0000","last_unstale":"2024-09-16T15:42:51.671430+0000","last_undegraded":"2024-09-16T15:42:51.671430+0000","last_fullsized":"2024-09-16T15:42:51.671430+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:41.161965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221945+0000","last_change":"2024-09-16T15:42:50.246550+0000","last_active":"2024-09-16T15:42:51.221945+0000","last_peered":"2024-09-16T15:42:51.221945+0000","last_clean":"2024-09-16T15:42:51.221945+0000","last_became_active":"2024-09-16T15:42:50.246299+0000","last_became_peered":"2024-09-16T15:42:50.246299+0000","last_unstale":"2024-09-16T15:42:51.221945+0000","last_undegraded":"2024-09-16T15:42:51.221945+0000","last_fullsized":"2024-09-16T15:42:51.221945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:37.350000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886751+0000","last_change":"2024-09-16T15:42:50.238140+0000","last_active":"2024-09-16T15:42:51.886751+0000","last_peered":"2024-09-16T15:42:51.886751+0000","last_clean":"2024-09-16T15:42:51.886751+0000","last_became_active":"2024-09-16T15:42:50.237877+0000","last_became_peered":"2024-09-16T15:42:50.237877+0000","last_unstale":"2024-09-16T15:42:51.886751+0000","last_undegraded":"2024-09-16T15:42:51.886751+0000","last_fullsized":"2024-09-16T15:42:51.886751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:36.177284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671331+0000","last_change":"2024-09-16T15:42:50.235453+0000","last_active":"2024-09-16T15:42:51.671331+0000","last_peered":"2024-09-16T15:42:51.671331+0000","last_clean":"2024-09-16T15:42:51.671331+0000","last_became_active":"2024-09-16T15:42:50.235194+0000","last_became_peered":"2024-09-16T15:42:50.235194+0000","last_unstale":"2024-09-16T15:42:51.671331+0000","last_undegraded":"2024-09-16T15:42:51.671331+0000","last_fullsized":"2024-09-16T15:42:51.671331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:02.139017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886619+0000","last_change":"2024-09-16T15:42:50.237574+0000","last_active":"2024-09-16T15:42:51.886619+0000","last_peered":"2024-09-16T15:42:51.886619+0000","last_clean":"2024-09-16T15:42:51.886619+0000","last_became_active":"2024-09-16T15:42:50.236242+0000","last_became_peered":"2024-09-16T15:42:50.236242+0000","last_unstale":"2024-09-16T15:42:51.886619+0000","last_undegraded":"2024-09-16T15:42:51.886619+0000","last_fullsized":"2024-09-16T15:42:51.886619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:35.801391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222105+0000","last_change":"2024-09-16T15:42:50.243951+0000","last_active":"2024-09-16T15:42:51.222105+0000","last_peered":"2024-09-16T15:42:51.222105+0000","last_clean":"2024-09-16T15:42:51.222105+0000","last_became_active":"2024-09-16T15:42:50.243705+0000","last_became_peered":"2024-09-16T15:42:50.243705+0000","last_unstale":"2024-09-16T15:42:51.222105+0000","last_undegraded":"2024-09-16T15:42:51.222105+0000","last_fullsized":"2024-09-16T15:42:51.222105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:08.110180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221417+0000","last_change":"2024-09-16T15:42:50.242601+0000","last_active":"2024-09-16T15:42:51.221417+0000","last_peered":"2024-09-16T15:42:51.221417+0000","last_clean":"2024-09-16T15:42:51.221417+0000","last_became_active":"2024-09-16T15:42:50.239915+0000","last_became_peered":"2024-09-16T15:42:50.239915+0000","last_unstale":"2024-09-16T15:42:51.221417+0000","last_undegraded":"2024-09-16T15:42:51.221417+0000","last_fullsized":"2024-09-16T15:42:51.221417+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:30.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221817+0000","last_change":"2024-09-16T15:42:50.237521+0000","last_active":"2024-09-16T15:42:51.221817+0000","last_peered":"2024-09-16T15:42:51.221817+0000","last_clean":"2024-09-16T15:42:51.221817+0000","last_became_active":"2024-09-16T15:42:50.236895+0000","last_became_peered":"2024-09-16T15:42:50.236895+0000","last_unstale":"2024-09-16T15:42:51.221817+0000","last_undegraded":"2024-09-16T15:42:51.221817+0000","last_fullsized":"2024-09-16T15:42:51.221817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:28.871447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632142+0000","last_change":"2024-09-16T15:42:50.244749+0000","last_active":"2024-09-16T15:42:51.632142+0000","last_peered":"2024-09-16T15:42:51.632142+0000","last_clean":"2024-09-16T15:42:51.632142+0000","last_became_active":"2024-09-16T15:42:50.244425+0000","last_became_peered":"2024-09-16T15:42:50.244425+0000","last_unstale":"2024-09-16T15:42:51.632142+0000","last_undegraded":"2024-09-16T15:42:51.632142+0000","last_fullsized":"2024-09-16T15:42:51.632142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:49.583061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632378+0000","last_change":"2024-09-16T15:42:49.230741+0000","last_active":"2024-09-16T15:42:51.632378+0000","last_peered":"2024-09-16T15:42:51.632378+0000","last_clean":"2024-09-16T15:42:51.632378+0000","last_became_active":"2024-09-16T15:42:49.230054+0000","last_became_peered":"2024-09-16T15:42:49.230054+0000","last_unstale":"2024-09-16T15:42:51.632378+0000","last_undegraded":"2024-09-16T15:42:51.632378+0000","last_fullsized":"2024-09-16T15:42:51.632378+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:06:05.634754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533214+0000","last_change":"2024-09-16T15:42:50.249173+0000","last_active":"2024-09-16T15:42:51.533214+0000","last_peered":"2024-09-16T15:42:51.533214+0000","last_clean":"2024-09-16T15:42:51.533214+0000","last_became_active":"2024-09-16T15:42:50.247280+0000","last_became_peered":"2024-09-16T15:42:50.247280+0000","last_unstale":"2024-09-16T15:42:51.533214+0000","last_undegraded":"2024-09-16T15:42:51.533214+0000","last_fullsized":"2024-09-16T15:42:51.533214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:03.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240547+0000","last_change":"2024-09-16T15:42:49.227671+0000","last_active":"2024-09-16T15:42:51.240547+0000","last_peered":"2024-09-16T15:42:51.240547+0000","last_clean":"2024-09-16T15:42:51.240547+0000","last_became_active":"2024-09-16T15:42:49.227333+0000","last_became_peered":"2024-09-16T15:42:49.227333+0000","last_unstale":"2024-09-16T15:42:51.240547+0000","last_undegraded":"2024-09-16T15:42:51.240547+0000","last_fullsized":"2024-09-16T15:42:51.240547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:06:05.634754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222367+0000","last_change":"2024-09-16T15:42:50.243680+0000","last_active":"2024-09-16T15:42:51.222367+0000","last_peered":"2024-09-16T15:42:51.222367+0000","last_clean":"2024-09-16T15:42:51.222367+0000","last_became_active":"2024-09-16T15:42:50.242813+0000","last_became_peered":"2024-09-16T15:42:50.242813+0000","last_unstale":"2024-09-16T15:42:51.222367+0000","last_undegraded":"2024-09-16T15:42:51.222367+0000","last_fullsized":"2024-09-16T15:42:51.222367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:23.095221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671357+0000","last_change":"2024-09-16T15:42:49.228304+0000","last_active":"2024-09-16T15:42:51.671357+0000","last_peered":"2024-09-16T15:42:51.671357+0000","last_clean":"2024-09-16T15:42:51.671357+0000","last_became_active":"2024-09-16T15:42:49.227962+0000","last_became_peered":"2024-09-16T15:42:49.227962+0000","last_unstale":"2024-09-16T15:42:51.671357+0000","last_undegraded":"2024-09-16T15:42:51.671357+0000","last_fullsized":"2024-09-16T15:42:51.671357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:45.388826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632006+0000","last_change":"2024-09-16T15:42:50.240957+0000","last_active":"2024-09-16T15:42:51.632006+0000","last_peered":"2024-09-16T15:42:51.632006+0000","last_clean":"2024-09-16T15:42:51.632006+0000","last_became_active":"2024-09-16T15:42:50.240356+0000","last_became_peered":"2024-09-16T15:42:50.240356+0000","last_unstale":"2024-09-16T15:42:51.632006+0000","last_undegraded":"2024-09-16T15:42:51.632006+0000","last_fullsized":"2024-09-16T15:42:51.632006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:20.329239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222688+0000","last_change":"2024-09-16T15:42:49.215851+0000","last_active":"2024-09-16T15:42:51.222688+0000","last_peered":"2024-09-16T15:42:51.222688+0000","last_clean":"2024-09-16T15:42:51.222688+0000","last_became_active":"2024-09-16T15:42:49.215587+0000","last_became_peered":"2024-09-16T15:42:49.215587+0000","last_unstale":"2024-09-16T15:42:51.222688+0000","last_undegraded":"2024-09-16T15:42:51.222688+0000","last_fullsized":"2024-09-16T15:42:51.222688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:52.618435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221588+0000","last_change":"2024-09-16T15:42:50.238522+0000","last_active":"2024-09-16T15:42:51.221588+0000","last_peered":"2024-09-16T15:42:51.221588+0000","last_clean":"2024-09-16T15:42:51.221588+0000","last_became_active":"2024-09-16T15:42:50.238223+0000","last_became_peered":"2024-09-16T15:42:50.238223+0000","last_unstale":"2024-09-16T15:42:51.221588+0000","last_undegraded":"2024-09-16T15:42:51.221588+0000","last_fullsized":"2024-09-16T15:42:51.221588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:45.525365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886773+0000","last_change":"2024-09-16T15:42:49.220153+0000","last_active":"2024-09-16T15:42:51.886773+0000","last_peered":"2024-09-16T15:42:51.886773+0000","last_clean":"2024-09-16T15:42:51.886773+0000","last_became_active":"2024-09-16T15:42:49.219786+0000","last_became_peered":"2024-09-16T15:42:49.219786+0000","last_unstale":"2024-09-16T15:42:51.886773+0000","last_undegraded":"2024-09-16T15:42:51.886773+0000","last_fullsized":"2024-09-16T15:42:51.886773+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:51:54.851547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671591+0000","last_change":"2024-09-16T15:42:50.237887+0000","last_active":"2024-09-16T15:42:51.671591+0000","last_peered":"2024-09-16T15:42:51.671591+0000","last_clean":"2024-09-16T15:42:51.671591+0000","last_became_active":"2024-09-16T15:42:50.237604+0000","last_became_peered":"2024-09-16T15:42:50.237604+0000","last_unstale":"2024-09-16T15:42:51.671591+0000","last_undegraded":"2024-09-16T15:42:51.671591+0000","last_fullsized":"2024-09-16T15:42:51.671591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:36.020627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222428+0000","last_change":"2024-09-16T15:42:49.227675+0000","last_active":"2024-09-16T15:42:51.222428+0000","last_peered":"2024-09-16T15:42:51.222428+0000","last_clean":"2024-09-16T15:42:51.222428+0000","last_became_active":"2024-09-16T15:42:49.227417+0000","last_became_peered":"2024-09-16T15:42:49.227417+0000","last_unstale":"2024-09-16T15:42:51.222428+0000","last_undegraded":"2024-09-16T15:42:51.222428+0000","last_fullsized":"2024-09-16T15:42:51.222428+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:56.796817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532690+0000","last_change":"2024-09-16T15:42:50.240752+0000","last_active":"2024-09-16T15:42:51.532690+0000","last_peered":"2024-09-16T15:42:51.532690+0000","last_clean":"2024-09-16T15:42:51.532690+0000","last_became_active":"2024-09-16T15:42:50.240455+0000","last_became_peered":"2024-09-16T15:42:50.240455+0000","last_unstale":"2024-09-16T15:42:51.532690+0000","last_undegraded":"2024-09-16T15:42:51.532690+0000","last_fullsized":"2024-09-16T15:42:51.532690+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:23.877726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219262+0000","last_change":"2024-09-16T15:42:49.213568+0000","last_active":"2024-09-16T15:42:51.219262+0000","last_peered":"2024-09-16T15:42:51.219262+0000","last_clean":"2024-09-16T15:42:51.219262+0000","last_became_active":"2024-09-16T15:42:49.213239+0000","last_became_peered":"2024-09-16T15:42:49.213239+0000","last_unstale":"2024-09-16T15:42:51.219262+0000","last_undegraded":"2024-09-16T15:42:51.219262+0000","last_fullsized":"2024-09-16T15:42:51.219262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:38.219818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240496+0000","last_change":"2024-09-16T15:42:50.236445+0000","last_active":"2024-09-16T15:42:51.240496+0000","last_peered":"2024-09-16T15:42:51.240496+0000","last_clean":"2024-09-16T15:42:51.240496+0000","last_became_active":"2024-09-16T15:42:50.236140+0000","last_became_peered":"2024-09-16T15:42:50.236140+0000","last_unstale":"2024-09-16T15:42:51.240496+0000","last_undegraded":"2024-09-16T15:42:51.240496+0000","last_fullsized":"2024-09-16T15:42:51.240496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:01.692023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532596+0000","last_change":"2024-09-16T15:42:49.227614+0000","last_active":"2024-09-16T15:42:51.532596+0000","last_peered":"2024-09-16T15:42:51.532596+0000","last_clean":"2024-09-16T15:42:51.532596+0000","last_became_active":"2024-09-16T15:42:49.226862+0000","last_became_peered":"2024-09-16T15:42:49.226862+0000","last_unstale":"2024-09-16T15:42:51.532596+0000","last_undegraded":"2024-09-16T15:42:51.532596+0000","last_fullsized":"2024-09-16T15:42:51.532596+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:42.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221982+0000","last_change":"2024-09-16T15:42:50.242143+0000","last_active":"2024-09-16T15:42:51.221982+0000","last_peered":"2024-09-16T15:42:51.221982+0000","last_clean":"2024-09-16T15:42:51.221982+0000","last_became_active":"2024-09-16T15:42:50.241844+0000","last_became_peered":"2024-09-16T15:42:50.241844+0000","last_unstale":"2024-09-16T15:42:51.221982+0000","last_undegraded":"2024-09-16T15:42:51.221982+0000","last_fullsized":"2024-09-16T15:42:51.221982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:01.692023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671310+0000","last_change":"2024-09-16T15:42:49.218496+0000","last_active":"2024-09-16T15:42:51.671310+0000","last_peered":"2024-09-16T15:42:51.671310+0000","last_clean":"2024-09-16T15:42:51.671310+0000","last_became_active":"2024-09-16T15:42:49.218195+0000","last_became_peered":"2024-09-16T15:42:49.218195+0000","last_unstale":"2024-09-16T15:42:51.671310+0000","last_undegraded":"2024-09-16T15:42:51.671310+0000","last_fullsized":"2024-09-16T15:42:51.671310+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:33.304250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632271+0000","last_change":"2024-09-16T15:42:50.246020+0000","last_active":"2024-09-16T15:42:51.632271+0000","last_peered":"2024-09-16T15:42:51.632271+0000","last_clean":"2024-09-16T15:42:51.632271+0000","last_became_active":"2024-09-16T15:42:50.245660+0000","last_became_peered":"2024-09-16T15:42:50.245660+0000","last_unstale":"2024-09-16T15:42:51.632271+0000","last_undegraded":"2024-09-16T15:42:51.632271+0000","last_fullsized":"2024-09-16T15:42:51.632271+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:50.807347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222757+0000","last_change":"2024-09-16T15:42:49.225639+0000","last_active":"2024-09-16T15:42:51.222757+0000","last_peered":"2024-09-16T15:42:51.222757+0000","last_clean":"2024-09-16T15:42:51.222757+0000","last_became_active":"2024-09-16T15:42:49.225353+0000","last_became_peered":"2024-09-16T15:42:49.225353+0000","last_unstale":"2024-09-16T15:42:51.222757+0000","last_undegraded":"2024-09-16T15:42:51.222757+0000","last_fullsized":"2024-09-16T15:42:51.222757+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:31.795804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222591+0000","last_change":"2024-09-16T15:42:50.235425+0000","last_active":"2024-09-16T15:42:51.222591+0000","last_peered":"2024-09-16T15:42:51.222591+0000","last_clean":"2024-09-16T15:42:51.222591+0000","last_became_active":"2024-09-16T15:42:50.235052+0000","last_became_peered":"2024-09-16T15:42:50.235052+0000","last_unstale":"2024-09-16T15:42:51.222591+0000","last_undegraded":"2024-09-16T15:42:51.222591+0000","last_fullsized":"2024-09-16T15:42:51.222591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:55.386280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886834+0000","last_change":"2024-09-16T15:42:49.221761+0000","last_active":"2024-09-16T15:42:51.886834+0000","last_peered":"2024-09-16T15:42:51.886834+0000","last_clean":"2024-09-16T15:42:51.886834+0000","last_became_active":"2024-09-16T15:42:49.221491+0000","last_became_peered":"2024-09-16T15:42:49.221491+0000","last_unstale":"2024-09-16T15:42:51.886834+0000","last_undegraded":"2024-09-16T15:42:51.886834+0000","last_fullsized":"2024-09-16T15:42:51.886834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:59.840954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532952+0000","last_change":"2024-09-16T15:42:50.249860+0000","last_active":"2024-09-16T15:42:51.532952+0000","last_peered":"2024-09-16T15:42:51.532952+0000","last_clean":"2024-09-16T15:42:51.532952+0000","last_became_active":"2024-09-16T15:42:50.244511+0000","last_became_peered":"2024-09-16T15:42:50.244511+0000","last_unstale":"2024-09-16T15:42:51.532952+0000","last_undegraded":"2024-09-16T15:42:51.532952+0000","last_fullsized":"2024-09-16T15:42:51.532952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:19.917538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219656+0000","last_change":"2024-09-16T15:42:49.219530+0000","last_active":"2024-09-16T15:42:51.219656+0000","last_peered":"2024-09-16T15:42:51.219656+0000","last_clean":"2024-09-16T15:42:51.219656+0000","last_became_active":"2024-09-16T15:42:49.219019+0000","last_became_peered":"2024-09-16T15:42:49.219019+0000","last_unstale":"2024-09-16T15:42:51.219656+0000","last_undegraded":"2024-09-16T15:42:51.219656+0000","last_fullsized":"2024-09-16T15:42:51.219656+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:46.888005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631758+0000","last_change":"2024-09-16T15:42:50.245485+0000","last_active":"2024-09-16T15:42:51.631758+0000","last_peered":"2024-09-16T15:42:51.631758+0000","last_clean":"2024-09-16T15:42:51.631758+0000","last_became_active":"2024-09-16T15:42:50.244917+0000","last_became_peered":"2024-09-16T15:42:50.244917+0000","last_unstale":"2024-09-16T15:42:51.631758+0000","last_undegraded":"2024-09-16T15:42:51.631758+0000","last_fullsized":"2024-09-16T15:42:51.631758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:42.918396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219758+0000","last_change":"2024-09-16T15:42:49.225850+0000","last_active":"2024-09-16T15:42:51.219758+0000","last_peered":"2024-09-16T15:42:51.219758+0000","last_clean":"2024-09-16T15:42:51.219758+0000","last_became_active":"2024-09-16T15:42:49.225497+0000","last_became_peered":"2024-09-16T15:42:49.225497+0000","last_unstale":"2024-09-16T15:42:51.219758+0000","last_undegraded":"2024-09-16T15:42:51.219758+0000","last_fullsized":"2024-09-16T15:42:51.219758+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:15.332359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222175+0000","last_change":"2024-09-16T15:42:50.236674+0000","last_active":"2024-09-16T15:42:51.222175+0000","last_peered":"2024-09-16T15:42:51.222175+0000","last_clean":"2024-09-16T15:42:51.222175+0000","last_became_active":"2024-09-16T15:42:50.236411+0000","last_became_peered":"2024-09-16T15:42:50.236411+0000","last_unstale":"2024-09-16T15:42:51.222175+0000","last_undegraded":"2024-09-16T15:42:51.222175+0000","last_fullsized":"2024-09-16T15:42:51.222175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:41.139551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222537+0000","last_change":"2024-09-16T15:42:49.227184+0000","last_active":"2024-09-16T15:42:51.222537+0000","last_peered":"2024-09-16T15:42:51.222537+0000","last_clean":"2024-09-16T15:42:51.222537+0000","last_became_active":"2024-09-16T15:42:49.226894+0000","last_became_peered":"2024-09-16T15:42:49.226894+0000","last_unstale":"2024-09-16T15:42:51.222537+0000","last_undegraded":"2024-09-16T15:42:51.222537+0000","last_fullsized":"2024-09-16T15:42:51.222537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:13.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240447+0000","last_change":"2024-09-16T15:42:50.242332+0000","last_active":"2024-09-16T15:42:51.240447+0000","last_peered":"2024-09-16T15:42:51.240447+0000","last_clean":"2024-09-16T15:42:51.240447+0000","last_became_active":"2024-09-16T15:42:50.241852+0000","last_became_peered":"2024-09-16T15:42:50.241852+0000","last_unstale":"2024-09-16T15:42:51.240447+0000","last_undegraded":"2024-09-16T15:42:51.240447+0000","last_fullsized":"2024-09-16T15:42:51.240447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:39.104202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221221+0000","last_change":"2024-09-16T15:42:49.229983+0000","last_active":"2024-09-16T15:42:51.221221+0000","last_peered":"2024-09-16T15:42:51.221221+0000","last_clean":"2024-09-16T15:42:51.221221+0000","last_became_active":"2024-09-16T15:42:49.229708+0000","last_became_peered":"2024-09-16T15:42:49.229708+0000","last_unstale":"2024-09-16T15:42:51.221221+0000","last_undegraded":"2024-09-16T15:42:51.221221+0000","last_fullsized":"2024-09-16T15:42:51.221221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:50.565782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532800+0000","last_change":"2024-09-16T15:42:50.237423+0000","last_active":"2024-09-16T15:42:51.532800+0000","last_peered":"2024-09-16T15:42:51.532800+0000","last_clean":"2024-09-16T15:42:51.532800+0000","last_became_active":"2024-09-16T15:42:50.236994+0000","last_became_peered":"2024-09-16T15:42:50.236994+0000","last_unstale":"2024-09-16T15:42:51.532800+0000","last_undegraded":"2024-09-16T15:42:51.532800+0000","last_fullsized":"2024-09-16T15:42:51.532800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:55.202973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.670983+0000","last_change":"2024-09-16T15:42:49.229382+0000","last_active":"2024-09-16T15:42:51.670983+0000","last_peered":"2024-09-16T15:42:51.670983+0000","last_clean":"2024-09-16T15:42:51.670983+0000","last_became_active":"2024-09-16T15:42:49.229101+0000","last_became_peered":"2024-09-16T15:42:49.229101+0000","last_unstale":"2024-09-16T15:42:51.670983+0000","last_undegraded":"2024-09-16T15:42:51.670983+0000","last_fullsized":"2024-09-16T15:42:51.670983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:08:51.344506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671394+0000","last_change":"2024-09-16T15:42:50.229524+0000","last_active":"2024-09-16T15:42:51.671394+0000","last_peered":"2024-09-16T15:42:51.671394+0000","last_clean":"2024-09-16T15:42:51.671394+0000","last_became_active":"2024-09-16T15:42:50.229234+0000","last_became_peered":"2024-09-16T15:42:50.229234+0000","last_unstale":"2024-09-16T15:42:51.671394+0000","last_undegraded":"2024-09-16T15:42:51.671394+0000","last_fullsized":"2024-09-16T15:42:51.671394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:57.651014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221299+0000","last_change":"2024-09-16T15:42:49.226313+0000","last_active":"2024-09-16T15:42:51.221299+0000","last_peered":"2024-09-16T15:42:51.221299+0000","last_clean":"2024-09-16T15:42:51.221299+0000","last_became_active":"2024-09-16T15:42:49.226037+0000","last_became_peered":"2024-09-16T15:42:49.226037+0000","last_unstale":"2024-09-16T15:42:51.221299+0000","last_undegraded":"2024-09-16T15:42:51.221299+0000","last_fullsized":"2024-09-16T15:42:51.221299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:47.386114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632429+0000","last_change":"2024-09-16T15:42:50.246671+0000","last_active":"2024-09-16T15:42:51.632429+0000","last_peered":"2024-09-16T15:42:51.632429+0000","last_clean":"2024-09-16T15:42:51.632429+0000","last_became_active":"2024-09-16T15:42:50.246427+0000","last_became_peered":"2024-09-16T15:42:50.246427+0000","last_unstale":"2024-09-16T15:42:51.632429+0000","last_undegraded":"2024-09-16T15:42:51.632429+0000","last_fullsized":"2024-09-16T15:42:51.632429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.825762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671542+0000","last_change":"2024-09-16T15:42:49.225855+0000","last_active":"2024-09-16T15:42:51.671542+0000","last_peered":"2024-09-16T15:42:51.671542+0000","last_clean":"2024-09-16T15:42:51.671542+0000","last_became_active":"2024-09-16T15:42:49.225235+0000","last_became_peered":"2024-09-16T15:42:49.225235+0000","last_unstale":"2024-09-16T15:42:51.671542+0000","last_undegraded":"2024-09-16T15:42:51.671542+0000","last_fullsized":"2024-09-16T15:42:51.671542+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:04:06.304202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219664+0000","last_change":"2024-09-16T15:42:50.236713+0000","last_active":"2024-09-16T15:42:51.219664+0000","last_peered":"2024-09-16T15:42:51.219664+0000","last_clean":"2024-09-16T15:42:51.219664+0000","last_became_active":"2024-09-16T15:42:50.236233+0000","last_became_peered":"2024-09-16T15:42:50.236233+0000","last_unstale":"2024-09-16T15:42:51.219664+0000","last_undegraded":"2024-09-16T15:42:51.219664+0000","last_fullsized":"2024-09-16T15:42:51.219664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:21.447882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222463+0000","last_change":"2024-09-16T15:42:49.225896+0000","last_active":"2024-09-16T15:42:51.222463+0000","last_peered":"2024-09-16T15:42:51.222463+0000","last_clean":"2024-09-16T15:42:51.222463+0000","last_became_active":"2024-09-16T15:42:49.225606+0000","last_became_peered":"2024-09-16T15:42:49.225606+0000","last_unstale":"2024-09-16T15:42:51.222463+0000","last_undegraded":"2024-09-16T15:42:51.222463+0000","last_fullsized":"2024-09-16T15:42:51.222463+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:57.795996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533018+0000","last_change":"2024-09-16T15:42:50.235937+0000","last_active":"2024-09-16T15:42:51.533018+0000","last_peered":"2024-09-16T15:42:51.533018+0000","last_clean":"2024-09-16T15:42:51.533018+0000","last_became_active":"2024-09-16T15:42:50.234785+0000","last_became_peered":"2024-09-16T15:42:50.234785+0000","last_unstale":"2024-09-16T15:42:51.533018+0000","last_undegraded":"2024-09-16T15:42:51.533018+0000","last_fullsized":"2024-09-16T15:42:51.533018+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:56.537229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631773+0000","last_change":"2024-09-16T15:42:49.232299+0000","last_active":"2024-09-16T15:42:51.631773+0000","last_peered":"2024-09-16T15:42:51.631773+0000","last_clean":"2024-09-16T15:42:51.631773+0000","last_became_active":"2024-09-16T15:42:49.231909+0000","last_became_peered":"2024-09-16T15:42:49.231909+0000","last_unstale":"2024-09-16T15:42:51.631773+0000","last_undegraded":"2024-09-16T15:42:51.631773+0000","last_fullsized":"2024-09-16T15:42:51.631773+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:55.045860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886667+0000","last_change":"2024-09-16T15:42:50.227398+0000","last_active":"2024-09-16T15:42:51.886667+0000","last_peered":"2024-09-16T15:42:51.886667+0000","last_clean":"2024-09-16T15:42:51.886667+0000","last_became_active":"2024-09-16T15:42:50.226978+0000","last_became_peered":"2024-09-16T15:42:50.226978+0000","last_unstale":"2024-09-16T15:42:51.886667+0000","last_undegraded":"2024-09-16T15:42:51.886667+0000","last_fullsized":"2024-09-16T15:42:51.886667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:30.408132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222622+0000","last_change":"2024-09-16T15:42:49.227801+0000","last_active":"2024-09-16T15:42:51.222622+0000","last_peered":"2024-09-16T15:42:51.222622+0000","last_clean":"2024-09-16T15:42:51.222622+0000","last_became_active":"2024-09-16T15:42:49.227544+0000","last_became_peered":"2024-09-16T15:42:49.227544+0000","last_unstale":"2024-09-16T15:42:51.222622+0000","last_undegraded":"2024-09-16T15:42:51.222622+0000","last_fullsized":"2024-09-16T15:42:51.222622+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:21.793711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886376+0000","last_change":"2024-09-16T15:42:50.240845+0000","last_active":"2024-09-16T15:42:51.886376+0000","last_peered":"2024-09-16T15:42:51.886376+0000","last_clean":"2024-09-16T15:42:51.886376+0000","last_became_active":"2024-09-16T15:42:50.240428+0000","last_became_peered":"2024-09-16T15:42:50.240428+0000","last_unstale":"2024-09-16T15:42:51.886376+0000","last_undegraded":"2024-09-16T15:42:51.886376+0000","last_fullsized":"2024-09-16T15:42:51.886376+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:03.926546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221535+0000","last_change":"2024-09-16T15:42:49.227177+0000","last_active":"2024-09-16T15:42:51.221535+0000","last_peered":"2024-09-16T15:42:51.221535+0000","last_clean":"2024-09-16T15:42:51.221535+0000","last_became_active":"2024-09-16T15:42:49.226902+0000","last_became_peered":"2024-09-16T15:42:49.226902+0000","last_unstale":"2024-09-16T15:42:51.221535+0000","last_undegraded":"2024-09-16T15:42:51.221535+0000","last_fullsized":"2024-09-16T15:42:51.221535+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:23.002295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886455+0000","last_change":"2024-09-16T15:42:50.233607+0000","last_active":"2024-09-16T15:42:51.886455+0000","last_peered":"2024-09-16T15:42:51.886455+0000","last_clean":"2024-09-16T15:42:51.886455+0000","last_became_active":"2024-09-16T15:42:50.233301+0000","last_became_peered":"2024-09-16T15:42:50.233301+0000","last_unstale":"2024-09-16T15:42:51.886455+0000","last_undegraded":"2024-09-16T15:42:51.886455+0000","last_fullsized":"2024-09-16T15:42:51.886455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:51:57.347856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240326+0000","last_change":"2024-09-16T15:42:49.232604+0000","last_active":"2024-09-16T15:42:51.240326+0000","last_peered":"2024-09-16T15:42:51.240326+0000","last_clean":"2024-09-16T15:42:51.240326+0000","last_became_active":"2024-09-16T15:42:49.232083+0000","last_became_peered":"2024-09-16T15:42:49.232083+0000","last_unstale":"2024-09-16T15:42:51.240326+0000","last_undegraded":"2024-09-16T15:42:51.240326+0000","last_fullsized":"2024-09-16T15:42:51.240326+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:59.450333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240697+0000","last_change":"2024-09-16T15:42:50.249833+0000","last_active":"2024-09-16T15:42:51.240697+0000","last_peered":"2024-09-16T15:42:51.240697+0000","last_clean":"2024-09-16T15:42:51.240697+0000","last_became_active":"2024-09-16T15:42:50.248359+0000","last_became_peered":"2024-09-16T15:42:50.248359+0000","last_unstale":"2024-09-16T15:42:51.240697+0000","last_undegraded":"2024-09-16T15:42:51.240697+0000","last_fullsized":"2024-09-16T15:42:51.240697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:08.110180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221406+0000","last_change":"2024-09-16T15:42:49.226320+0000","last_active":"2024-09-16T15:42:51.221406+0000","last_peered":"2024-09-16T15:42:51.221406+0000","last_clean":"2024-09-16T15:42:51.221406+0000","last_became_active":"2024-09-16T15:42:49.226026+0000","last_became_peered":"2024-09-16T15:42:49.226026+0000","last_unstale":"2024-09-16T15:42:51.221406+0000","last_undegraded":"2024-09-16T15:42:51.221406+0000","last_fullsized":"2024-09-16T15:42:51.221406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:30.666342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222319+0000","last_change":"2024-09-16T15:42:50.245949+0000","last_active":"2024-09-16T15:42:51.222319+0000","last_peered":"2024-09-16T15:42:51.222319+0000","last_clean":"2024-09-16T15:42:51.222319+0000","last_became_active":"2024-09-16T15:42:50.245641+0000","last_became_peered":"2024-09-16T15:42:50.245641+0000","last_unstale":"2024-09-16T15:42:51.222319+0000","last_undegraded":"2024-09-16T15:42:51.222319+0000","last_fullsized":"2024-09-16T15:42:51.222319+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:05.134119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.239778+0000","last_change":"2024-09-16T15:42:49.230435+0000","last_active":"2024-09-16T15:42:51.239778+0000","last_peered":"2024-09-16T15:42:51.239778+0000","last_clean":"2024-09-16T15:42:51.239778+0000","last_became_active":"2024-09-16T15:42:49.230027+0000","last_became_peered":"2024-09-16T15:42:49.230027+0000","last_unstale":"2024-09-16T15:42:51.239778+0000","last_undegraded":"2024-09-16T15:42:51.239778+0000","last_fullsized":"2024-09-16T15:42:51.239778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:41.164728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631809+0000","last_change":"2024-09-16T15:42:50.245973+0000","last_active":"2024-09-16T15:42:51.631809+0000","last_peered":"2024-09-16T15:42:51.631809+0000","last_clean":"2024-09-16T15:42:51.631809+0000","last_became_active":"2024-09-16T15:42:50.244763+0000","last_became_peered":"2024-09-16T15:42:50.244763+0000","last_unstale":"2024-09-16T15:42:51.631809+0000","last_undegraded":"2024-09-16T15:42:51.631809+0000","last_fullsized":"2024-09-16T15:42:51.631809+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:52.395826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533079+0000","last_change":"2024-09-16T15:42:49.227358+0000","last_active":"2024-09-16T15:42:51.533079+0000","last_peered":"2024-09-16T15:42:51.533079+0000","last_clean":"2024-09-16T15:42:51.533079+0000","last_became_active":"2024-09-16T15:42:49.227081+0000","last_became_peered":"2024-09-16T15:42:49.227081+0000","last_unstale":"2024-09-16T15:42:51.533079+0000","last_undegraded":"2024-09-16T15:42:51.533079+0000","last_fullsized":"2024-09-16T15:42:51.533079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:34.159398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632194+0000","last_change":"2024-09-16T15:42:50.243478+0000","last_active":"2024-09-16T15:42:51.632194+0000","last_peered":"2024-09-16T15:42:51.632194+0000","last_clean":"2024-09-16T15:42:51.632194+0000","last_became_active":"2024-09-16T15:42:50.243195+0000","last_became_peered":"2024-09-16T15:42:50.243195+0000","last_unstale":"2024-09-16T15:42:51.632194+0000","last_undegraded":"2024-09-16T15:42:51.632194+0000","last_fullsized":"2024-09-16T15:42:51.632194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:07.694426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240604+0000","last_change":"2024-09-16T15:42:49.232999+0000","last_active":"2024-09-16T15:42:51.240604+0000","last_peered":"2024-09-16T15:42:51.240604+0000","last_clean":"2024-09-16T15:42:51.240604+0000","last_became_active":"2024-09-16T15:42:49.232433+0000","last_became_peered":"2024-09-16T15:42:49.232433+0000","last_unstale":"2024-09-16T15:42:51.240604+0000","last_undegraded":"2024-09-16T15:42:51.240604+0000","last_fullsized":"2024-09-16T15:42:51.240604+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:55.045860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533009+0000","last_change":"2024-09-16T15:42:50.243676+0000","last_active":"2024-09-16T15:42:51.533009+0000","last_peered":"2024-09-16T15:42:51.533009+0000","last_clean":"2024-09-16T15:42:51.533009+0000","last_became_active":"2024-09-16T15:42:50.242348+0000","last_became_peered":"2024-09-16T15:42:50.242348+0000","last_unstale":"2024-09-16T15:42:51.533009+0000","last_undegraded":"2024-09-16T15:42:51.533009+0000","last_fullsized":"2024-09-16T15:42:51.533009+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:28.756230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532794+0000","last_change":"2024-09-16T15:42:49.230727+0000","last_active":"2024-09-16T15:42:51.532794+0000","last_peered":"2024-09-16T15:42:51.532794+0000","last_clean":"2024-09-16T15:42:51.532794+0000","last_became_active":"2024-09-16T15:42:49.230456+0000","last_became_peered":"2024-09-16T15:42:49.230456+0000","last_unstale":"2024-09-16T15:42:51.532794+0000","last_undegraded":"2024-09-16T15:42:51.532794+0000","last_fullsized":"2024-09-16T15:42:51.532794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:46.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221704+0000","last_change":"2024-09-16T15:42:50.247358+0000","last_active":"2024-09-16T15:42:51.221704+0000","last_peered":"2024-09-16T15:42:51.221704+0000","last_clean":"2024-09-16T15:42:51.221704+0000","last_became_active":"2024-09-16T15:42:50.241832+0000","last_became_peered":"2024-09-16T15:42:50.241832+0000","last_unstale":"2024-09-16T15:42:51.221704+0000","last_undegraded":"2024-09-16T15:42:51.221704+0000","last_fullsized":"2024-09-16T15:42:51.221704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:55.912098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219202+0000","last_change":"2024-09-16T15:42:49.216345+0000","last_active":"2024-09-16T15:42:51.219202+0000","last_peered":"2024-09-16T15:42:51.219202+0000","last_clean":"2024-09-16T15:42:51.219202+0000","last_became_active":"2024-09-16T15:42:49.216038+0000","last_became_peered":"2024-09-16T15:42:49.216038+0000","last_unstale":"2024-09-16T15:42:51.219202+0000","last_undegraded":"2024-09-16T15:42:51.219202+0000","last_fullsized":"2024-09-16T15:42:51.219202+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:30.969988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219391+0000","last_change":"2024-09-16T15:42:50.224269+0000","last_active":"2024-09-16T15:42:51.219391+0000","last_peered":"2024-09-16T15:42:51.219391+0000","last_clean":"2024-09-16T15:42:51.219391+0000","last_became_active":"2024-09-16T15:42:50.223827+0000","last_became_peered":"2024-09-16T15:42:50.223827+0000","last_unstale":"2024-09-16T15:42:51.219391+0000","last_undegraded":"2024-09-16T15:42:51.219391+0000","last_fullsized":"2024-09-16T15:42:51.219391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:20.372919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671409+0000","last_change":"2024-09-16T15:42:49.227215+0000","last_active":"2024-09-16T15:42:51.671409+0000","last_peered":"2024-09-16T15:42:51.671409+0000","last_clean":"2024-09-16T15:42:51.671409+0000","last_became_active":"2024-09-16T15:42:49.226937+0000","last_became_peered":"2024-09-16T15:42:49.226937+0000","last_unstale":"2024-09-16T15:42:51.671409+0000","last_undegraded":"2024-09-16T15:42:51.671409+0000","last_fullsized":"2024-09-16T15:42:51.671409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:26.324867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221383+0000","last_change":"2024-09-16T15:42:50.240843+0000","last_active":"2024-09-16T15:42:51.221383+0000","last_peered":"2024-09-16T15:42:51.221383+0000","last_clean":"2024-09-16T15:42:51.221383+0000","last_became_active":"2024-09-16T15:42:50.237363+0000","last_became_peered":"2024-09-16T15:42:50.237363+0000","last_unstale":"2024-09-16T15:42:51.221383+0000","last_undegraded":"2024-09-16T15:42:51.221383+0000","last_fullsized":"2024-09-16T15:42:51.221383+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:59:03.289883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221918+0000","last_change":"2024-09-16T15:42:49.218784+0000","last_active":"2024-09-16T15:42:51.221918+0000","last_peered":"2024-09-16T15:42:51.221918+0000","last_clean":"2024-09-16T15:42:51.221918+0000","last_became_active":"2024-09-16T15:42:49.218359+0000","last_became_peered":"2024-09-16T15:42:49.218359+0000","last_unstale":"2024-09-16T15:42:51.221918+0000","last_undegraded":"2024-09-16T15:42:51.221918+0000","last_fullsized":"2024-09-16T15:42:51.221918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:19.859750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886579+0000","last_change":"2024-09-16T15:42:50.236702+0000","last_active":"2024-09-16T15:42:51.886579+0000","last_peered":"2024-09-16T15:42:51.886579+0000","last_clean":"2024-09-16T15:42:51.886579+0000","last_became_active":"2024-09-16T15:42:50.236407+0000","last_became_peered":"2024-09-16T15:42:50.236407+0000","last_unstale":"2024-09-16T15:42:51.886579+0000","last_undegraded":"2024-09-16T15:42:51.886579+0000","last_fullsized":"2024-09-16T15:42:51.886579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:55.799552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221894+0000","last_change":"2024-09-16T15:42:49.221746+0000","last_active":"2024-09-16T15:42:51.221894+0000","last_peered":"2024-09-16T15:42:51.221894+0000","last_clean":"2024-09-16T15:42:51.221894+0000","last_became_active":"2024-09-16T15:42:49.221418+0000","last_became_peered":"2024-09-16T15:42:49.221418+0000","last_unstale":"2024-09-16T15:42:51.221894+0000","last_undegraded":"2024-09-16T15:42:51.221894+0000","last_fullsized":"2024-09-16T15:42:51.221894+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:09.934857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":207,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631522+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:42:51.631522+0000","last_peered":"2024-09-16T15:42:51.631522+0000","last_clean":"2024-09-16T15:42:51.631522+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:42:51.631522+0000","last_undegraded":"2024-09-16T15:42:51.631522+0000","last_fullsized":"2024-09-16T15:42:51.631522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509498,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1495040,"data_stored":965744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509498,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":901120,"data_stored":374852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509496,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":897024,"data_stored":374850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":897024,"data_stored":374830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":901120,"data_stored":374872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509496,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":905216,"data_stored":376372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509498,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1486848,"data_stored":965128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36240,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":913408,"data_stored":376988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:54.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.667+0000 7fe792400640 1 -- 172.21.15.60:0/3301046134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78c073fb0 msgr2=0x7fe78c074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.667+0000 7fe792400640 1 --2- 172.21.15.60:0/3301046134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78c073fb0 0x7fe78c074440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe784005fd0 tx=0x7fe78402aa10 comp rx=0 tx=0).stop 2024-09-16T15:42:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.667+0000 7fe792400640 1 -- 172.21.15.60:0/3301046134 shutdown_connections 2024-09-16T15:42:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.667+0000 7fe792400640 1 --2- 172.21.15.60:0/3301046134 >> v2:172.21.15.100:3300/0 conn(0x7fe78c0ccf00 0x7fe78c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.667+0000 7fe792400640 1 --2- 172.21.15.60:0/3301046134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78c073fb0 0x7fe78c074440 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.667+0000 7fe792400640 1 -- 172.21.15.60:0/3301046134 >> 172.21.15.60:0/3301046134 conn(0x7fe78c0b1950 msgr2=0x7fe78c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.667+0000 7fe792400640 1 -- 172.21.15.60:0/3301046134 shutdown_connections 2024-09-16T15:42:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.667+0000 7fe792400640 1 -- 172.21.15.60:0/3301046134 wait complete. 2024-09-16T15:42:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.669+0000 7fe792400640 1 Processor -- start 2024-09-16T15:42:54.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.669+0000 7fe792400640 1 -- start start 2024-09-16T15:42:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe792400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78c073fb0 0x7fe78c1460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe792400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78c0ccf00 0x7fe78c1f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe792400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe78c15f5a0 0x7fe78c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe792400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c06f2e0 con 0x7fe78c0ccf00 2024-09-16T15:42:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe792400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c06f160 con 0x7fe78c15f5a0 2024-09-16T15:42:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe792400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe78c06f460 con 0x7fe78c073fb0 2024-09-16T15:42:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78c0ccf00 0x7fe78c1f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe78aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78c0ccf00 0x7fe78c1f6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60138/0 (socket says 172.21.15.60:60138) 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe78aa00640 1 -- 172.21.15.60:0/1118542115 learned_addr learned my addr 172.21.15.60:0/1118542115 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe78b400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78c073fb0 0x7fe78c1460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.670+0000 7fe78be00640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe78c15f5a0 0x7fe78c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe78aa00640 1 -- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78c073fb0 msgr2=0x7fe78c1460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe78aa00640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78c073fb0 0x7fe78c1460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe78aa00640 1 -- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe78c15f5a0 msgr2=0x7fe78c0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe78aa00640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe78c15f5a0 0x7fe78c0cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe78aa00640 1 -- 172.21.15.60:0/1118542115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c1f7280 con 0x7fe78c0ccf00 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe78aa00640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78c0ccf00 0x7fe78c1f6370 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe77800ea20 tx=0x7fe77800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe783400640 1 -- 172.21.15.60:0/1118542115 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77800cd80 con 0x7fe78c0ccf00 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c146630 con 0x7fe78c0ccf00 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe783400640 1 -- 172.21.15.60:0/1118542115 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe778005020 con 0x7fe78c0ccf00 2024-09-16T15:42:54.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe783400640 1 -- 172.21.15.60:0/1118542115 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778005450 con 0x7fe78c0ccf00 2024-09-16T15:42:54.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.671+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1f6fa0 con 0x7fe78c0ccf00 2024-09-16T15:42:54.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.672+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78c06f980 con 0x7fe78c0ccf00 2024-09-16T15:42:54.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.674+0000 7fe783400640 1 -- 172.21.15.60:0/1118542115 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fe7780047a0 con 0x7fe78c0ccf00 2024-09-16T15:42:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.674+0000 7fe783400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe76c078bb0 0x7fe76c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.675+0000 7fe783400640 1 -- 172.21.15.60:0/1118542115 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7fe77809bab0 con 0x7fe78c0ccf00 2024-09-16T15:42:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.675+0000 7fe78b400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe76c078bb0 0x7fe76c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.676+0000 7fe78b400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe76c078bb0 0x7fe76c07b070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe77c001720 tx=0x7fe77c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:54.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.679+0000 7fe783400640 1 -- 172.21.15.60:0/1118542115 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe778063470 con 0x7fe78c0ccf00 2024-09-16T15:42:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.871+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe78c155ce0 con 0x7fe76c078bb0 2024-09-16T15:42:54.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.876+0000 7fe783400640 1 -- 172.21.15.60:0/1118542115 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252058 (secure 0 0 0) 0x7fe78c155ce0 con 0x7fe76c078bb0 2024-09-16T15:42:54.881 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:54.882 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:42:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe76c078bb0 msgr2=0x7fe76c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe76c078bb0 0x7fe76c07b070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe77c001720 tx=0x7fe77c009040 comp rx=0 tx=0).stop 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78c0ccf00 msgr2=0x7fe78c1f6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78c0ccf00 0x7fe78c1f6370 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe77800ea20 tx=0x7fe77800ee20 comp rx=0 tx=0).stop 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 shutdown_connections 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fe76c078bb0 0x7fe76c07b070 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe78c15f5a0 0x7fe78c0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78c0ccf00 0x7fe78c1f6370 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 --2- 172.21.15.60:0/1118542115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78c073fb0 0x7fe78c1460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 >> 172.21.15.60:0/1118542115 conn(0x7fe78c0b1950 msgr2=0x7fe78c15e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 shutdown_connections 2024-09-16T15:42:54.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:54.880+0000 7fe792400640 1 -- 172.21.15.60:0/1118542115 wait complete. 2024-09-16T15:42:54.898 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-16T15:42:53.776252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249412,"kb_used_data":8200,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725308,"statfs":{"total":773094113280,"available":772838715392,"internally_reserved":0,"allocated":8396800,"data_stored":4183636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.575010"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632224+0000","last_change":"2024-09-16T15:42:50.246949+0000","last_active":"2024-09-16T15:42:51.632224+0000","last_peered":"2024-09-16T15:42:51.632224+0000","last_clean":"2024-09-16T15:42:51.632224+0000","last_became_active":"2024-09-16T15:42:50.245057+0000","last_became_peered":"2024-09-16T15:42:50.245057+0000","last_unstale":"2024-09-16T15:42:51.632224+0000","last_undegraded":"2024-09-16T15:42:51.632224+0000","last_fullsized":"2024-09-16T15:42:51.632224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:32.199533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222866+0000","last_change":"2024-09-16T15:42:50.241966+0000","last_active":"2024-09-16T15:42:51.222866+0000","last_peered":"2024-09-16T15:42:51.222866+0000","last_clean":"2024-09-16T15:42:51.222866+0000","last_became_active":"2024-09-16T15:42:50.240434+0000","last_became_peered":"2024-09-16T15:42:50.240434+0000","last_unstale":"2024-09-16T15:42:51.222866+0000","last_undegraded":"2024-09-16T15:42:51.222866+0000","last_fullsized":"2024-09-16T15:42:51.222866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:50.405364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886709+0000","last_change":"2024-09-16T15:42:50.232921+0000","last_active":"2024-09-16T15:42:51.886709+0000","last_peered":"2024-09-16T15:42:51.886709+0000","last_clean":"2024-09-16T15:42:51.886709+0000","last_became_active":"2024-09-16T15:42:50.232658+0000","last_became_peered":"2024-09-16T15:42:50.232658+0000","last_unstale":"2024-09-16T15:42:51.886709+0000","last_undegraded":"2024-09-16T15:42:51.886709+0000","last_fullsized":"2024-09-16T15:42:51.886709+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:26.597473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219437+0000","last_change":"2024-09-16T15:42:50.237383+0000","last_active":"2024-09-16T15:42:51.219437+0000","last_peered":"2024-09-16T15:42:51.219437+0000","last_clean":"2024-09-16T15:42:51.219437+0000","last_became_active":"2024-09-16T15:42:50.237112+0000","last_became_peered":"2024-09-16T15:42:50.237112+0000","last_unstale":"2024-09-16T15:42:51.219437+0000","last_undegraded":"2024-09-16T15:42:51.219437+0000","last_fullsized":"2024-09-16T15:42:51.219437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:44.775395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532971+0000","last_change":"2024-09-16T15:42:50.243001+0000","last_active":"2024-09-16T15:42:51.532971+0000","last_peered":"2024-09-16T15:42:51.532971+0000","last_clean":"2024-09-16T15:42:51.532971+0000","last_became_active":"2024-09-16T15:42:50.241449+0000","last_became_peered":"2024-09-16T15:42:50.241449+0000","last_unstale":"2024-09-16T15:42:51.532971+0000","last_undegraded":"2024-09-16T15:42:51.532971+0000","last_fullsized":"2024-09-16T15:42:51.532971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:41.128580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222308+0000","last_change":"2024-09-16T15:42:50.236079+0000","last_active":"2024-09-16T15:42:51.222308+0000","last_peered":"2024-09-16T15:42:51.222308+0000","last_clean":"2024-09-16T15:42:51.222308+0000","last_became_active":"2024-09-16T15:42:50.234626+0000","last_became_peered":"2024-09-16T15:42:50.234626+0000","last_unstale":"2024-09-16T15:42:51.222308+0000","last_undegraded":"2024-09-16T15:42:51.222308+0000","last_fullsized":"2024-09-16T15:42:51.222308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:18.670844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221523+0000","last_change":"2024-09-16T15:42:50.239884+0000","last_active":"2024-09-16T15:42:51.221523+0000","last_peered":"2024-09-16T15:42:51.221523+0000","last_clean":"2024-09-16T15:42:51.221523+0000","last_became_active":"2024-09-16T15:42:50.238583+0000","last_became_peered":"2024-09-16T15:42:50.238583+0000","last_unstale":"2024-09-16T15:42:51.221523+0000","last_undegraded":"2024-09-16T15:42:51.221523+0000","last_fullsized":"2024-09-16T15:42:51.221523+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:39.104202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221629+0000","last_change":"2024-09-16T15:42:50.237211+0000","last_active":"2024-09-16T15:42:51.221629+0000","last_peered":"2024-09-16T15:42:51.221629+0000","last_clean":"2024-09-16T15:42:51.221629+0000","last_became_active":"2024-09-16T15:42:50.235538+0000","last_became_peered":"2024-09-16T15:42:50.235538+0000","last_unstale":"2024-09-16T15:42:51.221629+0000","last_undegraded":"2024-09-16T15:42:51.221629+0000","last_fullsized":"2024-09-16T15:42:51.221629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:42.073739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533042+0000","last_change":"2024-09-16T15:42:50.248832+0000","last_active":"2024-09-16T15:42:51.533042+0000","last_peered":"2024-09-16T15:42:51.533042+0000","last_clean":"2024-09-16T15:42:51.533042+0000","last_became_active":"2024-09-16T15:42:50.244745+0000","last_became_peered":"2024-09-16T15:42:50.244745+0000","last_unstale":"2024-09-16T15:42:51.533042+0000","last_undegraded":"2024-09-16T15:42:51.533042+0000","last_fullsized":"2024-09-16T15:42:51.533042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:56.282754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671475+0000","last_change":"2024-09-16T15:42:50.250302+0000","last_active":"2024-09-16T15:42:51.671475+0000","last_peered":"2024-09-16T15:42:51.671475+0000","last_clean":"2024-09-16T15:42:51.671475+0000","last_became_active":"2024-09-16T15:42:50.248474+0000","last_became_peered":"2024-09-16T15:42:50.248474+0000","last_unstale":"2024-09-16T15:42:51.671475+0000","last_undegraded":"2024-09-16T15:42:51.671475+0000","last_fullsized":"2024-09-16T15:42:51.671475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:10:26.440645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631934+0000","last_change":"2024-09-16T15:42:50.242486+0000","last_active":"2024-09-16T15:42:51.631934+0000","last_peered":"2024-09-16T15:42:51.631934+0000","last_clean":"2024-09-16T15:42:51.631934+0000","last_became_active":"2024-09-16T15:42:50.240689+0000","last_became_peered":"2024-09-16T15:42:50.240689+0000","last_unstale":"2024-09-16T15:42:51.631934+0000","last_undegraded":"2024-09-16T15:42:51.631934+0000","last_fullsized":"2024-09-16T15:42:51.631934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:11.985920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219465+0000","last_change":"2024-09-16T15:42:50.237151+0000","last_active":"2024-09-16T15:42:51.219465+0000","last_peered":"2024-09-16T15:42:51.219465+0000","last_clean":"2024-09-16T15:42:51.219465+0000","last_became_active":"2024-09-16T15:42:50.236870+0000","last_became_peered":"2024-09-16T15:42:50.236870+0000","last_unstale":"2024-09-16T15:42:51.219465+0000","last_undegraded":"2024-09-16T15:42:51.219465+0000","last_fullsized":"2024-09-16T15:42:51.219465+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:37.469154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240847+0000","last_change":"2024-09-16T15:42:50.236288+0000","last_active":"2024-09-16T15:42:51.240847+0000","last_peered":"2024-09-16T15:42:51.240847+0000","last_clean":"2024-09-16T15:42:51.240847+0000","last_became_active":"2024-09-16T15:42:50.235987+0000","last_became_peered":"2024-09-16T15:42:50.235987+0000","last_unstale":"2024-09-16T15:42:51.240847+0000","last_undegraded":"2024-09-16T15:42:51.240847+0000","last_fullsized":"2024-09-16T15:42:51.240847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:41.139551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533185+0000","last_change":"2024-09-16T15:42:50.248623+0000","last_active":"2024-09-16T15:42:51.533185+0000","last_peered":"2024-09-16T15:42:51.533185+0000","last_clean":"2024-09-16T15:42:51.533185+0000","last_became_active":"2024-09-16T15:42:50.248296+0000","last_became_peered":"2024-09-16T15:42:50.248296+0000","last_unstale":"2024-09-16T15:42:51.533185+0000","last_undegraded":"2024-09-16T15:42:51.533185+0000","last_fullsized":"2024-09-16T15:42:51.533185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:15.570815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240472+0000","last_change":"2024-09-16T15:42:50.240837+0000","last_active":"2024-09-16T15:42:51.240472+0000","last_peered":"2024-09-16T15:42:51.240472+0000","last_clean":"2024-09-16T15:42:51.240472+0000","last_became_active":"2024-09-16T15:42:50.240516+0000","last_became_peered":"2024-09-16T15:42:50.240516+0000","last_unstale":"2024-09-16T15:42:51.240472+0000","last_undegraded":"2024-09-16T15:42:51.240472+0000","last_fullsized":"2024-09-16T15:42:51.240472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:50.681711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671048+0000","last_change":"2024-09-16T15:42:50.237124+0000","last_active":"2024-09-16T15:42:51.671048+0000","last_peered":"2024-09-16T15:42:51.671048+0000","last_clean":"2024-09-16T15:42:51.671048+0000","last_became_active":"2024-09-16T15:42:50.236795+0000","last_became_peered":"2024-09-16T15:42:50.236795+0000","last_unstale":"2024-09-16T15:42:51.671048+0000","last_undegraded":"2024-09-16T15:42:51.671048+0000","last_fullsized":"2024-09-16T15:42:51.671048+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:00.018854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533240+0000","last_change":"2024-09-16T15:42:50.247702+0000","last_active":"2024-09-16T15:42:51.533240+0000","last_peered":"2024-09-16T15:42:51.533240+0000","last_clean":"2024-09-16T15:42:51.533240+0000","last_became_active":"2024-09-16T15:42:50.244483+0000","last_became_peered":"2024-09-16T15:42:50.244483+0000","last_unstale":"2024-09-16T15:42:51.533240+0000","last_undegraded":"2024-09-16T15:42:51.533240+0000","last_fullsized":"2024-09-16T15:42:51.533240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:48:49.368461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219746+0000","last_change":"2024-09-16T15:42:50.241747+0000","last_active":"2024-09-16T15:42:51.219746+0000","last_peered":"2024-09-16T15:42:51.219746+0000","last_clean":"2024-09-16T15:42:51.219746+0000","last_became_active":"2024-09-16T15:42:50.241449+0000","last_became_peered":"2024-09-16T15:42:50.241449+0000","last_unstale":"2024-09-16T15:42:51.219746+0000","last_undegraded":"2024-09-16T15:42:51.219746+0000","last_fullsized":"2024-09-16T15:42:51.219746+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:18.883958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222354+0000","last_change":"2024-09-16T15:42:50.244421+0000","last_active":"2024-09-16T15:42:51.222354+0000","last_peered":"2024-09-16T15:42:51.222354+0000","last_clean":"2024-09-16T15:42:51.222354+0000","last_became_active":"2024-09-16T15:42:50.244148+0000","last_became_peered":"2024-09-16T15:42:50.244148+0000","last_unstale":"2024-09-16T15:42:51.222354+0000","last_undegraded":"2024-09-16T15:42:51.222354+0000","last_fullsized":"2024-09-16T15:42:51.222354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:21.839575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240471+0000","last_change":"2024-09-16T15:42:50.238173+0000","last_active":"2024-09-16T15:42:51.240471+0000","last_peered":"2024-09-16T15:42:51.240471+0000","last_clean":"2024-09-16T15:42:51.240471+0000","last_became_active":"2024-09-16T15:42:50.237909+0000","last_became_peered":"2024-09-16T15:42:50.237909+0000","last_unstale":"2024-09-16T15:42:51.240471+0000","last_undegraded":"2024-09-16T15:42:51.240471+0000","last_fullsized":"2024-09-16T15:42:51.240471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:45.525365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671481+0000","last_change":"2024-09-16T15:42:50.235046+0000","last_active":"2024-09-16T15:42:51.671481+0000","last_peered":"2024-09-16T15:42:51.671481+0000","last_clean":"2024-09-16T15:42:51.671481+0000","last_became_active":"2024-09-16T15:42:50.234185+0000","last_became_peered":"2024-09-16T15:42:50.234185+0000","last_unstale":"2024-09-16T15:42:51.671481+0000","last_undegraded":"2024-09-16T15:42:51.671481+0000","last_fullsized":"2024-09-16T15:42:51.671481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:09.791095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671125+0000","last_change":"2024-09-16T15:42:50.248296+0000","last_active":"2024-09-16T15:42:51.671125+0000","last_peered":"2024-09-16T15:42:51.671125+0000","last_clean":"2024-09-16T15:42:51.671125+0000","last_became_active":"2024-09-16T15:42:50.247898+0000","last_became_peered":"2024-09-16T15:42:50.247898+0000","last_unstale":"2024-09-16T15:42:51.671125+0000","last_undegraded":"2024-09-16T15:42:51.671125+0000","last_fullsized":"2024-09-16T15:42:51.671125+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:21:50.270344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631984+0000","last_change":"2024-09-16T15:42:50.236819+0000","last_active":"2024-09-16T15:42:51.631984+0000","last_peered":"2024-09-16T15:42:51.631984+0000","last_clean":"2024-09-16T15:42:51.631984+0000","last_became_active":"2024-09-16T15:42:50.236538+0000","last_became_peered":"2024-09-16T15:42:50.236538+0000","last_unstale":"2024-09-16T15:42:51.631984+0000","last_undegraded":"2024-09-16T15:42:51.631984+0000","last_fullsized":"2024-09-16T15:42:51.631984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:13.298137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632075+0000","last_change":"2024-09-16T15:42:50.244718+0000","last_active":"2024-09-16T15:42:51.632075+0000","last_peered":"2024-09-16T15:42:51.632075+0000","last_clean":"2024-09-16T15:42:51.632075+0000","last_became_active":"2024-09-16T15:42:50.244439+0000","last_became_peered":"2024-09-16T15:42:50.244439+0000","last_unstale":"2024-09-16T15:42:51.632075+0000","last_undegraded":"2024-09-16T15:42:51.632075+0000","last_fullsized":"2024-09-16T15:42:51.632075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:41.188566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671430+0000","last_change":"2024-09-16T15:42:50.239034+0000","last_active":"2024-09-16T15:42:51.671430+0000","last_peered":"2024-09-16T15:42:51.671430+0000","last_clean":"2024-09-16T15:42:51.671430+0000","last_became_active":"2024-09-16T15:42:50.236895+0000","last_became_peered":"2024-09-16T15:42:50.236895+0000","last_unstale":"2024-09-16T15:42:51.671430+0000","last_undegraded":"2024-09-16T15:42:51.671430+0000","last_fullsized":"2024-09-16T15:42:51.671430+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:41.161965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221945+0000","last_change":"2024-09-16T15:42:50.246550+0000","last_active":"2024-09-16T15:42:51.221945+0000","last_peered":"2024-09-16T15:42:51.221945+0000","last_clean":"2024-09-16T15:42:51.221945+0000","last_became_active":"2024-09-16T15:42:50.246299+0000","last_became_peered":"2024-09-16T15:42:50.246299+0000","last_unstale":"2024-09-16T15:42:51.221945+0000","last_undegraded":"2024-09-16T15:42:51.221945+0000","last_fullsized":"2024-09-16T15:42:51.221945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:37.350000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886751+0000","last_change":"2024-09-16T15:42:50.238140+0000","last_active":"2024-09-16T15:42:51.886751+0000","last_peered":"2024-09-16T15:42:51.886751+0000","last_clean":"2024-09-16T15:42:51.886751+0000","last_became_active":"2024-09-16T15:42:50.237877+0000","last_became_peered":"2024-09-16T15:42:50.237877+0000","last_unstale":"2024-09-16T15:42:51.886751+0000","last_undegraded":"2024-09-16T15:42:51.886751+0000","last_fullsized":"2024-09-16T15:42:51.886751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:36.177284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671331+0000","last_change":"2024-09-16T15:42:50.235453+0000","last_active":"2024-09-16T15:42:51.671331+0000","last_peered":"2024-09-16T15:42:51.671331+0000","last_clean":"2024-09-16T15:42:51.671331+0000","last_became_active":"2024-09-16T15:42:50.235194+0000","last_became_peered":"2024-09-16T15:42:50.235194+0000","last_unstale":"2024-09-16T15:42:51.671331+0000","last_undegraded":"2024-09-16T15:42:51.671331+0000","last_fullsized":"2024-09-16T15:42:51.671331+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:02.139017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886619+0000","last_change":"2024-09-16T15:42:50.237574+0000","last_active":"2024-09-16T15:42:51.886619+0000","last_peered":"2024-09-16T15:42:51.886619+0000","last_clean":"2024-09-16T15:42:51.886619+0000","last_became_active":"2024-09-16T15:42:50.236242+0000","last_became_peered":"2024-09-16T15:42:50.236242+0000","last_unstale":"2024-09-16T15:42:51.886619+0000","last_undegraded":"2024-09-16T15:42:51.886619+0000","last_fullsized":"2024-09-16T15:42:51.886619+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:35.801391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222105+0000","last_change":"2024-09-16T15:42:50.243951+0000","last_active":"2024-09-16T15:42:51.222105+0000","last_peered":"2024-09-16T15:42:51.222105+0000","last_clean":"2024-09-16T15:42:51.222105+0000","last_became_active":"2024-09-16T15:42:50.243705+0000","last_became_peered":"2024-09-16T15:42:50.243705+0000","last_unstale":"2024-09-16T15:42:51.222105+0000","last_undegraded":"2024-09-16T15:42:51.222105+0000","last_fullsized":"2024-09-16T15:42:51.222105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:08.110180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221417+0000","last_change":"2024-09-16T15:42:50.242601+0000","last_active":"2024-09-16T15:42:51.221417+0000","last_peered":"2024-09-16T15:42:51.221417+0000","last_clean":"2024-09-16T15:42:51.221417+0000","last_became_active":"2024-09-16T15:42:50.239915+0000","last_became_peered":"2024-09-16T15:42:50.239915+0000","last_unstale":"2024-09-16T15:42:51.221417+0000","last_undegraded":"2024-09-16T15:42:51.221417+0000","last_fullsized":"2024-09-16T15:42:51.221417+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:30.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221817+0000","last_change":"2024-09-16T15:42:50.237521+0000","last_active":"2024-09-16T15:42:51.221817+0000","last_peered":"2024-09-16T15:42:51.221817+0000","last_clean":"2024-09-16T15:42:51.221817+0000","last_became_active":"2024-09-16T15:42:50.236895+0000","last_became_peered":"2024-09-16T15:42:50.236895+0000","last_unstale":"2024-09-16T15:42:51.221817+0000","last_undegraded":"2024-09-16T15:42:51.221817+0000","last_fullsized":"2024-09-16T15:42:51.221817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:28.871447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632142+0000","last_change":"2024-09-16T15:42:50.244749+0000","last_active":"2024-09-16T15:42:51.632142+0000","last_peered":"2024-09-16T15:42:51.632142+0000","last_clean":"2024-09-16T15:42:51.632142+0000","last_became_active":"2024-09-16T15:42:50.244425+0000","last_became_peered":"2024-09-16T15:42:50.244425+0000","last_unstale":"2024-09-16T15:42:51.632142+0000","last_undegraded":"2024-09-16T15:42:51.632142+0000","last_fullsized":"2024-09-16T15:42:51.632142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:49.583061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632378+0000","last_change":"2024-09-16T15:42:49.230741+0000","last_active":"2024-09-16T15:42:51.632378+0000","last_peered":"2024-09-16T15:42:51.632378+0000","last_clean":"2024-09-16T15:42:51.632378+0000","last_became_active":"2024-09-16T15:42:49.230054+0000","last_became_peered":"2024-09-16T15:42:49.230054+0000","last_unstale":"2024-09-16T15:42:51.632378+0000","last_undegraded":"2024-09-16T15:42:51.632378+0000","last_fullsized":"2024-09-16T15:42:51.632378+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:06:05.634754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533214+0000","last_change":"2024-09-16T15:42:50.249173+0000","last_active":"2024-09-16T15:42:51.533214+0000","last_peered":"2024-09-16T15:42:51.533214+0000","last_clean":"2024-09-16T15:42:51.533214+0000","last_became_active":"2024-09-16T15:42:50.247280+0000","last_became_peered":"2024-09-16T15:42:50.247280+0000","last_unstale":"2024-09-16T15:42:51.533214+0000","last_undegraded":"2024-09-16T15:42:51.533214+0000","last_fullsized":"2024-09-16T15:42:51.533214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:03.036533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240547+0000","last_change":"2024-09-16T15:42:49.227671+0000","last_active":"2024-09-16T15:42:51.240547+0000","last_peered":"2024-09-16T15:42:51.240547+0000","last_clean":"2024-09-16T15:42:51.240547+0000","last_became_active":"2024-09-16T15:42:49.227333+0000","last_became_peered":"2024-09-16T15:42:49.227333+0000","last_unstale":"2024-09-16T15:42:51.240547+0000","last_undegraded":"2024-09-16T15:42:51.240547+0000","last_fullsized":"2024-09-16T15:42:51.240547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:06:05.634754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222367+0000","last_change":"2024-09-16T15:42:50.243680+0000","last_active":"2024-09-16T15:42:51.222367+0000","last_peered":"2024-09-16T15:42:51.222367+0000","last_clean":"2024-09-16T15:42:51.222367+0000","last_became_active":"2024-09-16T15:42:50.242813+0000","last_became_peered":"2024-09-16T15:42:50.242813+0000","last_unstale":"2024-09-16T15:42:51.222367+0000","last_undegraded":"2024-09-16T15:42:51.222367+0000","last_fullsized":"2024-09-16T15:42:51.222367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:23.095221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671357+0000","last_change":"2024-09-16T15:42:49.228304+0000","last_active":"2024-09-16T15:42:51.671357+0000","last_peered":"2024-09-16T15:42:51.671357+0000","last_clean":"2024-09-16T15:42:51.671357+0000","last_became_active":"2024-09-16T15:42:49.227962+0000","last_became_peered":"2024-09-16T15:42:49.227962+0000","last_unstale":"2024-09-16T15:42:51.671357+0000","last_undegraded":"2024-09-16T15:42:51.671357+0000","last_fullsized":"2024-09-16T15:42:51.671357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:45.388826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632006+0000","last_change":"2024-09-16T15:42:50.240957+0000","last_active":"2024-09-16T15:42:51.632006+0000","last_peered":"2024-09-16T15:42:51.632006+0000","last_clean":"2024-09-16T15:42:51.632006+0000","last_became_active":"2024-09-16T15:42:50.240356+0000","last_became_peered":"2024-09-16T15:42:50.240356+0000","last_unstale":"2024-09-16T15:42:51.632006+0000","last_undegraded":"2024-09-16T15:42:51.632006+0000","last_fullsized":"2024-09-16T15:42:51.632006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:20.329239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222688+0000","last_change":"2024-09-16T15:42:49.215851+0000","last_active":"2024-09-16T15:42:51.222688+0000","last_peered":"2024-09-16T15:42:51.222688+0000","last_clean":"2024-09-16T15:42:51.222688+0000","last_became_active":"2024-09-16T15:42:49.215587+0000","last_became_peered":"2024-09-16T15:42:49.215587+0000","last_unstale":"2024-09-16T15:42:51.222688+0000","last_undegraded":"2024-09-16T15:42:51.222688+0000","last_fullsized":"2024-09-16T15:42:51.222688+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:52.618435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221588+0000","last_change":"2024-09-16T15:42:50.238522+0000","last_active":"2024-09-16T15:42:51.221588+0000","last_peered":"2024-09-16T15:42:51.221588+0000","last_clean":"2024-09-16T15:42:51.221588+0000","last_became_active":"2024-09-16T15:42:50.238223+0000","last_became_peered":"2024-09-16T15:42:50.238223+0000","last_unstale":"2024-09-16T15:42:51.221588+0000","last_undegraded":"2024-09-16T15:42:51.221588+0000","last_fullsized":"2024-09-16T15:42:51.221588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:45.525365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886773+0000","last_change":"2024-09-16T15:42:49.220153+0000","last_active":"2024-09-16T15:42:51.886773+0000","last_peered":"2024-09-16T15:42:51.886773+0000","last_clean":"2024-09-16T15:42:51.886773+0000","last_became_active":"2024-09-16T15:42:49.219786+0000","last_became_peered":"2024-09-16T15:42:49.219786+0000","last_unstale":"2024-09-16T15:42:51.886773+0000","last_undegraded":"2024-09-16T15:42:51.886773+0000","last_fullsized":"2024-09-16T15:42:51.886773+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:51:54.851547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671591+0000","last_change":"2024-09-16T15:42:50.237887+0000","last_active":"2024-09-16T15:42:51.671591+0000","last_peered":"2024-09-16T15:42:51.671591+0000","last_clean":"2024-09-16T15:42:51.671591+0000","last_became_active":"2024-09-16T15:42:50.237604+0000","last_became_peered":"2024-09-16T15:42:50.237604+0000","last_unstale":"2024-09-16T15:42:51.671591+0000","last_undegraded":"2024-09-16T15:42:51.671591+0000","last_fullsized":"2024-09-16T15:42:51.671591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:36.020627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222428+0000","last_change":"2024-09-16T15:42:49.227675+0000","last_active":"2024-09-16T15:42:51.222428+0000","last_peered":"2024-09-16T15:42:51.222428+0000","last_clean":"2024-09-16T15:42:51.222428+0000","last_became_active":"2024-09-16T15:42:49.227417+0000","last_became_peered":"2024-09-16T15:42:49.227417+0000","last_unstale":"2024-09-16T15:42:51.222428+0000","last_undegraded":"2024-09-16T15:42:51.222428+0000","last_fullsized":"2024-09-16T15:42:51.222428+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:56.796817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532690+0000","last_change":"2024-09-16T15:42:50.240752+0000","last_active":"2024-09-16T15:42:51.532690+0000","last_peered":"2024-09-16T15:42:51.532690+0000","last_clean":"2024-09-16T15:42:51.532690+0000","last_became_active":"2024-09-16T15:42:50.240455+0000","last_became_peered":"2024-09-16T15:42:50.240455+0000","last_unstale":"2024-09-16T15:42:51.532690+0000","last_undegraded":"2024-09-16T15:42:51.532690+0000","last_fullsized":"2024-09-16T15:42:51.532690+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:23.877726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219262+0000","last_change":"2024-09-16T15:42:49.213568+0000","last_active":"2024-09-16T15:42:51.219262+0000","last_peered":"2024-09-16T15:42:51.219262+0000","last_clean":"2024-09-16T15:42:51.219262+0000","last_became_active":"2024-09-16T15:42:49.213239+0000","last_became_peered":"2024-09-16T15:42:49.213239+0000","last_unstale":"2024-09-16T15:42:51.219262+0000","last_undegraded":"2024-09-16T15:42:51.219262+0000","last_fullsized":"2024-09-16T15:42:51.219262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:38.219818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240496+0000","last_change":"2024-09-16T15:42:50.236445+0000","last_active":"2024-09-16T15:42:51.240496+0000","last_peered":"2024-09-16T15:42:51.240496+0000","last_clean":"2024-09-16T15:42:51.240496+0000","last_became_active":"2024-09-16T15:42:50.236140+0000","last_became_peered":"2024-09-16T15:42:50.236140+0000","last_unstale":"2024-09-16T15:42:51.240496+0000","last_undegraded":"2024-09-16T15:42:51.240496+0000","last_fullsized":"2024-09-16T15:42:51.240496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:01.692023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532596+0000","last_change":"2024-09-16T15:42:49.227614+0000","last_active":"2024-09-16T15:42:51.532596+0000","last_peered":"2024-09-16T15:42:51.532596+0000","last_clean":"2024-09-16T15:42:51.532596+0000","last_became_active":"2024-09-16T15:42:49.226862+0000","last_became_peered":"2024-09-16T15:42:49.226862+0000","last_unstale":"2024-09-16T15:42:51.532596+0000","last_undegraded":"2024-09-16T15:42:51.532596+0000","last_fullsized":"2024-09-16T15:42:51.532596+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:42.658997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221982+0000","last_change":"2024-09-16T15:42:50.242143+0000","last_active":"2024-09-16T15:42:51.221982+0000","last_peered":"2024-09-16T15:42:51.221982+0000","last_clean":"2024-09-16T15:42:51.221982+0000","last_became_active":"2024-09-16T15:42:50.241844+0000","last_became_peered":"2024-09-16T15:42:50.241844+0000","last_unstale":"2024-09-16T15:42:51.221982+0000","last_undegraded":"2024-09-16T15:42:51.221982+0000","last_fullsized":"2024-09-16T15:42:51.221982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:01.692023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671310+0000","last_change":"2024-09-16T15:42:49.218496+0000","last_active":"2024-09-16T15:42:51.671310+0000","last_peered":"2024-09-16T15:42:51.671310+0000","last_clean":"2024-09-16T15:42:51.671310+0000","last_became_active":"2024-09-16T15:42:49.218195+0000","last_became_peered":"2024-09-16T15:42:49.218195+0000","last_unstale":"2024-09-16T15:42:51.671310+0000","last_undegraded":"2024-09-16T15:42:51.671310+0000","last_fullsized":"2024-09-16T15:42:51.671310+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:33.304250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632271+0000","last_change":"2024-09-16T15:42:50.246020+0000","last_active":"2024-09-16T15:42:51.632271+0000","last_peered":"2024-09-16T15:42:51.632271+0000","last_clean":"2024-09-16T15:42:51.632271+0000","last_became_active":"2024-09-16T15:42:50.245660+0000","last_became_peered":"2024-09-16T15:42:50.245660+0000","last_unstale":"2024-09-16T15:42:51.632271+0000","last_undegraded":"2024-09-16T15:42:51.632271+0000","last_fullsized":"2024-09-16T15:42:51.632271+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:50.807347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222757+0000","last_change":"2024-09-16T15:42:49.225639+0000","last_active":"2024-09-16T15:42:51.222757+0000","last_peered":"2024-09-16T15:42:51.222757+0000","last_clean":"2024-09-16T15:42:51.222757+0000","last_became_active":"2024-09-16T15:42:49.225353+0000","last_became_peered":"2024-09-16T15:42:49.225353+0000","last_unstale":"2024-09-16T15:42:51.222757+0000","last_undegraded":"2024-09-16T15:42:51.222757+0000","last_fullsized":"2024-09-16T15:42:51.222757+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:31.795804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222591+0000","last_change":"2024-09-16T15:42:50.235425+0000","last_active":"2024-09-16T15:42:51.222591+0000","last_peered":"2024-09-16T15:42:51.222591+0000","last_clean":"2024-09-16T15:42:51.222591+0000","last_became_active":"2024-09-16T15:42:50.235052+0000","last_became_peered":"2024-09-16T15:42:50.235052+0000","last_unstale":"2024-09-16T15:42:51.222591+0000","last_undegraded":"2024-09-16T15:42:51.222591+0000","last_fullsized":"2024-09-16T15:42:51.222591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:55.386280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886834+0000","last_change":"2024-09-16T15:42:49.221761+0000","last_active":"2024-09-16T15:42:51.886834+0000","last_peered":"2024-09-16T15:42:51.886834+0000","last_clean":"2024-09-16T15:42:51.886834+0000","last_became_active":"2024-09-16T15:42:49.221491+0000","last_became_peered":"2024-09-16T15:42:49.221491+0000","last_unstale":"2024-09-16T15:42:51.886834+0000","last_undegraded":"2024-09-16T15:42:51.886834+0000","last_fullsized":"2024-09-16T15:42:51.886834+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:59.840954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532952+0000","last_change":"2024-09-16T15:42:50.249860+0000","last_active":"2024-09-16T15:42:51.532952+0000","last_peered":"2024-09-16T15:42:51.532952+0000","last_clean":"2024-09-16T15:42:51.532952+0000","last_became_active":"2024-09-16T15:42:50.244511+0000","last_became_peered":"2024-09-16T15:42:50.244511+0000","last_unstale":"2024-09-16T15:42:51.532952+0000","last_undegraded":"2024-09-16T15:42:51.532952+0000","last_fullsized":"2024-09-16T15:42:51.532952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:19.917538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219656+0000","last_change":"2024-09-16T15:42:49.219530+0000","last_active":"2024-09-16T15:42:51.219656+0000","last_peered":"2024-09-16T15:42:51.219656+0000","last_clean":"2024-09-16T15:42:51.219656+0000","last_became_active":"2024-09-16T15:42:49.219019+0000","last_became_peered":"2024-09-16T15:42:49.219019+0000","last_unstale":"2024-09-16T15:42:51.219656+0000","last_undegraded":"2024-09-16T15:42:51.219656+0000","last_fullsized":"2024-09-16T15:42:51.219656+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:46.888005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631758+0000","last_change":"2024-09-16T15:42:50.245485+0000","last_active":"2024-09-16T15:42:51.631758+0000","last_peered":"2024-09-16T15:42:51.631758+0000","last_clean":"2024-09-16T15:42:51.631758+0000","last_became_active":"2024-09-16T15:42:50.244917+0000","last_became_peered":"2024-09-16T15:42:50.244917+0000","last_unstale":"2024-09-16T15:42:51.631758+0000","last_undegraded":"2024-09-16T15:42:51.631758+0000","last_fullsized":"2024-09-16T15:42:51.631758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:42.918396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219758+0000","last_change":"2024-09-16T15:42:49.225850+0000","last_active":"2024-09-16T15:42:51.219758+0000","last_peered":"2024-09-16T15:42:51.219758+0000","last_clean":"2024-09-16T15:42:51.219758+0000","last_became_active":"2024-09-16T15:42:49.225497+0000","last_became_peered":"2024-09-16T15:42:49.225497+0000","last_unstale":"2024-09-16T15:42:51.219758+0000","last_undegraded":"2024-09-16T15:42:51.219758+0000","last_fullsized":"2024-09-16T15:42:51.219758+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:15.332359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222175+0000","last_change":"2024-09-16T15:42:50.236674+0000","last_active":"2024-09-16T15:42:51.222175+0000","last_peered":"2024-09-16T15:42:51.222175+0000","last_clean":"2024-09-16T15:42:51.222175+0000","last_became_active":"2024-09-16T15:42:50.236411+0000","last_became_peered":"2024-09-16T15:42:50.236411+0000","last_unstale":"2024-09-16T15:42:51.222175+0000","last_undegraded":"2024-09-16T15:42:51.222175+0000","last_fullsized":"2024-09-16T15:42:51.222175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:41.139551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222537+0000","last_change":"2024-09-16T15:42:49.227184+0000","last_active":"2024-09-16T15:42:51.222537+0000","last_peered":"2024-09-16T15:42:51.222537+0000","last_clean":"2024-09-16T15:42:51.222537+0000","last_became_active":"2024-09-16T15:42:49.226894+0000","last_became_peered":"2024-09-16T15:42:49.226894+0000","last_unstale":"2024-09-16T15:42:51.222537+0000","last_undegraded":"2024-09-16T15:42:51.222537+0000","last_fullsized":"2024-09-16T15:42:51.222537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:13.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240447+0000","last_change":"2024-09-16T15:42:50.242332+0000","last_active":"2024-09-16T15:42:51.240447+0000","last_peered":"2024-09-16T15:42:51.240447+0000","last_clean":"2024-09-16T15:42:51.240447+0000","last_became_active":"2024-09-16T15:42:50.241852+0000","last_became_peered":"2024-09-16T15:42:50.241852+0000","last_unstale":"2024-09-16T15:42:51.240447+0000","last_undegraded":"2024-09-16T15:42:51.240447+0000","last_fullsized":"2024-09-16T15:42:51.240447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:39.104202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221221+0000","last_change":"2024-09-16T15:42:49.229983+0000","last_active":"2024-09-16T15:42:51.221221+0000","last_peered":"2024-09-16T15:42:51.221221+0000","last_clean":"2024-09-16T15:42:51.221221+0000","last_became_active":"2024-09-16T15:42:49.229708+0000","last_became_peered":"2024-09-16T15:42:49.229708+0000","last_unstale":"2024-09-16T15:42:51.221221+0000","last_undegraded":"2024-09-16T15:42:51.221221+0000","last_fullsized":"2024-09-16T15:42:51.221221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:50.565782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532800+0000","last_change":"2024-09-16T15:42:50.237423+0000","last_active":"2024-09-16T15:42:51.532800+0000","last_peered":"2024-09-16T15:42:51.532800+0000","last_clean":"2024-09-16T15:42:51.532800+0000","last_became_active":"2024-09-16T15:42:50.236994+0000","last_became_peered":"2024-09-16T15:42:50.236994+0000","last_unstale":"2024-09-16T15:42:51.532800+0000","last_undegraded":"2024-09-16T15:42:51.532800+0000","last_fullsized":"2024-09-16T15:42:51.532800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:55.202973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.670983+0000","last_change":"2024-09-16T15:42:49.229382+0000","last_active":"2024-09-16T15:42:51.670983+0000","last_peered":"2024-09-16T15:42:51.670983+0000","last_clean":"2024-09-16T15:42:51.670983+0000","last_became_active":"2024-09-16T15:42:49.229101+0000","last_became_peered":"2024-09-16T15:42:49.229101+0000","last_unstale":"2024-09-16T15:42:51.670983+0000","last_undegraded":"2024-09-16T15:42:51.670983+0000","last_fullsized":"2024-09-16T15:42:51.670983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:08:51.344506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671394+0000","last_change":"2024-09-16T15:42:50.229524+0000","last_active":"2024-09-16T15:42:51.671394+0000","last_peered":"2024-09-16T15:42:51.671394+0000","last_clean":"2024-09-16T15:42:51.671394+0000","last_became_active":"2024-09-16T15:42:50.229234+0000","last_became_peered":"2024-09-16T15:42:50.229234+0000","last_unstale":"2024-09-16T15:42:51.671394+0000","last_undegraded":"2024-09-16T15:42:51.671394+0000","last_fullsized":"2024-09-16T15:42:51.671394+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:57.651014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221299+0000","last_change":"2024-09-16T15:42:49.226313+0000","last_active":"2024-09-16T15:42:51.221299+0000","last_peered":"2024-09-16T15:42:51.221299+0000","last_clean":"2024-09-16T15:42:51.221299+0000","last_became_active":"2024-09-16T15:42:49.226037+0000","last_became_peered":"2024-09-16T15:42:49.226037+0000","last_unstale":"2024-09-16T15:42:51.221299+0000","last_undegraded":"2024-09-16T15:42:51.221299+0000","last_fullsized":"2024-09-16T15:42:51.221299+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:47.386114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632429+0000","last_change":"2024-09-16T15:42:50.246671+0000","last_active":"2024-09-16T15:42:51.632429+0000","last_peered":"2024-09-16T15:42:51.632429+0000","last_clean":"2024-09-16T15:42:51.632429+0000","last_became_active":"2024-09-16T15:42:50.246427+0000","last_became_peered":"2024-09-16T15:42:50.246427+0000","last_unstale":"2024-09-16T15:42:51.632429+0000","last_undegraded":"2024-09-16T15:42:51.632429+0000","last_fullsized":"2024-09-16T15:42:51.632429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.825762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671542+0000","last_change":"2024-09-16T15:42:49.225855+0000","last_active":"2024-09-16T15:42:51.671542+0000","last_peered":"2024-09-16T15:42:51.671542+0000","last_clean":"2024-09-16T15:42:51.671542+0000","last_became_active":"2024-09-16T15:42:49.225235+0000","last_became_peered":"2024-09-16T15:42:49.225235+0000","last_unstale":"2024-09-16T15:42:51.671542+0000","last_undegraded":"2024-09-16T15:42:51.671542+0000","last_fullsized":"2024-09-16T15:42:51.671542+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:04:06.304202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219664+0000","last_change":"2024-09-16T15:42:50.236713+0000","last_active":"2024-09-16T15:42:51.219664+0000","last_peered":"2024-09-16T15:42:51.219664+0000","last_clean":"2024-09-16T15:42:51.219664+0000","last_became_active":"2024-09-16T15:42:50.236233+0000","last_became_peered":"2024-09-16T15:42:50.236233+0000","last_unstale":"2024-09-16T15:42:51.219664+0000","last_undegraded":"2024-09-16T15:42:51.219664+0000","last_fullsized":"2024-09-16T15:42:51.219664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:21.447882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222463+0000","last_change":"2024-09-16T15:42:49.225896+0000","last_active":"2024-09-16T15:42:51.222463+0000","last_peered":"2024-09-16T15:42:51.222463+0000","last_clean":"2024-09-16T15:42:51.222463+0000","last_became_active":"2024-09-16T15:42:49.225606+0000","last_became_peered":"2024-09-16T15:42:49.225606+0000","last_unstale":"2024-09-16T15:42:51.222463+0000","last_undegraded":"2024-09-16T15:42:51.222463+0000","last_fullsized":"2024-09-16T15:42:51.222463+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:57.795996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533018+0000","last_change":"2024-09-16T15:42:50.235937+0000","last_active":"2024-09-16T15:42:51.533018+0000","last_peered":"2024-09-16T15:42:51.533018+0000","last_clean":"2024-09-16T15:42:51.533018+0000","last_became_active":"2024-09-16T15:42:50.234785+0000","last_became_peered":"2024-09-16T15:42:50.234785+0000","last_unstale":"2024-09-16T15:42:51.533018+0000","last_undegraded":"2024-09-16T15:42:51.533018+0000","last_fullsized":"2024-09-16T15:42:51.533018+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:56.537229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631773+0000","last_change":"2024-09-16T15:42:49.232299+0000","last_active":"2024-09-16T15:42:51.631773+0000","last_peered":"2024-09-16T15:42:51.631773+0000","last_clean":"2024-09-16T15:42:51.631773+0000","last_became_active":"2024-09-16T15:42:49.231909+0000","last_became_peered":"2024-09-16T15:42:49.231909+0000","last_unstale":"2024-09-16T15:42:51.631773+0000","last_undegraded":"2024-09-16T15:42:51.631773+0000","last_fullsized":"2024-09-16T15:42:51.631773+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:55.045860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886667+0000","last_change":"2024-09-16T15:42:50.227398+0000","last_active":"2024-09-16T15:42:51.886667+0000","last_peered":"2024-09-16T15:42:51.886667+0000","last_clean":"2024-09-16T15:42:51.886667+0000","last_became_active":"2024-09-16T15:42:50.226978+0000","last_became_peered":"2024-09-16T15:42:50.226978+0000","last_unstale":"2024-09-16T15:42:51.886667+0000","last_undegraded":"2024-09-16T15:42:51.886667+0000","last_fullsized":"2024-09-16T15:42:51.886667+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:30.408132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222622+0000","last_change":"2024-09-16T15:42:49.227801+0000","last_active":"2024-09-16T15:42:51.222622+0000","last_peered":"2024-09-16T15:42:51.222622+0000","last_clean":"2024-09-16T15:42:51.222622+0000","last_became_active":"2024-09-16T15:42:49.227544+0000","last_became_peered":"2024-09-16T15:42:49.227544+0000","last_unstale":"2024-09-16T15:42:51.222622+0000","last_undegraded":"2024-09-16T15:42:51.222622+0000","last_fullsized":"2024-09-16T15:42:51.222622+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:21.793711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886376+0000","last_change":"2024-09-16T15:42:50.240845+0000","last_active":"2024-09-16T15:42:51.886376+0000","last_peered":"2024-09-16T15:42:51.886376+0000","last_clean":"2024-09-16T15:42:51.886376+0000","last_became_active":"2024-09-16T15:42:50.240428+0000","last_became_peered":"2024-09-16T15:42:50.240428+0000","last_unstale":"2024-09-16T15:42:51.886376+0000","last_undegraded":"2024-09-16T15:42:51.886376+0000","last_fullsized":"2024-09-16T15:42:51.886376+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:03.926546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221535+0000","last_change":"2024-09-16T15:42:49.227177+0000","last_active":"2024-09-16T15:42:51.221535+0000","last_peered":"2024-09-16T15:42:51.221535+0000","last_clean":"2024-09-16T15:42:51.221535+0000","last_became_active":"2024-09-16T15:42:49.226902+0000","last_became_peered":"2024-09-16T15:42:49.226902+0000","last_unstale":"2024-09-16T15:42:51.221535+0000","last_undegraded":"2024-09-16T15:42:51.221535+0000","last_fullsized":"2024-09-16T15:42:51.221535+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:23.002295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886455+0000","last_change":"2024-09-16T15:42:50.233607+0000","last_active":"2024-09-16T15:42:51.886455+0000","last_peered":"2024-09-16T15:42:51.886455+0000","last_clean":"2024-09-16T15:42:51.886455+0000","last_became_active":"2024-09-16T15:42:50.233301+0000","last_became_peered":"2024-09-16T15:42:50.233301+0000","last_unstale":"2024-09-16T15:42:51.886455+0000","last_undegraded":"2024-09-16T15:42:51.886455+0000","last_fullsized":"2024-09-16T15:42:51.886455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:51:57.347856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240326+0000","last_change":"2024-09-16T15:42:49.232604+0000","last_active":"2024-09-16T15:42:51.240326+0000","last_peered":"2024-09-16T15:42:51.240326+0000","last_clean":"2024-09-16T15:42:51.240326+0000","last_became_active":"2024-09-16T15:42:49.232083+0000","last_became_peered":"2024-09-16T15:42:49.232083+0000","last_unstale":"2024-09-16T15:42:51.240326+0000","last_undegraded":"2024-09-16T15:42:51.240326+0000","last_fullsized":"2024-09-16T15:42:51.240326+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:59.450333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240697+0000","last_change":"2024-09-16T15:42:50.249833+0000","last_active":"2024-09-16T15:42:51.240697+0000","last_peered":"2024-09-16T15:42:51.240697+0000","last_clean":"2024-09-16T15:42:51.240697+0000","last_became_active":"2024-09-16T15:42:50.248359+0000","last_became_peered":"2024-09-16T15:42:50.248359+0000","last_unstale":"2024-09-16T15:42:51.240697+0000","last_undegraded":"2024-09-16T15:42:51.240697+0000","last_fullsized":"2024-09-16T15:42:51.240697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:08.110180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221406+0000","last_change":"2024-09-16T15:42:49.226320+0000","last_active":"2024-09-16T15:42:51.221406+0000","last_peered":"2024-09-16T15:42:51.221406+0000","last_clean":"2024-09-16T15:42:51.221406+0000","last_became_active":"2024-09-16T15:42:49.226026+0000","last_became_peered":"2024-09-16T15:42:49.226026+0000","last_unstale":"2024-09-16T15:42:51.221406+0000","last_undegraded":"2024-09-16T15:42:51.221406+0000","last_fullsized":"2024-09-16T15:42:51.221406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:30.666342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.222319+0000","last_change":"2024-09-16T15:42:50.245949+0000","last_active":"2024-09-16T15:42:51.222319+0000","last_peered":"2024-09-16T15:42:51.222319+0000","last_clean":"2024-09-16T15:42:51.222319+0000","last_became_active":"2024-09-16T15:42:50.245641+0000","last_became_peered":"2024-09-16T15:42:50.245641+0000","last_unstale":"2024-09-16T15:42:51.222319+0000","last_undegraded":"2024-09-16T15:42:51.222319+0000","last_fullsized":"2024-09-16T15:42:51.222319+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:05.134119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"74'1","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.239778+0000","last_change":"2024-09-16T15:42:49.230435+0000","last_active":"2024-09-16T15:42:51.239778+0000","last_peered":"2024-09-16T15:42:51.239778+0000","last_clean":"2024-09-16T15:42:51.239778+0000","last_became_active":"2024-09-16T15:42:49.230027+0000","last_became_peered":"2024-09-16T15:42:49.230027+0000","last_unstale":"2024-09-16T15:42:51.239778+0000","last_undegraded":"2024-09-16T15:42:51.239778+0000","last_fullsized":"2024-09-16T15:42:51.239778+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:41.164728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631809+0000","last_change":"2024-09-16T15:42:50.245973+0000","last_active":"2024-09-16T15:42:51.631809+0000","last_peered":"2024-09-16T15:42:51.631809+0000","last_clean":"2024-09-16T15:42:51.631809+0000","last_became_active":"2024-09-16T15:42:50.244763+0000","last_became_peered":"2024-09-16T15:42:50.244763+0000","last_unstale":"2024-09-16T15:42:51.631809+0000","last_undegraded":"2024-09-16T15:42:51.631809+0000","last_fullsized":"2024-09-16T15:42:51.631809+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:52.395826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533079+0000","last_change":"2024-09-16T15:42:49.227358+0000","last_active":"2024-09-16T15:42:51.533079+0000","last_peered":"2024-09-16T15:42:51.533079+0000","last_clean":"2024-09-16T15:42:51.533079+0000","last_became_active":"2024-09-16T15:42:49.227081+0000","last_became_peered":"2024-09-16T15:42:49.227081+0000","last_unstale":"2024-09-16T15:42:51.533079+0000","last_undegraded":"2024-09-16T15:42:51.533079+0000","last_fullsized":"2024-09-16T15:42:51.533079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:34.159398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.632194+0000","last_change":"2024-09-16T15:42:50.243478+0000","last_active":"2024-09-16T15:42:51.632194+0000","last_peered":"2024-09-16T15:42:51.632194+0000","last_clean":"2024-09-16T15:42:51.632194+0000","last_became_active":"2024-09-16T15:42:50.243195+0000","last_became_peered":"2024-09-16T15:42:50.243195+0000","last_unstale":"2024-09-16T15:42:51.632194+0000","last_undegraded":"2024-09-16T15:42:51.632194+0000","last_fullsized":"2024-09-16T15:42:51.632194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:07.694426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.240604+0000","last_change":"2024-09-16T15:42:49.232999+0000","last_active":"2024-09-16T15:42:51.240604+0000","last_peered":"2024-09-16T15:42:51.240604+0000","last_clean":"2024-09-16T15:42:51.240604+0000","last_became_active":"2024-09-16T15:42:49.232433+0000","last_became_peered":"2024-09-16T15:42:49.232433+0000","last_unstale":"2024-09-16T15:42:51.240604+0000","last_undegraded":"2024-09-16T15:42:51.240604+0000","last_fullsized":"2024-09-16T15:42:51.240604+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:55.045860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.533009+0000","last_change":"2024-09-16T15:42:50.243676+0000","last_active":"2024-09-16T15:42:51.533009+0000","last_peered":"2024-09-16T15:42:51.533009+0000","last_clean":"2024-09-16T15:42:51.533009+0000","last_became_active":"2024-09-16T15:42:50.242348+0000","last_became_peered":"2024-09-16T15:42:50.242348+0000","last_unstale":"2024-09-16T15:42:51.533009+0000","last_undegraded":"2024-09-16T15:42:51.533009+0000","last_fullsized":"2024-09-16T15:42:51.533009+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:28.756230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.532794+0000","last_change":"2024-09-16T15:42:49.230727+0000","last_active":"2024-09-16T15:42:51.532794+0000","last_peered":"2024-09-16T15:42:51.532794+0000","last_clean":"2024-09-16T15:42:51.532794+0000","last_became_active":"2024-09-16T15:42:49.230456+0000","last_became_peered":"2024-09-16T15:42:49.230456+0000","last_unstale":"2024-09-16T15:42:51.532794+0000","last_undegraded":"2024-09-16T15:42:51.532794+0000","last_fullsized":"2024-09-16T15:42:51.532794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:46.590247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221704+0000","last_change":"2024-09-16T15:42:50.247358+0000","last_active":"2024-09-16T15:42:51.221704+0000","last_peered":"2024-09-16T15:42:51.221704+0000","last_clean":"2024-09-16T15:42:51.221704+0000","last_became_active":"2024-09-16T15:42:50.241832+0000","last_became_peered":"2024-09-16T15:42:50.241832+0000","last_unstale":"2024-09-16T15:42:51.221704+0000","last_undegraded":"2024-09-16T15:42:51.221704+0000","last_fullsized":"2024-09-16T15:42:51.221704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:55.912098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219202+0000","last_change":"2024-09-16T15:42:49.216345+0000","last_active":"2024-09-16T15:42:51.219202+0000","last_peered":"2024-09-16T15:42:51.219202+0000","last_clean":"2024-09-16T15:42:51.219202+0000","last_became_active":"2024-09-16T15:42:49.216038+0000","last_became_peered":"2024-09-16T15:42:49.216038+0000","last_unstale":"2024-09-16T15:42:51.219202+0000","last_undegraded":"2024-09-16T15:42:51.219202+0000","last_fullsized":"2024-09-16T15:42:51.219202+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:30.969988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.219391+0000","last_change":"2024-09-16T15:42:50.224269+0000","last_active":"2024-09-16T15:42:51.219391+0000","last_peered":"2024-09-16T15:42:51.219391+0000","last_clean":"2024-09-16T15:42:51.219391+0000","last_became_active":"2024-09-16T15:42:50.223827+0000","last_became_peered":"2024-09-16T15:42:50.223827+0000","last_unstale":"2024-09-16T15:42:51.219391+0000","last_undegraded":"2024-09-16T15:42:51.219391+0000","last_fullsized":"2024-09-16T15:42:51.219391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:44:20.372919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.671409+0000","last_change":"2024-09-16T15:42:49.227215+0000","last_active":"2024-09-16T15:42:51.671409+0000","last_peered":"2024-09-16T15:42:51.671409+0000","last_clean":"2024-09-16T15:42:51.671409+0000","last_became_active":"2024-09-16T15:42:49.226937+0000","last_became_peered":"2024-09-16T15:42:49.226937+0000","last_unstale":"2024-09-16T15:42:51.671409+0000","last_undegraded":"2024-09-16T15:42:51.671409+0000","last_fullsized":"2024-09-16T15:42:51.671409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:26.324867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221383+0000","last_change":"2024-09-16T15:42:50.240843+0000","last_active":"2024-09-16T15:42:51.221383+0000","last_peered":"2024-09-16T15:42:51.221383+0000","last_clean":"2024-09-16T15:42:51.221383+0000","last_became_active":"2024-09-16T15:42:50.237363+0000","last_became_peered":"2024-09-16T15:42:50.237363+0000","last_unstale":"2024-09-16T15:42:51.221383+0000","last_undegraded":"2024-09-16T15:42:51.221383+0000","last_fullsized":"2024-09-16T15:42:51.221383+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:59:03.289883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221918+0000","last_change":"2024-09-16T15:42:49.218784+0000","last_active":"2024-09-16T15:42:51.221918+0000","last_peered":"2024-09-16T15:42:51.221918+0000","last_clean":"2024-09-16T15:42:51.221918+0000","last_became_active":"2024-09-16T15:42:49.218359+0000","last_became_peered":"2024-09-16T15:42:49.218359+0000","last_unstale":"2024-09-16T15:42:51.221918+0000","last_undegraded":"2024-09-16T15:42:51.221918+0000","last_fullsized":"2024-09-16T15:42:51.221918+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:19.859750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.886579+0000","last_change":"2024-09-16T15:42:50.236702+0000","last_active":"2024-09-16T15:42:51.886579+0000","last_peered":"2024-09-16T15:42:51.886579+0000","last_clean":"2024-09-16T15:42:51.886579+0000","last_became_active":"2024-09-16T15:42:50.236407+0000","last_became_peered":"2024-09-16T15:42:50.236407+0000","last_unstale":"2024-09-16T15:42:51.886579+0000","last_undegraded":"2024-09-16T15:42:51.886579+0000","last_fullsized":"2024-09-16T15:42:51.886579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:49.199972+0000","last_clean_scrub_stamp":"2024-09-16T15:42:49.199972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:55.799552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.221894+0000","last_change":"2024-09-16T15:42:49.221746+0000","last_active":"2024-09-16T15:42:51.221894+0000","last_peered":"2024-09-16T15:42:51.221894+0000","last_clean":"2024-09-16T15:42:51.221894+0000","last_became_active":"2024-09-16T15:42:49.221418+0000","last_became_peered":"2024-09-16T15:42:49.221418+0000","last_unstale":"2024-09-16T15:42:51.221894+0000","last_undegraded":"2024-09-16T15:42:51.221894+0000","last_fullsized":"2024-09-16T15:42:51.221894+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:48.194058+0000","last_clean_scrub_stamp":"2024-09-16T15:42:48.194058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:09.934857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":207,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-16T15:42:51.631522+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:42:51.631522+0000","last_peered":"2024-09-16T15:42:51.631522+0000","last_clean":"2024-09-16T15:42:51.631522+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:42:51.631522+0000","last_undegraded":"2024-09-16T15:42:51.631522+0000","last_fullsized":"2024-09-16T15:42:51.631522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509498,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28496,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343344,"statfs":{"total":96636764160,"available":96607584256,"internally_reserved":0,"allocated":1495040,"data_stored":965744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509498,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":901120,"data_stored":374852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509496,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":897024,"data_stored":374850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":897024,"data_stored":374830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":901120,"data_stored":374872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509496,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":905216,"data_stored":376372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509498,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1486848,"data_stored":965128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542202,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36240,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":913408,"data_stored":376988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:54.902 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:42:54.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:42:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.107+0000 7f9152800640 1 -- 172.21.15.60:0/396162061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134003a50 msgr2=0x7f9134003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.107+0000 7f9152800640 1 --2- 172.21.15.60:0/396162061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9134003a50 0x7f9134003e70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9140006f80 tx=0x7f9134005830 comp rx=0 tx=0).stop 2024-09-16T15:42:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.107+0000 7f9151e00640 1 -- 172.21.15.60:0/396162061 reap_dead start 2024-09-16T15:42:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.107+0000 7f9152800640 1 -- 172.21.15.60:0/396162061 shutdown_connections 2024-09-16T15:42:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.107+0000 7f9152800640 1 -- 172.21.15.60:0/396162061 >> 172.21.15.60:0/396162061 conn(0x7f914c0cc490 msgr2=0x7f914c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.107+0000 7f9152800640 1 -- 172.21.15.60:0/396162061 shutdown_connections 2024-09-16T15:42:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.107+0000 7f9152800640 1 -- 172.21.15.60:0/396162061 wait complete. 2024-09-16T15:42:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.109+0000 7f9152800640 1 Processor -- start 2024-09-16T15:42:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.109+0000 7f9152800640 1 -- start start 2024-09-16T15:42:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9152800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134003a50 0x7f914c0ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9152800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c0cd550 0x7f914c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9152800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914c0ce2b0 0x7f914c0ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9152800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0ca4f0 con 0x7f914c0ce2b0 2024-09-16T15:42:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9152800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0ca370 con 0x7f914c0cd550 2024-09-16T15:42:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9152800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f914c06eac0 con 0x7f9134003a50 2024-09-16T15:42:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9151e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914c0ce2b0 0x7f914c0ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9151400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134003a50 0x7f914c0ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9151e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914c0ce2b0 0x7f914c0ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60162/0 (socket says 172.21.15.60:60162) 2024-09-16T15:42:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.110+0000 7f9151e00640 1 -- 172.21.15.60:0/165593768 learned_addr learned my addr 172.21.15.60:0/165593768 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9150a00640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c0cd550 0x7f914c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9151e00640 1 -- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134003a50 msgr2=0x7f914c0ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9151e00640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134003a50 0x7f914c0ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9151e00640 1 -- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c0cd550 msgr2=0x7f914c14b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9151e00640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c0cd550 0x7f914c14b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9151e00640 1 -- 172.21.15.60:0/165593768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c0ae860 con 0x7f914c0ce2b0 2024-09-16T15:42:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9151e00640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914c0ce2b0 0x7f914c0ce730 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9144011b20 tx=0x7f9144011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9149600640 1 -- 172.21.15.60:0/165593768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144004b20 con 0x7f914c0ce2b0 2024-09-16T15:42:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9149600640 1 -- 172.21.15.60:0/165593768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9144004cc0 con 0x7f914c0ce2b0 2024-09-16T15:42:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c0aeb40 con 0x7f914c0ce2b0 2024-09-16T15:42:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.111+0000 7f9149600640 1 -- 172.21.15.60:0/165593768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144003bd0 con 0x7f914c0ce2b0 2024-09-16T15:42:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.112+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c14b9c0 con 0x7f914c0ce2b0 2024-09-16T15:42:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.113+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914c0cc930 con 0x7f914c0ce2b0 2024-09-16T15:42:55.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.114+0000 7f9149600640 1 -- 172.21.15.60:0/165593768 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f9144004030 con 0x7f914c0ce2b0 2024-09-16T15:42:55.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.115+0000 7f9149600640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f912c078be0 0x7f912c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:55.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.115+0000 7f9149600640 1 -- 172.21.15.60:0/165593768 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f914409bb80 con 0x7f914c0ce2b0 2024-09-16T15:42:55.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.116+0000 7f9151400640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f912c078be0 0x7f912c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:55.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.116+0000 7f9151400640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f912c078be0 0x7f912c07b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9140006f80 tx=0x7f914c0c4b40 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.120+0000 7f9149600640 1 -- 172.21.15.60:0/165593768 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f914401f060 con 0x7f914c0ce2b0 2024-09-16T15:42:55.183 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-09-16T15:42:55.181+0000 7f5565000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:42:55.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.387+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f914c073680 con 0x7f914c0ce2b0 2024-09-16T15:42:55.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.390+0000 7f9149600640 1 -- 172.21.15.60:0/165593768 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f9144063b90 con 0x7f914c0ce2b0 2024-09-16T15:42:55.392 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:42:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:42:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:42:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:42:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:42:55.393 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:42:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f912c078be0 msgr2=0x7f912c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f912c078be0 0x7f912c07b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9140006f80 tx=0x7f914c0c4b40 comp rx=0 tx=0).stop 2024-09-16T15:42:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914c0ce2b0 msgr2=0x7f914c0ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914c0ce2b0 0x7f914c0ce730 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9144011b20 tx=0x7f9144011f20 comp rx=0 tx=0).stop 2024-09-16T15:42:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 shutdown_connections 2024-09-16T15:42:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f912c078be0 0x7f912c07b0a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914c0ce2b0 0x7f914c0ce730 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f914c0cd550 0x7f914c14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 --2- 172.21.15.60:0/165593768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9134003a50 0x7f914c0ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 >> 172.21.15.60:0/165593768 conn(0x7f914c0cc490 msgr2=0x7f914c075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 shutdown_connections 2024-09-16T15:42:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.393+0000 7f9152800640 1 -- 172.21.15.60:0/165593768 wait complete. 2024-09-16T15:42:55.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.569+0000 7f0160a00640 1 -- 172.21.15.60:0/105208403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 msgr2=0x7f015c0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.569+0000 7f0160a00640 1 --2- 172.21.15.60:0/105208403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 0x7f015c0d2370 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0154005fd0 tx=0x7f015402a8b0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.569+0000 7f0160a00640 1 -- 172.21.15.60:0/105208403 shutdown_connections 2024-09-16T15:42:55.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.569+0000 7f0160a00640 1 --2- 172.21.15.60:0/105208403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 0x7f015c0d2370 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.569+0000 7f0160a00640 1 -- 172.21.15.60:0/105208403 >> 172.21.15.60:0/105208403 conn(0x7f015c0d3df0 msgr2=0x7f015c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.569+0000 7f0160a00640 1 -- 172.21.15.60:0/105208403 shutdown_connections 2024-09-16T15:42:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.569+0000 7f0160a00640 1 -- 172.21.15.60:0/105208403 wait complete. 2024-09-16T15:42:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.570+0000 7f0160a00640 1 Processor -- start 2024-09-16T15:42:55.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.570+0000 7f0160a00640 1 -- start start 2024-09-16T15:42:55.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.571+0000 7f0160a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f015c0d4980 0x7f015c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:55.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.571+0000 7f0160a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f015c1612d0 0x7f015c14ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:55.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.571+0000 7f0160a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 0x7f015c149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:55.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.571+0000 7f0160a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f015c1690d0 con 0x7f015c1616b0 2024-09-16T15:42:55.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.571+0000 7f0160a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f015c168f50 con 0x7f015c0d4980 2024-09-16T15:42:55.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.571+0000 7f0160a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f015c169250 con 0x7f015c1612d0 2024-09-16T15:42:55.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 0x7f015c149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 0x7f015c149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60172/0 (socket says 172.21.15.60:60172) 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 -- 172.21.15.60:0/909611178 learned_addr learned my addr 172.21.15.60:0/909611178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015a800640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f015c0d4980 0x7f015c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f0159e00640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f015c1612d0 0x7f015c14ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 -- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f015c1612d0 msgr2=0x7f015c14ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f015c1612d0 0x7f015c14ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 -- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f015c0d4980 msgr2=0x7f015c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f015c0d4980 0x7f015c0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 -- 172.21.15.60:0/909611178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c0cff10 con 0x7f015c1616b0 2024-09-16T15:42:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.572+0000 7f015b200640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 0x7f015c149f20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0154009cd0 tx=0x7f0154003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.573+0000 7f0152a00640 1 -- 172.21.15.60:0/909611178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154031b90 con 0x7f015c1616b0 2024-09-16T15:42:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.573+0000 7f0152a00640 1 -- 172.21.15.60:0/909611178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0154031e20 con 0x7f015c1616b0 2024-09-16T15:42:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.573+0000 7f0152a00640 1 -- 172.21.15.60:0/909611178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01540332f0 con 0x7f015c1616b0 2024-09-16T15:42:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.573+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c0d01f0 con 0x7f015c1616b0 2024-09-16T15:42:55.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.573+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c0cfc00 con 0x7f015c1616b0 2024-09-16T15:42:55.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.574+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c06eae0 con 0x7f015c1616b0 2024-09-16T15:42:55.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.575+0000 7f0152a00640 1 -- 172.21.15.60:0/909611178 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f0154031660 con 0x7f015c1616b0 2024-09-16T15:42:55.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.576+0000 7f0152a00640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f013c078be0 0x7f013c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:55.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.576+0000 7f0152a00640 1 -- 172.21.15.60:0/909611178 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f01540bc350 con 0x7f015c1616b0 2024-09-16T15:42:55.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.577+0000 7f015a800640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f013c078be0 0x7f013c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:55.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.577+0000 7f015a800640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f013c078be0 0x7f013c07b0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f014c0017c0 tx=0x7f014c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:55.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.581+0000 7f0152a00640 1 -- 172.21.15.60:0/909611178 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f015c06eae0 con 0x7f015c1616b0 2024-09-16T15:42:55.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.815+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f015c0b3da0 con 0x7f015c1616b0 2024-09-16T15:42:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.816+0000 7f0152a00640 1 -- 172.21.15.60:0/909611178 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f015403f030 con 0x7f015c1616b0 2024-09-16T15:42:55.818 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:42:55.818 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":47,"btime":"2024-09-16T15:42:53:237452+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:42:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 47 2024-09-16T15:42:55.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f013c078be0 msgr2=0x7f013c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f013c078be0 0x7f013c07b0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f014c0017c0 tx=0x7f014c009040 comp rx=0 tx=0).stop 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 msgr2=0x7f015c149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 0x7f015c149f20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0154009cd0 tx=0x7f0154003ba0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 shutdown_connections 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f013c078be0 0x7f013c07b0a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f015c1616b0 0x7f015c149f20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f015c1612d0 0x7f015c14ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 --2- 172.21.15.60:0/909611178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f015c0d4980 0x7f015c0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 >> 172.21.15.60:0/909611178 conn(0x7f015c0d3df0 msgr2=0x7f015c0b5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 shutdown_connections 2024-09-16T15:42:55.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:55.818+0000 7f0160a00640 1 -- 172.21.15.60:0/909611178 wait complete. 2024-09-16T15:42:56.238 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:42:49.653917+0000', 'modified': '2024-09-16T15:42:52.679067+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6153}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6153': {'gid': 6153, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/1266112466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 1266112466}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1266112466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6153, 'qdb_cluster': [6153]} 2024-09-16T15:42:56.238 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:42:56.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-16T15:42:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.402+0000 7f5e8bc00640 1 -- 172.21.15.60:0/4011970616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e84161550 msgr2=0x7f5e8406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.402+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/4011970616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e84161550 0x7f5e8406ef50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5e74005fd0 tx=0x7f5e7402a540 comp rx=0 tx=0).stop 2024-09-16T15:42:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.402+0000 7f5e8bc00640 1 -- 172.21.15.60:0/4011970616 shutdown_connections 2024-09-16T15:42:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.402+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/4011970616 >> v2:172.21.15.100:3300/0 conn(0x7f5e84072df0 0x7f5e840732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.402+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/4011970616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e84161550 0x7f5e8406ef50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.402+0000 7f5e8bc00640 1 -- 172.21.15.60:0/4011970616 >> 172.21.15.60:0/4011970616 conn(0x7f5e840b1950 msgr2=0x7f5e840b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.402+0000 7f5e8bc00640 1 -- 172.21.15.60:0/4011970616 shutdown_connections 2024-09-16T15:42:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.402+0000 7f5e8bc00640 1 -- 172.21.15.60:0/4011970616 wait complete. 2024-09-16T15:42:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.403+0000 7f5e8bc00640 1 Processor -- start 2024-09-16T15:42:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.403+0000 7f5e8bc00640 1 -- start start 2024-09-16T15:42:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.404+0000 7f5e8bc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e84072df0 0x7f5e8414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.404+0000 7f5e8bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e840af3d0 0x7f5e841569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.404+0000 7f5e8bc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e84161550 0x7f5e84156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.404+0000 7f5e8bc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840737e0 con 0x7f5e84161550 2024-09-16T15:42:56.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.404+0000 7f5e8bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8406fdb0 con 0x7f5e840af3d0 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.404+0000 7f5e8bc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5e84073960 con 0x7f5e84072df0 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e84161550 0x7f5e84156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e84161550 0x7f5e84156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60182/0 (socket says 172.21.15.60:60182) 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 -- 172.21.15.60:0/1825362772 learned_addr learned my addr 172.21.15.60:0/1825362772 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89200640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e84072df0 0x7f5e8414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e7be00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e840af3d0 0x7f5e841569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 -- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e84072df0 msgr2=0x7f5e8414c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e84072df0 0x7f5e8414c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 -- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e840af3d0 msgr2=0x7f5e841569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e840af3d0 0x7f5e841569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 -- 172.21.15.60:0/1825362772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e841f4c30 con 0x7f5e84161550 2024-09-16T15:42:56.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.405+0000 7f5e89c00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e84161550 0x7f5e84156f20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00eb10 tx=0x7f5e7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.406+0000 7f5e81600640 1 -- 172.21.15.60:0/1825362772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c004b20 con 0x7f5e84161550 2024-09-16T15:42:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.406+0000 7f5e81600640 1 -- 172.21.15.60:0/1825362772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e7c004ce0 con 0x7f5e84161550 2024-09-16T15:42:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.406+0000 7f5e8bc00640 1 -- 172.21.15.60:0/1825362772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e841f4ef0 con 0x7f5e84161550 2024-09-16T15:42:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.406+0000 7f5e81600640 1 -- 172.21.15.60:0/1825362772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c005280 con 0x7f5e84161550 2024-09-16T15:42:56.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.406+0000 7f5e8bc00640 1 -- 172.21.15.60:0/1825362772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e841f54b0 con 0x7f5e84161550 2024-09-16T15:42:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.408+0000 7f5e81600640 1 -- 172.21.15.60:0/1825362772 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f5e7c005420 con 0x7f5e84161550 2024-09-16T15:42:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.409+0000 7f5e81600640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5e60078bb0 0x7f5e6007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.409+0000 7f5e81600640 1 -- 172.21.15.60:0/1825362772 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f5e7c09bd40 con 0x7f5e84161550 2024-09-16T15:42:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.409+0000 7f5e89200640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5e60078bb0 0x7f5e6007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.410+0000 7f5e8ec4f740 1 Processor -- start 2024-09-16T15:42:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.410+0000 7f5e8ec4f740 1 -- start start 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89200640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5e60078bb0 0x7f5e6007b070 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5e74033000 tx=0x7f5e74004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdcac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85a20 0x564afbdc9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564afbd08430 con 0x564afbdca8b0 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564afbcf85d0 con 0x564afbd85fa0 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x564afbce7a00 con 0x564afbd85a20 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdcac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdcac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60188/0 (socket says 172.21.15.60:60188) 2024-09-16T15:42:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89200640 1 -- 172.21.15.60:0/5825608 learned_addr learned my addr 172.21.15.60:0/5825608 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89c00640 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89200640 1 -- 172.21.15.60:0/5825608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85a20 msgr2=0x564afbdc9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89200640 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85a20 0x564afbdc9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89200640 1 -- 172.21.15.60:0/5825608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 msgr2=0x564afbdc98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.411+0000 7f5e89200640 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.412+0000 7f5e89200640 1 -- 172.21.15.60:0/5825608 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564afbcb0520 con 0x564afbdca8b0 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.412+0000 7f5e89c00640 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.412+0000 7f5e89200640 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdcac90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5e74034040 tx=0x7f5e7403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a4a03029923a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.412+0000 7f5e79600640 1 -- 172.21.15.60:0/5825608 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74035040 con 0x564afbdca8b0 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.412+0000 7f5e79600640 1 -- 172.21.15.60:0/5825608 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e7404c070 con 0x564afbdca8b0 2024-09-16T15:42:56.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.412+0000 7f5e79600640 1 -- 172.21.15.60:0/5825608 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7403b6c0 con 0x564afbdca8b0 2024-09-16T15:42:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/5825608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 msgr2=0x564afbdcac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdcac90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5e74034040 tx=0x7f5e7403b000 comp rx=0 tx=0).stop 2024-09-16T15:42:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/5825608 shutdown_connections 2024-09-16T15:42:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85a20 0x564afbdc9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/5825608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdcac90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/5825608 >> 172.21.15.60:0/5825608 conn(0x564afbd80ff0 msgr2=0x564afbd84a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/5825608 shutdown_connections 2024-09-16T15:42:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.413+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/5825608 wait complete. 2024-09-16T15:42:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.414+0000 7f5e8ec4f740 1 -- start start 2024-09-16T15:42:56.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.414+0000 7f5e8ec4f740 1 Processor -- start 2024-09-16T15:42:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdc3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbddccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564afbc8e330 con 0x564afbdca8b0 2024-09-16T15:42:56.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564afbceb810 con 0x564afbd85fa0 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x564afba58620 con 0x564afbdc4100 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e8ec4f740 10 client.0 _ll_register_callbacks cb 0x564afbc93420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdc3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdc3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60194/0 (socket says 172.21.15.60:60194) 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e7be00640 1 -- 172.21.15.60:0/1891924573 learned_addr learned my addr 172.21.15.60:0/1891924573 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e89200640 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.415+0000 7f5e89c00640 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbddccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e7be00640 1 -- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 msgr2=0x564afbddccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e7be00640 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbddccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e7be00640 1 -- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 msgr2=0x564afbdc9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e7be00640 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e7be00640 1 -- 172.21.15.60:0/1891924573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564afba49850 con 0x564afbdca8b0 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e89c00640 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbddccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:56.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e7be00640 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdc3bc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c001d40 tx=0x7f5e6c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850ce5738f26c1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e72000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e72000640 1 -- 172.21.15.60:0/1891924573 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c019060 con 0x564afbdca8b0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e72000640 1 -- 172.21.15.60:0/1891924573 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f5e6c014ad0 con 0x564afbdca8b0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/1891924573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7404c070 con 0x564afbdca8b0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e72000640 1 -- 172.21.15.60:0/1891924573 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5e6c014cd0 con 0x564afbdca8b0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e8ec4f740 10 client.6279 fetch_fsmap learned FSMap version 47 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.416+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/1891924573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x564afbac06b0 con 0x564afbdca8b0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e72000640 1 -- 172.21.15.60:0/1891924573 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e7404c070 con 0x564afbdca8b0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e72000640 1 -- 172.21.15.60:0/1891924573 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7f5e6c0050d0 con 0x564afbdca8b0 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e8ec4f740 10 client.6279 fetch_fsmap finished waiting for FSMap version 47 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e8ec4f740 10 client.6279 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e8ec4f740 20 client.6279 populate_metadata read hostname 'smithi060' 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] conn(0x564afbdc1b80 0x564afbdadfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e8ec4f740 4 client.6279 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:42:56.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/1891924573 --> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564afbcfa520 con 0x564afbdc1b80 2024-09-16T15:42:56.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.417+0000 7f5e89200640 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] conn(0x564afbdc1b80 0x564afbdadfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.418+0000 7f5e89200640 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] conn(0x564afbdc1b80 0x564afbdadfb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=453bdf40a68dd7c2 server_cookie=257fc243b7575bc6 in_seq=0 out_seq=0 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.418+0000 7f5e72000640 10 client.6279 ms_handle_connect on v2:172.21.15.60:6834/1266112466 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.418+0000 7f5e72000640 1 -- 172.21.15.60:0/1891924573 <== mds.0 v2:172.21.15.60:6834/1266112466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564afbcfa520 con 0x564afbdc1b80 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.418+0000 7f5e72000640 10 client.6279 handle_command_reply: tid=0 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.418+0000 7f5e8ec4f740 1 client.6279 shutdown 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/1891924573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 msgr2=0x564afbdc3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdc3bc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c001d40 tx=0x7f5e6c00e530 comp rx=0 tx=0).stop 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/1891924573 shutdown_connections 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] conn(0x564afbdc1b80 0x564afbdadfb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbddccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdca8b0 0x564afbdc3bc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/1891924573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbd85fa0 0x564afbdc9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e72000640 0 client.6279 ms_handle_reset on v2:172.21.15.60:6834/1266112466 2024-09-16T15:42:56.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/1891924573 >> 172.21.15.60:0/1891924573 conn(0x564afbd85a20 msgr2=0x564afbd7fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:56.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/1891924573 shutdown_connections 2024-09-16T15:42:56.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/1891924573 wait complete. 2024-09-16T15:42:56.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.419+0000 7f5e8ec4f740 20 client.6279 trim_cache size 0 max 16384 2024-09-16T15:42:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.438+0000 7f5e8ec4f740 1 Processor -- start 2024-09-16T15:42:56.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.438+0000 7f5e8ec4f740 1 -- start start 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.439+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85fa0 0x564afbd84250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.439+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdc4100 0x564afbd84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.439+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 0x564afbded170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.439+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840cc230 con 0x564afbdc4100 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.439+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e841606f0 con 0x564afbdca8b0 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.439+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5e840cbbe0 con 0x564afbd85fa0 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.439+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdc4100 0x564afbd84790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdc4100 0x564afbd84790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60206/0 (socket says 172.21.15.60:60206) 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e7be00640 1 -- 172.21.15.60:0/560768996 learned_addr learned my addr 172.21.15.60:0/560768996 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e89c00640 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 0x564afbded170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e89200640 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85fa0 0x564afbd84250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e7be00640 1 -- 172.21.15.60:0/560768996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85fa0 msgr2=0x564afbd84250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e7be00640 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85fa0 0x564afbd84250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e7be00640 1 -- 172.21.15.60:0/560768996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 msgr2=0x564afbded170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e7be00640 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 0x564afbded170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e7be00640 1 -- 172.21.15.60:0/560768996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564afbdc3f30 con 0x564afbdc4100 2024-09-16T15:42:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e7be00640 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdc4100 0x564afbd84790 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c001620 tx=0x7f5e6c000e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f668d2bade3cb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e78c00640 1 -- 172.21.15.60:0/560768996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c02d070 con 0x564afbdc4100 2024-09-16T15:42:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e78c00640 1 -- 172.21.15.60:0/560768996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e6c00ea30 con 0x564afbdc4100 2024-09-16T15:42:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.440+0000 7f5e78c00640 1 -- 172.21.15.60:0/560768996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c007170 con 0x564afbdc4100 2024-09-16T15:42:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/560768996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdc4100 msgr2=0x564afbd84790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdc4100 0x564afbd84790 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c001620 tx=0x7f5e6c000e40 comp rx=0 tx=0).stop 2024-09-16T15:42:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/560768996 shutdown_connections 2024-09-16T15:42:56.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 0x564afbded170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbdc4100 0x564afbd84790 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/560768996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbd85fa0 0x564afbd84250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/560768996 >> 172.21.15.60:0/560768996 conn(0x564afbd85a20 msgr2=0x564afbdadd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/560768996 shutdown_connections 2024-09-16T15:42:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.441+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/560768996 wait complete. 2024-09-16T15:42:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.442+0000 7f5e8ec4f740 1 -- start start 2024-09-16T15:42:56.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.442+0000 7f5e8ec4f740 1 Processor -- start 2024-09-16T15:42:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.443+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbd85fa0 0x564afbe2ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.443+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbdd7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.443+0000 7f5e8ec4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 0x564afbd84250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.443+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x564afbdc1a00 con 0x564afbd85fa0 2024-09-16T15:42:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.443+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564afbdaed20 con 0x564afbdca8b0 2024-09-16T15:42:56.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.443+0000 7f5e8ec4f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x564afbdc1fc0 con 0x564afbdc4100 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.443+0000 7f5e8ec4f740 10 client.0 _ll_register_callbacks cb 0x7f5e6c007d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbd85fa0 0x564afbe2ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 0x564afbd84250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbd85fa0 0x564afbe2ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60214/0 (socket says 172.21.15.60:60214) 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e7be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbdd7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 -- 172.21.15.60:0/3361854446 learned_addr learned my addr 172.21.15.60:0/3361854446 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 -- 172.21.15.60:0/3361854446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 msgr2=0x564afbdd7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbdd7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 -- 172.21.15.60:0/3361854446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 msgr2=0x564afbd84250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 0x564afbd84250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 -- 172.21.15.60:0/3361854446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564afbe2f680 con 0x564afbd85fa0 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e89200640 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbd85fa0 0x564afbe2ef50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5e740099f0 tx=0x7f5e740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bd76fe7aa9f247 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e72000640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:42:56.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e72000640 1 -- 172.21.15.60:0/3361854446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7405e060 con 0x564afbd85fa0 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e72000640 1 -- 172.21.15.60:0/3361854446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f5e7403ba50 con 0x564afbd85fa0 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.444+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/3361854446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e6c00ea30 con 0x564afbd85fa0 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e72000640 1 -- 172.21.15.60:0/3361854446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5e74064050 con 0x564afbd85fa0 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e8ec4f740 10 client.6285 fetch_fsmap learned FSMap version 47 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/3361854446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x564afbd82470 con 0x564afbd85fa0 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e72000640 1 -- 172.21.15.60:0/3361854446 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e6c00ea30 con 0x564afbd85fa0 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e72000640 1 -- 172.21.15.60:0/3361854446 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7f5e74067020 con 0x564afbd85fa0 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e8ec4f740 10 client.6285 fetch_fsmap finished waiting for FSMap version 47 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e8ec4f740 10 client.6285 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e8ec4f740 20 client.6285 populate_metadata read hostname 'smithi060' 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] conn(0x564afbddc140 0x564afbe4a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e8ec4f740 4 client.6285 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/3361854446 --> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] -- command(tid 0: {"prefix": "status"}) -- 0x564afbcf5fd0 con 0x564afbddc140 2024-09-16T15:42:56.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.445+0000 7f5e7be00640 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] conn(0x564afbddc140 0x564afbe4a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e72000640 10 client.6285 ms_handle_connect on v2:172.21.15.60:6834/1266112466 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e7be00640 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] conn(0x564afbddc140 0x564afbe4a460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bcfde0548b51766 server_cookie=cb1ed3087ce9bacc in_seq=0 out_seq=0 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e72000640 1 -- 172.21.15.60:0/3361854446 <== mds.0 v2:172.21.15.60:6834/1266112466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564afbcf5fd0 con 0x564afbddc140 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e72000640 10 client.6285 handle_command_reply: tid=0 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 client.6285 shutdown 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/3361854446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbd85fa0 msgr2=0x564afbe2ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbd85fa0 0x564afbe2ef50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5e740099f0 tx=0x7f5e740046c0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/3361854446 shutdown_connections 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.60:6834/1266112466,v1:172.21.15.60:6835/1266112466] conn(0x564afbddc140 0x564afbe4a460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564afbdca8b0 0x564afbd84250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x564afbdc4100 0x564afbdd7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e72000640 0 client.6285 ms_handle_reset on v2:172.21.15.60:6834/1266112466 2024-09-16T15:42:56.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 --2- 172.21.15.60:0/3361854446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564afbd85fa0 0x564afbe2ef50 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/3361854446 >> 172.21.15.60:0/3361854446 conn(0x564afbd85a20 msgr2=0x564afbdae420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/3361854446 shutdown_connections 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.446+0000 7f5e8ec4f740 1 -- 172.21.15.60:0/3361854446 wait complete. 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8ec4f740 20 client.6285 trim_cache size 0 max 16384 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6153, 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:42:56.452 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:42:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:42:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2209153869999998, 2024-09-16T15:42:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 47, 2024-09-16T15:42:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 74, 2024-09-16T15:42:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 74, 2024-09-16T15:42:56.453 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.561551722000001 2024-09-16T15:42:56.453 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 -- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5e60078bb0 msgr2=0x7f5e6007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5e60078bb0 0x7f5e6007b070 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5e74033000 tx=0x7f5e74004730 comp rx=0 tx=0).stop 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 -- 172.21.15.60:0/1825362772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e84161550 msgr2=0x7f5e84156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e84161550 0x7f5e84156f20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00eb10 tx=0x7f5e7c00c2d0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 -- 172.21.15.60:0/1825362772 shutdown_connections 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f5e60078bb0 0x7f5e6007b070 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e84161550 0x7f5e84156f20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e840af3d0 0x7f5e841569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 --2- 172.21.15.60:0/1825362772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e84072df0 0x7f5e8414c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 -- 172.21.15.60:0/1825362772 >> 172.21.15.60:0/1825362772 conn(0x7f5e840b1950 msgr2=0x7f5e8415ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 -- 172.21.15.60:0/1825362772 shutdown_connections 2024-09-16T15:42:56.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.447+0000 7f5e8bc00640 1 -- 172.21.15.60:0/1825362772 wait complete. 2024-09-16T15:42:56.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:42:56.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.623+0000 7f3f9fe00640 1 -- 172.21.15.60:0/2863431894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 msgr2=0x7f3f980b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.623+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/2863431894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 0x7f3f980b2440 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3f9806dac0 tx=0x7f3f8402a560 comp rx=0 tx=0).stop 2024-09-16T15:42:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.623+0000 7f3f9fe00640 1 -- 172.21.15.60:0/2863431894 shutdown_connections 2024-09-16T15:42:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.623+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/2863431894 >> v2:172.21.15.100:3300/0 conn(0x7f3f98073fb0 0x7f3f98074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.623+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/2863431894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 0x7f3f980b2440 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.623+0000 7f3f9fe00640 1 -- 172.21.15.60:0/2863431894 >> 172.21.15.60:0/2863431894 conn(0x7f3f980b1950 msgr2=0x7f3f980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.623+0000 7f3f9fe00640 1 -- 172.21.15.60:0/2863431894 shutdown_connections 2024-09-16T15:42:56.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.623+0000 7f3f9fe00640 1 -- 172.21.15.60:0/2863431894 wait complete. 2024-09-16T15:42:56.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.625+0000 7f3f9fe00640 1 Processor -- start 2024-09-16T15:42:56.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.625+0000 7f3f9fe00640 1 -- start start 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f98073fb0 0x7f3f98155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 0x7f3f9815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f9815f5a0 0x7f3f9815bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9806f2e0 con 0x7f3f980ccf00 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9806f160 con 0x7f3f9815f5a0 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9806f460 con 0x7f3f98073fb0 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 0x7f3f9815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 0x7f3f9815b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60238/0 (socket says 172.21.15.60:60238) 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9ca00640 1 -- 172.21.15.60:0/3827117298 learned_addr learned my addr 172.21.15.60:0/3827117298 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9d400640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f98073fb0 0x7f3f98155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.626+0000 7f3f9de00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f9815f5a0 0x7f3f9815bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f9ca00640 1 -- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f98073fb0 msgr2=0x7f3f98155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f9ca00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f98073fb0 0x7f3f98155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f9ca00640 1 -- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f9815f5a0 msgr2=0x7f3f9815bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f9ca00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f9815f5a0 0x7f3f9815bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f9ca00640 1 -- 172.21.15.60:0/3827117298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f981f5820 con 0x7f3f980ccf00 2024-09-16T15:42:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f9ca00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 0x7f3f9815b7e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009850 tx=0x7f3f84003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f95600640 1 -- 172.21.15.60:0/3827117298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8403f070 con 0x7f3f980ccf00 2024-09-16T15:42:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f95600640 1 -- 172.21.15.60:0/3827117298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f840047a0 con 0x7f3f980ccf00 2024-09-16T15:42:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f95600640 1 -- 172.21.15.60:0/3827117298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84035030 con 0x7f3f980ccf00 2024-09-16T15:42:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.627+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981555d0 con 0x7f3f980ccf00 2024-09-16T15:42:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.628+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f981f4a60 con 0x7f3f980ccf00 2024-09-16T15:42:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.629+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9806f9f0 con 0x7f3f980ccf00 2024-09-16T15:42:56.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.630+0000 7f3f95600640 1 -- 172.21.15.60:0/3827117298 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f3f84042070 con 0x7f3f980ccf00 2024-09-16T15:42:56.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.631+0000 7f3f95600640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3f78078bb0 0x7f3f7807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:56.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.631+0000 7f3f95600640 1 -- 172.21.15.60:0/3827117298 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f3f840bbd90 con 0x7f3f980ccf00 2024-09-16T15:42:56.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.631+0000 7f3f9d400640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3f78078bb0 0x7f3f7807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:56.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.632+0000 7f3f9d400640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3f78078bb0 0x7f3f7807b070 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f9809acb0 tx=0x7f3f8c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:56.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.635+0000 7f3f95600640 1 -- 172.21.15.60:0/3827117298 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3f84083890 con 0x7f3f980ccf00 2024-09-16T15:42:56.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.868+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f980cc160 con 0x7f3f980ccf00 2024-09-16T15:42:56.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.868+0000 7f3f95600640 1 -- 172.21.15.60:0/3827117298 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f3f840882c0 con 0x7f3f980ccf00 2024-09-16T15:42:56.870 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.870+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3f78078bb0 msgr2=0x7f3f7807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.870+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3f78078bb0 0x7f3f7807b070 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f9809acb0 tx=0x7f3f8c009040 comp rx=0 tx=0).stop 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.870+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 msgr2=0x7f3f9815b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.870+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 0x7f3f9815b7e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009850 tx=0x7f3f84003910 comp rx=0 tx=0).stop 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.871+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 shutdown_connections 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.871+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3f78078bb0 0x7f3f7807b070 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.871+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f9815f5a0 0x7f3f9815bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.871+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f980ccf00 0x7f3f9815b7e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.871+0000 7f3f9fe00640 1 --2- 172.21.15.60:0/3827117298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f98073fb0 0x7f3f98155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.871+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 >> 172.21.15.60:0/3827117298 conn(0x7f3f980b1950 msgr2=0x7f3f9809cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:56.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.871+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 shutdown_connections 2024-09-16T15:42:56.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:56.871+0000 7f3f9fe00640 1 -- 172.21.15.60:0/3827117298 wait complete. 2024-09-16T15:42:56.889 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:42:56.889 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:42:56.889 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:42:56.890 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:42:56.890 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:56.890 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:42:56.890 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:42:56.890 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:42:56.890 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:56.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:56.890 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:42:56.926 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42472sec preferred_lft 42472sec 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:56.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:56.928 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:42:57.019 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:42:57.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:57.022 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:42:57.058 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:42:57.058 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:42:57.058 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:42:57.058 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:42:57.058 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:42:57.058 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:42:57.058 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:42:57.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:42:57.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:42:57.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:42:57.059 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:42:57.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:42:57.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:57.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:57.257 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:42:57.291 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:42:57.291 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:42:57.292 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:57.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:57.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:42:57.389 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:57.389 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:57.425 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:57.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:57.425 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:57.500 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:42:57.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:57.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:57.629 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:57.629 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:42:57.657 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:57.657 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:42:57.657 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:42:57.657 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:42:57.657 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:42:57.660 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:42:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:42:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:42:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:42:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:42:57.663 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:42:57.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:57.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:57.718 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:42:57.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:42:57.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:57.815 INFO:teuthology.orchestra.run.smithi060.stdout:client5675 2024-09-16T15:42:57.816 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:42:57.816 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:57.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:42:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.049+0000 7f9d18e00640 1 -- 172.21.15.60:0/3865068870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 msgr2=0x7f9d140732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.049+0000 7f9d18e00640 1 --2- 172.21.15.60:0/3865068870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 0x7f9d140732b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00acf0 tx=0x7f9d0c02bb00 comp rx=0 tx=0).stop 2024-09-16T15:42:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.050+0000 7f9d18e00640 1 -- 172.21.15.60:0/3865068870 shutdown_connections 2024-09-16T15:42:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.050+0000 7f9d18e00640 1 --2- 172.21.15.60:0/3865068870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 0x7f9d140732b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.050+0000 7f9d18e00640 1 -- 172.21.15.60:0/3865068870 >> 172.21.15.60:0/3865068870 conn(0x7f9d140b1950 msgr2=0x7f9d140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.050+0000 7f9d18e00640 1 -- 172.21.15.60:0/3865068870 shutdown_connections 2024-09-16T15:42:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.050+0000 7f9d18e00640 1 -- 172.21.15.60:0/3865068870 wait complete. 2024-09-16T15:42:58.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.051+0000 7f9d18e00640 1 Processor -- start 2024-09-16T15:42:58.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.051+0000 7f9d18e00640 1 -- start start 2024-09-16T15:42:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.052+0000 7f9d18e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d14074150 0x7f9d1414f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.052+0000 7f9d18e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 0x7f9d1415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.052+0000 7f9d18e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d140b43f0 0x7f9d1415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.052+0000 7f9d18e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14073970 con 0x7f9d140b43f0 2024-09-16T15:42:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.052+0000 7f9d18e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d140737f0 con 0x7f9d140746a0 2024-09-16T15:42:58.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.052+0000 7f9d18e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9d14073af0 con 0x7f9d14074150 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d140b43f0 0x7f9d1415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d140b43f0 0x7f9d1415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60244/0 (socket says 172.21.15.60:60244) 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 -- 172.21.15.60:0/3340468863 learned_addr learned my addr 172.21.15.60:0/3340468863 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d11e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d14074150 0x7f9d1414f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d0be00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 0x7f9d1415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 -- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d14074150 msgr2=0x7f9d1414f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d14074150 0x7f9d1414f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 -- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 msgr2=0x7f9d1415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 0x7f9d1415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 -- 172.21.15.60:0/3340468863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d141f5be0 con 0x7f9d140b43f0 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d11e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d14074150 0x7f9d1414f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d0be00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 0x7f9d1415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:58.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.053+0000 7f9d12800640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d140b43f0 0x7f9d1415b730 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00ae10 tx=0x7f9d0c030e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:58.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.054+0000 7f9d0a000640 1 -- 172.21.15.60:0/3340468863 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c046070 con 0x7f9d140b43f0 2024-09-16T15:42:58.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.054+0000 7f9d0a000640 1 -- 172.21.15.60:0/3340468863 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d0c004250 con 0x7f9d140b43f0 2024-09-16T15:42:58.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.054+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d141f6210 con 0x7f9d140b43f0 2024-09-16T15:42:58.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.054+0000 7f9d0a000640 1 -- 172.21.15.60:0/3340468863 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c004660 con 0x7f9d140b43f0 2024-09-16T15:42:58.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.054+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d141f4e80 con 0x7f9d140b43f0 2024-09-16T15:42:58.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.055+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d140b47d0 con 0x7f9d140b43f0 2024-09-16T15:42:58.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.056+0000 7f9d0a000640 1 -- 172.21.15.60:0/3340468863 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f9d0c049070 con 0x7f9d140b43f0 2024-09-16T15:42:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.057+0000 7f9d0a000640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9cf0078bb0 0x7f9cf007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.058+0000 7f9d0a000640 1 -- 172.21.15.60:0/3340468863 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f9d0c0bc270 con 0x7f9d140b43f0 2024-09-16T15:42:58.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.058+0000 7f9d11e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9cf0078bb0 0x7f9cf007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:58.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.059+0000 7f9d11e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9cf0078bb0 0x7f9cf007b070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9d040015d0 tx=0x7f9d04009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:58.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.062+0000 7f9d0a000640 1 -- 172.21.15.60:0/3340468863 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d0c033070 con 0x7f9d140b43f0 2024-09-16T15:42:58.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.300+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9d140cf4b0 con 0x7f9d140b43f0 2024-09-16T15:42:58.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.300+0000 7f9d0a000640 1 -- 172.21.15.60:0/3340468863 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f9d0c084410 con 0x7f9d140b43f0 2024-09-16T15:42:58.302 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.302+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9cf0078bb0 msgr2=0x7f9cf007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.302+0000 7f9d18e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9cf0078bb0 0x7f9cf007b070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9d040015d0 tx=0x7f9d04009040 comp rx=0 tx=0).stop 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.302+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d140b43f0 msgr2=0x7f9d1415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.302+0000 7f9d18e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d140b43f0 0x7f9d1415b730 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00ae10 tx=0x7f9d0c030e50 comp rx=0 tx=0).stop 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.303+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 shutdown_connections 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.303+0000 7f9d18e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9cf0078bb0 0x7f9cf007b070 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.303+0000 7f9d18e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d140b43f0 0x7f9d1415b730 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.303+0000 7f9d18e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d140746a0 0x7f9d1415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.303+0000 7f9d18e00640 1 --2- 172.21.15.60:0/3340468863 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d14074150 0x7f9d1414f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.303+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 >> 172.21.15.60:0/3340468863 conn(0x7f9d140b1950 msgr2=0x7f9d1415cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.303+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 shutdown_connections 2024-09-16T15:42:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:58.303+0000 7f9d18e00640 1 -- 172.21.15.60:0/3340468863 wait complete. 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:42:58.320 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:42:58.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:58.321 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:42:58.360 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:42:58.360 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:42:58.360 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:42:58.361 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42471sec preferred_lft 42471sec 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:42:58.362 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:42:58.363 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:58.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:58.363 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:42:58.460 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:42:58.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:58.464 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:42:58.500 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:42:58.500 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:42:58.500 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:42:58.500 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:42:58.500 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:42:58.501 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:42:58.501 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:42:58.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:42:58.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:42:58.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:42:58.501 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:42:58.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:42:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:42:58.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:42:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:58.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:58.696 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:42:58.730 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:42:58.731 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:42:58.731 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:42:58.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:58.732 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:42:58.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:58.827 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:58.862 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:42:58.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:58.862 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:58.936 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:42:58.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:58.936 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:59.064 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:59.064 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:42:59.092 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:42:59.092 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:42:59.092 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:42:59.092 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:42:59.092 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:42:59.094 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:42:59.095 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:42:59.095 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:42:59.095 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:42:59.095 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:42:59.097 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:42:59.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:59.098 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:42:59.152 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:42:59.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:42:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:42:59.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:59.241 INFO:teuthology.orchestra.run.smithi100.stdout:client5681 2024-09-16T15:42:59.241 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:42:59.242 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:59.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:42:59.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.451+0000 7fedfec00640 1 -- 172.21.15.60:0/2152551541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf80cea50 msgr2=0x7fedf80b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:59.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.451+0000 7fedfec00640 1 --2- 172.21.15.60:0/2152551541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf80cea50 0x7fedf80b3890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fede4009860 tx=0x7fede402a510 comp rx=0 tx=0).stop 2024-09-16T15:42:59.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.451+0000 7fedfec00640 1 -- 172.21.15.60:0/2152551541 shutdown_connections 2024-09-16T15:42:59.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.451+0000 7fedfec00640 1 --2- 172.21.15.60:0/2152551541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf80cea50 0x7fedf80b3890 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:59.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.451+0000 7fedfec00640 1 -- 172.21.15.60:0/2152551541 >> 172.21.15.60:0/2152551541 conn(0x7fedf80d0290 msgr2=0x7fedf80d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:59.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.451+0000 7fedfec00640 1 -- 172.21.15.60:0/2152551541 shutdown_connections 2024-09-16T15:42:59.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.451+0000 7fedfec00640 1 -- 172.21.15.60:0/2152551541 wait complete. 2024-09-16T15:42:59.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.453+0000 7fedfec00640 1 Processor -- start 2024-09-16T15:42:59.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.453+0000 7fedfec00640 1 -- start start 2024-09-16T15:42:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfec00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedf80cde60 0x7fedf80b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedf80cea50 0x7fedf8099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfec00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf815e190 0x7fedf80aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfec00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fedf815e6f0 con 0x7fedf80cde60 2024-09-16T15:42:59.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fedf815e570 con 0x7fedf80cea50 2024-09-16T15:42:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfec00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fedf80b44e0 con 0x7fedf815e190 2024-09-16T15:42:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfd800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedf80cde60 0x7fedf80b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfd800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedf80cde60 0x7fedf80b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52036/0 (socket says 172.21.15.60:52036) 2024-09-16T15:42:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.454+0000 7fedfd800640 1 -- 172.21.15.60:0/215276151 learned_addr learned my addr 172.21.15.60:0/215276151 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:42:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfce00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedf80cea50 0x7fedf8099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfe200640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf815e190 0x7fedf80aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfd800640 1 -- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf815e190 msgr2=0x7fedf80aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfd800640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf815e190 0x7fedf80aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfd800640 1 -- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedf80cea50 msgr2=0x7fedf8099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfd800640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedf80cea50 0x7fedf8099820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfd800640 1 -- 172.21.15.60:0/215276151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf80b0fa0 con 0x7fedf80cde60 2024-09-16T15:42:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfe200640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf815e190 0x7fedf80aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:59.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfd800640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedf80cde60 0x7fedf80b0a60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fedec00c440 tx=0x7fedec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:59.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedf5600640 1 -- 172.21.15.60:0/215276151 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec016020 con 0x7fedf80cde60 2024-09-16T15:42:59.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedf5600640 1 -- 172.21.15.60:0/215276151 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedec012920 con 0x7fedf80cde60 2024-09-16T15:42:59.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedf5600640 1 -- 172.21.15.60:0/215276151 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec012d70 con 0x7fedf80cde60 2024-09-16T15:42:59.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf80af9c0 con 0x7fedf80cde60 2024-09-16T15:42:59.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.455+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf80aff80 con 0x7fedf80cde60 2024-09-16T15:42:59.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.457+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf80b4a00 con 0x7fedf80cde60 2024-09-16T15:42:59.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.457+0000 7fedf5600640 1 -- 172.21.15.60:0/215276151 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fedec019020 con 0x7fedf80cde60 2024-09-16T15:42:59.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.458+0000 7fedf5600640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fedd8078bb0 0x7fedd807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:59.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.459+0000 7fedf5600640 1 -- 172.21.15.60:0/215276151 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7fedec09a8e0 con 0x7fedf80cde60 2024-09-16T15:42:59.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.459+0000 7fedfce00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fedd8078bb0 0x7fedd807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:59.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.459+0000 7fedfce00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fedd8078bb0 0x7fedd807b070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fede4034000 tx=0x7fede4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.463+0000 7fedf5600640 1 -- 172.21.15.60:0/215276151 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fedec020070 con 0x7fedf80cde60 2024-09-16T15:42:59.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:42:59.698+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fedf80b5de0 con 0x7fedf80cde60 2024-09-16T15:43:00.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.408+0000 7fedf5600640 1 -- 172.21.15.60:0/215276151 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v48) ==== 136+0+0 (secure 0 0 0) 0x7fedec062a80 con 0x7fedf80cde60 2024-09-16T15:43:00.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fedd8078bb0 msgr2=0x7fedd807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fedd8078bb0 0x7fedd807b070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fede4034000 tx=0x7fede4004730 comp rx=0 tx=0).stop 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedf80cde60 msgr2=0x7fedf80b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedf80cde60 0x7fedf80b0a60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fedec00c440 tx=0x7fedec00c900 comp rx=0 tx=0).stop 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 shutdown_connections 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fedd8078bb0 0x7fedd807b070 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fedf815e190 0x7fedf80aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedf80cea50 0x7fedf8099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 --2- 172.21.15.60:0/215276151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fedf80cde60 0x7fedf80b0a60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 >> 172.21.15.60:0/215276151 conn(0x7fedf80d0290 msgr2=0x7fedf80cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 shutdown_connections 2024-09-16T15:43:00.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.411+0000 7fedfec00640 1 -- 172.21.15.60:0/215276151 wait complete. 2024-09-16T15:43:00.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:43:00.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.654+0000 7fc8b0600640 1 -- 172.21.15.60:0/100886803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a8073fb0 msgr2=0x7fc8a8074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:00.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.654+0000 7fc8b0600640 1 --2- 172.21.15.60:0/100886803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a8073fb0 0x7fc8a8074460 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0005fd0 tx=0x7fc8a002a980 comp rx=0 tx=0).stop 2024-09-16T15:43:00.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.654+0000 7fc8b0600640 1 -- 172.21.15.60:0/100886803 shutdown_connections 2024-09-16T15:43:00.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.654+0000 7fc8b0600640 1 --2- 172.21.15.60:0/100886803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a8073fb0 0x7fc8a8074460 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.654+0000 7fc8b0600640 1 --2- 172.21.15.60:0/100886803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a815f5a0 0x7fc8a80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.654+0000 7fc8b0600640 1 -- 172.21.15.60:0/100886803 >> 172.21.15.60:0/100886803 conn(0x7fc8a80b1950 msgr2=0x7fc8a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:00.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.654+0000 7fc8b0600640 1 -- 172.21.15.60:0/100886803 shutdown_connections 2024-09-16T15:43:00.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.654+0000 7fc8b0600640 1 -- 172.21.15.60:0/100886803 wait complete. 2024-09-16T15:43:00.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.655+0000 7fc8b0600640 1 Processor -- start 2024-09-16T15:43:00.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.656+0000 7fc8b0600640 1 -- start start 2024-09-16T15:43:00.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.656+0000 7fc8b0600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a8073fb0 0x7fc8a80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:00.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.656+0000 7fc8b0600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a80ccf00 0x7fc8a81f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:00.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.656+0000 7fc8b0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a815f5a0 0x7fc8a81f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:00.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.656+0000 7fc8b0600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a806f2e0 con 0x7fc8a80ccf00 2024-09-16T15:43:00.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.656+0000 7fc8b0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a806f160 con 0x7fc8a815f5a0 2024-09-16T15:43:00.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.656+0000 7fc8b0600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a806f460 con 0x7fc8a8073fb0 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a80ccf00 0x7fc8a81f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a80ccf00 0x7fc8a81f46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52058/0 (socket says 172.21.15.60:52058) 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 -- 172.21.15.60:0/3142139359 learned_addr learned my addr 172.21.15.60:0/3142139359 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8adc00640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a8073fb0 0x7fc8a80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ae600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a815f5a0 0x7fc8a81f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 -- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a8073fb0 msgr2=0x7fc8a80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a8073fb0 0x7fc8a80cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 -- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a815f5a0 msgr2=0x7fc8a81f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a815f5a0 0x7fc8a81f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 -- 172.21.15.60:0/3142139359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a814fe40 con 0x7fc8a80ccf00 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ae600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a815f5a0 0x7fc8a81f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:00.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.657+0000 7fc8ad200640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a80ccf00 0x7fc8a81f46d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc89400c400 tx=0x7fc89400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.658+0000 7fc8a5600640 1 -- 172.21.15.60:0/3142139359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894011790 con 0x7fc8a80ccf00 2024-09-16T15:43:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.658+0000 7fc8a5600640 1 -- 172.21.15.60:0/3142139359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc894017070 con 0x7fc8a80ccf00 2024-09-16T15:43:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.658+0000 7fc8a5600640 1 -- 172.21.15.60:0/3142139359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894011d00 con 0x7fc8a80ccf00 2024-09-16T15:43:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.658+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a80d0430 con 0x7fc8a80ccf00 2024-09-16T15:43:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.658+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a80caa30 con 0x7fc8a80ccf00 2024-09-16T15:43:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.659+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a806f9f0 con 0x7fc8a80ccf00 2024-09-16T15:43:00.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.660+0000 7fc8a5600640 1 -- 172.21.15.60:0/3142139359 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fc89401c460 con 0x7fc8a80ccf00 2024-09-16T15:43:00.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.661+0000 7fc8a5600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc888078bb0 0x7fc88807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:00.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.661+0000 7fc8a5600640 1 -- 172.21.15.60:0/3142139359 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7fc89409acf0 con 0x7fc8a80ccf00 2024-09-16T15:43:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.662+0000 7fc8adc00640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc888078bb0 0x7fc88807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:00.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.663+0000 7fc8adc00640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc888078bb0 0x7fc88807b070 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80b2e00 tx=0x7fc89c00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:00.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.665+0000 7fc8a5600640 1 -- 172.21.15.60:0/3142139359 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc894062ed0 con 0x7fc8a80ccf00 2024-09-16T15:43:00.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:43:00.870 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:43:00.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.890+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc8a80cb1a0 con 0x7fc888078bb0 2024-09-16T15:43:00.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.892+0000 7fc8a5600640 1 -- 172.21.15.60:0/3142139359 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc8a80cb1a0 con 0x7fc888078bb0 2024-09-16T15:43:00.894 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:43:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:43:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:00.894 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:43:00.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc888078bb0 msgr2=0x7fc88807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:00.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc888078bb0 0x7fc88807b070 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80b2e00 tx=0x7fc89c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a80ccf00 msgr2=0x7fc8a81f46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a80ccf00 0x7fc8a81f46d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc89400c400 tx=0x7fc89400c8c0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 shutdown_connections 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fc888078bb0 0x7fc88807b070 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8a815f5a0 0x7fc8a81f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8a80ccf00 0x7fc8a81f46d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 --2- 172.21.15.60:0/3142139359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8a8073fb0 0x7fc8a80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 >> 172.21.15.60:0/3142139359 conn(0x7fc8a80b1950 msgr2=0x7fc8a809d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 shutdown_connections 2024-09-16T15:43:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:00.894+0000 7fc8b0600640 1 -- 172.21.15.60:0/3142139359 wait complete. 2024-09-16T15:43:00.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:43:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.113+0000 7fa2ef600640 1 -- 172.21.15.60:0/346767455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 msgr2=0x7fa2e80caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.113+0000 7fa2ef600640 1 --2- 172.21.15.60:0/346767455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 0x7fa2e80caa00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80c7070 tx=0x7fa2dc02a540 comp rx=0 tx=0).stop 2024-09-16T15:43:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.113+0000 7fa2ef600640 1 -- 172.21.15.60:0/346767455 shutdown_connections 2024-09-16T15:43:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.114+0000 7fa2ef600640 1 --2- 172.21.15.60:0/346767455 >> v2:172.21.15.100:3300/0 conn(0x7fa2e815e720 0x7fa2e815b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.114+0000 7fa2ef600640 1 --2- 172.21.15.60:0/346767455 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 0x7fa2e80caa00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.114+0000 7fa2ef600640 1 -- 172.21.15.60:0/346767455 >> 172.21.15.60:0/346767455 conn(0x7fa2e80b1ba0 msgr2=0x7fa2e80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.114+0000 7fa2ef600640 1 -- 172.21.15.60:0/346767455 shutdown_connections 2024-09-16T15:43:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.114+0000 7fa2ef600640 1 -- 172.21.15.60:0/346767455 wait complete. 2024-09-16T15:43:01.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.115+0000 7fa2ef600640 1 Processor -- start 2024-09-16T15:43:01.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.115+0000 7fa2ef600640 1 -- start start 2024-09-16T15:43:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.116+0000 7fa2ef600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e806fd10 0x7fa2e80aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.116+0000 7fa2ef600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 0x7fa2e81e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.116+0000 7fa2ef600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2e815e720 0x7fa2e81e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.116+0000 7fa2ef600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80cdd00 con 0x7fa2e80cf200 2024-09-16T15:43:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.116+0000 7fa2ef600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80cde80 con 0x7fa2e806fd10 2024-09-16T15:43:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.116+0000 7fa2ef600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e80ce000 con 0x7fa2e815e720 2024-09-16T15:43:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.116+0000 7fa2e7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 0x7fa2e81e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 0x7fa2e81e88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52080/0 (socket says 172.21.15.60:52080) 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e7e00640 1 -- 172.21.15.60:0/2922644759 learned_addr learned my addr 172.21.15.60:0/2922644759 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2ecc00640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e806fd10 0x7fa2e80aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2ed600640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2e815e720 0x7fa2e81e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e7e00640 1 -- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2e815e720 msgr2=0x7fa2e81e8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e7e00640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2e815e720 0x7fa2e81e8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e7e00640 1 -- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e806fd10 msgr2=0x7fa2e80aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e7e00640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e806fd10 0x7fa2e80aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e7e00640 1 -- 172.21.15.60:0/2922644759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e80b54a0 con 0x7fa2e80cf200 2024-09-16T15:43:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e7e00640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 0x7fa2e81e88e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400b410 tx=0x7fa2d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:01.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.117+0000 7fa2e4c00640 1 -- 172.21.15.60:0/2922644759 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4015020 con 0x7fa2e80cf200 2024-09-16T15:43:01.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.118+0000 7fa2e4c00640 1 -- 172.21.15.60:0/2922644759 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2d4011940 con 0x7fa2e80cf200 2024-09-16T15:43:01.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.118+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e81f7fe0 con 0x7fa2e80cf200 2024-09-16T15:43:01.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.118+0000 7fa2e4c00640 1 -- 172.21.15.60:0/2922644759 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4011d90 con 0x7fa2e80cf200 2024-09-16T15:43:01.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.118+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e81f8560 con 0x7fa2e80cf200 2024-09-16T15:43:01.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.119+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e80ce560 con 0x7fa2e80cf200 2024-09-16T15:43:01.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.120+0000 7fa2e4c00640 1 -- 172.21.15.60:0/2922644759 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7fa2d4018020 con 0x7fa2e80cf200 2024-09-16T15:43:01.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.121+0000 7fa2e4c00640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa2c8078bb0 0x7fa2c807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:01.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.121+0000 7fa2e4c00640 1 -- 172.21.15.60:0/2922644759 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7fa2d409ae00 con 0x7fa2e80cf200 2024-09-16T15:43:01.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.122+0000 7fa2ecc00640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa2c8078bb0 0x7fa2c807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:01.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.122+0000 7fa2ecc00640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa2c8078bb0 0x7fa2c807b070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc034000 tx=0x7fa2dc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:01.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.125+0000 7fa2e4c00640 1 -- 172.21.15.60:0/2922644759 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa2d401f020 con 0x7fa2e80cf200 2024-09-16T15:43:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.325+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa2e80b2320 con 0x7fa2e80cf200 2024-09-16T15:43:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.325+0000 7fa2e4c00640 1 -- 172.21.15.60:0/2922644759 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fa2d40628c0 con 0x7fa2e80cf200 2024-09-16T15:43:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa2c8078bb0 msgr2=0x7fa2c807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa2c8078bb0 0x7fa2c807b070 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc034000 tx=0x7fa2dc004730 comp rx=0 tx=0).stop 2024-09-16T15:43:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 msgr2=0x7fa2e81e88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 0x7fa2e81e88e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400b410 tx=0x7fa2d400b8d0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 shutdown_connections 2024-09-16T15:43:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7fa2c8078bb0 0x7fa2c807b070 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2e815e720 0x7fa2e81e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2e80cf200 0x7fa2e81e88e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 --2- 172.21.15.60:0/2922644759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e806fd10 0x7fa2e80aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 >> 172.21.15.60:0/2922644759 conn(0x7fa2e80b1ba0 msgr2=0x7fa2e80afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 shutdown_connections 2024-09-16T15:43:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.328+0000 7fa2ef600640 1 -- 172.21.15.60:0/2922644759 wait complete. 2024-09-16T15:43:01.345 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:01.346 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:01.381 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:01.381 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:01.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:43:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.658+0000 7f3791000640 1 -- 172.21.15.60:0/3513841155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0ccfd0 msgr2=0x7f378c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.658+0000 7f3791000640 1 --2- 172.21.15.60:0/3513841155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0ccfd0 0x7f378c0730b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3784009be0 tx=0x7f378402a970 comp rx=0 tx=0).stop 2024-09-16T15:43:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.658+0000 7f3791000640 1 -- 172.21.15.60:0/3513841155 shutdown_connections 2024-09-16T15:43:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.658+0000 7f3791000640 1 --2- 172.21.15.60:0/3513841155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0ccfd0 0x7f378c0730b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.658+0000 7f3791000640 1 -- 172.21.15.60:0/3513841155 >> 172.21.15.60:0/3513841155 conn(0x7f378c0b1950 msgr2=0x7f378c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:01.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.658+0000 7f3791000640 1 -- 172.21.15.60:0/3513841155 shutdown_connections 2024-09-16T15:43:01.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.658+0000 7f3791000640 1 -- 172.21.15.60:0/3513841155 wait complete. 2024-09-16T15:43:01.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.659+0000 7f3791000640 1 Processor -- start 2024-09-16T15:43:01.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3791000640 1 -- start start 2024-09-16T15:43:01.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3791000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c0ccfd0 0x7f378c13d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:01.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3791000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0cf300 0x7f378c1f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3791000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c1445d0 0x7f378c1f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3791000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0735f0 con 0x7f378c1445d0 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3791000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0b4f60 con 0x7f378c0cf300 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3791000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f378c073770 con 0x7f378c0ccfd0 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f378a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c1445d0 0x7f378c1f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f378a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c1445d0 0x7f378c1f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52086/0 (socket says 172.21.15.60:52086) 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f378a800640 1 -- 172.21.15.60:0/1143128652 learned_addr learned my addr 172.21.15.60:0/1143128652 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3781400640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0cf300 0x7f378c1f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f3789e00640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c0ccfd0 0x7f378c13d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f378a800640 1 -- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c0ccfd0 msgr2=0x7f378c13d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f378a800640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c0ccfd0 0x7f378c13d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f378a800640 1 -- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0cf300 msgr2=0x7f378c1f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f378a800640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0cf300 0x7f378c1f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.660+0000 7f378a800640 1 -- 172.21.15.60:0/1143128652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c0cbf80 con 0x7f378c1445d0 2024-09-16T15:43:01.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.661+0000 7f378a800640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c1445d0 0x7f378c1f8af0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3784009d00 tx=0x7f3784031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:01.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.661+0000 7f3782a00640 1 -- 172.21.15.60:0/1143128652 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784003c30 con 0x7f378c1445d0 2024-09-16T15:43:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.661+0000 7f3782a00640 1 -- 172.21.15.60:0/1143128652 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3784003dd0 con 0x7f378c1445d0 2024-09-16T15:43:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.661+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378c13dc10 con 0x7f378c1445d0 2024-09-16T15:43:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.661+0000 7f3782a00640 1 -- 172.21.15.60:0/1143128652 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37840042a0 con 0x7f378c1445d0 2024-09-16T15:43:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.661+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378c13e6f0 con 0x7f378c1445d0 2024-09-16T15:43:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.662+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378c073c20 con 0x7f378c1445d0 2024-09-16T15:43:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.664+0000 7f3782a00640 1 -- 172.21.15.60:0/1143128652 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f3784004440 con 0x7f378c1445d0 2024-09-16T15:43:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.664+0000 7f3782a00640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3768078bb0 0x7f376807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.664+0000 7f3782a00640 1 -- 172.21.15.60:0/1143128652 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f37840bc8c0 con 0x7f378c1445d0 2024-09-16T15:43:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.665+0000 7f3789e00640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3768078bb0 0x7f376807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.665+0000 7f3789e00640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3768078bb0 0x7f376807b070 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f378c159430 tx=0x7f377c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:01.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.669+0000 7f3782a00640 1 -- 172.21.15.60:0/1143128652 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f37840848e0 con 0x7f378c1445d0 2024-09-16T15:43:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.880+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f378c070a20 con 0x7f378c1445d0 2024-09-16T15:43:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.880+0000 7f3782a00640 1 -- 172.21.15.60:0/1143128652 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) ==== 80+0+0 (secure 0 0 0) 0x7f37840841e0 con 0x7f378c1445d0 2024-09-16T15:43:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:43:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3768078bb0 msgr2=0x7f376807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3768078bb0 0x7f376807b070 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f378c159430 tx=0x7f377c009040 comp rx=0 tx=0).stop 2024-09-16T15:43:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c1445d0 msgr2=0x7f378c1f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c1445d0 0x7f378c1f8af0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3784009d00 tx=0x7f3784031e60 comp rx=0 tx=0).stop 2024-09-16T15:43:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 shutdown_connections 2024-09-16T15:43:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f3768078bb0 0x7f376807b070 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f378c1445d0 0x7f378c1f8af0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f378c0cf300 0x7f378c1f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 --2- 172.21.15.60:0/1143128652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f378c0ccfd0 0x7f378c13d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 >> 172.21.15.60:0/1143128652 conn(0x7f378c0b1950 msgr2=0x7f378c0d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 shutdown_connections 2024-09-16T15:43:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:01.883+0000 7f3791000640 1 -- 172.21.15.60:0/1143128652 wait complete. 2024-09-16T15:43:01.901 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:43:01.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:01.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:43:01.964 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:43:01.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:01.964 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:43:02.000 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:02.000 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:02.041 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:02.041 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:02.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:43:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.279+0000 7f3396200640 1 -- 172.21.15.60:0/3820301342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901616b0 msgr2=0x7f33900d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.279+0000 7f3396200640 1 --2- 172.21.15.60:0/3820301342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901616b0 0x7f33900d21a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f33900c7390 tx=0x7f338802aac0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.279+0000 7f3396200640 1 -- 172.21.15.60:0/3820301342 shutdown_connections 2024-09-16T15:43:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.279+0000 7f3396200640 1 --2- 172.21.15.60:0/3820301342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901616b0 0x7f33900d21a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.279+0000 7f3396200640 1 --2- 172.21.15.60:0/3820301342 >> v2:172.21.15.100:3300/0 conn(0x7f33900d4980 0x7f33900d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.279+0000 7f3396200640 1 -- 172.21.15.60:0/3820301342 >> 172.21.15.60:0/3820301342 conn(0x7f33900d3df0 msgr2=0x7f33900d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.279+0000 7f3396200640 1 -- 172.21.15.60:0/3820301342 shutdown_connections 2024-09-16T15:43:02.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.279+0000 7f3396200640 1 -- 172.21.15.60:0/3820301342 wait complete. 2024-09-16T15:43:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.280+0000 7f3396200640 1 Processor -- start 2024-09-16T15:43:02.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.280+0000 7f3396200640 1 -- start start 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.281+0000 7f3396200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33900d4980 0x7f33901f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.281+0000 7f3396200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901612d0 0x7f339006e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.281+0000 7f3396200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33901616b0 0x7f339015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.281+0000 7f3396200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f33900b1bb0 con 0x7f33901612d0 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.281+0000 7f3396200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33900b1a30 con 0x7f33901616b0 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.281+0000 7f3396200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f33900b1d30 con 0x7f33900d4980 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901612d0 0x7f339006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901612d0 0x7f339006e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52106/0 (socket says 172.21.15.60:52106) 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 -- 172.21.15.60:0/427141102 learned_addr learned my addr 172.21.15.60:0/427141102 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3395800640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33901616b0 0x7f339015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3394e00640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33900d4980 0x7f33901f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:02.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 -- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33900d4980 msgr2=0x7f33901f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33900d4980 0x7f33901f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 -- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33901616b0 msgr2=0x7f339015dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33901616b0 0x7f339015dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 -- 172.21.15.60:0/427141102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3390156460 con 0x7f33901612d0 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3394e00640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33900d4980 0x7f33901f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.282+0000 7f3387e00640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901612d0 0x7f339006e790 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f337800d950 tx=0x7f337800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.283+0000 7f338d600640 1 -- 172.21.15.60:0/427141102 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378017070 con 0x7f33901612d0 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.283+0000 7f338d600640 1 -- 172.21.15.60:0/427141102 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3378009d20 con 0x7f33901612d0 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.283+0000 7f338d600640 1 -- 172.21.15.60:0/427141102 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780106c0 con 0x7f33901612d0 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.283+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339015e290 con 0x7f33901612d0 2024-09-16T15:43:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.283+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33901f8560 con 0x7f33901612d0 2024-09-16T15:43:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.284+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33900b2250 con 0x7f33901612d0 2024-09-16T15:43:02.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.286+0000 7f338d600640 1 -- 172.21.15.60:0/427141102 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f3378010860 con 0x7f33901612d0 2024-09-16T15:43:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.287+0000 7f338d600640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f336c078bb0 0x7f336c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.287+0000 7f338d600640 1 -- 172.21.15.60:0/427141102 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f337809b400 con 0x7f33901612d0 2024-09-16T15:43:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.287+0000 7f3394e00640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f336c078bb0 0x7f336c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:02.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.289+0000 7f3394e00640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f336c078bb0 0x7f336c07b070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3380007fd0 tx=0x7f3380009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:02.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.291+0000 7f338d600640 1 -- 172.21.15.60:0/427141102 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3378062dc0 con 0x7f33901612d0 2024-09-16T15:43:02.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.501+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f339015fac0 con 0x7f33901612d0 2024-09-16T15:43:02.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.501+0000 7f338d600640 1 -- 172.21.15.60:0/427141102 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) ==== 80+0+0 (secure 0 0 0) 0x7f33780677f0 con 0x7f33901612d0 2024-09-16T15:43:02.503 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:43:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f336c078bb0 msgr2=0x7f336c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f336c078bb0 0x7f336c07b070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3380007fd0 tx=0x7f3380009040 comp rx=0 tx=0).stop 2024-09-16T15:43:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901612d0 msgr2=0x7f339006e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901612d0 0x7f339006e790 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f337800d950 tx=0x7f337800de10 comp rx=0 tx=0).stop 2024-09-16T15:43:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 shutdown_connections 2024-09-16T15:43:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f336c078bb0 0x7f336c07b070 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33901616b0 0x7f339015dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33901612d0 0x7f339006e790 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 --2- 172.21.15.60:0/427141102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33900d4980 0x7f33901f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 >> 172.21.15.60:0/427141102 conn(0x7f33900d3df0 msgr2=0x7f33900ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:02.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 shutdown_connections 2024-09-16T15:43:02.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.504+0000 7f3396200640 1 -- 172.21.15.60:0/427141102 wait complete. 2024-09-16T15:43:02.521 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:43:02.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:02.522 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:43:02.589 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:43:02.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:02.589 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:43:02.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:02.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.784+0000 7f928a800640 1 -- 172.21.15.60:0/3566558446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92840b73c0 msgr2=0x7f9284071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.784+0000 7f928a800640 1 --2- 172.21.15.60:0/3566558446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92840b73c0 0x7f9284071bf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f92840cea00 tx=0x7f927402a560 comp rx=0 tx=0).stop 2024-09-16T15:43:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.784+0000 7f928a800640 1 -- 172.21.15.60:0/3566558446 shutdown_connections 2024-09-16T15:43:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.784+0000 7f928a800640 1 --2- 172.21.15.60:0/3566558446 >> v2:172.21.15.100:3300/0 conn(0x7f9284071260 0x7f92840cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.784+0000 7f928a800640 1 --2- 172.21.15.60:0/3566558446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92840b73c0 0x7f9284071bf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.784+0000 7f928a800640 1 -- 172.21.15.60:0/3566558446 >> 172.21.15.60:0/3566558446 conn(0x7f928406d230 msgr2=0x7f92840688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.784+0000 7f928a800640 1 -- 172.21.15.60:0/3566558446 shutdown_connections 2024-09-16T15:43:02.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.784+0000 7f928a800640 1 -- 172.21.15.60:0/3566558446 wait complete. 2024-09-16T15:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.786+0000 7f928a800640 1 Processor -- start 2024-09-16T15:43:02.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.786+0000 7f928a800640 1 -- start start 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f928a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284071260 0x7f928415b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f928a800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b73c0 0x7f928415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f928a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92840cefa0 0x7f9284157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f928a800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f92840b7920 con 0x7f9284071260 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f928a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92840b77a0 con 0x7f92840cefa0 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f928a800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f92840d2350 con 0x7f92840b73c0 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f9283400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284071260 0x7f928415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f9283400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284071260 0x7f928415b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52122/0 (socket says 172.21.15.60:52122) 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f9283400640 1 -- 172.21.15.60:0/298011695 learned_addr learned my addr 172.21.15.60:0/298011695 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f927aa00640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b73c0 0x7f928415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.787+0000 7f9283e00640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92840cefa0 0x7f9284157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f9283400640 1 -- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b73c0 msgr2=0x7f928415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f9283400640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b73c0 0x7f928415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f9283400640 1 -- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92840cefa0 msgr2=0x7f9284157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f9283400640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92840cefa0 0x7f9284157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f9283400640 1 -- 172.21.15.60:0/298011695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92841f5550 con 0x7f9284071260 2024-09-16T15:43:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f9283400640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284071260 0x7f928415b340 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f92740099c0 tx=0x7f92740038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f927be00640 1 -- 172.21.15.60:0/298011695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927403f070 con 0x7f9284071260 2024-09-16T15:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f927be00640 1 -- 172.21.15.60:0/298011695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92740047a0 con 0x7f9284071260 2024-09-16T15:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f927be00640 1 -- 172.21.15.60:0/298011695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274035030 con 0x7f9284071260 2024-09-16T15:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92841452f0 con 0x7f9284071260 2024-09-16T15:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.788+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92841458b0 con 0x7f9284071260 2024-09-16T15:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.789+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92840d28e0 con 0x7f9284071260 2024-09-16T15:43:02.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.792+0000 7f927be00640 1 -- 172.21.15.60:0/298011695 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f9274042070 con 0x7f9284071260 2024-09-16T15:43:02.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.792+0000 7f927be00640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9260078bb0 0x7f926007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:02.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.792+0000 7f927be00640 1 -- 172.21.15.60:0/298011695 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f92740bbfa0 con 0x7f9284071260 2024-09-16T15:43:02.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.792+0000 7f927aa00640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9260078bb0 0x7f926007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:02.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.793+0000 7f927aa00640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9260078bb0 0x7f926007b070 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92841461a0 tx=0x7f926c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:02.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:02.797+0000 7f927be00640 1 -- 172.21.15.60:0/298011695 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9274083af0 con 0x7f9284071260 2024-09-16T15:43:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.031+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92840c54d0 con 0x7f9284071260 2024-09-16T15:43:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.032+0000 7f927be00640 1 -- 172.21.15.60:0/298011695 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5231 (secure 0 0 0) 0x7f9274088520 con 0x7f9284071260 2024-09-16T15:43:03.033 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:03.033 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"btime":"2024-09-16T15:43:00:402258+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:43:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.034+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9260078bb0 msgr2=0x7f926007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9260078bb0 0x7f926007b070 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92841461a0 tx=0x7f926c005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284071260 msgr2=0x7f928415b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284071260 0x7f928415b340 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f92740099c0 tx=0x7f92740038a0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 shutdown_connections 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f9260078bb0 0x7f926007b070 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92840cefa0 0x7f9284157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92840b73c0 0x7f928415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 --2- 172.21.15.60:0/298011695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9284071260 0x7f928415b340 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 >> 172.21.15.60:0/298011695 conn(0x7f928406d230 msgr2=0x7f92840688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:03.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 shutdown_connections 2024-09-16T15:43:03.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.035+0000 7f928a800640 1 -- 172.21.15.60:0/298011695 wait complete. 2024-09-16T15:43:03.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-16T15:43:03.231 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-09-16T15:43:03.232 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-16T15:43:03.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol32220 2024-09-16T15:43:03.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.393+0000 7f9000e00640 1 -- 172.21.15.60:0/1798016521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 msgr2=0x7f8ffc073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.393+0000 7f9000e00640 1 --2- 172.21.15.60:0/1798016521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 0x7f8ffc073440 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009830 tx=0x7f8fec02a480 comp rx=0 tx=0).stop 2024-09-16T15:43:03.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.393+0000 7f9000e00640 1 -- 172.21.15.60:0/1798016521 shutdown_connections 2024-09-16T15:43:03.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.393+0000 7f9000e00640 1 --2- 172.21.15.60:0/1798016521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 0x7f8ffc073440 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.393+0000 7f9000e00640 1 -- 172.21.15.60:0/1798016521 >> 172.21.15.60:0/1798016521 conn(0x7f8ffc0b1840 msgr2=0x7f8ffc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:03.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.393+0000 7f9000e00640 1 -- 172.21.15.60:0/1798016521 shutdown_connections 2024-09-16T15:43:03.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.393+0000 7f9000e00640 1 -- 172.21.15.60:0/1798016521 wait complete. 2024-09-16T15:43:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.395+0000 7f9000e00640 1 Processor -- start 2024-09-16T15:43:03.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.395+0000 7f9000e00640 1 -- start start 2024-09-16T15:43:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f9000e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc06efa0 0x7f8ffc0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f9000e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ffc0cccd0 0x7f8ffc15d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f9000e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 0x7f8ffc1422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f9000e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc0ce6a0 con 0x7f8ffc145750 2024-09-16T15:43:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f9000e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc0ce520 con 0x7f8ffc06efa0 2024-09-16T15:43:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f9000e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8ffc0ce820 con 0x7f8ffc0cccd0 2024-09-16T15:43:03.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f8ffa800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 0x7f8ffc1422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f8ffa800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 0x7f8ffc1422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52138/0 (socket says 172.21.15.60:52138) 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f8ffa800640 1 -- 172.21.15.60:0/2948410163 learned_addr learned my addr 172.21.15.60:0/2948410163 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f8ff9e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc06efa0 0x7f8ffc0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f8ff1400640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ffc0cccd0 0x7f8ffc15d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f8ffa800640 1 -- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ffc0cccd0 msgr2=0x7f8ffc15d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f8ffa800640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ffc0cccd0 0x7f8ffc15d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.396+0000 7f8ffa800640 1 -- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc06efa0 msgr2=0x7f8ffc0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f8ffa800640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc06efa0 0x7f8ffc0acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f8ffa800640 1 -- 172.21.15.60:0/2948410163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc1f6c60 con 0x7f8ffc145750 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f8ff1400640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ffc0cccd0 0x7f8ffc15d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f8ffa800640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 0x7f8ffc1422f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400c780 tx=0x7f8ff400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f8ff2a00640 1 -- 172.21.15.60:0/2948410163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4004b60 con 0x7f8ffc145750 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f8ff2a00640 1 -- 172.21.15.60:0/2948410163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ff4004df0 con 0x7f8ffc145750 2024-09-16T15:43:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f8ff2a00640 1 -- 172.21.15.60:0/2948410163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400aa10 con 0x7f8ffc145750 2024-09-16T15:43:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc1f60e0 con 0x7f8ffc145750 2024-09-16T15:43:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.397+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1f66a0 con 0x7f8ffc145750 2024-09-16T15:43:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.398+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ffc073590 con 0x7f8ffc145750 2024-09-16T15:43:03.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.399+0000 7f8ff2a00640 1 -- 172.21.15.60:0/2948410163 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f8ff400abb0 con 0x7f8ffc145750 2024-09-16T15:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.401+0000 7f8ff2a00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8fd8078bb0 0x7f8fd807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.401+0000 7f8ff2a00640 1 -- 172.21.15.60:0/2948410163 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f8ff409b1e0 con 0x7f8ffc145750 2024-09-16T15:43:03.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.401+0000 7f8ff9e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8fd8078bb0 0x7f8fd807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:03.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.402+0000 7f8ff9e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8fd8078bb0 0x7f8fd807b070 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8fec034000 tx=0x7f8fec004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:03.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.405+0000 7f8ff2a00640 1 -- 172.21.15.60:0/2948410163 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ffc073590 con 0x7f8ffc145750 2024-09-16T15:43:03.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.627+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 --> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol32220", "target": ["mon-mgr", ""]}) -- 0x7f8ffc15d830 con 0x7f8fd8078bb0 2024-09-16T15:43:03.636 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:43:03.635+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:03.636 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:43:03.635+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:03.636 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:43:03.635+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:03.636 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:43:03.635+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:03.636 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:43:03.635+0000 7fa50ac00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:03.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.674+0000 7f8ff2a00640 1 -- 172.21.15.60:0/2948410163 <== mgr.4115 v2:172.21.15.100:6800/2198567224 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8fec0043b0 con 0x7f8fd8078bb0 2024-09-16T15:43:03.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8fd8078bb0 msgr2=0x7f8fd807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8fd8078bb0 0x7f8fd807b070 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8fec034000 tx=0x7f8fec004730 comp rx=0 tx=0).stop 2024-09-16T15:43:03.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 msgr2=0x7f8ffc1422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 0x7f8ffc1422f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400c780 tx=0x7f8ff400cc40 comp rx=0 tx=0).stop 2024-09-16T15:43:03.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 shutdown_connections 2024-09-16T15:43:03.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f8fd8078bb0 0x7f8fd807b070 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc145750 0x7f8ffc1422f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ffc0cccd0 0x7f8ffc15d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 --2- 172.21.15.60:0/2948410163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ffc06efa0 0x7f8ffc0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 >> 172.21.15.60:0/2948410163 conn(0x7f8ffc0b1840 msgr2=0x7f8ffc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:03.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 shutdown_connections 2024-09-16T15:43:03.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.677+0000 7f9000e00640 1 -- 172.21.15.60:0/2948410163 wait complete. 2024-09-16T15:43:03.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.856+0000 7f24cd400640 1 -- 172.21.15.60:0/3405419035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ac004800 msgr2=0x7f24ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.856+0000 7f24cd400640 1 --2- 172.21.15.60:0/3405419035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ac004800 0x7f24ac004c80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009b50 tx=0x7f24ac005830 comp rx=0 tx=0).stop 2024-09-16T15:43:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.856+0000 7f24c7200640 1 -- 172.21.15.60:0/3405419035 reap_dead start 2024-09-16T15:43:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.856+0000 7f24cd400640 1 -- 172.21.15.60:0/3405419035 shutdown_connections 2024-09-16T15:43:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.856+0000 7f24cd400640 1 -- 172.21.15.60:0/3405419035 >> 172.21.15.60:0/3405419035 conn(0x7f24c80b1950 msgr2=0x7f24c80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.856+0000 7f24cd400640 1 -- 172.21.15.60:0/3405419035 shutdown_connections 2024-09-16T15:43:03.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.857+0000 7f24cd400640 1 -- 172.21.15.60:0/3405419035 wait complete. 2024-09-16T15:43:03.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.858+0000 7f24cd400640 1 Processor -- start 2024-09-16T15:43:03.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.858+0000 7f24cd400640 1 -- start start 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24cd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ac004800 0x7f24c81f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24cd400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c809b8d0 0x7f24c81548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24cd400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c809cd40 0x7f24c809d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24cd400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80745c0 con 0x7f24c809cd40 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24cd400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8074440 con 0x7f24ac004800 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24cd400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f24c8074740 con 0x7f24c809b8d0 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24c7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c809cd40 0x7f24c809d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24c7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c809cd40 0x7f24c809d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52152/0 (socket says 172.21.15.60:52152) 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.859+0000 7f24c7200640 1 -- 172.21.15.60:0/1849062628 learned_addr learned my addr 172.21.15.60:0/1849062628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:03.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24bde00640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c809b8d0 0x7f24c81548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24c6800640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ac004800 0x7f24c81f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24c7200640 1 -- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c809b8d0 msgr2=0x7f24c81548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24c7200640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c809b8d0 0x7f24c81548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24c7200640 1 -- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ac004800 msgr2=0x7f24c81f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24c7200640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ac004800 0x7f24c81f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24c7200640 1 -- 172.21.15.60:0/1849062628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80ca220 con 0x7f24c809cd40 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24c6800640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ac004800 0x7f24c81f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24c7200640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c809cd40 0x7f24c809d1f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f24c8074390 tx=0x7f24c0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.860+0000 7f24bf400640 1 -- 172.21.15.60:0/1849062628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c000ee10 con 0x7f24c809cd40 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.861+0000 7f24bf400640 1 -- 172.21.15.60:0/1849062628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c0015070 con 0x7f24c809cd40 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.861+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c80ca4c0 con 0x7f24c809cd40 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.861+0000 7f24bf400640 1 -- 172.21.15.60:0/1849062628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0003d20 con 0x7f24c809cd40 2024-09-16T15:43:03.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.861+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c80caa80 con 0x7f24c809cd40 2024-09-16T15:43:03.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.862+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c80cacc0 con 0x7f24c809cd40 2024-09-16T15:43:03.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.863+0000 7f24bf400640 1 -- 172.21.15.60:0/1849062628 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 11) ==== 101593+0+0 (secure 0 0 0) 0x7f24c0004030 con 0x7f24c809cd40 2024-09-16T15:43:03.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.864+0000 7f24bf400640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f24a4078bb0 0x7f24a407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:03.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.864+0000 7f24bf400640 1 -- 172.21.15.60:0/1849062628 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f24c009b640 con 0x7f24c809cd40 2024-09-16T15:43:03.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.864+0000 7f24c6800640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f24a4078bb0 0x7f24a407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:03.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.865+0000 7f24c6800640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f24a4078bb0 0x7f24a407b070 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24b8001860 tx=0x7f24b8001ce0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:03.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:03.868+0000 7f24bf400640 1 -- 172.21.15.60:0/1849062628 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f24c80cacc0 con 0x7f24c809cd40 2024-09-16T15:43:04.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.102+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24c80b2440 con 0x7f24c809cd40 2024-09-16T15:43:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.103+0000 7f24bf400640 1 -- 172.21.15.60:0/1849062628 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5231 (secure 0 0 0) 0x7f24c0063090 con 0x7f24c809cd40 2024-09-16T15:43:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:04.104 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"btime":"2024-09-16T15:43:00:402258+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:43:04.104 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-09-16T15:43:04.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f24a4078bb0 msgr2=0x7f24a407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f24a4078bb0 0x7f24a407b070 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24b8001860 tx=0x7f24b8001ce0 comp rx=0 tx=0).stop 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c809cd40 msgr2=0x7f24c809d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c809cd40 0x7f24c809d1f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f24c8074390 tx=0x7f24c0010f40 comp rx=0 tx=0).stop 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 shutdown_connections 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f24a4078bb0 0x7f24a407b070 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c809cd40 0x7f24c809d1f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c809b8d0 0x7f24c81548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 --2- 172.21.15.60:0/1849062628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ac004800 0x7f24c81f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 >> 172.21.15.60:0/1849062628 conn(0x7f24c80b1950 msgr2=0x7f24c80d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.105+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 shutdown_connections 2024-09-16T15:43:04.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.106+0000 7f24cd400640 1 -- 172.21.15.60:0/1849062628 wait complete. 2024-09-16T15:43:04.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-16T15:43:04.299 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":5711,"entity":{"name":{"type":"client","num":5711},"addr":{"type":"any","addr":"172.21.15.100:0","nonce":2641588278}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":0.64699379800000001,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.967820253891247,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5711 172.21.15.100:0/2641588278","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"x","hostname":"smithi100","pid":"35825","root":"/"}}] 2024-09-16T15:43:04.299 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi100", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35825", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.100:0", "nonce": 2641588278, "type": "any" }, "name": { "num": 5711, "type": "client" } }, "id": 5711, "inst": "client.5711 172.21.15.100:0/2641588278", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.967820253891247 }, "state": "open", "uptime": 0.646993798 } ] 2024-09-16T15:43:04.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T15:43:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.461+0000 7f03d1600640 1 -- 172.21.15.60:0/2792371331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 msgr2=0x7f03cc06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.461+0000 7f03d1600640 1 --2- 172.21.15.60:0/2792371331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 0x7f03cc06ddd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f03bc005fd0 tx=0x7f03bc02a670 comp rx=0 tx=0).stop 2024-09-16T15:43:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.462+0000 7f03d1600640 1 -- 172.21.15.60:0/2792371331 shutdown_connections 2024-09-16T15:43:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.462+0000 7f03d1600640 1 --2- 172.21.15.60:0/2792371331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 0x7f03cc06ddd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.462+0000 7f03d1600640 1 -- 172.21.15.60:0/2792371331 >> 172.21.15.60:0/2792371331 conn(0x7f03cc0d1c40 msgr2=0x7f03cc0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.462+0000 7f03d1600640 1 -- 172.21.15.60:0/2792371331 shutdown_connections 2024-09-16T15:43:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.462+0000 7f03d1600640 1 -- 172.21.15.60:0/2792371331 wait complete. 2024-09-16T15:43:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.463+0000 7f03d1600640 1 Processor -- start 2024-09-16T15:43:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.463+0000 7f03d1600640 1 -- start start 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.464+0000 7f03d1600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03cc0b54a0 0x7f03cc0b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.464+0000 7f03d1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 0x7f03cc09f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.464+0000 7f03d1600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03cc15e2c0 0x7f03cc15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.464+0000 7f03d1600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc0b74e0 con 0x7f03cc15dee0 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.464+0000 7f03d1600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc0b7360 con 0x7f03cc0b54a0 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.464+0000 7f03d1600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f03cc0b7660 con 0x7f03cc15e2c0 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.464+0000 7f03cb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 0x7f03cc09f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.464+0000 7f03cb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 0x7f03cc09f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52170/0 (socket says 172.21.15.60:52170) 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03cb400640 1 -- 172.21.15.60:0/1438715831 learned_addr learned my addr 172.21.15.60:0/1438715831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03cbe00640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03cc0b54a0 0x7f03cc0b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03d0c00640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03cc15e2c0 0x7f03cc15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03cb400640 1 -- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03cc15e2c0 msgr2=0x7f03cc15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03cb400640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03cc15e2c0 0x7f03cc15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03cb400640 1 -- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03cc0b54a0 msgr2=0x7f03cc0b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03cb400640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03cc0b54a0 0x7f03cc0b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03cb400640 1 -- 172.21.15.60:0/1438715831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc0b11f0 con 0x7f03cc15dee0 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03cb400640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 0x7f03cc09f2d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c3c0 tx=0x7f03b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.465+0000 7f03c3e00640 1 -- 172.21.15.60:0/1438715831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8017070 con 0x7f03cc15dee0 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.466+0000 7f03c3e00640 1 -- 172.21.15.60:0/1438715831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03b80035f0 con 0x7f03cc15dee0 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.466+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc0b2720 con 0x7f03cc15dee0 2024-09-16T15:43:04.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.466+0000 7f03c3e00640 1 -- 172.21.15.60:0/1438715831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8003a20 con 0x7f03cc15dee0 2024-09-16T15:43:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.466+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc09d140 con 0x7f03cc15dee0 2024-09-16T15:43:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.467+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03cc06ddd0 con 0x7f03cc15dee0 2024-09-16T15:43:04.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.468+0000 7f03c3e00640 1 -- 172.21.15.60:0/1438715831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) ==== 101644+0+0 (secure 0 0 0) 0x7f03b801a070 con 0x7f03cc15dee0 2024-09-16T15:43:04.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.469+0000 7f03c3e00640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f03ac078c40 0x7f03ac07b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:04.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.469+0000 7f03c3e00640 1 -- 172.21.15.60:0/1438715831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7f03b809a3e0 con 0x7f03cc15dee0 2024-09-16T15:43:04.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.469+0000 7f03cbe00640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f03ac078c40 0x7f03ac07b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:04.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.470+0000 7f03cbe00640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f03ac078c40 0x7f03ac07b100 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f03bc033000 tx=0x7f03bc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:04.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.473+0000 7f03c3e00640 1 -- 172.21.15.60:0/1438715831 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f03cc0b11f0 con 0x7f03cc15dee0 2024-09-16T15:43:04.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.719+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f03cc06eb50 con 0x7f03cc15dee0 2024-09-16T15:43:04.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.726+0000 7f03c3e00640 1 -- 172.21.15.60:0/1438715831 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v12) ==== 81+0+530125 (secure 0 0 0) 0x7f03b8061cd0 con 0x7f03cc15dee0 2024-09-16T15:43:04.733 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:04.733 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:43:04.733 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 12, 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "active_gid": 4115, 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "active_name": "x", 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "active_addrs": { 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:6800", 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2198567224 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:6801", 2024-09-16T15:43:04.734 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2198567224 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "active_addr": "172.21.15.100:6801/2198567224", 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "active_change": "2024-09-16T15:38:13.695929+0000", 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "active_mgr_features": 4540719139924082687, 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "available": true, 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "standbys": [ 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 4129, 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "y", 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_features": 4540719139924082687, 2024-09-16T15:43:04.735 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.736 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.737 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.738 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.739 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.740 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.741 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.742 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.743 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.744 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.746 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.747 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-09-16T15:43:04.748 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-09-16T15:43:04.749 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.750 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.751 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.757 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T15:43:04.758 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T15:43:04.759 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.760 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.761 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.762 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.764 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.765 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-09-16T15:43:04.766 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.767 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T15:43:04.768 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.769 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.771 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.772 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.773 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.775 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.776 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-09-16T15:43:04.777 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.778 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.779 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.781 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-09-16T15:43:04.782 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-09-16T15:43:04.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-09-16T15:43:04.784 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.785 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.787 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.788 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-09-16T15:43:04.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-09-16T15:43:04.790 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nginx": { 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nginx", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.791 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nginx container image", 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.792 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_oauth2_proxy": { 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_oauth2_proxy", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-16T15:43:04.794 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "oauth2-proxy container image", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.795 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T15:43:04.796 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_samba": { 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_samba", 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.797 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Samba/SMB container image", 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_samba_metrics": { 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_samba_metrics", 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.800 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.801 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T15:43:04.802 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-09-16T15:43:04.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-09-16T15:43:04.804 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-09-16T15:43:04.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-09-16T15:43:04.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "grafana_dashboards_path": { 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "grafana_dashboards_path", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.808 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.810 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.811 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.812 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.813 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.815 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.816 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.817 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-09-16T15:43:04.818 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.819 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:04.820 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.821 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-09-16T15:43:04.822 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.823 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.824 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.825 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.827 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.828 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.829 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.831 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.832 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_keepalive_count_max": { 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_keepalive_count_max", 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-09-16T15:43:04.833 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_keepalive_interval": { 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_keepalive_interval", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.834 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.835 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.836 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T15:43:04.837 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.838 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.839 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.840 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.841 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.842 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.843 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.844 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.845 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.846 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.847 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.848 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.849 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.850 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-09-16T15:43:04.851 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T15:43:04.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.855 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.857 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.858 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T15:43:04.859 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T15:43:04.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.861 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.862 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.863 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.864 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.865 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.866 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.867 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "[]", 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.870 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "MULTICLUSTER_CONFIG": { 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "{}", 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.871 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.872 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.873 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.874 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.875 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.876 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.877 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.878 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.879 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.880 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.881 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.883 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.884 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.886 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.887 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.889 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "UNSAFE_TLS_v1_2": { 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.890 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.891 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.892 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.908 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.909 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.910 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.911 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-09-16T15:43:04.912 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-09-16T15:43:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.914 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.916 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.917 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.918 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.920 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.927 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.928 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.929 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.931 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.932 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.933 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-09-16T15:43:04.934 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.935 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.936 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.937 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-09-16T15:43:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.939 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:04.941 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.942 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.943 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.944 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-09-16T15:43:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-09-16T15:43:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-09-16T15:43:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-09-16T15:43:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-09-16T15:43:05.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.032 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.033 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.034 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.035 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.036 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-09-16T15:43:05.037 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-09-16T15:43:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-09-16T15:43:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-09-16T15:43:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-09-16T15:43:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "max_requests": { 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_requests", 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-16T15:43:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-09-16T15:43:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T15:43:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-09-16T15:43:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-09-16T15:43:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-09-16T15:43:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smb", 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "internal_store_backend": { 2024-09-16T15:43:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "internal_store_backend", 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "update_orchestration": { 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "update_orchestration", 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.140 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.141 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.142 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.143 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.144 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.145 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.146 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.148 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.149 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.150 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-09-16T15:43:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.152 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.153 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.154 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.155 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.156 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.157 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.158 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.159 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-09-16T15:43:05.160 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-09-16T15:43:05.161 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-09-16T15:43:05.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.163 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.164 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.165 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T15:43:05.167 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.168 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.169 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-09-16T15:43:05.170 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.171 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.172 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.173 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.174 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.175 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.176 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.177 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.178 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.179 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.180 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.181 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.183 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.184 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.185 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.186 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.187 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.188 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.189 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.190 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.191 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.192 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.193 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-09-16T15:43:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-09-16T15:43:05.195 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-09-16T15:43:05.196 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.197 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_no_wait": { 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.199 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.200 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.201 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:05.202 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.203 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.204 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.205 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.207 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.208 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.210 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.211 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.212 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: "modules": [ 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: "iostat", 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: "nfs", 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: "restful" 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.213 INFO:teuthology.orchestra.run.smithi060.stdout: "available_modules": [ 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "alerts", 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.214 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.215 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.216 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.217 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.218 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.219 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_destination": { 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_destination", 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.220 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_from_name": { 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_from_name", 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "Ceph", 2024-09-16T15:43:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Email From: name", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_host": { 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_host", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP server", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_password": { 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_password", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Password to authenticate with", 2024-09-16T15:43:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_port": { 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_port", 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "465", 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP port", 2024-09-16T15:43:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_sender": { 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_sender", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SMTP envelope sender", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_ssl": { 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_ssl", 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.226 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "smtp_user": { 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smtp_user", 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "User to authenticate as", 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.227 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.228 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "balancer", 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "active": { 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "active", 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.229 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_time": { 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_time", 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0000", 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.230 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "begin_weekday": { 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "begin_weekday", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T15:43:05.231 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_max_iterations": { 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_max_iterations", 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "25", 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "250", 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T15:43:05.232 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_metrics": { 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_metrics", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.233 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_compat_step": { 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crush_compat_step", 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.5", 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0.001", 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "0.999", 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "aggressiveness of optimization", 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "end_time": { 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_time", 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2359", 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "end_weekday": { 2024-09-16T15:43:05.235 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "end_weekday", 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "0", 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "6", 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.236 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.237 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.239 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.240 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.240 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.240 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.240 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.240 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.240 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.241 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.242 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "min_score": { 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_score", 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.243 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-09-16T15:43:05.244 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "upmap", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-compat", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "none", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "read", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap", 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap-read" 2024-09-16T15:43:05.245 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Balancer mode", 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_ids": { 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_ids", 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.246 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.247 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.248 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_deviation": { 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_deviation", 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.249 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "upmap_max_optimizations": { 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "upmap_max_optimizations", 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.250 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm", 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_down_multiplier": { 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_down_multiplier", 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.252 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_refresh_rate": { 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_refresh_rate", 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "20", 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.253 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "agent_starting_port": { 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "agent_starting_port", 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4721", 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.254 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_ptrace": { 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_ptrace", 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_interval": { 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_interval", 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.256 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "autotune_memory_target_ratio": { 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.257 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0.7", 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_log_destination": { 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephadm_log_destination", 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.258 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "file", 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "file,syslog", 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "syslog" 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "cgroups_split": { 2024-09-16T15:43:05.259 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cgroups_split", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "config_checks_enabled": { 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_checks_enabled", 2024-09-16T15:43:05.260 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "config_dashboard": { 2024-09-16T15:43:05.261 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "config_dashboard", 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_alertmanager": { 2024-09-16T15:43:05.262 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_alertmanager", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_base": { 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_base", 2024-09-16T15:43:05.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Container image name, without the tag", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_elasticsearch": { 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_elasticsearch", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.264 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "elasticsearch container image", 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_grafana": { 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_grafana", 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.265 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_haproxy": { 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_haproxy", 2024-09-16T15:43:05.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "HAproxy container image", 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.267 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_agent": { 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_agent", 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger agent container image", 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.268 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_collector": { 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_collector", 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger collector container image", 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.269 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_jaeger_query": { 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_jaeger_query", 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Jaeger query container image", 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_keepalived": { 2024-09-16T15:43:05.270 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_keepalived", 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Keepalived container image", 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.271 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_loki": { 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_loki", 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Loki container image", 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.272 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nginx": { 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nginx", 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nginx container image", 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.273 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_node_exporter": { 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_node_exporter", 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.274 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_nvmeof": { 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_nvmeof", 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Nvme-of container image", 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.275 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_oauth2_proxy": { 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_oauth2_proxy", 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "oauth2-proxy container image", 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.276 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_prometheus": { 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_prometheus", 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Prometheus container image", 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.277 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_promtail": { 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_promtail", 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.278 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Promtail container image", 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_samba": { 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_samba", 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.279 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Samba/SMB container image", 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_samba_metrics": { 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_samba_metrics", 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.280 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image_snmp_gateway": { 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_image_snmp_gateway", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T15:43:05.281 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "SNMP Gateway container image", 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "container_init": { 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "container_init", 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.282 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "daemon_cache_timeout": { 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "daemon_cache_timeout", 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.283 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "default_cephadm_command_timeout": { 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.284 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "900", 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "default_registry": { 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "default_registry", 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.285 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "docker.io", 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "device_cache_timeout": { 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_cache_timeout", 2024-09-16T15:43:05.286 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1800", 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache device inventory", 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "device_enhanced_scan": { 2024-09-16T15:43:05.287 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_enhanced_scan", 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "facts_cache_timeout": { 2024-09-16T15:43:05.288 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "facts_cache_timeout", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "seconds to cache host facts data", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "grafana_dashboards_path": { 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "grafana_dashboards_path", 2024-09-16T15:43:05.289 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "host_check_interval": { 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "host_check_interval", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.290 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to perform a host check", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "hw_monitoring": { 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hw_monitoring", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.291 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "inventory_list_all": { 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "inventory_list_all", 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.293 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "log_refresh_metadata": { 2024-09-16T15:43:05.294 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_refresh_metadata", 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.295 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.296 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.297 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.298 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.299 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "*", 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.300 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "max_count_per_host": { 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_count_per_host", 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.301 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of daemons per service per host", 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd_draining_count": { 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_osd_draining_count", 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.302 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "migration_current": { 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "migration_current", 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.303 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "internal - do not modify", 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": { 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mode", 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "root", 2024-09-16T15:43:05.304 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm-package", 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "root" 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "oob_default_addr": { 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "oob_default_addr", 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.305 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "169.254.1.1", 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "prometheus_alerts_path": { 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus_alerts_path", 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.306 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_insecure": { 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_insecure", 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.307 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_password": { 2024-09-16T15:43:05.308 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_password", 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_url": { 2024-09-16T15:43:05.309 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_url", 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.310 INFO:teuthology.orchestra.run.smithi060.stdout: "registry_username": { 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "registry_username", 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "secure_monitoring_stack": { 2024-09-16T15:43:05.311 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secure_monitoring_stack", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "service_discovery_port": { 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "service_discovery_port", 2024-09-16T15:43:05.312 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8765", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "cephadm service discovery port", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.313 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_config_file": { 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_config_file", 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.314 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_keepalive_count_max": { 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_keepalive_count_max", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-09-16T15:43:05.315 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "ssh_keepalive_interval": { 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssh_keepalive_interval", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.316 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "use_agent": { 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_agent", 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.317 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "use_repo_digest": { 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "use_repo_digest", 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.318 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_failed_host_check": { 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_failed_host_check", 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.319 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_daemons": { 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_daemons", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.320 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_on_stray_hosts": { 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_on_stray_hosts", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.321 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crash", 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.322 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.323 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.324 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.325 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.326 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "retain_interval": { 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retain_interval", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "31536000", 2024-09-16T15:43:05.327 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.328 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_recent_interval": { 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_recent_interval", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1209600", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dashboard", 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.330 INFO:teuthology.orchestra.run.smithi060.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.332 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.333 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_ENABLED": { 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.334 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.335 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.336 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.337 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.338 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.339 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.340 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.341 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.342 INFO:teuthology.orchestra.run.smithi060.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T15:43:05.343 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T15:43:05.344 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.345 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_URL": { 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_URL", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.346 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.347 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.348 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.349 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.350 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.351 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "[]", 2024-09-16T15:43:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "MULTICLUSTER_CONFIG": { 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "{}", 2024-09-16T15:43:05.353 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.354 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.355 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.356 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.357 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.358 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.359 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.360 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.361 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.362 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.363 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8", 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T15:43:05.366 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "45", 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T15:43:05.367 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T15:43:05.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "admin", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.369 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.370 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "UNSAFE_TLS_v1_2": { 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.371 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.372 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.373 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.374 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "cross_origin_url": { 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cross_origin_url", 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.375 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "crt_file": { 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "crt_file", 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.376 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "debug": { 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "debug", 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.377 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Enable/disable debug options", 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "jwt_token_ttl": { 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "jwt_token_ttl", 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.378 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "28800", 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.379 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.380 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.381 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.382 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.383 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.384 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.385 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "motd": { 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "motd", 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "The message of the day", 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.386 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect_resolve_ip_addr": { 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.387 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.388 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8080", 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.389 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.390 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.391 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl_server_port": { 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl_server_port", 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.392 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8443", 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.393 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "redirect", 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "redirect" 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-09-16T15:43:05.394 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.395 INFO:teuthology.orchestra.run.smithi060.stdout: "url_prefix": { 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url_prefix", 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_monitoring": { 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_monitoring", 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.397 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "monitor device health metrics", 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.398 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.401 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.402 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mark_out_threshold": { 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mark_out_threshold", 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "2419200", 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.403 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": { 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pool_name", 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "device_health_metrics", 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "retention_period": { 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "retention_period", 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15552000", 2024-09-16T15:43:05.405 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long to retain device health metrics", 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_frequency": { 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_frequency", 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-09-16T15:43:05.406 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "self_heal": { 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "self_heal", 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.407 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.408 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.409 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "warn_threshold": { 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "warn_threshold", 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.410 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "7257600", 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "diskprediction_local", 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.411 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.412 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.413 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.414 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.415 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.416 INFO:teuthology.orchestra.run.smithi060.stdout: "predict_interval": { 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predict_interval", 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "86400", 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "predictor_model": { 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "predictor_model", 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "prophetstor", 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "600", 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "influx", 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": false, 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "influxdb python module not found", 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "batch_size": { 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "batch_size", 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5000", 2024-09-16T15:43:05.421 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "database": { 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "database", 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.422 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "ceph", 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": { 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "hostname", 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.423 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server hostname", 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.424 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "30", 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "5", 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.425 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.426 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.427 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.428 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.429 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "password": { 2024-09-16T15:43:05.430 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "password", 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "password of InfluxDB server user", 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.431 INFO:teuthology.orchestra.run.smithi060.stdout: "port": { 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "port", 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "8086", 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "InfluxDB server port", 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "ssl": { 2024-09-16T15:43:05.433 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "ssl", 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "false", 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "threads": { 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threads", 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "32", 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "username": { 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "username", 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "username of InfluxDB server user", 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "verify_ssl": { 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "verify_ssl", 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "true", 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "insights", 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.438 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.439 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.442 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.443 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "iostat", 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.445 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.446 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.447 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.448 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.449 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.450 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "localpool", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "failure_domain": { 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "failure_domain", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "host", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "failure domain for any created local pool", 2024-09-16T15:43:05.451 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.452 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.454 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.455 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.456 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": { 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "min_size", 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default min_size for any created local pool", 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.457 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "num_rep": { 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "num_rep", 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3", 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default replica count for any created local pool", 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.458 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": { 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_num", 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "128", 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T15:43:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "prefix": { 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prefix", 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "name prefix for any created local pool", 2024-09-16T15:43:05.460 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.461 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "subtree": { 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subtree", 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "rack", 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.462 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mds_autoscaler", 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.463 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.464 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.465 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.466 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.467 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.468 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirroring", 2024-09-16T15:43:05.469 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.471 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.472 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.473 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.474 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.475 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "nfs", 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.476 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.477 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.478 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.479 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.480 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.481 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.482 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "fail_fs": { 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fail_fs", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.484 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.485 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.486 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.487 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.488 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator": { 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "orchestrator", 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm", 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "rook", 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: "test_orchestrator" 2024-09-16T15:43:05.489 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Orchestrator backend", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.490 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_perf_query", 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.491 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.496 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "osd_support", 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.497 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.498 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.499 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.500 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.501 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.502 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "pg_autoscaler", 2024-09-16T15:43:05.503 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.504 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.505 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.506 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.507 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.508 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.509 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.510 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "threshold": { 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "threshold", 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "3.0", 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1.0", 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "scaling threshold", 2024-09-16T15:43:05.511 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "progress", 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_pg_recovery_event": { 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_pg_recovery_event", 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.512 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.513 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.514 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.515 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.516 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.517 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.518 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "max_completed_events": { 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_completed_events", 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.519 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "50", 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "number of past completed events to remember", 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "sleep_interval": { 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sleep_interval", 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.520 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "how long the module is going to sleep", 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.521 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "prometheus", 2024-09-16T15:43:05.522 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "cache": { 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cache", 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.523 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "exclude_perf_counters": { 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "exclude_perf_counters", 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.524 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.525 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.526 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.527 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.528 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.529 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools": { 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools", 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.530 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "300", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.531 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "scrape_interval": { 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "scrape_interval", 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15.0", 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.532 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "::", 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.533 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "9283", 2024-09-16T15:43:05.534 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.535 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "stale_cache_strategy": { 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stale_cache_strategy", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "log", 2024-09-16T15:43:05.536 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_behaviour": { 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_behaviour", 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "default", 2024-09-16T15:43:05.537 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "default", 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "error" 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "standby_error_status_code": { 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "standby_error_status_code", 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.538 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "400", 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "599", 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.539 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.540 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.541 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.542 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.543 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_snap_create": { 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_snap_create", 2024-09-16T15:43:05.544 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10", 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "mirror_snapshot_schedule": { 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T15:43:05.545 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.546 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "trash_purge_schedule": { 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "trash_purge_schedule", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.547 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "restful", 2024-09-16T15:43:05.548 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_auth": { 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enable_auth", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "key_file": { 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "key_file", 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.550 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.551 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.552 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.553 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.554 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "max_requests": { 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_requests", 2024-09-16T15:43:05.555 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "500", 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "server_addr": { 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_addr", 2024-09-16T15:43:05.556 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "server_port": { 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "server_port", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.557 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.558 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.559 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rgw", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.560 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.561 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.562 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.563 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.564 INFO:teuthology.orchestra.run.smithi060.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "5", 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.566 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rook", 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.567 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.569 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.570 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.571 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.572 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "storage_class": { 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "storage_class", 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "local", 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.573 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "selftest", 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.574 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.575 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.576 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.577 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.578 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "roption1": { 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption1", 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.579 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "roption2": { 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "roption2", 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.580 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "xyz", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption1": { 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption1", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.581 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption2": { 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption2", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.582 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption3": { 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption3", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "float", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.583 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption4": { 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption4", 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.584 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption5": { 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption5", 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.585 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption6": { 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption6", 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.586 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "rwoption7": { 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rwoption7", 2024-09-16T15:43:05.587 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "1", 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "42", 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.588 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "testkey": { 2024-09-16T15:43:05.589 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testkey", 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "testlkey": { 2024-09-16T15:43:05.590 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testlkey", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "testnewline": { 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "testnewline", 2024-09-16T15:43:05.591 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.592 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "smb", 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "internal_store_backend": { 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "internal_store_backend", 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.593 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.594 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.595 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.596 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.597 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.598 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.599 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "update_orchestration": { 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "update_orchestration", 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.600 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snap_schedule", 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_m_granularity": { 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "allow_m_granularity", 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.602 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "dump_on_update": { 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "dump_on_update", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "dump database to debug log on update", 2024-09-16T15:43:05.603 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.605 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.606 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.607 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.608 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.609 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "stats", 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.610 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.611 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.612 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.613 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.614 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.615 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "status", 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.616 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.617 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.619 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.620 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.621 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telegraf", 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "address": { 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "address", 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.622 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.623 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "15", 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.624 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.625 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.626 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.627 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.628 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.629 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "telemetry", 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_basic": { 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_basic", 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.630 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_crash": { 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_crash", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.631 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_device": { 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_device", 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.632 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_ident": { 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_ident", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.633 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "channel_perf": { 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "channel_perf", 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.634 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "contact": { 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "contact", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.635 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "description": { 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "description", 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.636 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "device_url": { 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "device_url", 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "enabled": { 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "enabled", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.638 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "24", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "8", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.639 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "last_opt_revision": { 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "last_opt_revision", 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "1", 2024-09-16T15:43:05.640 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard": { 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.641 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "leaderboard_description": { 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "leaderboard_description", 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.642 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.643 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.644 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.645 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.646 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.647 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "organization": { 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "organization", 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.648 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "proxy": { 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "proxy", 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.649 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.650 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "url": { 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "url", 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.651 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "test_orchestrator", 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.652 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.653 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.654 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.655 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.656 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.657 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.659 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.660 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.661 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.662 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.663 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "max_concurrent_clones": { 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "max_concurrent_clones", 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "4", 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.664 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "periodic_async_work": { 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "periodic_async_work", 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Periodically check for async work", 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_delay": { 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_delay", 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.666 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot_clone_no_wait": { 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "True", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.667 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.668 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "can_run": true, 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "error_string": "", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "module_options": { 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "discovery_interval": { 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "discovery_interval", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "uint", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "100", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.669 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "identifier": { 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "identifier", 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.670 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "interval": { 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "interval", 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "secs", 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "60", 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.671 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "log_level": { 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_level", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.672 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster": { 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.673 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_cluster_level": { 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_cluster_level", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "info", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.674 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [ 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "", 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "critical", 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "debug", 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "error", 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "info", 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "warning" 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": { 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "log_to_file", 2024-09-16T15:43:05.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "bool", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "False", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "sqlite3_killpoint": { 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "sqlite3_killpoint", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.676 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "dev", 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "0", 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_host": { 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_host", 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.677 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_port": { 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_port", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "int", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.678 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "10051", 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "zabbix_sender": { 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "zabbix_sender", 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "str", 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "level": "advanced", 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 0, 2024-09-16T15:43:05.679 INFO:teuthology.orchestra.run.smithi060.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "min": "", 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "max": "", 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "enum_allowed": [], 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "desc": "", 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "long_desc": "", 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "tags": [], 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "see_also": [] 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "services": {}, 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "always_on_modules": { 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "octopus": [ 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-09-16T15:43:05.680 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "pacific": [ 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-09-16T15:43:05.681 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "quincy": [ 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-09-16T15:43:05.682 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "reef": [ 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:43:05.683 INFO:teuthology.orchestra.run.smithi060.stdout: "squid": [ 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "balancer", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "crash", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "devicehealth", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "orchestrator", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscaler", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "progress", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_support", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "status", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "telemetry", 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "volumes" 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "last_failure_osd_epoch": 0, 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "active_clients": [ 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.684 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "devicehealth", 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4072261497 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "libcephsqlite", 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4216665037 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "rbd_support", 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1852740290 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.686 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1587892015 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.687 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 4177407071 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3427025651 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.688 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2997397432 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 690787621 2024-09-16T15:43:05.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2410226558 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.690 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 883853961 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "volumes", 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.100:0", 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2641588278 2024-09-16T15:43:05.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.692 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.692 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:05.692 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:05.692 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:43:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f03ac078c40 msgr2=0x7f03ac07b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f03ac078c40 0x7f03ac07b100 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f03bc033000 tx=0x7f03bc004730 comp rx=0 tx=0).stop 2024-09-16T15:43:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 msgr2=0x7f03cc09f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 0x7f03cc09f2d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f03b800c3c0 tx=0x7f03b800c880 comp rx=0 tx=0).stop 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 shutdown_connections 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7f03ac078c40 0x7f03ac07b100 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03cc15e2c0 0x7f03cc15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03cc15dee0 0x7f03cc09f2d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.731+0000 7f03d1600640 1 --2- 172.21.15.60:0/1438715831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03cc0b54a0 0x7f03cc0b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.732+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 >> 172.21.15.60:0/1438715831 conn(0x7f03cc0d1c40 msgr2=0x7f03cc0c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.732+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 shutdown_connections 2024-09-16T15:43:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:04.732+0000 7f03d1600640 1 -- 172.21.15.60:0/1438715831 wait complete. 2024-09-16T15:43:05.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2024-09-16T15:43:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.871+0000 7ff9aba00640 1 -- 172.21.15.60:0/722636652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a415f5a0 msgr2=0x7ff9a40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.871+0000 7ff9aba00640 1 --2- 172.21.15.60:0/722636652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a415f5a0 0x7ff9a40b2440 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff998009830 tx=0x7ff99802a530 comp rx=0 tx=0).stop 2024-09-16T15:43:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.871+0000 7ff9aba00640 1 -- 172.21.15.60:0/722636652 shutdown_connections 2024-09-16T15:43:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.871+0000 7ff9aba00640 1 --2- 172.21.15.60:0/722636652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a415f5a0 0x7ff9a40b2440 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.871+0000 7ff9aba00640 1 -- 172.21.15.60:0/722636652 >> 172.21.15.60:0/722636652 conn(0x7ff9a40b1950 msgr2=0x7ff9a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.872+0000 7ff9aba00640 1 -- 172.21.15.60:0/722636652 shutdown_connections 2024-09-16T15:43:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.872+0000 7ff9aba00640 1 -- 172.21.15.60:0/722636652 wait complete. 2024-09-16T15:43:05.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.873+0000 7ff9aba00640 1 Processor -- start 2024-09-16T15:43:05.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.873+0000 7ff9aba00640 1 -- start start 2024-09-16T15:43:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.874+0000 7ff9aba00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a4073fb0 0x7ff9a41f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.874+0000 7ff9aba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40ccf00 0x7ff9a41f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.874+0000 7ff9aba00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a415f5a0 0x7ff9a4157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.874+0000 7ff9aba00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a406f9f0 con 0x7ff9a415f5a0 2024-09-16T15:43:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.874+0000 7ff9aba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a406f870 con 0x7ff9a40ccf00 2024-09-16T15:43:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.874+0000 7ff9aba00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a4074420 con 0x7ff9a4073fb0 2024-09-16T15:43:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.874+0000 7ff9a9a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a415f5a0 0x7ff9a4157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.874+0000 7ff9a9a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a415f5a0 0x7ff9a4157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52180/0 (socket says 172.21.15.60:52180) 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9a00640 1 -- 172.21.15.60:0/882930892 learned_addr learned my addr 172.21.15.60:0/882930892 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9000640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a4073fb0 0x7ff9a41f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a3e00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40ccf00 0x7ff9a41f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9a00640 1 -- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a4073fb0 msgr2=0x7ff9a41f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9a00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a4073fb0 0x7ff9a41f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9a00640 1 -- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40ccf00 msgr2=0x7ff9a41f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9a00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40ccf00 0x7ff9a41f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9a00640 1 -- 172.21.15.60:0/882930892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a41f2ec0 con 0x7ff9a415f5a0 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9000640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a4073fb0 0x7ff9a41f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a9a00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a415f5a0 0x7ff9a4157600 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00cbc0 tx=0x7ff99c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:05.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c016020 con 0x7ff9a415f5a0 2024-09-16T15:43:05.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff99c00add0 con 0x7ff9a415f5a0 2024-09-16T15:43:05.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.875+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a41f31a0 con 0x7ff9a415f5a0 2024-09-16T15:43:05.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.876+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c012680 con 0x7ff9a415f5a0 2024-09-16T15:43:05.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.876+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a4157b40 con 0x7ff9a415f5a0 2024-09-16T15:43:05.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.877+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a41f36c0 con 0x7ff9a415f5a0 2024-09-16T15:43:05.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.878+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 12) ==== 101644+0+0 (secure 0 0 0) 0x7ff99c019020 con 0x7ff9a415f5a0 2024-09-16T15:43:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.879+0000 7ff9a0c00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff984078c40 0x7ff98407b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.879+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5709+0+0 (secure 0 0 0) 0x7ff99c09b4e0 con 0x7ff9a415f5a0 2024-09-16T15:43:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.879+0000 7ff9a9000640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff984078c40 0x7ff98407b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.880+0000 7ff9a9000640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff984078c40 0x7ff98407b100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff998034000 tx=0x7ff998004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:05.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:05.883+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff9a41f36c0 con 0x7ff9a415f5a0 2024-09-16T15:43:06.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.126+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) -- 0x7ff9a40b2440 con 0x7ff9a415f5a0 2024-09-16T15:43:06.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.131+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7ff99c020030 con 0x7ff9a415f5a0 2024-09-16T15:43:06.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.133+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 13) ==== 50665+0+0 (secure 0 0 0) 0x7ff99c05ef90 con 0x7ff9a415f5a0 2024-09-16T15:43:06.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9a0c00640 1 -- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff984078c40 msgr2=0x7ff98407b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:06.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9a0c00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff984078c40 0x7ff98407b100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff998034000 tx=0x7ff998004730 comp rx=0 tx=0).stop 2024-09-16T15:43:06.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a415f5a0 msgr2=0x7ff9a4157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:06.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9aba00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a415f5a0 0x7ff9a4157600 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00cbc0 tx=0x7ff99c00a430 comp rx=0 tx=0).stop 2024-09-16T15:43:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 shutdown_connections 2024-09-16T15:43:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9aba00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:6800/2198567224,v1:172.21.15.100:6801/2198567224] conn(0x7ff984078c40 0x7ff98407b100 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9aba00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a415f5a0 0x7ff9a4157600 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9aba00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a40ccf00 0x7ff9a41f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9aba00640 1 --2- 172.21.15.60:0/882930892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a4073fb0 0x7ff9a41f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.134+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 >> 172.21.15.60:0/882930892 conn(0x7ff9a40b1950 msgr2=0x7ff9a4075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.135+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 shutdown_connections 2024-09-16T15:43:06.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.135+0000 7ff9aba00640 1 -- 172.21.15.60:0/882930892 wait complete. 2024-09-16T15:43:06.139 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-09-16T15:43:06.138+0000 7fa58fc00640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-16T15:43:06.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:06.262 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.260+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.262 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.260+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.262 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.260+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.262 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.260+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.262 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.260+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.261+0000 7fcb8c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.261+0000 7fcb8c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.261+0000 7fcb8c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.261+0000 7fcb8c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:43:06.261+0000 7fcb8c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:43:06.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.359+0000 7fd68f400640 1 -- 172.21.15.60:0/1451634033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 msgr2=0x7fd6880acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:06.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.359+0000 7fd68f400640 1 --2- 172.21.15.60:0/1451634033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 0x7fd6880acd30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd688075130 tx=0x7fd67402a580 comp rx=0 tx=0).stop 2024-09-16T15:43:06.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.359+0000 7fd68f400640 1 -- 172.21.15.60:0/1451634033 shutdown_connections 2024-09-16T15:43:06.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.359+0000 7fd68f400640 1 --2- 172.21.15.60:0/1451634033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 0x7fd6880acd30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.359+0000 7fd68f400640 1 --2- 172.21.15.60:0/1451634033 >> v2:172.21.15.100:3300/0 conn(0x7fd68815f5a0 0x7fd6880b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.359+0000 7fd68f400640 1 -- 172.21.15.60:0/1451634033 >> 172.21.15.60:0/1451634033 conn(0x7fd6880b1950 msgr2=0x7fd6880b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:06.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.359+0000 7fd68f400640 1 -- 172.21.15.60:0/1451634033 shutdown_connections 2024-09-16T15:43:06.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.359+0000 7fd68f400640 1 -- 172.21.15.60:0/1451634033 wait complete. 2024-09-16T15:43:06.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.361+0000 7fd68f400640 1 Processor -- start 2024-09-16T15:43:06.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.361+0000 7fd68f400640 1 -- start start 2024-09-16T15:43:06.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd68f400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6880ad270 0x7fd6881558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:06.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd68f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 0x7fd6881f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:06.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd68f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd68815f5a0 0x7fd6881f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:06.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd68f400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd68806f160 con 0x7fd6880ccf00 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd68f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd68806f2e0 con 0x7fd68815f5a0 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd68f400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd68806f460 con 0x7fd6880ad270 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd687e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 0x7fd6881f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd687e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 0x7fd6881f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52290/0 (socket says 172.21.15.60:52290) 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd687e00640 1 -- 172.21.15.60:0/4085763640 learned_addr learned my addr 172.21.15.60:0/4085763640 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd68ca00640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6880ad270 0x7fd6881558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd68d400640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd68815f5a0 0x7fd6881f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd687e00640 1 -- 172.21.15.60:0/4085763640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6880ad270 msgr2=0x7fd6881558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.362+0000 7fd687e00640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6880ad270 0x7fd6881558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd687e00640 1 -- 172.21.15.60:0/4085763640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd68815f5a0 msgr2=0x7fd6881f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd687e00640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd68815f5a0 0x7fd6881f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd687e00640 1 -- 172.21.15.60:0/4085763640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6881f5090 con 0x7fd6880ccf00 2024-09-16T15:43:06.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd687e00640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 0x7fd6881f4840 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd674009850 tx=0x7fd674003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:06.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd684c00640 1 -- 172.21.15.60:0/4085763640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67403f070 con 0x7fd6880ccf00 2024-09-16T15:43:06.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd684c00640 1 -- 172.21.15.60:0/4085763640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6740047a0 con 0x7fd6880ccf00 2024-09-16T15:43:06.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd684c00640 1 -- 172.21.15.60:0/4085763640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674035030 con 0x7fd6880ccf00 2024-09-16T15:43:06.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6881f5370 con 0x7fd6880ccf00 2024-09-16T15:43:06.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.363+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6880cf3d0 con 0x7fd6880ccf00 2024-09-16T15:43:06.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.364+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68806f9f0 con 0x7fd6880ccf00 2024-09-16T15:43:06.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.364+0000 7fd684c00640 1 -- 172.21.15.60:0/4085763640 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) ==== 50665+0+0 (secure 0 0 0) 0x7fd674042070 con 0x7fd6880ccf00 2024-09-16T15:43:06.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.365+0000 7fd684c00640 1 -- 172.21.15.60:0/4085763640 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6268+0+0 (secure 0 0 0) 0x7fd674074370 con 0x7fd6880ccf00 2024-09-16T15:43:06.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.370+0000 7fd684c00640 1 -- 172.21.15.60:0/4085763640 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd67403b460 con 0x7fd6880ccf00 2024-09-16T15:43:06.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.611+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd688155e30 con 0x7fd6880ccf00 2024-09-16T15:43:06.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.612+0000 7fd684c00640 1 -- 172.21.15.60:0/4085763640 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5231 (secure 0 0 0) 0x7fd6740740d0 con 0x7fd6880ccf00 2024-09-16T15:43:06.613 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:06.613 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"btime":"2024-09-16T15:43:00:402258+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:43:06.613 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-09-16T15:43:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 msgr2=0x7fd6881f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 0x7fd6881f4840 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd674009850 tx=0x7fd674003910 comp rx=0 tx=0).stop 2024-09-16T15:43:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 shutdown_connections 2024-09-16T15:43:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd68815f5a0 0x7fd6881f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6880ccf00 0x7fd6881f4840 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 --2- 172.21.15.60:0/4085763640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6880ad270 0x7fd6881558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 >> 172.21.15.60:0/4085763640 conn(0x7fd6880b1950 msgr2=0x7fd6880b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 shutdown_connections 2024-09-16T15:43:06.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:06.614+0000 7fd68f400640 1 -- 172.21.15.60:0/4085763640 wait complete. 2024-09-16T15:43:06.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-16T15:43:06.808 INFO:teuthology.orchestra.run.smithi060.stdout:[{"id":6229,"entity":{"name":{"type":"client","num":6229},"addr":{"type":"any","addr":"172.21.15.60:0","nonce":475213615}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.528994929,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9975731712212872,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6229 172.21.15.60:0/475213615","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"y","hostname":"smithi060","pid":"35878","root":"/"}},{"id":6226,"entity":{"name":{"type":"client","num":6226},"addr":{"type":"any","addr":"172.21.15.60:0","nonce":4113350793}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.52999492000000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9975731712212872,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6226 172.21.15.60:0/4113350793","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"33f95d2ae1bcbe7eb04404303f179eeeaa6982df","ceph_version":"ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)","entity_id":"y","hostname":"smithi060","pid":"35878","root":"/"}}] 2024-09-16T15:43:06.809 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35878", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.60:0", "nonce": 475213615, "type": "any" }, "name": { "num": 6229, "type": "client" } }, "id": 6229, "inst": "client.6229 172.21.15.60:0/475213615", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9975731712212872 }, "state": "open", "uptime": 0.528994929 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "33f95d2ae1bcbe7eb04404303f179eeeaa6982df", "ceph_version": "ceph version 19.3.0-4938-g33f95d2a (33f95d2ae1bcbe7eb04404303f179eeeaa6982df) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi060", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35878", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.60:0", "nonce": 4113350793, "type": "any" }, "name": { "num": 6226, "type": "client" } }, "id": 6226, "inst": "client.6226 172.21.15.60:0/4113350793", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9975731712212872 }, "state": "open", "uptime": 0.52999492 } ] 2024-09-16T15:43:06.809 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-16T15:43:06.809 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:43:06.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:43:06.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:43:06.998 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:43:06.999 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:43:06.999 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:06.999 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:07.034 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:43:07.034 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:43:07.034 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:07.034 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:07.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:43:07.070 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:43:07.070 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:07.070 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:07.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:43:07.109 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:43:07.109 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:07.109 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:07.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:43:07.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:07.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.309+0000 7f3098600640 1 -- 172.21.15.60:0/504176739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f309015f5a0 msgr2=0x7f30900b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.309+0000 7f3098600640 1 --2- 172.21.15.60:0/504176739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f309015f5a0 0x7f30900b2440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3084009830 tx=0x7f308402a530 comp rx=0 tx=0).stop 2024-09-16T15:43:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.309+0000 7f3098600640 1 -- 172.21.15.60:0/504176739 shutdown_connections 2024-09-16T15:43:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.309+0000 7f3098600640 1 --2- 172.21.15.60:0/504176739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f309015f5a0 0x7f30900b2440 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.309+0000 7f3098600640 1 -- 172.21.15.60:0/504176739 >> 172.21.15.60:0/504176739 conn(0x7f30900b1950 msgr2=0x7f30900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.309+0000 7f3098600640 1 -- 172.21.15.60:0/504176739 shutdown_connections 2024-09-16T15:43:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.309+0000 7f3098600640 1 -- 172.21.15.60:0/504176739 wait complete. 2024-09-16T15:43:07.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.311+0000 7f3098600640 1 Processor -- start 2024-09-16T15:43:07.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.311+0000 7f3098600640 1 -- start start 2024-09-16T15:43:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3098600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3090073fb0 0x7f3090150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3098600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30900ccf00 0x7f30900cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3098600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309015f5a0 0x7f30900cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3098600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f309006f9f0 con 0x7f309015f5a0 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3098600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f309006f870 con 0x7f30900ccf00 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3098600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3090074420 con 0x7f3090073fb0 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3096600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309015f5a0 0x7f30900cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3096600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309015f5a0 0x7f30900cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52292/0 (socket says 172.21.15.60:52292) 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3096600640 1 -- 172.21.15.60:0/3349496509 learned_addr learned my addr 172.21.15.60:0/3349496509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3095200640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30900ccf00 0x7f30900cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.312+0000 7f3095c00640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3090073fb0 0x7f3090150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f3096600640 1 -- 172.21.15.60:0/3349496509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3090073fb0 msgr2=0x7f3090150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f3096600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3090073fb0 0x7f3090150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f3096600640 1 -- 172.21.15.60:0/3349496509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30900ccf00 msgr2=0x7f30900cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f3096600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30900ccf00 0x7f30900cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f3096600640 1 -- 172.21.15.60:0/3349496509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30901540c0 con 0x7f309015f5a0 2024-09-16T15:43:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f3096600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309015f5a0 0x7f30900cf1a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f308800c770 tx=0x7f308800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:07.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f308d600640 1 -- 172.21.15.60:0/3349496509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088003da0 con 0x7f309015f5a0 2024-09-16T15:43:07.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f308d600640 1 -- 172.21.15.60:0/3349496509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3088005070 con 0x7f309015f5a0 2024-09-16T15:43:07.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f308d600640 1 -- 172.21.15.60:0/3349496509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088005500 con 0x7f309015f5a0 2024-09-16T15:43:07.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30901f7c50 con 0x7f309015f5a0 2024-09-16T15:43:07.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.313+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30901f8210 con 0x7f309015f5a0 2024-09-16T15:43:07.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.314+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30900b2440 con 0x7f309015f5a0 2024-09-16T15:43:07.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.314+0000 7f308d600640 1 -- 172.21.15.60:0/3349496509 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 14) ==== 51000+0+0 (secure 0 0 0) 0x7f30880056a0 con 0x7f309015f5a0 2024-09-16T15:43:07.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.315+0000 7f308d600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f307003e710 0x7f3070040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:07.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.315+0000 7f308d600640 1 -- 172.21.15.60:0/3349496509 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6268+0+0 (secure 0 0 0) 0x7f30880538e0 con 0x7f309015f5a0 2024-09-16T15:43:07.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.315+0000 7f3095c00640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f307003e710 0x7f3070040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:07.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.316+0000 7f3095c00640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f307003e710 0x7f3070040bd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3084034000 tx=0x7f3084004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:07.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.320+0000 7f308d600640 1 -- 172.21.15.60:0/3349496509 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f308801ac60 con 0x7f309015f5a0 2024-09-16T15:43:07.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.552+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30900b2070 con 0x7f309015f5a0 2024-09-16T15:43:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.553+0000 7f308d600640 1 -- 172.21.15.60:0/3349496509 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5231 (secure 0 0 0) 0x7f3088010d10 con 0x7f309015f5a0 2024-09-16T15:43:07.554 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:07.554 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"btime":"2024-09-16T15:43:00:402258+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:43:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f307003e710 msgr2=0x7f3070040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f307003e710 0x7f3070040bd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3084034000 tx=0x7f3084004730 comp rx=0 tx=0).stop 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309015f5a0 msgr2=0x7f30900cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309015f5a0 0x7f30900cf1a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f308800c770 tx=0x7f308800cc30 comp rx=0 tx=0).stop 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 shutdown_connections 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f307003e710 0x7f3070040bd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f309015f5a0 0x7f30900cf1a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30900ccf00 0x7f30900cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 --2- 172.21.15.60:0/3349496509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3090073fb0 0x7f3090150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 >> 172.21.15.60:0/3349496509 conn(0x7f30900b1950 msgr2=0x7f30900af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:07.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.555+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 shutdown_connections 2024-09-16T15:43:07.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.556+0000 7f3098600640 1 -- 172.21.15.60:0/3349496509 wait complete. 2024-09-16T15:43:07.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:07.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.732+0000 7f5aec000640 1 -- 172.21.15.60:0/2033585807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5acc005280 msgr2=0x7f5acc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.732+0000 7f5aec000640 1 --2- 172.21.15.60:0/2033585807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5acc005280 0x7f5acc00b9c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001f30 tx=0x7f5acc005830 comp rx=0 tx=0).stop 2024-09-16T15:43:07.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.732+0000 7f5aec000640 1 -- 172.21.15.60:0/2033585807 shutdown_connections 2024-09-16T15:43:07.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.732+0000 7f5aec000640 1 --2- 172.21.15.60:0/2033585807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5acc005280 0x7f5acc00b9c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.732+0000 7f5aec000640 1 -- 172.21.15.60:0/2033585807 >> 172.21.15.60:0/2033585807 conn(0x7f5ae40b1950 msgr2=0x7f5ae40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:07.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.732+0000 7f5aec000640 1 -- 172.21.15.60:0/2033585807 shutdown_connections 2024-09-16T15:43:07.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.732+0000 7f5aec000640 1 -- 172.21.15.60:0/2033585807 wait complete. 2024-09-16T15:43:07.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.733+0000 7f5aec000640 1 Processor -- start 2024-09-16T15:43:07.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.734+0000 7f5aec000640 1 -- start start 2024-09-16T15:43:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.734+0000 7f5aec000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5acc005280 0x7f5ae414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.734+0000 7f5aec000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae40cee60 0x7f5ae41601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aec000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4160730 0x7f5ae40cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aec000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae406f2e0 con 0x7f5ae4160730 2024-09-16T15:43:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aec000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae406f160 con 0x7f5acc005280 2024-09-16T15:43:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aec000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae406f460 con 0x7f5ae40cee60 2024-09-16T15:43:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aea000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4160730 0x7f5ae40cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:07.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5ae9600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5acc005280 0x7f5ae414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aea000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4160730 0x7f5ae40cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52320/0 (socket says 172.21.15.60:52320) 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aea000640 1 -- 172.21.15.60:0/4246102200 learned_addr learned my addr 172.21.15.60:0/4246102200 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5ae8c00640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae40cee60 0x7f5ae41601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aea000640 1 -- 172.21.15.60:0/4246102200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae40cee60 msgr2=0x7f5ae41601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aea000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae40cee60 0x7f5ae41601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aea000640 1 -- 172.21.15.60:0/4246102200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5acc005280 msgr2=0x7f5ae414d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aea000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5acc005280 0x7f5ae414d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5aea000640 1 -- 172.21.15.60:0/4246102200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae40cdae0 con 0x7f5ae4160730 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.735+0000 7f5ae9600640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5acc005280 0x7f5ae414d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.736+0000 7f5aea000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4160730 0x7f5ae40cfc80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5adc02ade0 tx=0x7f5adc001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:07.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.736+0000 7f5ae1600640 1 -- 172.21.15.60:0/4246102200 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc003d80 con 0x7f5ae4160730 2024-09-16T15:43:07.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.736+0000 7f5ae1600640 1 -- 172.21.15.60:0/4246102200 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5adc033560 con 0x7f5ae4160730 2024-09-16T15:43:07.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.736+0000 7f5ae1600640 1 -- 172.21.15.60:0/4246102200 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0339b0 con 0x7f5ae4160730 2024-09-16T15:43:07.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.736+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae40cddc0 con 0x7f5ae4160730 2024-09-16T15:43:07.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.736+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae40ce380 con 0x7f5ae4160730 2024-09-16T15:43:07.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.737+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae4160bb0 con 0x7f5ae4160730 2024-09-16T15:43:07.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.737+0000 7f5ae1600640 1 -- 172.21.15.60:0/4246102200 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 14) ==== 51000+0+0 (secure 0 0 0) 0x7f5adc033b50 con 0x7f5ae4160730 2024-09-16T15:43:07.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.738+0000 7f5ae1600640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ac403e710 0x7f5ac4040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:07.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.738+0000 7f5ae1600640 1 -- 172.21.15.60:0/4246102200 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6268+0+0 (secure 0 0 0) 0x7f5adc073750 con 0x7f5ae4160730 2024-09-16T15:43:07.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.738+0000 7f5ae9600640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ac403e710 0x7f5ac4040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:07.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.739+0000 7f5ae9600640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ac403e710 0x7f5ac4040bd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009fc0 tx=0x7f5ad8005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:07.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.742+0000 7f5ae1600640 1 -- 172.21.15.60:0/4246102200 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5adc045020 con 0x7f5ae4160730 2024-09-16T15:43:07.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.976+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ae415fcd0 con 0x7f5ae4160730 2024-09-16T15:43:07.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.977+0000 7f5ae1600640 1 -- 172.21.15.60:0/4246102200 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5231 (secure 0 0 0) 0x7f5adc006260 con 0x7f5ae4160730 2024-09-16T15:43:07.978 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:07.978 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"btime":"2024-09-16T15:43:00:402258+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:43:07.978 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ac403e710 msgr2=0x7f5ac4040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ac403e710 0x7f5ac4040bd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009fc0 tx=0x7f5ad8005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4160730 msgr2=0x7f5ae40cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4160730 0x7f5ae40cfc80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5adc02ade0 tx=0x7f5adc001c70 comp rx=0 tx=0).stop 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 shutdown_connections 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ac403e710 0x7f5ac4040bd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4160730 0x7f5ae40cfc80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ae40cee60 0x7f5ae41601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 --2- 172.21.15.60:0/4246102200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5acc005280 0x7f5ae414d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 >> 172.21.15.60:0/4246102200 conn(0x7f5ae40b1950 msgr2=0x7f5ae40ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.979+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 shutdown_connections 2024-09-16T15:43:07.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:07.980+0000 7f5aec000640 1 -- 172.21.15.60:0/4246102200 wait complete. 2024-09-16T15:43:07.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.155+0000 7fbc7a800640 1 -- 172.21.15.60:0/2763608168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7415bc30 msgr2=0x7fbc740b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.155+0000 7fbc7a800640 1 --2- 172.21.15.60:0/2763608168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7415bc30 0x7fbc740b28f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009830 tx=0x7fbc6802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.156+0000 7fbc7a800640 1 -- 172.21.15.60:0/2763608168 shutdown_connections 2024-09-16T15:43:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.156+0000 7fbc7a800640 1 --2- 172.21.15.60:0/2763608168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc7415bc30 0x7fbc740b28f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.156+0000 7fbc7a800640 1 -- 172.21.15.60:0/2763608168 >> 172.21.15.60:0/2763608168 conn(0x7fbc740cfa20 msgr2=0x7fbc740cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.156+0000 7fbc7a800640 1 -- 172.21.15.60:0/2763608168 shutdown_connections 2024-09-16T15:43:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.156+0000 7fbc7a800640 1 -- 172.21.15.60:0/2763608168 wait complete. 2024-09-16T15:43:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.157+0000 7fbc7a800640 1 Processor -- start 2024-09-16T15:43:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.157+0000 7fbc7a800640 1 -- start start 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc7a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc740d0540 0x7fbc741f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc7a800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc7415bc30 0x7fbc740b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc7a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc740b4ff0 0x7fbc74147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc7a800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbc74074660 con 0x7fbc740d0540 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc7a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7406fa30 con 0x7fbc740b4ff0 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc7a800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbc74148980 con 0x7fbc7415bc30 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc79400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc740d0540 0x7fbc741f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc79400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc740d0540 0x7fbc741f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52322/0 (socket says 172.21.15.60:52322) 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.158+0000 7fbc79400640 1 -- 172.21.15.60:0/4156775757 learned_addr learned my addr 172.21.15.60:0/4156775757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:08.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.159+0000 7fbc79e00640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc740b4ff0 0x7fbc74147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.159+0000 7fbc78a00640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc7415bc30 0x7fbc740b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.159+0000 7fbc79400640 1 -- 172.21.15.60:0/4156775757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc7415bc30 msgr2=0x7fbc740b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.159+0000 7fbc79400640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc7415bc30 0x7fbc740b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.159+0000 7fbc79400640 1 -- 172.21.15.60:0/4156775757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc740b4ff0 msgr2=0x7fbc74147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.159+0000 7fbc79400640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc740b4ff0 0x7fbc74147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.159+0000 7fbc79400640 1 -- 172.21.15.60:0/4156775757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc74145aa0 con 0x7fbc740d0540 2024-09-16T15:43:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.159+0000 7fbc79400640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc740d0540 0x7fbc741f1d30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009780 tx=0x7fbc68033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.160+0000 7fbc71600640 1 -- 172.21.15.60:0/4156775757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68005c90 con 0x7fbc740d0540 2024-09-16T15:43:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.160+0000 7fbc71600640 1 -- 172.21.15.60:0/4156775757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc68031e50 con 0x7fbc740d0540 2024-09-16T15:43:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.160+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc74145d80 con 0x7fbc740d0540 2024-09-16T15:43:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.160+0000 7fbc71600640 1 -- 172.21.15.60:0/4156775757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68005280 con 0x7fbc740d0540 2024-09-16T15:43:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.160+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc74155530 con 0x7fbc740d0540 2024-09-16T15:43:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.161+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc740b5400 con 0x7fbc740d0540 2024-09-16T15:43:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.161+0000 7fbc71600640 1 -- 172.21.15.60:0/4156775757 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 14) ==== 51000+0+0 (secure 0 0 0) 0x7fbc68005420 con 0x7fbc740d0540 2024-09-16T15:43:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.161+0000 7fbc71600640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbc5403e710 0x7fbc54040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.162+0000 7fbc71600640 1 -- 172.21.15.60:0/4156775757 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6268+0+0 (secure 0 0 0) 0x7fbc68074940 con 0x7fbc740d0540 2024-09-16T15:43:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.162+0000 7fbc78a00640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbc5403e710 0x7fbc54040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.162+0000 7fbc78a00640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbc5403e710 0x7fbc54040bd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc60007fd0 tx=0x7fbc60009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:08.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.166+0000 7fbc71600640 1 -- 172.21.15.60:0/4156775757 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbc740b5400 con 0x7fbc740d0540 2024-09-16T15:43:08.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.363+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc740b28f0 con 0x7fbc740d0540 2024-09-16T15:43:08.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.364+0000 7fbc71600640 1 -- 172.21.15.60:0/4156775757 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17550 (secure 0 0 0) 0x7fbc6803f030 con 0x7fbc740d0540 2024-09-16T15:43:08.365 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:08.365 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:06.127970+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:47.898202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:48.609707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/2997397432":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/883853961":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/2410226558":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4072261497":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4177407071":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4216665037":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/1852740290":"2024-09-17T15:43:06.127928+0000","172.21.15.100:6801/2198567224":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/2641588278":"2024-09-17T15:43:06.127928+0000","172.21.15.100:6800/2198567224":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/1587892015":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/3427025651":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/690787621":"2024-09-17T15:43:06.127928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:08.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbc5403e710 msgr2=0x7fbc54040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbc5403e710 0x7fbc54040bd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc60007fd0 tx=0x7fbc60009040 comp rx=0 tx=0).stop 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc740d0540 msgr2=0x7fbc741f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc740d0540 0x7fbc741f1d30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009780 tx=0x7fbc68033c20 comp rx=0 tx=0).stop 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 shutdown_connections 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbc5403e710 0x7fbc54040bd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc740b4ff0 0x7fbc74147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc7415bc30 0x7fbc740b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 --2- 172.21.15.60:0/4156775757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc740d0540 0x7fbc741f1d30 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 >> 172.21.15.60:0/4156775757 conn(0x7fbc740cfa20 msgr2=0x7fbc74151680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 shutdown_connections 2024-09-16T15:43:08.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.366+0000 7fbc7a800640 1 -- 172.21.15.60:0/4156775757 wait complete. 2024-09-16T15:43:08.383 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:43:08.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:08.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.543+0000 7fd33c200640 1 -- 172.21.15.60:0/75426826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 msgr2=0x7fd3340acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.543+0000 7fd33c200640 1 --2- 172.21.15.60:0/75426826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 0x7fd3340acd30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd334075130 tx=0x7fd32002a560 comp rx=0 tx=0).stop 2024-09-16T15:43:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.543+0000 7fd33c200640 1 -- 172.21.15.60:0/75426826 shutdown_connections 2024-09-16T15:43:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.543+0000 7fd33c200640 1 --2- 172.21.15.60:0/75426826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 0x7fd3340acd30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.543+0000 7fd33c200640 1 --2- 172.21.15.60:0/75426826 >> v2:172.21.15.100:3300/0 conn(0x7fd33415f5a0 0x7fd3340b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.543+0000 7fd33c200640 1 -- 172.21.15.60:0/75426826 >> 172.21.15.60:0/75426826 conn(0x7fd3340b1950 msgr2=0x7fd3340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.543+0000 7fd33c200640 1 -- 172.21.15.60:0/75426826 shutdown_connections 2024-09-16T15:43:08.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.543+0000 7fd33c200640 1 -- 172.21.15.60:0/75426826 wait complete. 2024-09-16T15:43:08.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.545+0000 7fd33c200640 1 Processor -- start 2024-09-16T15:43:08.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.545+0000 7fd33c200640 1 -- start start 2024-09-16T15:43:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd33c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3340ad270 0x7fd3341f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd33c200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 0x7fd33414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd33c200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33415f5a0 0x7fd334153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd33c200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd33406f160 con 0x7fd3340ccf00 2024-09-16T15:43:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd33c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd33406f2e0 con 0x7fd3340ad270 2024-09-16T15:43:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd33c200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd33406f460 con 0x7fd33415f5a0 2024-09-16T15:43:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd338e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 0x7fd33414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd338e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 0x7fd33414e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52370/0 (socket says 172.21.15.60:52370) 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd338e00640 1 -- 172.21.15.60:0/3544782692 learned_addr learned my addr 172.21.15.60:0/3544782692 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd339800640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3340ad270 0x7fd3341f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.546+0000 7fd33a200640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33415f5a0 0x7fd334153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd338e00640 1 -- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33415f5a0 msgr2=0x7fd334153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd338e00640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33415f5a0 0x7fd334153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd338e00640 1 -- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3340ad270 msgr2=0x7fd3341f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd338e00640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3340ad270 0x7fd3341f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd338e00640 1 -- 172.21.15.60:0/3544782692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3341f5290 con 0x7fd3340ccf00 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd33a200640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33415f5a0 0x7fd334153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:08.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd338e00640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 0x7fd33414e670 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd320009850 tx=0x7fd320003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd331600640 1 -- 172.21.15.60:0/3544782692 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32003f070 con 0x7fd3340ccf00 2024-09-16T15:43:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd331600640 1 -- 172.21.15.60:0/3544782692 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3200047a0 con 0x7fd3340ccf00 2024-09-16T15:43:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd331600640 1 -- 172.21.15.60:0/3544782692 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320035030 con 0x7fd3340ccf00 2024-09-16T15:43:08.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.547+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3341f5570 con 0x7fd3340ccf00 2024-09-16T15:43:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.548+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3341f5be0 con 0x7fd3340ccf00 2024-09-16T15:43:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.548+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd33406f9f0 con 0x7fd3340ccf00 2024-09-16T15:43:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.549+0000 7fd331600640 1 -- 172.21.15.60:0/3544782692 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 14) ==== 51000+0+0 (secure 0 0 0) 0x7fd320042070 con 0x7fd3340ccf00 2024-09-16T15:43:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.549+0000 7fd331600640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd31403e710 0x7fd314040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.549+0000 7fd331600640 1 -- 172.21.15.60:0/3544782692 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6268+0+0 (secure 0 0 0) 0x7fd3200749b0 con 0x7fd3340ccf00 2024-09-16T15:43:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.549+0000 7fd339800640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd31403e710 0x7fd314040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.550+0000 7fd339800640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd31403e710 0x7fd314040bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd32800bb50 tx=0x7fd32800b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:08.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.554+0000 7fd331600640 1 -- 172.21.15.60:0/3544782692 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd32003b460 con 0x7fd3340ccf00 2024-09-16T15:43:08.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.787+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd33415e040 con 0x7fd3340ccf00 2024-09-16T15:43:08.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.788+0000 7fd331600640 1 -- 172.21.15.60:0/3544782692 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5231 (secure 0 0 0) 0x7fd320042360 con 0x7fd3340ccf00 2024-09-16T15:43:08.789 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:08.789 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"btime":"2024-09-16T15:43:00:402258+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:43:08.790 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd31403e710 msgr2=0x7fd314040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd31403e710 0x7fd314040bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd32800bb50 tx=0x7fd32800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 msgr2=0x7fd33414e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 0x7fd33414e670 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd320009850 tx=0x7fd320003910 comp rx=0 tx=0).stop 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 shutdown_connections 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd31403e710 0x7fd314040bd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd33415f5a0 0x7fd334153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3340ccf00 0x7fd33414e670 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 --2- 172.21.15.60:0/3544782692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3340ad270 0x7fd3341f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.790+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 >> 172.21.15.60:0/3544782692 conn(0x7fd3340b1950 msgr2=0x7fd33415d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:08.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.791+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 shutdown_connections 2024-09-16T15:43:08.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.791+0000 7fd33c200640 1 -- 172.21.15.60:0/3544782692 wait complete. 2024-09-16T15:43:08.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.967+0000 7fd1c7e00640 1 -- 172.21.15.60:0/2082380792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c80b5370 msgr2=0x7fd1c80b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.967+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/2082380792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c80b5370 0x7fd1c80b3bc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005fd0 tx=0x7fd1b402a540 comp rx=0 tx=0).stop 2024-09-16T15:43:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.967+0000 7fd1c7e00640 1 -- 172.21.15.60:0/2082380792 shutdown_connections 2024-09-16T15:43:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.967+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/2082380792 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c80b5370 0x7fd1c80b3bc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.967+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/2082380792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1c80b4100 0x7fd1c80b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.967+0000 7fd1c7e00640 1 -- 172.21.15.60:0/2082380792 >> 172.21.15.60:0/2082380792 conn(0x7fd1c80d3df0 msgr2=0x7fd1c80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.967+0000 7fd1c7e00640 1 -- 172.21.15.60:0/2082380792 shutdown_connections 2024-09-16T15:43:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.967+0000 7fd1c7e00640 1 -- 172.21.15.60:0/2082380792 wait complete. 2024-09-16T15:43:08.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.968+0000 7fd1c7e00640 1 Processor -- start 2024-09-16T15:43:08.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.968+0000 7fd1c7e00640 1 -- start start 2024-09-16T15:43:08.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.969+0000 7fd1c7e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1c80b4100 0x7fd1c80caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.969+0000 7fd1c7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c80b5370 0x7fd1c80ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.969+0000 7fd1c7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c8161440 0x7fd1c80b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.969+0000 7fd1c7e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80b2040 con 0x7fd1c8161440 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.969+0000 7fd1c7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c80b1ec0 con 0x7fd1c80b5370 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.969+0000 7fd1c7e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c80b21c0 con 0x7fd1c80b4100 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c8161440 0x7fd1c80b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c6a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1c80b4100 0x7fd1c80caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c80b5370 0x7fd1c80ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c8161440 0x7fd1c80b8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42136/0 (socket says 172.21.15.60:42136) 2024-09-16T15:43:08.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 -- 172.21.15.60:0/4209604667 learned_addr learned my addr 172.21.15.60:0/4209604667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:08.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 -- 172.21.15.60:0/4209604667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1c80b4100 msgr2=0x7fd1c80caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1c80b4100 0x7fd1c80caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 -- 172.21.15.60:0/4209604667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c80b5370 msgr2=0x7fd1c80ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:08.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c80b5370 0x7fd1c80ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:08.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 -- 172.21.15.60:0/4209604667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c81f4ed0 con 0x7fd1c8161440 2024-09-16T15:43:08.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.970+0000 7fd1c7400640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c8161440 0x7fd1c80b8df0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000eb40 tx=0x7fd1c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:08.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.971+0000 7fd1bf400640 1 -- 172.21.15.60:0/4209604667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0004b20 con 0x7fd1c8161440 2024-09-16T15:43:08.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.971+0000 7fd1bf400640 1 -- 172.21.15.60:0/4209604667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1c0004ce0 con 0x7fd1c8161440 2024-09-16T15:43:08.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.971+0000 7fd1bf400640 1 -- 172.21.15.60:0/4209604667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0005280 con 0x7fd1c8161440 2024-09-16T15:43:08.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.971+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c81f5170 con 0x7fd1c8161440 2024-09-16T15:43:08.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.971+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c815f580 con 0x7fd1c8161440 2024-09-16T15:43:08.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.972+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c80b1770 con 0x7fd1c8161440 2024-09-16T15:43:08.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.972+0000 7fd1bf400640 1 -- 172.21.15.60:0/4209604667 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 14) ==== 51000+0+0 (secure 0 0 0) 0x7fd1c0005420 con 0x7fd1c8161440 2024-09-16T15:43:08.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.972+0000 7fd1bf400640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1a803e710 0x7fd1a8040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:08.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.972+0000 7fd1bf400640 1 -- 172.21.15.60:0/4209604667 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6268+0+0 (secure 0 0 0) 0x7fd1c00547c0 con 0x7fd1c8161440 2024-09-16T15:43:08.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.973+0000 7fd1c6a00640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1a803e710 0x7fd1a8040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:08.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.973+0000 7fd1c6a00640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1a803e710 0x7fd1a8040bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1c806d6d0 tx=0x7fd1b8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:08.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:08.977+0000 7fd1bf400640 1 -- 172.21.15.60:0/4209604667 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd1c00058f0 con 0x7fd1c8161440 2024-09-16T15:43:09.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.212+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1c81578c0 con 0x7fd1c8161440 2024-09-16T15:43:09.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.213+0000 7fd1bf400640 1 -- 172.21.15.60:0/4209604667 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5231 (secure 0 0 0) 0x7fd1c0021cf0 con 0x7fd1c8161440 2024-09-16T15:43:09.214 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:09.214 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"btime":"2024-09-16T15:43:00:402258+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:49.653917+0000","modified":"2024-09-16T15:42:52.679067+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1266112466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1266112466},{"type":"v1","addr":"172.21.15.60:6835","nonce":1266112466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-16T15:43:09.214 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.215+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1a803e710 msgr2=0x7fd1a8040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.215+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1a803e710 0x7fd1a8040bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1c806d6d0 tx=0x7fd1b8009040 comp rx=0 tx=0).stop 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.215+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c8161440 msgr2=0x7fd1c80b8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.215+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c8161440 0x7fd1c80b8df0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000eb40 tx=0x7fd1c000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.216+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 shutdown_connections 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.216+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1a803e710 0x7fd1a8040bd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.216+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1c8161440 0x7fd1c80b8df0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.216+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1c80b5370 0x7fd1c80ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.216+0000 7fd1c7e00640 1 --2- 172.21.15.60:0/4209604667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1c80b4100 0x7fd1c80caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.216+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 >> 172.21.15.60:0/4209604667 conn(0x7fd1c80d3df0 msgr2=0x7fd1c80d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.216+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 shutdown_connections 2024-09-16T15:43:09.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.216+0000 7fd1c7e00640 1 -- 172.21.15.60:0/4209604667 wait complete. 2024-09-16T15:43:09.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:09.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.391+0000 7f44d6000640 1 -- 172.21.15.60:0/3585783896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 msgr2=0x7f44d006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.391+0000 7f44d6000640 1 --2- 172.21.15.60:0/3585783896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 0x7f44d006f160 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f44c8005fd0 tx=0x7f44c802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.392+0000 7f44d6000640 1 -- 172.21.15.60:0/3585783896 shutdown_connections 2024-09-16T15:43:09.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.392+0000 7f44d6000640 1 --2- 172.21.15.60:0/3585783896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 0x7f44d006f160 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.392+0000 7f44d6000640 1 -- 172.21.15.60:0/3585783896 >> 172.21.15.60:0/3585783896 conn(0x7f44d00b1950 msgr2=0x7f44d00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:09.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.392+0000 7f44d6000640 1 -- 172.21.15.60:0/3585783896 shutdown_connections 2024-09-16T15:43:09.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.392+0000 7f44d6000640 1 -- 172.21.15.60:0/3585783896 wait complete. 2024-09-16T15:43:09.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.393+0000 7f44d6000640 1 Processor -- start 2024-09-16T15:43:09.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.393+0000 7f44d6000640 1 -- start start 2024-09-16T15:43:09.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.394+0000 7f44d6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 0x7f44d00b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:09.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.394+0000 7f44d6000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44d00ccf00 0x7f44d00ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.394+0000 7f44d6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d015f5a0 0x7f44d00cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.394+0000 7f44d6000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f44d0074840 con 0x7f44d00b2440 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.394+0000 7f44d6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f44d00746c0 con 0x7f44d015f5a0 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.394+0000 7f44d6000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f44d006f6a0 con 0x7f44d00ccf00 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.394+0000 7f44cf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 0x7f44d00b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 0x7f44d00b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42162/0 (socket says 172.21.15.60:42162) 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cf400640 1 -- 172.21.15.60:0/78211858 learned_addr learned my addr 172.21.15.60:0/78211858 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cfe00640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d015f5a0 0x7f44d00cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44c6a00640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44d00ccf00 0x7f44d00ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cf400640 1 -- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44d00ccf00 msgr2=0x7f44d00ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cf400640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44d00ccf00 0x7f44d00ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cf400640 1 -- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d015f5a0 msgr2=0x7f44d00cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cf400640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d015f5a0 0x7f44d00cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cf400640 1 -- 172.21.15.60:0/78211858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d00c9170 con 0x7f44d00b2440 2024-09-16T15:43:09.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44c6a00640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44d00ccf00 0x7f44d00ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:09.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44cf400640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 0x7f44d00b3cb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f44c000c370 tx=0x7f44c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:09.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44c7e00640 1 -- 172.21.15.60:0/78211858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0017070 con 0x7f44d00b2440 2024-09-16T15:43:09.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44c7e00640 1 -- 172.21.15.60:0/78211858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44c00035f0 con 0x7f44d00b2440 2024-09-16T15:43:09.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44c7e00640 1 -- 172.21.15.60:0/78211858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0003a40 con 0x7f44d00b2440 2024-09-16T15:43:09.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.395+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d00c9410 con 0x7f44d00b2440 2024-09-16T15:43:09.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.396+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d00c99d0 con 0x7f44d00b2440 2024-09-16T15:43:09.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.397+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d006f160 con 0x7f44d00b2440 2024-09-16T15:43:09.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.397+0000 7f44c7e00640 1 -- 172.21.15.60:0/78211858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 14) ==== 51000+0+0 (secure 0 0 0) 0x7f44c001a070 con 0x7f44d00b2440 2024-09-16T15:43:09.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.397+0000 7f44c7e00640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44ac03e710 0x7f44ac040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:09.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.397+0000 7f44c7e00640 1 -- 172.21.15.60:0/78211858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6268+0+0 (secure 0 0 0) 0x7f44c0052c90 con 0x7f44d00b2440 2024-09-16T15:43:09.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.398+0000 7f44c6a00640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44ac03e710 0x7f44ac040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:09.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.398+0000 7f44c6a00640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44ac03e710 0x7f44ac040bd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f44b8007fd0 tx=0x7f44b8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:09.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.402+0000 7f44c7e00640 1 -- 172.21.15.60:0/78211858 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f44c0003dc0 con 0x7f44d00b2440 2024-09-16T15:43:09.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.446+0000 7f44c7e00640 1 -- 172.21.15.60:0/78211858 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 15) ==== 51056+0+0 (secure 0 0 0) 0x7f44c008a090 con 0x7f44d00b2440 2024-09-16T15:43:09.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.607+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44d00b1f80 con 0x7f44d00b2440 2024-09-16T15:43:09.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.608+0000 7f44c7e00640 1 -- 172.21.15.60:0/78211858 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+17550 (secure 0 0 0) 0x7f44c0015ba0 con 0x7f44d00b2440 2024-09-16T15:43:09.609 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:09.609 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":75,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:06.127970+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:47.898202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:48.609707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/2997397432":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/883853961":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/2410226558":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4072261497":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4177407071":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4216665037":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/1852740290":"2024-09-17T15:43:06.127928+0000","172.21.15.100:6801/2198567224":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/2641588278":"2024-09-17T15:43:06.127928+0000","172.21.15.100:6800/2198567224":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/1587892015":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/3427025651":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/690787621":"2024-09-17T15:43:06.127928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:09.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44ac03e710 msgr2=0x7f44ac040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44ac03e710 0x7f44ac040bd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f44b8007fd0 tx=0x7f44b8009040 comp rx=0 tx=0).stop 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 msgr2=0x7f44d00b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 0x7f44d00b3cb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f44c000c370 tx=0x7f44c000c830 comp rx=0 tx=0).stop 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 shutdown_connections 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44ac03e710 0x7f44ac040bd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44d015f5a0 0x7f44d00cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44d00ccf00 0x7f44d00ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 --2- 172.21.15.60:0/78211858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44d00b2440 0x7f44d00b3cb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 >> 172.21.15.60:0/78211858 conn(0x7f44d00b1950 msgr2=0x7f44d00ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 shutdown_connections 2024-09-16T15:43:09.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.610+0000 7f44d6000640 1 -- 172.21.15.60:0/78211858 wait complete. 2024-09-16T15:43:09.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:43:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.788+0000 7f8a5c000640 1 -- 172.21.15.60:0/349937506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a54070580 msgr2=0x7f8a54151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.788+0000 7f8a5c000640 1 --2- 172.21.15.60:0/349937506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a54070580 0x7f8a54151e60 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c005fd0 tx=0x7f8a4c02ab30 comp rx=0 tx=0).stop 2024-09-16T15:43:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.788+0000 7f8a5c000640 1 -- 172.21.15.60:0/349937506 shutdown_connections 2024-09-16T15:43:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.788+0000 7f8a5c000640 1 --2- 172.21.15.60:0/349937506 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a54070580 0x7f8a54151e60 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.788+0000 7f8a5c000640 1 -- 172.21.15.60:0/349937506 >> 172.21.15.60:0/349937506 conn(0x7f8a540b2230 msgr2=0x7f8a540b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:09.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.788+0000 7f8a5c000640 1 -- 172.21.15.60:0/349937506 shutdown_connections 2024-09-16T15:43:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.788+0000 7f8a5c000640 1 -- 172.21.15.60:0/349937506 wait complete. 2024-09-16T15:43:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.789+0000 7f8a5c000640 1 Processor -- start 2024-09-16T15:43:09.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.789+0000 7f8a5c000640 1 -- start start 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.790+0000 7f8a5c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a54070580 0x7f8a540ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.790+0000 7f8a5c000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a540730c0 0x7f8a540c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.790+0000 7f8a5c000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a5415bdc0 0x7f8a540c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.790+0000 7f8a5c000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5415c320 con 0x7f8a540730c0 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.790+0000 7f8a5c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5415c1a0 con 0x7f8a54070580 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.790+0000 7f8a5c000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8a54073530 con 0x7f8a5415bdc0 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a540730c0 0x7f8a540c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a540730c0 0x7f8a540c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42190/0 (socket says 172.21.15.60:42190) 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 -- 172.21.15.60:0/2461847746 learned_addr learned my addr 172.21.15.60:0/2461847746 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a59600640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a54070580 0x7f8a540ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:09.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a5a000640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a5415bdc0 0x7f8a540c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:09.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 -- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a5415bdc0 msgr2=0x7f8a540c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a5415bdc0 0x7f8a540c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 -- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a54070580 msgr2=0x7f8a540ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a54070580 0x7f8a540ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 -- 172.21.15.60:0/2461847746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a540ab510 con 0x7f8a540730c0 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a5a000640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a5415bdc0 0x7f8a540c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.791+0000 7f8a58c00640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a540730c0 0x7f8a540c54a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000c500 tx=0x7f8a4000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.792+0000 7f8a51600640 1 -- 172.21.15.60:0/2461847746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400052d0 con 0x7f8a540730c0 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.792+0000 7f8a51600640 1 -- 172.21.15.60:0/2461847746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a40005470 con 0x7f8a540730c0 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.792+0000 7f8a51600640 1 -- 172.21.15.60:0/2461847746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40012870 con 0x7f8a540730c0 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.792+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a540ab7b0 con 0x7f8a540730c0 2024-09-16T15:43:09.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.792+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a540abd70 con 0x7f8a540730c0 2024-09-16T15:43:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.793+0000 7f8a51600640 1 -- 172.21.15.60:0/2461847746 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 15) ==== 51056+0+0 (secure 0 0 0) 0x7f8a400035f0 con 0x7f8a540730c0 2024-09-16T15:43:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.793+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a54073990 con 0x7f8a540730c0 2024-09-16T15:43:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.794+0000 7f8a51600640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8a3403e7a0 0x7f8a34040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.794+0000 7f8a51600640 1 -- 172.21.15.60:0/2461847746 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6268+0+0 (secure 0 0 0) 0x7f8a400535a0 con 0x7f8a540730c0 2024-09-16T15:43:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.794+0000 7f8a59600640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8a3403e7a0 0x7f8a34040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.794+0000 7f8a59600640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8a3403e7a0 0x7f8a34040c60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a48001740 tx=0x7f8a48009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:09.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:09.798+0000 7f8a51600640 1 -- 172.21.15.60:0/2461847746 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8a4001b400 con 0x7f8a540730c0 2024-09-16T15:43:10.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.034+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8a540ac200 con 0x7f8a540730c0 2024-09-16T15:43:10.036 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:43:10.035+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:43:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.037+0000 7f8a51600640 1 -- 172.21.15.60:0/2461847746 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7f8a4001b780 con 0x7f8a540730c0 2024-09-16T15:43:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:43:10.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8a3403e7a0 msgr2=0x7f8a34040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:10.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8a3403e7a0 0x7f8a34040c60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a48001740 tx=0x7f8a48009040 comp rx=0 tx=0).stop 2024-09-16T15:43:10.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a540730c0 msgr2=0x7f8a540c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:10.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a540730c0 0x7f8a540c54a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000c500 tx=0x7f8a4000c9c0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 shutdown_connections 2024-09-16T15:43:10.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8a3403e7a0 0x7f8a34040c60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a5415bdc0 0x7f8a540c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a540730c0 0x7f8a540c54a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 --2- 172.21.15.60:0/2461847746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a54070580 0x7f8a540ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 >> 172.21.15.60:0/2461847746 conn(0x7f8a540b2230 msgr2=0x7f8a54070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:10.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 shutdown_connections 2024-09-16T15:43:10.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.040+0000 7f8a5c000640 1 -- 172.21.15.60:0/2461847746 wait complete. 2024-09-16T15:43:10.052 INFO:tasks.ceph.mds.a.smithi060.stderr: -8762> 2024-09-16T15:42:55.181+0000 7f5565000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:43:10.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:43:10.108 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:43:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.220+0000 7fcfa3c00640 1 -- 172.21.15.60:0/3408856536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 msgr2=0x7fcf9c0b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.220+0000 7fcfa3c00640 1 --2- 172.21.15.60:0/3408856536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 0x7fcf9c0b2b40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009830 tx=0x7fcf8802a5b0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.221+0000 7fcfa3c00640 1 -- 172.21.15.60:0/3408856536 shutdown_connections 2024-09-16T15:43:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.221+0000 7fcfa3c00640 1 --2- 172.21.15.60:0/3408856536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 0x7fcf9c0b2b40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.221+0000 7fcfa3c00640 1 -- 172.21.15.60:0/3408856536 >> 172.21.15.60:0/3408856536 conn(0x7fcf9c0b1840 msgr2=0x7fcf9c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.221+0000 7fcfa3c00640 1 -- 172.21.15.60:0/3408856536 shutdown_connections 2024-09-16T15:43:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.221+0000 7fcfa3c00640 1 -- 172.21.15.60:0/3408856536 wait complete. 2024-09-16T15:43:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.222+0000 7fcfa3c00640 1 Processor -- start 2024-09-16T15:43:10.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.222+0000 7fcfa3c00640 1 -- start start 2024-09-16T15:43:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa3c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 0x7fcf9c1f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa3c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf9c0cec60 0x7fcf9c146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa3c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf9c161760 0x7fcf9c1469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa3c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c162520 con 0x7fcf9c0af190 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa3c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c1623a0 con 0x7fcf9c0cec60 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa3c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c1626a0 con 0x7fcf9c161760 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 0x7fcf9c1f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 0x7fcf9c1f9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42218/0 (socket says 172.21.15.60:42218) 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.223+0000 7fcfa1200640 1 -- 172.21.15.60:0/4009019232 learned_addr learned my addr 172.21.15.60:0/4009019232 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcf9be00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf9c0cec60 0x7fcf9c146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcfa1c00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf9c161760 0x7fcf9c1469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcfa1200640 1 -- 172.21.15.60:0/4009019232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf9c161760 msgr2=0x7fcf9c1469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcfa1200640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf9c161760 0x7fcf9c1469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcfa1200640 1 -- 172.21.15.60:0/4009019232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf9c0cec60 msgr2=0x7fcf9c146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcfa1200640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf9c0cec60 0x7fcf9c146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:10.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcfa1200640 1 -- 172.21.15.60:0/4009019232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c146f00 con 0x7fcf9c0af190 2024-09-16T15:43:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcfa1200640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 0x7fcf9c1f9270 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c001eb0 tx=0x7fcf8c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcf98c00640 1 -- 172.21.15.60:0/4009019232 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c011790 con 0x7fcf9c0af190 2024-09-16T15:43:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c1471d0 con 0x7fcf9c0af190 2024-09-16T15:43:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.224+0000 7fcf98c00640 1 -- 172.21.15.60:0/4009019232 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf8c017070 con 0x7fcf9c0af190 2024-09-16T15:43:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.225+0000 7fcf98c00640 1 -- 172.21.15.60:0/4009019232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c011ca0 con 0x7fcf9c0af190 2024-09-16T15:43:10.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.225+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c147790 con 0x7fcf9c0af190 2024-09-16T15:43:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.226+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c06a370 con 0x7fcf9c0af190 2024-09-16T15:43:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.226+0000 7fcf98c00640 1 -- 172.21.15.60:0/4009019232 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 15) ==== 51056+0+0 (secure 0 0 0) 0x7fcf8c01c460 con 0x7fcf9c0af190 2024-09-16T15:43:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.226+0000 7fcf98c00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf7c03e7d0 0x7fcf7c040c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.226+0000 7fcf98c00640 1 -- 172.21.15.60:0/4009019232 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6354+0+0 (secure 0 0 0) 0x7fcf8c053800 con 0x7fcf9c0af190 2024-09-16T15:43:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.226+0000 7fcf9be00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf7c03e7d0 0x7fcf7c040c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.227+0000 7fcf9be00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf7c03e7d0 0x7fcf7c040c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf88034000 tx=0x7fcf88004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:10.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.231+0000 7fcf98c00640 1 -- 172.21.15.60:0/4009019232 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcf9c06a370 con 0x7fcf9c0af190 2024-09-16T15:43:10.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:10.467+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcf9c06ef50 con 0x7fcf9c0af190 2024-09-16T15:43:11.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.043+0000 7fcf98c00640 1 -- 172.21.15.60:0/4009019232 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7fcf9c06ef50 con 0x7fcf9c0af190 2024-09-16T15:43:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.046+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf7c03e7d0 msgr2=0x7fcf7c040c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:11.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.046+0000 7fcfa3c00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf7c03e7d0 0x7fcf7c040c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf88034000 tx=0x7fcf88004730 comp rx=0 tx=0).stop 2024-09-16T15:43:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.046+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 msgr2=0x7fcf9c1f9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.046+0000 7fcfa3c00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 0x7fcf9c1f9270 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c001eb0 tx=0x7fcf8c00c890 comp rx=0 tx=0).stop 2024-09-16T15:43:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.047+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 shutdown_connections 2024-09-16T15:43:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.047+0000 7fcfa3c00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf7c03e7d0 0x7fcf7c040c90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.047+0000 7fcfa3c00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf9c161760 0x7fcf9c1469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.047+0000 7fcfa3c00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf9c0cec60 0x7fcf9c146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:11.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.047+0000 7fcfa3c00640 1 --2- 172.21.15.60:0/4009019232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf9c0af190 0x7fcf9c1f9270 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.047+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 >> 172.21.15.60:0/4009019232 conn(0x7fcf9c0b1840 msgr2=0x7fcf9c0adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.047+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 shutdown_connections 2024-09-16T15:43:11.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.048+0000 7fcfa3c00640 1 -- 172.21.15.60:0/4009019232 wait complete. 2024-09-16T15:43:11.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:43:11.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.236+0000 7fc6afe00640 1 -- 172.21.15.60:0/2771783596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b015bda0 msgr2=0x7fc6b0073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.236+0000 7fc6afe00640 1 --2- 172.21.15.60:0/2771783596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b015bda0 0x7fc6b0073010 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009830 tx=0x7fc6a002a510 comp rx=0 tx=0).stop 2024-09-16T15:43:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.236+0000 7fc6afe00640 1 -- 172.21.15.60:0/2771783596 shutdown_connections 2024-09-16T15:43:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.236+0000 7fc6afe00640 1 --2- 172.21.15.60:0/2771783596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6b006e590 0x7fc6b006fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.236+0000 7fc6afe00640 1 --2- 172.21.15.60:0/2771783596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b015bda0 0x7fc6b0073010 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.236+0000 7fc6afe00640 1 -- 172.21.15.60:0/2771783596 >> 172.21.15.60:0/2771783596 conn(0x7fc6b00cfa70 msgr2=0x7fc6b00cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.236+0000 7fc6afe00640 1 -- 172.21.15.60:0/2771783596 shutdown_connections 2024-09-16T15:43:11.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.236+0000 7fc6afe00640 1 -- 172.21.15.60:0/2771783596 wait complete. 2024-09-16T15:43:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.238+0000 7fc6afe00640 1 Processor -- start 2024-09-16T15:43:11.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.238+0000 7fc6afe00640 1 -- start start 2024-09-16T15:43:11.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b006e590 0x7fc6b01f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:11.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6afe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b0070380 0x7fc6b00c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:11.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6afe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6b015bda0 0x7fc6b0150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:11.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6afe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b00cc350 con 0x7fc6b0070380 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6afe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b00cc1d0 con 0x7fc6b006e590 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6afe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b00740e0 con 0x7fc6b015bda0 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6a7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b0070380 0x7fc6b00c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6a7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b0070380 0x7fc6b00c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42236/0 (socket says 172.21.15.60:42236) 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6a7e00640 1 -- 172.21.15.60:0/1667323443 learned_addr learned my addr 172.21.15.60:0/1667323443 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6af400640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6b015bda0 0x7fc6b0150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6aea00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b006e590 0x7fc6b01f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6a7e00640 1 -- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6b015bda0 msgr2=0x7fc6b0150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6a7e00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6b015bda0 0x7fc6b0150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6a7e00640 1 -- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b006e590 msgr2=0x7fc6b01f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6a7e00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b006e590 0x7fc6b01f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6a7e00640 1 -- 172.21.15.60:0/1667323443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b01f83d0 con 0x7fc6b0070380 2024-09-16T15:43:11.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6aea00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b006e590 0x7fc6b01f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.239+0000 7fc6af400640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6b015bda0 0x7fc6b0150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.240+0000 7fc6a7e00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b0070380 0x7fc6b00c5be0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc69800d950 tx=0x7fc69800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.240+0000 7fc6a6a00640 1 -- 172.21.15.60:0/1667323443 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698017070 con 0x7fc6b0070380 2024-09-16T15:43:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.240+0000 7fc6a6a00640 1 -- 172.21.15.60:0/1667323443 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc698009d20 con 0x7fc6b0070380 2024-09-16T15:43:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.240+0000 7fc6a6a00640 1 -- 172.21.15.60:0/1667323443 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6980106c0 con 0x7fc6b0070380 2024-09-16T15:43:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.240+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b01f8680 con 0x7fc6b0070380 2024-09-16T15:43:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.240+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b01f8c40 con 0x7fc6b0070380 2024-09-16T15:43:11.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.241+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b0074610 con 0x7fc6b0070380 2024-09-16T15:43:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.242+0000 7fc6a6a00640 1 -- 172.21.15.60:0/1667323443 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fc69801a070 con 0x7fc6b0070380 2024-09-16T15:43:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.244+0000 7fc6a6a00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc68c078970 0x7fc68c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.244+0000 7fc6a6a00640 1 -- 172.21.15.60:0/1667323443 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6354+0+0 (secure 0 0 0) 0x7fc69809afe0 con 0x7fc6b0070380 2024-09-16T15:43:11.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.244+0000 7fc6aea00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc68c078970 0x7fc68c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:11.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.245+0000 7fc6aea00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc68c078970 0x7fc68c07ae30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0034000 tx=0x7fc6a0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:11.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.248+0000 7fc6a6a00640 1 -- 172.21.15.60:0/1667323443 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc6980631b0 con 0x7fc6b0070380 2024-09-16T15:43:11.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:11.446+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc6b0070060 con 0x7fc6b0070380 2024-09-16T15:43:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.045+0000 7fc6a6a00640 1 -- 172.21.15.60:0/1667323443 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) ==== 181+0+0 (secure 0 0 0) 0x7fc698062ab0 con 0x7fc6b0070380 2024-09-16T15:43:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:43:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc68c078970 msgr2=0x7fc68c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc68c078970 0x7fc68c07ae30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0034000 tx=0x7fc6a0004730 comp rx=0 tx=0).stop 2024-09-16T15:43:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b0070380 msgr2=0x7fc6b00c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b0070380 0x7fc6b00c5be0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc69800d950 tx=0x7fc69800de10 comp rx=0 tx=0).stop 2024-09-16T15:43:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 shutdown_connections 2024-09-16T15:43:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc68c078970 0x7fc68c07ae30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6b015bda0 0x7fc6b0150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6b0070380 0x7fc6b00c5be0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 --2- 172.21.15.60:0/1667323443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6b006e590 0x7fc6b01f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 >> 172.21.15.60:0/1667323443 conn(0x7fc6b00cfa70 msgr2=0x7fc6b00cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 shutdown_connections 2024-09-16T15:43:12.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.050+0000 7fc6afe00640 1 -- 172.21.15.60:0/1667323443 wait complete. 2024-09-16T15:43:12.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:43:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.245+0000 7f7d97200640 1 -- 172.21.15.60:0/3831357513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 msgr2=0x7f7d9006f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.245+0000 7f7d97200640 1 --2- 172.21.15.60:0/3831357513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 0x7f7d9006f300 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7d80005fd0 tx=0x7f7d8002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.246+0000 7f7d97200640 1 -- 172.21.15.60:0/3831357513 shutdown_connections 2024-09-16T15:43:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.246+0000 7f7d97200640 1 --2- 172.21.15.60:0/3831357513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d900b4770 0x7f7d9006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.246+0000 7f7d97200640 1 --2- 172.21.15.60:0/3831357513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 0x7f7d9006f300 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.246+0000 7f7d97200640 1 -- 172.21.15.60:0/3831357513 >> 172.21.15.60:0/3831357513 conn(0x7f7d900b18e0 msgr2=0x7f7d900b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.246+0000 7f7d97200640 1 -- 172.21.15.60:0/3831357513 shutdown_connections 2024-09-16T15:43:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.246+0000 7f7d97200640 1 -- 172.21.15.60:0/3831357513 wait complete. 2024-09-16T15:43:12.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.247+0000 7f7d97200640 1 Processor -- start 2024-09-16T15:43:12.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.247+0000 7f7d97200640 1 -- start start 2024-09-16T15:43:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.248+0000 7f7d97200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d9006ec30 0x7f7d90156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.248+0000 7f7d97200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d900b4770 0x7f7d901f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.248+0000 7f7d97200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 0x7f7d901f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.248+0000 7f7d97200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7d901622c0 con 0x7f7d9015dfc0 2024-09-16T15:43:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.248+0000 7f7d97200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7d900ae8f0 con 0x7f7d9006ec30 2024-09-16T15:43:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.248+0000 7f7d97200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7d90162440 con 0x7f7d900b4770 2024-09-16T15:43:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.248+0000 7f7d95200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 0x7f7d901f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d95200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 0x7f7d901f8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42244/0 (socket says 172.21.15.60:42244) 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d95200640 1 -- 172.21.15.60:0/2120559242 learned_addr learned my addr 172.21.15.60:0/2120559242 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d87400640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d900b4770 0x7f7d901f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d8fe00640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d9006ec30 0x7f7d90156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d95200640 1 -- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d900b4770 msgr2=0x7f7d901f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d95200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d900b4770 0x7f7d901f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d95200640 1 -- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d9006ec30 msgr2=0x7f7d90156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d95200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d9006ec30 0x7f7d90156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d95200640 1 -- 172.21.15.60:0/2120559242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9015c910 con 0x7f7d9015dfc0 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d8fe00640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d9006ec30 0x7f7d90156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:12.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d95200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 0x7f7d901f8f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800c7a0 tx=0x7f7d8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:12.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d8cc00640 1 -- 172.21.15.60:0/2120559242 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88003b70 con 0x7f7d9015dfc0 2024-09-16T15:43:12.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d8cc00640 1 -- 172.21.15.60:0/2120559242 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d88003d10 con 0x7f7d9015dfc0 2024-09-16T15:43:12.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.249+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9015cbf0 con 0x7f7d9015dfc0 2024-09-16T15:43:12.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.250+0000 7f7d8cc00640 1 -- 172.21.15.60:0/2120559242 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800aab0 con 0x7f7d9015dfc0 2024-09-16T15:43:12.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.250+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9015d1b0 con 0x7f7d9015dfc0 2024-09-16T15:43:12.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.251+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d90162960 con 0x7f7d9015dfc0 2024-09-16T15:43:12.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.252+0000 7f7d8cc00640 1 -- 172.21.15.60:0/2120559242 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f7d8800ac50 con 0x7f7d9015dfc0 2024-09-16T15:43:12.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.253+0000 7f7d8cc00640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d6c0789a0 0x7f7d6c07ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:12.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.254+0000 7f7d8cc00640 1 -- 172.21.15.60:0/2120559242 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5916+0+0 (secure 0 0 0) 0x7f7d8809aa30 con 0x7f7d9015dfc0 2024-09-16T15:43:12.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.254+0000 7f7d8fe00640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d6c0789a0 0x7f7d6c07ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:12.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.254+0000 7f7d8fe00640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d6c0789a0 0x7f7d6c07ae60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d80033000 tx=0x7f7d80004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:12.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.258+0000 7f7d8cc00640 1 -- 172.21.15.60:0/2120559242 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7d88062da0 con 0x7f7d9015dfc0 2024-09-16T15:43:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:12.455+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d900b07e0 con 0x7f7d9015dfc0 2024-09-16T15:43:13.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.457+0000 7f7d8cc00640 1 -- 172.21.15.60:0/2120559242 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) ==== 169+0+0 (secure 0 0 0) 0x7f7d880626a0 con 0x7f7d9015dfc0 2024-09-16T15:43:13.459 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.461+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d6c0789a0 msgr2=0x7f7d6c07ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.461+0000 7f7d97200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d6c0789a0 0x7f7d6c07ae60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d80033000 tx=0x7f7d80004730 comp rx=0 tx=0).stop 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.461+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 msgr2=0x7f7d901f8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.461+0000 7f7d97200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 0x7f7d901f8f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800c7a0 tx=0x7f7d8800cc60 comp rx=0 tx=0).stop 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.462+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 shutdown_connections 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.462+0000 7f7d97200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d6c0789a0 0x7f7d6c07ae60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.462+0000 7f7d97200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d9015dfc0 0x7f7d901f8f20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.462+0000 7f7d97200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d900b4770 0x7f7d901f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.462+0000 7f7d97200640 1 --2- 172.21.15.60:0/2120559242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d9006ec30 0x7f7d90156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.462+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 >> 172.21.15.60:0/2120559242 conn(0x7f7d900b18e0 msgr2=0x7f7d900b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:13.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.462+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 shutdown_connections 2024-09-16T15:43:13.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.462+0000 7f7d97200640 1 -- 172.21.15.60:0/2120559242 wait complete. 2024-09-16T15:43:13.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:43:13.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.649+0000 7fb257200640 1 -- 172.21.15.60:0/1292774412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb250161550 msgr2=0x7fb25006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:13.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.649+0000 7fb257200640 1 --2- 172.21.15.60:0/1292774412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb250161550 0x7fb25006ef50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb240009860 tx=0x7fb24002a510 comp rx=0 tx=0).stop 2024-09-16T15:43:13.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.649+0000 7fb257200640 1 -- 172.21.15.60:0/1292774412 shutdown_connections 2024-09-16T15:43:13.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.649+0000 7fb257200640 1 --2- 172.21.15.60:0/1292774412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb250161550 0x7fb25006ef50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.649+0000 7fb257200640 1 -- 172.21.15.60:0/1292774412 >> 172.21.15.60:0/1292774412 conn(0x7fb2500b1950 msgr2=0x7fb2500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:13.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.649+0000 7fb257200640 1 -- 172.21.15.60:0/1292774412 shutdown_connections 2024-09-16T15:43:13.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.649+0000 7fb257200640 1 -- 172.21.15.60:0/1292774412 wait complete. 2024-09-16T15:43:13.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.651+0000 7fb257200640 1 Processor -- start 2024-09-16T15:43:13.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.651+0000 7fb257200640 1 -- start start 2024-09-16T15:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb257200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb250072df0 0x7fb2500d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb257200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2500af3d0 0x7fb2501f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb257200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb250161550 0x7fb25009ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb257200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb2500739b0 con 0x7fb2500af3d0 2024-09-16T15:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb257200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb25006fdb0 con 0x7fb250072df0 2024-09-16T15:43:13.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb257200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb250073b30 con 0x7fb250161550 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb24f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2500af3d0 0x7fb2501f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb24f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2500af3d0 0x7fb2501f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42248/0 (socket says 172.21.15.60:42248) 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb24f400640 1 -- 172.21.15.60:0/749792438 learned_addr learned my addr 172.21.15.60:0/749792438 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb24fe00640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb250072df0 0x7fb2500d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.652+0000 7fb255200640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb250161550 0x7fb25009ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb24f400640 1 -- 172.21.15.60:0/749792438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb250161550 msgr2=0x7fb25009ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb24f400640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb250161550 0x7fb25009ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb24f400640 1 -- 172.21.15.60:0/749792438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb250072df0 msgr2=0x7fb2500d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb24f400640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb250072df0 0x7fb2500d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb24f400640 1 -- 172.21.15.60:0/749792438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2501f52c0 con 0x7fb2500af3d0 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb24f400640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2500af3d0 0x7fb2501f4b70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00c4b0 tx=0x7fb23c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb247e00640 1 -- 172.21.15.60:0/749792438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c017070 con 0x7fb2500af3d0 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb247e00640 1 -- 172.21.15.60:0/749792438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb23c0053c0 con 0x7fb2500af3d0 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb247e00640 1 -- 172.21.15.60:0/749792438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c003850 con 0x7fb2500af3d0 2024-09-16T15:43:13.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2501f55a0 con 0x7fb2500af3d0 2024-09-16T15:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.653+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2500cec60 con 0x7fb2500af3d0 2024-09-16T15:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.654+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb250073f60 con 0x7fb2500af3d0 2024-09-16T15:43:13.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.655+0000 7fb247e00640 1 -- 172.21.15.60:0/749792438 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fb23c01a070 con 0x7fb2500af3d0 2024-09-16T15:43:13.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.657+0000 7fb247e00640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb230078970 0x7fb23007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:13.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.657+0000 7fb247e00640 1 -- 172.21.15.60:0/749792438 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7fb23c099e70 con 0x7fb2500af3d0 2024-09-16T15:43:13.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.657+0000 7fb24fe00640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb230078970 0x7fb23007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:13.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.658+0000 7fb24fe00640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb230078970 0x7fb23007ae30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb240034000 tx=0x7fb240004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.662+0000 7fb247e00640 1 -- 172.21.15.60:0/749792438 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb23c061d30 con 0x7fb2500af3d0 2024-09-16T15:43:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.870+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fb2500adf50 con 0x7fb2500af3d0 2024-09-16T15:43:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.874+0000 7fb247e00640 1 -- 172.21.15.60:0/749792438 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7fb23c066760 con 0x7fb2500af3d0 2024-09-16T15:43:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb230078970 msgr2=0x7fb23007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb230078970 0x7fb23007ae30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb240034000 tx=0x7fb240004730 comp rx=0 tx=0).stop 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2500af3d0 msgr2=0x7fb2501f4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2500af3d0 0x7fb2501f4b70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb23c00c4b0 tx=0x7fb23c00c970 comp rx=0 tx=0).stop 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 shutdown_connections 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb230078970 0x7fb23007ae30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb250161550 0x7fb25009ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2500af3d0 0x7fb2501f4b70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 --2- 172.21.15.60:0/749792438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb250072df0 0x7fb2500d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 >> 172.21.15.60:0/749792438 conn(0x7fb2500b1950 msgr2=0x7fb25006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 shutdown_connections 2024-09-16T15:43:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:13.877+0000 7fb257200640 1 -- 172.21.15.60:0/749792438 wait complete. 2024-09-16T15:43:13.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-16T15:43:14.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.053+0000 7f48c6200640 1 -- 172.21.15.60:0/3405103752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 msgr2=0x7f48c0074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.053+0000 7f48c6200640 1 --2- 172.21.15.60:0/3405103752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 0x7f48c0074420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f48c00cfbb0 tx=0x7f48b802aa30 comp rx=0 tx=0).stop 2024-09-16T15:43:14.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.053+0000 7f48c6200640 1 -- 172.21.15.60:0/3405103752 shutdown_connections 2024-09-16T15:43:14.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.053+0000 7f48c6200640 1 --2- 172.21.15.60:0/3405103752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 0x7f48c0074420 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.053+0000 7f48c6200640 1 --2- 172.21.15.60:0/3405103752 >> v2:172.21.15.100:3300/0 conn(0x7f48c00ccf00 0x7f48c00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.053+0000 7f48c6200640 1 -- 172.21.15.60:0/3405103752 >> 172.21.15.60:0/3405103752 conn(0x7f48c00b1950 msgr2=0x7f48c00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:14.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.053+0000 7f48c6200640 1 -- 172.21.15.60:0/3405103752 shutdown_connections 2024-09-16T15:43:14.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.053+0000 7f48c6200640 1 -- 172.21.15.60:0/3405103752 wait complete. 2024-09-16T15:43:14.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.055+0000 7f48c6200640 1 Processor -- start 2024-09-16T15:43:14.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.055+0000 7f48c6200640 1 -- start start 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48c6200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 0x7f48c01f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48c6200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c00ccf00 0x7f48c00cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48c6200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c015f5a0 0x7f48c00b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48c6200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f48c006f2e0 con 0x7f48c0073fb0 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48c6200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f48c006f160 con 0x7f48c015f5a0 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48c6200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f48c006f460 con 0x7f48c00ccf00 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48bf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 0x7f48c01f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48bf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 0x7f48c01f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42272/0 (socket says 172.21.15.60:42272) 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48bf400640 1 -- 172.21.15.60:0/3468304696 learned_addr learned my addr 172.21.15.60:0/3468304696 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48b6a00640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c00ccf00 0x7f48c00cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:14.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.056+0000 7f48bfe00640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c015f5a0 0x7f48c00b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48bf400640 1 -- 172.21.15.60:0/3468304696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c00ccf00 msgr2=0x7f48c00cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48bf400640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c00ccf00 0x7f48c00cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48bf400640 1 -- 172.21.15.60:0/3468304696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c015f5a0 msgr2=0x7f48c00b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48bf400640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c015f5a0 0x7f48c00b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48bf400640 1 -- 172.21.15.60:0/3468304696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c01f5a20 con 0x7f48c0073fb0 2024-09-16T15:43:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48bf400640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 0x7f48c01f54e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c3d0 tx=0x7f48b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48b7e00640 1 -- 172.21.15.60:0/3468304696 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0017070 con 0x7f48c0073fb0 2024-09-16T15:43:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48b7e00640 1 -- 172.21.15.60:0/3468304696 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48b00053c0 con 0x7f48c0073fb0 2024-09-16T15:43:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48b7e00640 1 -- 172.21.15.60:0/3468304696 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0003850 con 0x7f48c0073fb0 2024-09-16T15:43:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c00cfc30 con 0x7f48c0073fb0 2024-09-16T15:43:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.057+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c00d01f0 con 0x7f48c0073fb0 2024-09-16T15:43:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.058+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c006f9f0 con 0x7f48c0073fb0 2024-09-16T15:43:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.060+0000 7f48b7e00640 1 -- 172.21.15.60:0/3468304696 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f48b001a070 con 0x7f48c0073fb0 2024-09-16T15:43:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.061+0000 7f48b7e00640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f489c078970 0x7f489c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.061+0000 7f48b7e00640 1 -- 172.21.15.60:0/3468304696 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7f48b0099e30 con 0x7f48c0073fb0 2024-09-16T15:43:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.061+0000 7f48b6a00640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f489c078970 0x7f489c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:14.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.061+0000 7f48b6a00640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f489c078970 0x7f489c07ae30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f48a8005d80 tx=0x7f48a800b390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:14.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.064+0000 7f48b7e00640 1 -- 172.21.15.60:0/3468304696 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f48b0061c60 con 0x7f48c0073fb0 2024-09-16T15:43:14.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.332+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f48c00aff20 con 0x7f48c0073fb0 2024-09-16T15:43:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.459+0000 7f48b7e00640 1 -- 172.21.15.60:0/3468304696 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v261) ==== 131+0+0 (secure 0 0 0) 0x7f48b0066690 con 0x7f48c0073fb0 2024-09-16T15:43:14.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f489c078970 msgr2=0x7f489c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f489c078970 0x7f489c07ae30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f48a8005d80 tx=0x7f48a800b390 comp rx=0 tx=0).stop 2024-09-16T15:43:14.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 msgr2=0x7f48c01f54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 0x7f48c01f54e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c3d0 tx=0x7f48b000c890 comp rx=0 tx=0).stop 2024-09-16T15:43:14.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 shutdown_connections 2024-09-16T15:43:14.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f489c078970 0x7f489c07ae30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48c015f5a0 0x7f48c00b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c00ccf00 0x7f48c00cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 --2- 172.21.15.60:0/3468304696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c0073fb0 0x7f48c01f54e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 >> 172.21.15.60:0/3468304696 conn(0x7f48c00b1950 msgr2=0x7f48c009cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:14.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 shutdown_connections 2024-09-16T15:43:14.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.461+0000 7f48c6200640 1 -- 172.21.15.60:0/3468304696 wait complete. 2024-09-16T15:43:14.478 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-16T15:43:14.478 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-16T15:43:14.478 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:43:14.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:14.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.638+0000 7f915d600640 1 -- 172.21.15.60:0/2998645263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91580cc8d0 msgr2=0x7f915815cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.638+0000 7f915d600640 1 --2- 172.21.15.60:0/2998645263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91580cc8d0 0x7f915815cb40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9148009830 tx=0x7f914802a510 comp rx=0 tx=0).stop 2024-09-16T15:43:14.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.638+0000 7f915d600640 1 -- 172.21.15.60:0/2998645263 shutdown_connections 2024-09-16T15:43:14.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.638+0000 7f915d600640 1 --2- 172.21.15.60:0/2998645263 >> v2:172.21.15.100:3300/0 conn(0x7f91580d1620 0x7f91580cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.638+0000 7f915d600640 1 --2- 172.21.15.60:0/2998645263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91580cc8d0 0x7f915815cb40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.638+0000 7f915d600640 1 -- 172.21.15.60:0/2998645263 >> 172.21.15.60:0/2998645263 conn(0x7f91580d0b30 msgr2=0x7f91580d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:14.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.638+0000 7f915d600640 1 -- 172.21.15.60:0/2998645263 shutdown_connections 2024-09-16T15:43:14.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.638+0000 7f915d600640 1 -- 172.21.15.60:0/2998645263 wait complete. 2024-09-16T15:43:14.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.640+0000 7f915d600640 1 Processor -- start 2024-09-16T15:43:14.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.640+0000 7f915d600640 1 -- start start 2024-09-16T15:43:14.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f915d600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9158073ee0 0x7f91581f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:14.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f915d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91580cc8d0 0x7f91580b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f915d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91580d1620 0x7f91580b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f915d600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9158074390 con 0x7f91580d1620 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f915d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f91581565e0 con 0x7f91580cc8d0 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f915d600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9158074510 con 0x7f9158073ee0 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f915cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91580d1620 0x7f91580b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f9157e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9158073ee0 0x7f91581f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f914f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91580cc8d0 0x7f91580b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f9157e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9158073ee0 0x7f91581f2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:44870/0 (socket says 172.21.15.60:44870) 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f9157e00640 1 -- 172.21.15.60:0/3740741706 learned_addr learned my addr 172.21.15.60:0/3740741706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:14.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.641+0000 7f915cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91580d1620 0x7f91580b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42276/0 (socket says 172.21.15.60:42276) 2024-09-16T15:43:14.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.642+0000 7f9157e00640 1 -- 172.21.15.60:0/3740741706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91580cc8d0 msgr2=0x7f91580b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.642+0000 7f9157e00640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91580cc8d0 0x7f91580b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.642+0000 7f9157e00640 1 -- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91580d1620 msgr2=0x7f91580b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.642+0000 7f9157e00640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91580d1620 0x7f91580b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.642+0000 7f9157e00640 1 -- 172.21.15.60:0/3740741706 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91580b3d30 con 0x7f9158073ee0 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.642+0000 7f915cc00640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91580d1620 0x7f91580b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.642+0000 7f9157e00640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9158073ee0 0x7f91581f2020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9148009950 tx=0x7f9148032c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.642+0000 7f9154c00640 1 -- 172.21.15.60:0/3740741706 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148040070 con 0x7f9158073ee0 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.643+0000 7f9154c00640 1 -- 172.21.15.60:0/3740741706 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9148005310 con 0x7f9158073ee0 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.643+0000 7f9154c00640 1 -- 172.21.15.60:0/3740741706 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148005c90 con 0x7f9158073ee0 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.643+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91580b4010 con 0x7f9158073ee0 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.643+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91580b45d0 con 0x7f9158073ee0 2024-09-16T15:43:14.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.644+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9158070080 con 0x7f9158073ee0 2024-09-16T15:43:14.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.645+0000 7f9154c00640 1 -- 172.21.15.60:0/3740741706 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f9148039040 con 0x7f9158073ee0 2024-09-16T15:43:14.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.646+0000 7f9154c00640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9134078970 0x7f913407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:14.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.646+0000 7f9154c00640 1 -- 172.21.15.60:0/3740741706 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7f91480bbfc0 con 0x7f9158073ee0 2024-09-16T15:43:14.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.646+0000 7f914f400640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9134078970 0x7f913407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:14.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.647+0000 7f914f400640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9134078970 0x7f913407ae30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f91580b6ca0 tx=0x7f9140005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:14.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.650+0000 7f9154c00640 1 -- 172.21.15.60:0/3740741706 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9148083e70 con 0x7f9158073ee0 2024-09-16T15:43:14.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.848+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91580c40b0 con 0x7f9158073ee0 2024-09-16T15:43:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.849+0000 7f9154c00640 1 -- 172.21.15.60:0/3740741706 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+13856 (secure 0 0 0) 0x7f91480888a0 con 0x7f9158073ee0 2024-09-16T15:43:14.851 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:14.851 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:13.449581+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1266112466":"2024-09-17T15:43:10.035595+0000","172.21.15.60:6834/1266112466":"2024-09-17T15:43:10.035595+0000","172.21.15.100:0/690787621":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/3427025651":"2024-09-17T15:43:06.127928+0000","172.21.15.100:6800/2198567224":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/2641588278":"2024-09-17T15:43:06.127928+0000","172.21.15.100:6801/2198567224":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/1587892015":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4177407071":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4072261497":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/2410226558":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/1852740290":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/4216665037":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/883853961":"2024-09-17T15:43:06.127928+0000","172.21.15.100:0/2997397432":"2024-09-17T15:43:06.127928+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9134078970 msgr2=0x7f913407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9134078970 0x7f913407ae30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f91580b6ca0 tx=0x7f9140005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9158073ee0 msgr2=0x7f91581f2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9158073ee0 0x7f91581f2020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9148009950 tx=0x7f9148032c10 comp rx=0 tx=0).stop 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 shutdown_connections 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9134078970 0x7f913407ae30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91580d1620 0x7f91580b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91580cc8d0 0x7f91580b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 --2- 172.21.15.60:0/3740741706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9158073ee0 0x7f91581f2020 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 >> 172.21.15.60:0/3740741706 conn(0x7f91580d0b30 msgr2=0x7f9158153a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 shutdown_connections 2024-09-16T15:43:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:14.852+0000 7f915d600640 1 -- 172.21.15.60:0/3740741706 wait complete. 2024-09-16T15:43:14.869 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:43:14.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:43:15.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.033+0000 7f92faa00640 1 -- 172.21.15.60:0/1611185744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8004020 msgr2=0x7f92d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:15.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.033+0000 7f92faa00640 1 --2- 172.21.15.60:0/1611185744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92d8004020 0x7f92d8004800 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009b50 tx=0x7f92d8005830 comp rx=0 tx=0).stop 2024-09-16T15:43:15.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.033+0000 7f92f8a00640 1 -- 172.21.15.60:0/1611185744 reap_dead start 2024-09-16T15:43:15.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.033+0000 7f92faa00640 1 -- 172.21.15.60:0/1611185744 shutdown_connections 2024-09-16T15:43:15.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.033+0000 7f92faa00640 1 -- 172.21.15.60:0/1611185744 >> 172.21.15.60:0/1611185744 conn(0x7f92f40b1840 msgr2=0x7f92f40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:15.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.034+0000 7f92faa00640 1 -- 172.21.15.60:0/1611185744 shutdown_connections 2024-09-16T15:43:15.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.034+0000 7f92faa00640 1 -- 172.21.15.60:0/1611185744 wait complete. 2024-09-16T15:43:15.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.035+0000 7f92faa00640 1 Processor -- start 2024-09-16T15:43:15.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.035+0000 7f92faa00640 1 -- start start 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92faa00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8004020 0x7f92f41526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92faa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f414e290 0x7f92f41f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92faa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92f41f8600 0x7f92f41f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92faa00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f92f40b2ed0 con 0x7f92f41f8600 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92faa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92f40b2d50 con 0x7f92f414e290 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92faa00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f92f40b3050 con 0x7f92d8004020 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92f8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92f41f8600 0x7f92f41f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92f8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92f41f8600 0x7f92f41f4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42302/0 (socket says 172.21.15.60:42302) 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.036+0000 7f92f8a00640 1 -- 172.21.15.60:0/3419109825 learned_addr learned my addr 172.21.15.60:0/3419109825 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92f3400640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f414e290 0x7f92f41f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:15.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92f3e00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8004020 0x7f92f41526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:15.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92f8a00640 1 -- 172.21.15.60:0/3419109825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8004020 msgr2=0x7f92f41526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:15.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92f8a00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8004020 0x7f92f41526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92f8a00640 1 -- 172.21.15.60:0/3419109825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f414e290 msgr2=0x7f92f41f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:15.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92f8a00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f414e290 0x7f92f41f80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92f8a00640 1 -- 172.21.15.60:0/3419109825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f4153200 con 0x7f92f41f8600 2024-09-16T15:43:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92f8a00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92f41f8600 0x7f92f41f4ac0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f92f40af120 tx=0x7f92ec00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92ebe00640 1 -- 172.21.15.60:0/3419109825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec004b20 con 0x7f92f41f8600 2024-09-16T15:43:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92ebe00640 1 -- 172.21.15.60:0/3419109825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92ec004cc0 con 0x7f92f41f8600 2024-09-16T15:43:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.037+0000 7f92ebe00640 1 -- 172.21.15.60:0/3419109825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec003bd0 con 0x7f92f41f8600 2024-09-16T15:43:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.038+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f41534e0 con 0x7f92f41f8600 2024-09-16T15:43:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.038+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f4152be0 con 0x7f92f41f8600 2024-09-16T15:43:15.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.039+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f40cf0b0 con 0x7f92f41f8600 2024-09-16T15:43:15.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.040+0000 7f92ebe00640 1 -- 172.21.15.60:0/3419109825 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f92ec004030 con 0x7f92f41f8600 2024-09-16T15:43:15.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.041+0000 7f92ebe00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f92d4078970 0x7f92d407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:15.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.041+0000 7f92ebe00640 1 -- 172.21.15.60:0/3419109825 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7f92ec09af40 con 0x7f92f41f8600 2024-09-16T15:43:15.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.041+0000 7f92f3e00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f92d4078970 0x7f92d407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:15.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.042+0000 7f92f3e00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f92d4078970 0x7f92d407ae30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000fb0 tx=0x7f92e4006040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:15.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.045+0000 7f92ebe00640 1 -- 172.21.15.60:0/3419109825 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f92ec062d70 con 0x7f92f41f8600 2024-09-16T15:43:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.238+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92f414ece0 con 0x7f92f41f8600 2024-09-16T15:43:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.239+0000 7f92ebe00640 1 -- 172.21.15.60:0/3419109825 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f92ec0677a0 con 0x7f92f41f8600 2024-09-16T15:43:15.240 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.241+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f92d4078970 msgr2=0x7f92d407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.241+0000 7f92faa00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f92d4078970 0x7f92d407ae30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000fb0 tx=0x7f92e4006040 comp rx=0 tx=0).stop 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.241+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92f41f8600 msgr2=0x7f92f41f4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.241+0000 7f92faa00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92f41f8600 0x7f92f41f4ac0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f92f40af120 tx=0x7f92ec00f3c0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.242+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 shutdown_connections 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.242+0000 7f92faa00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f92d4078970 0x7f92d407ae30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.242+0000 7f92faa00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92f41f8600 0x7f92f41f4ac0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.242+0000 7f92faa00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92f414e290 0x7f92f41f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.242+0000 7f92faa00640 1 --2- 172.21.15.60:0/3419109825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92d8004020 0x7f92f41526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.242+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 >> 172.21.15.60:0/3419109825 conn(0x7f92f40b1840 msgr2=0x7f92f40adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.242+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 shutdown_connections 2024-09-16T15:43:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.242+0000 7f92faa00640 1 -- 172.21.15.60:0/3419109825 wait complete. 2024-09-16T15:43:15.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-16T15:43:15.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.417+0000 7f48a4800640 1 -- 172.21.15.60:0/1502503767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 msgr2=0x7f489c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:15.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.417+0000 7f48a4800640 1 --2- 172.21.15.60:0/1502503767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 0x7f489c0b2440 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f489c076720 tx=0x7f489002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.418+0000 7f48a4800640 1 -- 172.21.15.60:0/1502503767 shutdown_connections 2024-09-16T15:43:15.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.418+0000 7f48a4800640 1 --2- 172.21.15.60:0/1502503767 >> v2:172.21.15.100:3300/0 conn(0x7f489c0ad270 0x7f489c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.418+0000 7f48a4800640 1 --2- 172.21.15.60:0/1502503767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f489c0ccf00 0x7f489c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.418+0000 7f48a4800640 1 --2- 172.21.15.60:0/1502503767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 0x7f489c0b2440 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.418+0000 7f48a4800640 1 -- 172.21.15.60:0/1502503767 >> 172.21.15.60:0/1502503767 conn(0x7f489c0b1950 msgr2=0x7f489c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:15.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.418+0000 7f48a4800640 1 -- 172.21.15.60:0/1502503767 shutdown_connections 2024-09-16T15:43:15.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.418+0000 7f48a4800640 1 -- 172.21.15.60:0/1502503767 wait complete. 2024-09-16T15:43:15.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.419+0000 7f48a4800640 1 Processor -- start 2024-09-16T15:43:15.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.419+0000 7f48a4800640 1 -- start start 2024-09-16T15:43:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.420+0000 7f48a4800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c0ad270 0x7f489c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.420+0000 7f48a4800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f489c0ccf00 0x7f489c1f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.420+0000 7f48a4800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 0x7f489c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.420+0000 7f48a4800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f489c06f160 con 0x7f489c15f5a0 2024-09-16T15:43:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.420+0000 7f48a4800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f489c074780 con 0x7f489c0ad270 2024-09-16T15:43:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.420+0000 7f48a4800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f489c06f2e0 con 0x7f489c0ccf00 2024-09-16T15:43:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 0x7f489c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:15.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 0x7f489c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42318/0 (socket says 172.21.15.60:42318) 2024-09-16T15:43:15.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 -- 172.21.15.60:0/1495658298 learned_addr learned my addr 172.21.15.60:0/1495658298 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:15.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 -- 172.21.15.60:0/1495658298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f489c0ccf00 msgr2=0x7f489c1f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:15.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f489c0ccf00 0x7f489c1f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 -- 172.21.15.60:0/1495658298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c0ad270 msgr2=0x7f489c156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:15.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c0ad270 0x7f489c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:15.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 -- 172.21.15.60:0/1495658298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c1565d0 con 0x7f489c15f5a0 2024-09-16T15:43:15.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f48a2800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 0x7f489c0cff20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f489800da90 tx=0x7f489800df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f4896000640 1 -- 172.21.15.60:0/1495658298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898017070 con 0x7f489c15f5a0 2024-09-16T15:43:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f4896000640 1 -- 172.21.15.60:0/1495658298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4898003970 con 0x7f489c15f5a0 2024-09-16T15:43:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.421+0000 7f4896000640 1 -- 172.21.15.60:0/1495658298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898003d80 con 0x7f489c15f5a0 2024-09-16T15:43:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.422+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c156b30 con 0x7f489c15f5a0 2024-09-16T15:43:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.422+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c1f5d50 con 0x7f489c15f5a0 2024-09-16T15:43:15.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.423+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c06f870 con 0x7f489c15f5a0 2024-09-16T15:43:15.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.424+0000 7f4896000640 1 -- 172.21.15.60:0/1495658298 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f4898003f20 con 0x7f489c15f5a0 2024-09-16T15:43:15.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.425+0000 7f4896000640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4880078970 0x7f488007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:15.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.425+0000 7f4896000640 1 -- 172.21.15.60:0/1495658298 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7f489809afd0 con 0x7f489c15f5a0 2024-09-16T15:43:15.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.425+0000 7f48a1e00640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4880078970 0x7f488007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:15.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.426+0000 7f48a1e00640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4880078970 0x7f488007ae30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4890033000 tx=0x7f48900046c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.429+0000 7f4896000640 1 -- 172.21.15.60:0/1495658298 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f489c1565d0 con 0x7f489c15f5a0 2024-09-16T15:43:15.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:15.701+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f489c1f5f20 con 0x7f489c15f5a0 2024-09-16T15:43:16.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.466+0000 7f4896000640 1 -- 172.21.15.60:0/1495658298 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v263) ==== 149+0+0 (secure 0 0 0) 0x7f4898063450 con 0x7f489c15f5a0 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4880078970 msgr2=0x7f488007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4880078970 0x7f488007ae30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4890033000 tx=0x7f48900046c0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 msgr2=0x7f489c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 0x7f489c0cff20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f489800da90 tx=0x7f489800df50 comp rx=0 tx=0).stop 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 shutdown_connections 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4880078970 0x7f488007ae30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f489c15f5a0 0x7f489c0cff20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f489c0ccf00 0x7f489c1f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 --2- 172.21.15.60:0/1495658298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f489c0ad270 0x7f489c156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.469+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 >> 172.21.15.60:0/1495658298 conn(0x7f489c0b1950 msgr2=0x7f489c09d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.470+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 shutdown_connections 2024-09-16T15:43:16.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.470+0000 7f48a4800640 1 -- 172.21.15.60:0/1495658298 wait complete. 2024-09-16T15:43:16.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:43:16.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 -- 172.21.15.60:0/1646099411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c0ad270 msgr2=0x7f706c073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:16.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 --2- 172.21.15.60:0/1646099411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c0ad270 0x7f706c073fb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f706c0723f0 tx=0x7f706402a9a0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 -- 172.21.15.60:0/1646099411 shutdown_connections 2024-09-16T15:43:16.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 --2- 172.21.15.60:0/1646099411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c0ad270 0x7f706c073fb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 --2- 172.21.15.60:0/1646099411 >> v2:172.21.15.100:3300/0 conn(0x7f706c0ccf00 0x7f706c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 --2- 172.21.15.60:0/1646099411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f706c15f5a0 0x7f706c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 -- 172.21.15.60:0/1646099411 >> 172.21.15.60:0/1646099411 conn(0x7f706c0b1950 msgr2=0x7f706c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:16.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 -- 172.21.15.60:0/1646099411 shutdown_connections 2024-09-16T15:43:16.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.649+0000 7f7074600640 1 -- 172.21.15.60:0/1646099411 wait complete. 2024-09-16T15:43:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.651+0000 7f7074600640 1 Processor -- start 2024-09-16T15:43:16.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.651+0000 7f7074600640 1 -- start start 2024-09-16T15:43:16.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7074600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706c0ad270 0x7f706c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:16.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7074600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f706c0ccf00 0x7f706c1465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7074600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c15f5a0 0x7f706c154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7074600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f706c06f160 con 0x7f706c15f5a0 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7074600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f706c074780 con 0x7f706c0ad270 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7074600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f706c06f2e0 con 0x7f706c0ccf00 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7072600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c15f5a0 0x7f706c154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7072600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c15f5a0 0x7f706c154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42326/0 (socket says 172.21.15.60:42326) 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7072600640 1 -- 172.21.15.60:0/1812109651 learned_addr learned my addr 172.21.15.60:0/1812109651 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.652+0000 7f7071200640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f706c0ccf00 0x7f706c1465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7071c00640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706c0ad270 0x7f706c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:16.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7072600640 1 -- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f706c0ccf00 msgr2=0x7f706c1465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:16.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7072600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f706c0ccf00 0x7f706c1465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7072600640 1 -- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706c0ad270 msgr2=0x7f706c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:16.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7072600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706c0ad270 0x7f706c0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7072600640 1 -- 172.21.15.60:0/1812109651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c150ed0 con 0x7f706c15f5a0 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7071200640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f706c0ccf00 0x7f706c1465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7071c00640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706c0ad270 0x7f706c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7072600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c15f5a0 0x7f706c154ff0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f706402af90 tx=0x7f7064003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7069600640 1 -- 172.21.15.60:0/1812109651 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064046070 con 0x7f706c15f5a0 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7069600640 1 -- 172.21.15.60:0/1812109651 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f706402f600 con 0x7f706c15f5a0 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7069600640 1 -- 172.21.15.60:0/1812109651 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064033050 con 0x7f706c15f5a0 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.653+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706c1511b0 con 0x7f706c15f5a0 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.654+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706c151770 con 0x7f706c15f5a0 2024-09-16T15:43:16.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.655+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706c06f870 con 0x7f706c15f5a0 2024-09-16T15:43:16.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.656+0000 7f7069600640 1 -- 172.21.15.60:0/1812109651 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f7064049070 con 0x7f706c15f5a0 2024-09-16T15:43:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.657+0000 7f7069600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f704c078970 0x7f704c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.657+0000 7f7069600640 1 -- 172.21.15.60:0/1812109651 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7f70640bb7b0 con 0x7f706c15f5a0 2024-09-16T15:43:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.657+0000 7f7071c00640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f704c078970 0x7f704c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:16.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.658+0000 7f7071c00640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f704c078970 0x7f704c07ae30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f706000bbc0 tx=0x7f7060005ef0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:16.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.661+0000 7f7069600640 1 -- 172.21.15.60:0/1812109651 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7064083c40 con 0x7f706c15f5a0 2024-09-16T15:43:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.855+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f706c157b90 con 0x7f706c15f5a0 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.856+0000 7f7069600640 1 -- 172.21.15.60:0/1812109651 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+1998 (secure 0 0 0) 0x7f706408a080 con 0x7f706c15f5a0 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:43:16.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:43:16.858 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:43:16.859 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:43:16.859 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f704c078970 msgr2=0x7f704c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f704c078970 0x7f704c07ae30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f706000bbc0 tx=0x7f7060005ef0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c15f5a0 msgr2=0x7f706c154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c15f5a0 0x7f706c154ff0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f706402af90 tx=0x7f7064003ef0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 shutdown_connections 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f704c078970 0x7f704c07ae30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f706c15f5a0 0x7f706c154ff0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f706c0ccf00 0x7f706c1465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 --2- 172.21.15.60:0/1812109651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f706c0ad270 0x7f706c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.858+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 >> 172.21.15.60:0/1812109651 conn(0x7f706c0b1950 msgr2=0x7f706c09c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.859+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 shutdown_connections 2024-09-16T15:43:16.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:16.859+0000 7f7074600640 1 -- 172.21.15.60:0/1812109651 wait complete. 2024-09-16T15:43:16.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:43:17.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 -- 172.21.15.60:0/647343342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79180ccb00 msgr2=0x7f791815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:17.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 --2- 172.21.15.60:0/647343342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79180ccb00 0x7f791815f9f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f79180d0cd0 tx=0x7f791002a980 comp rx=0 tx=0).stop 2024-09-16T15:43:17.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 -- 172.21.15.60:0/647343342 shutdown_connections 2024-09-16T15:43:17.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 --2- 172.21.15.60:0/647343342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79180ccb00 0x7f791815f9f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 --2- 172.21.15.60:0/647343342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f791806efa0 0x7f79180cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 --2- 172.21.15.60:0/647343342 >> v2:172.21.15.100:3300/0 conn(0x7f7918145750 0x7f7918073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 -- 172.21.15.60:0/647343342 >> 172.21.15.60:0/647343342 conn(0x7f79180b1840 msgr2=0x7f79180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:17.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 -- 172.21.15.60:0/647343342 shutdown_connections 2024-09-16T15:43:17.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.037+0000 7f791d200640 1 -- 172.21.15.60:0/647343342 wait complete. 2024-09-16T15:43:17.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.039+0000 7f791d200640 1 Processor -- start 2024-09-16T15:43:17.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.039+0000 7f791d200640 1 -- start start 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f791d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f791806efa0 0x7f79180c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f791d200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79180ccb00 0x7f79181442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f791d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7918145750 0x7f7918134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f791d200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f79181600b0 con 0x7f791806efa0 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f791d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f791815ff30 con 0x7f7918145750 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f791d200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7918160230 con 0x7f79180ccb00 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f7915e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f791806efa0 0x7f79180c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f7915e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f791806efa0 0x7f79180c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42332/0 (socket says 172.21.15.60:42332) 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f7915e00640 1 -- 172.21.15.60:0/2129249243 learned_addr learned my addr 172.21.15.60:0/2129249243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f7916800640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7918145750 0x7f7918134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:17.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.040+0000 7f7915400640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79180ccb00 0x7f79181442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:17.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f7915e00640 1 -- 172.21.15.60:0/2129249243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79180ccb00 msgr2=0x7f79181442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:17.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f7915e00640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79180ccb00 0x7f79181442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f7915e00640 1 -- 172.21.15.60:0/2129249243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7918145750 msgr2=0x7f7918134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:17.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f7915e00640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7918145750 0x7f7918134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f7915e00640 1 -- 172.21.15.60:0/2129249243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79180c5150 con 0x7f791806efa0 2024-09-16T15:43:17.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f7915e00640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f791806efa0 0x7f79180c4ab0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f790800e7c0 tx=0x7f790800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:17.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f790e000640 1 -- 172.21.15.60:0/2129249243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800ccc0 con 0x7f791806efa0 2024-09-16T15:43:17.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f790e000640 1 -- 172.21.15.60:0/2129249243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7908010430 con 0x7f791806efa0 2024-09-16T15:43:17.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f790e000640 1 -- 172.21.15.60:0/2129249243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908010860 con 0x7f791806efa0 2024-09-16T15:43:17.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79180c5430 con 0x7f791806efa0 2024-09-16T15:43:17.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.041+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79180c59f0 con 0x7f791806efa0 2024-09-16T15:43:17.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.043+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7918160710 con 0x7f791806efa0 2024-09-16T15:43:17.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.044+0000 7f790e000640 1 -- 172.21.15.60:0/2129249243 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f7908010a00 con 0x7f791806efa0 2024-09-16T15:43:17.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.045+0000 7f790e000640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f78f8078970 0x7f78f807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:17.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.045+0000 7f790e000640 1 -- 172.21.15.60:0/2129249243 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7f790809b3f0 con 0x7f791806efa0 2024-09-16T15:43:17.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.045+0000 7f7915400640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f78f8078970 0x7f78f807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:17.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.046+0000 7f7915400640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f78f8078970 0x7f78f807ae30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79181e8a30 tx=0x7f7904005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:17.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.049+0000 7f790e000640 1 -- 172.21.15.60:0/2129249243 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f79080632b0 con 0x7f791806efa0 2024-09-16T15:43:17.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.247+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f79180adda0 con 0x7f791806efa0 2024-09-16T15:43:17.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.251+0000 7f790e000640 1 -- 172.21.15.60:0/2129249243 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f7908067ce0 con 0x7f791806efa0 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f78f8078970 msgr2=0x7f78f807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f78f8078970 0x7f78f807ae30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79181e8a30 tx=0x7f7904005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f791806efa0 msgr2=0x7f79180c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f791806efa0 0x7f79180c4ab0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f790800e7c0 tx=0x7f790800ec80 comp rx=0 tx=0).stop 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 shutdown_connections 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f78f8078970 0x7f78f807ae30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7918145750 0x7f7918134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79180ccb00 0x7f79181442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.253+0000 7f791d200640 1 --2- 172.21.15.60:0/2129249243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f791806efa0 0x7f79180c4ab0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.254+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 >> 172.21.15.60:0/2129249243 conn(0x7f79180b1840 msgr2=0x7f79180ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.254+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 shutdown_connections 2024-09-16T15:43:17.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.254+0000 7f791d200640 1 -- 172.21.15.60:0/2129249243 wait complete. 2024-09-16T15:43:17.270 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:43:17.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:43:17.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:43:17.450 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:43:17.450 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:17.450 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:17.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:43:17.486 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:17.486 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:17.523 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:43:17.523 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:17.523 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:17.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:43:17.562 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:17.562 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:17.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:43:17.599 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:43:17.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:17.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.761+0000 7f21b7c00640 1 -- 172.21.15.60:0/1647768855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 msgr2=0x7f21b00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.761+0000 7f21b7c00640 1 --2- 172.21.15.60:0/1647768855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 0x7f21b00b2440 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005fd0 tx=0x7f21a402a540 comp rx=0 tx=0).stop 2024-09-16T15:43:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.761+0000 7f21b7c00640 1 -- 172.21.15.60:0/1647768855 shutdown_connections 2024-09-16T15:43:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.761+0000 7f21b7c00640 1 --2- 172.21.15.60:0/1647768855 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ccf00 0x7f21b00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.761+0000 7f21b7c00640 1 --2- 172.21.15.60:0/1647768855 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 0x7f21b00b2440 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.761+0000 7f21b7c00640 1 -- 172.21.15.60:0/1647768855 >> 172.21.15.60:0/1647768855 conn(0x7f21b00b1950 msgr2=0x7f21b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.761+0000 7f21b7c00640 1 -- 172.21.15.60:0/1647768855 shutdown_connections 2024-09-16T15:43:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.761+0000 7f21b7c00640 1 -- 172.21.15.60:0/1647768855 wait complete. 2024-09-16T15:43:17.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.763+0000 7f21b7c00640 1 Processor -- start 2024-09-16T15:43:17.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.763+0000 7f21b7c00640 1 -- start start 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b7c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ad270 0x7f21b01f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b7c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ccf00 0x7f21b00b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b7c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 0x7f21b0153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b7c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f21b006f2e0 con 0x7f21b015f5a0 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b7c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21b006f160 con 0x7f21b00ccf00 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b7c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f21b006f460 con 0x7f21b00ad270 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b5c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 0x7f21b0153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b5200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ad270 0x7f21b01f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:17.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b5c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 0x7f21b0153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42350/0 (socket says 172.21.15.60:42350) 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b5c00640 1 -- 172.21.15.60:0/2161960864 learned_addr learned my addr 172.21.15.60:0/2161960864 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21afe00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ccf00 0x7f21b00b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b5c00640 1 -- 172.21.15.60:0/2161960864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ad270 msgr2=0x7f21b01f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b5c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ad270 0x7f21b01f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b5c00640 1 -- 172.21.15.60:0/2161960864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ccf00 msgr2=0x7f21b00b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.764+0000 7f21b5c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ccf00 0x7f21b00b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.765+0000 7f21b5c00640 1 -- 172.21.15.60:0/2161960864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b00b3290 con 0x7f21b015f5a0 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.765+0000 7f21b5c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 0x7f21b0153560 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f21a800c770 tx=0x7f21a800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.765+0000 7f21acc00640 1 -- 172.21.15.60:0/2161960864 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8017070 con 0x7f21b015f5a0 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.765+0000 7f21acc00640 1 -- 172.21.15.60:0/2161960864 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21a8005070 con 0x7f21b015f5a0 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.765+0000 7f21acc00640 1 -- 172.21.15.60:0/2161960864 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a80054c0 con 0x7f21b015f5a0 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.765+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b00b3530 con 0x7f21b015f5a0 2024-09-16T15:43:17.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.765+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b00b3dd0 con 0x7f21b015f5a0 2024-09-16T15:43:17.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.766+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b006fce0 con 0x7f21b015f5a0 2024-09-16T15:43:17.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.767+0000 7f21acc00640 1 -- 172.21.15.60:0/2161960864 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f21a801a070 con 0x7f21b015f5a0 2024-09-16T15:43:17.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.768+0000 7f21acc00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f21900789a0 0x7f219007ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:17.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.768+0000 7f21acc00640 1 -- 172.21.15.60:0/2161960864 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7f21a8099fe0 con 0x7f21b015f5a0 2024-09-16T15:43:17.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.768+0000 7f21b5200640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f21900789a0 0x7f219007ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:17.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.769+0000 7f21b5200640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f21900789a0 0x7f219007ae60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f21a4033000 tx=0x7f21a4004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:17.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:17.772+0000 7f21acc00640 1 -- 172.21.15.60:0/2161960864 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f21a8061ec0 con 0x7f21b015f5a0 2024-09-16T15:43:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.006+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21b00743a0 con 0x7f21b015f5a0 2024-09-16T15:43:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.006+0000 7f21acc00640 1 -- 172.21.15.60:0/2161960864 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+3917 (secure 0 0 0) 0x7f21a80668f0 con 0x7f21b015f5a0 2024-09-16T15:43:18.008 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:18.008 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":50,"btime":"2024-09-16T15:43:11:038106+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2096551376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2096551376},{"type":"v1","addr":"172.21.15.100:6837","nonce":2096551376}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/2980284352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2980284352},{"type":"v1","addr":"172.21.15.100:6836","nonce":2980284352}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1592560510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1592560510},{"type":"v1","addr":"172.21.15.60:6833","nonce":1592560510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2563710828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":2563710828},{"type":"v1","addr":"172.21.15.60:6835","nonce":2563710828}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[]} 2024-09-16T15:43:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 50 2024-09-16T15:43:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f21900789a0 msgr2=0x7f219007ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f21900789a0 0x7f219007ae60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f21a4033000 tx=0x7f21a4004730 comp rx=0 tx=0).stop 2024-09-16T15:43:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 msgr2=0x7f21b0153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 0x7f21b0153560 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f21a800c770 tx=0x7f21a800cc30 comp rx=0 tx=0).stop 2024-09-16T15:43:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 shutdown_connections 2024-09-16T15:43:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f21900789a0 0x7f219007ae60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015f5a0 0x7f21b0153560 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:18.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ccf00 0x7f21b00b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 --2- 172.21.15.60:0/2161960864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ad270 0x7f21b01f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 >> 172.21.15.60:0/2161960864 conn(0x7f21b00b1950 msgr2=0x7f21b015e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 shutdown_connections 2024-09-16T15:43:18.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.009+0000 7f21b7c00640 1 -- 172.21.15.60:0/2161960864 wait complete. 2024-09-16T15:43:18.027 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:43:18.027 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:43:18.027 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:43:18.027 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:43:18.027 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:43:18.028 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:43:18.028 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:43:18.028 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:43:18.028 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:43:18.028 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:43:18.028 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:43:18.028 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:43:18.028 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:43:18.029 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:43:18.029 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:43:18.029 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:43:18.043 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:43:18.043 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:43:18.044 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:43:18.044 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:43:18.045 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:43:18.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:43:18.046 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:43:18.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:43:18.087 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:43:18.087 DEBUG:teuthology.parallel:result is None 2024-09-16T15:43:18.089 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:43:18.089 DEBUG:teuthology.parallel:result is None 2024-09-16T15:43:18.090 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:43:18.090 DEBUG:teuthology.parallel:result is None 2024-09-16T15:43:18.092 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:43:18.093 DEBUG:teuthology.parallel:result is None 2024-09-16T15:43:18.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:43:18.218 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:43:18.218 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:43:18.223 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:43:18.223 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.259+0000 7f7dfe600640 1 -- 172.21.15.60:0/3346531039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df8073fb0 msgr2=0x7f7df8074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.259+0000 7f7dfe600640 1 --2- 172.21.15.60:0/3346531039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df8073fb0 0x7f7df8074440 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005fd0 tx=0x7f7df002aa00 comp rx=0 tx=0).stop 2024-09-16T15:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.259+0000 7f7dfe600640 1 -- 172.21.15.60:0/3346531039 shutdown_connections 2024-09-16T15:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.259+0000 7f7dfe600640 1 --2- 172.21.15.60:0/3346531039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df8073fb0 0x7f7df8074440 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.259+0000 7f7dfe600640 1 --2- 172.21.15.60:0/3346531039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7df80ccf00 0x7f7df80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.259+0000 7f7dfe600640 1 -- 172.21.15.60:0/3346531039 >> 172.21.15.60:0/3346531039 conn(0x7f7df80b1950 msgr2=0x7f7df80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:18.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.259+0000 7f7dfe600640 1 -- 172.21.15.60:0/3346531039 shutdown_connections 2024-09-16T15:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.259+0000 7f7dfe600640 1 -- 172.21.15.60:0/3346531039 wait complete. 2024-09-16T15:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.261+0000 7f7dfe600640 1 Processor -- start 2024-09-16T15:43:18.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.261+0000 7f7dfe600640 1 -- start start 2024-09-16T15:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7dfe600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7df8073fb0 0x7f7df80cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7dfe600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df80ccf00 0x7f7df81f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7dfe600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df815f5a0 0x7f7df80ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:18.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7dfe600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7df806f2e0 con 0x7f7df80ccf00 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7dfe600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7df806f160 con 0x7f7df815f5a0 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7dfe600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7df806f460 con 0x7f7df8073fb0 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df80ccf00 0x7f7df81f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df80ccf00 0x7f7df81f87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42382/0 (socket says 172.21.15.60:42382) 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df6a00640 1 -- 172.21.15.60:0/1342594922 learned_addr learned my addr 172.21.15.60:0/1342594922 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df7400640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7df8073fb0 0x7f7df80cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df6a00640 1 -- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7df8073fb0 msgr2=0x7f7df80cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df7e00640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df815f5a0 0x7f7df80ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df6a00640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7df8073fb0 0x7f7df80cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df6a00640 1 -- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df815f5a0 msgr2=0x7f7df80ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df6a00640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df815f5a0 0x7f7df80ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df6a00640 1 -- 172.21.15.60:0/1342594922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df81f8d10 con 0x7f7df80ccf00 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df7400640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7df8073fb0 0x7f7df80cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:18.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.262+0000 7f7df7e00640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df815f5a0 0x7f7df80ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:18.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.263+0000 7f7df6a00640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df80ccf00 0x7f7df81f87d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7de400e9f0 tx=0x7f7de400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:18.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.263+0000 7f7def400640 1 -- 172.21.15.60:0/1342594922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de400cd80 con 0x7f7df80ccf00 2024-09-16T15:43:18.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.263+0000 7f7def400640 1 -- 172.21.15.60:0/1342594922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7de4010430 con 0x7f7df80ccf00 2024-09-16T15:43:18.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.263+0000 7f7def400640 1 -- 172.21.15.60:0/1342594922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4010840 con 0x7f7df80ccf00 2024-09-16T15:43:18.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.263+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df81f4bc0 con 0x7f7df80ccf00 2024-09-16T15:43:18.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.263+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df81f5180 con 0x7f7df80ccf00 2024-09-16T15:43:18.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.264+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df806f9f0 con 0x7f7df80ccf00 2024-09-16T15:43:18.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.265+0000 7f7def400640 1 -- 172.21.15.60:0/1342594922 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f7de40109e0 con 0x7f7df80ccf00 2024-09-16T15:43:18.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.266+0000 7f7def400640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dd8078970 0x7f7dd807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:18.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.266+0000 7f7def400640 1 -- 172.21.15.60:0/1342594922 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5502+0+0 (secure 0 0 0) 0x7f7de409b630 con 0x7f7df80ccf00 2024-09-16T15:43:18.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.267+0000 7f7df7400640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dd8078970 0x7f7dd807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:18.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.267+0000 7f7df7400640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dd8078970 0x7f7dd807ae30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7df809aa70 tx=0x7f7de8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:18.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.271+0000 7f7def400640 1 -- 172.21.15.60:0/1342594922 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7df81f8d10 con 0x7f7df80ccf00 2024-09-16T15:43:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:18.483+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7df809aaa0 con 0x7f7df80ccf00 2024-09-16T15:43:19.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.497+0000 7f7def400640 1 -- 172.21.15.60:0/1342594922 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) ==== 97+0+0 (secure 0 0 0) 0x7f7de4063c10 con 0x7f7df80ccf00 2024-09-16T15:43:19.499 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:43:19.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dd8078970 msgr2=0x7f7dd807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:19.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dd8078970 0x7f7dd807ae30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7df809aa70 tx=0x7f7de8009040 comp rx=0 tx=0).stop 2024-09-16T15:43:19.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df80ccf00 msgr2=0x7f7df81f87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:19.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df80ccf00 0x7f7df81f87d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7de400e9f0 tx=0x7f7de400eeb0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 shutdown_connections 2024-09-16T15:43:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dd8078970 0x7f7dd807ae30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7df815f5a0 0x7f7df80ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7df80ccf00 0x7f7df81f87d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 --2- 172.21.15.60:0/1342594922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7df8073fb0 0x7f7df80cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 >> 172.21.15.60:0/1342594922 conn(0x7f7df80b1950 msgr2=0x7f7df809c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 shutdown_connections 2024-09-16T15:43:19.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.501+0000 7f7dfe600640 1 -- 172.21.15.60:0/1342594922 wait complete. 2024-09-16T15:43:19.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:43:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.686+0000 7ffbb2a00640 1 -- 172.21.15.60:0/459243286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac073e00 msgr2=0x7ffbac168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.686+0000 7ffbb2a00640 1 --2- 172.21.15.60:0/459243286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac073e00 0x7ffbac168e10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0b4890 tx=0x7ffba402a800 comp rx=0 tx=0).stop 2024-09-16T15:43:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.687+0000 7ffbb2a00640 1 -- 172.21.15.60:0/459243286 shutdown_connections 2024-09-16T15:43:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.687+0000 7ffbb2a00640 1 --2- 172.21.15.60:0/459243286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac073e00 0x7ffbac168e10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.687+0000 7ffbb2a00640 1 -- 172.21.15.60:0/459243286 >> 172.21.15.60:0/459243286 conn(0x7ffbac0b1950 msgr2=0x7ffbac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.687+0000 7ffbb2a00640 1 -- 172.21.15.60:0/459243286 shutdown_connections 2024-09-16T15:43:19.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.687+0000 7ffbb2a00640 1 -- 172.21.15.60:0/459243286 wait complete. 2024-09-16T15:43:19.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.688+0000 7ffbb2a00640 1 Processor -- start 2024-09-16T15:43:19.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.688+0000 7ffbb2a00640 1 -- start start 2024-09-16T15:43:19.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.689+0000 7ffbb2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac073e00 0x7ffbac0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.689+0000 7ffbb2a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbac0af3d0 0x7ffbac0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.689+0000 7ffbb2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac161550 0x7ffbac09bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.689+0000 7ffbb2a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1694d0 con 0x7ffbac161550 2024-09-16T15:43:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.689+0000 7ffbb2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac169350 con 0x7ffbac073e00 2024-09-16T15:43:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.689+0000 7ffbb2a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffbac169650 con 0x7ffbac0af3d0 2024-09-16T15:43:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac161550 0x7ffbac09bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac161550 0x7ffbac09bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36426/0 (socket says 172.21.15.60:36426) 2024-09-16T15:43:19.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 -- 172.21.15.60:0/2401897202 learned_addr learned my addr 172.21.15.60:0/2401897202 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:19.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffba3400640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbac0af3d0 0x7ffbac0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:19.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 -- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbac0af3d0 msgr2=0x7ffbac0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbabe00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac073e00 0x7ffbac0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbac0af3d0 0x7ffbac0c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 -- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac073e00 msgr2=0x7ffbac0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac073e00 0x7ffbac0afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 -- 172.21.15.60:0/2401897202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbac09ab30 con 0x7ffbac161550 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffba3400640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbac0af3d0 0x7ffbac0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbabe00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac073e00 0x7ffbac0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.690+0000 7ffbb0a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac161550 0x7ffbac09bbb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ffba4035040 tx=0x7ffba4033e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.691+0000 7ffba8c00640 1 -- 172.21.15.60:0/2401897202 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4042020 con 0x7ffbac161550 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.691+0000 7ffba8c00640 1 -- 172.21.15.60:0/2401897202 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffba4004040 con 0x7ffbac161550 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.691+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbac09ae10 con 0x7ffbac161550 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.691+0000 7ffba8c00640 1 -- 172.21.15.60:0/2401897202 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4037040 con 0x7ffbac161550 2024-09-16T15:43:19.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.691+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbac09b6b0 con 0x7ffbac161550 2024-09-16T15:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.692+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbac06ef50 con 0x7ffbac161550 2024-09-16T15:43:19.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.693+0000 7ffba8c00640 1 -- 172.21.15.60:0/2401897202 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7ffba4045020 con 0x7ffbac161550 2024-09-16T15:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.695+0000 7ffba8c00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb88078970 0x7ffb8807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.695+0000 7ffba8c00640 1 -- 172.21.15.60:0/2401897202 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7ffba40bb450 con 0x7ffbac161550 2024-09-16T15:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.695+0000 7ffbabe00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb88078970 0x7ffb8807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:19.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.696+0000 7ffbabe00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb88078970 0x7ffb8807ae30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c001220 tx=0x7ffb9c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:19.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.699+0000 7ffba8c00640 1 -- 172.21.15.60:0/2401897202 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffba404c070 con 0x7ffbac161550 2024-09-16T15:43:19.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.968+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ffbac1628d0 con 0x7ffbac161550 2024-09-16T15:43:19.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.969+0000 7ffba8c00640 1 -- 172.21.15.60:0/2401897202 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7ffba402f070 con 0x7ffbac161550 2024-09-16T15:43:19.970 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:19.970 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:43:19.970 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:43:19.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.970 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:43:19.970 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:43:19.970 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:19.971 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:19.973 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:43:19.974 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.977 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:43:19.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:43:19.979 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:43:19.980 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.981 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:19.983 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:19.983 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:43:19.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb88078970 msgr2=0x7ffb8807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:19.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb88078970 0x7ffb8807ae30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c001220 tx=0x7ffb9c009040 comp rx=0 tx=0).stop 2024-09-16T15:43:19.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac161550 msgr2=0x7ffbac09bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac161550 0x7ffbac09bbb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ffba4035040 tx=0x7ffba4033e80 comp rx=0 tx=0).stop 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 shutdown_connections 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb88078970 0x7ffb8807ae30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbac161550 0x7ffbac09bbb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbac0af3d0 0x7ffbac0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 --2- 172.21.15.60:0/2401897202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbac073e00 0x7ffbac0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 >> 172.21.15.60:0/2401897202 conn(0x7ffbac0b1950 msgr2=0x7ffbac15f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 shutdown_connections 2024-09-16T15:43:19.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:19.971+0000 7ffbb2a00640 1 -- 172.21.15.60:0/2401897202 wait complete. 2024-09-16T15:43:19.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.147+0000 7f83a8a00640 1 -- 172.21.15.60:0/1705268755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 msgr2=0x7f83a406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.147+0000 7f83a8a00640 1 --2- 172.21.15.60:0/1705268755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 0x7f83a406ef50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f83a40752e0 tx=0x7f839402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.147+0000 7f83a8a00640 1 -- 172.21.15.60:0/1705268755 shutdown_connections 2024-09-16T15:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.147+0000 7f83a8a00640 1 --2- 172.21.15.60:0/1705268755 >> v2:172.21.15.100:3300/0 conn(0x7f83a40af3d0 0x7f83a406f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.147+0000 7f83a8a00640 1 --2- 172.21.15.60:0/1705268755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 0x7f83a406ef50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.147+0000 7f83a8a00640 1 -- 172.21.15.60:0/1705268755 >> 172.21.15.60:0/1705268755 conn(0x7f83a40b1950 msgr2=0x7f83a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.147+0000 7f83a8a00640 1 -- 172.21.15.60:0/1705268755 shutdown_connections 2024-09-16T15:43:20.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.147+0000 7f83a8a00640 1 -- 172.21.15.60:0/1705268755 wait complete. 2024-09-16T15:43:20.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.149+0000 7f83a8a00640 1 Processor -- start 2024-09-16T15:43:20.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.149+0000 7f83a8a00640 1 -- start start 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a8a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a406f9d0 0x7f83a414c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a40af3d0 0x7f83a40ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 0x7f83a409d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a8a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4073f80 con 0x7f83a4161550 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4073e00 con 0x7f83a40af3d0 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a8a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f83a4074100 con 0x7f83a406f9d0 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 0x7f83a409d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 0x7f83a409d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36454/0 (socket says 172.21.15.60:36454) 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a2800640 1 -- 172.21.15.60:0/3258304461 learned_addr learned my addr 172.21.15.60:0/3258304461 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:20.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f8399400640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a40af3d0 0x7f83a40ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.150+0000 7f83a1e00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a406f9d0 0x7f83a414c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f83a2800640 1 -- 172.21.15.60:0/3258304461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a406f9d0 msgr2=0x7f83a414c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f83a2800640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a406f9d0 0x7f83a414c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f83a2800640 1 -- 172.21.15.60:0/3258304461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a40af3d0 msgr2=0x7f83a40ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f83a2800640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a40af3d0 0x7f83a40ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f83a2800640 1 -- 172.21.15.60:0/3258304461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a414cb90 con 0x7f83a4161550 2024-09-16T15:43:20.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f83a2800640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 0x7f83a409d6d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f839c00c780 tx=0x7f839c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f839aa00640 1 -- 172.21.15.60:0/3258304461 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c004b60 con 0x7f83a4161550 2024-09-16T15:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f839aa00640 1 -- 172.21.15.60:0/3258304461 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f839c004df0 con 0x7f83a4161550 2024-09-16T15:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f839aa00640 1 -- 172.21.15.60:0/3258304461 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c00aa70 con 0x7f83a4161550 2024-09-16T15:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.151+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a409ae90 con 0x7f83a4161550 2024-09-16T15:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.152+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a409b450 con 0x7f83a4161550 2024-09-16T15:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.153+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a41626c0 con 0x7f83a4161550 2024-09-16T15:43:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.154+0000 7f839aa00640 1 -- 172.21.15.60:0/3258304461 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f839c00ac10 con 0x7f83a4161550 2024-09-16T15:43:20.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.155+0000 7f839aa00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8380078970 0x7f838007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:20.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.155+0000 7f839aa00640 1 -- 172.21.15.60:0/3258304461 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f839c09aaa0 con 0x7f83a4161550 2024-09-16T15:43:20.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.155+0000 7f83a1e00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8380078970 0x7f838007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:20.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.156+0000 7f83a1e00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8380078970 0x7f838007ae30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8394033000 tx=0x7f8394004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:20.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.159+0000 7f839aa00640 1 -- 172.21.15.60:0/3258304461 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f839c020070 con 0x7f83a4161550 2024-09-16T15:43:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.357+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f83a415e8c0 con 0x7f83a4161550 2024-09-16T15:43:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.358+0000 7f839aa00640 1 -- 172.21.15.60:0/3258304461 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f839c062ad0 con 0x7f83a4161550 2024-09-16T15:43:20.359 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:20.359 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:43:20.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:20.359 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:43:20.359 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:43:20.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:20.359 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:43:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8380078970 msgr2=0x7f838007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8380078970 0x7f838007ae30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8394033000 tx=0x7f8394004730 comp rx=0 tx=0).stop 2024-09-16T15:43:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 msgr2=0x7f83a409d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 0x7f83a409d6d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f839c00c780 tx=0x7f839c00cc40 comp rx=0 tx=0).stop 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 shutdown_connections 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8380078970 0x7f838007ae30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83a4161550 0x7f83a409d6d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83a40af3d0 0x7f83a40ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 --2- 172.21.15.60:0/3258304461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83a406f9d0 0x7f83a414c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 >> 172.21.15.60:0/3258304461 conn(0x7f83a40b1950 msgr2=0x7f83a415ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 shutdown_connections 2024-09-16T15:43:20.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.360+0000 7f83a8a00640 1 -- 172.21.15.60:0/3258304461 wait complete. 2024-09-16T15:43:20.377 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:43:20.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:43:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.542+0000 7f0d7b200640 1 -- 172.21.15.60:0/2144698761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7415e710 msgr2=0x7f0d740d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.542+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2144698761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7415e710 0x7f0d740d4a20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009830 tx=0x7f0d6802a520 comp rx=0 tx=0).stop 2024-09-16T15:43:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.542+0000 7f0d7b200640 1 -- 172.21.15.60:0/2144698761 shutdown_connections 2024-09-16T15:43:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.542+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2144698761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d7415eaf0 0x7f0d7415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.542+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2144698761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7415e710 0x7f0d740d4a20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.542+0000 7f0d7b200640 1 -- 172.21.15.60:0/2144698761 >> 172.21.15.60:0/2144698761 conn(0x7f0d740d3f00 msgr2=0x7f0d740d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.542+0000 7f0d7b200640 1 -- 172.21.15.60:0/2144698761 shutdown_connections 2024-09-16T15:43:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.542+0000 7f0d7b200640 1 -- 172.21.15.60:0/2144698761 wait complete. 2024-09-16T15:43:20.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.544+0000 7f0d7b200640 1 Processor -- start 2024-09-16T15:43:20.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.544+0000 7f0d7b200640 1 -- start start 2024-09-16T15:43:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d7b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d740d08d0 0x7f0d741f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d7b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7415e710 0x7f0d740b9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d7b200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d7415eaf0 0x7f0d740c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d7b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0d740d10f0 con 0x7f0d740d08d0 2024-09-16T15:43:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d7b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d740d0f70 con 0x7f0d7415e710 2024-09-16T15:43:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d7b200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0d740d1270 con 0x7f0d7415eaf0 2024-09-16T15:43:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d79e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d740d08d0 0x7f0d741f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d79e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d740d08d0 0x7f0d741f7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36462/0 (socket says 172.21.15.60:36462) 2024-09-16T15:43:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d79e00640 1 -- 172.21.15.60:0/2254594976 learned_addr learned my addr 172.21.15.60:0/2254594976 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d79400640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7415e710 0x7f0d740b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.545+0000 7f0d7a800640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d7415eaf0 0x7f0d740c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:20.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d79e00640 1 -- 172.21.15.60:0/2254594976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d7415eaf0 msgr2=0x7f0d740c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:20.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d79e00640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d7415eaf0 0x7f0d740c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d79e00640 1 -- 172.21.15.60:0/2254594976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7415e710 msgr2=0x7f0d740b9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:20.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d79e00640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7415e710 0x7f0d740b9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:20.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d79e00640 1 -- 172.21.15.60:0/2254594976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d740b9bb0 con 0x7f0d740d08d0 2024-09-16T15:43:20.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d79e00640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d740d08d0 0x7f0d741f7a30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c00 tx=0x7f0d68032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:20.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d6e000640 1 -- 172.21.15.60:0/2254594976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68040070 con 0x7f0d740d08d0 2024-09-16T15:43:20.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d6e000640 1 -- 172.21.15.60:0/2254594976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d680038e0 con 0x7f0d740d08d0 2024-09-16T15:43:20.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d740c9b40 con 0x7f0d740d08d0 2024-09-16T15:43:20.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.546+0000 7f0d6e000640 1 -- 172.21.15.60:0/2254594976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68003d30 con 0x7f0d740d08d0 2024-09-16T15:43:20.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.547+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d740ca100 con 0x7f0d740d08d0 2024-09-16T15:43:20.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.548+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d740d4a20 con 0x7f0d740d08d0 2024-09-16T15:43:20.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.549+0000 7f0d6e000640 1 -- 172.21.15.60:0/2254594976 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f0d68039040 con 0x7f0d740d08d0 2024-09-16T15:43:20.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.550+0000 7f0d6e000640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d580789a0 0x7f0d5807ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:20.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.550+0000 7f0d6e000640 1 -- 172.21.15.60:0/2254594976 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4857+0+0 (secure 0 0 0) 0x7f0d680bbb30 con 0x7f0d740d08d0 2024-09-16T15:43:20.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.550+0000 7f0d79400640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d580789a0 0x7f0d5807ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:20.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.551+0000 7f0d79400640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d580789a0 0x7f0d5807ae60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d7406d800 tx=0x7f0d64005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:20.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.554+0000 7f0d6e000640 1 -- 172.21.15.60:0/2254594976 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0d68083d10 con 0x7f0d740d08d0 2024-09-16T15:43:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:20.750+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0d740b9e80 con 0x7f0d740d08d0 2024-09-16T15:43:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.506+0000 7f0d6e000640 1 -- 172.21.15.60:0/2254594976 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f0d68088740 con 0x7f0d740d08d0 2024-09-16T15:43:21.507 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:43:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.509+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d580789a0 msgr2=0x7f0d5807ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.509+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d580789a0 0x7f0d5807ae60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d7406d800 tx=0x7f0d64005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.509+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d740d08d0 msgr2=0x7f0d741f7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.509+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d740d08d0 0x7f0d741f7a30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c00 tx=0x7f0d68032c10 comp rx=0 tx=0).stop 2024-09-16T15:43:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.510+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 shutdown_connections 2024-09-16T15:43:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.510+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d580789a0 0x7f0d5807ae60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.510+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d7415eaf0 0x7f0d740c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.510+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d7415e710 0x7f0d740b9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.510+0000 7f0d7b200640 1 --2- 172.21.15.60:0/2254594976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d740d08d0 0x7f0d741f7a30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.510+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 >> 172.21.15.60:0/2254594976 conn(0x7f0d740d3f00 msgr2=0x7f0d7415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.510+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 shutdown_connections 2024-09-16T15:43:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.510+0000 7f0d7b200640 1 -- 172.21.15.60:0/2254594976 wait complete. 2024-09-16T15:43:21.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:43:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.699+0000 7fd671400640 1 -- 172.21.15.60:0/156745371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 msgr2=0x7fd66c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.699+0000 7fd671400640 1 --2- 172.21.15.60:0/156745371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 0x7fd66c0cf070 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0b2470 tx=0x7fd66402aab0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.699+0000 7fd671400640 1 -- 172.21.15.60:0/156745371 shutdown_connections 2024-09-16T15:43:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.699+0000 7fd671400640 1 --2- 172.21.15.60:0/156745371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 0x7fd66c0cf070 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.699+0000 7fd671400640 1 --2- 172.21.15.60:0/156745371 >> v2:172.21.15.100:3300/0 conn(0x7fd66c0af190 0x7fd66c0b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.699+0000 7fd671400640 1 -- 172.21.15.60:0/156745371 >> 172.21.15.60:0/156745371 conn(0x7fd66c0b1840 msgr2=0x7fd66c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.699+0000 7fd671400640 1 -- 172.21.15.60:0/156745371 shutdown_connections 2024-09-16T15:43:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.699+0000 7fd671400640 1 -- 172.21.15.60:0/156745371 wait complete. 2024-09-16T15:43:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.700+0000 7fd671400640 1 Processor -- start 2024-09-16T15:43:21.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.701+0000 7fd671400640 1 -- start start 2024-09-16T15:43:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.701+0000 7fd671400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c0af190 0x7fd66c1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.701+0000 7fd671400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 0x7fd66c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.701+0000 7fd671400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd66c161760 0x7fd66c0b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.701+0000 7fd671400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c162310 con 0x7fd66c0cec60 2024-09-16T15:43:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.701+0000 7fd671400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c162190 con 0x7fd66c0af190 2024-09-16T15:43:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.701+0000 7fd671400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c162490 con 0x7fd66c161760 2024-09-16T15:43:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 0x7fd66c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:21.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 0x7fd66c0b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36470/0 (socket says 172.21.15.60:36470) 2024-09-16T15:43:21.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 -- 172.21.15.60:0/1624758465 learned_addr learned my addr 172.21.15.60:0/1624758465 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:21.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd66a800640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c0af190 0x7fd66c1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:21.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd66b200640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd66c161760 0x7fd66c0b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:21.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 -- 172.21.15.60:0/1624758465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd66c161760 msgr2=0x7fd66c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:21.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd66c161760 0x7fd66c0b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 -- 172.21.15.60:0/1624758465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c0af190 msgr2=0x7fd66c1527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:21.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c0af190 0x7fd66c1527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:21.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 -- 172.21.15.60:0/1624758465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c1f49f0 con 0x7fd66c0cec60 2024-09-16T15:43:21.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.702+0000 7fd661e00640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 0x7fd66c0b07f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd65400d950 tx=0x7fd65400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:21.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.703+0000 7fd663400640 1 -- 172.21.15.60:0/1624758465 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654017070 con 0x7fd66c0cec60 2024-09-16T15:43:21.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.703+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c1f4c60 con 0x7fd66c0cec60 2024-09-16T15:43:21.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.703+0000 7fd663400640 1 -- 172.21.15.60:0/1624758465 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd654009d20 con 0x7fd66c0cec60 2024-09-16T15:43:21.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.703+0000 7fd663400640 1 -- 172.21.15.60:0/1624758465 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654004280 con 0x7fd66c0cec60 2024-09-16T15:43:21.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.703+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c1f5220 con 0x7fd66c0cec60 2024-09-16T15:43:21.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.704+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c0cf070 con 0x7fd66c0cec60 2024-09-16T15:43:21.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.705+0000 7fd663400640 1 -- 172.21.15.60:0/1624758465 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fd654005020 con 0x7fd66c0cec60 2024-09-16T15:43:21.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.706+0000 7fd663400640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd648078970 0x7fd64807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:21.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.706+0000 7fd663400640 1 -- 172.21.15.60:0/1624758465 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5227+0+0 (secure 0 0 0) 0x7fd65409aed0 con 0x7fd66c0cec60 2024-09-16T15:43:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.707+0000 7fd66a800640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd648078970 0x7fd64807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.707+0000 7fd66a800640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd648078970 0x7fd64807ae30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd66c154cc0 tx=0x7fd65c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.710+0000 7fd663400640 1 -- 172.21.15.60:0/1624758465 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd654062f20 con 0x7fd66c0cec60 2024-09-16T15:43:21.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:21.907+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd66c0b3430 con 0x7fd66c0cec60 2024-09-16T15:43:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.512+0000 7fd663400640 1 -- 172.21.15.60:0/1624758465 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7fd654067950 con 0x7fd66c0cec60 2024-09-16T15:43:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.515+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd648078970 msgr2=0x7fd64807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.515+0000 7fd671400640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd648078970 0x7fd64807ae30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd66c154cc0 tx=0x7fd65c009040 comp rx=0 tx=0).stop 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.515+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 msgr2=0x7fd66c0b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.515+0000 7fd671400640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 0x7fd66c0b07f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd65400d950 tx=0x7fd65400de10 comp rx=0 tx=0).stop 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.516+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 shutdown_connections 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.516+0000 7fd671400640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd648078970 0x7fd64807ae30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.516+0000 7fd671400640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd66c161760 0x7fd66c0b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.516+0000 7fd671400640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd66c0cec60 0x7fd66c0b07f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.516+0000 7fd671400640 1 --2- 172.21.15.60:0/1624758465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd66c0af190 0x7fd66c1527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.516+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 >> 172.21.15.60:0/1624758465 conn(0x7fd66c0b1840 msgr2=0x7fd66c0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.516+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 shutdown_connections 2024-09-16T15:43:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.516+0000 7fd671400640 1 -- 172.21.15.60:0/1624758465 wait complete. 2024-09-16T15:43:22.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:43:22.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.716+0000 7f8625800640 1 -- 172.21.15.60:0/958484925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8604004d40 msgr2=0x7f86040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:22.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.716+0000 7f8625800640 1 --2- 172.21.15.60:0/958484925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8604004d40 0x7f86040051c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8618009ed0 tx=0x7f8604005830 comp rx=0 tx=0).stop 2024-09-16T15:43:22.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.716+0000 7f861f400640 1 -- 172.21.15.60:0/958484925 reap_dead start 2024-09-16T15:43:22.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.716+0000 7f8625800640 1 -- 172.21.15.60:0/958484925 shutdown_connections 2024-09-16T15:43:22.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.716+0000 7f8625800640 1 -- 172.21.15.60:0/958484925 >> 172.21.15.60:0/958484925 conn(0x7f86200b1840 msgr2=0x7f86200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:22.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.716+0000 7f8625800640 1 -- 172.21.15.60:0/958484925 shutdown_connections 2024-09-16T15:43:22.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.716+0000 7f8625800640 1 -- 172.21.15.60:0/958484925 wait complete. 2024-09-16T15:43:22.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.718+0000 7f8625800640 1 Processor -- start 2024-09-16T15:43:22.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.718+0000 7f8625800640 1 -- start start 2024-09-16T15:43:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8625800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8604004d40 0x7f8620156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8625800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86200ab440 0x7f86201f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8625800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86201f5ce0 0x7f862013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8625800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f86200737d0 con 0x7f86200ab440 2024-09-16T15:43:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8625800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8620073650 con 0x7f86201f5ce0 2024-09-16T15:43:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8625800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f86200ce0f0 con 0x7f8604004d40 2024-09-16T15:43:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8616000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86200ab440 0x7f86201f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8616000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86200ab440 0x7f86201f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36490/0 (socket says 172.21.15.60:36490) 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f8616000640 1 -- 172.21.15.60:0/4284729705 learned_addr learned my addr 172.21.15.60:0/4284729705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f861ea00640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8604004d40 0x7f8620156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.719+0000 7f861f400640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86201f5ce0 0x7f862013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8616000640 1 -- 172.21.15.60:0/4284729705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8604004d40 msgr2=0x7f8620156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8616000640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8604004d40 0x7f8620156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8616000640 1 -- 172.21.15.60:0/4284729705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86201f5ce0 msgr2=0x7f862013ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8616000640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86201f5ce0 0x7f862013ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8616000640 1 -- 172.21.15.60:0/4284729705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86201f3640 con 0x7f86200ab440 2024-09-16T15:43:22.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8616000640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86200ab440 0x7f86201f3100 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f86080066e0 tx=0x7f86080039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:22.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8617400640 1 -- 172.21.15.60:0/4284729705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608015070 con 0x7f86200ab440 2024-09-16T15:43:22.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8617400640 1 -- 172.21.15.60:0/4284729705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8608011920 con 0x7f86200ab440 2024-09-16T15:43:22.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8617400640 1 -- 172.21.15.60:0/4284729705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608011d70 con 0x7f86200ab440 2024-09-16T15:43:22.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.720+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86201f6630 con 0x7f86200ab440 2024-09-16T15:43:22.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.721+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86201f2900 con 0x7f86200ab440 2024-09-16T15:43:22.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.722+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86201f6b50 con 0x7f86200ab440 2024-09-16T15:43:22.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.723+0000 7f8617400640 1 -- 172.21.15.60:0/4284729705 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f8608018070 con 0x7f86200ab440 2024-09-16T15:43:22.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.724+0000 7f8617400640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f85fc078970 0x7f85fc07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:22.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.724+0000 7f8617400640 1 -- 172.21.15.60:0/4284729705 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5609+0+0 (secure 0 0 0) 0x7f860809b4c0 con 0x7f86200ab440 2024-09-16T15:43:22.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.724+0000 7f861ea00640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f85fc078970 0x7f85fc07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:22.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.725+0000 7f861ea00640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f85fc078970 0x7f85fc07ae30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86201f5c40 tx=0x7f8610005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:22.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.729+0000 7f8617400640 1 -- 172.21.15.60:0/4284729705 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f86201f6b50 con 0x7f86200ab440 2024-09-16T15:43:22.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:22.962+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f86201f26c0 con 0x7f86200ab440 2024-09-16T15:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.515+0000 7f8617400640 1 -- 172.21.15.60:0/4284729705 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v53) ==== 174+0+0 (secure 0 0 0) 0x7f86201f3640 con 0x7f86200ab440 2024-09-16T15:43:23.516 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 12 and data pool 13 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.518+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f85fc078970 msgr2=0x7f85fc07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.518+0000 7f8625800640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f85fc078970 0x7f85fc07ae30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86201f5c40 tx=0x7f8610005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.518+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86200ab440 msgr2=0x7f86201f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.518+0000 7f8625800640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86200ab440 0x7f86201f3100 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f86080066e0 tx=0x7f86080039c0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.519+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 shutdown_connections 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.519+0000 7f8625800640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f85fc078970 0x7f85fc07ae30 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f86201f5c40 tx=0x7f8610005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.519+0000 7f8625800640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86201f5ce0 0x7f862013ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.519+0000 7f8625800640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86200ab440 0x7f86201f3100 secure :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f86080066e0 tx=0x7f86080039c0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.519+0000 7f8625800640 1 --2- 172.21.15.60:0/4284729705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8604004d40 0x7f8620156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.519+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 >> 172.21.15.60:0/4284729705 conn(0x7f86200b1840 msgr2=0x7f862006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:23.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.520+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 shutdown_connections 2024-09-16T15:43:23.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.520+0000 7f8625800640 1 -- 172.21.15.60:0/4284729705 wait complete. 2024-09-16T15:43:23.560 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:23.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.722+0000 7f1ae1800640 1 -- 172.21.15.60:0/166214796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc0ccf00 msgr2=0x7f1adc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:23.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.722+0000 7f1ae1800640 1 --2- 172.21.15.60:0/166214796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc0ccf00 0x7f1adc0b2440 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1adc06dac0 tx=0x7f1ac402a560 comp rx=0 tx=0).stop 2024-09-16T15:43:23.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.723+0000 7f1ae1800640 1 -- 172.21.15.60:0/166214796 shutdown_connections 2024-09-16T15:43:23.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.723+0000 7f1ae1800640 1 --2- 172.21.15.60:0/166214796 >> v2:172.21.15.100:3300/0 conn(0x7f1adc073fb0 0x7f1adc074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.723+0000 7f1ae1800640 1 --2- 172.21.15.60:0/166214796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc0ccf00 0x7f1adc0b2440 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.723+0000 7f1ae1800640 1 -- 172.21.15.60:0/166214796 >> 172.21.15.60:0/166214796 conn(0x7f1adc0b1950 msgr2=0x7f1adc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:23.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.723+0000 7f1ae1800640 1 -- 172.21.15.60:0/166214796 shutdown_connections 2024-09-16T15:43:23.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.723+0000 7f1ae1800640 1 -- 172.21.15.60:0/166214796 wait complete. 2024-09-16T15:43:23.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.724+0000 7f1ae1800640 1 Processor -- start 2024-09-16T15:43:23.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.724+0000 7f1ae1800640 1 -- start start 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.725+0000 7f1ae1800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc073fb0 0x7f1adc14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.725+0000 7f1ae1800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1adc0ccf00 0x7f1adc14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.725+0000 7f1ae1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1adc15f5a0 0x7f1adc15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.725+0000 7f1ae1800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc06f2e0 con 0x7f1adc073fb0 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.725+0000 7f1ae1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc06f160 con 0x7f1adc15f5a0 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.725+0000 7f1ae1800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1adc06f460 con 0x7f1adc0ccf00 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc073fb0 0x7f1adc14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc073fb0 0x7f1adc14e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36498/0 (socket says 172.21.15.60:36498) 2024-09-16T15:43:23.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 -- 172.21.15.60:0/1422262387 learned_addr learned my addr 172.21.15.60:0/1422262387 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1ad2000640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1adc0ccf00 0x7f1adc14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adb400640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1adc15f5a0 0x7f1adc15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 -- 172.21.15.60:0/1422262387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1adc0ccf00 msgr2=0x7f1adc14c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1adc0ccf00 0x7f1adc14c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 -- 172.21.15.60:0/1422262387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1adc15f5a0 msgr2=0x7f1adc15a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1adc15f5a0 0x7f1adc15a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 -- 172.21.15.60:0/1422262387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc15b4b0 con 0x7f1adc073fb0 2024-09-16T15:43:23.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.726+0000 7f1adaa00640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc073fb0 0x7f1adc14e2b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c3d0 tx=0x7f1acc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:23.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.727+0000 7f1ad3400640 1 -- 172.21.15.60:0/1422262387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc017070 con 0x7f1adc073fb0 2024-09-16T15:43:23.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.727+0000 7f1ad3400640 1 -- 172.21.15.60:0/1422262387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1acc0053c0 con 0x7f1adc073fb0 2024-09-16T15:43:23.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.727+0000 7f1ad3400640 1 -- 172.21.15.60:0/1422262387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc003850 con 0x7f1adc073fb0 2024-09-16T15:43:23.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.727+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1adc15aef0 con 0x7f1adc073fb0 2024-09-16T15:43:23.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.727+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1adc0cdb80 con 0x7f1adc073fb0 2024-09-16T15:43:23.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.728+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1adc06f9f0 con 0x7f1adc073fb0 2024-09-16T15:43:23.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.729+0000 7f1ad3400640 1 -- 172.21.15.60:0/1422262387 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f1acc01a070 con 0x7f1adc073fb0 2024-09-16T15:43:23.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.730+0000 7f1ad3400640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ab8078970 0x7f1ab807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:23.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.730+0000 7f1ad3400640 1 -- 172.21.15.60:0/1422262387 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f1acc099f00 con 0x7f1adc073fb0 2024-09-16T15:43:23.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.730+0000 7f1ad2000640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ab8078970 0x7f1ab807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:23.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.731+0000 7f1ad2000640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ab8078970 0x7f1ab807ae30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4033000 tx=0x7f1ac4004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:23.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.735+0000 7f1ad3400640 1 -- 172.21.15.60:0/1422262387 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1acc061c60 con 0x7f1adc073fb0 2024-09-16T15:43:23.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.932+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1adc0b3800 con 0x7f1adc073fb0 2024-09-16T15:43:23.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.934+0000 7f1ad3400640 1 -- 172.21.15.60:0/1422262387 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16733 (secure 0 0 0) 0x7f1acc066690 con 0x7f1adc073fb0 2024-09-16T15:43:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:23.935 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:22.963745+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:20.760401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:21.917247+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ab8078970 msgr2=0x7f1ab807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ab8078970 0x7f1ab807ae30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4033000 tx=0x7f1ac4004730 comp rx=0 tx=0).stop 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc073fb0 msgr2=0x7f1adc14e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc073fb0 0x7f1adc14e2b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c3d0 tx=0x7f1acc00c890 comp rx=0 tx=0).stop 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 shutdown_connections 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ab8078970 0x7f1ab807ae30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1adc15f5a0 0x7f1adc15a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1adc0ccf00 0x7f1adc14c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 --2- 172.21.15.60:0/1422262387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1adc073fb0 0x7f1adc14e2b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 >> 172.21.15.60:0/1422262387 conn(0x7f1adc0b1950 msgr2=0x7f1adc153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 shutdown_connections 2024-09-16T15:43:23.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:23.936+0000 7f1ae1800640 1 -- 172.21.15.60:0/1422262387 wait complete. 2024-09-16T15:43:23.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.114+0000 7f2f1b800640 1 -- 172.21.15.60:0/3869140201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 msgr2=0x7f2f140b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.114+0000 7f2f1b800640 1 --2- 172.21.15.60:0/3869140201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 0x7f2f140b2440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005fd0 tx=0x7f2f0402a540 comp rx=0 tx=0).stop 2024-09-16T15:43:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.114+0000 7f2f1b800640 1 -- 172.21.15.60:0/3869140201 shutdown_connections 2024-09-16T15:43:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.114+0000 7f2f1b800640 1 --2- 172.21.15.60:0/3869140201 >> v2:172.21.15.100:3300/0 conn(0x7f2f140ccf00 0x7f2f140acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.114+0000 7f2f1b800640 1 --2- 172.21.15.60:0/3869140201 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 0x7f2f140b2440 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.114+0000 7f2f1b800640 1 -- 172.21.15.60:0/3869140201 >> 172.21.15.60:0/3869140201 conn(0x7f2f140b1950 msgr2=0x7f2f140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.115+0000 7f2f1b800640 1 -- 172.21.15.60:0/3869140201 shutdown_connections 2024-09-16T15:43:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.115+0000 7f2f1b800640 1 -- 172.21.15.60:0/3869140201 wait complete. 2024-09-16T15:43:24.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.116+0000 7f2f1b800640 1 Processor -- start 2024-09-16T15:43:24.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.116+0000 7f2f1b800640 1 -- start start 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.117+0000 7f2f1b800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f140ad270 0x7f2f1415e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.117+0000 7f2f1b800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f140ccf00 0x7f2f140cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.117+0000 7f2f1b800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 0x7f2f14156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.117+0000 7f2f1b800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1406f2e0 con 0x7f2f140ad270 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.117+0000 7f2f1b800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1406f160 con 0x7f2f140ccf00 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.117+0000 7f2f1b800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1406f460 con 0x7f2f1415f5a0 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f140ad270 0x7f2f1415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f140ad270 0x7f2f1415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36502/0 (socket says 172.21.15.60:36502) 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 -- 172.21.15.60:0/1943737642 learned_addr learned my addr 172.21.15.60:0/1943737642 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f0be00640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f140ccf00 0x7f2f140cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:24.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 -- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 msgr2=0x7f2f14156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f19800640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 0x7f2f14156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 0x7f2f14156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 -- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f140ccf00 msgr2=0x7f2f140cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f140ccf00 0x7f2f140cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 -- 172.21.15.60:0/1943737642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f140cbc30 con 0x7f2f140ad270 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f19800640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 0x7f2f14156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f0be00640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f140ccf00 0x7f2f140cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.118+0000 7f2f18e00640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f140ad270 0x7f2f1415e820 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2f04034040 tx=0x7f2f040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.119+0000 7f2f11600640 1 -- 172.21.15.60:0/1943737642 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0403e030 con 0x7f2f140ad270 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.119+0000 7f2f11600640 1 -- 172.21.15.60:0/1943737642 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f04003d90 con 0x7f2f140ad270 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.119+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f140cbf10 con 0x7f2f140ad270 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.119+0000 7f2f11600640 1 -- 172.21.15.60:0/1943737642 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04035040 con 0x7f2f140ad270 2024-09-16T15:43:24.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.119+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f140cc4d0 con 0x7f2f140ad270 2024-09-16T15:43:24.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.120+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1406f980 con 0x7f2f140ad270 2024-09-16T15:43:24.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.121+0000 7f2f11600640 1 -- 172.21.15.60:0/1943737642 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f2f04041030 con 0x7f2f140ad270 2024-09-16T15:43:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.122+0000 7f2f11600640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ef0078970 0x7f2ef007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.122+0000 7f2f11600640 1 -- 172.21.15.60:0/1943737642 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f2f040bb670 con 0x7f2f140ad270 2024-09-16T15:43:24.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.122+0000 7f2f0be00640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ef0078970 0x7f2ef007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:24.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.123+0000 7f2f0be00640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ef0078970 0x7f2ef007ae30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f140cb100 tx=0x7f2efc005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.126+0000 7f2f11600640 1 -- 172.21.15.60:0/1943737642 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2f040839a0 con 0x7f2f140ad270 2024-09-16T15:43:24.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.326+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f14069ac0 con 0x7f2f140ad270 2024-09-16T15:43:24.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.328+0000 7f2f11600640 1 -- 172.21.15.60:0/1943737642 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16733 (secure 0 0 0) 0x7f2f040832a0 con 0x7f2f140ad270 2024-09-16T15:43:24.329 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:24.329 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":86,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:22.963745+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:20.760401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:21.917247+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:24.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ef0078970 msgr2=0x7f2ef007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ef0078970 0x7f2ef007ae30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f140cb100 tx=0x7f2efc005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f140ad270 msgr2=0x7f2f1415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f140ad270 0x7f2f1415e820 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2f04034040 tx=0x7f2f040038a0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 shutdown_connections 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ef0078970 0x7f2ef007ae30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f1415f5a0 0x7f2f14156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f140ccf00 0x7f2f140cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 --2- 172.21.15.60:0/1943737642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f140ad270 0x7f2f1415e820 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.330+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 >> 172.21.15.60:0/1943737642 conn(0x7f2f140b1950 msgr2=0x7f2f140b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.331+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 shutdown_connections 2024-09-16T15:43:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.331+0000 7f2f1b800640 1 -- 172.21.15.60:0/1943737642 wait complete. 2024-09-16T15:43:24.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:43:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.510+0000 7f6cb0600640 1 -- 172.21.15.60:0/2884582749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca806efa0 msgr2=0x7f6ca80cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.510+0000 7f6cb0600640 1 --2- 172.21.15.60:0/2884582749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca806efa0 0x7f6ca80cc5c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80d0cd0 tx=0x7f6c9402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.510+0000 7f6cb0600640 1 -- 172.21.15.60:0/2884582749 shutdown_connections 2024-09-16T15:43:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.510+0000 7f6cb0600640 1 --2- 172.21.15.60:0/2884582749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca806efa0 0x7f6ca80cc5c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.510+0000 7f6cb0600640 1 --2- 172.21.15.60:0/2884582749 >> v2:172.21.15.100:3300/0 conn(0x7f6ca8145750 0x7f6ca8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.510+0000 7f6cb0600640 1 -- 172.21.15.60:0/2884582749 >> 172.21.15.60:0/2884582749 conn(0x7f6ca80b1840 msgr2=0x7f6ca80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.510+0000 7f6cb0600640 1 -- 172.21.15.60:0/2884582749 shutdown_connections 2024-09-16T15:43:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.510+0000 7f6cb0600640 1 -- 172.21.15.60:0/2884582749 wait complete. 2024-09-16T15:43:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.512+0000 7f6cb0600640 1 Processor -- start 2024-09-16T15:43:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.512+0000 7f6cb0600640 1 -- start start 2024-09-16T15:43:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cb0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca806efa0 0x7f6ca81436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cb0600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ca80ccb00 0x7f6ca81430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cb0600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca8145750 0x7f6ca8144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cb0600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca80ced40 con 0x7f6ca8145750 2024-09-16T15:43:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cb0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca80cebc0 con 0x7f6ca806efa0 2024-09-16T15:43:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cb0600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca80ceec0 con 0x7f6ca80ccb00 2024-09-16T15:43:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cae600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca8145750 0x7f6ca8144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:24.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cae600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca8145750 0x7f6ca8144160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36506/0 (socket says 172.21.15.60:36506) 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cadc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca806efa0 0x7f6ca81436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cae600640 1 -- 172.21.15.60:0/1933436306 learned_addr learned my addr 172.21.15.60:0/1933436306 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.513+0000 7f6cad200640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ca80ccb00 0x7f6ca81430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6cae600640 1 -- 172.21.15.60:0/1933436306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ca80ccb00 msgr2=0x7f6ca81430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6cae600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ca80ccb00 0x7f6ca81430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6cae600640 1 -- 172.21.15.60:0/1933436306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca806efa0 msgr2=0x7f6ca81436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6cae600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca806efa0 0x7f6ca81436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6cae600640 1 -- 172.21.15.60:0/1933436306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca81ead00 con 0x7f6ca8145750 2024-09-16T15:43:24.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6cae600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca8145750 0x7f6ca8144160 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000c7b0 tx=0x7f6ca000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6ca5600640 1 -- 172.21.15.60:0/1933436306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0004b60 con 0x7f6ca8145750 2024-09-16T15:43:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6ca5600640 1 -- 172.21.15.60:0/1933436306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ca0004df0 con 0x7f6ca8145750 2024-09-16T15:43:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6ca5600640 1 -- 172.21.15.60:0/1933436306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000aa90 con 0x7f6ca8145750 2024-09-16T15:43:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.514+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca80c5610 con 0x7f6ca8145750 2024-09-16T15:43:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.515+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca80c5bd0 con 0x7f6ca8145750 2024-09-16T15:43:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.516+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca80cc5c0 con 0x7f6ca8145750 2024-09-16T15:43:24.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.517+0000 7f6ca5600640 1 -- 172.21.15.60:0/1933436306 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f6ca000ac30 con 0x7f6ca8145750 2024-09-16T15:43:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.518+0000 7f6ca5600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c88078970 0x7f6c8807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.518+0000 7f6ca5600640 1 -- 172.21.15.60:0/1933436306 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f6ca009ad30 con 0x7f6ca8145750 2024-09-16T15:43:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.518+0000 7f6cadc00640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c88078970 0x7f6c8807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:24.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.519+0000 7f6cadc00640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c88078970 0x7f6c8807ae30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c005fd0 tx=0x7f6c9c00a250 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:24.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.522+0000 7f6ca5600640 1 -- 172.21.15.60:0/1933436306 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6ca0020070 con 0x7f6ca8145750 2024-09-16T15:43:24.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:24.761+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6ca80c5da0 con 0x7f6ca8145750 2024-09-16T15:43:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.546+0000 7f6ca5600640 1 -- 172.21.15.60:0/1933436306 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) ==== 118+0+0 (secure 0 0 0) 0x7f6ca0062b20 con 0x7f6ca8145750 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c88078970 msgr2=0x7f6c8807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c88078970 0x7f6c8807ae30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c005fd0 tx=0x7f6c9c00a250 comp rx=0 tx=0).stop 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca8145750 msgr2=0x7f6ca8144160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca8145750 0x7f6ca8144160 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000c7b0 tx=0x7f6ca000cc70 comp rx=0 tx=0).stop 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 shutdown_connections 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c88078970 0x7f6c8807ae30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ca8145750 0x7f6ca8144160 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ca80ccb00 0x7f6ca81430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 --2- 172.21.15.60:0/1933436306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca806efa0 0x7f6ca81436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.549+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 >> 172.21.15.60:0/1933436306 conn(0x7f6ca80b1840 msgr2=0x7f6ca80b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.550+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 shutdown_connections 2024-09-16T15:43:25.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.550+0000 7f6cb0600640 1 -- 172.21.15.60:0/1933436306 wait complete. 2024-09-16T15:43:25.567 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:43:25.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:43:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.733+0000 7f7bf2000640 1 -- 172.21.15.60:0/1005153431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 msgr2=0x7f7bd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.733+0000 7f7bf2000640 1 --2- 172.21.15.60:0/1005153431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 0x7f7bd000b9c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7be4005fd0 tx=0x7f7bd0005830 comp rx=0 tx=0).stop 2024-09-16T15:43:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.733+0000 7f7bf2000640 1 -- 172.21.15.60:0/1005153431 shutdown_connections 2024-09-16T15:43:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.733+0000 7f7bf2000640 1 --2- 172.21.15.60:0/1005153431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 0x7f7bd000b9c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.733+0000 7f7bf2000640 1 -- 172.21.15.60:0/1005153431 >> 172.21.15.60:0/1005153431 conn(0x7f7bec0b1840 msgr2=0x7f7bec0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.734+0000 7f7bf2000640 1 -- 172.21.15.60:0/1005153431 shutdown_connections 2024-09-16T15:43:25.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.734+0000 7f7bf2000640 1 -- 172.21.15.60:0/1005153431 wait complete. 2024-09-16T15:43:25.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.735+0000 7f7bf2000640 1 Processor -- start 2024-09-16T15:43:25.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.735+0000 7f7bf2000640 1 -- start start 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.736+0000 7f7bf2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 0x7f7bec15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.736+0000 7f7bf2000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bec1f5f00 0x7f7bec0aa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.736+0000 7f7bf2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bec141d60 0x7f7bec0aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.736+0000 7f7bf2000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec0737d0 con 0x7f7bd00052e0 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.736+0000 7f7bf2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec073650 con 0x7f7bec141d60 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.736+0000 7f7bf2000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7bec0cc5c0 con 0x7f7bec1f5f00 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.736+0000 7f7beb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 0x7f7bec15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.736+0000 7f7beb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 0x7f7bec15e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36526/0 (socket says 172.21.15.60:36526) 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7beb400640 1 -- 172.21.15.60:0/613643894 learned_addr learned my addr 172.21.15.60:0/613643894 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7be2a00640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bec1f5f00 0x7f7bec0aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7bebe00640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bec141d60 0x7f7bec0aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7beb400640 1 -- 172.21.15.60:0/613643894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bec1f5f00 msgr2=0x7f7bec0aa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:25.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7beb400640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bec1f5f00 0x7f7bec0aa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7beb400640 1 -- 172.21.15.60:0/613643894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bec141d60 msgr2=0x7f7bec0aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7beb400640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bec141d60 0x7f7bec0aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7beb400640 1 -- 172.21.15.60:0/613643894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec1f6400 con 0x7f7bd00052e0 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7beb400640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 0x7f7bec15e8e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0066b0 tx=0x7f7bdc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7be3e00640 1 -- 172.21.15.60:0/613643894 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc015070 con 0x7f7bd00052e0 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7be3e00640 1 -- 172.21.15.60:0/613643894 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bdc011840 con 0x7f7bd00052e0 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.737+0000 7f7be3e00640 1 -- 172.21.15.60:0/613643894 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc011c50 con 0x7f7bd00052e0 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.738+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bec1f66e0 con 0x7f7bd00052e0 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.738+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bec1f6ca0 con 0x7f7bd00052e0 2024-09-16T15:43:25.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.739+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bec06edc0 con 0x7f7bd00052e0 2024-09-16T15:43:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.740+0000 7f7be3e00640 1 -- 172.21.15.60:0/613643894 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f7bdc018070 con 0x7f7bd00052e0 2024-09-16T15:43:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.741+0000 7f7be3e00640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bc80789a0 0x7f7bc807ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.741+0000 7f7be3e00640 1 -- 172.21.15.60:0/613643894 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f7bdc09b750 con 0x7f7bd00052e0 2024-09-16T15:43:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.741+0000 7f7be2a00640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bc80789a0 0x7f7bc807ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:25.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.742+0000 7f7be2a00640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bc80789a0 0x7f7bc807ae60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4000c00 tx=0x7f7bd4006b70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.745+0000 7f7be3e00640 1 -- 172.21.15.60:0/613643894 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7bec06edc0 con 0x7f7bd00052e0 2024-09-16T15:43:25.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:25.978+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7bec15f9f0 con 0x7f7bd00052e0 2024-09-16T15:43:26.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.552+0000 7f7be3e00640 1 -- 172.21.15.60:0/613643894 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) ==== 122+0+0 (secure 0 0 0) 0x7f7bdc063580 con 0x7f7bd00052e0 2024-09-16T15:43:26.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bc80789a0 msgr2=0x7f7bc807ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:26.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bc80789a0 0x7f7bc807ae60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4000c00 tx=0x7f7bd4006b70 comp rx=0 tx=0).stop 2024-09-16T15:43:26.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 msgr2=0x7f7bec15e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:26.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 0x7f7bec15e8e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0066b0 tx=0x7f7bdc0039c0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 shutdown_connections 2024-09-16T15:43:26.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bc80789a0 0x7f7bc807ae60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bec141d60 0x7f7bec0aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bec1f5f00 0x7f7bec0aa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 --2- 172.21.15.60:0/613643894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd00052e0 0x7f7bec15e8e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.555+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 >> 172.21.15.60:0/613643894 conn(0x7f7bec0b1840 msgr2=0x7f7bec0cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:26.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.556+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 shutdown_connections 2024-09-16T15:43:26.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.556+0000 7f7bf2000640 1 -- 172.21.15.60:0/613643894 wait complete. 2024-09-16T15:43:26.574 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:43:26.575 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.737+0000 7fe0ef200640 1 -- 172.21.15.60:0/1623681823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0f00b4050 msgr2=0x7fe0f00b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.737+0000 7fe0ef200640 1 --2- 172.21.15.60:0/1623681823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0f00b4050 0x7fe0f00b1770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009830 tx=0x7fe0e802a410 comp rx=0 tx=0).stop 2024-09-16T15:43:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.737+0000 7fe0ef200640 1 -- 172.21.15.60:0/1623681823 shutdown_connections 2024-09-16T15:43:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.737+0000 7fe0ef200640 1 --2- 172.21.15.60:0/1623681823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0f00b4050 0x7fe0f00b1770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.737+0000 7fe0ef200640 1 --2- 172.21.15.60:0/1623681823 >> v2:172.21.15.100:3300/0 conn(0x7fe0f0161440 0x7fe0f00b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.737+0000 7fe0ef200640 1 -- 172.21.15.60:0/1623681823 >> 172.21.15.60:0/1623681823 conn(0x7fe0f00d3df0 msgr2=0x7fe0f00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.737+0000 7fe0ef200640 1 -- 172.21.15.60:0/1623681823 shutdown_connections 2024-09-16T15:43:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.737+0000 7fe0ef200640 1 -- 172.21.15.60:0/1623681823 wait complete. 2024-09-16T15:43:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.739+0000 7fe0ef200640 1 Processor -- start 2024-09-16T15:43:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.739+0000 7fe0ef200640 1 -- start start 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0ef200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f00b34b0 0x7fe0f00d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0ef200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0f00b4050 0x7fe0f00b8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0ef200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0f0161440 0x7fe0f00b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0ef200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f00b2040 con 0x7fe0f00b4050 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0ef200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f00b1ec0 con 0x7fe0f00b34b0 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0ef200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f00b21c0 con 0x7fe0f0161440 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0e5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0f00b4050 0x7fe0f00b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0e5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0f00b4050 0x7fe0f00b8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36542/0 (socket says 172.21.15.60:36542) 2024-09-16T15:43:26.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0e5400640 1 -- 172.21.15.60:0/926981516 learned_addr learned my addr 172.21.15.60:0/926981516 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0ee800640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0f0161440 0x7fe0f00b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0ede00640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f00b34b0 0x7fe0f00d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0e5400640 1 -- 172.21.15.60:0/926981516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0f0161440 msgr2=0x7fe0f00b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0e5400640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0f0161440 0x7fe0f00b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0e5400640 1 -- 172.21.15.60:0/926981516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f00b34b0 msgr2=0x7fe0f00d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0e5400640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f00b34b0 0x7fe0f00d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.740+0000 7fe0e5400640 1 -- 172.21.15.60:0/926981516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01f5180 con 0x7fe0f00b4050 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.741+0000 7fe0e5400640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0f00b4050 0x7fe0f00b8420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800c4d0 tx=0x7fe0d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.741+0000 7fe0e6a00640 1 -- 172.21.15.60:0/926981516 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8017070 con 0x7fe0f00b4050 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.741+0000 7fe0e6a00640 1 -- 172.21.15.60:0/926981516 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0d80035f0 con 0x7fe0f00b4050 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.741+0000 7fe0e6a00640 1 -- 172.21.15.60:0/926981516 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8003a40 con 0x7fe0f00b4050 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.741+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01f5440 con 0x7fe0f00b4050 2024-09-16T15:43:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.741+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f00cfbc0 con 0x7fe0f00b4050 2024-09-16T15:43:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.742+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f00b1770 con 0x7fe0f00b4050 2024-09-16T15:43:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.743+0000 7fe0e6a00640 1 -- 172.21.15.60:0/926981516 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fe0d801a070 con 0x7fe0f00b4050 2024-09-16T15:43:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.744+0000 7fe0e6a00640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe0cc078970 0x7fe0cc07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.744+0000 7fe0e6a00640 1 -- 172.21.15.60:0/926981516 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fe0d8099f70 con 0x7fe0f00b4050 2024-09-16T15:43:26.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.745+0000 7fe0ede00640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe0cc078970 0x7fe0cc07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:26.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.745+0000 7fe0ede00640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe0cc078970 0x7fe0cc07ae30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0155d70 tx=0x7fe0e0009500 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:26.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.748+0000 7fe0e6a00640 1 -- 172.21.15.60:0/926981516 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe0d8061cd0 con 0x7fe0f00b4050 2024-09-16T15:43:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.980+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0f00b2960 con 0x7fe0f00b4050 2024-09-16T15:43:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.980+0000 7fe0e6a00640 1 -- 172.21.15.60:0/926981516 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5231 (secure 0 0 0) 0x7fe0d8066700 con 0x7fe0f00b4050 2024-09-16T15:43:26.982 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:26.982 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":57,"btime":"2024-09-16T15:43:26:547677+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1388326737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1388326737},{"type":"v1","addr":"172.21.15.100:6835","nonce":1388326737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6382,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/2485723644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2485723644},{"type":"v1","addr":"172.21.15.100:6801","nonce":2485723644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2496703614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2496703614},{"type":"v1","addr":"172.21.15.60:6835","nonce":2496703614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:22.963712+0000","modified":"2024-09-16T15:43:25.979087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3103295284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3103295284},{"type":"v1","addr":"172.21.15.60:6834","nonce":3103295284}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":6}]} 2024-09-16T15:43:26.982 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 57 2024-09-16T15:43:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe0cc078970 msgr2=0x7fe0cc07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe0cc078970 0x7fe0cc07ae30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0155d70 tx=0x7fe0e0009500 comp rx=0 tx=0).stop 2024-09-16T15:43:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0f00b4050 msgr2=0x7fe0f00b8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0f00b4050 0x7fe0f00b8420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800c4d0 tx=0x7fe0d800c990 comp rx=0 tx=0).stop 2024-09-16T15:43:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 shutdown_connections 2024-09-16T15:43:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe0cc078970 0x7fe0cc07ae30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0f0161440 0x7fe0f00b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0f00b4050 0x7fe0f00b8420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 --2- 172.21.15.60:0/926981516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f00b34b0 0x7fe0f00d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 >> 172.21.15.60:0/926981516 conn(0x7fe0f00d3df0 msgr2=0x7fe0f00d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 shutdown_connections 2024-09-16T15:43:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:26.983+0000 7fe0ef200640 1 -- 172.21.15.60:0/926981516 wait complete. 2024-09-16T15:43:27.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:27.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.160+0000 7f8e14800640 1 -- 172.21.15.60:0/607896793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c073fb0 msgr2=0x7f8e0c074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.160+0000 7f8e14800640 1 --2- 172.21.15.60:0/607896793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c073fb0 0x7f8e0c074420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c075130 tx=0x7f8e0802a820 comp rx=0 tx=0).stop 2024-09-16T15:43:27.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.160+0000 7f8e14800640 1 -- 172.21.15.60:0/607896793 shutdown_connections 2024-09-16T15:43:27.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.160+0000 7f8e14800640 1 --2- 172.21.15.60:0/607896793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c073fb0 0x7f8e0c074420 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.160+0000 7f8e14800640 1 --2- 172.21.15.60:0/607896793 >> v2:172.21.15.100:3300/0 conn(0x7f8e0c15f5a0 0x7f8e0c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.160+0000 7f8e14800640 1 -- 172.21.15.60:0/607896793 >> 172.21.15.60:0/607896793 conn(0x7f8e0c0b1950 msgr2=0x7f8e0c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:27.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.160+0000 7f8e14800640 1 -- 172.21.15.60:0/607896793 shutdown_connections 2024-09-16T15:43:27.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.160+0000 7f8e14800640 1 -- 172.21.15.60:0/607896793 wait complete. 2024-09-16T15:43:27.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.162+0000 7f8e14800640 1 Processor -- start 2024-09-16T15:43:27.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.162+0000 7f8e14800640 1 -- start start 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e14800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0c073fb0 0x7f8e0c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e14800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c0ccf00 0x7f8e0c1f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e14800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e0c15f5a0 0x7f8e0c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e14800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c06f2e0 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e14800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c06f160 con 0x7f8e0c15f5a0 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e14800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c06f460 con 0x7f8e0c073fb0 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e11400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c0ccf00 0x7f8e0c1f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e11400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c0ccf00 0x7f8e0c1f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36552/0 (socket says 172.21.15.60:36552) 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.163+0000 7f8e11400640 1 -- 172.21.15.60:0/2964573476 learned_addr learned my addr 172.21.15.60:0/2964573476 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e12800640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e0c15f5a0 0x7f8e0c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e11e00640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0c073fb0 0x7f8e0c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e11400640 1 -- 172.21.15.60:0/2964573476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0c073fb0 msgr2=0x7f8e0c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e11400640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0c073fb0 0x7f8e0c158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e11400640 1 -- 172.21.15.60:0/2964573476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e0c15f5a0 msgr2=0x7f8e0c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e11400640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e0c15f5a0 0x7f8e0c156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e11400640 1 -- 172.21.15.60:0/2964573476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c157ef0 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e11400640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c0ccf00 0x7f8e0c1f5c70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00c530 tx=0x7f8dfc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:27.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e06000640 1 -- 172.21.15.60:0/2964573476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc017070 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e06000640 1 -- 172.21.15.60:0/2964573476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dfc0035f0 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e06000640 1 -- 172.21.15.60:0/2964573476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc003a40 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c1581d0 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.164+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c158790 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.165+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c144f60 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.167+0000 7f8e06000640 1 -- 172.21.15.60:0/2964573476 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f8dfc01a070 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.168+0000 7f8e06000640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8df0078970 0x7f8df007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.168+0000 7f8e06000640 1 -- 172.21.15.60:0/2964573476 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f8dfc099f90 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.168+0000 7f8e11e00640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8df0078970 0x7f8df007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.169+0000 7f8e11e00640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8df0078970 0x7f8df007ae30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ac0 tx=0x7f8e00005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:27.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.172+0000 7f8e06000640 1 -- 172.21.15.60:0/2964573476 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8e0c144f60 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.370+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e0c06f9f0 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.371+0000 7f8e06000640 1 -- 172.21.15.60:0/2964573476 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16720 (secure 0 0 0) 0x7f8dfc0623d0 con 0x7f8e0c0ccf00 2024-09-16T15:43:27.372 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:27.372 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":87,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:24.529297+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:20.760401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:21.917247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8df0078970 msgr2=0x7f8df007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8df0078970 0x7f8df007ae30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ac0 tx=0x7f8e00005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c0ccf00 msgr2=0x7f8e0c1f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c0ccf00 0x7f8e0c1f5c70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00c530 tx=0x7f8dfc00c9f0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 shutdown_connections 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8df0078970 0x7f8df007ae30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e0c15f5a0 0x7f8e0c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0c0ccf00 0x7f8e0c1f5c70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.373+0000 7f8e14800640 1 --2- 172.21.15.60:0/2964573476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0c073fb0 0x7f8e0c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.374+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 >> 172.21.15.60:0/2964573476 conn(0x7f8e0c0b1950 msgr2=0x7f8e0c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:27.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.374+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 shutdown_connections 2024-09-16T15:43:27.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.374+0000 7f8e14800640 1 -- 172.21.15.60:0/2964573476 wait complete. 2024-09-16T15:43:27.390 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:43:27.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:43:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.550+0000 7f547ca00640 1 -- 172.21.15.60:0/3950271857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54780d0540 msgr2=0x7f547806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.550+0000 7f547ca00640 1 --2- 172.21.15.60:0/3950271857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54780d0540 0x7f547806f320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5470005fd0 tx=0x7f547002aa90 comp rx=0 tx=0).stop 2024-09-16T15:43:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.550+0000 7f547ca00640 1 -- 172.21.15.60:0/3950271857 shutdown_connections 2024-09-16T15:43:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.550+0000 7f547ca00640 1 --2- 172.21.15.60:0/3950271857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54780d0540 0x7f547806f320 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.550+0000 7f547ca00640 1 --2- 172.21.15.60:0/3950271857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54780725d0 0x7f54780b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.550+0000 7f547ca00640 1 -- 172.21.15.60:0/3950271857 >> 172.21.15.60:0/3950271857 conn(0x7f54780cfa20 msgr2=0x7f54780cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.550+0000 7f547ca00640 1 -- 172.21.15.60:0/3950271857 shutdown_connections 2024-09-16T15:43:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.550+0000 7f547ca00640 1 -- 172.21.15.60:0/3950271857 wait complete. 2024-09-16T15:43:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.551+0000 7f547ca00640 1 Processor -- start 2024-09-16T15:43:27.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.552+0000 7f547ca00640 1 -- start start 2024-09-16T15:43:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.552+0000 7f547ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54780d0540 0x7f5478148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.552+0000 7f547ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f547815bc30 0x7f54780c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.552+0000 7f547ca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54780c9640 0x7f54780ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f547ca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5478074660 con 0x7f54780d0540 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f547ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f547806f860 con 0x7f547815bc30 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f547ca00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f54781f88d0 con 0x7f54780c9640 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5476800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54780d0540 0x7f5478148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5476800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54780d0540 0x7f5478148c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36562/0 (socket says 172.21.15.60:36562) 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5476800640 1 -- 172.21.15.60:0/4246047558 learned_addr learned my addr 172.21.15.60:0/4246047558 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5477200640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54780c9640 0x7f54780ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5475e00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f547815bc30 0x7f54780c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5476800640 1 -- 172.21.15.60:0/4246047558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54780c9640 msgr2=0x7f54780ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5476800640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54780c9640 0x7f54780ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5476800640 1 -- 172.21.15.60:0/4246047558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f547815bc30 msgr2=0x7f54780c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5476800640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f547815bc30 0x7f54780c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.553+0000 7f5476800640 1 -- 172.21.15.60:0/4246047558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54781f7cc0 con 0x7f54780d0540 2024-09-16T15:43:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.554+0000 7f5476800640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54780d0540 0x7f5478148c90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f546800c3d0 tx=0x7f546800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.556+0000 7f546ea00640 1 -- 172.21.15.60:0/4246047558 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468017070 con 0x7f54780d0540 2024-09-16T15:43:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.556+0000 7f546ea00640 1 -- 172.21.15.60:0/4246047558 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54680053c0 con 0x7f54780d0540 2024-09-16T15:43:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.556+0000 7f546ea00640 1 -- 172.21.15.60:0/4246047558 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468003850 con 0x7f54780d0540 2024-09-16T15:43:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.556+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54781f7fa0 con 0x7f54780d0540 2024-09-16T15:43:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.556+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54781f8560 con 0x7f54780d0540 2024-09-16T15:43:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.557+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547806fdf0 con 0x7f54780d0540 2024-09-16T15:43:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.558+0000 7f546ea00640 1 -- 172.21.15.60:0/4246047558 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f546801a070 con 0x7f54780d0540 2024-09-16T15:43:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.559+0000 7f546ea00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5458078970 0x7f545807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.560+0000 7f546ea00640 1 -- 172.21.15.60:0/4246047558 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f5468099e50 con 0x7f54780d0540 2024-09-16T15:43:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.560+0000 7f5475e00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5458078970 0x7f545807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.560+0000 7f5475e00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5458078970 0x7f545807ae30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f54781f7aa0 tx=0x7f5464005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:27.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.564+0000 7f546ea00640 1 -- 172.21.15.60:0/4246047558 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5468061c40 con 0x7f54780d0540 2024-09-16T15:43:27.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.755+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54780b2990 con 0x7f5458078970 2024-09-16T15:43:27.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.761+0000 7f546ea00640 1 -- 172.21.15.60:0/4246047558 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240188 (secure 0 0 0) 0x7f54780b2990 con 0x7f5458078970 2024-09-16T15:43:27.764 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:27.765 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:43:27.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5458078970 msgr2=0x7f545807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5458078970 0x7f545807ae30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f54781f7aa0 tx=0x7f5464005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54780d0540 msgr2=0x7f5478148c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54780d0540 0x7f5478148c90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f546800c3d0 tx=0x7f546800c890 comp rx=0 tx=0).stop 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 shutdown_connections 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5458078970 0x7f545807ae30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54780c9640 0x7f54780ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f547815bc30 0x7f54780c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 --2- 172.21.15.60:0/4246047558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54780d0540 0x7f5478148c90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 >> 172.21.15.60:0/4246047558 conn(0x7f54780cfa20 msgr2=0x7f54780cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 shutdown_connections 2024-09-16T15:43:27.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.764+0000 7f547ca00640 1 -- 172.21.15.60:0/4246047558 wait complete. 2024-09-16T15:43:27.781 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-16T15:43:26.165023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250340,"kb_used_data":9124,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724380,"statfs":{"total":773094113280,"available":772837765120,"internally_reserved":0,"allocated":9342976,"data_stored":4645845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.657889"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.511710+0000","last_change":"2024-09-16T15:43:21.511710+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.511710+0000","last_undegraded":"2024-09-16T15:43:21.511710+0000","last_fullsized":"2024-09-16T15:43:21.511710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.513120+0000","last_change":"2024-09-16T15:43:21.513120+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.513120+0000","last_undegraded":"2024-09-16T15:43:21.513120+0000","last_fullsized":"2024-09-16T15:43:21.513120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.517613+0000","last_change":"2024-09-16T15:43:21.517613+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.517613+0000","last_undegraded":"2024-09-16T15:43:21.517613+0000","last_fullsized":"2024-09-16T15:43:21.517613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.517531+0000","last_change":"2024-09-16T15:43:21.517531+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.517531+0000","last_undegraded":"2024-09-16T15:43:21.517531+0000","last_fullsized":"2024-09-16T15:43:21.517531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.513072+0000","last_change":"2024-09-16T15:43:21.513072+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.513072+0000","last_undegraded":"2024-09-16T15:43:21.513072+0000","last_fullsized":"2024-09-16T15:43:21.513072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.513658+0000","last_change":"2024-09-16T15:43:21.513658+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.513658+0000","last_undegraded":"2024-09-16T15:43:21.513658+0000","last_fullsized":"2024-09-16T15:43:21.513658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.517084+0000","last_change":"2024-09-16T15:43:21.517084+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.517084+0000","last_undegraded":"2024-09-16T15:43:21.517084+0000","last_fullsized":"2024-09-16T15:43:21.517084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.512754+0000","last_change":"2024-09-16T15:43:21.512754+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.512754+0000","last_undegraded":"2024-09-16T15:43:21.512754+0000","last_fullsized":"2024-09-16T15:43:21.512754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510948+0000","last_change":"2024-09-16T15:43:21.510948+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510948+0000","last_undegraded":"2024-09-16T15:43:21.510948+0000","last_fullsized":"2024-09-16T15:43:21.510948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510478+0000","last_change":"2024-09-16T15:43:21.510478+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510478+0000","last_undegraded":"2024-09-16T15:43:21.510478+0000","last_fullsized":"2024-09-16T15:43:21.510478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510779+0000","last_change":"2024-09-16T15:43:21.510779+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510779+0000","last_undegraded":"2024-09-16T15:43:21.510779+0000","last_fullsized":"2024-09-16T15:43:21.510779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.514423+0000","last_change":"2024-09-16T15:43:21.514423+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.514423+0000","last_undegraded":"2024-09-16T15:43:21.514423+0000","last_fullsized":"2024-09-16T15:43:21.514423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.512773+0000","last_change":"2024-09-16T15:43:21.512773+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.512773+0000","last_undegraded":"2024-09-16T15:43:21.512773+0000","last_fullsized":"2024-09-16T15:43:21.512773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510059+0000","last_change":"2024-09-16T15:43:21.510059+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510059+0000","last_undegraded":"2024-09-16T15:43:21.510059+0000","last_fullsized":"2024-09-16T15:43:21.510059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.513554+0000","last_change":"2024-09-16T15:43:21.513554+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.513554+0000","last_undegraded":"2024-09-16T15:43:21.513554+0000","last_fullsized":"2024-09-16T15:43:21.513554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.516500+0000","last_change":"2024-09-16T15:43:21.516500+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.516500+0000","last_undegraded":"2024-09-16T15:43:21.516500+0000","last_fullsized":"2024-09-16T15:43:21.516500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510477+0000","last_change":"2024-09-16T15:43:21.510477+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510477+0000","last_undegraded":"2024-09-16T15:43:21.510477+0000","last_fullsized":"2024-09-16T15:43:21.510477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.517448+0000","last_change":"2024-09-16T15:43:21.517448+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.517448+0000","last_undegraded":"2024-09-16T15:43:21.517448+0000","last_fullsized":"2024-09-16T15:43:21.517448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.516931+0000","last_change":"2024-09-16T15:43:21.516931+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.516931+0000","last_undegraded":"2024-09-16T15:43:21.516931+0000","last_fullsized":"2024-09-16T15:43:21.516931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510078+0000","last_change":"2024-09-16T15:43:21.510078+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510078+0000","last_undegraded":"2024-09-16T15:43:21.510078+0000","last_fullsized":"2024-09-16T15:43:21.510078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.514030+0000","last_change":"2024-09-16T15:43:21.514030+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.514030+0000","last_undegraded":"2024-09-16T15:43:21.514030+0000","last_fullsized":"2024-09-16T15:43:21.514030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.514057+0000","last_change":"2024-09-16T15:43:21.514057+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.514057+0000","last_undegraded":"2024-09-16T15:43:21.514057+0000","last_fullsized":"2024-09-16T15:43:21.514057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510061+0000","last_change":"2024-09-16T15:43:21.510061+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510061+0000","last_undegraded":"2024-09-16T15:43:21.510061+0000","last_fullsized":"2024-09-16T15:43:21.510061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.512798+0000","last_change":"2024-09-16T15:43:21.512798+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.512798+0000","last_undegraded":"2024-09-16T15:43:21.512798+0000","last_fullsized":"2024-09-16T15:43:21.512798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.512954+0000","last_change":"2024-09-16T15:43:21.512954+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.512954+0000","last_undegraded":"2024-09-16T15:43:21.512954+0000","last_fullsized":"2024-09-16T15:43:21.512954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510318+0000","last_change":"2024-09-16T15:43:21.510318+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510318+0000","last_undegraded":"2024-09-16T15:43:21.510318+0000","last_fullsized":"2024-09-16T15:43:21.510318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.510076+0000","last_change":"2024-09-16T15:43:21.510076+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.510076+0000","last_undegraded":"2024-09-16T15:43:21.510076+0000","last_fullsized":"2024-09-16T15:43:21.510076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.516930+0000","last_change":"2024-09-16T15:43:21.516930+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.516930+0000","last_undegraded":"2024-09-16T15:43:21.516930+0000","last_fullsized":"2024-09-16T15:43:21.516930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.513096+0000","last_change":"2024-09-16T15:43:21.513096+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.513096+0000","last_undegraded":"2024-09-16T15:43:21.513096+0000","last_fullsized":"2024-09-16T15:43:21.513096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.509721+0000","last_change":"2024-09-16T15:43:21.509721+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.509721+0000","last_undegraded":"2024-09-16T15:43:21.509721+0000","last_fullsized":"2024-09-16T15:43:21.509721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.513060+0000","last_change":"2024-09-16T15:43:21.513060+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.513060+0000","last_undegraded":"2024-09-16T15:43:21.513060+0000","last_fullsized":"2024-09-16T15:43:21.513060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:43:22.507245+0000","last_change":"2024-09-16T15:43:22.507245+0000","last_active":"2024-09-16T15:43:22.507245+0000","last_peered":"2024-09-16T15:43:22.507245+0000","last_clean":"2024-09-16T15:43:22.507245+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:22.507245+0000","last_undegraded":"2024-09-16T15:43:22.507245+0000","last_fullsized":"2024-09-16T15:43:22.507245+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-09-16T15:43:21.509658+0000","last_change":"2024-09-16T15:43:21.509658+0000","last_active":"2024-09-16T15:43:21.500182+0000","last_peered":"2024-09-16T15:43:21.500182+0000","last_clean":"2024-09-16T15:43:21.500182+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:43:21.509658+0000","last_undegraded":"2024-09-16T15:43:21.509658+0000","last_fullsized":"2024-09-16T15:43:21.509658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":250,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T15:43:21.509423+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:43:21.509423+0000","last_peered":"2024-09-16T15:43:21.509423+0000","last_clean":"2024-09-16T15:43:21.509423+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:43:21.509423+0000","last_undegraded":"2024-09-16T15:43:21.509423+0000","last_fullsized":"2024-09-16T15:43:21.509423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28600,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343240,"statfs":{"total":96636764160,"available":96607477760,"internally_reserved":0,"allocated":1605632,"data_stored":1019871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509504,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1011712,"data_stored":429503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509502,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1024000,"data_stored":435320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1024000,"data_stored":435320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1011712,"data_stored":429503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1024000,"data_stored":435320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335492,"statfs":{"total":96636764160,"available":96599543808,"internally_reserved":0,"allocated":1024000,"data_stored":435320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509504,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32768,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1617920,"data_stored":1025688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:43:27.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:43:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.980+0000 7fad37200640 1 -- 172.21.15.60:0/272180390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b34b0 msgr2=0x7fad380b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.980+0000 7fad37200640 1 --2- 172.21.15.60:0/272180390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b34b0 0x7fad380b3940 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fad380b6930 tx=0x7fad2402a4b0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.980+0000 7fad37200640 1 -- 172.21.15.60:0/272180390 shutdown_connections 2024-09-16T15:43:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.980+0000 7fad37200640 1 --2- 172.21.15.60:0/272180390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b34b0 0x7fad380b3940 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.980+0000 7fad37200640 1 --2- 172.21.15.60:0/272180390 >> v2:172.21.15.100:3300/0 conn(0x7fad38161440 0x7fad380b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.980+0000 7fad37200640 1 -- 172.21.15.60:0/272180390 >> 172.21.15.60:0/272180390 conn(0x7fad380d3df0 msgr2=0x7fad380d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.980+0000 7fad37200640 1 -- 172.21.15.60:0/272180390 shutdown_connections 2024-09-16T15:43:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.980+0000 7fad37200640 1 -- 172.21.15.60:0/272180390 wait complete. 2024-09-16T15:43:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.981+0000 7fad37200640 1 Processor -- start 2024-09-16T15:43:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.981+0000 7fad37200640 1 -- start start 2024-09-16T15:43:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.982+0000 7fad37200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad380b34b0 0x7fad381564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.982+0000 7fad37200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b3e80 0x7fad380c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.982+0000 7fad37200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad38161440 0x7fad380b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.982+0000 7fad37200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fad380b2900 con 0x7fad380b3e80 2024-09-16T15:43:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.982+0000 7fad37200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad380b2780 con 0x7fad380b34b0 2024-09-16T15:43:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.982+0000 7fad37200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fad380b2a80 con 0x7fad38161440 2024-09-16T15:43:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b3e80 0x7fad380c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b3e80 0x7fad380c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36578/0 (socket says 172.21.15.60:36578) 2024-09-16T15:43:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35400640 1 -- 172.21.15.60:0/842956780 learned_addr learned my addr 172.21.15.60:0/842956780 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35e00640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad380b34b0 0x7fad381564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad36800640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad38161440 0x7fad380b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35400640 1 -- 172.21.15.60:0/842956780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad38161440 msgr2=0x7fad380b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35400640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad38161440 0x7fad380b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35400640 1 -- 172.21.15.60:0/842956780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad380b34b0 msgr2=0x7fad381564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35400640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad380b34b0 0x7fad381564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:27.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.983+0000 7fad35400640 1 -- 172.21.15.60:0/842956780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad38156a10 con 0x7fad380b3e80 2024-09-16T15:43:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.984+0000 7fad35400640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b3e80 0x7fad380c7920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fad24009920 tx=0x7fad24032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.984+0000 7fad2e000640 1 -- 172.21.15.60:0/842956780 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24040070 con 0x7fad380b3e80 2024-09-16T15:43:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.984+0000 7fad2e000640 1 -- 172.21.15.60:0/842956780 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad24005310 con 0x7fad380b3e80 2024-09-16T15:43:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.984+0000 7fad2e000640 1 -- 172.21.15.60:0/842956780 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240047a0 con 0x7fad380b3e80 2024-09-16T15:43:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.984+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad381f47f0 con 0x7fad380b3e80 2024-09-16T15:43:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.984+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad381f4db0 con 0x7fad380b3e80 2024-09-16T15:43:27.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.985+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3815c2c0 con 0x7fad380b3e80 2024-09-16T15:43:27.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.986+0000 7fad2e000640 1 -- 172.21.15.60:0/842956780 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fad24039040 con 0x7fad380b3e80 2024-09-16T15:43:27.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.988+0000 7fad2e000640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fad18078970 0x7fad1807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:27.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.988+0000 7fad2e000640 1 -- 172.21.15.60:0/842956780 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fad240bbce0 con 0x7fad380b3e80 2024-09-16T15:43:27.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.988+0000 7fad35e00640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fad18078970 0x7fad1807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:27.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.989+0000 7fad35e00640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fad18078970 0x7fad1807ae30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad3806e830 tx=0x7fad2800b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:27.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:27.992+0000 7fad2e000640 1 -- 172.21.15.60:0/842956780 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fad3815c2c0 con 0x7fad380b3e80 2024-09-16T15:43:28.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.183+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad381603c0 con 0x7fad18078970 2024-09-16T15:43:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.188+0000 7fad2e000640 1 -- 172.21.15.60:0/842956780 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252080 (secure 0 0 0) 0x7fad381603c0 con 0x7fad18078970 2024-09-16T15:43:28.192 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:28.193 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:43:28.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fad18078970 msgr2=0x7fad1807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fad18078970 0x7fad1807ae30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad3806e830 tx=0x7fad2800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b3e80 msgr2=0x7fad380c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b3e80 0x7fad380c7920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fad24009920 tx=0x7fad24032c10 comp rx=0 tx=0).stop 2024-09-16T15:43:28.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 shutdown_connections 2024-09-16T15:43:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fad18078970 0x7fad1807ae30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fad38161440 0x7fad380b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fad380b3e80 0x7fad380c7920 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 --2- 172.21.15.60:0/842956780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad380b34b0 0x7fad381564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 >> 172.21.15.60:0/842956780 conn(0x7fad380d3df0 msgr2=0x7fad3806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 shutdown_connections 2024-09-16T15:43:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.192+0000 7fad37200640 1 -- 172.21.15.60:0/842956780 wait complete. 2024-09-16T15:43:28.209 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-16T15:43:28.165353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263188,"kb_used_data":9512,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711532,"statfs":{"total":773094113280,"available":772824608768,"internally_reserved":0,"allocated":9740288,"data_stored":4822916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001264"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543062+0000","last_change":"2024-09-16T15:43:23.533293+0000","last_active":"2024-09-16T15:43:24.543062+0000","last_peered":"2024-09-16T15:43:24.543062+0000","last_clean":"2024-09-16T15:43:24.543062+0000","last_became_active":"2024-09-16T15:43:23.532820+0000","last_became_peered":"2024-09-16T15:43:23.532820+0000","last_unstale":"2024-09-16T15:43:24.543062+0000","last_undegraded":"2024-09-16T15:43:24.543062+0000","last_fullsized":"2024-09-16T15:43:24.543062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:35.877884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993911+0000","last_change":"2024-09-16T15:43:23.539557+0000","last_active":"2024-09-16T15:43:24.993911+0000","last_peered":"2024-09-16T15:43:24.993911+0000","last_clean":"2024-09-16T15:43:24.993911+0000","last_became_active":"2024-09-16T15:43:23.531296+0000","last_became_peered":"2024-09-16T15:43:23.531296+0000","last_unstale":"2024-09-16T15:43:24.993911+0000","last_undegraded":"2024-09-16T15:43:24.993911+0000","last_fullsized":"2024-09-16T15:43:24.993911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:25:09.155703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543786+0000","last_change":"2024-09-16T15:43:23.537593+0000","last_active":"2024-09-16T15:43:24.543786+0000","last_peered":"2024-09-16T15:43:24.543786+0000","last_clean":"2024-09-16T15:43:24.543786+0000","last_became_active":"2024-09-16T15:43:23.537289+0000","last_became_peered":"2024-09-16T15:43:23.537289+0000","last_unstale":"2024-09-16T15:43:24.543786+0000","last_undegraded":"2024-09-16T15:43:24.543786+0000","last_fullsized":"2024-09-16T15:43:24.543786+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:30:38.656343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539824+0000","last_change":"2024-09-16T15:43:23.554433+0000","last_active":"2024-09-16T15:43:24.539824+0000","last_peered":"2024-09-16T15:43:24.539824+0000","last_clean":"2024-09-16T15:43:24.539824+0000","last_became_active":"2024-09-16T15:43:23.554171+0000","last_became_peered":"2024-09-16T15:43:23.554171+0000","last_unstale":"2024-09-16T15:43:24.539824+0000","last_undegraded":"2024-09-16T15:43:24.539824+0000","last_fullsized":"2024-09-16T15:43:24.539824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:28.452856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993638+0000","last_change":"2024-09-16T15:43:23.530653+0000","last_active":"2024-09-16T15:43:24.993638+0000","last_peered":"2024-09-16T15:43:24.993638+0000","last_clean":"2024-09-16T15:43:24.993638+0000","last_became_active":"2024-09-16T15:43:23.529690+0000","last_became_peered":"2024-09-16T15:43:23.529690+0000","last_unstale":"2024-09-16T15:43:24.993638+0000","last_undegraded":"2024-09-16T15:43:24.993638+0000","last_fullsized":"2024-09-16T15:43:24.993638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:18.271989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993225+0000","last_change":"2024-09-16T15:43:23.544210+0000","last_active":"2024-09-16T15:43:24.993225+0000","last_peered":"2024-09-16T15:43:24.993225+0000","last_clean":"2024-09-16T15:43:24.993225+0000","last_became_active":"2024-09-16T15:43:23.543870+0000","last_became_peered":"2024-09-16T15:43:23.543870+0000","last_unstale":"2024-09-16T15:43:24.993225+0000","last_undegraded":"2024-09-16T15:43:24.993225+0000","last_fullsized":"2024-09-16T15:43:24.993225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:29.427189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539469+0000","last_change":"2024-09-16T15:43:23.549527+0000","last_active":"2024-09-16T15:43:24.539469+0000","last_peered":"2024-09-16T15:43:24.539469+0000","last_clean":"2024-09-16T15:43:24.539469+0000","last_became_active":"2024-09-16T15:43:23.549270+0000","last_became_peered":"2024-09-16T15:43:23.549270+0000","last_unstale":"2024-09-16T15:43:24.539469+0000","last_undegraded":"2024-09-16T15:43:24.539469+0000","last_fullsized":"2024-09-16T15:43:24.539469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:28:28.184349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993139+0000","last_change":"2024-09-16T15:43:23.547346+0000","last_active":"2024-09-16T15:43:24.993139+0000","last_peered":"2024-09-16T15:43:24.993139+0000","last_clean":"2024-09-16T15:43:24.993139+0000","last_became_active":"2024-09-16T15:43:23.546422+0000","last_became_peered":"2024-09-16T15:43:23.546422+0000","last_unstale":"2024-09-16T15:43:24.993139+0000","last_undegraded":"2024-09-16T15:43:24.993139+0000","last_fullsized":"2024-09-16T15:43:24.993139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:22.690674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993164+0000","last_change":"2024-09-16T15:43:23.564642+0000","last_active":"2024-09-16T15:43:24.993164+0000","last_peered":"2024-09-16T15:43:24.993164+0000","last_clean":"2024-09-16T15:43:24.993164+0000","last_became_active":"2024-09-16T15:43:23.564357+0000","last_became_peered":"2024-09-16T15:43:23.564357+0000","last_unstale":"2024-09-16T15:43:24.993164+0000","last_undegraded":"2024-09-16T15:43:24.993164+0000","last_fullsized":"2024-09-16T15:43:24.993164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:49.687402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543689+0000","last_change":"2024-09-16T15:43:23.546243+0000","last_active":"2024-09-16T15:43:24.543689+0000","last_peered":"2024-09-16T15:43:24.543689+0000","last_clean":"2024-09-16T15:43:24.543689+0000","last_became_active":"2024-09-16T15:43:23.545875+0000","last_became_peered":"2024-09-16T15:43:23.545875+0000","last_unstale":"2024-09-16T15:43:24.543689+0000","last_undegraded":"2024-09-16T15:43:24.543689+0000","last_fullsized":"2024-09-16T15:43:24.543689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:50.887500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993866+0000","last_change":"2024-09-16T15:43:23.530102+0000","last_active":"2024-09-16T15:43:24.993866+0000","last_peered":"2024-09-16T15:43:24.993866+0000","last_clean":"2024-09-16T15:43:24.993866+0000","last_became_active":"2024-09-16T15:43:23.529761+0000","last_became_peered":"2024-09-16T15:43:23.529761+0000","last_unstale":"2024-09-16T15:43:24.993866+0000","last_undegraded":"2024-09-16T15:43:24.993866+0000","last_fullsized":"2024-09-16T15:43:24.993866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:14:09.827867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.542787+0000","last_change":"2024-09-16T15:43:23.547342+0000","last_active":"2024-09-16T15:43:24.542787+0000","last_peered":"2024-09-16T15:43:24.542787+0000","last_clean":"2024-09-16T15:43:24.542787+0000","last_became_active":"2024-09-16T15:43:23.545780+0000","last_became_peered":"2024-09-16T15:43:23.545780+0000","last_unstale":"2024-09-16T15:43:24.542787+0000","last_undegraded":"2024-09-16T15:43:24.542787+0000","last_fullsized":"2024-09-16T15:43:24.542787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:55.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.544196+0000","last_change":"2024-09-16T15:43:23.537345+0000","last_active":"2024-09-16T15:43:24.544196+0000","last_peered":"2024-09-16T15:43:24.544196+0000","last_clean":"2024-09-16T15:43:24.544196+0000","last_became_active":"2024-09-16T15:43:23.537073+0000","last_became_peered":"2024-09-16T15:43:23.537073+0000","last_unstale":"2024-09-16T15:43:24.544196+0000","last_undegraded":"2024-09-16T15:43:24.544196+0000","last_fullsized":"2024-09-16T15:43:24.544196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:13:19.822581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539729+0000","last_change":"2024-09-16T15:43:23.554039+0000","last_active":"2024-09-16T15:43:24.539729+0000","last_peered":"2024-09-16T15:43:24.539729+0000","last_clean":"2024-09-16T15:43:24.539729+0000","last_became_active":"2024-09-16T15:43:23.553779+0000","last_became_peered":"2024-09-16T15:43:23.553779+0000","last_unstale":"2024-09-16T15:43:24.539729+0000","last_undegraded":"2024-09-16T15:43:24.539729+0000","last_fullsized":"2024-09-16T15:43:24.539729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:43.431019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993055+0000","last_change":"2024-09-16T15:43:23.546986+0000","last_active":"2024-09-16T15:43:24.993055+0000","last_peered":"2024-09-16T15:43:24.993055+0000","last_clean":"2024-09-16T15:43:24.993055+0000","last_became_active":"2024-09-16T15:43:23.546734+0000","last_became_peered":"2024-09-16T15:43:23.546734+0000","last_unstale":"2024-09-16T15:43:24.993055+0000","last_undegraded":"2024-09-16T15:43:24.993055+0000","last_fullsized":"2024-09-16T15:43:24.993055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:05.953479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993548+0000","last_change":"2024-09-16T15:43:23.531950+0000","last_active":"2024-09-16T15:43:24.993548+0000","last_peered":"2024-09-16T15:43:24.993548+0000","last_clean":"2024-09-16T15:43:24.993548+0000","last_became_active":"2024-09-16T15:43:23.530529+0000","last_became_peered":"2024-09-16T15:43:23.530529+0000","last_unstale":"2024-09-16T15:43:24.993548+0000","last_undegraded":"2024-09-16T15:43:24.993548+0000","last_fullsized":"2024-09-16T15:43:24.993548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:37.438615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994137+0000","last_change":"2024-09-16T15:43:23.539415+0000","last_active":"2024-09-16T15:43:24.994137+0000","last_peered":"2024-09-16T15:43:24.994137+0000","last_clean":"2024-09-16T15:43:24.994137+0000","last_became_active":"2024-09-16T15:43:23.539093+0000","last_became_peered":"2024-09-16T15:43:23.539093+0000","last_unstale":"2024-09-16T15:43:24.994137+0000","last_undegraded":"2024-09-16T15:43:24.994137+0000","last_fullsized":"2024-09-16T15:43:24.994137+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:00.743092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.540761+0000","last_change":"2024-09-16T15:43:23.551541+0000","last_active":"2024-09-16T15:43:24.540761+0000","last_peered":"2024-09-16T15:43:24.540761+0000","last_clean":"2024-09-16T15:43:24.540761+0000","last_became_active":"2024-09-16T15:43:23.551193+0000","last_became_peered":"2024-09-16T15:43:23.551193+0000","last_unstale":"2024-09-16T15:43:24.540761+0000","last_undegraded":"2024-09-16T15:43:24.540761+0000","last_fullsized":"2024-09-16T15:43:24.540761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:23.577616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994806+0000","last_change":"2024-09-16T15:43:23.549363+0000","last_active":"2024-09-16T15:43:24.994806+0000","last_peered":"2024-09-16T15:43:24.994806+0000","last_clean":"2024-09-16T15:43:24.994806+0000","last_became_active":"2024-09-16T15:43:23.549108+0000","last_became_peered":"2024-09-16T15:43:23.549108+0000","last_unstale":"2024-09-16T15:43:24.994806+0000","last_undegraded":"2024-09-16T15:43:24.994806+0000","last_fullsized":"2024-09-16T15:43:24.994806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:12.251007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539884+0000","last_change":"2024-09-16T15:43:23.561372+0000","last_active":"2024-09-16T15:43:24.539884+0000","last_peered":"2024-09-16T15:43:24.539884+0000","last_clean":"2024-09-16T15:43:24.539884+0000","last_became_active":"2024-09-16T15:43:23.560458+0000","last_became_peered":"2024-09-16T15:43:23.560458+0000","last_unstale":"2024-09-16T15:43:24.539884+0000","last_undegraded":"2024-09-16T15:43:24.539884+0000","last_fullsized":"2024-09-16T15:43:24.539884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:33.326127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539782+0000","last_change":"2024-09-16T15:43:23.548266+0000","last_active":"2024-09-16T15:43:24.539782+0000","last_peered":"2024-09-16T15:43:24.539782+0000","last_clean":"2024-09-16T15:43:24.539782+0000","last_became_active":"2024-09-16T15:43:23.546411+0000","last_became_peered":"2024-09-16T15:43:23.546411+0000","last_unstale":"2024-09-16T15:43:24.539782+0000","last_undegraded":"2024-09-16T15:43:24.539782+0000","last_fullsized":"2024-09-16T15:43:24.539782+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:33.517557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993532+0000","last_change":"2024-09-16T15:43:23.546925+0000","last_active":"2024-09-16T15:43:24.993532+0000","last_peered":"2024-09-16T15:43:24.993532+0000","last_clean":"2024-09-16T15:43:24.993532+0000","last_became_active":"2024-09-16T15:43:23.545868+0000","last_became_peered":"2024-09-16T15:43:23.545868+0000","last_unstale":"2024-09-16T15:43:24.993532+0000","last_undegraded":"2024-09-16T15:43:24.993532+0000","last_fullsized":"2024-09-16T15:43:24.993532+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:09.706561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.542934+0000","last_change":"2024-09-16T15:43:23.538943+0000","last_active":"2024-09-16T15:43:24.542934+0000","last_peered":"2024-09-16T15:43:24.542934+0000","last_clean":"2024-09-16T15:43:24.542934+0000","last_became_active":"2024-09-16T15:43:23.538666+0000","last_became_peered":"2024-09-16T15:43:23.538666+0000","last_unstale":"2024-09-16T15:43:24.542934+0000","last_undegraded":"2024-09-16T15:43:24.542934+0000","last_fullsized":"2024-09-16T15:43:24.542934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:24.508463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539219+0000","last_change":"2024-09-16T15:43:23.557459+0000","last_active":"2024-09-16T15:43:24.539219+0000","last_peered":"2024-09-16T15:43:24.539219+0000","last_clean":"2024-09-16T15:43:24.539219+0000","last_became_active":"2024-09-16T15:43:23.556228+0000","last_became_peered":"2024-09-16T15:43:23.556228+0000","last_unstale":"2024-09-16T15:43:24.539219+0000","last_undegraded":"2024-09-16T15:43:24.539219+0000","last_fullsized":"2024-09-16T15:43:24.539219+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:39.876902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994056+0000","last_change":"2024-09-16T15:43:23.530961+0000","last_active":"2024-09-16T15:43:24.994056+0000","last_peered":"2024-09-16T15:43:24.994056+0000","last_clean":"2024-09-16T15:43:24.994056+0000","last_became_active":"2024-09-16T15:43:23.530651+0000","last_became_peered":"2024-09-16T15:43:23.530651+0000","last_unstale":"2024-09-16T15:43:24.994056+0000","last_undegraded":"2024-09-16T15:43:24.994056+0000","last_fullsized":"2024-09-16T15:43:24.994056+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:03.105659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543761+0000","last_change":"2024-09-16T15:43:23.543239+0000","last_active":"2024-09-16T15:43:24.543761+0000","last_peered":"2024-09-16T15:43:24.543761+0000","last_clean":"2024-09-16T15:43:24.543761+0000","last_became_active":"2024-09-16T15:43:23.542761+0000","last_became_peered":"2024-09-16T15:43:23.542761+0000","last_unstale":"2024-09-16T15:43:24.543761+0000","last_undegraded":"2024-09-16T15:43:24.543761+0000","last_fullsized":"2024-09-16T15:43:24.543761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:12.958116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993430+0000","last_change":"2024-09-16T15:43:23.547333+0000","last_active":"2024-09-16T15:43:24.993430+0000","last_peered":"2024-09-16T15:43:24.993430+0000","last_clean":"2024-09-16T15:43:24.993430+0000","last_became_active":"2024-09-16T15:43:23.544926+0000","last_became_peered":"2024-09-16T15:43:23.544926+0000","last_unstale":"2024-09-16T15:43:24.993430+0000","last_undegraded":"2024-09-16T15:43:24.993430+0000","last_fullsized":"2024-09-16T15:43:24.993430+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:18.558134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543050+0000","last_change":"2024-09-16T15:43:23.533368+0000","last_active":"2024-09-16T15:43:24.543050+0000","last_peered":"2024-09-16T15:43:24.543050+0000","last_clean":"2024-09-16T15:43:24.543050+0000","last_became_active":"2024-09-16T15:43:23.533095+0000","last_became_peered":"2024-09-16T15:43:23.533095+0000","last_unstale":"2024-09-16T15:43:24.543050+0000","last_undegraded":"2024-09-16T15:43:24.543050+0000","last_fullsized":"2024-09-16T15:43:24.543050+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:08.782939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993969+0000","last_change":"2024-09-16T15:43:23.553745+0000","last_active":"2024-09-16T15:43:24.993969+0000","last_peered":"2024-09-16T15:43:24.993969+0000","last_clean":"2024-09-16T15:43:24.993969+0000","last_became_active":"2024-09-16T15:43:23.553440+0000","last_became_peered":"2024-09-16T15:43:23.553440+0000","last_unstale":"2024-09-16T15:43:24.993969+0000","last_undegraded":"2024-09-16T15:43:24.993969+0000","last_fullsized":"2024-09-16T15:43:24.993969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:54.766479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993541+0000","last_change":"2024-09-16T15:43:23.547065+0000","last_active":"2024-09-16T15:43:24.993541+0000","last_peered":"2024-09-16T15:43:24.993541+0000","last_clean":"2024-09-16T15:43:24.993541+0000","last_became_active":"2024-09-16T15:43:23.544758+0000","last_became_peered":"2024-09-16T15:43:23.544758+0000","last_unstale":"2024-09-16T15:43:24.993541+0000","last_undegraded":"2024-09-16T15:43:24.993541+0000","last_fullsized":"2024-09-16T15:43:24.993541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:07.407008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993117+0000","last_change":"2024-09-16T15:43:23.546043+0000","last_active":"2024-09-16T15:43:24.993117+0000","last_peered":"2024-09-16T15:43:24.993117+0000","last_clean":"2024-09-16T15:43:24.993117+0000","last_became_active":"2024-09-16T15:43:23.545773+0000","last_became_peered":"2024-09-16T15:43:23.545773+0000","last_unstale":"2024-09-16T15:43:24.993117+0000","last_undegraded":"2024-09-16T15:43:24.993117+0000","last_fullsized":"2024-09-16T15:43:24.993117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:09.484557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.542759+0000","last_change":"2024-09-16T15:43:23.528553+0000","last_active":"2024-09-16T15:43:24.542759+0000","last_peered":"2024-09-16T15:43:24.542759+0000","last_clean":"2024-09-16T15:43:24.542759+0000","last_became_active":"2024-09-16T15:43:23.528322+0000","last_became_peered":"2024-09-16T15:43:23.528322+0000","last_unstale":"2024-09-16T15:43:24.542759+0000","last_undegraded":"2024-09-16T15:43:24.542759+0000","last_fullsized":"2024-09-16T15:43:24.542759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:10.031744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994131+0000","last_change":"2024-09-16T15:43:23.530339+0000","last_active":"2024-09-16T15:43:24.994131+0000","last_peered":"2024-09-16T15:43:24.994131+0000","last_clean":"2024-09-16T15:43:24.994131+0000","last_became_active":"2024-09-16T15:43:23.529935+0000","last_became_peered":"2024-09-16T15:43:23.529935+0000","last_unstale":"2024-09-16T15:43:24.994131+0000","last_undegraded":"2024-09-16T15:43:24.994131+0000","last_fullsized":"2024-09-16T15:43:24.994131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:09.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543168+0000","last_change":"2024-09-16T15:43:22.532461+0000","last_active":"2024-09-16T15:43:24.543168+0000","last_peered":"2024-09-16T15:43:24.543168+0000","last_clean":"2024-09-16T15:43:24.543168+0000","last_became_active":"2024-09-16T15:43:22.532155+0000","last_became_peered":"2024-09-16T15:43:22.532155+0000","last_unstale":"2024-09-16T15:43:24.543168+0000","last_undegraded":"2024-09-16T15:43:24.543168+0000","last_fullsized":"2024-09-16T15:43:24.543168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:37.997533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543100+0000","last_change":"2024-09-16T15:43:23.562146+0000","last_active":"2024-09-16T15:43:24.543100+0000","last_peered":"2024-09-16T15:43:24.543100+0000","last_clean":"2024-09-16T15:43:24.543100+0000","last_became_active":"2024-09-16T15:43:23.561794+0000","last_became_peered":"2024-09-16T15:43:23.561794+0000","last_unstale":"2024-09-16T15:43:24.543100+0000","last_undegraded":"2024-09-16T15:43:24.543100+0000","last_fullsized":"2024-09-16T15:43:24.543100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:36.242908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994141+0000","last_change":"2024-09-16T15:43:22.537086+0000","last_active":"2024-09-16T15:43:24.994141+0000","last_peered":"2024-09-16T15:43:24.994141+0000","last_clean":"2024-09-16T15:43:24.994141+0000","last_became_active":"2024-09-16T15:43:22.536691+0000","last_became_peered":"2024-09-16T15:43:22.536691+0000","last_unstale":"2024-09-16T15:43:24.994141+0000","last_undegraded":"2024-09-16T15:43:24.994141+0000","last_fullsized":"2024-09-16T15:43:24.994141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:13.488776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543294+0000","last_change":"2024-09-16T15:43:23.543597+0000","last_active":"2024-09-16T15:43:24.543294+0000","last_peered":"2024-09-16T15:43:24.543294+0000","last_clean":"2024-09-16T15:43:24.543294+0000","last_became_active":"2024-09-16T15:43:23.542895+0000","last_became_peered":"2024-09-16T15:43:23.542895+0000","last_unstale":"2024-09-16T15:43:24.543294+0000","last_undegraded":"2024-09-16T15:43:24.543294+0000","last_fullsized":"2024-09-16T15:43:24.543294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:43.076333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994604+0000","last_change":"2024-09-16T15:43:22.539126+0000","last_active":"2024-09-16T15:43:24.994604+0000","last_peered":"2024-09-16T15:43:24.994604+0000","last_clean":"2024-09-16T15:43:24.994604+0000","last_became_active":"2024-09-16T15:43:22.538876+0000","last_became_peered":"2024-09-16T15:43:22.538876+0000","last_unstale":"2024-09-16T15:43:24.994604+0000","last_undegraded":"2024-09-16T15:43:24.994604+0000","last_fullsized":"2024-09-16T15:43:24.994604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:08.477494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.544017+0000","last_change":"2024-09-16T15:43:23.548094+0000","last_active":"2024-09-16T15:43:24.544017+0000","last_peered":"2024-09-16T15:43:24.544017+0000","last_clean":"2024-09-16T15:43:24.544017+0000","last_became_active":"2024-09-16T15:43:23.547800+0000","last_became_peered":"2024-09-16T15:43:23.547800+0000","last_unstale":"2024-09-16T15:43:24.544017+0000","last_undegraded":"2024-09-16T15:43:24.544017+0000","last_fullsized":"2024-09-16T15:43:24.544017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:05.953479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994447+0000","last_change":"2024-09-16T15:43:22.543758+0000","last_active":"2024-09-16T15:43:24.994447+0000","last_peered":"2024-09-16T15:43:24.994447+0000","last_clean":"2024-09-16T15:43:24.994447+0000","last_became_active":"2024-09-16T15:43:22.542365+0000","last_became_peered":"2024-09-16T15:43:22.542365+0000","last_unstale":"2024-09-16T15:43:24.994447+0000","last_undegraded":"2024-09-16T15:43:24.994447+0000","last_fullsized":"2024-09-16T15:43:24.994447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:36.431552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993862+0000","last_change":"2024-09-16T15:43:23.552102+0000","last_active":"2024-09-16T15:43:24.993862+0000","last_peered":"2024-09-16T15:43:24.993862+0000","last_clean":"2024-09-16T15:43:24.993862+0000","last_became_active":"2024-09-16T15:43:23.548726+0000","last_became_peered":"2024-09-16T15:43:23.548726+0000","last_unstale":"2024-09-16T15:43:24.993862+0000","last_undegraded":"2024-09-16T15:43:24.993862+0000","last_fullsized":"2024-09-16T15:43:24.993862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:34:33.428697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993984+0000","last_change":"2024-09-16T15:43:22.536530+0000","last_active":"2024-09-16T15:43:24.993984+0000","last_peered":"2024-09-16T15:43:24.993984+0000","last_clean":"2024-09-16T15:43:24.993984+0000","last_became_active":"2024-09-16T15:43:22.535590+0000","last_became_peered":"2024-09-16T15:43:22.535590+0000","last_unstale":"2024-09-16T15:43:24.993984+0000","last_undegraded":"2024-09-16T15:43:24.993984+0000","last_fullsized":"2024-09-16T15:43:24.993984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:52.629449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993948+0000","last_change":"2024-09-16T15:43:23.544631+0000","last_active":"2024-09-16T15:43:24.993948+0000","last_peered":"2024-09-16T15:43:24.993948+0000","last_clean":"2024-09-16T15:43:24.993948+0000","last_became_active":"2024-09-16T15:43:23.544331+0000","last_became_peered":"2024-09-16T15:43:23.544331+0000","last_unstale":"2024-09-16T15:43:24.993948+0000","last_undegraded":"2024-09-16T15:43:24.993948+0000","last_fullsized":"2024-09-16T15:43:24.993948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:39.137862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993341+0000","last_change":"2024-09-16T15:43:22.538769+0000","last_active":"2024-09-16T15:43:24.993341+0000","last_peered":"2024-09-16T15:43:24.993341+0000","last_clean":"2024-09-16T15:43:24.993341+0000","last_became_active":"2024-09-16T15:43:22.538330+0000","last_became_peered":"2024-09-16T15:43:22.538330+0000","last_unstale":"2024-09-16T15:43:24.993341+0000","last_undegraded":"2024-09-16T15:43:24.993341+0000","last_fullsized":"2024-09-16T15:43:24.993341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:43:24.696035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543016+0000","last_change":"2024-09-16T15:43:23.558754+0000","last_active":"2024-09-16T15:43:24.543016+0000","last_peered":"2024-09-16T15:43:24.543016+0000","last_clean":"2024-09-16T15:43:24.543016+0000","last_became_active":"2024-09-16T15:43:23.558454+0000","last_became_peered":"2024-09-16T15:43:23.558454+0000","last_unstale":"2024-09-16T15:43:24.543016+0000","last_undegraded":"2024-09-16T15:43:24.543016+0000","last_fullsized":"2024-09-16T15:43:24.543016+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:24.363754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993211+0000","last_change":"2024-09-16T15:43:22.540698+0000","last_active":"2024-09-16T15:43:24.993211+0000","last_peered":"2024-09-16T15:43:24.993211+0000","last_clean":"2024-09-16T15:43:24.993211+0000","last_became_active":"2024-09-16T15:43:22.536708+0000","last_became_peered":"2024-09-16T15:43:22.536708+0000","last_unstale":"2024-09-16T15:43:24.993211+0000","last_undegraded":"2024-09-16T15:43:24.993211+0000","last_fullsized":"2024-09-16T15:43:24.993211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:08.101601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993501+0000","last_change":"2024-09-16T15:43:23.552234+0000","last_active":"2024-09-16T15:43:24.993501+0000","last_peered":"2024-09-16T15:43:24.993501+0000","last_clean":"2024-09-16T15:43:24.993501+0000","last_became_active":"2024-09-16T15:43:23.551980+0000","last_became_peered":"2024-09-16T15:43:23.551980+0000","last_unstale":"2024-09-16T15:43:24.993501+0000","last_undegraded":"2024-09-16T15:43:24.993501+0000","last_fullsized":"2024-09-16T15:43:24.993501+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:13:19.822581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993272+0000","last_change":"2024-09-16T15:43:22.531042+0000","last_active":"2024-09-16T15:43:24.993272+0000","last_peered":"2024-09-16T15:43:24.993272+0000","last_clean":"2024-09-16T15:43:24.993272+0000","last_became_active":"2024-09-16T15:43:22.530736+0000","last_became_peered":"2024-09-16T15:43:22.530736+0000","last_unstale":"2024-09-16T15:43:24.993272+0000","last_undegraded":"2024-09-16T15:43:24.993272+0000","last_fullsized":"2024-09-16T15:43:24.993272+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:08.870907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994352+0000","last_change":"2024-09-16T15:43:23.556645+0000","last_active":"2024-09-16T15:43:24.994352+0000","last_peered":"2024-09-16T15:43:24.994352+0000","last_clean":"2024-09-16T15:43:24.994352+0000","last_became_active":"2024-09-16T15:43:23.555655+0000","last_became_peered":"2024-09-16T15:43:23.555655+0000","last_unstale":"2024-09-16T15:43:24.994352+0000","last_undegraded":"2024-09-16T15:43:24.994352+0000","last_fullsized":"2024-09-16T15:43:24.994352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:26.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543365+0000","last_change":"2024-09-16T15:43:22.534944+0000","last_active":"2024-09-16T15:43:24.543365+0000","last_peered":"2024-09-16T15:43:24.543365+0000","last_clean":"2024-09-16T15:43:24.543365+0000","last_became_active":"2024-09-16T15:43:22.534565+0000","last_became_peered":"2024-09-16T15:43:22.534565+0000","last_unstale":"2024-09-16T15:43:24.543365+0000","last_undegraded":"2024-09-16T15:43:24.543365+0000","last_fullsized":"2024-09-16T15:43:24.543365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:56:04.914130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993379+0000","last_change":"2024-09-16T15:43:23.543734+0000","last_active":"2024-09-16T15:43:24.993379+0000","last_peered":"2024-09-16T15:43:24.993379+0000","last_clean":"2024-09-16T15:43:24.993379+0000","last_became_active":"2024-09-16T15:43:23.543451+0000","last_became_peered":"2024-09-16T15:43:23.543451+0000","last_unstale":"2024-09-16T15:43:24.993379+0000","last_undegraded":"2024-09-16T15:43:24.993379+0000","last_fullsized":"2024-09-16T15:43:24.993379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:12:18.655079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543752+0000","last_change":"2024-09-16T15:43:22.532955+0000","last_active":"2024-09-16T15:43:24.543752+0000","last_peered":"2024-09-16T15:43:24.543752+0000","last_clean":"2024-09-16T15:43:24.543752+0000","last_became_active":"2024-09-16T15:43:22.532651+0000","last_became_peered":"2024-09-16T15:43:22.532651+0000","last_unstale":"2024-09-16T15:43:24.543752+0000","last_undegraded":"2024-09-16T15:43:24.543752+0000","last_fullsized":"2024-09-16T15:43:24.543752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:03.850090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994465+0000","last_change":"2024-09-16T15:43:23.559265+0000","last_active":"2024-09-16T15:43:24.994465+0000","last_peered":"2024-09-16T15:43:24.994465+0000","last_clean":"2024-09-16T15:43:24.994465+0000","last_became_active":"2024-09-16T15:43:23.558420+0000","last_became_peered":"2024-09-16T15:43:23.558420+0000","last_unstale":"2024-09-16T15:43:24.994465+0000","last_undegraded":"2024-09-16T15:43:24.994465+0000","last_fullsized":"2024-09-16T15:43:24.994465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:19.625198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.542591+0000","last_change":"2024-09-16T15:43:22.535865+0000","last_active":"2024-09-16T15:43:24.542591+0000","last_peered":"2024-09-16T15:43:24.542591+0000","last_clean":"2024-09-16T15:43:24.542591+0000","last_became_active":"2024-09-16T15:43:22.535574+0000","last_became_peered":"2024-09-16T15:43:22.535574+0000","last_unstale":"2024-09-16T15:43:24.542591+0000","last_undegraded":"2024-09-16T15:43:24.542591+0000","last_fullsized":"2024-09-16T15:43:24.542591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:23.923009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993635+0000","last_change":"2024-09-16T15:43:23.540539+0000","last_active":"2024-09-16T15:43:24.993635+0000","last_peered":"2024-09-16T15:43:24.993635+0000","last_clean":"2024-09-16T15:43:24.993635+0000","last_became_active":"2024-09-16T15:43:23.539969+0000","last_became_peered":"2024-09-16T15:43:23.539969+0000","last_unstale":"2024-09-16T15:43:24.993635+0000","last_undegraded":"2024-09-16T15:43:24.993635+0000","last_fullsized":"2024-09-16T15:43:24.993635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:29.427189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993163+0000","last_change":"2024-09-16T15:43:22.541387+0000","last_active":"2024-09-16T15:43:24.993163+0000","last_peered":"2024-09-16T15:43:24.993163+0000","last_clean":"2024-09-16T15:43:24.993163+0000","last_became_active":"2024-09-16T15:43:22.541168+0000","last_became_peered":"2024-09-16T15:43:22.541168+0000","last_unstale":"2024-09-16T15:43:24.993163+0000","last_undegraded":"2024-09-16T15:43:24.993163+0000","last_fullsized":"2024-09-16T15:43:24.993163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:41.982965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993859+0000","last_change":"2024-09-16T15:43:23.544480+0000","last_active":"2024-09-16T15:43:24.993859+0000","last_peered":"2024-09-16T15:43:24.993859+0000","last_clean":"2024-09-16T15:43:24.993859+0000","last_became_active":"2024-09-16T15:43:23.544199+0000","last_became_peered":"2024-09-16T15:43:23.544199+0000","last_unstale":"2024-09-16T15:43:24.993859+0000","last_undegraded":"2024-09-16T15:43:24.993859+0000","last_fullsized":"2024-09-16T15:43:24.993859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:39.137862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994031+0000","last_change":"2024-09-16T15:43:22.535066+0000","last_active":"2024-09-16T15:43:24.994031+0000","last_peered":"2024-09-16T15:43:24.994031+0000","last_clean":"2024-09-16T15:43:24.994031+0000","last_became_active":"2024-09-16T15:43:22.534760+0000","last_became_peered":"2024-09-16T15:43:22.534760+0000","last_unstale":"2024-09-16T15:43:24.994031+0000","last_undegraded":"2024-09-16T15:43:24.994031+0000","last_fullsized":"2024-09-16T15:43:24.994031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:23.921060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.544412+0000","last_change":"2024-09-16T15:43:23.540881+0000","last_active":"2024-09-16T15:43:24.544412+0000","last_peered":"2024-09-16T15:43:24.544412+0000","last_clean":"2024-09-16T15:43:24.544412+0000","last_became_active":"2024-09-16T15:43:23.540420+0000","last_became_peered":"2024-09-16T15:43:23.540420+0000","last_unstale":"2024-09-16T15:43:24.544412+0000","last_undegraded":"2024-09-16T15:43:24.544412+0000","last_fullsized":"2024-09-16T15:43:24.544412+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:26.933751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993484+0000","last_change":"2024-09-16T15:43:22.535202+0000","last_active":"2024-09-16T15:43:24.993484+0000","last_peered":"2024-09-16T15:43:24.993484+0000","last_clean":"2024-09-16T15:43:24.993484+0000","last_became_active":"2024-09-16T15:43:22.534936+0000","last_became_peered":"2024-09-16T15:43:22.534936+0000","last_unstale":"2024-09-16T15:43:24.993484+0000","last_undegraded":"2024-09-16T15:43:24.993484+0000","last_fullsized":"2024-09-16T15:43:24.993484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:07.883933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993410+0000","last_change":"2024-09-16T15:43:23.550597+0000","last_active":"2024-09-16T15:43:24.993410+0000","last_peered":"2024-09-16T15:43:24.993410+0000","last_clean":"2024-09-16T15:43:24.993410+0000","last_became_active":"2024-09-16T15:43:23.550312+0000","last_became_peered":"2024-09-16T15:43:23.550312+0000","last_unstale":"2024-09-16T15:43:24.993410+0000","last_undegraded":"2024-09-16T15:43:24.993410+0000","last_fullsized":"2024-09-16T15:43:24.993410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:30.625726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.992874+0000","last_change":"2024-09-16T15:43:22.533297+0000","last_active":"2024-09-16T15:43:24.992874+0000","last_peered":"2024-09-16T15:43:24.992874+0000","last_clean":"2024-09-16T15:43:24.992874+0000","last_became_active":"2024-09-16T15:43:22.532982+0000","last_became_peered":"2024-09-16T15:43:22.532982+0000","last_unstale":"2024-09-16T15:43:24.992874+0000","last_undegraded":"2024-09-16T15:43:24.992874+0000","last_fullsized":"2024-09-16T15:43:24.992874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:30:24.221761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993175+0000","last_change":"2024-09-16T15:43:23.529600+0000","last_active":"2024-09-16T15:43:24.993175+0000","last_peered":"2024-09-16T15:43:24.993175+0000","last_clean":"2024-09-16T15:43:24.993175+0000","last_became_active":"2024-09-16T15:43:23.528076+0000","last_became_peered":"2024-09-16T15:43:23.528076+0000","last_unstale":"2024-09-16T15:43:24.993175+0000","last_undegraded":"2024-09-16T15:43:24.993175+0000","last_fullsized":"2024-09-16T15:43:24.993175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:12:16.444711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993548+0000","last_change":"2024-09-16T15:43:22.536803+0000","last_active":"2024-09-16T15:43:24.993548+0000","last_peered":"2024-09-16T15:43:24.993548+0000","last_clean":"2024-09-16T15:43:24.993548+0000","last_became_active":"2024-09-16T15:43:22.536511+0000","last_became_peered":"2024-09-16T15:43:22.536511+0000","last_unstale":"2024-09-16T15:43:24.993548+0000","last_undegraded":"2024-09-16T15:43:24.993548+0000","last_fullsized":"2024-09-16T15:43:24.993548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:28.099761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543971+0000","last_change":"2024-09-16T15:43:23.543958+0000","last_active":"2024-09-16T15:43:24.543971+0000","last_peered":"2024-09-16T15:43:24.543971+0000","last_clean":"2024-09-16T15:43:24.543971+0000","last_became_active":"2024-09-16T15:43:23.543606+0000","last_became_peered":"2024-09-16T15:43:23.543606+0000","last_unstale":"2024-09-16T15:43:24.543971+0000","last_undegraded":"2024-09-16T15:43:24.543971+0000","last_fullsized":"2024-09-16T15:43:24.543971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:11.818951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543501+0000","last_change":"2024-09-16T15:43:22.537951+0000","last_active":"2024-09-16T15:43:24.543501+0000","last_peered":"2024-09-16T15:43:24.543501+0000","last_clean":"2024-09-16T15:43:24.543501+0000","last_became_active":"2024-09-16T15:43:22.537677+0000","last_became_peered":"2024-09-16T15:43:22.537677+0000","last_unstale":"2024-09-16T15:43:24.543501+0000","last_undegraded":"2024-09-16T15:43:24.543501+0000","last_fullsized":"2024-09-16T15:43:24.543501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:16.975007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543303+0000","last_change":"2024-09-16T15:43:23.532351+0000","last_active":"2024-09-16T15:43:24.543303+0000","last_peered":"2024-09-16T15:43:24.543303+0000","last_clean":"2024-09-16T15:43:24.543303+0000","last_became_active":"2024-09-16T15:43:23.531154+0000","last_became_peered":"2024-09-16T15:43:23.531154+0000","last_unstale":"2024-09-16T15:43:24.543303+0000","last_undegraded":"2024-09-16T15:43:24.543303+0000","last_fullsized":"2024-09-16T15:43:24.543303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:03.340649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994816+0000","last_change":"2024-09-16T15:43:22.541953+0000","last_active":"2024-09-16T15:43:24.994816+0000","last_peered":"2024-09-16T15:43:24.994816+0000","last_clean":"2024-09-16T15:43:24.994816+0000","last_became_active":"2024-09-16T15:43:22.541699+0000","last_became_peered":"2024-09-16T15:43:22.541699+0000","last_unstale":"2024-09-16T15:43:24.994816+0000","last_undegraded":"2024-09-16T15:43:24.994816+0000","last_fullsized":"2024-09-16T15:43:24.994816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:34:57.402121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993849+0000","last_change":"2024-09-16T15:43:23.529056+0000","last_active":"2024-09-16T15:43:24.993849+0000","last_peered":"2024-09-16T15:43:24.993849+0000","last_clean":"2024-09-16T15:43:24.993849+0000","last_became_active":"2024-09-16T15:43:23.528764+0000","last_became_peered":"2024-09-16T15:43:23.528764+0000","last_unstale":"2024-09-16T15:43:24.993849+0000","last_undegraded":"2024-09-16T15:43:24.993849+0000","last_fullsized":"2024-09-16T15:43:24.993849+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:08.236438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994533+0000","last_change":"2024-09-16T15:43:22.540946+0000","last_active":"2024-09-16T15:43:24.994533+0000","last_peered":"2024-09-16T15:43:24.994533+0000","last_clean":"2024-09-16T15:43:24.994533+0000","last_became_active":"2024-09-16T15:43:22.538247+0000","last_became_peered":"2024-09-16T15:43:22.538247+0000","last_unstale":"2024-09-16T15:43:24.994533+0000","last_undegraded":"2024-09-16T15:43:24.994533+0000","last_fullsized":"2024-09-16T15:43:24.994533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:17.264927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539725+0000","last_change":"2024-09-16T15:43:23.550961+0000","last_active":"2024-09-16T15:43:24.539725+0000","last_peered":"2024-09-16T15:43:24.539725+0000","last_clean":"2024-09-16T15:43:24.539725+0000","last_became_active":"2024-09-16T15:43:23.550509+0000","last_became_peered":"2024-09-16T15:43:23.550509+0000","last_unstale":"2024-09-16T15:43:24.539725+0000","last_undegraded":"2024-09-16T15:43:24.539725+0000","last_fullsized":"2024-09-16T15:43:24.539725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:42:34.084754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994048+0000","last_change":"2024-09-16T15:43:22.529658+0000","last_active":"2024-09-16T15:43:24.994048+0000","last_peered":"2024-09-16T15:43:24.994048+0000","last_clean":"2024-09-16T15:43:24.994048+0000","last_became_active":"2024-09-16T15:43:22.529267+0000","last_became_peered":"2024-09-16T15:43:22.529267+0000","last_unstale":"2024-09-16T15:43:24.994048+0000","last_undegraded":"2024-09-16T15:43:24.994048+0000","last_fullsized":"2024-09-16T15:43:24.994048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:11.951053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993495+0000","last_change":"2024-09-16T15:43:23.547154+0000","last_active":"2024-09-16T15:43:24.993495+0000","last_peered":"2024-09-16T15:43:24.993495+0000","last_clean":"2024-09-16T15:43:24.993495+0000","last_became_active":"2024-09-16T15:43:23.546909+0000","last_became_peered":"2024-09-16T15:43:23.546909+0000","last_unstale":"2024-09-16T15:43:24.993495+0000","last_undegraded":"2024-09-16T15:43:24.993495+0000","last_fullsized":"2024-09-16T15:43:24.993495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:16.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539810+0000","last_change":"2024-09-16T15:43:22.541325+0000","last_active":"2024-09-16T15:43:24.539810+0000","last_peered":"2024-09-16T15:43:24.539810+0000","last_clean":"2024-09-16T15:43:24.539810+0000","last_became_active":"2024-09-16T15:43:22.541044+0000","last_became_peered":"2024-09-16T15:43:22.541044+0000","last_unstale":"2024-09-16T15:43:24.539810+0000","last_undegraded":"2024-09-16T15:43:24.539810+0000","last_fullsized":"2024-09-16T15:43:24.539810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:17.215651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539413+0000","last_change":"2024-09-16T15:43:23.549024+0000","last_active":"2024-09-16T15:43:24.539413+0000","last_peered":"2024-09-16T15:43:24.539413+0000","last_clean":"2024-09-16T15:43:24.539413+0000","last_became_active":"2024-09-16T15:43:23.548064+0000","last_became_peered":"2024-09-16T15:43:23.548064+0000","last_unstale":"2024-09-16T15:43:24.539413+0000","last_undegraded":"2024-09-16T15:43:24.539413+0000","last_fullsized":"2024-09-16T15:43:24.539413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:43.532163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.539318+0000","last_change":"2024-09-16T15:43:22.533513+0000","last_active":"2024-09-16T15:43:24.539318+0000","last_peered":"2024-09-16T15:43:24.539318+0000","last_clean":"2024-09-16T15:43:24.539318+0000","last_became_active":"2024-09-16T15:43:22.529919+0000","last_became_peered":"2024-09-16T15:43:22.529919+0000","last_unstale":"2024-09-16T15:43:24.539318+0000","last_undegraded":"2024-09-16T15:43:24.539318+0000","last_fullsized":"2024-09-16T15:43:24.539318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:44.711344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993086+0000","last_change":"2024-09-16T15:43:23.544427+0000","last_active":"2024-09-16T15:43:24.993086+0000","last_peered":"2024-09-16T15:43:24.993086+0000","last_clean":"2024-09-16T15:43:24.993086+0000","last_became_active":"2024-09-16T15:43:23.544123+0000","last_became_peered":"2024-09-16T15:43:23.544123+0000","last_unstale":"2024-09-16T15:43:24.993086+0000","last_undegraded":"2024-09-16T15:43:24.993086+0000","last_fullsized":"2024-09-16T15:43:24.993086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:06.911238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.992679+0000","last_change":"2024-09-16T15:43:22.533796+0000","last_active":"2024-09-16T15:43:24.992679+0000","last_peered":"2024-09-16T15:43:24.992679+0000","last_clean":"2024-09-16T15:43:24.992679+0000","last_became_active":"2024-09-16T15:43:22.533507+0000","last_became_peered":"2024-09-16T15:43:22.533507+0000","last_unstale":"2024-09-16T15:43:24.992679+0000","last_undegraded":"2024-09-16T15:43:24.992679+0000","last_fullsized":"2024-09-16T15:43:24.992679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:23.356691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994041+0000","last_change":"2024-09-16T15:43:23.542048+0000","last_active":"2024-09-16T15:43:24.994041+0000","last_peered":"2024-09-16T15:43:24.994041+0000","last_clean":"2024-09-16T15:43:24.994041+0000","last_became_active":"2024-09-16T15:43:23.541737+0000","last_became_peered":"2024-09-16T15:43:23.541737+0000","last_unstale":"2024-09-16T15:43:24.994041+0000","last_undegraded":"2024-09-16T15:43:24.994041+0000","last_fullsized":"2024-09-16T15:43:24.994041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:19.625198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994229+0000","last_change":"2024-09-16T15:43:22.532839+0000","last_active":"2024-09-16T15:43:24.994229+0000","last_peered":"2024-09-16T15:43:24.994229+0000","last_clean":"2024-09-16T15:43:24.994229+0000","last_became_active":"2024-09-16T15:43:22.531596+0000","last_became_peered":"2024-09-16T15:43:22.531596+0000","last_unstale":"2024-09-16T15:43:24.994229+0000","last_undegraded":"2024-09-16T15:43:24.994229+0000","last_fullsized":"2024-09-16T15:43:24.994229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:55.113976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543927+0000","last_change":"2024-09-16T15:43:23.545463+0000","last_active":"2024-09-16T15:43:24.543927+0000","last_peered":"2024-09-16T15:43:24.543927+0000","last_clean":"2024-09-16T15:43:24.543927+0000","last_became_active":"2024-09-16T15:43:23.544866+0000","last_became_peered":"2024-09-16T15:43:23.544866+0000","last_unstale":"2024-09-16T15:43:24.543927+0000","last_undegraded":"2024-09-16T15:43:24.543927+0000","last_fullsized":"2024-09-16T15:43:24.543927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:55:39.367717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993809+0000","last_change":"2024-09-16T15:43:22.537994+0000","last_active":"2024-09-16T15:43:24.993809+0000","last_peered":"2024-09-16T15:43:24.993809+0000","last_clean":"2024-09-16T15:43:24.993809+0000","last_became_active":"2024-09-16T15:43:22.537686+0000","last_became_peered":"2024-09-16T15:43:22.537686+0000","last_unstale":"2024-09-16T15:43:24.993809+0000","last_undegraded":"2024-09-16T15:43:24.993809+0000","last_fullsized":"2024-09-16T15:43:24.993809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:10:31.394867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543702+0000","last_change":"2024-09-16T15:43:23.548528+0000","last_active":"2024-09-16T15:43:24.543702+0000","last_peered":"2024-09-16T15:43:24.543702+0000","last_clean":"2024-09-16T15:43:24.543702+0000","last_became_active":"2024-09-16T15:43:23.548203+0000","last_became_peered":"2024-09-16T15:43:23.548203+0000","last_unstale":"2024-09-16T15:43:24.543702+0000","last_undegraded":"2024-09-16T15:43:24.543702+0000","last_fullsized":"2024-09-16T15:43:24.543702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:47.077064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.544311+0000","last_change":"2024-09-16T15:43:22.532550+0000","last_active":"2024-09-16T15:43:24.544311+0000","last_peered":"2024-09-16T15:43:24.544311+0000","last_clean":"2024-09-16T15:43:24.544311+0000","last_became_active":"2024-09-16T15:43:22.532223+0000","last_became_peered":"2024-09-16T15:43:22.532223+0000","last_unstale":"2024-09-16T15:43:24.544311+0000","last_undegraded":"2024-09-16T15:43:24.544311+0000","last_fullsized":"2024-09-16T15:43:24.544311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:07.423505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994249+0000","last_change":"2024-09-16T15:43:23.540310+0000","last_active":"2024-09-16T15:43:24.994249+0000","last_peered":"2024-09-16T15:43:24.994249+0000","last_clean":"2024-09-16T15:43:24.994249+0000","last_became_active":"2024-09-16T15:43:23.539554+0000","last_became_peered":"2024-09-16T15:43:23.539554+0000","last_unstale":"2024-09-16T15:43:24.994249+0000","last_undegraded":"2024-09-16T15:43:24.994249+0000","last_fullsized":"2024-09-16T15:43:24.994249+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:06.358536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543768+0000","last_change":"2024-09-16T15:43:22.535202+0000","last_active":"2024-09-16T15:43:24.543768+0000","last_peered":"2024-09-16T15:43:24.543768+0000","last_clean":"2024-09-16T15:43:24.543768+0000","last_became_active":"2024-09-16T15:43:22.534862+0000","last_became_peered":"2024-09-16T15:43:22.534862+0000","last_unstale":"2024-09-16T15:43:24.543768+0000","last_undegraded":"2024-09-16T15:43:24.543768+0000","last_fullsized":"2024-09-16T15:43:24.543768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:35.466819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993421+0000","last_change":"2024-09-16T15:43:23.550392+0000","last_active":"2024-09-16T15:43:24.993421+0000","last_peered":"2024-09-16T15:43:24.993421+0000","last_clean":"2024-09-16T15:43:24.993421+0000","last_became_active":"2024-09-16T15:43:23.550107+0000","last_became_peered":"2024-09-16T15:43:23.550107+0000","last_unstale":"2024-09-16T15:43:24.993421+0000","last_undegraded":"2024-09-16T15:43:24.993421+0000","last_fullsized":"2024-09-16T15:43:24.993421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:18.558134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994698+0000","last_change":"2024-09-16T15:43:22.540179+0000","last_active":"2024-09-16T15:43:24.994698+0000","last_peered":"2024-09-16T15:43:24.994698+0000","last_clean":"2024-09-16T15:43:24.994698+0000","last_became_active":"2024-09-16T15:43:22.535835+0000","last_became_peered":"2024-09-16T15:43:22.535835+0000","last_unstale":"2024-09-16T15:43:24.994698+0000","last_undegraded":"2024-09-16T15:43:24.994698+0000","last_fullsized":"2024-09-16T15:43:24.994698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:17.551071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.994803+0000","last_change":"2024-09-16T15:43:23.554824+0000","last_active":"2024-09-16T15:43:24.994803+0000","last_peered":"2024-09-16T15:43:24.994803+0000","last_clean":"2024-09-16T15:43:24.994803+0000","last_became_active":"2024-09-16T15:43:23.543659+0000","last_became_peered":"2024-09-16T15:43:23.543659+0000","last_unstale":"2024-09-16T15:43:24.994803+0000","last_undegraded":"2024-09-16T15:43:24.994803+0000","last_fullsized":"2024-09-16T15:43:24.994803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:09.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993375+0000","last_change":"2024-09-16T15:43:22.537291+0000","last_active":"2024-09-16T15:43:24.993375+0000","last_peered":"2024-09-16T15:43:24.993375+0000","last_clean":"2024-09-16T15:43:24.993375+0000","last_became_active":"2024-09-16T15:43:22.536806+0000","last_became_peered":"2024-09-16T15:43:22.536806+0000","last_unstale":"2024-09-16T15:43:24.993375+0000","last_undegraded":"2024-09-16T15:43:24.993375+0000","last_fullsized":"2024-09-16T15:43:24.993375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:44.286130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543211+0000","last_change":"2024-09-16T15:43:23.543370+0000","last_active":"2024-09-16T15:43:24.543211+0000","last_peered":"2024-09-16T15:43:24.543211+0000","last_clean":"2024-09-16T15:43:24.543211+0000","last_became_active":"2024-09-16T15:43:23.543008+0000","last_became_peered":"2024-09-16T15:43:23.543008+0000","last_unstale":"2024-09-16T15:43:24.543211+0000","last_undegraded":"2024-09-16T15:43:24.543211+0000","last_fullsized":"2024-09-16T15:43:24.543211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:08.890996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543329+0000","last_change":"2024-09-16T15:43:22.534871+0000","last_active":"2024-09-16T15:43:24.543329+0000","last_peered":"2024-09-16T15:43:24.543329+0000","last_clean":"2024-09-16T15:43:24.543329+0000","last_became_active":"2024-09-16T15:43:22.534215+0000","last_became_peered":"2024-09-16T15:43:22.534215+0000","last_unstale":"2024-09-16T15:43:24.543329+0000","last_undegraded":"2024-09-16T15:43:24.543329+0000","last_fullsized":"2024-09-16T15:43:24.543329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:39.055825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543175+0000","last_change":"2024-09-16T15:43:23.549112+0000","last_active":"2024-09-16T15:43:24.543175+0000","last_peered":"2024-09-16T15:43:24.543175+0000","last_clean":"2024-09-16T15:43:24.543175+0000","last_became_active":"2024-09-16T15:43:23.548745+0000","last_became_peered":"2024-09-16T15:43:23.548745+0000","last_unstale":"2024-09-16T15:43:24.543175+0000","last_undegraded":"2024-09-16T15:43:24.543175+0000","last_fullsized":"2024-09-16T15:43:24.543175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:48:40.705361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993161+0000","last_change":"2024-09-16T15:43:22.533087+0000","last_active":"2024-09-16T15:43:24.993161+0000","last_peered":"2024-09-16T15:43:24.993161+0000","last_clean":"2024-09-16T15:43:24.993161+0000","last_became_active":"2024-09-16T15:43:22.532670+0000","last_became_peered":"2024-09-16T15:43:22.532670+0000","last_unstale":"2024-09-16T15:43:24.993161+0000","last_undegraded":"2024-09-16T15:43:24.993161+0000","last_fullsized":"2024-09-16T15:43:24.993161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:45.598347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.993894+0000","last_change":"2024-09-16T15:43:23.546118+0000","last_active":"2024-09-16T15:43:24.993894+0000","last_peered":"2024-09-16T15:43:24.993894+0000","last_clean":"2024-09-16T15:43:24.993894+0000","last_became_active":"2024-09-16T15:43:23.545751+0000","last_became_peered":"2024-09-16T15:43:23.545751+0000","last_unstale":"2024-09-16T15:43:24.993894+0000","last_undegraded":"2024-09-16T15:43:24.993894+0000","last_fullsized":"2024-09-16T15:43:24.993894+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:22.507245+0000","last_clean_scrub_stamp":"2024-09-16T15:43:22.507245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:44.496635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.544071+0000","last_change":"2024-09-16T15:43:22.533565+0000","last_active":"2024-09-16T15:43:24.544071+0000","last_peered":"2024-09-16T15:43:24.544071+0000","last_clean":"2024-09-16T15:43:24.544071+0000","last_became_active":"2024-09-16T15:43:22.533311+0000","last_became_peered":"2024-09-16T15:43:22.533311+0000","last_unstale":"2024-09-16T15:43:24.544071+0000","last_undegraded":"2024-09-16T15:43:24.544071+0000","last_fullsized":"2024-09-16T15:43:24.544071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:21.500182+0000","last_clean_scrub_stamp":"2024-09-16T15:43:21.500182+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:58.897683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":256,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T15:43:24.543269+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:43:24.543269+0000","last_peered":"2024-09-16T15:43:24.543269+0000","last_clean":"2024-09-16T15:43:24.543269+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:43:24.543269+0000","last_undegraded":"2024-09-16T15:43:24.543269+0000","last_fullsized":"2024-09-16T15:43:24.543269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1658880,"data_stored":1045108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509505,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1073152,"data_stored":455364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509503,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":456304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":454670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509504,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":456282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509503,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":455252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36396,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335444,"statfs":{"total":96636764160,"available":96599494656,"internally_reserved":0,"allocated":1073152,"data_stored":454864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509505,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":1045072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:43:28.213 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:43:28.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:43:28.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.417+0000 7f3f61200640 1 -- 172.21.15.60:0/178334938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f40005210 msgr2=0x7f3f4000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.417+0000 7f3f61200640 1 --2- 172.21.15.60:0/178334938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f40005210 0x7f3f4000b9c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3f54005fd0 tx=0x7f3f40005830 comp rx=0 tx=0).stop 2024-09-16T15:43:28.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.417+0000 7f3f61200640 1 -- 172.21.15.60:0/178334938 shutdown_connections 2024-09-16T15:43:28.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.417+0000 7f3f61200640 1 --2- 172.21.15.60:0/178334938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f40005210 0x7f3f4000b9c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.417+0000 7f3f61200640 1 -- 172.21.15.60:0/178334938 >> 172.21.15.60:0/178334938 conn(0x7f3f5c0cc490 msgr2=0x7f3f5c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:28.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.417+0000 7f3f61200640 1 -- 172.21.15.60:0/178334938 shutdown_connections 2024-09-16T15:43:28.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.417+0000 7f3f61200640 1 -- 172.21.15.60:0/178334938 wait complete. 2024-09-16T15:43:28.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.419+0000 7f3f61200640 1 Processor -- start 2024-09-16T15:43:28.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.419+0000 7f3f61200640 1 -- start start 2024-09-16T15:43:28.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f61200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f40005210 0x7f3f5c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:28.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f61200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c156020 0x7f3f5c1f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:28.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f61200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c145e40 0x7f3f5c1f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:28.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f61200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0ca4f0 con 0x7f3f5c145e40 2024-09-16T15:43:28.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f61200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0ca370 con 0x7f3f40005210 2024-09-16T15:43:28.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f61200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c15fa50 con 0x7f3f5c156020 2024-09-16T15:43:28.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f5be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c145e40 0x7f3f5c1f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:28.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f5be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c145e40 0x7f3f5c1f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36592/0 (socket says 172.21.15.60:36592) 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f5be00640 1 -- 172.21.15.60:0/3811591556 learned_addr learned my addr 172.21.15.60:0/3811591556 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f52a00640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c156020 0x7f3f5c1f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.420+0000 7f3f5b400640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f40005210 0x7f3f5c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f5be00640 1 -- 172.21.15.60:0/3811591556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c156020 msgr2=0x7f3f5c1f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f5be00640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c156020 0x7f3f5c1f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f5be00640 1 -- 172.21.15.60:0/3811591556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f40005210 msgr2=0x7f3f5c0b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f5be00640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f40005210 0x7f3f5c0b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f5be00640 1 -- 172.21.15.60:0/3811591556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c153180 con 0x7f3f5c145e40 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f5be00640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c145e40 0x7f3f5c1f61e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3f5402aea0 tx=0x7f3f54004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f53e00640 1 -- 172.21.15.60:0/3811591556 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54009200 con 0x7f3f5c145e40 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f53e00640 1 -- 172.21.15.60:0/3811591556 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f54004650 con 0x7f3f5c145e40 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c0b52f0 con 0x7f3f5c145e40 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f53e00640 1 -- 172.21.15.60:0/3811591556 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54009ad0 con 0x7f3f5c145e40 2024-09-16T15:43:28.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.421+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c153420 con 0x7f3f5c145e40 2024-09-16T15:43:28.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.422+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c06a150 con 0x7f3f5c145e40 2024-09-16T15:43:28.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.423+0000 7f3f53e00640 1 -- 172.21.15.60:0/3811591556 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f3f54044020 con 0x7f3f5c145e40 2024-09-16T15:43:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.424+0000 7f3f53e00640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f38078970 0x7f3f3807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:28.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.424+0000 7f3f53e00640 1 -- 172.21.15.60:0/3811591556 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f3f540bb800 con 0x7f3f5c145e40 2024-09-16T15:43:28.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.425+0000 7f3f5b400640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f38078970 0x7f3f3807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:28.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.425+0000 7f3f5b400640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f38078970 0x7f3f3807ae30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c000fb0 tx=0x7f3f4c006720 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:28.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.429+0000 7f3f53e00640 1 -- 172.21.15.60:0/3811591556 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3f5404b020 con 0x7f3f5c145e40 2024-09-16T15:43:28.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.698+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3f5c15ff40 con 0x7f3f5c145e40 2024-09-16T15:43:28.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.701+0000 7f3f53e00640 1 -- 172.21.15.60:0/3811591556 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f3f5403ae20 con 0x7f3f5c145e40 2024-09-16T15:43:28.702 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:43:28.702 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:43:28.702 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:43:28.702 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:43:28.702 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:43:28.702 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f38078970 msgr2=0x7f3f3807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f38078970 0x7f3f3807ae30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c000fb0 tx=0x7f3f4c006720 comp rx=0 tx=0).stop 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c145e40 msgr2=0x7f3f5c1f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c145e40 0x7f3f5c1f61e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3f5402aea0 tx=0x7f3f54004250 comp rx=0 tx=0).stop 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 shutdown_connections 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f38078970 0x7f3f3807ae30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5c145e40 0x7f3f5c1f61e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5c156020 0x7f3f5c1f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 --2- 172.21.15.60:0/3811591556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f40005210 0x7f3f5c0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.703+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 >> 172.21.15.60:0/3811591556 conn(0x7f3f5c0cc490 msgr2=0x7f3f5c0b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.704+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 shutdown_connections 2024-09-16T15:43:28.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.704+0000 7f3f61200640 1 -- 172.21.15.60:0/3811591556 wait complete. 2024-09-16T15:43:28.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.880+0000 7f562f800640 1 -- 172.21.15.60:0/1752785971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562815bac0 msgr2=0x7f56280cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.880+0000 7f562f800640 1 --2- 172.21.15.60:0/1752785971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562815bac0 0x7f56280cd8f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f561c009830 tx=0x7f561c02a520 comp rx=0 tx=0).stop 2024-09-16T15:43:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.880+0000 7f562f800640 1 -- 172.21.15.60:0/1752785971 shutdown_connections 2024-09-16T15:43:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.880+0000 7f562f800640 1 --2- 172.21.15.60:0/1752785971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5628071490 0x7f562815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.880+0000 7f562f800640 1 --2- 172.21.15.60:0/1752785971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f562815bac0 0x7f56280cd8f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.880+0000 7f562f800640 1 -- 172.21.15.60:0/1752785971 >> 172.21.15.60:0/1752785971 conn(0x7f56280b2b90 msgr2=0x7f56280b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.880+0000 7f562f800640 1 -- 172.21.15.60:0/1752785971 shutdown_connections 2024-09-16T15:43:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.880+0000 7f562f800640 1 -- 172.21.15.60:0/1752785971 wait complete. 2024-09-16T15:43:28.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.882+0000 7f562f800640 1 Processor -- start 2024-09-16T15:43:28.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.882+0000 7f562f800640 1 -- start start 2024-09-16T15:43:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562f800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5628071490 0x7f56281451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562f800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f562815bac0 0x7f5628159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562f800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f562815bea0 0x7f56281332e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562f800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f56281600a0 con 0x7f562815bea0 2024-09-16T15:43:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562f800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f562815ff20 con 0x7f5628071490 2024-09-16T15:43:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562f800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f562806cb70 con 0x7f562815bac0 2024-09-16T15:43:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f562815bea0 0x7f56281332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:28.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f562815bea0 0x7f56281332e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38586/0 (socket says 172.21.15.60:38586) 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.883+0000 7f562d800640 1 -- 172.21.15.60:0/3962490713 learned_addr learned my addr 172.21.15.60:0/3962490713 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562ce00640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5628071490 0x7f56281451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f5627e00640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f562815bac0 0x7f5628159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562d800640 1 -- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f562815bac0 msgr2=0x7f5628159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562d800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f562815bac0 0x7f5628159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562d800640 1 -- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5628071490 msgr2=0x7f56281451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562d800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5628071490 0x7f56281451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562d800640 1 -- 172.21.15.60:0/3962490713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56281f5520 con 0x7f562815bea0 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f5627e00640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f562815bac0 0x7f5628159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562ce00640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5628071490 0x7f56281451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562d800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f562815bea0 0x7f56281332e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f562000eb40 tx=0x7f562000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:28.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f5624c00640 1 -- 172.21.15.60:0/3962490713 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620004b20 con 0x7f562815bea0 2024-09-16T15:43:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f5624c00640 1 -- 172.21.15.60:0/3962490713 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5620004ce0 con 0x7f562815bea0 2024-09-16T15:43:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56280b4260 con 0x7f562815bea0 2024-09-16T15:43:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.884+0000 7f5624c00640 1 -- 172.21.15.60:0/3962490713 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620005280 con 0x7f562815bea0 2024-09-16T15:43:28.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.885+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5628133820 con 0x7f562815bea0 2024-09-16T15:43:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.886+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562815f7d0 con 0x7f562815bea0 2024-09-16T15:43:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.887+0000 7f5624c00640 1 -- 172.21.15.60:0/3962490713 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f5620005420 con 0x7f562815bea0 2024-09-16T15:43:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.888+0000 7f5624c00640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5608078970 0x7f560807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.888+0000 7f5624c00640 1 -- 172.21.15.60:0/3962490713 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f562009b900 con 0x7f562815bea0 2024-09-16T15:43:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.888+0000 7f562ce00640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5608078970 0x7f560807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.889+0000 7f562ce00640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5608078970 0x7f560807ae30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f561c034000 tx=0x7f561c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:28.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:28.892+0000 7f5624c00640 1 -- 172.21.15.60:0/3962490713 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5620063d40 con 0x7f562815bea0 2024-09-16T15:43:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.123+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5628074f10 con 0x7f562815bea0 2024-09-16T15:43:29.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.123+0000 7f5624c00640 1 -- 172.21.15.60:0/3962490713 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5231 (secure 0 0 0) 0x7f562009b070 con 0x7f562815bea0 2024-09-16T15:43:29.124 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:29.125 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":57,"btime":"2024-09-16T15:43:26:547677+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1388326737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1388326737},{"type":"v1","addr":"172.21.15.100:6835","nonce":1388326737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6382,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/2485723644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2485723644},{"type":"v1","addr":"172.21.15.100:6801","nonce":2485723644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6391,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2496703614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2496703614},{"type":"v1","addr":"172.21.15.60:6835","nonce":2496703614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:22.963712+0000","modified":"2024-09-16T15:43:25.979087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3103295284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3103295284},{"type":"v1","addr":"172.21.15.60:6834","nonce":3103295284}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":6}]} 2024-09-16T15:43:29.125 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 57 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.125+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5608078970 msgr2=0x7f560807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.125+0000 7f562f800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5608078970 0x7f560807ae30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f561c034000 tx=0x7f561c004730 comp rx=0 tx=0).stop 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f562815bea0 msgr2=0x7f56281332e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f562815bea0 0x7f56281332e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f562000eb40 tx=0x7f562000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 shutdown_connections 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5608078970 0x7f560807ae30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f562815bea0 0x7f56281332e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f562815bac0 0x7f5628159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 --2- 172.21.15.60:0/3962490713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5628071490 0x7f56281451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 >> 172.21.15.60:0/3962490713 conn(0x7f56280b2b90 msgr2=0x7f56280b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:29.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 shutdown_connections 2024-09-16T15:43:29.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.126+0000 7f562f800640 1 -- 172.21.15.60:0/3962490713 wait complete. 2024-09-16T15:43:29.492 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-09-16T15:43:29.491+0000 7f82d6a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:43:29.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:43:22.963712+0000', 'modified': '2024-09-16T15:43:25.979087+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6576}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6576': {'gid': 6576, 'name': 'c', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6834/3103295284', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 3103295284}, {'type': 'v1', 'addr': '172.21.15.60:6834', 'nonce': 3103295284}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6576, 'qdb_cluster': [6576]} 2024-09-16T15:43:29.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:43:29.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:43:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.711+0000 7f5138e00640 1 -- 172.21.15.60:0/2389275816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5134161440 msgr2=0x7f51340b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.711+0000 7f5138e00640 1 --2- 172.21.15.60:0/2389275816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5134161440 0x7f51340b5370 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f51340d4ea0 tx=0x7f512402a890 comp rx=0 tx=0).stop 2024-09-16T15:43:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.711+0000 7f5138e00640 1 -- 172.21.15.60:0/2389275816 shutdown_connections 2024-09-16T15:43:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.711+0000 7f5138e00640 1 --2- 172.21.15.60:0/2389275816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5134161440 0x7f51340b5370 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.711+0000 7f5138e00640 1 -- 172.21.15.60:0/2389275816 >> 172.21.15.60:0/2389275816 conn(0x7f51340d3df0 msgr2=0x7f51340d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.711+0000 7f5138e00640 1 -- 172.21.15.60:0/2389275816 shutdown_connections 2024-09-16T15:43:29.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.711+0000 7f5138e00640 1 -- 172.21.15.60:0/2389275816 wait complete. 2024-09-16T15:43:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.713+0000 7f5138e00640 1 Processor -- start 2024-09-16T15:43:29.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.713+0000 7f5138e00640 1 -- start start 2024-09-16T15:43:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f5138e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340b34b0 0x7f51340cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f5138e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4050 0x7f51341f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f5138e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134161440 0x7f51341f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f5138e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51340b2210 con 0x7f51340b4050 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f5138e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51340b2090 con 0x7f51340b34b0 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f5138e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f51340b2390 con 0x7f5134161440 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f512aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4050 0x7f51341f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f512aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4050 0x7f51341f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38594/0 (socket says 172.21.15.60:38594) 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f512aa00640 1 -- 172.21.15.60:0/2245380414 learned_addr learned my addr 172.21.15.60:0/2245380414 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f5133400640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340b34b0 0x7f51340cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.714+0000 7f5133e00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134161440 0x7f51341f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512aa00640 1 -- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134161440 msgr2=0x7f51341f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512aa00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134161440 0x7f51341f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512aa00640 1 -- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340b34b0 msgr2=0x7f51340cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512aa00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340b34b0 0x7f51340cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512aa00640 1 -- 172.21.15.60:0/2245380414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51340cdad0 con 0x7f51340b4050 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f5133e00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134161440 0x7f51341f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512aa00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4050 0x7f51341f7c50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c530 tx=0x7f511c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512be00640 1 -- 172.21.15.60:0/2245380414 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0052d0 con 0x7f51340b4050 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512be00640 1 -- 172.21.15.60:0/2245380414 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f511c005470 con 0x7f51340b4050 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f512be00640 1 -- 172.21.15.60:0/2245380414 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c012870 con 0x7f51340b4050 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f5138e00640 1 -- 172.21.15.60:0/2245380414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51340cddb0 con 0x7f51340b4050 2024-09-16T15:43:29.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.715+0000 7f5138e00640 1 -- 172.21.15.60:0/2245380414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51340cccc0 con 0x7f51340b4050 2024-09-16T15:43:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.717+0000 7f512be00640 1 -- 172.21.15.60:0/2245380414 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f511c0035f0 con 0x7f51340b4050 2024-09-16T15:43:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.718+0000 7f512be00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5110078970 0x7f511007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.719+0000 7f512be00640 1 -- 172.21.15.60:0/2245380414 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f511c09a680 con 0x7f51340b4050 2024-09-16T15:43:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.719+0000 7f5133400640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5110078970 0x7f511007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.719+0000 7f513d29f740 1 Processor -- start 2024-09-16T15:43:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.719+0000 7f513d29f740 1 -- start start 2024-09-16T15:43:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.719+0000 7f5133400640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5110078970 0x7f511007ae30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5124034000 tx=0x7f5124004ab0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f513d29f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8ed1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f513d29f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8cc10 0x55c6c8ed0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f513d29f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8ed0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f513d29f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c6c8e0f430 con 0x55c6c8e8d190 2024-09-16T15:43:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f513d29f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c6c8dff5d0 con 0x55c6c8ed1aa0 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f513d29f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c6c8deea00 con 0x55c6c8e8cc10 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f5133e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8ed0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f5133e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8ed0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38608/0 (socket says 172.21.15.60:38608) 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f5133e00640 1 -- 172.21.15.60:0/1810512978 learned_addr learned my addr 172.21.15.60:0/1810512978 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f512aa00640 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8cc10 0x55c6c8ed0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.720+0000 7f5133400640 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8ed1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5133e00640 1 -- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8cc10 msgr2=0x55c6c8ed0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5133e00640 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8cc10 0x55c6c8ed0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5133e00640 1 -- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 msgr2=0x55c6c8ed1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5133e00640 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8ed1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5133e00640 1 -- 172.21.15.60:0/1810512978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c8db7520 con 0x55c6c8e8d190 2024-09-16T15:43:29.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f512aa00640 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8cc10 0x55c6c8ed0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5133e00640 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8ed0ac0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f512c0015b0 tx=0x7f512c001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b402782b50e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5123e00640 1 -- 172.21.15.60:0/1810512978 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c00e740 con 0x55c6c8e8d190 2024-09-16T15:43:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5123e00640 1 -- 172.21.15.60:0/1810512978 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f512c019070 con 0x55c6c8e8d190 2024-09-16T15:43:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.721+0000 7f5123e00640 1 -- 172.21.15.60:0/1810512978 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c00ec10 con 0x55c6c8e8d190 2024-09-16T15:43:29.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 -- 172.21.15.60:0/1810512978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 msgr2=0x55c6c8ed0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8ed0ac0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f512c0015b0 tx=0x7f512c001a70 comp rx=0 tx=0).stop 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 -- 172.21.15.60:0/1810512978 shutdown_connections 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8ed0ac0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8cc10 0x55c6c8ed0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 --2- 172.21.15.60:0/1810512978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8ed1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 -- 172.21.15.60:0/1810512978 >> 172.21.15.60:0/1810512978 conn(0x55c6c8e881e0 msgr2=0x55c6c8e8bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 -- 172.21.15.60:0/1810512978 shutdown_connections 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.722+0000 7f513d29f740 1 -- 172.21.15.60:0/1810512978 wait complete. 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.723+0000 7f513d29f740 1 -- start start 2024-09-16T15:43:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.723+0000 7f513d29f740 1 Processor -- start 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f513d29f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8eb5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f513d29f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f513d29f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ee3a70 0x55c6c8ee3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f513d29f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c6c8d95330 con 0x55c6c8e8d190 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f513d29f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c6c8df2810 con 0x55c6c8ed1aa0 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f513d29f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c6c8b5f620 con 0x55c6c8ee3a70 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f513d29f740 10 client.0 _ll_register_callbacks cb 0x55c6c8d9a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f5133400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8eb5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f5133400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8eb5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38618/0 (socket says 172.21.15.60:38618) 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f5133400640 1 -- 172.21.15.60:0/1076972916 learned_addr learned my addr 172.21.15.60:0/1076972916 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f5133e00640 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ee3a70 0x55c6c8ee3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.724+0000 7f512aa00640 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5133400640 1 -- 172.21.15.60:0/1076972916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ee3a70 msgr2=0x55c6c8ee3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5133400640 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ee3a70 0x55c6c8ee3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5133400640 1 -- 172.21.15.60:0/1076972916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 msgr2=0x55c6c8eb0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5133400640 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5133400640 1 -- 172.21.15.60:0/1076972916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c8b50850 con 0x55c6c8e8d190 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5133400640 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8eb5cc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5124009ca0 tx=0x7f5124009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f27d597c3f6898 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5120c00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5120c00640 1 -- 172.21.15.60:0/1076972916 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124048b80 con 0x55c6c8e8d190 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f5120c00640 1 -- 172.21.15.60:0/1076972916 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f5124048d20 con 0x55c6c8e8d190 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.725+0000 7f513d29f740 1 -- 172.21.15.60:0/1076972916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f512c019070 con 0x55c6c8e8d190 2024-09-16T15:43:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f5120c00640 1 -- 172.21.15.60:0/1076972916 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5124031420 con 0x55c6c8e8d190 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f513d29f740 10 client.6687 fetch_fsmap learned FSMap version 57 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f513d29f740 1 -- 172.21.15.60:0/1076972916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55c6c8bc76b0 con 0x55c6c8e8d190 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f5120c00640 1 -- 172.21.15.60:0/1076972916 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f512c019070 con 0x55c6c8e8d190 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f5120c00640 1 -- 172.21.15.60:0/1076972916 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 57) ==== 3502+0+0 (secure 0 0 0) 0x7f5124059050 con 0x55c6c8e8d190 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f513d29f740 10 client.6687 fetch_fsmap finished waiting for FSMap version 57 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f513d29f740 10 client.6687 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f513d29f740 20 client.6687 populate_metadata read hostname 'smithi060' 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f513d29f740 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] conn(0x55c6c8eb38f0 0x55c6c8eb3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f513d29f740 4 client.6687 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f513d29f740 1 -- 172.21.15.60:0/1076972916 --> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c8e01520 con 0x55c6c8eb38f0 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.726+0000 7f512aa00640 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] conn(0x55c6c8eb38f0 0x55c6c8eb3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.727+0000 7f512aa00640 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] conn(0x55c6c8eb38f0 0x55c6c8eb3cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f56e3fb05e5fbe server_cookie=5e9e030435399474 in_seq=0 out_seq=0 2024-09-16T15:43:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.727+0000 7f5120c00640 10 client.6687 ms_handle_connect on v2:172.21.15.60:6832/3103295284 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f5120c00640 1 -- 172.21.15.60:0/1076972916 <== mds.0 v2:172.21.15.60:6832/3103295284 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6c8e01520 con 0x55c6c8eb38f0 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f5120c00640 10 client.6687 handle_command_reply: tid=0 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 client.6687 shutdown 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 -- 172.21.15.60:0/1076972916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 msgr2=0x55c6c8eb5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8eb5cc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5124009ca0 tx=0x7f5124009500 comp rx=0 tx=0).stop 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 -- 172.21.15.60:0/1076972916 shutdown_connections 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] conn(0x55c6c8eb38f0 0x55c6c8eb3cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ee3a70 0x55c6c8ee3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 --2- 172.21.15.60:0/1076972916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8e8d190 0x55c6c8eb5cc0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f5120c00640 0 client.6687 ms_handle_reset on v2:172.21.15.60:6832/3103295284 2024-09-16T15:43:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 -- 172.21.15.60:0/1076972916 >> 172.21.15.60:0/1076972916 conn(0x55c6c8e8cc10 msgr2=0x55c6c8eb23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 -- 172.21.15.60:0/1076972916 shutdown_connections 2024-09-16T15:43:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 1 -- 172.21.15.60:0/1076972916 wait complete. 2024-09-16T15:43:29.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.728+0000 7f513d29f740 20 client.6687 trim_cache size 0 max 16384 2024-09-16T15:43:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.747+0000 7f513d29f740 1 Processor -- start 2024-09-16T15:43:29.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.747+0000 7f513d29f740 1 -- start start 2024-09-16T15:43:29.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.748+0000 7f513d29f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8d190 0x55c6c8eb0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.748+0000 7f513d29f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.748+0000 7f513d29f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ee3a70 0x55c6c8ef2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.748+0000 7f513d29f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5134200f20 con 0x55c6c8ed1aa0 2024-09-16T15:43:29.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.748+0000 7f513d29f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51341fffe0 con 0x55c6c8ee3a70 2024-09-16T15:43:29.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.748+0000 7f513d29f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f51341f54e0 con 0x55c6c8e8d190 2024-09-16T15:43:29.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.748+0000 7f512aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f512aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38634/0 (socket says 172.21.15.60:38634) 2024-09-16T15:43:29.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f512aa00640 1 -- 172.21.15.60:0/986922055 learned_addr learned my addr 172.21.15.60:0/986922055 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f5133e00640 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ee3a70 0x55c6c8ef2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f5133400640 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8d190 0x55c6c8eb0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f512aa00640 1 -- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8d190 msgr2=0x55c6c8eb0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f512aa00640 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8d190 0x55c6c8eb0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f512aa00640 1 -- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ee3a70 msgr2=0x55c6c8ef2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f512aa00640 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ee3a70 0x55c6c8ef2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f512aa00640 1 -- 172.21.15.60:0/986922055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c8edff20 con 0x55c6c8ed1aa0 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f5133e00640 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ee3a70 0x55c6c8ef2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f512aa00640 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb1250 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f511c0620c0 tx=0x7f511c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d05c62d5816fe43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f5123400640 1 -- 172.21.15.60:0/986922055 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0623c0 con 0x55c6c8ed1aa0 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f5123400640 1 -- 172.21.15.60:0/986922055 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f511c066ea0 con 0x55c6c8ed1aa0 2024-09-16T15:43:29.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.749+0000 7f5123400640 1 -- 172.21.15.60:0/986922055 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c06bd50 con 0x55c6c8ed1aa0 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 -- 172.21.15.60:0/986922055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ed1aa0 msgr2=0x55c6c8eb1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb1250 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f511c0620c0 tx=0x7f511c005040 comp rx=0 tx=0).stop 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 -- 172.21.15.60:0/986922055 shutdown_connections 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8ee3a70 0x55c6c8ef2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ed1aa0 0x55c6c8eb1250 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 --2- 172.21.15.60:0/986922055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8e8d190 0x55c6c8eb0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 -- 172.21.15.60:0/986922055 >> 172.21.15.60:0/986922055 conn(0x55c6c8e8cc10 msgr2=0x55c6c8ec6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 -- 172.21.15.60:0/986922055 shutdown_connections 2024-09-16T15:43:29.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.750+0000 7f513d29f740 1 -- 172.21.15.60:0/986922055 wait complete. 2024-09-16T15:43:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.751+0000 7f513d29f740 1 -- start start 2024-09-16T15:43:29.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.751+0000 7f513d29f740 1 Processor -- start 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f513d29f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8e8d190 0x55c6c8f476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f513d29f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f513d29f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ee3a70 0x55c6c8eb1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f513d29f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c6c8eb0b90 con 0x55c6c8ee3a70 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f513d29f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51340b7980 con 0x55c6c8e8d190 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f513d29f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c6c8ecb000 con 0x55c6c8ed1aa0 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f513d29f740 10 client.0 _ll_register_callbacks cb 0x7f5124003dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f5133e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ee3a70 0x55c6c8eb1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f5133e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ee3a70 0x55c6c8eb1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38642/0 (socket says 172.21.15.60:38642) 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.752+0000 7f5133e00640 1 -- 172.21.15.60:0/1656106741 learned_addr learned my addr 172.21.15.60:0/1656106741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5133400640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8e8d190 0x55c6c8f476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5133e00640 1 -- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ed1aa0 msgr2=0x55c6c8eb0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f512aa00640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5133e00640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5133e00640 1 -- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8e8d190 msgr2=0x55c6c8f476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5133e00640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8e8d190 0x55c6c8f476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5133e00640 1 -- 172.21.15.60:0/1656106741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c8eb3fd0 con 0x55c6c8ee3a70 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f512aa00640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5133400640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8e8d190 0x55c6c8f476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5133e00640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ee3a70 0x55c6c8eb1250 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f512c016040 tx=0x7f512c00ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6495983620235107 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5120c00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5120c00640 1 -- 172.21.15.60:0/1656106741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c005450 con 0x55c6c8ee3a70 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f5120c00640 1 -- 172.21.15.60:0/1656106741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f512c006890 con 0x55c6c8ee3a70 2024-09-16T15:43:29.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.753+0000 7f513d29f740 1 -- 172.21.15.60:0/1656106741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511c066ea0 con 0x55c6c8ee3a70 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f5120c00640 1 -- 172.21.15.60:0/1656106741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f512c02f050 con 0x55c6c8ee3a70 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f513d29f740 10 client.6693 fetch_fsmap learned FSMap version 57 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f513d29f740 1 -- 172.21.15.60:0/1656106741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55c6c8eca1a0 con 0x55c6c8ee3a70 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f5120c00640 1 -- 172.21.15.60:0/1656106741 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f511c066ea0 con 0x55c6c8ee3a70 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f5120c00640 1 -- 172.21.15.60:0/1656106741 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 57) ==== 3502+0+0 (secure 0 0 0) 0x7f512c032020 con 0x55c6c8ee3a70 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f513d29f740 10 client.6693 fetch_fsmap finished waiting for FSMap version 57 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f513d29f740 10 client.6693 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f513d29f740 20 client.6693 populate_metadata read hostname 'smithi060' 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f513d29f740 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] conn(0x55c6c8f54ca0 0x55c6c8edede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f513d29f740 4 client.6693 mds_command: new command op to 6576 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f513d29f740 1 -- 172.21.15.60:0/1656106741 --> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] -- command(tid 0: {"prefix": "status"}) -- 0x55c6c8dfcfd0 con 0x55c6c8f54ca0 2024-09-16T15:43:29.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.754+0000 7f5133400640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] conn(0x55c6c8f54ca0 0x55c6c8edede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f5133400640 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] conn(0x55c6c8f54ca0 0x55c6c8edede0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c79c42e6ab3466 server_cookie=bf1cb66f6faa9bf4 in_seq=0 out_seq=0 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f5120c00640 10 client.6693 ms_handle_connect on v2:172.21.15.60:6832/3103295284 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f5120c00640 1 -- 172.21.15.60:0/1656106741 <== mds.0 v2:172.21.15.60:6832/3103295284 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55c6c8dfcfd0 con 0x55c6c8f54ca0 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f5120c00640 10 client.6693 handle_command_reply: tid=0 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 client.6693 shutdown 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 -- 172.21.15.60:0/1656106741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ee3a70 msgr2=0x55c6c8eb1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ee3a70 0x55c6c8eb1250 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f512c016040 tx=0x7f512c00ff10 comp rx=0 tx=0).stop 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 -- 172.21.15.60:0/1656106741 shutdown_connections 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.60:6832/3103295284,v1:172.21.15.60:6834/3103295284] conn(0x55c6c8f54ca0 0x55c6c8edede0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c6c8ee3a70 0x55c6c8eb1250 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c6c8ed1aa0 0x55c6c8eb0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 --2- 172.21.15.60:0/1656106741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c6c8e8d190 0x55c6c8f476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f5120c00640 0 client.6693 ms_handle_reset on v2:172.21.15.60:6832/3103295284 2024-09-16T15:43:29.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 -- 172.21.15.60:0/1656106741 >> 172.21.15.60:0/1656106741 conn(0x55c6c8e8cc10 msgr2=0x55c6c8eb02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:29.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 -- 172.21.15.60:0/1656106741 shutdown_connections 2024-09-16T15:43:29.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.755+0000 7f513d29f740 1 -- 172.21.15.60:0/1656106741 wait complete. 2024-09-16T15:43:29.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f513d29f740 20 client.6693 trim_cache size 0 max 16384 2024-09-16T15:43:29.762 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:43:29.762 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6576, 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2392010029999998, 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 57, 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 87, 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 87, 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.529688536 2024-09-16T15:43:29.763 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:43:29.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 -- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5110078970 msgr2=0x7f511007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5110078970 0x7f511007ae30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5124034000 tx=0x7f5124004ab0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 -- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4050 msgr2=0x7f51341f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4050 0x7f51341f7c50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c530 tx=0x7f511c00c9f0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 -- 172.21.15.60:0/2245380414 shutdown_connections 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5110078970 0x7f511007ae30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5134161440 0x7f51341f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51340b4050 0x7f51341f7c50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 --2- 172.21.15.60:0/2245380414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51340b34b0 0x7f51340cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 -- 172.21.15.60:0/2245380414 >> 172.21.15.60:0/2245380414 conn(0x7f51340d3df0 msgr2=0x7f51340d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 -- 172.21.15.60:0/2245380414 shutdown_connections 2024-09-16T15:43:29.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.756+0000 7f5138e00640 1 -- 172.21.15.60:0/2245380414 wait complete. 2024-09-16T15:43:29.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:43:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.932+0000 7fe6e0e00640 1 -- 172.21.15.60:0/2041621737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6dc0cdd00 msgr2=0x7fe6dc0adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.932+0000 7fe6e0e00640 1 --2- 172.21.15.60:0/2041621737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6dc0cdd00 0x7fe6dc0adcd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005fd0 tx=0x7fe6d402a840 comp rx=0 tx=0).stop 2024-09-16T15:43:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.933+0000 7fe6e0e00640 1 -- 172.21.15.60:0/2041621737 shutdown_connections 2024-09-16T15:43:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.933+0000 7fe6e0e00640 1 --2- 172.21.15.60:0/2041621737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6dc0cdd00 0x7fe6dc0adcd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.933+0000 7fe6e0e00640 1 -- 172.21.15.60:0/2041621737 >> 172.21.15.60:0/2041621737 conn(0x7fe6dc0cfa20 msgr2=0x7fe6dc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:29.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.933+0000 7fe6e0e00640 1 -- 172.21.15.60:0/2041621737 shutdown_connections 2024-09-16T15:43:29.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.933+0000 7fe6e0e00640 1 -- 172.21.15.60:0/2041621737 wait complete. 2024-09-16T15:43:29.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.934+0000 7fe6e0e00640 1 Processor -- start 2024-09-16T15:43:29.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.935+0000 7fe6e0e00640 1 -- start start 2024-09-16T15:43:29.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6e0e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6dc06fb90 0x7fe6dc1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6e0e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6dc0ad880 0x7fe6dc0c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6e0e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6dc0cdd00 0x7fe6dc1f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6e0e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc15eb40 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6e0e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc15e9c0 con 0x7fe6dc06fb90 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6e0e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc15ecc0 con 0x7fe6dc0ad880 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6dc0cdd00 0x7fe6dc1f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6dbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6dc0cdd00 0x7fe6dc1f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38650/0 (socket says 172.21.15.60:38650) 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6dbe00640 1 -- 172.21.15.60:0/4026655857 learned_addr learned my addr 172.21.15.60:0/4026655857 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6db400640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6dc06fb90 0x7fe6dc1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6d2a00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6dc0ad880 0x7fe6dc0c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6dbe00640 1 -- 172.21.15.60:0/4026655857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6dc0ad880 msgr2=0x7fe6dc0c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6dbe00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6dc0ad880 0x7fe6dc0c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6dbe00640 1 -- 172.21.15.60:0/4026655857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6dc06fb90 msgr2=0x7fe6dc1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6dbe00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6dc06fb90 0x7fe6dc1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.936+0000 7fe6dbe00640 1 -- 172.21.15.60:0/4026655857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc0c75d0 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.937+0000 7fe6dbe00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6dc0cdd00 0x7fe6dc1f7af0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009cd0 tx=0x7fe6d40315f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.937+0000 7fe6d3e00640 1 -- 172.21.15.60:0/4026655857 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4042030 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.937+0000 7fe6d3e00640 1 -- 172.21.15.60:0/4026655857 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6d4031ae0 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.937+0000 7fe6d3e00640 1 -- 172.21.15.60:0/4026655857 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4003970 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.937+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc0c7b60 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.937+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc157d00 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.938+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc15e390 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.939+0000 7fe6d3e00640 1 -- 172.21.15.60:0/4026655857 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fe6d4045070 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.940+0000 7fe6d3e00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe6b8078970 0x7fe6b807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:29.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.940+0000 7fe6d3e00640 1 -- 172.21.15.60:0/4026655857 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fe6d40bbe20 con 0x7fe6dc0cdd00 2024-09-16T15:43:29.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.941+0000 7fe6db400640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe6b8078970 0x7fe6b807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:29.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.941+0000 7fe6db400640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe6b8078970 0x7fe6b807ae30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc001eb0 tx=0x7fe6cc009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:29.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:29.945+0000 7fe6d3e00640 1 -- 172.21.15.60:0/4026655857 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe6dc15e390 con 0x7fe6dc0cdd00 2024-09-16T15:43:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.181+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe6dc15f930 con 0x7fe6dc0cdd00 2024-09-16T15:43:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.182+0000 7fe6d3e00640 1 -- 172.21.15.60:0/4026655857 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) ==== 53+0+73 (secure 0 0 0) 0x7fe6d4083b80 con 0x7fe6dc0cdd00 2024-09-16T15:43:30.183 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe6b8078970 msgr2=0x7fe6b807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe6b8078970 0x7fe6b807ae30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc001eb0 tx=0x7fe6cc009040 comp rx=0 tx=0).stop 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6dc0cdd00 msgr2=0x7fe6dc1f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6dc0cdd00 0x7fe6dc1f7af0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009cd0 tx=0x7fe6d40315f0 comp rx=0 tx=0).stop 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 shutdown_connections 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe6b8078970 0x7fe6b807ae30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6dc0cdd00 0x7fe6dc1f7af0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6dc0ad880 0x7fe6dc0c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 --2- 172.21.15.60:0/4026655857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6dc06fb90 0x7fe6dc1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.184+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 >> 172.21.15.60:0/4026655857 conn(0x7fe6dc0cfa20 msgr2=0x7fe6dc160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.185+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 shutdown_connections 2024-09-16T15:43:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30.185+0000 7fe6e0e00640 1 -- 172.21.15.60:0/4026655857 wait complete. 2024-09-16T15:43:30.201 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:43:30.202 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:43:30.202 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:43:30.202 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:43:30.202 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:43:30.202 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:43:30.202 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:43:30.202 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:43:30.202 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:43:30.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:30.202 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:43:30.237 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:43:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:43:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42439sec preferred_lft 42439sec 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:30.239 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:43:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:43:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:43:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:30.240 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:43:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:43:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:43:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:30.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:30.241 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:43:30.334 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:43:30.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:30.337 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:43:30.373 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:43:30.374 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:43:30.374 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:43:30.374 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:43:30.374 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:43:30.374 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:43:30.374 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:43:30.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:43:30.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:43:30.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:43:30.374 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:43:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:43:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:43:30.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:30.574 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:43:30.609 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:43:30.610 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:43:30.610 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:43:30.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:30.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:43:30.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:43:30.707 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:43:30.742 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:43:30.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:43:30.742 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:43:30.816 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:43:30.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:30.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:43:30.945 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:43:30.946 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:43:30.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:43:30.973 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:43:30.973 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:43:30.974 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:43:30.974 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:43:30.974 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:43:30.977 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:43:30.979 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:43:30.979 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:43:30.979 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:43:30.979 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:43:30.979 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:43:30.979 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:43:30.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:30.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:43:31.075 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:43:31.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:43:31.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:43:31.213 INFO:teuthology.orchestra.run.smithi060.stdout:client6702 2024-09-16T15:43:31.213 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:31.213 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:31.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:43:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.449+0000 7ff57f200640 1 -- 172.21.15.60:0/1962517969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 msgr2=0x7ff564004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.449+0000 7ff57f200640 1 --2- 172.21.15.60:0/1962517969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 0x7ff564004c80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c2e0 tx=0x7ff564005830 comp rx=0 tx=0).stop 2024-09-16T15:43:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.449+0000 7ff57f200640 1 -- 172.21.15.60:0/1962517969 shutdown_connections 2024-09-16T15:43:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.449+0000 7ff57f200640 1 --2- 172.21.15.60:0/1962517969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 0x7ff564004c80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.449+0000 7ff57f200640 1 -- 172.21.15.60:0/1962517969 >> 172.21.15.60:0/1962517969 conn(0x7ff5800cfa20 msgr2=0x7ff5800cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.449+0000 7ff57f200640 1 -- 172.21.15.60:0/1962517969 shutdown_connections 2024-09-16T15:43:31.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.449+0000 7ff57f200640 1 -- 172.21.15.60:0/1962517969 wait complete. 2024-09-16T15:43:31.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.450+0000 7ff57f200640 1 Processor -- start 2024-09-16T15:43:31.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.451+0000 7ff57f200640 1 -- start start 2024-09-16T15:43:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.451+0000 7ff57f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 0x7ff5801f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800b10e0 0x7ff580150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5800ce630 0x7ff5800cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800b2c80 con 0x7ff564004800 2024-09-16T15:43:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800b2b00 con 0x7ff5800ce630 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff580073d40 con 0x7ff5800b10e0 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 0x7ff5801f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 0x7ff5801f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38670/0 (socket says 172.21.15.60:38670) 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57de00640 1 -- 172.21.15.60:0/2127189877 learned_addr learned my addr 172.21.15.60:0/2127189877 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57e800640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5800ce630 0x7ff5800cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57d400640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800b10e0 0x7ff580150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57de00640 1 -- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800b10e0 msgr2=0x7ff580150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57de00640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800b10e0 0x7ff580150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57de00640 1 -- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5800ce630 msgr2=0x7ff5800cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57de00640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5800ce630 0x7ff5800cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57de00640 1 -- 172.21.15.60:0/2127189877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5801f5820 con 0x7ff564004800 2024-09-16T15:43:31.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.452+0000 7ff57d400640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800b10e0 0x7ff580150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:31.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.453+0000 7ff57de00640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 0x7ff5801f52e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff570011780 tx=0x7ff570011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:31.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.453+0000 7ff576000640 1 -- 172.21.15.60:0/2127189877 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000fd50 con 0x7ff564004800 2024-09-16T15:43:31.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.453+0000 7ff576000640 1 -- 172.21.15.60:0/2127189877 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff570011e90 con 0x7ff564004800 2024-09-16T15:43:31.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.453+0000 7ff576000640 1 -- 172.21.15.60:0/2127189877 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570004290 con 0x7ff564004800 2024-09-16T15:43:31.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.453+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5800b0040 con 0x7ff564004800 2024-09-16T15:43:31.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.453+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5800b0600 con 0x7ff564004800 2024-09-16T15:43:31.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.454+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58015b490 con 0x7ff564004800 2024-09-16T15:43:31.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.455+0000 7ff576000640 1 -- 172.21.15.60:0/2127189877 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7ff5700035f0 con 0x7ff564004800 2024-09-16T15:43:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.456+0000 7ff576000640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff560078970 0x7ff56007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.456+0000 7ff576000640 1 -- 172.21.15.60:0/2127189877 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7ff57009b780 con 0x7ff564004800 2024-09-16T15:43:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.457+0000 7ff57d400640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff560078970 0x7ff56007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.457+0000 7ff57d400640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff560078970 0x7ff56007ae30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff56c033000 tx=0x7ff56c001a60 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:31.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.461+0000 7ff576000640 1 -- 172.21.15.60:0/2127189877 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff58015b490 con 0x7ff564004800 2024-09-16T15:43:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.702+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff5800742a0 con 0x7ff564004800 2024-09-16T15:43:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.702+0000 7ff576000640 1 -- 172.21.15.60:0/2127189877 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) ==== 53+0+73 (secure 0 0 0) 0x7ff570063570 con 0x7ff564004800 2024-09-16T15:43:31.703 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:43:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff560078970 msgr2=0x7ff56007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff560078970 0x7ff56007ae30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff56c033000 tx=0x7ff56c001a60 comp rx=0 tx=0).stop 2024-09-16T15:43:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 msgr2=0x7ff5801f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 0x7ff5801f52e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff570011780 tx=0x7ff570011c40 comp rx=0 tx=0).stop 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 shutdown_connections 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff560078970 0x7ff56007ae30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5800ce630 0x7ff5800cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800b10e0 0x7ff580150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 --2- 172.21.15.60:0/2127189877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff564004800 0x7ff5801f52e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 >> 172.21.15.60:0/2127189877 conn(0x7ff5800cfa20 msgr2=0x7ff5800ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 shutdown_connections 2024-09-16T15:43:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:31.705+0000 7ff57f200640 1 -- 172.21.15.60:0/2127189877 wait complete. 2024-09-16T15:43:31.721 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:43:31.721 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:43:31.721 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:43:31.721 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:43:31.722 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:43:31.722 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:43:31.722 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:43:31.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:43:31.722 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:43:31.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:31.722 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:43:31.762 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42437sec preferred_lft 42437sec 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:43:31.763 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:43:31.764 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:43:31.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:31.765 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:43:31.862 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:43:31.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:31.865 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:43:31.901 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:43:31.902 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:43:31.902 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:43:31.902 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:43:31.902 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:43:31.902 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:43:31.902 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:43:31.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:43:31.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:43:31.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:43:31.902 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:43:32.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:43:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:43:32.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:43:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:43:32.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:32.098 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:43:32.133 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:43:32.133 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:43:32.134 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:43:32.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:32.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:43:32.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:43:32.229 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:43:32.264 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:43:32.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:43:32.264 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:43:32.340 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:43:32.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:32.340 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:43:32.468 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:43:32.468 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:43:32.497 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:43:32.497 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:43:32.497 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:43:32.497 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:43:32.497 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:43:32.499 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:43:32.499 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:43:32.500 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:43:32.500 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:43:32.500 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:43:32.502 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:43:32.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:32.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:43:32.558 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:43:32.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:43:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:43:32.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:43:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:43:32.649 INFO:teuthology.orchestra.run.smithi100.stdout:client5939 2024-09-16T15:43:32.649 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:32.649 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:32.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:43:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.861+0000 7f0ead800640 1 -- 172.21.15.60:0/2321279417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c003a00 msgr2=0x7f0e8c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.861+0000 7f0ead800640 1 --2- 172.21.15.60:0/2321279417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c003a00 0x7f0e8c003de0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009ed0 tx=0x7f0e8c005830 comp rx=0 tx=0).stop 2024-09-16T15:43:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.861+0000 7f0ead800640 1 -- 172.21.15.60:0/2321279417 shutdown_connections 2024-09-16T15:43:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.861+0000 7f0ead800640 1 --2- 172.21.15.60:0/2321279417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c003a00 0x7f0e8c003de0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.861+0000 7f0ead800640 1 -- 172.21.15.60:0/2321279417 >> 172.21.15.60:0/2321279417 conn(0x7f0ea80b18e0 msgr2=0x7f0ea80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.861+0000 7f0ead800640 1 -- 172.21.15.60:0/2321279417 shutdown_connections 2024-09-16T15:43:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.861+0000 7f0ead800640 1 -- 172.21.15.60:0/2321279417 wait complete. 2024-09-16T15:43:32.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.863+0000 7f0ead800640 1 Processor -- start 2024-09-16T15:43:32.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.863+0000 7f0ead800640 1 -- start start 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ead800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c003a00 0x7f0ea8159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ead800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea8151aa0 0x7f0ea81451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ead800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea8156440 0x7f0ea80cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ead800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea806f830 con 0x7f0ea8156440 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ead800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea806f6b0 con 0x7f0ea8151aa0 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ead800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea806f9b0 con 0x7f0e8c003a00 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ea6a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c003a00 0x7f0ea8159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea8156440 0x7f0ea80cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ea6a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c003a00 0x7f0ea8159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:42576/0 (socket says 172.21.15.60:42576) 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ea6a00640 1 -- 172.21.15.60:0/972604058 learned_addr learned my addr 172.21.15.60:0/972604058 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.864+0000 7f0ea6000640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea8151aa0 0x7f0ea81451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ea6a00640 1 -- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea8151aa0 msgr2=0x7f0ea81451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ea6a00640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea8151aa0 0x7f0ea81451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ea6a00640 1 -- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea8156440 msgr2=0x7f0ea80cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ea6a00640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea8156440 0x7f0ea80cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ea6a00640 1 -- 172.21.15.60:0/972604058 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea8147bf0 con 0x7f0e8c003a00 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ea7400640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea8156440 0x7f0ea80cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ea6000640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea8151aa0 0x7f0ea81451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ea6a00640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c003a00 0x7f0ea8159f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009e20 tx=0x7f0e98033cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0e9ea00640 1 -- 172.21.15.60:0/972604058 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98038040 con 0x7f0e8c003a00 2024-09-16T15:43:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0e9ea00640 1 -- 172.21.15.60:0/972604058 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e98041070 con 0x7f0e8c003a00 2024-09-16T15:43:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0e9ea00640 1 -- 172.21.15.60:0/972604058 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98009200 con 0x7f0e8c003a00 2024-09-16T15:43:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.865+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea8147ed0 con 0x7f0e8c003a00 2024-09-16T15:43:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.866+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea8148490 con 0x7f0e8c003a00 2024-09-16T15:43:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.866+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea8069360 con 0x7f0e8c003a00 2024-09-16T15:43:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.868+0000 7f0e9ea00640 1 -- 172.21.15.60:0/972604058 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f0e9803c650 con 0x7f0e8c003a00 2024-09-16T15:43:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.869+0000 7f0e9ea00640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e880789a0 0x7f0e8807ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.869+0000 7f0e9ea00640 1 -- 172.21.15.60:0/972604058 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f0e980bb940 con 0x7f0e8c003a00 2024-09-16T15:43:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.869+0000 7f0ea6000640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e880789a0 0x7f0e8807ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:32.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.870+0000 7f0ea6000640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e880789a0 0x7f0e8807ae60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8151e80 tx=0x7f0e940067a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:32.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:32.872+0000 7f0e9ea00640 1 -- 172.21.15.60:0/972604058 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0e98083fc0 con 0x7f0e8c003a00 2024-09-16T15:43:33.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.110+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f0ea8148870 con 0x7f0e8c003a00 2024-09-16T15:43:33.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.578+0000 7f0e9ea00640 1 -- 172.21.15.60:0/972604058 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v58) ==== 136+0+0 (secure 0 0 0) 0x7f0ea8148870 con 0x7f0e8c003a00 2024-09-16T15:43:33.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e880789a0 msgr2=0x7f0e8807ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e880789a0 0x7f0e8807ae60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8151e80 tx=0x7f0e940067a0 comp rx=0 tx=0).stop 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c003a00 msgr2=0x7f0ea8159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c003a00 0x7f0ea8159f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009e20 tx=0x7f0e98033cb0 comp rx=0 tx=0).stop 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 shutdown_connections 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e880789a0 0x7f0e8807ae60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ea8156440 0x7f0ea80cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ea8151aa0 0x7f0ea81451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 --2- 172.21.15.60:0/972604058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c003a00 0x7f0ea8159f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 >> 172.21.15.60:0/972604058 conn(0x7f0ea80b18e0 msgr2=0x7f0ea80ad4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:33.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 shutdown_connections 2024-09-16T15:43:33.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.581+0000 7f0ead800640 1 -- 172.21.15.60:0/972604058 wait complete. 2024-09-16T15:43:33.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:43:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.763+0000 7f2a6e200640 1 -- 172.21.15.60:0/1170305426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a50004050 msgr2=0x7f2a50004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.763+0000 7f2a6e200640 1 --2- 172.21.15.60:0/1170305426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a50004050 0x7f2a50004800 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009b20 tx=0x7f2a50005830 comp rx=0 tx=0).stop 2024-09-16T15:43:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.763+0000 7f2a6d800640 1 -- 172.21.15.60:0/1170305426 reap_dead start 2024-09-16T15:43:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.763+0000 7f2a6e200640 1 -- 172.21.15.60:0/1170305426 shutdown_connections 2024-09-16T15:43:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.763+0000 7f2a6e200640 1 -- 172.21.15.60:0/1170305426 >> 172.21.15.60:0/1170305426 conn(0x7f2a680d1c40 msgr2=0x7f2a680d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.763+0000 7f2a6e200640 1 -- 172.21.15.60:0/1170305426 shutdown_connections 2024-09-16T15:43:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.763+0000 7f2a6e200640 1 -- 172.21.15.60:0/1170305426 wait complete. 2024-09-16T15:43:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.765+0000 7f2a6e200640 1 Processor -- start 2024-09-16T15:43:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.765+0000 7f2a6e200640 1 -- start start 2024-09-16T15:43:33.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a50004050 0x7f2a6809dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6e200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a680b2a90 0x7f2a681f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a681f8270 0x7f2a681f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6e200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6806eef0 con 0x7f2a681f8270 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6806ed70 con 0x7f2a50004050 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6e200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6806f070 con 0x7f2a680b2a90 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a681f8270 0x7f2a681f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6d800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a681f8270 0x7f2a681f86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38702/0 (socket says 172.21.15.60:38702) 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6d800640 1 -- 172.21.15.60:0/4226494724 learned_addr learned my addr 172.21.15.60:0/4226494724 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a67e00640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a680b2a90 0x7f2a681f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6ce00640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a50004050 0x7f2a6809dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6d800640 1 -- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a680b2a90 msgr2=0x7f2a681f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6d800640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a680b2a90 0x7f2a681f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6d800640 1 -- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a50004050 msgr2=0x7f2a6809dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6d800640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a50004050 0x7f2a6809dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6d800640 1 -- 172.21.15.60:0/4226494724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a680b2f10 con 0x7f2a681f8270 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.766+0000 7f2a6ce00640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a50004050 0x7f2a6809dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.767+0000 7f2a6d800640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a681f8270 0x7f2a681f86f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2a6806fc90 tx=0x7f2a60001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.767+0000 7f2a64c00640 1 -- 172.21.15.60:0/4226494724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60015070 con 0x7f2a681f8270 2024-09-16T15:43:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.767+0000 7f2a64c00640 1 -- 172.21.15.60:0/4226494724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a60011920 con 0x7f2a681f8270 2024-09-16T15:43:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.767+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6809e2b0 con 0x7f2a681f8270 2024-09-16T15:43:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.767+0000 7f2a64c00640 1 -- 172.21.15.60:0/4226494724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60011d70 con 0x7f2a681f8270 2024-09-16T15:43:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.767+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6809e870 con 0x7f2a681f8270 2024-09-16T15:43:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.768+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a681f7d80 con 0x7f2a681f8270 2024-09-16T15:43:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.769+0000 7f2a64c00640 1 -- 172.21.15.60:0/4226494724 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f2a60018070 con 0x7f2a681f8270 2024-09-16T15:43:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.770+0000 7f2a64c00640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2a480789a0 0x7f2a4807ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.770+0000 7f2a64c00640 1 -- 172.21.15.60:0/4226494724 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f2a6009b840 con 0x7f2a681f8270 2024-09-16T15:43:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.771+0000 7f2a6ce00640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2a480789a0 0x7f2a4807ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.771+0000 7f2a6ce00640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2a480789a0 0x7f2a4807ae60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c001eb0 tx=0x7f2a5c006720 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.775+0000 7f2a64c00640 1 -- 172.21.15.60:0/4226494724 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2a60063c80 con 0x7f2a681f8270 2024-09-16T15:43:33.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.994+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f2a6809ea40 con 0x7f2a480789a0 2024-09-16T15:43:33.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.995+0000 7f2a64c00640 1 -- 172.21.15.60:0/4226494724 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f2a5c0036b0 con 0x7f2a480789a0 2024-09-16T15:43:33.997 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:43:33.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:33.997 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:43:33.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:33.997 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2a480789a0 msgr2=0x7f2a4807ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2a480789a0 0x7f2a4807ae60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c001eb0 tx=0x7f2a5c006720 comp rx=0 tx=0).stop 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a681f8270 msgr2=0x7f2a681f86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a681f8270 0x7f2a681f86f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2a6806fc90 tx=0x7f2a60001c10 comp rx=0 tx=0).stop 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 shutdown_connections 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2a480789a0 0x7f2a4807ae60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a681f8270 0x7f2a681f86f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a680b2a90 0x7f2a681f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 --2- 172.21.15.60:0/4226494724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a50004050 0x7f2a6809dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 >> 172.21.15.60:0/4226494724 conn(0x7f2a680d1c40 msgr2=0x7f2a6815e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:34.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.998+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 shutdown_connections 2024-09-16T15:43:34.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:33.999+0000 7f2a6e200640 1 -- 172.21.15.60:0/4226494724 wait complete. 2024-09-16T15:43:34.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:43:34.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.177+0000 7fe68c200640 1 -- 172.21.15.60:0/2229182704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68415e250 msgr2=0x7fe6840d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.177+0000 7fe68c200640 1 --2- 172.21.15.60:0/2229182704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68415e250 0x7fe6840d0ac0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe678009900 tx=0x7fe67802a600 comp rx=0 tx=0).stop 2024-09-16T15:43:34.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.177+0000 7fe68c200640 1 -- 172.21.15.60:0/2229182704 shutdown_connections 2024-09-16T15:43:34.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.177+0000 7fe68c200640 1 --2- 172.21.15.60:0/2229182704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe68406ed00 0x7fe684161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.177+0000 7fe68c200640 1 --2- 172.21.15.60:0/2229182704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe68415e250 0x7fe6840d0ac0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.177+0000 7fe68c200640 1 -- 172.21.15.60:0/2229182704 >> 172.21.15.60:0/2229182704 conn(0x7fe6840d1e20 msgr2=0x7fe6840d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:34.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.177+0000 7fe68c200640 1 -- 172.21.15.60:0/2229182704 shutdown_connections 2024-09-16T15:43:34.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.177+0000 7fe68c200640 1 -- 172.21.15.60:0/2229182704 wait complete. 2024-09-16T15:43:34.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.179+0000 7fe68c200640 1 Processor -- start 2024-09-16T15:43:34.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.179+0000 7fe68c200640 1 -- start start 2024-09-16T15:43:34.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe68c200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe68406ed00 0x7fe684161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe68c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6840d1000 0x7fe6841f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe68c200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe68415e250 0x7fe68415e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe68c200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841624f0 con 0x7fe68406ed00 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe68c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe684162370 con 0x7fe6840d1000 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe68c200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe6840cef10 con 0x7fe68415e250 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe689800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe68406ed00 0x7fe684161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe689800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe68406ed00 0x7fe684161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38708/0 (socket says 172.21.15.60:38708) 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe689800640 1 -- 172.21.15.60:0/3651722762 learned_addr learned my addr 172.21.15.60:0/3651722762 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.180+0000 7fe688e00640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6840d1000 0x7fe6841f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe68a200640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe68415e250 0x7fe68415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe689800640 1 -- 172.21.15.60:0/3651722762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe68415e250 msgr2=0x7fe68415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe689800640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe68415e250 0x7fe68415e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe689800640 1 -- 172.21.15.60:0/3651722762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6840d1000 msgr2=0x7fe6841f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe689800640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6840d1000 0x7fe6841f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe689800640 1 -- 172.21.15.60:0/3651722762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6841f5820 con 0x7fe68406ed00 2024-09-16T15:43:34.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe689800640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe68406ed00 0x7fe684161490 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe678000c00 tx=0x7fe6780038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe681600640 1 -- 172.21.15.60:0/3651722762 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67803e030 con 0x7fe68406ed00 2024-09-16T15:43:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe681600640 1 -- 172.21.15.60:0/3651722762 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe678003d90 con 0x7fe68406ed00 2024-09-16T15:43:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe681600640 1 -- 172.21.15.60:0/3651722762 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678035040 con 0x7fe68406ed00 2024-09-16T15:43:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.181+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6841f5b00 con 0x7fe68406ed00 2024-09-16T15:43:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.182+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6840c6960 con 0x7fe68406ed00 2024-09-16T15:43:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.183+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6840cf410 con 0x7fe68406ed00 2024-09-16T15:43:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.184+0000 7fe681600640 1 -- 172.21.15.60:0/3651722762 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fe678041030 con 0x7fe68406ed00 2024-09-16T15:43:34.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.185+0000 7fe681600640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe664078970 0x7fe66407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.185+0000 7fe681600640 1 -- 172.21.15.60:0/3651722762 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fe6780bb4a0 con 0x7fe68406ed00 2024-09-16T15:43:34.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.185+0000 7fe688e00640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe664078970 0x7fe66407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.186+0000 7fe688e00640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe664078970 0x7fe66407ae30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6700095f0 tx=0x7fe670005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:34.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.189+0000 7fe681600640 1 -- 172.21.15.60:0/3651722762 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe678083290 con 0x7fe68406ed00 2024-09-16T15:43:34.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.383+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe684161f40 con 0x7fe68406ed00 2024-09-16T15:43:34.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.384+0000 7fe681600640 1 -- 172.21.15.60:0/3651722762 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fe678087cc0 con 0x7fe68406ed00 2024-09-16T15:43:34.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe664078970 msgr2=0x7fe66407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe664078970 0x7fe66407ae30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6700095f0 tx=0x7fe670005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe68406ed00 msgr2=0x7fe684161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe68406ed00 0x7fe684161490 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe678000c00 tx=0x7fe6780038a0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 shutdown_connections 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe664078970 0x7fe66407ae30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe68415e250 0x7fe68415e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6840d1000 0x7fe6841f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 --2- 172.21.15.60:0/3651722762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe68406ed00 0x7fe684161490 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 >> 172.21.15.60:0/3651722762 conn(0x7fe6840d1e20 msgr2=0x7fe6840aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 shutdown_connections 2024-09-16T15:43:34.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.386+0000 7fe68c200640 1 -- 172.21.15.60:0/3651722762 wait complete. 2024-09-16T15:43:34.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-16T15:43:34.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.565+0000 7fef52400640 1 -- 172.21.15.60:0/1863179052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef4c0b5480 msgr2=0x7fef4c0d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.565+0000 7fef52400640 1 --2- 172.21.15.60:0/1863179052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef4c0b5480 0x7fef4c0d2830 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fef4c06cdc0 tx=0x7fef3402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.565+0000 7fef52400640 1 -- 172.21.15.60:0/1863179052 shutdown_connections 2024-09-16T15:43:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.565+0000 7fef52400640 1 --2- 172.21.15.60:0/1863179052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef4c0b5480 0x7fef4c0d2830 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.565+0000 7fef52400640 1 --2- 172.21.15.60:0/1863179052 >> v2:172.21.15.100:3300/0 conn(0x7fef4c161460 0x7fef4c0d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.565+0000 7fef52400640 1 -- 172.21.15.60:0/1863179052 >> 172.21.15.60:0/1863179052 conn(0x7fef4c09d0b0 msgr2=0x7fef4c0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.565+0000 7fef52400640 1 -- 172.21.15.60:0/1863179052 shutdown_connections 2024-09-16T15:43:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.565+0000 7fef52400640 1 -- 172.21.15.60:0/1863179052 wait complete. 2024-09-16T15:43:34.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.567+0000 7fef52400640 1 Processor -- start 2024-09-16T15:43:34.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.567+0000 7fef52400640 1 -- start start 2024-09-16T15:43:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef52400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef4c0b4f40 0x7fef4c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef52400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef4c0b5480 0x7fef4c09db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef52400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef4c161460 0x7fef4c0b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef52400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c0cef30 con 0x7fef4c0b5480 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef52400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c0b4610 con 0x7fef4c161460 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef52400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c0cf0b0 con 0x7fef4c0b4f40 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef43e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef4c0b5480 0x7fef4c09db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef43e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef4c0b5480 0x7fef4c09db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38728/0 (socket says 172.21.15.60:38728) 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef43e00640 1 -- 172.21.15.60:0/1589057743 learned_addr learned my addr 172.21.15.60:0/1589057743 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef51000640 1 --2- 172.21.15.60:0/1589057743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef4c0b4f40 0x7fef4c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.568+0000 7fef51a00640 1 --2- 172.21.15.60:0/1589057743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef4c161460 0x7fef4c0b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef43e00640 1 -- 172.21.15.60:0/1589057743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef4c0b4f40 msgr2=0x7fef4c15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef43e00640 1 --2- 172.21.15.60:0/1589057743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef4c0b4f40 0x7fef4c15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef43e00640 1 -- 172.21.15.60:0/1589057743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef4c161460 msgr2=0x7fef4c0b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef43e00640 1 --2- 172.21.15.60:0/1589057743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef4c161460 0x7fef4c0b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef43e00640 1 -- 172.21.15.60:0/1589057743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c1f82b0 con 0x7fef4c0b5480 2024-09-16T15:43:34.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef43e00640 1 --2- 172.21.15.60:0/1589057743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef4c0b5480 0x7fef4c09db90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fef34009850 tx=0x7fef34003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef49600640 1 -- 172.21.15.60:0/1589057743 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3403f070 con 0x7fef4c0b5480 2024-09-16T15:43:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef49600640 1 -- 172.21.15.60:0/1589057743 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef340047a0 con 0x7fef4c0b5480 2024-09-16T15:43:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef49600640 1 -- 172.21.15.60:0/1589057743 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34035030 con 0x7fef4c0b5480 2024-09-16T15:43:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef52400640 1 -- 172.21.15.60:0/1589057743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c1f8590 con 0x7fef4c0b5480 2024-09-16T15:43:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.569+0000 7fef52400640 1 -- 172.21.15.60:0/1589057743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c1f8b50 con 0x7fef4c0b5480 2024-09-16T15:43:34.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.571+0000 7fef52400640 1 -- 172.21.15.60:0/1589057743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c0d3780 con 0x7fef4c0b5480 2024-09-16T15:43:34.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.572+0000 7fef49600640 1 -- 172.21.15.60:0/1589057743 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fef34042070 con 0x7fef4c0b5480 2024-09-16T15:43:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.573+0000 7fef49600640 1 --2- 172.21.15.60:0/1589057743 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef28078970 0x7fef2807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.573+0000 7fef49600640 1 -- 172.21.15.60:0/1589057743 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fef340bb8d0 con 0x7fef4c0b5480 2024-09-16T15:43:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.573+0000 7fef51000640 1 --2- 172.21.15.60:0/1589057743 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef28078970 0x7fef2807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.574+0000 7fef51000640 1 --2- 172.21.15.60:0/1589057743 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef28078970 0x7fef2807ae30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef3c004630 tx=0x7fef3c00b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:34.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.577+0000 7fef49600640 1 -- 172.21.15.60:0/1589057743 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fef4c0d3780 con 0x7fef4c0b5480 2024-09-16T15:43:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-16T15:43:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:fs volume create [] : Create a CephFS volume 2024-09-16T15:43:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-16T15:43:34.809 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:43:34.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-16T15:43:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.970+0000 7fac8ac00640 1 -- 172.21.15.60:0/1901029924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac84145750 msgr2=0x7fac84073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.970+0000 7fac8ac00640 1 --2- 172.21.15.60:0/1901029924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac84145750 0x7fac84073440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fac74005fd0 tx=0x7fac7402a540 comp rx=0 tx=0).stop 2024-09-16T15:43:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.970+0000 7fac8ac00640 1 -- 172.21.15.60:0/1901029924 shutdown_connections 2024-09-16T15:43:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.970+0000 7fac8ac00640 1 --2- 172.21.15.60:0/1901029924 >> v2:172.21.15.100:3300/0 conn(0x7fac840cccd0 0x7fac840cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.970+0000 7fac8ac00640 1 --2- 172.21.15.60:0/1901029924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac84145750 0x7fac84073440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.970+0000 7fac8ac00640 1 -- 172.21.15.60:0/1901029924 >> 172.21.15.60:0/1901029924 conn(0x7fac840b1840 msgr2=0x7fac840b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.970+0000 7fac8ac00640 1 -- 172.21.15.60:0/1901029924 shutdown_connections 2024-09-16T15:43:34.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.970+0000 7fac8ac00640 1 -- 172.21.15.60:0/1901029924 wait complete. 2024-09-16T15:43:34.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.972+0000 7fac8ac00640 1 Processor -- start 2024-09-16T15:43:34.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.972+0000 7fac8ac00640 1 -- start start 2024-09-16T15:43:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac8ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8406efa0 0x7fac841f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac8ac00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac840cccd0 0x7fac840b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac8ac00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac84145750 0x7fac840cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac8ac00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fac840ce480 con 0x7fac840cccd0 2024-09-16T15:43:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac8ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac840ce300 con 0x7fac8406efa0 2024-09-16T15:43:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac8ac00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fac840ce600 con 0x7fac84145750 2024-09-16T15:43:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac83400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac840cccd0 0x7fac840b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac83400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac840cccd0 0x7fac840b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38734/0 (socket says 172.21.15.60:38734) 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac83400640 1 -- 172.21.15.60:0/4063768559 learned_addr learned my addr 172.21.15.60:0/4063768559 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac88c00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac84145750 0x7fac840cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.973+0000 7fac83e00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8406efa0 0x7fac841f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac83400640 1 -- 172.21.15.60:0/4063768559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac84145750 msgr2=0x7fac840cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac83400640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac84145750 0x7fac840cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac83400640 1 -- 172.21.15.60:0/4063768559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8406efa0 msgr2=0x7fac841f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac83400640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8406efa0 0x7fac841f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac83400640 1 -- 172.21.15.60:0/4063768559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac840cbd50 con 0x7fac840cccd0 2024-09-16T15:43:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac83400640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac840cccd0 0x7fac840b42a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c450 tx=0x7fac7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac7be00640 1 -- 172.21.15.60:0/4063768559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac700052d0 con 0x7fac840cccd0 2024-09-16T15:43:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac7be00640 1 -- 172.21.15.60:0/4063768559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac70005470 con 0x7fac840cccd0 2024-09-16T15:43:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac7be00640 1 -- 172.21.15.60:0/4063768559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70012870 con 0x7fac840cccd0 2024-09-16T15:43:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.974+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac840b3200 con 0x7fac840cccd0 2024-09-16T15:43:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.975+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac840b37c0 con 0x7fac840cccd0 2024-09-16T15:43:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.976+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac840d0780 con 0x7fac840cccd0 2024-09-16T15:43:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.977+0000 7fac7be00640 1 -- 172.21.15.60:0/4063768559 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fac700035f0 con 0x7fac840cccd0 2024-09-16T15:43:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.978+0000 7fac7be00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac640789a0 0x7fac6407ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:34.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.979+0000 7fac7be00640 1 -- 172.21.15.60:0/4063768559 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fac7009a600 con 0x7fac840cccd0 2024-09-16T15:43:34.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.979+0000 7fac83e00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac640789a0 0x7fac6407ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:34.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.979+0000 7fac83e00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac640789a0 0x7fac6407ae60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac74033000 tx=0x7fac74004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:34.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:34.983+0000 7fac7be00640 1 -- 172.21.15.60:0/4063768559 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fac840cbd50 con 0x7fac840cccd0 2024-09-16T15:43:35.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:35.204+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7fac84069a10 con 0x7fac640789a0 2024-09-16T15:43:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.596+0000 7fac7be00640 1 -- 172.21.15.60:0/4063768559 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7fac84069a10 con 0x7fac640789a0 2024-09-16T15:43:37.598 INFO:teuthology.orchestra.run.smithi060.stderr:Volume created successfully (no MDS daemons created) 2024-09-16T15:43:37.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.601+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac640789a0 msgr2=0x7fac6407ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.601+0000 7fac8ac00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac640789a0 0x7fac6407ae60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac74033000 tx=0x7fac74004730 comp rx=0 tx=0).stop 2024-09-16T15:43:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.601+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac840cccd0 msgr2=0x7fac840b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.601+0000 7fac8ac00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac840cccd0 0x7fac840b42a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c450 tx=0x7fac7000c910 comp rx=0 tx=0).stop 2024-09-16T15:43:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.602+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 shutdown_connections 2024-09-16T15:43:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.602+0000 7fac8ac00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac640789a0 0x7fac6407ae60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:37.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.602+0000 7fac8ac00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac84145750 0x7fac840cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:37.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.602+0000 7fac8ac00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac840cccd0 0x7fac840b42a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:37.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.602+0000 7fac8ac00640 1 --2- 172.21.15.60:0/4063768559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac8406efa0 0x7fac841f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:37.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.602+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 >> 172.21.15.60:0/4063768559 conn(0x7fac840b1840 msgr2=0x7fac840cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:37.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.603+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 shutdown_connections 2024-09-16T15:43:37.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.604+0000 7fac8ac00640 1 -- 172.21.15.60:0/4063768559 wait complete. 2024-09-16T15:43:37.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-16T15:43:37.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.808+0000 7fc122e00640 1 -- 172.21.15.60:0/1309950010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c15ba90 msgr2=0x7fc11c15ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.808+0000 7fc122e00640 1 --2- 172.21.15.60:0/1309950010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c15ba90 0x7fc11c15ef90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0d0c60 tx=0x7fc10c02a560 comp rx=0 tx=0).stop 2024-09-16T15:43:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.808+0000 7fc122e00640 1 -- 172.21.15.60:0/1309950010 shutdown_connections 2024-09-16T15:43:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.808+0000 7fc122e00640 1 --2- 172.21.15.60:0/1309950010 >> v2:172.21.15.100:3300/0 conn(0x7fc11c0cb390 0x7fc11c0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.808+0000 7fc122e00640 1 --2- 172.21.15.60:0/1309950010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c15ba90 0x7fc11c15ef90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.808+0000 7fc122e00640 1 -- 172.21.15.60:0/1309950010 >> 172.21.15.60:0/1309950010 conn(0x7fc11c0d0d40 msgr2=0x7fc11c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.808+0000 7fc122e00640 1 -- 172.21.15.60:0/1309950010 shutdown_connections 2024-09-16T15:43:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.808+0000 7fc122e00640 1 -- 172.21.15.60:0/1309950010 wait complete. 2024-09-16T15:43:37.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.810+0000 7fc122e00640 1 Processor -- start 2024-09-16T15:43:37.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.810+0000 7fc122e00640 1 -- start start 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc122e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c06cb70 0x7fc11c0b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc122e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c0cb390 0x7fc11c154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc122e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c15ba90 0x7fc11c1f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc122e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0cefa0 con 0x7fc11c06cb70 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc122e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0cee20 con 0x7fc11c15ba90 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc122e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c0cf120 con 0x7fc11c0cb390 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc11be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c06cb70 0x7fc11c0b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc11be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c06cb70 0x7fc11c0b69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38748/0 (socket says 172.21.15.60:38748) 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc11be00640 1 -- 172.21.15.60:0/163637702 learned_addr learned my addr 172.21.15.60:0/163637702 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc120e00640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c15ba90 0x7fc11c1f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:37.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc11b400640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c0cb390 0x7fc11c154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc11be00640 1 -- 172.21.15.60:0/163637702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c0cb390 msgr2=0x7fc11c154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.811+0000 7fc11be00640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c0cb390 0x7fc11c154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc11be00640 1 -- 172.21.15.60:0/163637702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c15ba90 msgr2=0x7fc11c1f4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc11be00640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc11c15ba90 0x7fc11c1f4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc11be00640 1 -- 172.21.15.60:0/163637702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c15dd80 con 0x7fc11c06cb70 2024-09-16T15:43:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc11b400640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc11c0cb390 0x7fc11c154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc11be00640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc11c06cb70 0x7fc11c0b69b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc10c034040 tx=0x7fc10c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc113e00640 1 -- 172.21.15.60:0/163637702 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c03e030 con 0x7fc11c06cb70 2024-09-16T15:43:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc113e00640 1 -- 172.21.15.60:0/163637702 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc10c0047a0 con 0x7fc11c06cb70 2024-09-16T15:43:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc113e00640 1 -- 172.21.15.60:0/163637702 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c035040 con 0x7fc11c06cb70 2024-09-16T15:43:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc122e00640 1 -- 172.21.15.60:0/163637702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c154da0 con 0x7fc11c06cb70 2024-09-16T15:43:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.812+0000 7fc122e00640 1 -- 172.21.15.60:0/163637702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c155360 con 0x7fc11c06cb70 2024-09-16T15:43:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.813+0000 7fc122e00640 1 -- 172.21.15.60:0/163637702 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c0cf710 con 0x7fc11c06cb70 2024-09-16T15:43:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.814+0000 7fc113e00640 1 -- 172.21.15.60:0/163637702 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fc10c041030 con 0x7fc11c06cb70 2024-09-16T15:43:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.816+0000 7fc113e00640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc0fc078970 0x7fc0fc07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.816+0000 7fc113e00640 1 -- 172.21.15.60:0/163637702 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6543+0+0 (secure 0 0 0) 0x7fc10c0bb6f0 con 0x7fc11c06cb70 2024-09-16T15:43:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.816+0000 7fc11b400640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc0fc078970 0x7fc0fc07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.817+0000 7fc11b400640 1 --2- 172.21.15.60:0/163637702 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc0fc078970 0x7fc0fc07ae30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc11c15eaa0 tx=0x7fc108009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:37.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:37.821+0000 7fc113e00640 1 -- 172.21.15.60:0/163637702 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc11c15f4d0 con 0x7fc11c06cb70 2024-09-16T15:43:38.047 INFO:teuthology.orchestra.run.smithi060.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-16T15:43:38.047 INFO:teuthology.orchestra.run.smithi060.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-16T15:43:38.047 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-16T15:43:38.055 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:43:38.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-16T15:43:38.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.219+0000 7f0638200640 1 -- 172.21.15.60:0/3894440157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06300747a0 msgr2=0x7f06300cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:38.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.219+0000 7f0638200640 1 --2- 172.21.15.60:0/3894440157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06300747a0 0x7f06300cc680 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0628009830 tx=0x7f062802a510 comp rx=0 tx=0).stop 2024-09-16T15:43:38.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.219+0000 7f0638200640 1 -- 172.21.15.60:0/3894440157 shutdown_connections 2024-09-16T15:43:38.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.219+0000 7f0638200640 1 --2- 172.21.15.60:0/3894440157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06300747a0 0x7f06300cc680 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:38.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.219+0000 7f0638200640 1 --2- 172.21.15.60:0/3894440157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f063015c3f0 0x7f0630074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:38.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.219+0000 7f0638200640 1 -- 172.21.15.60:0/3894440157 >> 172.21.15.60:0/3894440157 conn(0x7f06300b1ba0 msgr2=0x7f06300b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:38.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.219+0000 7f0638200640 1 -- 172.21.15.60:0/3894440157 shutdown_connections 2024-09-16T15:43:38.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.219+0000 7f0638200640 1 -- 172.21.15.60:0/3894440157 wait complete. 2024-09-16T15:43:38.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.220+0000 7f0638200640 1 Processor -- start 2024-09-16T15:43:38.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.221+0000 7f0638200640 1 -- start start 2024-09-16T15:43:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0638200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06300747a0 0x7f06301f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0638200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06301561e0 0x7f06301f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0638200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015c3f0 0x7f06300ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0638200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f06300ccbc0 con 0x7f06300747a0 2024-09-16T15:43:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0638200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06300ccd40 con 0x7f063015c3f0 2024-09-16T15:43:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0638200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f06300ccec0 con 0x7f06301561e0 2024-09-16T15:43:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0635800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06300747a0 0x7f06301f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0635800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06300747a0 0x7f06301f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38766/0 (socket says 172.21.15.60:38766) 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0635800640 1 -- 172.21.15.60:0/2716955422 learned_addr learned my addr 172.21.15.60:0/2716955422 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0634e00640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06301561e0 0x7f06301f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0636200640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015c3f0 0x7f06300ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0635800640 1 -- 172.21.15.60:0/2716955422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06301561e0 msgr2=0x7f06301f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0635800640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06301561e0 0x7f06301f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0635800640 1 -- 172.21.15.60:0/2716955422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015c3f0 msgr2=0x7f06300ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0635800640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015c3f0 0x7f06300ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0635800640 1 -- 172.21.15.60:0/2716955422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06301f44e0 con 0x7f06300747a0 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.222+0000 7f0636200640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f063015c3f0 0x7f06300ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.223+0000 7f0635800640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06300747a0 0x7f06301f8690 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f062400cba0 tx=0x7f062400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.223+0000 7f062d600640 1 -- 172.21.15.60:0/2716955422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624016020 con 0x7f06300747a0 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.223+0000 7f062d600640 1 -- 172.21.15.60:0/2716955422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f062400ae30 con 0x7f06300747a0 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.223+0000 7f062d600640 1 -- 172.21.15.60:0/2716955422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624012660 con 0x7f06300747a0 2024-09-16T15:43:38.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.223+0000 7f0638200640 1 -- 172.21.15.60:0/2716955422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06300cb100 con 0x7f06300747a0 2024-09-16T15:43:38.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.223+0000 7f0638200640 1 -- 172.21.15.60:0/2716955422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06301f7e90 con 0x7f06300747a0 2024-09-16T15:43:38.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.224+0000 7f0638200640 1 -- 172.21.15.60:0/2716955422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063006ad10 con 0x7f06300747a0 2024-09-16T15:43:38.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.226+0000 7f062d600640 1 -- 172.21.15.60:0/2716955422 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f0624019020 con 0x7f06300747a0 2024-09-16T15:43:38.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.227+0000 7f062d600640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0610078970 0x7f061007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:38.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.227+0000 7f062d600640 1 -- 172.21.15.60:0/2716955422 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6543+0+0 (secure 0 0 0) 0x7f062409b8d0 con 0x7f06300747a0 2024-09-16T15:43:38.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.228+0000 7f0634e00640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0610078970 0x7f061007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:38.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.228+0000 7f0634e00640 1 --2- 172.21.15.60:0/2716955422 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0610078970 0x7f061007ae30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f063009daf0 tx=0x7f061c00b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:38.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.231+0000 7f062d600640 1 -- 172.21.15.60:0/2716955422 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f063006ad10 con 0x7f06300747a0 2024-09-16T15:43:38.453 INFO:teuthology.orchestra.run.smithi060.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-16T15:43:38.453 INFO:teuthology.orchestra.run.smithi060.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] [] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-16T15:43:38.453 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-16T15:43:38.462 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:43:38.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-16T15:43:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.629+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1774029831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 msgr2=0x7fbb881604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.629+0000 7fbb8dc00640 1 --2- 172.21.15.60:0/1774029831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 0x7fbb881604b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbb78005fd0 tx=0x7fbb7802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:43:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.629+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1774029831 shutdown_connections 2024-09-16T15:43:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.629+0000 7fbb8dc00640 1 --2- 172.21.15.60:0/1774029831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 0x7fbb881604b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:38.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.629+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1774029831 >> 172.21.15.60:0/1774029831 conn(0x7fbb880d1e20 msgr2=0x7fbb880d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:38.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.629+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1774029831 shutdown_connections 2024-09-16T15:43:38.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.629+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1774029831 wait complete. 2024-09-16T15:43:38.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.631+0000 7fbb8dc00640 1 Processor -- start 2024-09-16T15:43:38.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.631+0000 7fbb8dc00640 1 -- start start 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb8dc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb880cd6f0 0x7fbb881f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb8dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb880d08e0 0x7fbb881efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb8dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 0x7fbb881f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb8dc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbb880cf9f0 con 0x7fbb8815bf10 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb8dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8815c400 con 0x7fbb880d08e0 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb8dc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbb880cfb70 con 0x7fbb880cd6f0 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb87400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 0x7fbb881f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb87400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 0x7fbb881f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:38792/0 (socket says 172.21.15.60:38792) 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb87400640 1 -- 172.21.15.60:0/1556274534 learned_addr learned my addr 172.21.15.60:0/1556274534 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb86a00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb880cd6f0 0x7fbb881f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:38.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.632+0000 7fbb86000640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb880d08e0 0x7fbb881efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.633+0000 7fbb87400640 1 -- 172.21.15.60:0/1556274534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb880cd6f0 msgr2=0x7fbb881f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.633+0000 7fbb87400640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb880cd6f0 0x7fbb881f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.633+0000 7fbb87400640 1 -- 172.21.15.60:0/1556274534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb880d08e0 msgr2=0x7fbb881efe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.633+0000 7fbb87400640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb880d08e0 0x7fbb881efe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.633+0000 7fbb87400640 1 -- 172.21.15.60:0/1556274534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb881f2b40 con 0x7fbb8815bf10 2024-09-16T15:43:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.633+0000 7fbb87400640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 0x7fbb881f3020 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c780 tx=0x7fbb8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.635+0000 7fbb7ea00640 1 -- 172.21.15.60:0/1556274534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80004b60 con 0x7fbb8815bf10 2024-09-16T15:43:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.635+0000 7fbb7ea00640 1 -- 172.21.15.60:0/1556274534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb80004df0 con 0x7fbb8815bf10 2024-09-16T15:43:38.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.635+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8809cd70 con 0x7fbb8815bf10 2024-09-16T15:43:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.635+0000 7fbb7ea00640 1 -- 172.21.15.60:0/1556274534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000aa90 con 0x7fbb8815bf10 2024-09-16T15:43:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.635+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8809ca90 con 0x7fbb8815bf10 2024-09-16T15:43:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.636+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb881f2930 con 0x7fbb8815bf10 2024-09-16T15:43:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.638+0000 7fbb7ea00640 1 -- 172.21.15.60:0/1556274534 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fbb8000ac30 con 0x7fbb8815bf10 2024-09-16T15:43:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.638+0000 7fbb7ea00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb68078970 0x7fbb6807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.639+0000 7fbb7ea00640 1 -- 172.21.15.60:0/1556274534 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6543+0+0 (secure 0 0 0) 0x7fbb8009b090 con 0x7fbb8815bf10 2024-09-16T15:43:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.639+0000 7fbb86a00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb68078970 0x7fbb6807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.639+0000 7fbb86a00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb68078970 0x7fbb6807ae30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb78033000 tx=0x7fbb78004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:38.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.643+0000 7fbb7ea00640 1 -- 172.21.15.60:0/1556274534 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb80020070 con 0x7fbb8815bf10 2024-09-16T15:43:38.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:38.866+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7fbb881f2d10 con 0x7fbb68078970 2024-09-16T15:43:38.871 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:43:38.871+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:43:38.919 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:43:41.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.887+0000 7fbb7ea00640 1 -- 172.21.15.60:0/1556274534 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7fbb881f2d10 con 0x7fbb68078970 2024-09-16T15:43:41.889 INFO:teuthology.orchestra.run.smithi060.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-16T15:43:41.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb68078970 msgr2=0x7fbb6807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:41.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb68078970 0x7fbb6807ae30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb78033000 tx=0x7fbb78004730 comp rx=0 tx=0).stop 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 msgr2=0x7fbb881f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 0x7fbb881f3020 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000c780 tx=0x7fbb8000cc40 comp rx=0 tx=0).stop 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 shutdown_connections 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb68078970 0x7fbb6807ae30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb8815bf10 0x7fbb881f3020 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb880d08e0 0x7fbb881efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 --2- 172.21.15.60:0/1556274534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb880cd6f0 0x7fbb881f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 >> 172.21.15.60:0/1556274534 conn(0x7fbb880d1e20 msgr2=0x7fbb880b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 shutdown_connections 2024-09-16T15:43:41.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:41.892+0000 7fbb8dc00640 1 -- 172.21.15.60:0/1556274534 wait complete. 2024-09-16T15:43:41.913 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:43:41.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:43:42.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:43:42.109 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:43:42.109 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:43:42.110 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:42.110 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:42.148 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:42.148 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:42.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:43:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.431+0000 7feeb7200640 1 -- 172.21.15.60:0/2177227911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00b2440 msgr2=0x7feeb006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.431+0000 7feeb7200640 1 --2- 172.21.15.60:0/2177227911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00b2440 0x7feeb006f160 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7feea8005fd0 tx=0x7feea802aa10 comp rx=0 tx=0).stop 2024-09-16T15:43:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.431+0000 7feeb7200640 1 -- 172.21.15.60:0/2177227911 shutdown_connections 2024-09-16T15:43:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.431+0000 7feeb7200640 1 --2- 172.21.15.60:0/2177227911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00b2440 0x7feeb006f160 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.431+0000 7feeb7200640 1 -- 172.21.15.60:0/2177227911 >> 172.21.15.60:0/2177227911 conn(0x7feeb00b1950 msgr2=0x7feeb00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.431+0000 7feeb7200640 1 -- 172.21.15.60:0/2177227911 shutdown_connections 2024-09-16T15:43:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.432+0000 7feeb7200640 1 -- 172.21.15.60:0/2177227911 wait complete. 2024-09-16T15:43:42.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.433+0000 7feeb7200640 1 Processor -- start 2024-09-16T15:43:42.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.433+0000 7feeb7200640 1 -- start start 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.434+0000 7feeb7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00b2440 0x7feeb00d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.434+0000 7feeb7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00ccf00 0x7feeb00ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.434+0000 7feeb7200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feeb015f5a0 0x7feeb015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.434+0000 7feeb7200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7feeb0074840 con 0x7feeb00ccf00 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.434+0000 7feeb7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feeb00746c0 con 0x7feeb00b2440 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.434+0000 7feeb7200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7feeb006f6a0 con 0x7feeb015f5a0 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.434+0000 7feeaf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00ccf00 0x7feeb00ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeaf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00ccf00 0x7feeb00ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59136/0 (socket says 172.21.15.60:59136) 2024-09-16T15:43:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeaf400640 1 -- 172.21.15.60:0/3590514250 learned_addr learned my addr 172.21.15.60:0/3590514250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:42.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeb5200640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feeb015f5a0 0x7feeb015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:42.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeafe00640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00b2440 0x7feeb00d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:42.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeaf400640 1 -- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feeb015f5a0 msgr2=0x7feeb015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:42.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeaf400640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feeb015f5a0 0x7feeb015ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:42.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeaf400640 1 -- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00b2440 msgr2=0x7feeb00d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:42.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeaf400640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00b2440 0x7feeb00d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:42.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeaf400640 1 -- 172.21.15.60:0/3590514250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb00cf460 con 0x7feeb00ccf00 2024-09-16T15:43:42.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeb5200640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feeb015f5a0 0x7feeb015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.435+0000 7feeaf400640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00ccf00 0x7feeb00ce4f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c430 tx=0x7fee9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.436+0000 7feea7e00640 1 -- 172.21.15.60:0/3590514250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c011790 con 0x7feeb00ccf00 2024-09-16T15:43:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.436+0000 7feea7e00640 1 -- 172.21.15.60:0/3590514250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee9c017070 con 0x7feeb00ccf00 2024-09-16T15:43:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.436+0000 7feea7e00640 1 -- 172.21.15.60:0/3590514250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c011d00 con 0x7feeb00ccf00 2024-09-16T15:43:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.436+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb00cf740 con 0x7feeb00ccf00 2024-09-16T15:43:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.436+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb00cfd00 con 0x7feeb00ccf00 2024-09-16T15:43:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.437+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb006f160 con 0x7feeb00ccf00 2024-09-16T15:43:42.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.438+0000 7feea7e00640 1 -- 172.21.15.60:0/3590514250 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fee9c01c460 con 0x7feeb00ccf00 2024-09-16T15:43:42.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.440+0000 7feea7e00640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fee90078970 0x7fee9007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:42.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.440+0000 7feea7e00640 1 -- 172.21.15.60:0/3590514250 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7fee9c09ab80 con 0x7feeb00ccf00 2024-09-16T15:43:42.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.440+0000 7feeafe00640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fee90078970 0x7fee9007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:42.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.440+0000 7feeafe00640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fee90078970 0x7fee9007ae30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feea0007fd0 tx=0x7feea0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.444+0000 7feea7e00640 1 -- 172.21.15.60:0/3590514250 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fee9c062ed0 con 0x7feeb00ccf00 2024-09-16T15:43:42.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.651+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feeb00acd30 con 0x7feeb00ccf00 2024-09-16T15:43:42.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.651+0000 7feea7e00640 1 -- 172.21.15.60:0/3590514250 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v95) ==== 80+0+122 (secure 0 0 0) 0x7fee9c0627d0 con 0x7feeb00ccf00 2024-09-16T15:43:42.652 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/2496703614 2024-09-17T15:43:38.871416+0000 2024-09-16T15:43:42.652 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/2496703614 2024-09-17T15:43:38.871416+0000 2024-09-16T15:43:42.652 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-09-16T15:43:42.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fee90078970 msgr2=0x7fee9007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fee90078970 0x7fee9007ae30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feea0007fd0 tx=0x7feea0009040 comp rx=0 tx=0).stop 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00ccf00 msgr2=0x7feeb00ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00ccf00 0x7feeb00ce4f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c430 tx=0x7fee9c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 shutdown_connections 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fee90078970 0x7fee9007ae30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feeb015f5a0 0x7feeb015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feeb00ccf00 0x7feeb00ce4f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 --2- 172.21.15.60:0/3590514250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feeb00b2440 0x7feeb00d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 >> 172.21.15.60:0/3590514250 conn(0x7feeb00b1950 msgr2=0x7feeb00ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 shutdown_connections 2024-09-16T15:43:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:42.653+0000 7feeb7200640 1 -- 172.21.15.60:0/3590514250 wait complete. 2024-09-16T15:43:42.672 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:43:42.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:42.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:43:42.734 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:43:42.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:42.735 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:43:42.770 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:43:42.770 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:42.770 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:42.810 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:42.810 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:42.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:43:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.051+0000 7ff627400640 1 -- 172.21.15.60:0/637495884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 msgr2=0x7ff620074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.051+0000 7ff627400640 1 --2- 172.21.15.60:0/637495884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 0x7ff620074420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff6200cfbb0 tx=0x7ff61802aa30 comp rx=0 tx=0).stop 2024-09-16T15:43:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.051+0000 7ff627400640 1 -- 172.21.15.60:0/637495884 shutdown_connections 2024-09-16T15:43:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.051+0000 7ff627400640 1 --2- 172.21.15.60:0/637495884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 0x7ff620074420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.051+0000 7ff627400640 1 --2- 172.21.15.60:0/637495884 >> v2:172.21.15.100:3300/0 conn(0x7ff6200ccf00 0x7ff6200b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.051+0000 7ff627400640 1 -- 172.21.15.60:0/637495884 >> 172.21.15.60:0/637495884 conn(0x7ff6200b1950 msgr2=0x7ff6200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.051+0000 7ff627400640 1 -- 172.21.15.60:0/637495884 shutdown_connections 2024-09-16T15:43:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.051+0000 7ff627400640 1 -- 172.21.15.60:0/637495884 wait complete. 2024-09-16T15:43:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.053+0000 7ff627400640 1 Processor -- start 2024-09-16T15:43:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.053+0000 7ff627400640 1 -- start start 2024-09-16T15:43:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff627400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 0x7ff6201546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff627400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6200ccf00 0x7ff6201f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff627400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff62015f5a0 0x7ff6200cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff627400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff62006f2e0 con 0x7ff620073fb0 2024-09-16T15:43:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff627400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff62006f160 con 0x7ff62015f5a0 2024-09-16T15:43:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff627400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff62006f460 con 0x7ff6200ccf00 2024-09-16T15:43:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff624a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 0x7ff6201546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff624a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 0x7ff6201546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59144/0 (socket says 172.21.15.60:59144) 2024-09-16T15:43:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.054+0000 7ff624a00640 1 -- 172.21.15.60:0/583630891 learned_addr learned my addr 172.21.15.60:0/583630891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff617e00640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6200ccf00 0x7ff6201f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff625400640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff62015f5a0 0x7ff6200cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff624a00640 1 -- 172.21.15.60:0/583630891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6200ccf00 msgr2=0x7ff6201f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff624a00640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6200ccf00 0x7ff6201f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff624a00640 1 -- 172.21.15.60:0/583630891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff62015f5a0 msgr2=0x7ff6200cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff624a00640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff62015f5a0 0x7ff6200cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff624a00640 1 -- 172.21.15.60:0/583630891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62009aa00 con 0x7ff620073fb0 2024-09-16T15:43:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff624a00640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 0x7ff6201546b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff61000c3c0 tx=0x7ff61000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff61d600640 1 -- 172.21.15.60:0/583630891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610017070 con 0x7ff620073fb0 2024-09-16T15:43:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff61d600640 1 -- 172.21.15.60:0/583630891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6100035f0 con 0x7ff620073fb0 2024-09-16T15:43:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff61d600640 1 -- 172.21.15.60:0/583630891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610003a80 con 0x7ff620073fb0 2024-09-16T15:43:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.055+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6201f4e50 con 0x7ff620073fb0 2024-09-16T15:43:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.056+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6200cdc80 con 0x7ff620073fb0 2024-09-16T15:43:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.057+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6201f5370 con 0x7ff620073fb0 2024-09-16T15:43:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.058+0000 7ff61d600640 1 -- 172.21.15.60:0/583630891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7ff61001a070 con 0x7ff620073fb0 2024-09-16T15:43:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.059+0000 7ff61d600640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff5fc0789a0 0x7ff5fc07ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.059+0000 7ff61d600640 1 -- 172.21.15.60:0/583630891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7ff610099e90 con 0x7ff620073fb0 2024-09-16T15:43:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.059+0000 7ff617e00640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff5fc0789a0 0x7ff5fc07ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.060+0000 7ff617e00640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff5fc0789a0 0x7ff5fc07ae60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff62009cf80 tx=0x7ff608005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.063+0000 7ff61d600640 1 -- 172.21.15.60:0/583630891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff6201f5370 con 0x7ff620073fb0 2024-09-16T15:43:43.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.271+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6200aced0 con 0x7ff620073fb0 2024-09-16T15:43:43.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.272+0000 7ff61d600640 1 -- 172.21.15.60:0/583630891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v95) ==== 80+0+122 (secure 0 0 0) 0x7ff61009b080 con 0x7ff620073fb0 2024-09-16T15:43:43.273 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6835/2496703614 2024-09-17T15:43:38.871416+0000 2024-09-16T15:43:43.273 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6833/2496703614 2024-09-17T15:43:38.871416+0000 2024-09-16T15:43:43.273 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-09-16T15:43:43.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff5fc0789a0 msgr2=0x7ff5fc07ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff5fc0789a0 0x7ff5fc07ae60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff62009cf80 tx=0x7ff608005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:43.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 msgr2=0x7ff6201546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 0x7ff6201546b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff61000c3c0 tx=0x7ff61000c880 comp rx=0 tx=0).stop 2024-09-16T15:43:43.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 shutdown_connections 2024-09-16T15:43:43.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff5fc0789a0 0x7ff5fc07ae60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff62015f5a0 0x7ff6200cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6200ccf00 0x7ff6201f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 --2- 172.21.15.60:0/583630891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff620073fb0 0x7ff6201546b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 >> 172.21.15.60:0/583630891 conn(0x7ff6200b1950 msgr2=0x7ff62015dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:43.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 shutdown_connections 2024-09-16T15:43:43.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.274+0000 7ff627400640 1 -- 172.21.15.60:0/583630891 wait complete. 2024-09-16T15:43:43.291 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:43:43.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:43.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:43:43.360 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:43:43.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:43.360 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:43:43.397 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:43:43.397 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:43.397 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:43.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:43:43.433 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:43:43.433 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:43.433 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:43.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:43:43.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.632+0000 7ff570800640 1 -- 172.21.15.60:0/3452834384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff568073fb0 msgr2=0x7ff568074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.632+0000 7ff570800640 1 --2- 172.21.15.60:0/3452834384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff568073fb0 0x7ff568074440 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff564005fd0 tx=0x7ff56402aa00 comp rx=0 tx=0).stop 2024-09-16T15:43:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.632+0000 7ff570800640 1 -- 172.21.15.60:0/3452834384 shutdown_connections 2024-09-16T15:43:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.632+0000 7ff570800640 1 --2- 172.21.15.60:0/3452834384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff568073fb0 0x7ff568074440 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.632+0000 7ff570800640 1 --2- 172.21.15.60:0/3452834384 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5680ccf00 0x7ff5680b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.632+0000 7ff570800640 1 -- 172.21.15.60:0/3452834384 >> 172.21.15.60:0/3452834384 conn(0x7ff5680b1950 msgr2=0x7ff5680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.632+0000 7ff570800640 1 -- 172.21.15.60:0/3452834384 shutdown_connections 2024-09-16T15:43:43.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.633+0000 7ff570800640 1 -- 172.21.15.60:0/3452834384 wait complete. 2024-09-16T15:43:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.634+0000 7ff570800640 1 Processor -- start 2024-09-16T15:43:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.634+0000 7ff570800640 1 -- start start 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.635+0000 7ff570800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff568073fb0 0x7ff568146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.635+0000 7ff570800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5680ccf00 0x7ff568143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.635+0000 7ff570800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff56815f5a0 0x7ff5681436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.635+0000 7ff570800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff56806f2e0 con 0x7ff5680ccf00 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.635+0000 7ff570800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff56806f160 con 0x7ff568073fb0 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.635+0000 7ff570800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff56806f460 con 0x7ff56815f5a0 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5680ccf00 0x7ff568143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5680ccf00 0x7ff568143190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59166/0 (socket says 172.21.15.60:59166) 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 -- 172.21.15.60:0/4121390126 learned_addr learned my addr 172.21.15.60:0/4121390126 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56de00640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff568073fb0 0x7ff568146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 -- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff56815f5a0 msgr2=0x7ff5681436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56e800640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff56815f5a0 0x7ff5681436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff56815f5a0 0x7ff5681436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 -- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff568073fb0 msgr2=0x7ff568146880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff568073fb0 0x7ff568146880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 -- 172.21.15.60:0/4121390126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56814fd90 con 0x7ff5680ccf00 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56de00640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff568073fb0 0x7ff568146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56e800640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff56815f5a0 0x7ff5681436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff56d400640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5680ccf00 0x7ff568143190 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff55800e9f0 tx=0x7ff55800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.636+0000 7ff562000640 1 -- 172.21.15.60:0/4121390126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55800cd80 con 0x7ff5680ccf00 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.637+0000 7ff562000640 1 -- 172.21.15.60:0/4121390126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff558005020 con 0x7ff5680ccf00 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.637+0000 7ff562000640 1 -- 172.21.15.60:0/4121390126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558005450 con 0x7ff5680ccf00 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.637+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56814d650 con 0x7ff5680ccf00 2024-09-16T15:43:43.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.637+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff568150000 con 0x7ff5680ccf00 2024-09-16T15:43:43.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.638+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56806f9f0 con 0x7ff5680ccf00 2024-09-16T15:43:43.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.639+0000 7ff562000640 1 -- 172.21.15.60:0/4121390126 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7ff5580047a0 con 0x7ff5680ccf00 2024-09-16T15:43:43.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.640+0000 7ff562000640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff54c078970 0x7ff54c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:43.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.640+0000 7ff562000640 1 -- 172.21.15.60:0/4121390126 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7ff55809b720 con 0x7ff5680ccf00 2024-09-16T15:43:43.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.640+0000 7ff56de00640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff54c078970 0x7ff54c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:43.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.641+0000 7ff56de00640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff54c078970 0x7ff54c07ae30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0016a0 tx=0x7ff55c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.644+0000 7ff562000640 1 -- 172.21.15.60:0/4121390126 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff558063bc0 con 0x7ff5680ccf00 2024-09-16T15:43:43.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.878+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5681504a0 con 0x7ff5680ccf00 2024-09-16T15:43:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.878+0000 7ff562000640 1 -- 172.21.15.60:0/4121390126 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5231 (secure 0 0 0) 0x7ff5580634c0 con 0x7ff5680ccf00 2024-09-16T15:43:43.879 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:43.879 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"btime":"2024-09-16T15:43:39:873967+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1388326737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1388326737},{"type":"v1","addr":"172.21.15.100:6835","nonce":1388326737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6382,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/2485723644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2485723644},{"type":"v1","addr":"172.21.15.100:6801","nonce":2485723644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2204286380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2204286380},{"type":"v1","addr":"172.21.15.60:6835","nonce":2204286380}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:22.963712+0000","modified":"2024-09-16T15:43:25.979087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3103295284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3103295284},{"type":"v1","addr":"172.21.15.60:6834","nonce":3103295284}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":6}]} 2024-09-16T15:43:43.880 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff54c078970 msgr2=0x7ff54c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff54c078970 0x7ff54c07ae30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0016a0 tx=0x7ff55c009040 comp rx=0 tx=0).stop 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5680ccf00 msgr2=0x7ff568143190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5680ccf00 0x7ff568143190 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff55800e9f0 tx=0x7ff55800edf0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 shutdown_connections 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff54c078970 0x7ff54c07ae30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff56815f5a0 0x7ff5681436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5680ccf00 0x7ff568143190 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.880+0000 7ff570800640 1 --2- 172.21.15.60:0/4121390126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff568073fb0 0x7ff568146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.881+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 >> 172.21.15.60:0/4121390126 conn(0x7ff5680b1950 msgr2=0x7ff5680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.881+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 shutdown_connections 2024-09-16T15:43:43.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:43.881+0000 7ff570800640 1 -- 172.21.15.60:0/4121390126 wait complete. 2024-09-16T15:43:43.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.058+0000 7f6901000640 1 -- 172.21.15.60:0/4255348433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 msgr2=0x7f68fc073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.058+0000 7f6901000640 1 --2- 172.21.15.60:0/4255348433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 0x7f68fc073fb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f68fc075130 tx=0x7f68f402a980 comp rx=0 tx=0).stop 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.058+0000 7f6901000640 1 -- 172.21.15.60:0/4255348433 shutdown_connections 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.058+0000 7f6901000640 1 --2- 172.21.15.60:0/4255348433 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 0x7f68fc073fb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.058+0000 7f6901000640 1 --2- 172.21.15.60:0/4255348433 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68fc0ccf00 0x7f68fc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.059+0000 7f6901000640 1 --2- 172.21.15.60:0/4255348433 >> v2:172.21.15.100:3300/0 conn(0x7f68fc15f5a0 0x7f68fc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.059+0000 7f6901000640 1 -- 172.21.15.60:0/4255348433 >> 172.21.15.60:0/4255348433 conn(0x7f68fc0b1950 msgr2=0x7f68fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.059+0000 7f6901000640 1 -- 172.21.15.60:0/4255348433 shutdown_connections 2024-09-16T15:43:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.059+0000 7f6901000640 1 -- 172.21.15.60:0/4255348433 wait complete. 2024-09-16T15:43:44.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.060+0000 7f6901000640 1 Processor -- start 2024-09-16T15:43:44.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.060+0000 7f6901000640 1 -- start start 2024-09-16T15:43:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.061+0000 7f6901000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 0x7f68fc12a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.061+0000 7f6901000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68fc0ccf00 0x7f68fc1f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.061+0000 7f6901000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68fc15f5a0 0x7f68fc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.061+0000 7f6901000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc06f160 con 0x7f68fc0ad270 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.061+0000 7f6901000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc074780 con 0x7f68fc15f5a0 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.061+0000 7f6901000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc06f2e0 con 0x7f68fc0ccf00 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.061+0000 7f68f9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 0x7f68fc12a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 0x7f68fc12a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59178/0 (socket says 172.21.15.60:59178) 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f9e00640 1 -- 172.21.15.60:0/1030486162 learned_addr learned my addr 172.21.15.60:0/1030486162 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f9e00640 1 -- 172.21.15.60:0/1030486162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68fc0ccf00 msgr2=0x7f68fc1f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f9e00640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68fc0ccf00 0x7f68fc1f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f9e00640 1 -- 172.21.15.60:0/1030486162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68fc15f5a0 msgr2=0x7f68fc0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f9e00640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68fc15f5a0 0x7f68fc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f9e00640 1 -- 172.21.15.60:0/1030486162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc1f4d70 con 0x7f68fc0ad270 2024-09-16T15:43:44.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f9e00640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 0x7f68fc12a230 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00e790 tx=0x7f68ec00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f2a00640 1 -- 172.21.15.60:0/1030486162 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00eea0 con 0x7f68fc0ad270 2024-09-16T15:43:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f2a00640 1 -- 172.21.15.60:0/1030486162 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68ec00cdb0 con 0x7f68fc0ad270 2024-09-16T15:43:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.062+0000 7f68f2a00640 1 -- 172.21.15.60:0/1030486162 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec010690 con 0x7f68fc0ad270 2024-09-16T15:43:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.063+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc1f4fe0 con 0x7f68fc0ad270 2024-09-16T15:43:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.063+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc1f55a0 con 0x7f68fc0ad270 2024-09-16T15:43:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.064+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc0cb0e0 con 0x7f68fc0ad270 2024-09-16T15:43:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.065+0000 7f68f2a00640 1 -- 172.21.15.60:0/1030486162 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f68ec010830 con 0x7f68fc0ad270 2024-09-16T15:43:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.066+0000 7f68f2a00640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f68d80789a0 0x7f68d807ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.066+0000 7f68f2a00640 1 -- 172.21.15.60:0/1030486162 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7f68ec09b590 con 0x7f68fc0ad270 2024-09-16T15:43:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.066+0000 7f68f1400640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f68d80789a0 0x7f68d807ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.067+0000 7f68f1400640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f68d80789a0 0x7f68d807ae60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0ca8b0 tx=0x7f68e4009590 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.070+0000 7f68f2a00640 1 -- 172.21.15.60:0/1030486162 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f68ec063970 con 0x7f68fc0ad270 2024-09-16T15:43:44.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.303+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68fc06f810 con 0x7f68fc0ad270 2024-09-16T15:43:44.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.304+0000 7f68f2a00640 1 -- 172.21.15.60:0/1030486162 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5231 (secure 0 0 0) 0x7f68ec063270 con 0x7f68fc0ad270 2024-09-16T15:43:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:44.305 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"btime":"2024-09-16T15:43:39:873967+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1388326737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1388326737},{"type":"v1","addr":"172.21.15.100:6835","nonce":1388326737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6382,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/2485723644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2485723644},{"type":"v1","addr":"172.21.15.100:6801","nonce":2485723644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2204286380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2204286380},{"type":"v1","addr":"172.21.15.60:6835","nonce":2204286380}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:22.963712+0000","modified":"2024-09-16T15:43:25.979087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3103295284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3103295284},{"type":"v1","addr":"172.21.15.60:6834","nonce":3103295284}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":6}]} 2024-09-16T15:43:44.305 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-09-16T15:43:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f68d80789a0 msgr2=0x7f68d807ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f68d80789a0 0x7f68d807ae60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0ca8b0 tx=0x7f68e4009590 comp rx=0 tx=0).stop 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 msgr2=0x7f68fc12a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 0x7f68fc12a230 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00e790 tx=0x7f68ec00ec50 comp rx=0 tx=0).stop 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 shutdown_connections 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f68d80789a0 0x7f68d807ae60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68fc15f5a0 0x7f68fc0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68fc0ccf00 0x7f68fc1f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 --2- 172.21.15.60:0/1030486162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68fc0ad270 0x7f68fc12a230 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.306+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 >> 172.21.15.60:0/1030486162 conn(0x7f68fc0b1950 msgr2=0x7f68fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.307+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 shutdown_connections 2024-09-16T15:43:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.307+0000 7f6901000640 1 -- 172.21.15.60:0/1030486162 wait complete. 2024-09-16T15:43:44.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:44.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.485+0000 7ff8e8400640 1 -- 172.21.15.60:0/73892624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 msgr2=0x7ff8e00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.485+0000 7ff8e8400640 1 --2- 172.21.15.60:0/73892624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 0x7ff8e00732a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005fd0 tx=0x7ff8d802a820 comp rx=0 tx=0).stop 2024-09-16T15:43:44.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.485+0000 7ff8e8400640 1 -- 172.21.15.60:0/73892624 shutdown_connections 2024-09-16T15:43:44.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.485+0000 7ff8e8400640 1 --2- 172.21.15.60:0/73892624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 0x7ff8e00732a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.485+0000 7ff8e8400640 1 --2- 172.21.15.60:0/73892624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e0161550 0x7ff8e006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.485+0000 7ff8e8400640 1 -- 172.21.15.60:0/73892624 >> 172.21.15.60:0/73892624 conn(0x7ff8e00b1950 msgr2=0x7ff8e00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:44.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.485+0000 7ff8e8400640 1 -- 172.21.15.60:0/73892624 shutdown_connections 2024-09-16T15:43:44.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.485+0000 7ff8e8400640 1 -- 172.21.15.60:0/73892624 wait complete. 2024-09-16T15:43:44.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.487+0000 7ff8e8400640 1 Processor -- start 2024-09-16T15:43:44.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.487+0000 7ff8e8400640 1 -- start start 2024-09-16T15:43:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e8400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 0x7ff8e01f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e8400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e00af3d0 0x7ff8e00b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e8400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e0161550 0x7ff8e01f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e8400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e00737e0 con 0x7ff8e00af3d0 2024-09-16T15:43:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e8400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e006fdb0 con 0x7ff8e0072df0 2024-09-16T15:43:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e8400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff8e0073960 con 0x7ff8e0161550 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e00af3d0 0x7ff8e00b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e00af3d0 0x7ff8e00b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59194/0 (socket says 172.21.15.60:59194) 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e5000640 1 -- 172.21.15.60:0/2549771413 learned_addr learned my addr 172.21.15.60:0/2549771413 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e5a00640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 0x7ff8e01f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.488+0000 7ff8e6400640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e0161550 0x7ff8e01f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e5000640 1 -- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e0161550 msgr2=0x7ff8e01f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e5000640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e0161550 0x7ff8e01f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e5000640 1 -- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 msgr2=0x7ff8e01f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e5000640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 0x7ff8e01f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e5000640 1 -- 172.21.15.60:0/2549771413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e01f4a80 con 0x7ff8e00af3d0 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e5a00640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 0x7ff8e01f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e5000640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e00af3d0 0x7ff8e00b0d90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00c420 tx=0x7ff8cc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8dd600640 1 -- 172.21.15.60:0/2549771413 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc011790 con 0x7ff8e00af3d0 2024-09-16T15:43:44.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8dd600640 1 -- 172.21.15.60:0/2549771413 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8cc017070 con 0x7ff8e00af3d0 2024-09-16T15:43:44.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8dd600640 1 -- 172.21.15.60:0/2549771413 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc011c80 con 0x7ff8e00af3d0 2024-09-16T15:43:44.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e01f85b0 con 0x7ff8e00af3d0 2024-09-16T15:43:44.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.489+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01f8b30 con 0x7ff8e00af3d0 2024-09-16T15:43:44.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.490+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e0073ec0 con 0x7ff8e00af3d0 2024-09-16T15:43:44.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.491+0000 7ff8dd600640 1 -- 172.21.15.60:0/2549771413 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7ff8cc01c460 con 0x7ff8e00af3d0 2024-09-16T15:43:44.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.492+0000 7ff8dd600640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff8c0078970 0x7ff8c007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.493+0000 7ff8dd600640 1 -- 172.21.15.60:0/2549771413 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7ff8cc09a890 con 0x7ff8e00af3d0 2024-09-16T15:43:44.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.493+0000 7ff8e5a00640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff8c0078970 0x7ff8c007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.493+0000 7ff8e5a00640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff8c0078970 0x7ff8c007ae30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff8e006e8f0 tx=0x7ff8d4005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:44.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.497+0000 7ff8dd600640 1 -- 172.21.15.60:0/2549771413 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff8e01f4a80 con 0x7ff8e00af3d0 2024-09-16T15:43:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.697+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8e01f4ec0 con 0x7ff8e00af3d0 2024-09-16T15:43:44.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.698+0000 7ff8dd600640 1 -- 172.21.15.60:0/2549771413 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16849 (secure 0 0 0) 0x7ff8cc062620 con 0x7ff8e00af3d0 2024-09-16T15:43:44.699 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:44.699 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:41.881993+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:20.760401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:21.917247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2496703614":"2024-09-17T15:43:38.871416+0000","172.21.15.60:6833/2496703614":"2024-09-17T15:43:38.871416+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff8c0078970 msgr2=0x7ff8c007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff8c0078970 0x7ff8c007ae30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff8e006e8f0 tx=0x7ff8d4005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e00af3d0 msgr2=0x7ff8e00b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e00af3d0 0x7ff8e00b0d90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00c420 tx=0x7ff8cc00c8e0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 shutdown_connections 2024-09-16T15:43:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff8c0078970 0x7ff8c007ae30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8e0161550 0x7ff8e01f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff8e00af3d0 0x7ff8e00b0d90 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 --2- 172.21.15.60:0/2549771413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8e0072df0 0x7ff8e01f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 >> 172.21.15.60:0/2549771413 conn(0x7ff8e00b1950 msgr2=0x7ff8e00ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 shutdown_connections 2024-09-16T15:43:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.701+0000 7ff8e8400640 1 -- 172.21.15.60:0/2549771413 wait complete. 2024-09-16T15:43:44.720 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:43:44.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:44.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 -- 172.21.15.60:0/42492624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c0b3e50 msgr2=0x7f446c0d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 --2- 172.21.15.60:0/42492624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c0b3e50 0x7f446c0d1d60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f446c0b6930 tx=0x7f446402a9a0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 -- 172.21.15.60:0/42492624 shutdown_connections 2024-09-16T15:43:44.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 --2- 172.21.15.60:0/42492624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c0b3e50 0x7f446c0d1d60 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 --2- 172.21.15.60:0/42492624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f446c0b34b0 0x7f446c0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 --2- 172.21.15.60:0/42492624 >> v2:172.21.15.100:3300/0 conn(0x7f446c161440 0x7f446c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 -- 172.21.15.60:0/42492624 >> 172.21.15.60:0/42492624 conn(0x7f446c0d3df0 msgr2=0x7f446c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:44.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 -- 172.21.15.60:0/42492624 shutdown_connections 2024-09-16T15:43:44.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.879+0000 7f4471a00640 1 -- 172.21.15.60:0/42492624 wait complete. 2024-09-16T15:43:44.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.881+0000 7f4471a00640 1 Processor -- start 2024-09-16T15:43:44.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.881+0000 7f4471a00640 1 -- start start 2024-09-16T15:43:44.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f446c0b34b0 0x7f446c0b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f446c0b3e50 0x7f446c1f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c161440 0x7f446c1f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0d2420 con 0x7f446c161440 2024-09-16T15:43:44.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f446c0d22a0 con 0x7f446c0b34b0 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f446c0d25a0 con 0x7f446c0b3e50 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c161440 0x7f446c1f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c161440 0x7f446c1f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59214/0 (socket says 172.21.15.60:59214) 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471000640 1 -- 172.21.15.60:0/2823512512 learned_addr learned my addr 172.21.15.60:0/2823512512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f446be00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f446c0b34b0 0x7f446c0b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f446b400640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f446c0b3e50 0x7f446c1f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471000640 1 -- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f446c0b3e50 msgr2=0x7f446c1f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471000640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f446c0b3e50 0x7f446c1f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471000640 1 -- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f446c0b34b0 msgr2=0x7f446c0b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471000640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f446c0b34b0 0x7f446c0b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f4471000640 1 -- 172.21.15.60:0/2823512512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446c0b8a20 con 0x7f446c161440 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f446be00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f446c0b34b0 0x7f446c0b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:44.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.882+0000 7f446b400640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f446c0b3e50 0x7f446c1f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:44.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.883+0000 7f4471000640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c161440 0x7f446c1f7c60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4464009d60 tx=0x7f4464003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:44.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.883+0000 7f4463e00640 1 -- 172.21.15.60:0/2823512512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464047070 con 0x7f446c161440 2024-09-16T15:43:44.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.883+0000 7f4463e00640 1 -- 172.21.15.60:0/2823512512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f446402f600 con 0x7f446c161440 2024-09-16T15:43:44.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.883+0000 7f4463e00640 1 -- 172.21.15.60:0/2823512512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464033050 con 0x7f446c161440 2024-09-16T15:43:44.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.883+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446c0d0710 con 0x7f446c161440 2024-09-16T15:43:44.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.883+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446c0d0f10 con 0x7f446c161440 2024-09-16T15:43:44.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.884+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f446c0d2b30 con 0x7f446c161440 2024-09-16T15:43:44.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.885+0000 7f4463e00640 1 -- 172.21.15.60:0/2823512512 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f446404a070 con 0x7f446c161440 2024-09-16T15:43:44.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.886+0000 7f4463e00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f444c078970 0x7f444c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:44.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.886+0000 7f4463e00640 1 -- 172.21.15.60:0/2823512512 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7f44640bbc10 con 0x7f446c161440 2024-09-16T15:43:44.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.886+0000 7f446be00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f444c078970 0x7f444c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:44.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.887+0000 7f446be00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f444c078970 0x7f444c07ae30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f445c009b80 tx=0x7f445c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:44.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:44.890+0000 7f4463e00640 1 -- 172.21.15.60:0/2823512512 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f446c0b8a20 con 0x7f446c161440 2024-09-16T15:43:45.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.123+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f446c0760b0 con 0x7f446c161440 2024-09-16T15:43:45.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.124+0000 7f4463e00640 1 -- 172.21.15.60:0/2823512512 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5231 (secure 0 0 0) 0x7f4464083f60 con 0x7f446c161440 2024-09-16T15:43:45.125 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:45.125 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"btime":"2024-09-16T15:43:39:873967+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1388326737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1388326737},{"type":"v1","addr":"172.21.15.100:6835","nonce":1388326737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6382,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/2485723644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2485723644},{"type":"v1","addr":"172.21.15.100:6801","nonce":2485723644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2204286380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2204286380},{"type":"v1","addr":"172.21.15.60:6835","nonce":2204286380}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:22.963712+0000","modified":"2024-09-16T15:43:25.979087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3103295284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3103295284},{"type":"v1","addr":"172.21.15.60:6834","nonce":3103295284}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":6}]} 2024-09-16T15:43:45.125 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.126+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f444c078970 msgr2=0x7f444c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.126+0000 7f4471a00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f444c078970 0x7f444c07ae30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f445c009b80 tx=0x7f445c005f90 comp rx=0 tx=0).stop 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c161440 msgr2=0x7f446c1f7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c161440 0x7f446c1f7c60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4464009d60 tx=0x7f4464003ef0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 shutdown_connections 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f444c078970 0x7f444c07ae30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f446c161440 0x7f446c1f7c60 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f446c0b3e50 0x7f446c1f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 --2- 172.21.15.60:0/2823512512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f446c0b34b0 0x7f446c0b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 >> 172.21.15.60:0/2823512512 conn(0x7f446c0d3df0 msgr2=0x7f446c0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 shutdown_connections 2024-09-16T15:43:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.127+0000 7f4471a00640 1 -- 172.21.15.60:0/2823512512 wait complete. 2024-09-16T15:43:45.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.304+0000 7f2583000640 1 -- 172.21.15.60:0/2105630879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c15f5a0 msgr2=0x7f257c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.304+0000 7f2583000640 1 --2- 172.21.15.60:0/2105630879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c15f5a0 0x7f257c0b2440 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f256c005fd0 tx=0x7f256c02a540 comp rx=0 tx=0).stop 2024-09-16T15:43:45.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.304+0000 7f2583000640 1 -- 172.21.15.60:0/2105630879 shutdown_connections 2024-09-16T15:43:45.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.304+0000 7f2583000640 1 --2- 172.21.15.60:0/2105630879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f257c073fb0 0x7f257c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.304+0000 7f2583000640 1 --2- 172.21.15.60:0/2105630879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c15f5a0 0x7f257c0b2440 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.304+0000 7f2583000640 1 -- 172.21.15.60:0/2105630879 >> 172.21.15.60:0/2105630879 conn(0x7f257c0b1950 msgr2=0x7f257c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:45.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.304+0000 7f2583000640 1 -- 172.21.15.60:0/2105630879 shutdown_connections 2024-09-16T15:43:45.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.304+0000 7f2583000640 1 -- 172.21.15.60:0/2105630879 wait complete. 2024-09-16T15:43:45.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.305+0000 7f2583000640 1 Processor -- start 2024-09-16T15:43:45.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.306+0000 7f2583000640 1 -- start start 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.306+0000 7f2583000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f257c073fb0 0x7f257c153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.306+0000 7f2583000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c0ccf00 0x7f257c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.306+0000 7f2583000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f257c15f5a0 0x7f257c1f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.306+0000 7f2583000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f257c06f2e0 con 0x7f257c0ccf00 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.306+0000 7f2583000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f257c06f160 con 0x7f257c15f5a0 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.306+0000 7f2583000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f257c06f460 con 0x7f257c073fb0 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c0ccf00 0x7f257c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c0ccf00 0x7f257c0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59228/0 (socket says 172.21.15.60:59228) 2024-09-16T15:43:45.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 -- 172.21.15.60:0/417798969 learned_addr learned my addr 172.21.15.60:0/417798969 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257be00640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f257c073fb0 0x7f257c153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f2581000640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f257c15f5a0 0x7f257c1f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 -- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f257c073fb0 msgr2=0x7f257c153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f257c073fb0 0x7f257c153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 -- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f257c15f5a0 msgr2=0x7f257c1f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f257c15f5a0 0x7f257c1f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 -- 172.21.15.60:0/417798969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c0b3630 con 0x7f257c0ccf00 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f2581000640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f257c15f5a0 0x7f257c1f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.307+0000 7f257b400640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c0ccf00 0x7f257c0b2d50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f256800c450 tx=0x7f256800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.308+0000 7f2573e00640 1 -- 172.21.15.60:0/417798969 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680052d0 con 0x7f257c0ccf00 2024-09-16T15:43:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.308+0000 7f2573e00640 1 -- 172.21.15.60:0/417798969 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2568005470 con 0x7f257c0ccf00 2024-09-16T15:43:45.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.308+0000 7f2573e00640 1 -- 172.21.15.60:0/417798969 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568012870 con 0x7f257c0ccf00 2024-09-16T15:43:45.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.308+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257c0b3910 con 0x7f257c0ccf00 2024-09-16T15:43:45.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.308+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257c0b3ed0 con 0x7f257c0ccf00 2024-09-16T15:43:45.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.309+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257c06f980 con 0x7f257c0ccf00 2024-09-16T15:43:45.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.310+0000 7f2573e00640 1 -- 172.21.15.60:0/417798969 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f25680035f0 con 0x7f257c0ccf00 2024-09-16T15:43:45.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.312+0000 7f2573e00640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f255c078970 0x7f255c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:45.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.312+0000 7f2573e00640 1 -- 172.21.15.60:0/417798969 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7f256809a760 con 0x7f257c0ccf00 2024-09-16T15:43:45.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.312+0000 7f257be00640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f255c078970 0x7f255c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:45.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.312+0000 7f257be00640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f255c078970 0x7f255c07ae30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f256c033000 tx=0x7f256c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:45.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.316+0000 7f2573e00640 1 -- 172.21.15.60:0/417798969 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2568062ab0 con 0x7f257c0ccf00 2024-09-16T15:43:45.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.548+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f257c0ae470 con 0x7f257c0ccf00 2024-09-16T15:43:45.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.549+0000 7f2573e00640 1 -- 172.21.15.60:0/417798969 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5231 (secure 0 0 0) 0x7f25680623b0 con 0x7f257c0ccf00 2024-09-16T15:43:45.550 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:45.550 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"btime":"2024-09-16T15:43:39:873967+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1388326737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1388326737},{"type":"v1","addr":"172.21.15.100:6835","nonce":1388326737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6382,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/2485723644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2485723644},{"type":"v1","addr":"172.21.15.100:6801","nonce":2485723644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2204286380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2204286380},{"type":"v1","addr":"172.21.15.60:6835","nonce":2204286380}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:22.963712+0000","modified":"2024-09-16T15:43:25.979087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6576},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6576":{"gid":6576,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6834/3103295284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3103295284},{"type":"v1","addr":"172.21.15.60:6834","nonce":3103295284}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6576,"qdb_cluster":[6576]},"id":6}]} 2024-09-16T15:43:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f255c078970 msgr2=0x7f255c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f255c078970 0x7f255c07ae30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f256c033000 tx=0x7f256c004730 comp rx=0 tx=0).stop 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c0ccf00 msgr2=0x7f257c0b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c0ccf00 0x7f257c0b2d50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f256800c450 tx=0x7f256800c910 comp rx=0 tx=0).stop 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 shutdown_connections 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f255c078970 0x7f255c07ae30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f257c15f5a0 0x7f257c1f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f257c0ccf00 0x7f257c0b2d50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 --2- 172.21.15.60:0/417798969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f257c073fb0 0x7f257c153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.551+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 >> 172.21.15.60:0/417798969 conn(0x7f257c0b1950 msgr2=0x7f257c0cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.552+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 shutdown_connections 2024-09-16T15:43:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.552+0000 7f2583000640 1 -- 172.21.15.60:0/417798969 wait complete. 2024-09-16T15:43:45.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:45.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.728+0000 7eff3c200640 1 -- 172.21.15.60:0/104231110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 msgr2=0x7eff3406cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.728+0000 7eff3c200640 1 --2- 172.21.15.60:0/104231110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 0x7eff3406cb70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7eff20009830 tx=0x7eff2002a3f0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.728+0000 7eff3c200640 1 -- 172.21.15.60:0/104231110 shutdown_connections 2024-09-16T15:43:45.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.728+0000 7eff3c200640 1 --2- 172.21.15.60:0/104231110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3406d0b0 0x7eff34162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.728+0000 7eff3c200640 1 --2- 172.21.15.60:0/104231110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 0x7eff3406cb70 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.728+0000 7eff3c200640 1 -- 172.21.15.60:0/104231110 >> 172.21.15.60:0/104231110 conn(0x7eff340b2230 msgr2=0x7eff340b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:45.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.728+0000 7eff3c200640 1 -- 172.21.15.60:0/104231110 shutdown_connections 2024-09-16T15:43:45.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.728+0000 7eff3c200640 1 -- 172.21.15.60:0/104231110 wait complete. 2024-09-16T15:43:45.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.730+0000 7eff3c200640 1 Processor -- start 2024-09-16T15:43:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.730+0000 7eff3c200640 1 -- start start 2024-09-16T15:43:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff3c200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 0x7eff341571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff3c200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3406d0b0 0x7eff34145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff3c200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff340b3e20 0x7eff34146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff3c200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7eff34070e40 con 0x7eff340688b0 2024-09-16T15:43:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff3c200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff34070cc0 con 0x7eff340b3e20 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff3c200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff34162690 con 0x7eff3406d0b0 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff39800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 0x7eff341571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff39800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 0x7eff341571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59252/0 (socket says 172.21.15.60:59252) 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff39800640 1 -- 172.21.15.60:0/3551611658 learned_addr learned my addr 172.21.15.60:0/3551611658 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff38e00640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3406d0b0 0x7eff34145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff3a200640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff340b3e20 0x7eff34146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff39800640 1 -- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3406d0b0 msgr2=0x7eff34145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff39800640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3406d0b0 0x7eff34145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff39800640 1 -- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff340b3e20 msgr2=0x7eff34146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff39800640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff340b3e20 0x7eff34146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff39800640 1 -- 172.21.15.60:0/3551611658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff34156720 con 0x7eff340688b0 2024-09-16T15:43:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.731+0000 7eff38e00640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3406d0b0 0x7eff34145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.732+0000 7eff39800640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 0x7eff341571c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7eff2800c3c0 tx=0x7eff2800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.732+0000 7eff31600640 1 -- 172.21.15.60:0/3551611658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28017070 con 0x7eff340688b0 2024-09-16T15:43:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.732+0000 7eff31600640 1 -- 172.21.15.60:0/3551611658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff280035f0 con 0x7eff340688b0 2024-09-16T15:43:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.732+0000 7eff31600640 1 -- 172.21.15.60:0/3551611658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28003a80 con 0x7eff340688b0 2024-09-16T15:43:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.732+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff34157700 con 0x7eff340688b0 2024-09-16T15:43:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.732+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff341569c0 con 0x7eff340688b0 2024-09-16T15:43:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.733+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff340d0270 con 0x7eff340688b0 2024-09-16T15:43:45.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.734+0000 7eff31600640 1 -- 172.21.15.60:0/3551611658 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7eff2801a070 con 0x7eff340688b0 2024-09-16T15:43:45.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.735+0000 7eff31600640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff14078970 0x7eff1407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:45.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.735+0000 7eff31600640 1 -- 172.21.15.60:0/3551611658 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7eff28099fc0 con 0x7eff340688b0 2024-09-16T15:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.736+0000 7eff38e00640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff14078970 0x7eff1407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.736+0000 7eff38e00640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff14078970 0x7eff1407ae30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7eff20034000 tx=0x7eff20004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:45.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.740+0000 7eff31600640 1 -- 172.21.15.60:0/3551611658 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7eff28062310 con 0x7eff340688b0 2024-09-16T15:43:45.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.935+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff34161810 con 0x7eff340688b0 2024-09-16T15:43:45.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.936+0000 7eff31600640 1 -- 172.21.15.60:0/3551611658 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16849 (secure 0 0 0) 0x7eff28061c10 con 0x7eff340688b0 2024-09-16T15:43:45.938 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:45.938 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:41.881993+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:20.760401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:21.917247+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2496703614":"2024-09-17T15:43:38.871416+0000","172.21.15.60:6833/2496703614":"2024-09-17T15:43:38.871416+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:45.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff14078970 msgr2=0x7eff1407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff14078970 0x7eff1407ae30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7eff20034000 tx=0x7eff20004730 comp rx=0 tx=0).stop 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 msgr2=0x7eff341571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 0x7eff341571c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7eff2800c3c0 tx=0x7eff2800c880 comp rx=0 tx=0).stop 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 shutdown_connections 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff14078970 0x7eff1407ae30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff340b3e20 0x7eff34146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff3406d0b0 0x7eff34145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 --2- 172.21.15.60:0/3551611658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff340688b0 0x7eff341571c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 >> 172.21.15.60:0/3551611658 conn(0x7eff340b2230 msgr2=0x7eff340adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 shutdown_connections 2024-09-16T15:43:45.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:45.939+0000 7eff3c200640 1 -- 172.21.15.60:0/3551611658 wait complete. 2024-09-16T15:43:45.957 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:43:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.117+0000 7f7b72000640 1 -- 172.21.15.60:0/1099861705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c0ccf00 msgr2=0x7f7b6c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.117+0000 7f7b72000640 1 --2- 172.21.15.60:0/1099861705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c0ccf00 0x7f7b6c0b2440 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7b58005fd0 tx=0x7f7b5802a540 comp rx=0 tx=0).stop 2024-09-16T15:43:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.117+0000 7f7b72000640 1 -- 172.21.15.60:0/1099861705 shutdown_connections 2024-09-16T15:43:46.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.117+0000 7f7b72000640 1 --2- 172.21.15.60:0/1099861705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c0ccf00 0x7f7b6c0b2440 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.117+0000 7f7b72000640 1 -- 172.21.15.60:0/1099861705 >> 172.21.15.60:0/1099861705 conn(0x7f7b6c0b1950 msgr2=0x7f7b6c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:46.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.117+0000 7f7b72000640 1 -- 172.21.15.60:0/1099861705 shutdown_connections 2024-09-16T15:43:46.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.117+0000 7f7b72000640 1 -- 172.21.15.60:0/1099861705 wait complete. 2024-09-16T15:43:46.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.119+0000 7f7b72000640 1 Processor -- start 2024-09-16T15:43:46.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.119+0000 7f7b72000640 1 -- start start 2024-09-16T15:43:46.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.119+0000 7f7b72000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b6c073fb0 0x7f7b6c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:46.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b72000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c0ccf00 0x7f7b6c1f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:46.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b72000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c15f5a0 0x7f7b6c14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:46.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b72000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c06f9f0 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b72000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c06f870 con 0x7f7b6c0ccf00 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b72000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c074420 con 0x7f7b6c073fb0 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c15f5a0 0x7f7b6c14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c15f5a0 0x7f7b6c14c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59274/0 (socket says 172.21.15.60:59274) 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6be00640 1 -- 172.21.15.60:0/2525560369 learned_addr learned my addr 172.21.15.60:0/2525560369 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6aa00640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c0ccf00 0x7f7b6c1f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6b400640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b6c073fb0 0x7f7b6c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6be00640 1 -- 172.21.15.60:0/2525560369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b6c073fb0 msgr2=0x7f7b6c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6be00640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b6c073fb0 0x7f7b6c158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6be00640 1 -- 172.21.15.60:0/2525560369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c0ccf00 msgr2=0x7f7b6c1f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6be00640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c0ccf00 0x7f7b6c1f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.120+0000 7f7b6be00640 1 -- 172.21.15.60:0/2525560369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c1f2b70 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.121+0000 7f7b6be00640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c15f5a0 0x7f7b6c14c630 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400c7a0 tx=0x7f7b6400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:46.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.121+0000 7f7b63400640 1 -- 172.21.15.60:0/2525560369 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64003b70 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.121+0000 7f7b63400640 1 -- 172.21.15.60:0/2525560369 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b64003d30 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.121+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1f2e10 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.121+0000 7f7b63400640 1 -- 172.21.15.60:0/2525560369 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400aab0 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.121+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1f33d0 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.122+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c0b2440 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.123+0000 7f7b63400640 1 -- 172.21.15.60:0/2525560369 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f7b6400ac50 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.124+0000 7f7b63400640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b4c078970 0x7f7b4c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:46.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.124+0000 7f7b63400640 1 -- 172.21.15.60:0/2525560369 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5795+0+0 (secure 0 0 0) 0x7f7b6409aa60 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.125+0000 7f7b6b400640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b4c078970 0x7f7b4c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:46.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.125+0000 7f7b6b400640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b4c078970 0x7f7b4c07ae30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c006fd0 tx=0x7f7b5c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:46.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.129+0000 7f7b63400640 1 -- 172.21.15.60:0/2525560369 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7b64062e40 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.368+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7b6c0cc2d0 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.369 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:43:46.369+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:43:46.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.370+0000 7f7b63400640 1 -- 172.21.15.60:0/2525560369 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7f7b64062740 con 0x7f7b6c15f5a0 2024-09-16T15:43:46.371 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:43:46.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b4c078970 msgr2=0x7f7b4c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:46.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b4c078970 0x7f7b4c07ae30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c006fd0 tx=0x7f7b5c008040 comp rx=0 tx=0).stop 2024-09-16T15:43:46.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c15f5a0 msgr2=0x7f7b6c14c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:46.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c15f5a0 0x7f7b6c14c630 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400c7a0 tx=0x7f7b6400cc60 comp rx=0 tx=0).stop 2024-09-16T15:43:46.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 shutdown_connections 2024-09-16T15:43:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b4c078970 0x7f7b4c07ae30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b6c15f5a0 0x7f7b6c14c630 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b6c0ccf00 0x7f7b6c1f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 --2- 172.21.15.60:0/2525560369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b6c073fb0 0x7f7b6c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 >> 172.21.15.60:0/2525560369 conn(0x7f7b6c0b1950 msgr2=0x7f7b6c15e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 shutdown_connections 2024-09-16T15:43:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.374+0000 7f7b72000640 1 -- 172.21.15.60:0/2525560369 wait complete. 2024-09-16T15:43:46.385 INFO:tasks.ceph.mds.c.smithi060.stderr: -2638> 2024-09-16T15:43:29.491+0000 7f82d6a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:43:46.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:43:46.437 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:43:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.563+0000 7f5c8e400640 1 -- 172.21.15.60:0/536843150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 msgr2=0x7f5c880b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.563+0000 7f5c8e400640 1 --2- 172.21.15.60:0/536843150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 0x7f5c880b3890 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5c74005fd0 tx=0x7f5c7402a540 comp rx=0 tx=0).stop 2024-09-16T15:43:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.563+0000 7f5c8e400640 1 -- 172.21.15.60:0/536843150 shutdown_connections 2024-09-16T15:43:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.563+0000 7f5c8e400640 1 --2- 172.21.15.60:0/536843150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 0x7f5c880b3890 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.563+0000 7f5c8e400640 1 --2- 172.21.15.60:0/536843150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c880cf4b0 0x7f5c880cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.563+0000 7f5c8e400640 1 -- 172.21.15.60:0/536843150 >> 172.21.15.60:0/536843150 conn(0x7f5c880d3d50 msgr2=0x7f5c880d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.563+0000 7f5c8e400640 1 -- 172.21.15.60:0/536843150 shutdown_connections 2024-09-16T15:43:46.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.563+0000 7f5c8e400640 1 -- 172.21.15.60:0/536843150 wait complete. 2024-09-16T15:43:46.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.564+0000 7f5c8e400640 1 Processor -- start 2024-09-16T15:43:46.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.564+0000 7f5c8e400640 1 -- start start 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.565+0000 7f5c8e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 0x7f5c880c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.565+0000 7f5c8e400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c880cf4b0 0x7f5c880c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.565+0000 7f5c8e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c8815df60 0x7f5c880b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.565+0000 7f5c8e400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5c880d24b0 con 0x7f5c880b3450 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.565+0000 7f5c8e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c880d2330 con 0x7f5c8815df60 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.565+0000 7f5c8e400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5c880d2630 con 0x7f5c880cf4b0 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 0x7f5c880c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 0x7f5c880c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59304/0 (socket says 172.21.15.60:59304) 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 -- 172.21.15.60:0/3271688453 learned_addr learned my addr 172.21.15.60:0/3271688453 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c87e00640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c880cf4b0 0x7f5c880c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8da00640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c8815df60 0x7f5c880b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 -- 172.21.15.60:0/3271688453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c880cf4b0 msgr2=0x7f5c880c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c880cf4b0 0x7f5c880c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 -- 172.21.15.60:0/3271688453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c8815df60 msgr2=0x7f5c880b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c8815df60 0x7f5c880b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 -- 172.21.15.60:0/3271688453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c880c99f0 con 0x7f5c880b3450 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.566+0000 7f5c8d000640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 0x7f5c880c8400 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c3a0 tx=0x7f5c7800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.567+0000 7f5c84c00640 1 -- 172.21.15.60:0/3271688453 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78017070 con 0x7f5c880b3450 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.567+0000 7f5c84c00640 1 -- 172.21.15.60:0/3271688453 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c780053c0 con 0x7f5c880b3450 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.567+0000 7f5c84c00640 1 -- 172.21.15.60:0/3271688453 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78003850 con 0x7f5c880b3450 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.567+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c880c9cc0 con 0x7f5c880b3450 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.567+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c880c7920 con 0x7f5c880b3450 2024-09-16T15:43:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.568+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c880d2bc0 con 0x7f5c880b3450 2024-09-16T15:43:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.569+0000 7f5c84c00640 1 -- 172.21.15.60:0/3271688453 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f5c7801a070 con 0x7f5c880b3450 2024-09-16T15:43:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.570+0000 7f5c84c00640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5c68078970 0x7f5c6807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.570+0000 7f5c84c00640 1 -- 172.21.15.60:0/3271688453 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5881+0+0 (secure 0 0 0) 0x7f5c78099f30 con 0x7f5c880b3450 2024-09-16T15:43:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.570+0000 7f5c87e00640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5c68078970 0x7f5c6807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.571+0000 7f5c87e00640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5c68078970 0x7f5c6807ae30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c74033000 tx=0x7f5c74004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.574+0000 7f5c84c00640 1 -- 172.21.15.60:0/3271688453 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5c78061c70 con 0x7f5c880b3450 2024-09-16T15:43:46.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:46.808+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5c8815c140 con 0x7f5c880b3450 2024-09-16T15:43:47.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.372+0000 7f5c84c00640 1 -- 172.21.15.60:0/3271688453 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7f5c780666a0 con 0x7f5c880b3450 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.375+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5c68078970 msgr2=0x7f5c6807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.375+0000 7f5c8e400640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5c68078970 0x7f5c6807ae30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c74033000 tx=0x7f5c74004730 comp rx=0 tx=0).stop 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.375+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 msgr2=0x7f5c880c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.375+0000 7f5c8e400640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 0x7f5c880c8400 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c3a0 tx=0x7f5c7800c860 comp rx=0 tx=0).stop 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.376+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 shutdown_connections 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.376+0000 7f5c8e400640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5c68078970 0x7f5c6807ae30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.376+0000 7f5c8e400640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c8815df60 0x7f5c880b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.376+0000 7f5c8e400640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c880cf4b0 0x7f5c880c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.376+0000 7f5c8e400640 1 --2- 172.21.15.60:0/3271688453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c880b3450 0x7f5c880c8400 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.376+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 >> 172.21.15.60:0/3271688453 conn(0x7f5c880d3d50 msgr2=0x7f5c880b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:47.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.376+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 shutdown_connections 2024-09-16T15:43:47.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.376+0000 7f5c8e400640 1 -- 172.21.15.60:0/3271688453 wait complete. 2024-09-16T15:43:47.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:43:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.609+0000 7f7d4fe00640 1 -- 172.21.15.60:0/3022063835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bbb0 msgr2=0x7f7d500688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.609+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/3022063835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bbb0 0x7f7d500688b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009830 tx=0x7f7d4002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.609+0000 7f7d4fe00640 1 -- 172.21.15.60:0/3022063835 shutdown_connections 2024-09-16T15:43:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.609+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/3022063835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d500cd4d0 0x7f7d500afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.609+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/3022063835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bbb0 0x7f7d500688b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.609+0000 7f7d4fe00640 1 -- 172.21.15.60:0/3022063835 >> 172.21.15.60:0/3022063835 conn(0x7f7d500d03e0 msgr2=0x7f7d500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.609+0000 7f7d4fe00640 1 -- 172.21.15.60:0/3022063835 shutdown_connections 2024-09-16T15:43:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.609+0000 7f7d4fe00640 1 -- 172.21.15.60:0/3022063835 wait complete. 2024-09-16T15:43:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.610+0000 7f7d4fe00640 1 Processor -- start 2024-09-16T15:43:47.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.611+0000 7f7d4fe00640 1 -- start start 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.611+0000 7f7d4fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d500cd4d0 0x7f7d500cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.611+0000 7f7d4fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d5015bbb0 0x7f7d500cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.611+0000 7f7d4fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bf90 0x7f7d500ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.611+0000 7f7d4fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5015fe80 con 0x7f7d5015bbb0 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.611+0000 7f7d4fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5015fd00 con 0x7f7d5015bf90 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.611+0000 7f7d4fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7d50160000 con 0x7f7d500cd4d0 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d5015bbb0 0x7f7d500cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d5015bbb0 0x7f7d500cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59316/0 (socket says 172.21.15.60:59316) 2024-09-16T15:43:47.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 -- 172.21.15.60:0/917993830 learned_addr learned my addr 172.21.15.60:0/917993830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d4f400640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bf90 0x7f7d500ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d4ea00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d500cd4d0 0x7f7d500cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 -- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d500cd4d0 msgr2=0x7f7d500cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d500cd4d0 0x7f7d500cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 -- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bf90 msgr2=0x7f7d500ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bf90 0x7f7d500ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 -- 172.21.15.60:0/917993830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501f47e0 con 0x7f7d5015bbb0 2024-09-16T15:43:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d4f400640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bf90 0x7f7d500ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d4ea00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d500cd4d0 0x7f7d500cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.612+0000 7f7d46000640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d5015bbb0 0x7f7d500cdfb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c430 tx=0x7f7d3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.613+0000 7f7d47400640 1 -- 172.21.15.60:0/917993830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38017070 con 0x7f7d5015bbb0 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.613+0000 7f7d47400640 1 -- 172.21.15.60:0/917993830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d380035f0 con 0x7f7d5015bbb0 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.613+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d50157190 con 0x7f7d5015bbb0 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.613+0000 7f7d47400640 1 -- 172.21.15.60:0/917993830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38003a40 con 0x7f7d5015bbb0 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.613+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d50157a30 con 0x7f7d5015bbb0 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.614+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d50160540 con 0x7f7d5015bbb0 2024-09-16T15:43:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.615+0000 7f7d47400640 1 -- 172.21.15.60:0/917993830 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f7d3801a070 con 0x7f7d5015bbb0 2024-09-16T15:43:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.616+0000 7f7d47400640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d2c078970 0x7f7d2c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.616+0000 7f7d47400640 1 -- 172.21.15.60:0/917993830 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5881+0+0 (secure 0 0 0) 0x7f7d3809a010 con 0x7f7d5015bbb0 2024-09-16T15:43:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.616+0000 7f7d4ea00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d2c078970 0x7f7d2c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:47.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.617+0000 7f7d4ea00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d2c078970 0x7f7d2c07ae30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d40034000 tx=0x7f7d40004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:47.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.620+0000 7f7d47400640 1 -- 172.21.15.60:0/917993830 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7d500cbcb0 con 0x7f7d5015bbb0 2024-09-16T15:43:47.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:47.819+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d50073ab0 con 0x7f7d5015bbb0 2024-09-16T15:43:48.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.374+0000 7f7d47400640 1 -- 172.21.15.60:0/917993830 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7f7d380623a0 con 0x7f7d5015bbb0 2024-09-16T15:43:48.375 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:43:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d2c078970 msgr2=0x7f7d2c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d2c078970 0x7f7d2c07ae30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d40034000 tx=0x7f7d40004730 comp rx=0 tx=0).stop 2024-09-16T15:43:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d5015bbb0 msgr2=0x7f7d500cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d5015bbb0 0x7f7d500cdfb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c430 tx=0x7f7d3800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 shutdown_connections 2024-09-16T15:43:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7d2c078970 0x7f7d2c07ae30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d5015bf90 0x7f7d500ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d5015bbb0 0x7f7d500cdfb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 --2- 172.21.15.60:0/917993830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d500cd4d0 0x7f7d500cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.378+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 >> 172.21.15.60:0/917993830 conn(0x7f7d500d03e0 msgr2=0x7f7d500adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.379+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 shutdown_connections 2024-09-16T15:43:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.379+0000 7f7d4fe00640 1 -- 172.21.15.60:0/917993830 wait complete. 2024-09-16T15:43:48.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:43:48.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.568+0000 7f71b4a00640 1 -- 172.21.15.60:0/475135824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 msgr2=0x7f71b00b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:48.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.568+0000 7f71b4a00640 1 --2- 172.21.15.60:0/475135824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 0x7f71b00b0a90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005fd0 tx=0x7f71a802aa00 comp rx=0 tx=0).stop 2024-09-16T15:43:48.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.568+0000 7f71b4a00640 1 -- 172.21.15.60:0/475135824 shutdown_connections 2024-09-16T15:43:48.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.568+0000 7f71b4a00640 1 --2- 172.21.15.60:0/475135824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 0x7f71b00b0a90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.568+0000 7f71b4a00640 1 --2- 172.21.15.60:0/475135824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71b00cb720 0x7f71b00749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.568+0000 7f71b4a00640 1 -- 172.21.15.60:0/475135824 >> 172.21.15.60:0/475135824 conn(0x7f71b00d03e0 msgr2=0x7f71b00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:48.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.568+0000 7f71b4a00640 1 -- 172.21.15.60:0/475135824 shutdown_connections 2024-09-16T15:43:48.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.568+0000 7f71b4a00640 1 -- 172.21.15.60:0/475135824 wait complete. 2024-09-16T15:43:48.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.569+0000 7f71b4a00640 1 Processor -- start 2024-09-16T15:43:48.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.570+0000 7f71b4a00640 1 -- start start 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.570+0000 7f71b4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 0x7f71b00c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.570+0000 7f71b4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00cb720 0x7f71b015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71b4a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71b015f360 0x7f71b00b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71b4a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f71b0075080 con 0x7f71b00b05e0 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71b4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71b0074f00 con 0x7f71b00cb720 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71b4a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f71b00b0fd0 con 0x7f71b015f360 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ae800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 0x7f71b00c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ae800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 0x7f71b00c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59322/0 (socket says 172.21.15.60:59322) 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ae800640 1 -- 172.21.15.60:0/325196084 learned_addr learned my addr 172.21.15.60:0/325196084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ade00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00cb720 0x7f71b015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71af200640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71b015f360 0x7f71b00b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ae800640 1 -- 172.21.15.60:0/325196084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71b015f360 msgr2=0x7f71b00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ae800640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71b015f360 0x7f71b00b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ae800640 1 -- 172.21.15.60:0/325196084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00cb720 msgr2=0x7f71b015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ae800640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00cb720 0x7f71b015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.571+0000 7f71ae800640 1 -- 172.21.15.60:0/325196084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b01f7fd0 con 0x7f71b00b05e0 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.572+0000 7f71ae800640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 0x7f71b00c74b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f71a000b3f0 tx=0x7f71a000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.572+0000 7f71a6a00640 1 -- 172.21.15.60:0/325196084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0015020 con 0x7f71b00b05e0 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.572+0000 7f71a6a00640 1 -- 172.21.15.60:0/325196084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a0011920 con 0x7f71b00b05e0 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.572+0000 7f71a6a00640 1 -- 172.21.15.60:0/325196084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0011d70 con 0x7f71b00b05e0 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.572+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b01f8290 con 0x7f71b00b05e0 2024-09-16T15:43:48.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.572+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b01f8850 con 0x7f71b00b05e0 2024-09-16T15:43:48.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.573+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b00b0a90 con 0x7f71b00b05e0 2024-09-16T15:43:48.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.574+0000 7f71a6a00640 1 -- 172.21.15.60:0/325196084 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f71a0018020 con 0x7f71b00b05e0 2024-09-16T15:43:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.575+0000 7f71a6a00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f71900789a0 0x7f719007ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.575+0000 7f71a6a00640 1 -- 172.21.15.60:0/325196084 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5443+0+0 (secure 0 0 0) 0x7f71a009a580 con 0x7f71b00b05e0 2024-09-16T15:43:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.575+0000 7f71ade00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f71900789a0 0x7f719007ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.576+0000 7f71ade00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f71900789a0 0x7f719007ae60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71b0074db0 tx=0x7f719c00b390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.580+0000 7f71a6a00640 1 -- 172.21.15.60:0/325196084 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f71a001f030 con 0x7f71b00b05e0 2024-09-16T15:43:48.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:48.774+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f71b00af410 con 0x7f71b00b05e0 2024-09-16T15:43:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.385+0000 7f71a6a00640 1 -- 172.21.15.60:0/325196084 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7f71a0062470 con 0x7f71b00b05e0 2024-09-16T15:43:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:43:49.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.388+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f71900789a0 msgr2=0x7f719007ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.388+0000 7f71b4a00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f71900789a0 0x7f719007ae60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71b0074db0 tx=0x7f719c00b390 comp rx=0 tx=0).stop 2024-09-16T15:43:49.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.388+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 msgr2=0x7f71b00c74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.388+0000 7f71b4a00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 0x7f71b00c74b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f71a000b3f0 tx=0x7f71a000b8b0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.389+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 shutdown_connections 2024-09-16T15:43:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.389+0000 7f71b4a00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f71900789a0 0x7f719007ae60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.389+0000 7f71b4a00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71b015f360 0x7f71b00b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.389+0000 7f71b4a00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00cb720 0x7f71b015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.389+0000 7f71b4a00640 1 --2- 172.21.15.60:0/325196084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71b00b05e0 0x7f71b00c74b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.389+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 >> 172.21.15.60:0/325196084 conn(0x7f71b00d03e0 msgr2=0x7f71b00b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.389+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 shutdown_connections 2024-09-16T15:43:49.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.390+0000 7f71b4a00640 1 -- 172.21.15.60:0/325196084 wait complete. 2024-09-16T15:43:49.410 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:43:49.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.576+0000 7f9847400640 1 -- 172.21.15.60:0/4119327312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9840161550 msgr2=0x7f984006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.576+0000 7f9847400640 1 --2- 172.21.15.60:0/4119327312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9840161550 0x7f984006ef50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f98400752e0 tx=0x7f983402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.576+0000 7f9847400640 1 -- 172.21.15.60:0/4119327312 shutdown_connections 2024-09-16T15:43:49.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.576+0000 7f9847400640 1 --2- 172.21.15.60:0/4119327312 >> v2:172.21.15.100:3300/0 conn(0x7f98400af3d0 0x7f984006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.576+0000 7f9847400640 1 --2- 172.21.15.60:0/4119327312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9840161550 0x7f984006ef50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.576+0000 7f9847400640 1 -- 172.21.15.60:0/4119327312 >> 172.21.15.60:0/4119327312 conn(0x7f98400b1950 msgr2=0x7f98400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:49.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.576+0000 7f9847400640 1 -- 172.21.15.60:0/4119327312 shutdown_connections 2024-09-16T15:43:49.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.576+0000 7f9847400640 1 -- 172.21.15.60:0/4119327312 wait complete. 2024-09-16T15:43:49.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.577+0000 7f9847400640 1 Processor -- start 2024-09-16T15:43:49.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.578+0000 7f9847400640 1 -- start start 2024-09-16T15:43:49.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.578+0000 7f9847400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984006f9d0 0x7f98400cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:49.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.578+0000 7f9847400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98400af3d0 0x7f98401f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:49.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.578+0000 7f9847400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840161550 0x7f98401f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:49.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.578+0000 7f9847400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9840161a40 con 0x7f984006f9d0 2024-09-16T15:43:49.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.578+0000 7f9847400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9840074010 con 0x7f98400af3d0 2024-09-16T15:43:49.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.578+0000 7f9847400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f98401623a0 con 0x7f9840161550 2024-09-16T15:43:49.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984006f9d0 0x7f98400cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984006f9d0 0x7f98400cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58752/0 (socket says 172.21.15.60:58752) 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 -- 172.21.15.60:0/3791849656 learned_addr learned my addr 172.21.15.60:0/3791849656 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9845400640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840161550 0x7f98401f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f983fe00640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98400af3d0 0x7f98401f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 -- 172.21.15.60:0/3791849656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840161550 msgr2=0x7f98401f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840161550 0x7f98401f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 -- 172.21.15.60:0/3791849656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98400af3d0 msgr2=0x7f98401f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98400af3d0 0x7f98401f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 -- 172.21.15.60:0/3791849656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98401f5830 con 0x7f984006f9d0 2024-09-16T15:43:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.579+0000 7f9844a00640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984006f9d0 0x7f98400cd050 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9834034040 tx=0x7f98340039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.580+0000 7f983cc00640 1 -- 172.21.15.60:0/3791849656 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983403e030 con 0x7f984006f9d0 2024-09-16T15:43:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.580+0000 7f983cc00640 1 -- 172.21.15.60:0/3791849656 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98340047a0 con 0x7f984006f9d0 2024-09-16T15:43:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.580+0000 7f983cc00640 1 -- 172.21.15.60:0/3791849656 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834035040 con 0x7f984006f9d0 2024-09-16T15:43:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.580+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98400ad2d0 con 0x7f984006f9d0 2024-09-16T15:43:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.580+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98400ad890 con 0x7f984006f9d0 2024-09-16T15:43:49.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.581+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9840162850 con 0x7f984006f9d0 2024-09-16T15:43:49.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.582+0000 7f983cc00640 1 -- 172.21.15.60:0/3791849656 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f9834041030 con 0x7f984006f9d0 2024-09-16T15:43:49.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.583+0000 7f983cc00640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:49.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.583+0000 7f983cc00640 1 -- 172.21.15.60:0/3791849656 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7f98340bafa0 con 0x7f984006f9d0 2024-09-16T15:43:49.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.583+0000 7f983fe00640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:49.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.584+0000 7f983fe00640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98400d0720 tx=0x7f982c009110 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:49.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.587+0000 7f983cc00640 1 -- 172.21.15.60:0/3791849656 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f98401f5830 con 0x7f984006f9d0 2024-09-16T15:43:49.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.783+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f9840074420 con 0x7f984006f9d0 2024-09-16T15:43:49.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.787+0000 7f983cc00640 1 -- 172.21.15.60:0/3791849656 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7f9834082fa0 con 0x7f984006f9d0 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 msgr2=0x7f982007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98400d0720 tx=0x7f982c009110 comp rx=0 tx=0).stop 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984006f9d0 msgr2=0x7f98400cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984006f9d0 0x7f98400cd050 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9834034040 tx=0x7f98340039e0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 shutdown_connections 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840161550 0x7f98401f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98400af3d0 0x7f98401f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 --2- 172.21.15.60:0/3791849656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f984006f9d0 0x7f98400cd050 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.789+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 >> 172.21.15.60:0/3791849656 conn(0x7f98400b1950 msgr2=0x7f984015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.790+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 shutdown_connections 2024-09-16T15:43:49.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.790+0000 7f9847400640 1 -- 172.21.15.60:0/3791849656 wait complete. 2024-09-16T15:43:49.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-16T15:43:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.973+0000 7fdba2600640 1 -- 172.21.15.60:0/3114498027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb80004800 msgr2=0x7fdb80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.973+0000 7fdba2600640 1 --2- 172.21.15.60:0/3114498027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb80004800 0x7fdb80004c80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400c6e0 tx=0x7fdb80005830 comp rx=0 tx=0).stop 2024-09-16T15:43:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.973+0000 7fdba2600640 1 -- 172.21.15.60:0/3114498027 shutdown_connections 2024-09-16T15:43:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.973+0000 7fdba2600640 1 --2- 172.21.15.60:0/3114498027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb80004800 0x7fdb80004c80 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.973+0000 7fdba2600640 1 -- 172.21.15.60:0/3114498027 >> 172.21.15.60:0/3114498027 conn(0x7fdb9c0d1c40 msgr2=0x7fdb9c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.973+0000 7fdba2600640 1 -- 172.21.15.60:0/3114498027 shutdown_connections 2024-09-16T15:43:49.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.973+0000 7fdba2600640 1 -- 172.21.15.60:0/3114498027 wait complete. 2024-09-16T15:43:49.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.975+0000 7fdba2600640 1 Processor -- start 2024-09-16T15:43:49.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.975+0000 7fdba2600640 1 -- start start 2024-09-16T15:43:49.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdba2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80004800 0x7fdb9c153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:49.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdba2600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c153d50 0x7fdb9c1f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:49.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdba2600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c1f7f00 0x7fdb9c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdba2600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c06e810 con 0x7fdb9c153d50 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdba2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c06e690 con 0x7fdb80004800 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdba2600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c06e990 con 0x7fdb9c1f7f00 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdb93e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c153d50 0x7fdb9c1f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdb93e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c153d50 0x7fdb9c1f79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58766/0 (socket says 172.21.15.60:58766) 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdb93e00640 1 -- 172.21.15.60:0/2361945177 learned_addr learned my addr 172.21.15.60:0/2361945177 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdba1200640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80004800 0x7fdb9c153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdba1c00640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c1f7f00 0x7fdb9c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdb93e00640 1 -- 172.21.15.60:0/2361945177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c1f7f00 msgr2=0x7fdb9c0b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdb93e00640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c1f7f00 0x7fdb9c0b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdb93e00640 1 -- 172.21.15.60:0/2361945177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80004800 msgr2=0x7fdb9c153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdb93e00640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80004800 0x7fdb9c153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.976+0000 7fdb93e00640 1 -- 172.21.15.60:0/2361945177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c1f8530 con 0x7fdb9c153d50 2024-09-16T15:43:49.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.977+0000 7fdb93e00640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c153d50 0x7fdb9c1f79c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400c800 tx=0x7fdb84006380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.977+0000 7fdb99600640 1 -- 172.21.15.60:0/2361945177 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84004830 con 0x7fdb9c153d50 2024-09-16T15:43:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.977+0000 7fdb99600640 1 -- 172.21.15.60:0/2361945177 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb84039070 con 0x7fdb9c153d50 2024-09-16T15:43:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.977+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c1f8810 con 0x7fdb9c153d50 2024-09-16T15:43:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.977+0000 7fdb99600640 1 -- 172.21.15.60:0/2361945177 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84004c90 con 0x7fdb9c153d50 2024-09-16T15:43:49.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.977+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c09d690 con 0x7fdb9c153d50 2024-09-16T15:43:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.978+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c06eef0 con 0x7fdb9c153d50 2024-09-16T15:43:49.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.979+0000 7fdb99600640 1 -- 172.21.15.60:0/2361945177 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fdb84043030 con 0x7fdb9c153d50 2024-09-16T15:43:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.980+0000 7fdb99600640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb78078970 0x7fdb7807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:49.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.980+0000 7fdb99600640 1 -- 172.21.15.60:0/2361945177 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7fdb840ba0b0 con 0x7fdb9c153d50 2024-09-16T15:43:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.981+0000 7fdba1200640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb78078970 0x7fdb7807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:49.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.981+0000 7fdba1200640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb78078970 0x7fdb7807ae30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c001860 tx=0x7fdb8c001110 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:49.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:49.985+0000 7fdb99600640 1 -- 172.21.15.60:0/2361945177 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdb8404a060 con 0x7fdb9c153d50 2024-09-16T15:43:50.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.259+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7fdb9c160570 con 0x7fdb9c153d50 2024-09-16T15:43:50.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.386+0000 7fdb99600640 1 -- 172.21.15.60:0/2361945177 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v295) ==== 146+0+0 (secure 0 0 0) 0x7fdb84082830 con 0x7fdb9c153d50 2024-09-16T15:43:50.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb78078970 msgr2=0x7fdb7807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb78078970 0x7fdb7807ae30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c001860 tx=0x7fdb8c001110 comp rx=0 tx=0).stop 2024-09-16T15:43:50.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c153d50 msgr2=0x7fdb9c1f79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c153d50 0x7fdb9c1f79c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400c800 tx=0x7fdb84006380 comp rx=0 tx=0).stop 2024-09-16T15:43:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 shutdown_connections 2024-09-16T15:43:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb78078970 0x7fdb7807ae30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c1f7f00 0x7fdb9c0b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c153d50 0x7fdb9c1f79c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 --2- 172.21.15.60:0/2361945177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb80004800 0x7fdb9c153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 >> 172.21.15.60:0/2361945177 conn(0x7fdb9c0d1c40 msgr2=0x7fdb9c15f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 shutdown_connections 2024-09-16T15:43:50.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.390+0000 7fdba2600640 1 -- 172.21.15.60:0/2361945177 wait complete. 2024-09-16T15:43:50.407 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-16T15:43:50.407 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-16T15:43:50.407 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:43:50.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:50.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.568+0000 7fb4a1400640 1 -- 172.21.15.60:0/954776712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 msgr2=0x7fb4800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.568+0000 7fb4a1400640 1 --2- 172.21.15.60:0/954776712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 0x7fb4800051c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb494009ed0 tx=0x7fb480005830 comp rx=0 tx=0).stop 2024-09-16T15:43:50.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.568+0000 7fb49b200640 1 -- 172.21.15.60:0/954776712 reap_dead start 2024-09-16T15:43:50.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.568+0000 7fb4a1400640 1 -- 172.21.15.60:0/954776712 shutdown_connections 2024-09-16T15:43:50.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.568+0000 7fb4a1400640 1 -- 172.21.15.60:0/954776712 >> 172.21.15.60:0/954776712 conn(0x7fb49c0b1950 msgr2=0x7fb49c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:50.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.568+0000 7fb4a1400640 1 -- 172.21.15.60:0/954776712 shutdown_connections 2024-09-16T15:43:50.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.568+0000 7fb4a1400640 1 -- 172.21.15.60:0/954776712 wait complete. 2024-09-16T15:43:50.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.570+0000 7fb4a1400640 1 Processor -- start 2024-09-16T15:43:50.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.570+0000 7fb4a1400640 1 -- start start 2024-09-16T15:43:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 0x7fb49c148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb49c0b4040 0x7fb49c1f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb49c1f8b00 0x7fb49c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb4a1400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c074840 con 0x7fb480004d40 2024-09-16T15:43:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb4a1400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c0746c0 con 0x7fb49c0b4040 2024-09-16T15:43:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb4a1400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb49c06f160 con 0x7fb49c1f8b00 2024-09-16T15:43:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 0x7fb49c148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 0x7fb49c148c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58776/0 (socket says 172.21.15.60:58776) 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49a800640 1 -- 172.21.15.60:0/584556351 learned_addr learned my addr 172.21.15.60:0/584556351 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49b200640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb49c1f8b00 0x7fb49c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49a800640 1 -- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb49c1f8b00 msgr2=0x7fb49c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb493e00640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb49c0b4040 0x7fb49c1f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49a800640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb49c1f8b00 0x7fb49c152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49a800640 1 -- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb49c0b4040 msgr2=0x7fb49c1f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49a800640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb49c0b4040 0x7fb49c1f85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49a800640 1 -- 172.21.15.60:0/584556351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c0b2fa0 con 0x7fb480004d40 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb49b200640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb49c1f8b00 0x7fb49c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.571+0000 7fb493e00640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb49c0b4040 0x7fb49c1f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.572+0000 7fb49a800640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 0x7fb49c148c50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e6d0 tx=0x7fb48c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.572+0000 7fb492a00640 1 -- 172.21.15.60:0/584556351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00eea0 con 0x7fb480004d40 2024-09-16T15:43:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.572+0000 7fb492a00640 1 -- 172.21.15.60:0/584556351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb48c004440 con 0x7fb480004d40 2024-09-16T15:43:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.572+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c0b3280 con 0x7fb480004d40 2024-09-16T15:43:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.572+0000 7fb492a00640 1 -- 172.21.15.60:0/584556351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00caa0 con 0x7fb480004d40 2024-09-16T15:43:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.572+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c0b3840 con 0x7fb480004d40 2024-09-16T15:43:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.573+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49c06f650 con 0x7fb480004d40 2024-09-16T15:43:50.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.574+0000 7fb492a00640 1 -- 172.21.15.60:0/584556351 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fb48c00cc40 con 0x7fb480004d40 2024-09-16T15:43:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.575+0000 7fb492a00640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4780789a0 0x7fb47807ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.575+0000 7fb492a00640 1 -- 172.21.15.60:0/584556351 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7fb48c09ab80 con 0x7fb480004d40 2024-09-16T15:43:50.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.575+0000 7fb493e00640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4780789a0 0x7fb47807ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:50.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.576+0000 7fb493e00640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4780789a0 0x7fb47807ae60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb484000c00 tx=0x7fb484006b70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.579+0000 7fb492a00640 1 -- 172.21.15.60:0/584556351 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb48c063260 con 0x7fb480004d40 2024-09-16T15:43:50.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.779+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb49c0aced0 con 0x7fb480004d40 2024-09-16T15:43:50.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.780+0000 7fb492a00640 1 -- 172.21.15.60:0/584556351 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+13163 (secure 0 0 0) 0x7fb48c062b60 con 0x7fb480004d40 2024-09-16T15:43:50.781 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:50.782 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":98,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:49.375539+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6834/3103295284":"2024-09-17T15:43:46.369357+0000","172.21.15.60:6832/3103295284":"2024-09-17T15:43:46.369357+0000","172.21.15.60:6835/2496703614":"2024-09-17T15:43:38.871416+0000","172.21.15.60:6833/2496703614":"2024-09-17T15:43:38.871416+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.782+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4780789a0 msgr2=0x7fb47807ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.782+0000 7fb4a1400640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4780789a0 0x7fb47807ae60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb484000c00 tx=0x7fb484006b70 comp rx=0 tx=0).stop 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.782+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 msgr2=0x7fb49c148c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.782+0000 7fb4a1400640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 0x7fb49c148c50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00e6d0 tx=0x7fb48c00eb90 comp rx=0 tx=0).stop 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.783+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 shutdown_connections 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.783+0000 7fb4a1400640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4780789a0 0x7fb47807ae60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.783+0000 7fb4a1400640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb49c1f8b00 0x7fb49c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.783+0000 7fb4a1400640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb49c0b4040 0x7fb49c1f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.783+0000 7fb4a1400640 1 --2- 172.21.15.60:0/584556351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb480004d40 0x7fb49c148c50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.783+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 >> 172.21.15.60:0/584556351 conn(0x7fb49c0b1950 msgr2=0x7fb49c1542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.783+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 shutdown_connections 2024-09-16T15:43:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.783+0000 7fb4a1400640 1 -- 172.21.15.60:0/584556351 wait complete. 2024-09-16T15:43:50.800 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:43:50.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:43:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.962+0000 7fbb08600640 1 -- 172.21.15.60:0/414522866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb00145750 msgr2=0x7fbb00073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.962+0000 7fbb08600640 1 --2- 172.21.15.60:0/414522866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb00145750 0x7fbb00073440 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbb000cfb70 tx=0x7fbaf402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.962+0000 7fbb08600640 1 -- 172.21.15.60:0/414522866 shutdown_connections 2024-09-16T15:43:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.962+0000 7fbb08600640 1 --2- 172.21.15.60:0/414522866 >> v2:172.21.15.100:3300/0 conn(0x7fbb000cccd0 0x7fbb000cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.962+0000 7fbb08600640 1 --2- 172.21.15.60:0/414522866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb00145750 0x7fbb00073440 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.962+0000 7fbb08600640 1 -- 172.21.15.60:0/414522866 >> 172.21.15.60:0/414522866 conn(0x7fbb000b1840 msgr2=0x7fbb000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:50.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.962+0000 7fbb08600640 1 -- 172.21.15.60:0/414522866 shutdown_connections 2024-09-16T15:43:50.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.962+0000 7fbb08600640 1 -- 172.21.15.60:0/414522866 wait complete. 2024-09-16T15:43:50.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.963+0000 7fbb08600640 1 Processor -- start 2024-09-16T15:43:50.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.964+0000 7fbb08600640 1 -- start start 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.964+0000 7fbb08600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb0006efa0 0x7fbb001f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.964+0000 7fbb08600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb000cccd0 0x7fbb0013bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.964+0000 7fbb08600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb00145750 0x7fbb000c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.964+0000 7fbb08600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbb000ce480 con 0x7fbb000cccd0 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.964+0000 7fbb08600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb000ce300 con 0x7fbb0006efa0 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.964+0000 7fbb08600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbb000ce600 con 0x7fbb00145750 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb000cccd0 0x7fbb0013bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb000cccd0 0x7fbb0013bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58794/0 (socket says 172.21.15.60:58794) 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 -- 172.21.15.60:0/2453852419 learned_addr learned my addr 172.21.15.60:0/2453852419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05c00640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb0006efa0 0x7fbb001f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb06600640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb00145750 0x7fbb000c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:50.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 -- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb00145750 msgr2=0x7fbb000c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb00145750 0x7fbb000c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 -- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb0006efa0 msgr2=0x7fbb001f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb0006efa0 0x7fbb001f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 -- 172.21.15.60:0/2453852419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb001f9040 con 0x7fbb000cccd0 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05c00640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb0006efa0 0x7fbb001f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.965+0000 7fbb05200640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb000cccd0 0x7fbb0013bf40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c440 tx=0x7fbaec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.966+0000 7fbafd600640 1 -- 172.21.15.60:0/2453852419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec017070 con 0x7fbb000cccd0 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.966+0000 7fbafd600640 1 -- 172.21.15.60:0/2453852419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaec0035f0 con 0x7fbb000cccd0 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.966+0000 7fbafd600640 1 -- 172.21.15.60:0/2453852419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec003a40 con 0x7fbb000cccd0 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.966+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0013c480 con 0x7fbb000cccd0 2024-09-16T15:43:50.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.966+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb000b3490 con 0x7fbb000cccd0 2024-09-16T15:43:50.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.967+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb000cdbb0 con 0x7fbb000cccd0 2024-09-16T15:43:50.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.968+0000 7fbafd600640 1 -- 172.21.15.60:0/2453852419 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fbaec003be0 con 0x7fbb000cccd0 2024-09-16T15:43:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.969+0000 7fbafd600640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbae0078970 0x7fbae007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.969+0000 7fbafd600640 1 -- 172.21.15.60:0/2453852419 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7fbaec099c80 con 0x7fbb000cccd0 2024-09-16T15:43:50.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.969+0000 7fbb05c00640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbae0078970 0x7fbae007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:50.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.970+0000 7fbb05c00640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbae0078970 0x7fbae007ae30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4033000 tx=0x7fbaf4004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:50.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:50.973+0000 7fbafd600640 1 -- 172.21.15.60:0/2453852419 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb001f9040 con 0x7fbb000cccd0 2024-09-16T15:43:51.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.171+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb00143540 con 0x7fbb000cccd0 2024-09-16T15:43:51.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.172+0000 7fbafd600640 1 -- 172.21.15.60:0/2453852419 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fbaec062360 con 0x7fbb000cccd0 2024-09-16T15:43:51.173 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbae0078970 msgr2=0x7fbae007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbae0078970 0x7fbae007ae30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4033000 tx=0x7fbaf4004730 comp rx=0 tx=0).stop 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb000cccd0 msgr2=0x7fbb0013bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb000cccd0 0x7fbb0013bf40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00c440 tx=0x7fbaec00c900 comp rx=0 tx=0).stop 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 shutdown_connections 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbae0078970 0x7fbae007ae30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb00145750 0x7fbb000c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb000cccd0 0x7fbb0013bf40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 --2- 172.21.15.60:0/2453852419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb0006efa0 0x7fbb001f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.174+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 >> 172.21.15.60:0/2453852419 conn(0x7fbb000b1840 msgr2=0x7fbb000b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.175+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 shutdown_connections 2024-09-16T15:43:51.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.175+0000 7fbb08600640 1 -- 172.21.15.60:0/2453852419 wait complete. 2024-09-16T15:43:51.191 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-16T15:43:51.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.351+0000 7f15cf200640 1 -- 172.21.15.60:0/456551694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d00725d0 msgr2=0x7f15d00ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.351+0000 7f15cf200640 1 --2- 172.21.15.60:0/456551694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d00725d0 0x7f15d00ceae0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f15b8005fd0 tx=0x7f15b802b910 comp rx=0 tx=0).stop 2024-09-16T15:43:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.351+0000 7f15cf200640 1 -- 172.21.15.60:0/456551694 shutdown_connections 2024-09-16T15:43:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.351+0000 7f15cf200640 1 --2- 172.21.15.60:0/456551694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d00725d0 0x7f15d00ceae0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.351+0000 7f15cf200640 1 -- 172.21.15.60:0/456551694 >> 172.21.15.60:0/456551694 conn(0x7f15d00cfa20 msgr2=0x7f15d00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.351+0000 7f15cf200640 1 -- 172.21.15.60:0/456551694 shutdown_connections 2024-09-16T15:43:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.351+0000 7f15cf200640 1 -- 172.21.15.60:0/456551694 wait complete. 2024-09-16T15:43:51.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.353+0000 7f15cf200640 1 Processor -- start 2024-09-16T15:43:51.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.353+0000 7f15cf200640 1 -- start start 2024-09-16T15:43:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15cf200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15d00725d0 0x7f15d01f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15cf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0072a40 0x7f15d00b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15cf200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d015a280 0x7f15d014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15cf200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f15d006f750 con 0x7f15d015a280 2024-09-16T15:43:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15cf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15d006f5d0 con 0x7f15d0072a40 2024-09-16T15:43:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15cf200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f15d00cf020 con 0x7f15d00725d0 2024-09-16T15:43:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15ce800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d015a280 0x7f15d014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15ce800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d015a280 0x7f15d014d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58818/0 (socket says 172.21.15.60:58818) 2024-09-16T15:43:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15ce800640 1 -- 172.21.15.60:0/140728057 learned_addr learned my addr 172.21.15.60:0/140728057 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15c5400640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0072a40 0x7f15d00b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.354+0000 7f15cde00640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15d00725d0 0x7f15d01f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15ce800640 1 -- 172.21.15.60:0/140728057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15d00725d0 msgr2=0x7f15d01f7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15ce800640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15d00725d0 0x7f15d01f7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15ce800640 1 -- 172.21.15.60:0/140728057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0072a40 msgr2=0x7f15d00b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15ce800640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0072a40 0x7f15d00b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15ce800640 1 -- 172.21.15.60:0/140728057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d01f8ca0 con 0x7f15d015a280 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15ce800640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d015a280 0x7f15d014d8e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c770 tx=0x7f15c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15c6a00640 1 -- 172.21.15.60:0/140728057 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8017070 con 0x7f15d015a280 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15c6a00640 1 -- 172.21.15.60:0/140728057 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15c8005070 con 0x7f15d015a280 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15c6a00640 1 -- 172.21.15.60:0/140728057 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c80054c0 con 0x7f15d015a280 2024-09-16T15:43:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d01f8400 con 0x7f15d015a280 2024-09-16T15:43:51.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.355+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d01f89c0 con 0x7f15d015a280 2024-09-16T15:43:51.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.356+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d0157b20 con 0x7f15d015a280 2024-09-16T15:43:51.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.358+0000 7f15c6a00640 1 -- 172.21.15.60:0/140728057 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f15c801a070 con 0x7f15d015a280 2024-09-16T15:43:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.359+0000 7f15c6a00640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15ac0789a0 0x7f15ac07ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.359+0000 7f15c6a00640 1 -- 172.21.15.60:0/140728057 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7f15c8099e30 con 0x7f15d015a280 2024-09-16T15:43:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.359+0000 7f15cde00640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15ac0789a0 0x7f15ac07ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.360+0000 7f15cde00640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15ac0789a0 0x7f15ac07ae60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006fd0 tx=0x7f15c0009060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:51.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.363+0000 7f15c6a00640 1 -- 172.21.15.60:0/140728057 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f15c8061ec0 con 0x7f15d015a280 2024-09-16T15:43:51.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:51.634+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f15d00cb870 con 0x7f15d015a280 2024-09-16T15:43:52.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.395+0000 7f15c6a00640 1 -- 172.21.15.60:0/140728057 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v297) ==== 152+0+0 (secure 0 0 0) 0x7f15d00cb870 con 0x7f15d015a280 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.398+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15ac0789a0 msgr2=0x7f15ac07ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.398+0000 7f15cf200640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15ac0789a0 0x7f15ac07ae60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006fd0 tx=0x7f15c0009060 comp rx=0 tx=0).stop 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.398+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d015a280 msgr2=0x7f15d014d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.398+0000 7f15cf200640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d015a280 0x7f15d014d8e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c770 tx=0x7f15c800cc30 comp rx=0 tx=0).stop 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.399+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 shutdown_connections 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.399+0000 7f15cf200640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15ac0789a0 0x7f15ac07ae60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.399+0000 7f15cf200640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15d015a280 0x7f15d014d8e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.399+0000 7f15cf200640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15d0072a40 0x7f15d00b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.399+0000 7f15cf200640 1 --2- 172.21.15.60:0/140728057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15d00725d0 0x7f15d01f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.399+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 >> 172.21.15.60:0/140728057 conn(0x7f15d00cfa20 msgr2=0x7f15d00b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:52.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.399+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 shutdown_connections 2024-09-16T15:43:52.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.399+0000 7f15cf200640 1 -- 172.21.15.60:0/140728057 wait complete. 2024-09-16T15:43:52.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:43:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.583+0000 7ffac6800640 1 -- 172.21.15.60:0/4122611559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40039e0 msgr2=0x7ffaa4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.583+0000 7ffac6800640 1 --2- 172.21.15.60:0/4122611559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40039e0 0x7ffaa4003dc0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009b50 tx=0x7ffaa4005830 comp rx=0 tx=0).stop 2024-09-16T15:43:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.583+0000 7ffabfe00640 1 -- 172.21.15.60:0/4122611559 reap_dead start 2024-09-16T15:43:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.583+0000 7ffac6800640 1 -- 172.21.15.60:0/4122611559 shutdown_connections 2024-09-16T15:43:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.583+0000 7ffac6800640 1 -- 172.21.15.60:0/4122611559 >> 172.21.15.60:0/4122611559 conn(0x7ffac00b1950 msgr2=0x7ffac00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.583+0000 7ffac6800640 1 -- 172.21.15.60:0/4122611559 shutdown_connections 2024-09-16T15:43:52.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.583+0000 7ffac6800640 1 -- 172.21.15.60:0/4122611559 wait complete. 2024-09-16T15:43:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.585+0000 7ffac6800640 1 Processor -- start 2024-09-16T15:43:52.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.585+0000 7ffac6800640 1 -- start start 2024-09-16T15:43:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffac6800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa40039e0 0x7ffac014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffac6800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac0158af0 0x7ffac014bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffac6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac01490b0 0x7ffac01496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:52.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffac6800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffac006f9f0 con 0x7ffac01490b0 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffac6800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffac006f870 con 0x7ffac0158af0 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffac6800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffac00b2440 con 0x7ffaa40039e0 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffabfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac01490b0 0x7ffac01496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffabfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac01490b0 0x7ffac01496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58840/0 (socket says 172.21.15.60:58840) 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffabfe00640 1 -- 172.21.15.60:0/549539723 learned_addr learned my addr 172.21.15.60:0/549539723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffabf400640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa40039e0 0x7ffac014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.586+0000 7ffab6a00640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac0158af0 0x7ffac014bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffabfe00640 1 -- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa40039e0 msgr2=0x7ffac014b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffabfe00640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa40039e0 0x7ffac014b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffabfe00640 1 -- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac0158af0 msgr2=0x7ffac014bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffabfe00640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac0158af0 0x7ffac014bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffabfe00640 1 -- 172.21.15.60:0/549539723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac0158050 con 0x7ffac01490b0 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffab6a00640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac0158af0 0x7ffac014bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffabfe00640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac01490b0 0x7ffac01496b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ffab8006900 tx=0x7ffab8001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:52.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffab7e00640 1 -- 172.21.15.60:0/549539723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8011870 con 0x7ffac01490b0 2024-09-16T15:43:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffab7e00640 1 -- 172.21.15.60:0/549539723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffab8017070 con 0x7ffac01490b0 2024-09-16T15:43:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffab7e00640 1 -- 172.21.15.60:0/549539723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8011da0 con 0x7ffac01490b0 2024-09-16T15:43:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac00c9620 con 0x7ffac01490b0 2024-09-16T15:43:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.587+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac01582f0 con 0x7ffac01490b0 2024-09-16T15:43:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.589+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac0157e80 con 0x7ffac01490b0 2024-09-16T15:43:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.590+0000 7ffab7e00640 1 -- 172.21.15.60:0/549539723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7ffab801c460 con 0x7ffac01490b0 2024-09-16T15:43:52.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.590+0000 7ffab7e00640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa9c078970 0x7ffa9c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.591+0000 7ffab7e00640 1 -- 172.21.15.60:0/549539723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7ffab809b1b0 con 0x7ffac01490b0 2024-09-16T15:43:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.591+0000 7ffabf400640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa9c078970 0x7ffa9c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.592+0000 7ffabf400640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa9c078970 0x7ffa9c07ae30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffab0035000 tx=0x7ffab002d040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:52.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.595+0000 7ffab7e00640 1 -- 172.21.15.60:0/549539723 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffab8063800 con 0x7ffac01490b0 2024-09-16T15:43:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.791+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ffac015d920 con 0x7ffac01490b0 2024-09-16T15:43:52.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.791+0000 7ffab7e00640 1 -- 172.21.15.60:0/549539723 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+1998 (secure 0 0 0) 0x7ffab8063100 con 0x7ffac01490b0 2024-09-16T15:43:52.792 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:43:52.792 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:43:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:43:52.794 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:43:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa9c078970 msgr2=0x7ffa9c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa9c078970 0x7ffa9c07ae30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffab0035000 tx=0x7ffab002d040 comp rx=0 tx=0).stop 2024-09-16T15:43:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac01490b0 msgr2=0x7ffac01496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac01490b0 0x7ffac01496b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ffab8006900 tx=0x7ffab8001ab0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 shutdown_connections 2024-09-16T15:43:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa9c078970 0x7ffa9c07ae30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffac01490b0 0x7ffac01496b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffac0158af0 0x7ffac014bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 --2- 172.21.15.60:0/549539723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa40039e0 0x7ffac014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.793+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 >> 172.21.15.60:0/549539723 conn(0x7ffac00b1950 msgr2=0x7ffac00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.794+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 shutdown_connections 2024-09-16T15:43:52.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.794+0000 7ffac6800640 1 -- 172.21.15.60:0/549539723 wait complete. 2024-09-16T15:43:52.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:43:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.974+0000 7f3b79a00640 1 -- 172.21.15.60:0/4146973424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b740cec60 msgr2=0x7f3b740cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.974+0000 7f3b79a00640 1 --2- 172.21.15.60:0/4146973424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b740cec60 0x7f3b740cf070 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c005fd0 tx=0x7f3b6c02a930 comp rx=0 tx=0).stop 2024-09-16T15:43:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.974+0000 7f3b79a00640 1 -- 172.21.15.60:0/4146973424 shutdown_connections 2024-09-16T15:43:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.974+0000 7f3b79a00640 1 --2- 172.21.15.60:0/4146973424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b740cec60 0x7f3b740cf070 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.974+0000 7f3b79a00640 1 -- 172.21.15.60:0/4146973424 >> 172.21.15.60:0/4146973424 conn(0x7f3b740b1840 msgr2=0x7f3b740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:52.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.974+0000 7f3b79a00640 1 -- 172.21.15.60:0/4146973424 shutdown_connections 2024-09-16T15:43:52.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.974+0000 7f3b79a00640 1 -- 172.21.15.60:0/4146973424 wait complete. 2024-09-16T15:43:52.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.975+0000 7f3b79a00640 1 Processor -- start 2024-09-16T15:43:52.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.976+0000 7f3b79a00640 1 -- start start 2024-09-16T15:43:52.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.976+0000 7f3b79a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b740af190 0x7f3b74160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:52.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b79a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b740cec60 0x7f3b74072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b79a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b74161760 0x7f3b741510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b79a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b741623a0 con 0x7f3b74161760 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b79a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3b74070530 con 0x7f3b740cec60 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b79a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3b74162520 con 0x7f3b740af190 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b73400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b74161760 0x7f3b741510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b73400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b74161760 0x7f3b741510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58852/0 (socket says 172.21.15.60:58852) 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b73400640 1 -- 172.21.15.60:0/2476776321 learned_addr learned my addr 172.21.15.60:0/2476776321 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b6a000640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b740cec60 0x7f3b74072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b72a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b740af190 0x7f3b74160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b73400640 1 -- 172.21.15.60:0/2476776321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b740af190 msgr2=0x7f3b74160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b73400640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b740af190 0x7f3b74160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b73400640 1 -- 172.21.15.60:0/2476776321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b740cec60 msgr2=0x7f3b74072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b73400640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b740cec60 0x7f3b74072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.977+0000 7f3b73400640 1 -- 172.21.15.60:0/2476776321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b741f4680 con 0x7f3b74161760 2024-09-16T15:43:52.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.978+0000 7f3b73400640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b74161760 0x7f3b741510c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c006010 tx=0x7f3b6c0305f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.978+0000 7f3b6b400640 1 -- 172.21.15.60:0/2476776321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c003bf0 con 0x7f3b74161760 2024-09-16T15:43:52.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.978+0000 7f3b6b400640 1 -- 172.21.15.60:0/2476776321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b6c003d90 con 0x7f3b74161760 2024-09-16T15:43:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.978+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b741f4960 con 0x7f3b74161760 2024-09-16T15:43:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.978+0000 7f3b6b400640 1 -- 172.21.15.60:0/2476776321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0042a0 con 0x7f3b74161760 2024-09-16T15:43:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.978+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b741f4f20 con 0x7f3b74161760 2024-09-16T15:43:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.979+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b741629d0 con 0x7f3b74161760 2024-09-16T15:43:52.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.980+0000 7f3b6b400640 1 -- 172.21.15.60:0/2476776321 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f3b6c004440 con 0x7f3b74161760 2024-09-16T15:43:52.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.981+0000 7f3b6b400640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b50078970 0x7f3b5007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:52.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.981+0000 7f3b6b400640 1 -- 172.21.15.60:0/2476776321 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7f3b6c0bc0b0 con 0x7f3b74161760 2024-09-16T15:43:52.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.982+0000 7f3b72a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b50078970 0x7f3b5007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:52.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.982+0000 7f3b72a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b50078970 0x7f3b5007ae30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3b64007fd0 tx=0x7f3b64007b00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:52.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:52.985+0000 7f3b6b400640 1 -- 172.21.15.60:0/2476776321 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b6c084140 con 0x7f3b74161760 2024-09-16T15:43:53.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.184+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3b74151ec0 con 0x7f3b74161760 2024-09-16T15:43:53.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.188+0000 7f3b6b400640 1 -- 172.21.15.60:0/2476776321 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f3b6c088b70 con 0x7f3b74161760 2024-09-16T15:43:53.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b50078970 msgr2=0x7f3b5007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:53.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b50078970 0x7f3b5007ae30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3b64007fd0 tx=0x7f3b64007b00 comp rx=0 tx=0).stop 2024-09-16T15:43:53.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b74161760 msgr2=0x7f3b741510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:53.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b74161760 0x7f3b741510c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c006010 tx=0x7f3b6c0305f0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 shutdown_connections 2024-09-16T15:43:53.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b50078970 0x7f3b5007ae30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b74161760 0x7f3b741510c0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b740cec60 0x7f3b74072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 --2- 172.21.15.60:0/2476776321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b740af190 0x7f3b74160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 >> 172.21.15.60:0/2476776321 conn(0x7f3b740b1840 msgr2=0x7f3b740cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:53.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 shutdown_connections 2024-09-16T15:43:53.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.191+0000 7f3b79a00640 1 -- 172.21.15.60:0/2476776321 wait complete. 2024-09-16T15:43:53.209 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:43:53.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:43:53.304 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:43:53.388 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:43:53.388 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:53.389 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:53.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:43:53.424 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:53.424 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:53.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:43:53.461 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:43:53.461 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:53.501 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:43:53.501 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:43:53.501 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:53.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:43:53.537 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:43:53.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:43:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.701+0000 7f1b65a00640 1 -- 172.21.15.60:0/3466880151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 msgr2=0x7f1b600b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.701+0000 7f1b65a00640 1 --2- 172.21.15.60:0/3466880151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 0x7f1b600b2440 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f1b50005fd0 tx=0x7f1b5002a540 comp rx=0 tx=0).stop 2024-09-16T15:43:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.701+0000 7f1b65a00640 1 -- 172.21.15.60:0/3466880151 shutdown_connections 2024-09-16T15:43:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.701+0000 7f1b65a00640 1 --2- 172.21.15.60:0/3466880151 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b60073fb0 0x7f1b60074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.701+0000 7f1b65a00640 1 --2- 172.21.15.60:0/3466880151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 0x7f1b600b2440 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.701+0000 7f1b65a00640 1 -- 172.21.15.60:0/3466880151 >> 172.21.15.60:0/3466880151 conn(0x7f1b600b1950 msgr2=0x7f1b600b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.701+0000 7f1b65a00640 1 -- 172.21.15.60:0/3466880151 shutdown_connections 2024-09-16T15:43:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.701+0000 7f1b65a00640 1 -- 172.21.15.60:0/3466880151 wait complete. 2024-09-16T15:43:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.702+0000 7f1b65a00640 1 Processor -- start 2024-09-16T15:43:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.703+0000 7f1b65a00640 1 -- start start 2024-09-16T15:43:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.703+0000 7f1b65a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b60073fb0 0x7f1b6015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.703+0000 7f1b65a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b600ccf00 0x7f1b60156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.703+0000 7f1b65a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 0x7f1b60156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.703+0000 7f1b65a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6006f2e0 con 0x7f1b6015f5a0 2024-09-16T15:43:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.703+0000 7f1b65a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6006f160 con 0x7f1b600ccf00 2024-09-16T15:43:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.703+0000 7f1b65a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6006f460 con 0x7f1b60073fb0 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 0x7f1b60156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5ea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b60073fb0 0x7f1b6015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 0x7f1b60156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58868/0 (socket says 172.21.15.60:58868) 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 -- 172.21.15.60:0/517767929 learned_addr learned my addr 172.21.15.60:0/517767929 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b56000640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b600ccf00 0x7f1b60156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 -- 172.21.15.60:0/517767929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b60073fb0 msgr2=0x7f1b6015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b60073fb0 0x7f1b6015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 -- 172.21.15.60:0/517767929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b600ccf00 msgr2=0x7f1b60156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b600ccf00 0x7f1b60156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 -- 172.21.15.60:0/517767929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6015e540 con 0x7f1b6015f5a0 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.704+0000 7f1b5f400640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 0x7f1b60156ec0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800eaa0 tx=0x7f1b5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.705+0000 7f1b57400640 1 -- 172.21.15.60:0/517767929 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800cdb0 con 0x7f1b6015f5a0 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.705+0000 7f1b57400640 1 -- 172.21.15.60:0/517767929 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b58004c10 con 0x7f1b6015f5a0 2024-09-16T15:43:53.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.705+0000 7f1b57400640 1 -- 172.21.15.60:0/517767929 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58005020 con 0x7f1b6015f5a0 2024-09-16T15:43:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.705+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b600cb000 con 0x7f1b6015f5a0 2024-09-16T15:43:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.705+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b600cb5c0 con 0x7f1b6015f5a0 2024-09-16T15:43:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.706+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6009df30 con 0x7f1b6015f5a0 2024-09-16T15:43:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.707+0000 7f1b57400640 1 -- 172.21.15.60:0/517767929 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f1b580051c0 con 0x7f1b6015f5a0 2024-09-16T15:43:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.708+0000 7f1b57400640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b3c078970 0x7f1b3c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.708+0000 7f1b57400640 1 -- 172.21.15.60:0/517767929 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7f1b5809b210 con 0x7f1b6015f5a0 2024-09-16T15:43:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.708+0000 7f1b5ea00640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b3c078970 0x7f1b3c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:53.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.709+0000 7f1b5ea00640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b3c078970 0x7f1b3c07ae30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b50033000 tx=0x7f1b50004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:53.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.712+0000 7f1b57400640 1 -- 172.21.15.60:0/517767929 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1b58016030 con 0x7f1b6015f5a0 2024-09-16T15:43:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.944+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b6009dc50 con 0x7f1b6015f5a0 2024-09-16T15:43:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.945+0000 7f1b57400640 1 -- 172.21.15.60:0/517767929 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+3917 (secure 0 0 0) 0x7f1b580632a0 con 0x7f1b6015f5a0 2024-09-16T15:43:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:53.946 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":64,"btime":"2024-09-16T15:43:47:371379+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1388326737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1388326737},{"type":"v1","addr":"172.21.15.100:6835","nonce":1388326737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6382,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/2485723644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2485723644},{"type":"v1","addr":"172.21.15.100:6801","nonce":2485723644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2204286380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2204286380},{"type":"v1","addr":"172.21.15.60:6835","nonce":2204286380}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3670782349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3670782349},{"type":"v1","addr":"172.21.15.60:6834","nonce":3670782349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[]} 2024-09-16T15:43:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 64 2024-09-16T15:43:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b3c078970 msgr2=0x7f1b3c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b3c078970 0x7f1b3c07ae30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b50033000 tx=0x7f1b50004730 comp rx=0 tx=0).stop 2024-09-16T15:43:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 msgr2=0x7f1b60156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:53.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 0x7f1b60156ec0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800eaa0 tx=0x7f1b5800ef60 comp rx=0 tx=0).stop 2024-09-16T15:43:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 shutdown_connections 2024-09-16T15:43:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b3c078970 0x7f1b3c07ae30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b6015f5a0 0x7f1b60156ec0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b600ccf00 0x7f1b60156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 --2- 172.21.15.60:0/517767929 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b60073fb0 0x7f1b6015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:53.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 >> 172.21.15.60:0/517767929 conn(0x7f1b600b1950 msgr2=0x7f1b600cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:53.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 shutdown_connections 2024-09-16T15:43:53.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:53.947+0000 7f1b65a00640 1 -- 172.21.15.60:0/517767929 wait complete. 2024-09-16T15:43:53.964 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:43:53.964 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:43:53.964 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:43:53.965 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:43:53.965 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:43:53.965 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:43:53.965 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:43:53.965 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:43:53.965 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:43:53.965 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:43:53.966 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:43:53.966 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:43:53.966 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:43:53.966 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:43:53.966 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:43:53.966 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:43:53.980 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:43:53.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:43:53.981 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:43:53.981 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:43:53.982 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:43:53.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:43:53.983 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:43:53.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:43:54.025 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:43:54.025 DEBUG:teuthology.parallel:result is None 2024-09-16T15:43:54.026 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:43:54.026 DEBUG:teuthology.parallel:result is None 2024-09-16T15:43:54.027 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:43:54.027 DEBUG:teuthology.parallel:result is None 2024-09-16T15:43:54.028 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:43:54.028 DEBUG:teuthology.parallel:result is None 2024-09-16T15:43:54.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:43:54.153 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:43:54.153 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:43:54.159 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:43:54.160 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:43:54.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.196+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2658953981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc0ccf00 msgr2=0x7fa9dc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:54.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.196+0000 7fa9e2c00640 1 --2- 172.21.15.60:0/2658953981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc0ccf00 0x7fa9dc0b2440 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009830 tx=0x7fa9c802a530 comp rx=0 tx=0).stop 2024-09-16T15:43:54.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.196+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2658953981 shutdown_connections 2024-09-16T15:43:54.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.196+0000 7fa9e2c00640 1 --2- 172.21.15.60:0/2658953981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc0ccf00 0x7fa9dc0b2440 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:54.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.196+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2658953981 >> 172.21.15.60:0/2658953981 conn(0x7fa9dc0b1950 msgr2=0x7fa9dc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:54.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.196+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2658953981 shutdown_connections 2024-09-16T15:43:54.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.196+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2658953981 wait complete. 2024-09-16T15:43:54.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.198+0000 7fa9e2c00640 1 Processor -- start 2024-09-16T15:43:54.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.198+0000 7fa9e2c00640 1 -- start start 2024-09-16T15:43:54.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9e2c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc073fb0 0x7fa9dc1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:54.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9e2c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc0ccf00 0x7fa9dc15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:54.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9e2c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9dc15f5a0 0x7fa9dc0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:54.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9e2c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc06f9f0 con 0x7fa9dc073fb0 2024-09-16T15:43:54.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9e2c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc06f870 con 0x7fa9dc0ccf00 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9e2c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa9dc074420 con 0x7fa9dc15f5a0 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9dbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc073fb0 0x7fa9dc1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9dbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc073fb0 0x7fa9dc1520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58916/0 (socket says 172.21.15.60:58916) 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9dbe00640 1 -- 172.21.15.60:0/2584738622 learned_addr learned my addr 172.21.15.60:0/2584738622 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9e0c00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9dc15f5a0 0x7fa9dc0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9db400640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc0ccf00 0x7fa9dc15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9dbe00640 1 -- 172.21.15.60:0/2584738622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9dc15f5a0 msgr2=0x7fa9dc0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9dbe00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9dc15f5a0 0x7fa9dc0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9dbe00640 1 -- 172.21.15.60:0/2584738622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc0ccf00 msgr2=0x7fa9dc15c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9dbe00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc0ccf00 0x7fa9dc15c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.199+0000 7fa9dbe00640 1 -- 172.21.15.60:0/2584738622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc15c960 con 0x7fa9dc073fb0 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.200+0000 7fa9dbe00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc073fb0 0x7fa9dc1520e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00cbc0 tx=0x7fa9cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.200+0000 7fa9d3e00640 1 -- 172.21.15.60:0/2584738622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc016020 con 0x7fa9dc073fb0 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.200+0000 7fa9d3e00640 1 -- 172.21.15.60:0/2584738622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9cc00add0 con 0x7fa9dc073fb0 2024-09-16T15:43:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.200+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc1526c0 con 0x7fa9dc073fb0 2024-09-16T15:43:54.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.200+0000 7fa9d3e00640 1 -- 172.21.15.60:0/2584738622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc012680 con 0x7fa9dc073fb0 2024-09-16T15:43:54.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.200+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc152c80 con 0x7fa9dc073fb0 2024-09-16T15:43:54.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.201+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9dc0b2440 con 0x7fa9dc073fb0 2024-09-16T15:43:54.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.202+0000 7fa9d3e00640 1 -- 172.21.15.60:0/2584738622 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fa9cc019020 con 0x7fa9dc073fb0 2024-09-16T15:43:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.204+0000 7fa9d3e00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa9bc078970 0x7fa9bc07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.204+0000 7fa9d3e00640 1 -- 172.21.15.60:0/2584738622 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5029+0+0 (secure 0 0 0) 0x7fa9cc09abb0 con 0x7fa9dc073fb0 2024-09-16T15:43:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.204+0000 7fa9db400640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa9bc078970 0x7fa9bc07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:54.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.205+0000 7fa9db400640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa9bc078970 0x7fa9bc07ae30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8034000 tx=0x7fa9c8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.208+0000 7fa9d3e00640 1 -- 172.21.15.60:0/2584738622 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa9cc020030 con 0x7fa9dc073fb0 2024-09-16T15:43:54.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:54.419+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa9dc0b20f0 con 0x7fa9dc073fb0 2024-09-16T15:43:55.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.431+0000 7fa9d3e00640 1 -- 172.21.15.60:0/2584738622 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) ==== 97+0+0 (secure 0 0 0) 0x7fa9cc062bc0 con 0x7fa9dc073fb0 2024-09-16T15:43:55.432 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:43:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.434+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa9bc078970 msgr2=0x7fa9bc07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.434+0000 7fa9e2c00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa9bc078970 0x7fa9bc07ae30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8034000 tx=0x7fa9c8004730 comp rx=0 tx=0).stop 2024-09-16T15:43:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.434+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc073fb0 msgr2=0x7fa9dc1520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.434+0000 7fa9e2c00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc073fb0 0x7fa9dc1520e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00cbc0 tx=0x7fa9cc00a430 comp rx=0 tx=0).stop 2024-09-16T15:43:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.435+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 shutdown_connections 2024-09-16T15:43:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.435+0000 7fa9e2c00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa9bc078970 0x7fa9bc07ae30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.435+0000 7fa9e2c00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9dc15f5a0 0x7fa9dc0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.435+0000 7fa9e2c00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9dc0ccf00 0x7fa9dc15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.435+0000 7fa9e2c00640 1 --2- 172.21.15.60:0/2584738622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9dc073fb0 0x7fa9dc1520e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.435+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 >> 172.21.15.60:0/2584738622 conn(0x7fa9dc0b1950 msgr2=0x7fa9dc15dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.435+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 shutdown_connections 2024-09-16T15:43:55.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.435+0000 7fa9e2c00640 1 -- 172.21.15.60:0/2584738622 wait complete. 2024-09-16T15:43:55.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:43:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.619+0000 7f2ea9e00640 1 -- 172.21.15.60:0/3198161696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 msgr2=0x7f2ea40d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.619+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/3198161696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 0x7f2ea40d21a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40c7390 tx=0x7f2e9c02aac0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.619+0000 7f2ea9e00640 1 -- 172.21.15.60:0/3198161696 shutdown_connections 2024-09-16T15:43:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.619+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/3198161696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 0x7f2ea40d21a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.619+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/3198161696 >> v2:172.21.15.100:3300/0 conn(0x7f2ea40d4980 0x7f2ea40d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.619+0000 7f2ea9e00640 1 -- 172.21.15.60:0/3198161696 >> 172.21.15.60:0/3198161696 conn(0x7f2ea40d3df0 msgr2=0x7f2ea40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:55.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.619+0000 7f2ea9e00640 1 -- 172.21.15.60:0/3198161696 shutdown_connections 2024-09-16T15:43:55.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.619+0000 7f2ea9e00640 1 -- 172.21.15.60:0/3198161696 wait complete. 2024-09-16T15:43:55.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.620+0000 7f2ea9e00640 1 Processor -- start 2024-09-16T15:43:55.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.621+0000 7f2ea9e00640 1 -- start start 2024-09-16T15:43:55.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea40d4980 0x7f2ea415cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:55.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea41612d0 0x7f2ea415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:55.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 0x7f2ea40cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:55.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea40b1bb0 con 0x7f2ea41616b0 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea40b1a30 con 0x7f2ea40d4980 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea40b1d30 con 0x7f2ea41612d0 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 0x7f2ea40cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 0x7f2ea40cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58932/0 (socket says 172.21.15.60:58932) 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9400640 1 -- 172.21.15.60:0/1392017704 learned_addr learned my addr 172.21.15.60:0/1392017704 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea3e00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea41612d0 0x7f2ea415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea8a00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea40d4980 0x7f2ea415cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9400640 1 -- 172.21.15.60:0/1392017704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea41612d0 msgr2=0x7f2ea415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9400640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea41612d0 0x7f2ea415d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9400640 1 -- 172.21.15.60:0/1392017704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea40d4980 msgr2=0x7f2ea415cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9400640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea40d4980 0x7f2ea415cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.622+0000 7f2ea9400640 1 -- 172.21.15.60:0/1392017704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea40ca740 con 0x7f2ea41616b0 2024-09-16T15:43:55.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.623+0000 7f2ea9400640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 0x7f2ea40cb540 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009d60 tx=0x7f2e9c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:55.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.623+0000 7f2ea0c00640 1 -- 172.21.15.60:0/1392017704 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c044070 con 0x7f2ea41616b0 2024-09-16T15:43:55.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.623+0000 7f2ea0c00640 1 -- 172.21.15.60:0/1392017704 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e9c004250 con 0x7f2ea41616b0 2024-09-16T15:43:55.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.623+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea40b8c10 con 0x7f2ea41616b0 2024-09-16T15:43:55.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.623+0000 7f2ea0c00640 1 -- 172.21.15.60:0/1392017704 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c004680 con 0x7f2ea41616b0 2024-09-16T15:43:55.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.623+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41f4ea0 con 0x7f2ea41616b0 2024-09-16T15:43:55.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.624+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea40b2250 con 0x7f2ea41616b0 2024-09-16T15:43:55.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.625+0000 7f2ea0c00640 1 -- 172.21.15.60:0/1392017704 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f2e9c02c840 con 0x7f2ea41616b0 2024-09-16T15:43:55.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.626+0000 7f2ea0c00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2e84078970 0x7f2e8407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:55.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.626+0000 7f2ea0c00640 1 -- 172.21.15.60:0/1392017704 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4857+0+0 (secure 0 0 0) 0x7f2e9c0bae20 con 0x7f2ea41616b0 2024-09-16T15:43:55.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.626+0000 7f2ea8a00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2e84078970 0x7f2e8407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:55.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.627+0000 7f2ea8a00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2e84078970 0x7f2e8407ae30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40c85d0 tx=0x7f2e98009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:55.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.631+0000 7f2ea0c00640 1 -- 172.21.15.60:0/1392017704 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2e9c032030 con 0x7f2ea41616b0 2024-09-16T15:43:55.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.902+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2ea40b3500 con 0x7f2ea41616b0 2024-09-16T15:43:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.903+0000 7f2ea0c00640 1 -- 172.21.15.60:0/1392017704 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f2e9c004820 con 0x7f2ea41616b0 2024-09-16T15:43:55.904 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:55.904 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:43:55.904 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:43:55.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.904 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:55.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.907 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.909 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.911 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.913 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.914 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:43:55.916 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:43:55.917 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.918 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:55.918 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:43:55.918 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:43:55.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2e84078970 msgr2=0x7f2e8407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2e84078970 0x7f2e8407ae30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40c85d0 tx=0x7f2e98009040 comp rx=0 tx=0).stop 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 msgr2=0x7f2ea40cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 0x7f2ea40cb540 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009d60 tx=0x7f2e9c003f10 comp rx=0 tx=0).stop 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 shutdown_connections 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2e84078970 0x7f2e8407ae30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea41616b0 0x7f2ea40cb540 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea41612d0 0x7f2ea415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 --2- 172.21.15.60:0/1392017704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea40d4980 0x7f2ea415cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.905+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 >> 172.21.15.60:0/1392017704 conn(0x7f2ea40d3df0 msgr2=0x7f2ea40ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.906+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 shutdown_connections 2024-09-16T15:43:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:55.906+0000 7f2ea9e00640 1 -- 172.21.15.60:0/1392017704 wait complete. 2024-09-16T15:43:55.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:43:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 -- 172.21.15.60:0/2212218750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ccf00 msgr2=0x7f87bc0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 --2- 172.21.15.60:0/2212218750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ccf00 0x7f87bc0acd30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f87bc075130 tx=0x7f87a402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 -- 172.21.15.60:0/2212218750 shutdown_connections 2024-09-16T15:43:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 --2- 172.21.15.60:0/2212218750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0ad270 0x7f87bc073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 --2- 172.21.15.60:0/2212218750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ccf00 0x7f87bc0acd30 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 --2- 172.21.15.60:0/2212218750 >> v2:172.21.15.100:3300/0 conn(0x7f87bc15f5a0 0x7f87bc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 -- 172.21.15.60:0/2212218750 >> 172.21.15.60:0/2212218750 conn(0x7f87bc0b1950 msgr2=0x7f87bc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 -- 172.21.15.60:0/2212218750 shutdown_connections 2024-09-16T15:43:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.081+0000 7f87c3600640 1 -- 172.21.15.60:0/2212218750 wait complete. 2024-09-16T15:43:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.083+0000 7f87c3600640 1 Processor -- start 2024-09-16T15:43:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.083+0000 7f87c3600640 1 -- start start 2024-09-16T15:43:56.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c3600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad270 0x7f87bc0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:56.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c3600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0ccf00 0x7f87bc1f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:56.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c3600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15f5a0 0x7f87bc0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:56.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c3600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc074880 con 0x7f87bc0ad270 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c3600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc074700 con 0x7f87bc15f5a0 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c3600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f87bc06f160 con 0x7f87bc0ccf00 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c0c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad270 0x7f87bc0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c0c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad270 0x7f87bc0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58948/0 (socket says 172.21.15.60:58948) 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c0c00640 1 -- 172.21.15.60:0/2202381554 learned_addr learned my addr 172.21.15.60:0/2202381554 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87b3e00640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0ccf00 0x7f87bc1f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.084+0000 7f87c1600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15f5a0 0x7f87bc0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c0c00640 1 -- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0ccf00 msgr2=0x7f87bc1f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c0c00640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0ccf00 0x7f87bc1f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c0c00640 1 -- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15f5a0 msgr2=0x7f87bc0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c0c00640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15f5a0 0x7f87bc0cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c0c00640 1 -- 172.21.15.60:0/2202381554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc0b39a0 con 0x7f87bc0ad270 2024-09-16T15:43:56.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c1600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15f5a0 0x7f87bc0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c0c00640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad270 0x7f87bc0b33c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00e7c0 tx=0x7f87ac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87b9600640 1 -- 172.21.15.60:0/2202381554 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00ccc0 con 0x7f87bc0ad270 2024-09-16T15:43:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87b9600640 1 -- 172.21.15.60:0/2202381554 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87ac010430 con 0x7f87bc0ad270 2024-09-16T15:43:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87b9600640 1 -- 172.21.15.60:0/2202381554 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac010860 con 0x7f87bc0ad270 2024-09-16T15:43:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87bc0b3c80 con 0x7f87bc0ad270 2024-09-16T15:43:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.085+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87bc0b4240 con 0x7f87bc0ad270 2024-09-16T15:43:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.086+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87bc073fb0 con 0x7f87bc0ad270 2024-09-16T15:43:56.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.087+0000 7f87b9600640 1 -- 172.21.15.60:0/2202381554 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f87ac010a00 con 0x7f87bc0ad270 2024-09-16T15:43:56.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.089+0000 7f87b9600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8798078970 0x7f879807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:56.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.089+0000 7f87b9600640 1 -- 172.21.15.60:0/2202381554 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4857+0+0 (secure 0 0 0) 0x7f87ac09b310 con 0x7f87bc0ad270 2024-09-16T15:43:56.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.089+0000 7f87b3e00640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8798078970 0x7f879807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:56.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.090+0000 7f87b3e00640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8798078970 0x7f879807ae30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f87a4033000 tx=0x7f87a4004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:56.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.093+0000 7f87b9600640 1 -- 172.21.15.60:0/2202381554 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f87ac063990 con 0x7f87bc0ad270 2024-09-16T15:43:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.289+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f87bc0b13c0 con 0x7f87bc0ad270 2024-09-16T15:43:56.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.290+0000 7f87b9600640 1 -- 172.21.15.60:0/2202381554 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) ==== 84+0+65 (secure 0 0 0) 0x7f87ac063290 con 0x7f87bc0ad270 2024-09-16T15:43:56.291 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:56.291 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:43:56.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:43:56.291 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:43:56.291 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:43:56.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:43:56.291 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:43:56.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8798078970 msgr2=0x7f879807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:56.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8798078970 0x7f879807ae30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f87a4033000 tx=0x7f87a4004730 comp rx=0 tx=0).stop 2024-09-16T15:43:56.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad270 msgr2=0x7f87bc0b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:56.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad270 0x7f87bc0b33c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00e7c0 tx=0x7f87ac00ec80 comp rx=0 tx=0).stop 2024-09-16T15:43:56.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 shutdown_connections 2024-09-16T15:43:56.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8798078970 0x7f879807ae30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87bc15f5a0 0x7f87bc0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87bc0ccf00 0x7f87bc1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 --2- 172.21.15.60:0/2202381554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87bc0ad270 0x7f87bc0b33c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 >> 172.21.15.60:0/2202381554 conn(0x7f87bc0b1950 msgr2=0x7f87bc15e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:56.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 shutdown_connections 2024-09-16T15:43:56.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.292+0000 7f87c3600640 1 -- 172.21.15.60:0/2202381554 wait complete. 2024-09-16T15:43:56.309 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:43:56.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:43:56.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.475+0000 7f8459c00640 1 -- 172.21.15.60:0/1875601471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845415cff0 msgr2=0x7f845406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:56.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.475+0000 7f8459c00640 1 --2- 172.21.15.60:0/1875601471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845415cff0 0x7f845406e6f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f843c005fd0 tx=0x7f843c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.475+0000 7f8459c00640 1 -- 172.21.15.60:0/1875601471 shutdown_connections 2024-09-16T15:43:56.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.475+0000 7f8459c00640 1 --2- 172.21.15.60:0/1875601471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845415cff0 0x7f845406e6f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.475+0000 7f8459c00640 1 -- 172.21.15.60:0/1875601471 >> 172.21.15.60:0/1875601471 conn(0x7f84540b18e0 msgr2=0x7f84540b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:56.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.476+0000 7f8459c00640 1 -- 172.21.15.60:0/1875601471 shutdown_connections 2024-09-16T15:43:56.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.476+0000 7f8459c00640 1 -- 172.21.15.60:0/1875601471 wait complete. 2024-09-16T15:43:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.477+0000 7f8459c00640 1 Processor -- start 2024-09-16T15:43:56.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.477+0000 7f8459c00640 1 -- start start 2024-09-16T15:43:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.478+0000 7f8459c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845406f220 0x7f845414f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.478+0000 7f8459c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8454159aa0 0x7f84540cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.478+0000 7f8459c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845415cff0 0x7f845415e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.478+0000 7f8459c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f84540ccc20 con 0x7f845406f220 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.478+0000 7f8459c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f845406f6b0 con 0x7f845415cff0 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.478+0000 7f8459c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8454157880 con 0x7f8454159aa0 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.478+0000 7f8452a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845406f220 0x7f845414f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8452a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845406f220 0x7f845414f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58974/0 (socket says 172.21.15.60:58974) 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8452a00640 1 -- 172.21.15.60:0/3302008025 learned_addr learned my addr 172.21.15.60:0/3302008025 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f844a000640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8454159aa0 0x7f84540cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8453400640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845415cff0 0x7f845415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8452a00640 1 -- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8454159aa0 msgr2=0x7f84540cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:56.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8452a00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8454159aa0 0x7f84540cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8452a00640 1 -- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845415cff0 msgr2=0x7f845415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8452a00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845415cff0 0x7f845415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8452a00640 1 -- 172.21.15.60:0/3302008025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84540cfae0 con 0x7f845406f220 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8453400640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845415cff0 0x7f845415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8452a00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845406f220 0x7f845414f290 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f844400c410 tx=0x7f844400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f844b400640 1 -- 172.21.15.60:0/3302008025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444017070 con 0x7f845406f220 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f844b400640 1 -- 172.21.15.60:0/3302008025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84440035f0 con 0x7f845406f220 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f844b400640 1 -- 172.21.15.60:0/3302008025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444003a80 con 0x7f845406f220 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.479+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845414f7d0 con 0x7f845406f220 2024-09-16T15:43:56.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.480+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8454157510 con 0x7f845406f220 2024-09-16T15:43:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.481+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8454069360 con 0x7f845406f220 2024-09-16T15:43:56.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.482+0000 7f844b400640 1 -- 172.21.15.60:0/3302008025 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f844401a070 con 0x7f845406f220 2024-09-16T15:43:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.483+0000 7f844b400640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f84300789a0 0x7f843007ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.483+0000 7f844b400640 1 -- 172.21.15.60:0/3302008025 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4857+0+0 (secure 0 0 0) 0x7f8444099b50 con 0x7f845406f220 2024-09-16T15:43:56.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.483+0000 7f844a000640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f84300789a0 0x7f843007ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:56.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.484+0000 7f844a000640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f84300789a0 0x7f843007ae60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f843c033000 tx=0x7f843c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:56.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.487+0000 7f844b400640 1 -- 172.21.15.60:0/3302008025 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f84440010c0 con 0x7f845406f220 2024-09-16T15:43:56.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:56.683+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f845406e6f0 con 0x7f845406f220 2024-09-16T15:43:57.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.441+0000 7f844b400640 1 -- 172.21.15.60:0/3302008025 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) ==== 138+0+0 (secure 0 0 0) 0x7f8444061c00 con 0x7f845406f220 2024-09-16T15:43:57.442 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f84300789a0 msgr2=0x7f843007ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f84300789a0 0x7f843007ae60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f843c033000 tx=0x7f843c004730 comp rx=0 tx=0).stop 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845406f220 msgr2=0x7f845414f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845406f220 0x7f845414f290 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f844400c410 tx=0x7f844400c8d0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 shutdown_connections 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f84300789a0 0x7f843007ae60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f845415cff0 0x7f845415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8454159aa0 0x7f84540cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 --2- 172.21.15.60:0/3302008025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f845406f220 0x7f845414f290 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 >> 172.21.15.60:0/3302008025 conn(0x7f84540b18e0 msgr2=0x7f845415c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:57.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.445+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 shutdown_connections 2024-09-16T15:43:57.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.446+0000 7f8459c00640 1 -- 172.21.15.60:0/3302008025 wait complete. 2024-09-16T15:43:57.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:43:57.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.642+0000 7f2872e00640 1 -- 172.21.15.60:0/2153080584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c145750 msgr2=0x7f286c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:57.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.642+0000 7f2872e00640 1 --2- 172.21.15.60:0/2153080584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c145750 0x7f286c073440 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f285c005fd0 tx=0x7f285c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:43:57.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.642+0000 7f2872e00640 1 -- 172.21.15.60:0/2153080584 shutdown_connections 2024-09-16T15:43:57.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.642+0000 7f2872e00640 1 --2- 172.21.15.60:0/2153080584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c06efa0 0x7f286c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.642+0000 7f2872e00640 1 --2- 172.21.15.60:0/2153080584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c145750 0x7f286c073440 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.642+0000 7f2872e00640 1 -- 172.21.15.60:0/2153080584 >> 172.21.15.60:0/2153080584 conn(0x7f286c0b1840 msgr2=0x7f286c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:57.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.642+0000 7f2872e00640 1 -- 172.21.15.60:0/2153080584 shutdown_connections 2024-09-16T15:43:57.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.642+0000 7f2872e00640 1 -- 172.21.15.60:0/2153080584 wait complete. 2024-09-16T15:43:57.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.644+0000 7f2872e00640 1 Processor -- start 2024-09-16T15:43:57.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.644+0000 7f2872e00640 1 -- start start 2024-09-16T15:43:57.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.644+0000 7f2872e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c06efa0 0x7f286c0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:57.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2872e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ccb00 0x7f286c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:57.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2872e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c145750 0x7f286c1f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2872e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0b26b0 con 0x7f286c0ccb00 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2872e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f286c0cedd0 con 0x7f286c145750 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2872e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f286c160100 con 0x7f286c06efa0 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2863400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ccb00 0x7f286c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2863400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ccb00 0x7f286c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58998/0 (socket says 172.21.15.60:58998) 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2863400640 1 -- 172.21.15.60:0/1732408974 learned_addr learned my addr 172.21.15.60:0/1732408974 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f286be00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c06efa0 0x7f286c0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2870e00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c145750 0x7f286c1f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2863400640 1 -- 172.21.15.60:0/1732408974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c06efa0 msgr2=0x7f286c0cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2863400640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c06efa0 0x7f286c0cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2863400640 1 -- 172.21.15.60:0/1732408974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c145750 msgr2=0x7f286c1f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2863400640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c145750 0x7f286c1f48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:57.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.645+0000 7f2863400640 1 -- 172.21.15.60:0/1732408974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c0c9920 con 0x7f286c0ccb00 2024-09-16T15:43:57.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.646+0000 7f2863400640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ccb00 0x7f286c0cbc00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f285400d950 tx=0x7f285400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:57.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.646+0000 7f2868c00640 1 -- 172.21.15.60:0/1732408974 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854017070 con 0x7f286c0ccb00 2024-09-16T15:43:57.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.646+0000 7f2868c00640 1 -- 172.21.15.60:0/1732408974 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2854009d20 con 0x7f286c0ccb00 2024-09-16T15:43:57.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.646+0000 7f2868c00640 1 -- 172.21.15.60:0/1732408974 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854004280 con 0x7f286c0ccb00 2024-09-16T15:43:57.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.646+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c0c9c00 con 0x7f286c0ccb00 2024-09-16T15:43:57.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.646+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c0ca1c0 con 0x7f286c0ccb00 2024-09-16T15:43:57.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.647+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c1605b0 con 0x7f286c0ccb00 2024-09-16T15:43:57.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.650+0000 7f2868c00640 1 -- 172.21.15.60:0/1732408974 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f2854005020 con 0x7f286c0ccb00 2024-09-16T15:43:57.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.651+0000 7f2868c00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2848078970 0x7f284807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:57.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.651+0000 7f2868c00640 1 -- 172.21.15.60:0/1732408974 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5227+0+0 (secure 0 0 0) 0x7f285409af60 con 0x7f286c0ccb00 2024-09-16T15:43:57.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.651+0000 7f286be00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2848078970 0x7f284807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:57.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.652+0000 7f286be00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2848078970 0x7f284807ae30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f285c035000 tx=0x7f285c03b040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:57.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.654+0000 7f2868c00640 1 -- 172.21.15.60:0/1732408974 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2854062f20 con 0x7f286c0ccb00 2024-09-16T15:43:57.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:57.851+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f286c0b0530 con 0x7f286c0ccb00 2024-09-16T15:43:58.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.448+0000 7f2868c00640 1 -- 172.21.15.60:0/1732408974 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) ==== 170+0+0 (secure 0 0 0) 0x7f2854067950 con 0x7f286c0ccb00 2024-09-16T15:43:58.449 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:43:58.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2848078970 msgr2=0x7f284807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:58.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2848078970 0x7f284807ae30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f285c035000 tx=0x7f285c03b040 comp rx=0 tx=0).stop 2024-09-16T15:43:58.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ccb00 msgr2=0x7f286c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:58.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ccb00 0x7f286c0cbc00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f285400d950 tx=0x7f285400de10 comp rx=0 tx=0).stop 2024-09-16T15:43:58.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 shutdown_connections 2024-09-16T15:43:58.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2848078970 0x7f284807ae30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286c145750 0x7f286c1f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f286c0ccb00 0x7f286c0cbc00 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 --2- 172.21.15.60:0/1732408974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286c06efa0 0x7f286c0cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 >> 172.21.15.60:0/1732408974 conn(0x7f286c0b1840 msgr2=0x7f286c0b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:58.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 shutdown_connections 2024-09-16T15:43:58.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.452+0000 7f2872e00640 1 -- 172.21.15.60:0/1732408974 wait complete. 2024-09-16T15:43:58.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:43:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.655+0000 7fb0a9c00640 1 -- 172.21.15.60:0/441263957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a40b1880 msgr2=0x7fb0a40688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.655+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/441263957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a40b1880 0x7fb0a40688b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb094005fd0 tx=0x7fb09402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.655+0000 7fb0a9c00640 1 -- 172.21.15.60:0/441263957 shutdown_connections 2024-09-16T15:43:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.655+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/441263957 >> v2:172.21.15.100:3300/0 conn(0x7fb0a415f790 0x7fb0a415fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.655+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/441263957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a40b1880 0x7fb0a40688b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.655+0000 7fb0a9c00640 1 -- 172.21.15.60:0/441263957 >> 172.21.15.60:0/441263957 conn(0x7fb0a40b5680 msgr2=0x7fb0a40b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.655+0000 7fb0a9c00640 1 -- 172.21.15.60:0/441263957 shutdown_connections 2024-09-16T15:43:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.655+0000 7fb0a9c00640 1 -- 172.21.15.60:0/441263957 wait complete. 2024-09-16T15:43:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.657+0000 7fb0a9c00640 1 Processor -- start 2024-09-16T15:43:58.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.657+0000 7fb0a9c00640 1 -- start start 2024-09-16T15:43:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a9c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a40b1880 0x7fb0a409ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a9c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a415c1a0 0x7fb0a41589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a415f790 0x7fb0a4158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a9c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a415e980 con 0x7fb0a415c1a0 2024-09-16T15:43:58.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a9c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a40b1e70 con 0x7fb0a415f790 2024-09-16T15:43:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a9c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a415eb00 con 0x7fb0a40b1880 2024-09-16T15:43:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a415c1a0 0x7fb0a41589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a415c1a0 0x7fb0a41589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59014/0 (socket says 172.21.15.60:59014) 2024-09-16T15:43:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.658+0000 7fb0a3400640 1 -- 172.21.15.60:0/2456396615 learned_addr learned my addr 172.21.15.60:0/2456396615 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb0a3e00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a40b1880 0x7fb0a409ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb0a9200640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a415f790 0x7fb0a4158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb0a3400640 1 -- 172.21.15.60:0/2456396615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a40b1880 msgr2=0x7fb0a409ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb0a3400640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a40b1880 0x7fb0a409ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb0a3400640 1 -- 172.21.15.60:0/2456396615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a415f790 msgr2=0x7fb0a4158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb0a3400640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a415f790 0x7fb0a4158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb0a3400640 1 -- 172.21.15.60:0/2456396615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a409b1b0 con 0x7fb0a415c1a0 2024-09-16T15:43:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb0a3400640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a415c1a0 0x7fb0a41589a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb09000e790 tx=0x7fb09000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.659+0000 7fb09be00640 1 -- 172.21.15.60:0/2456396615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09000eea0 con 0x7fb0a415c1a0 2024-09-16T15:43:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.660+0000 7fb09be00640 1 -- 172.21.15.60:0/2456396615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb09000cdb0 con 0x7fb0a415c1a0 2024-09-16T15:43:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.660+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a41f2810 con 0x7fb0a415c1a0 2024-09-16T15:43:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.660+0000 7fb09be00640 1 -- 172.21.15.60:0/2456396615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090010690 con 0x7fb0a415c1a0 2024-09-16T15:43:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.660+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a41f2dd0 con 0x7fb0a415c1a0 2024-09-16T15:43:58.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.661+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a415cf00 con 0x7fb0a415c1a0 2024-09-16T15:43:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.662+0000 7fb09be00640 1 -- 172.21.15.60:0/2456396615 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fb090010830 con 0x7fb0a415c1a0 2024-09-16T15:43:58.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.663+0000 7fb09be00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb084078970 0x7fb08407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:58.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.663+0000 7fb09be00640 1 -- 172.21.15.60:0/2456396615 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5609+0+0 (secure 0 0 0) 0x7fb09009b490 con 0x7fb0a415c1a0 2024-09-16T15:43:58.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.663+0000 7fb0a3e00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb084078970 0x7fb08407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:58.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.664+0000 7fb0a3e00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb084078970 0x7fb08407ae30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb094033000 tx=0x7fb094004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:58.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.668+0000 7fb09be00640 1 -- 172.21.15.60:0/2456396615 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb0900632e0 con 0x7fb0a415c1a0 2024-09-16T15:43:58.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:58.902+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb0a41529c0 con 0x7fb0a415c1a0 2024-09-16T15:43:59.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.450+0000 7fb09be00640 1 -- 172.21.15.60:0/2456396615 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v67) ==== 174+0+0 (secure 0 0 0) 0x7fb090067d10 con 0x7fb0a415c1a0 2024-09-16T15:43:59.452 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 16 and data pool 17 2024-09-16T15:43:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.455+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb084078970 msgr2=0x7fb08407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb084078970 0x7fb08407ae30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb094033000 tx=0x7fb094004730 comp rx=0 tx=0).stop 2024-09-16T15:43:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a415c1a0 msgr2=0x7fb0a41589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:59.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a415c1a0 0x7fb0a41589a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb09000e790 tx=0x7fb09000ec50 comp rx=0 tx=0).stop 2024-09-16T15:43:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 shutdown_connections 2024-09-16T15:43:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb084078970 0x7fb08407ae30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0a415f790 0x7fb0a4158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0a415c1a0 0x7fb0a41589a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 --2- 172.21.15.60:0/2456396615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0a40b1880 0x7fb0a409ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.456+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 >> 172.21.15.60:0/2456396615 conn(0x7fb0a40b5680 msgr2=0x7fb0a415b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.457+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 shutdown_connections 2024-09-16T15:43:59.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.457+0000 7fb0a9c00640 1 -- 172.21.15.60:0/2456396615 wait complete. 2024-09-16T15:43:59.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:43:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.660+0000 7f4725000640 1 -- 172.21.15.60:0/446077664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200cf300 msgr2=0x7f47200729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.660+0000 7f4725000640 1 --2- 172.21.15.60:0/446077664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200cf300 0x7f47200729a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4708005fd0 tx=0x7f470802a660 comp rx=0 tx=0).stop 2024-09-16T15:43:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.661+0000 7f4725000640 1 -- 172.21.15.60:0/446077664 shutdown_connections 2024-09-16T15:43:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.661+0000 7f4725000640 1 --2- 172.21.15.60:0/446077664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47200ccfd0 0x7f4720072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.661+0000 7f4725000640 1 --2- 172.21.15.60:0/446077664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200cf300 0x7f47200729a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.661+0000 7f4725000640 1 -- 172.21.15.60:0/446077664 >> 172.21.15.60:0/446077664 conn(0x7f47200b1950 msgr2=0x7f47200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.661+0000 7f4725000640 1 -- 172.21.15.60:0/446077664 shutdown_connections 2024-09-16T15:43:59.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.661+0000 7f4725000640 1 -- 172.21.15.60:0/446077664 wait complete. 2024-09-16T15:43:59.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.662+0000 7f4725000640 1 Processor -- start 2024-09-16T15:43:59.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.662+0000 7f4725000640 1 -- start start 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.663+0000 7f4725000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200ccfd0 0x7f4720141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.663+0000 7f4725000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47200cf300 0x7f47200caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.663+0000 7f4725000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47201445d0 0x7f47200cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.663+0000 7f4725000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4720073420 con 0x7f47200ccfd0 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.663+0000 7f4725000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f47200b4eb0 con 0x7f47201445d0 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.663+0000 7f4725000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f47200735a0 con 0x7f47200cf300 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.663+0000 7f471de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200ccfd0 0x7f4720141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200ccfd0 0x7f4720141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49094/0 (socket says 172.21.15.60:49094) 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471de00640 1 -- 172.21.15.60:0/2690155266 learned_addr learned my addr 172.21.15.60:0/2690155266 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f4715400640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47200cf300 0x7f47200caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471e800640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47201445d0 0x7f47200cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471de00640 1 -- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47200cf300 msgr2=0x7f47200caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:59.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471de00640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47200cf300 0x7f47200caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471de00640 1 -- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47201445d0 msgr2=0x7f47200cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471de00640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47201445d0 0x7f47200cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471de00640 1 -- 172.21.15.60:0/2690155266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472009b5b0 con 0x7f47200ccfd0 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471e800640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47201445d0 0x7f47200cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f471de00640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200ccfd0 0x7f4720141a90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f471000c3a0 tx=0x7f471000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.664+0000 7f4716a00640 1 -- 172.21.15.60:0/2690155266 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710017070 con 0x7f47200ccfd0 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.665+0000 7f4716a00640 1 -- 172.21.15.60:0/2690155266 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47100053c0 con 0x7f47200ccfd0 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.665+0000 7f4716a00640 1 -- 172.21.15.60:0/2690155266 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710003850 con 0x7f47200ccfd0 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.665+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472009c330 con 0x7f47200ccfd0 2024-09-16T15:43:59.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.665+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472009b850 con 0x7f47200ccfd0 2024-09-16T15:43:59.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.666+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4720073a80 con 0x7f47200ccfd0 2024-09-16T15:43:59.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.667+0000 7f4716a00640 1 -- 172.21.15.60:0/2690155266 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f471001a070 con 0x7f47200ccfd0 2024-09-16T15:43:59.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.668+0000 7f4716a00640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f46fc078970 0x7f46fc07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:43:59.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.668+0000 7f4716a00640 1 -- 172.21.15.60:0/2690155266 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5709+0+0 (secure 0 0 0) 0x7f4710099f20 con 0x7f47200ccfd0 2024-09-16T15:43:59.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.668+0000 7f4715400640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f46fc078970 0x7f46fc07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:43:59.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.669+0000 7f4715400640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f46fc078970 0x7f46fc07ae30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4708033000 tx=0x7f4708004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:43:59.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.673+0000 7f4716a00640 1 -- 172.21.15.60:0/2690155266 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4710062360 con 0x7f47200ccfd0 2024-09-16T15:43:59.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.870+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4720070a20 con 0x7f47200ccfd0 2024-09-16T15:43:59.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.871+0000 7f4716a00640 1 -- 172.21.15.60:0/2690155266 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16700 (secure 0 0 0) 0x7f4710061c60 con 0x7f47200ccfd0 2024-09-16T15:43:59.872 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:43:59.873 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:58.903878+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:56.694179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:57.861894+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:43:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f46fc078970 msgr2=0x7f46fc07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f46fc078970 0x7f46fc07ae30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4708033000 tx=0x7f4708004730 comp rx=0 tx=0).stop 2024-09-16T15:43:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200ccfd0 msgr2=0x7f4720141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:43:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200ccfd0 0x7f4720141a90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f471000c3a0 tx=0x7f471000c860 comp rx=0 tx=0).stop 2024-09-16T15:43:59.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 shutdown_connections 2024-09-16T15:43:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f46fc078970 0x7f46fc07ae30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f47201445d0 0x7f47200cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47200cf300 0x7f47200caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 --2- 172.21.15.60:0/2690155266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47200ccfd0 0x7f4720141a90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:43:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 >> 172.21.15.60:0/2690155266 conn(0x7f47200b1950 msgr2=0x7f47200cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:43:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 shutdown_connections 2024-09-16T15:43:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:43:59.874+0000 7f4725000640 1 -- 172.21.15.60:0/2690155266 wait complete. 2024-09-16T15:43:59.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:44:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.050+0000 7fc46e600640 1 -- 172.21.15.60:0/3109782797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc46815f5a0 msgr2=0x7fc4680b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.050+0000 7fc46e600640 1 --2- 172.21.15.60:0/3109782797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc46815f5a0 0x7fc4680b2440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc458009830 tx=0x7fc45802a480 comp rx=0 tx=0).stop 2024-09-16T15:44:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.051+0000 7fc46e600640 1 -- 172.21.15.60:0/3109782797 shutdown_connections 2024-09-16T15:44:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.051+0000 7fc46e600640 1 --2- 172.21.15.60:0/3109782797 >> v2:172.21.15.100:3300/0 conn(0x7fc4680ccf00 0x7fc4680acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.051+0000 7fc46e600640 1 --2- 172.21.15.60:0/3109782797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc46815f5a0 0x7fc4680b2440 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.051+0000 7fc46e600640 1 -- 172.21.15.60:0/3109782797 >> 172.21.15.60:0/3109782797 conn(0x7fc4680b1950 msgr2=0x7fc4680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.051+0000 7fc46e600640 1 -- 172.21.15.60:0/3109782797 shutdown_connections 2024-09-16T15:44:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.051+0000 7fc46e600640 1 -- 172.21.15.60:0/3109782797 wait complete. 2024-09-16T15:44:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.052+0000 7fc46e600640 1 Processor -- start 2024-09-16T15:44:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.053+0000 7fc46e600640 1 -- start start 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.053+0000 7fc46e600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4680ad270 0x7fc46815e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.053+0000 7fc46e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4680ccf00 0x7fc4680cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.053+0000 7fc46e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46815f5a0 0x7fc468156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.053+0000 7fc46e600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc46806f2e0 con 0x7fc4680ccf00 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.053+0000 7fc46e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc46806f160 con 0x7fc46815f5a0 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.053+0000 7fc46e600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc46806f460 con 0x7fc4680ad270 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc466a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4680ccf00 0x7fc4680cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc466a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4680ccf00 0x7fc4680cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49096/0 (socket says 172.21.15.60:49096) 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc466a00640 1 -- 172.21.15.60:0/3806513488 learned_addr learned my addr 172.21.15.60:0/3806513488 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc467e00640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46815f5a0 0x7fc468156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc467400640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4680ad270 0x7fc46815e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc466a00640 1 -- 172.21.15.60:0/3806513488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4680ad270 msgr2=0x7fc46815e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc466a00640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4680ad270 0x7fc46815e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc466a00640 1 -- 172.21.15.60:0/3806513488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46815f5a0 msgr2=0x7fc468156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc466a00640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46815f5a0 0x7fc468156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.054+0000 7fc466a00640 1 -- 172.21.15.60:0/3806513488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4680cbc30 con 0x7fc4680ccf00 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.055+0000 7fc466a00640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4680ccf00 0x7fc4680cb170 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc45400e9d0 tx=0x7fc45400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.055+0000 7fc45f400640 1 -- 172.21.15.60:0/3806513488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45400cd80 con 0x7fc4680ccf00 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.055+0000 7fc45f400640 1 -- 172.21.15.60:0/3806513488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc454010430 con 0x7fc4680ccf00 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.055+0000 7fc45f400640 1 -- 172.21.15.60:0/3806513488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454010880 con 0x7fc4680ccf00 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.055+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4680cbf10 con 0x7fc4680ccf00 2024-09-16T15:44:00.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.055+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4680cc4d0 con 0x7fc4680ccf00 2024-09-16T15:44:00.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.056+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46806f980 con 0x7fc4680ccf00 2024-09-16T15:44:00.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.057+0000 7fc45f400640 1 -- 172.21.15.60:0/3806513488 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fc454009400 con 0x7fc4680ccf00 2024-09-16T15:44:00.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.058+0000 7fc45f400640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc448078970 0x7fc44807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:00.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.058+0000 7fc45f400640 1 -- 172.21.15.60:0/3806513488 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5709+0+0 (secure 0 0 0) 0x7fc45409b670 con 0x7fc4680ccf00 2024-09-16T15:44:00.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.059+0000 7fc467400640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc448078970 0x7fc44807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:00.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.059+0000 7fc467400640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc448078970 0x7fc44807ae30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc458034000 tx=0x7fc458004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:00.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.062+0000 7fc45f400640 1 -- 172.21.15.60:0/3806513488 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc4540633d0 con 0x7fc4680ccf00 2024-09-16T15:44:00.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.260+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc468069ac0 con 0x7fc4680ccf00 2024-09-16T15:44:00.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.261+0000 7fc45f400640 1 -- 172.21.15.60:0/3806513488 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16700 (secure 0 0 0) 0x7fc454067e00 con 0x7fc4680ccf00 2024-09-16T15:44:00.263 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:44:00.263 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:43:58.903878+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:56.694179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:57.861894+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:44:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc448078970 msgr2=0x7fc44807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc448078970 0x7fc44807ae30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc458034000 tx=0x7fc458004730 comp rx=0 tx=0).stop 2024-09-16T15:44:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4680ccf00 msgr2=0x7fc4680cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4680ccf00 0x7fc4680cb170 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc45400e9d0 tx=0x7fc45400ee90 comp rx=0 tx=0).stop 2024-09-16T15:44:00.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 shutdown_connections 2024-09-16T15:44:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc448078970 0x7fc44807ae30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46815f5a0 0x7fc468156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4680ccf00 0x7fc4680cb170 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 --2- 172.21.15.60:0/3806513488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4680ad270 0x7fc46815e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 >> 172.21.15.60:0/3806513488 conn(0x7fc4680b1950 msgr2=0x7fc4680b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 shutdown_connections 2024-09-16T15:44:00.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.264+0000 7fc46e600640 1 -- 172.21.15.60:0/3806513488 wait complete. 2024-09-16T15:44:00.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:44:00.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.445+0000 7eff82c00640 1 -- 172.21.15.60:0/4287759660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff60005230 msgr2=0x7eff6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:00.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.445+0000 7eff82c00640 1 --2- 172.21.15.60:0/4287759660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff60005230 0x7eff6000b9c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c610 tx=0x7eff60005830 comp rx=0 tx=0).stop 2024-09-16T15:44:00.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.445+0000 7eff82c00640 1 -- 172.21.15.60:0/4287759660 shutdown_connections 2024-09-16T15:44:00.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.445+0000 7eff82c00640 1 --2- 172.21.15.60:0/4287759660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff60005230 0x7eff6000b9c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.445+0000 7eff82c00640 1 -- 172.21.15.60:0/4287759660 >> 172.21.15.60:0/4287759660 conn(0x7eff7c0b1840 msgr2=0x7eff7c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:00.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.446+0000 7eff82c00640 1 -- 172.21.15.60:0/4287759660 shutdown_connections 2024-09-16T15:44:00.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.446+0000 7eff82c00640 1 -- 172.21.15.60:0/4287759660 wait complete. 2024-09-16T15:44:00.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.447+0000 7eff82c00640 1 Processor -- start 2024-09-16T15:44:00.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.447+0000 7eff82c00640 1 -- start start 2024-09-16T15:44:00.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff82c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff60005230 0x7eff7c1f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:00.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff82c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff7c1e7c70 0x7eff7c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff82c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c1e80f0 0x7eff7c0b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff82c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c0ce830 con 0x7eff7c1e7c70 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff82c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c0ce6b0 con 0x7eff7c1e80f0 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff82c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7eff7c0704c0 con 0x7eff60005230 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff7b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff7c1e7c70 0x7eff7c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff7b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff7c1e7c70 0x7eff7c0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49106/0 (socket says 172.21.15.60:49106) 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.448+0000 7eff7b400640 1 -- 172.21.15.60:0/1319570784 learned_addr learned my addr 172.21.15.60:0/1319570784 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff80c00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c1e80f0 0x7eff7c0b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:00.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff7be00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff60005230 0x7eff7c1f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:00.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff7b400640 1 -- 172.21.15.60:0/1319570784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff60005230 msgr2=0x7eff7c1f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:00.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff7b400640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff60005230 0x7eff7c1f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff7b400640 1 -- 172.21.15.60:0/1319570784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c1e80f0 msgr2=0x7eff7c0b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:00.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff7b400640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c1e80f0 0x7eff7c0b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff7b400640 1 -- 172.21.15.60:0/1319570784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c1e86c0 con 0x7eff7c1e7c70 2024-09-16T15:44:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff7b400640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff7c1e7c70 0x7eff7c0b0700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7eff68003e90 tx=0x7eff68001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff73e00640 1 -- 172.21.15.60:0/1319570784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68015070 con 0x7eff7c1e7c70 2024-09-16T15:44:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.449+0000 7eff73e00640 1 -- 172.21.15.60:0/1319570784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff68011900 con 0x7eff7c1e7c70 2024-09-16T15:44:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.450+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7c1f8400 con 0x7eff7c1e7c70 2024-09-16T15:44:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.450+0000 7eff73e00640 1 -- 172.21.15.60:0/1319570784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68011d30 con 0x7eff7c1e7c70 2024-09-16T15:44:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.450+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7c1f89c0 con 0x7eff7c1e7c70 2024-09-16T15:44:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.451+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7c06ef50 con 0x7eff7c1e7c70 2024-09-16T15:44:00.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.452+0000 7eff73e00640 1 -- 172.21.15.60:0/1319570784 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7eff68018070 con 0x7eff7c1e7c70 2024-09-16T15:44:00.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.453+0000 7eff73e00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff5c0789a0 0x7eff5c07ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:00.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.453+0000 7eff73e00640 1 -- 172.21.15.60:0/1319570784 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5709+0+0 (secure 0 0 0) 0x7eff6809b7d0 con 0x7eff7c1e7c70 2024-09-16T15:44:00.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.454+0000 7eff7be00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff5c0789a0 0x7eff5c07ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:00.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.454+0000 7eff7be00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff5c0789a0 0x7eff5c07ae60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7eff6c005e00 tx=0x7eff6c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:00.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.458+0000 7eff73e00640 1 -- 172.21.15.60:0/1319570784 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7eff7c06ef50 con 0x7eff7c1e7c70 2024-09-16T15:44:00.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:00.696+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7eff7c06f510 con 0x7eff7c1e7c70 2024-09-16T15:44:01.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:44:01.092 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:44:01.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.478+0000 7eff73e00640 1 -- 172.21.15.60:0/1319570784 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) ==== 118+0+0 (secure 0 0 0) 0x7eff7c06f510 con 0x7eff7c1e7c70 2024-09-16T15:44:01.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff5c0789a0 msgr2=0x7eff5c07ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff5c0789a0 0x7eff5c07ae60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7eff6c005e00 tx=0x7eff6c005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff7c1e7c70 msgr2=0x7eff7c0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff7c1e7c70 0x7eff7c0b0700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7eff68003e90 tx=0x7eff68001800 comp rx=0 tx=0).stop 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 shutdown_connections 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7eff5c0789a0 0x7eff5c07ae60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff7c1e80f0 0x7eff7c0b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff7c1e7c70 0x7eff7c0b0700 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 --2- 172.21.15.60:0/1319570784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff60005230 0x7eff7c1f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 >> 172.21.15.60:0/1319570784 conn(0x7eff7c0b1840 msgr2=0x7eff7c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 shutdown_connections 2024-09-16T15:44:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.481+0000 7eff82c00640 1 -- 172.21.15.60:0/1319570784 wait complete. 2024-09-16T15:44:01.500 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:44:01.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:44:01.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 -- 172.21.15.60:0/219215383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e406efa0 msgr2=0x7f40e40cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 --2- 172.21.15.60:0/219215383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e406efa0 0x7f40e40cc5c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f40e40d0cd0 tx=0x7f40d002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 -- 172.21.15.60:0/219215383 shutdown_connections 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 --2- 172.21.15.60:0/219215383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e40ccb00 0x7f40e415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 --2- 172.21.15.60:0/219215383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e406efa0 0x7f40e40cc5c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 --2- 172.21.15.60:0/219215383 >> v2:172.21.15.100:3300/0 conn(0x7f40e4145750 0x7f40e4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 -- 172.21.15.60:0/219215383 >> 172.21.15.60:0/219215383 conn(0x7f40e40b1840 msgr2=0x7f40e40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 -- 172.21.15.60:0/219215383 shutdown_connections 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.664+0000 7f40ec600640 1 -- 172.21.15.60:0/219215383 wait complete. 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.666+0000 7f40ec600640 1 Processor -- start 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.666+0000 7f40ec600640 1 -- start start 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40ec600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e406efa0 0x7f40e40ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:01.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40ec600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e40ccb00 0x7f40e413dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40ec600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e4145750 0x7f40e413e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40ec600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41602c0 con 0x7f40e40ccb00 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40ec600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4160140 con 0x7f40e4145750 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40ec600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f40e4160440 con 0x7f40e406efa0 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40e9200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e40ccb00 0x7f40e413dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40e9200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e40ccb00 0x7f40e413dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49126/0 (socket says 172.21.15.60:49126) 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40e9200640 1 -- 172.21.15.60:0/2842830261 learned_addr learned my addr 172.21.15.60:0/2842830261 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40e9c00640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e406efa0 0x7f40e40ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.667+0000 7f40ea600640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e4145750 0x7f40e413e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e9200640 1 -- 172.21.15.60:0/2842830261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e406efa0 msgr2=0x7f40e40ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e9200640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e406efa0 0x7f40e40ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e9200640 1 -- 172.21.15.60:0/2842830261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e4145750 msgr2=0x7f40e413e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e9200640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e4145750 0x7f40e413e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e9200640 1 -- 172.21.15.60:0/2842830261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e40aefc0 con 0x7f40e40ccb00 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e9200640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e40ccb00 0x7f40e413dca0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009850 tx=0x7f40d0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e1600640 1 -- 172.21.15.60:0/2842830261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d003f070 con 0x7f40e40ccb00 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e1600640 1 -- 172.21.15.60:0/2842830261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40d00047a0 con 0x7f40e40ccb00 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40e1600640 1 -- 172.21.15.60:0/2842830261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0035030 con 0x7f40e40ccb00 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e40aa4b0 con 0x7f40e40ccb00 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.668+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e41f5c00 con 0x7f40e40ccb00 2024-09-16T15:44:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.670+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e415f9f0 con 0x7f40e40ccb00 2024-09-16T15:44:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.671+0000 7f40e1600640 1 -- 172.21.15.60:0/2842830261 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f40d0042070 con 0x7f40e40ccb00 2024-09-16T15:44:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.672+0000 7f40e1600640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c4078970 0x7f40c407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.672+0000 7f40e1600640 1 -- 172.21.15.60:0/2842830261 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f40d00bb880 con 0x7f40e40ccb00 2024-09-16T15:44:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.672+0000 7f40e9c00640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c4078970 0x7f40c407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.673+0000 7f40e9c00640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c4078970 0x7f40c407ae30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009b30 tx=0x7f40d8005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.676+0000 7f40e1600640 1 -- 172.21.15.60:0/2842830261 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f40d0083710 con 0x7f40e40ccb00 2024-09-16T15:44:01.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:01.909+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f40e4069a10 con 0x7f40e40ccb00 2024-09-16T15:44:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.484+0000 7f40e1600640 1 -- 172.21.15.60:0/2842830261 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) ==== 122+0+0 (secure 0 0 0) 0x7f40d0088140 con 0x7f40e40ccb00 2024-09-16T15:44:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c4078970 msgr2=0x7f40c407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c4078970 0x7f40c407ae30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009b30 tx=0x7f40d8005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e40ccb00 msgr2=0x7f40e413dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e40ccb00 0x7f40e413dca0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009850 tx=0x7f40d0003820 comp rx=0 tx=0).stop 2024-09-16T15:44:02.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 shutdown_connections 2024-09-16T15:44:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c4078970 0x7f40c407ae30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e4145750 0x7f40e413e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e40ccb00 0x7f40e413dca0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 --2- 172.21.15.60:0/2842830261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e406efa0 0x7f40e40ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 >> 172.21.15.60:0/2842830261 conn(0x7f40e40b1840 msgr2=0x7f40e40d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 shutdown_connections 2024-09-16T15:44:02.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.487+0000 7f40ec600640 1 -- 172.21.15.60:0/2842830261 wait complete. 2024-09-16T15:44:02.505 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:44:02.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.666+0000 7fa798200640 1 -- 172.21.15.60:0/740067706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 msgr2=0x7fa7900cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.666+0000 7fa798200640 1 --2- 172.21.15.60:0/740067706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 0x7fa7900cea60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa77c005fd0 tx=0x7fa77c02a540 comp rx=0 tx=0).stop 2024-09-16T15:44:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.666+0000 7fa798200640 1 -- 172.21.15.60:0/740067706 shutdown_connections 2024-09-16T15:44:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.666+0000 7fa798200640 1 --2- 172.21.15.60:0/740067706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 0x7fa7900cea60 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.666+0000 7fa798200640 1 --2- 172.21.15.60:0/740067706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7900b73c0 0x7fa790071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.666+0000 7fa798200640 1 -- 172.21.15.60:0/740067706 >> 172.21.15.60:0/740067706 conn(0x7fa79006d230 msgr2=0x7fa7900688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.666+0000 7fa798200640 1 -- 172.21.15.60:0/740067706 shutdown_connections 2024-09-16T15:44:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.666+0000 7fa798200640 1 -- 172.21.15.60:0/740067706 wait complete. 2024-09-16T15:44:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.667+0000 7fa798200640 1 Processor -- start 2024-09-16T15:44:02.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.667+0000 7fa798200640 1 -- start start 2024-09-16T15:44:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.668+0000 7fa798200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 0x7fa7901525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.668+0000 7fa798200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7900b73c0 0x7fa7900b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.668+0000 7fa798200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7900cefa0 0x7fa7900cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.668+0000 7fa798200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa7900b7920 con 0x7fa790071260 2024-09-16T15:44:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.668+0000 7fa798200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7900b77a0 con 0x7fa7900b73c0 2024-09-16T15:44:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.668+0000 7fa798200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa7900d2350 con 0x7fa7900cefa0 2024-09-16T15:44:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 0x7fa7901525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 0x7fa7901525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49154/0 (socket says 172.21.15.60:49154) 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 -- 172.21.15.60:0/4090177231 learned_addr learned my addr 172.21.15.60:0/4090177231 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa794e00640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7900b73c0 0x7fa7900b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa796200640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7900cefa0 0x7fa7900cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 -- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7900cefa0 msgr2=0x7fa7900cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7900cefa0 0x7fa7900cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 -- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7900b73c0 msgr2=0x7fa7900b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7900b73c0 0x7fa7900b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 -- 172.21.15.60:0/4090177231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7900b2a60 con 0x7fa790071260 2024-09-16T15:44:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa794e00640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7900b73c0 0x7fa7900b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa795800640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 0x7fa7901525d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa78400d770 tx=0x7fa78400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.669+0000 7fa78d600640 1 -- 172.21.15.60:0/4090177231 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784017070 con 0x7fa790071260 2024-09-16T15:44:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.670+0000 7fa78d600640 1 -- 172.21.15.60:0/4090177231 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa784009d20 con 0x7fa790071260 2024-09-16T15:44:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.670+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7900b37e0 con 0x7fa790071260 2024-09-16T15:44:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.670+0000 7fa78d600640 1 -- 172.21.15.60:0/4090177231 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784005280 con 0x7fa790071260 2024-09-16T15:44:02.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.670+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7900b3da0 con 0x7fa790071260 2024-09-16T15:44:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.671+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7900d28e0 con 0x7fa790071260 2024-09-16T15:44:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.672+0000 7fa78d600640 1 -- 172.21.15.60:0/4090177231 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fa784005420 con 0x7fa790071260 2024-09-16T15:44:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.673+0000 7fa78d600640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa770078970 0x7fa77007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.673+0000 7fa78d600640 1 -- 172.21.15.60:0/4090177231 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fa78409ade0 con 0x7fa790071260 2024-09-16T15:44:02.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.673+0000 7fa794e00640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa770078970 0x7fa77007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:02.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.674+0000 7fa794e00640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa770078970 0x7fa77007ae30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa77c033000 tx=0x7fa77c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:02.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.677+0000 7fa78d600640 1 -- 172.21.15.60:0/4090177231 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa784063190 con 0x7fa790071260 2024-09-16T15:44:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.913+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa79006ad20 con 0x7fa790071260 2024-09-16T15:44:02.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.913+0000 7fa78d600640 1 -- 172.21.15.60:0/4090177231 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7fa784062a90 con 0x7fa790071260 2024-09-16T15:44:02.915 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:44:02.915 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"btime":"2024-09-16T15:44:02:480020+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/586556953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":586556953},{"type":"v1","addr":"172.21.15.100:6835","nonce":586556953}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2314857147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2314857147},{"type":"v1","addr":"172.21.15.60:6834","nonce":2314857147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1301436173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1301436173},{"type":"v1","addr":"172.21.15.60:6835","nonce":1301436173}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:58.903833+0000","modified":"2024-09-16T15:44:01.910153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6688":{"gid":6688,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6801/515865896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":515865896},{"type":"v1","addr":"172.21.15.100:6801","nonce":515865896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6688,"qdb_cluster":[6688]},"id":8}]} 2024-09-16T15:44:02.915 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-09-16T15:44:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa770078970 msgr2=0x7fa77007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa770078970 0x7fa77007ae30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa77c033000 tx=0x7fa77c004730 comp rx=0 tx=0).stop 2024-09-16T15:44:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 msgr2=0x7fa7901525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 0x7fa7901525d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa78400d770 tx=0x7fa78400dc30 comp rx=0 tx=0).stop 2024-09-16T15:44:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 shutdown_connections 2024-09-16T15:44:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa770078970 0x7fa77007ae30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7900cefa0 0x7fa7900cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7900b73c0 0x7fa7900b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 --2- 172.21.15.60:0/4090177231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa790071260 0x7fa7901525d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 >> 172.21.15.60:0/4090177231 conn(0x7fa79006d230 msgr2=0x7fa79015b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 shutdown_connections 2024-09-16T15:44:02.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:02.916+0000 7fa798200640 1 -- 172.21.15.60:0/4090177231 wait complete. 2024-09-16T15:44:02.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:44:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.092+0000 7f17cb200640 1 -- 172.21.15.60:0/1260503893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17cc0ce100 msgr2=0x7f17cc0cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.092+0000 7f17cb200640 1 --2- 172.21.15.60:0/1260503893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17cc0ce100 0x7f17cc0cc860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009860 tx=0x7f17bc02a5a0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.092+0000 7f17cb200640 1 -- 172.21.15.60:0/1260503893 shutdown_connections 2024-09-16T15:44:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.092+0000 7f17cb200640 1 --2- 172.21.15.60:0/1260503893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17cc0ce100 0x7f17cc0cc860 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.092+0000 7f17cb200640 1 -- 172.21.15.60:0/1260503893 >> 172.21.15.60:0/1260503893 conn(0x7f17cc0b1e90 msgr2=0x7f17cc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.093+0000 7f17cb200640 1 -- 172.21.15.60:0/1260503893 shutdown_connections 2024-09-16T15:44:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.093+0000 7f17cb200640 1 -- 172.21.15.60:0/1260503893 wait complete. 2024-09-16T15:44:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.094+0000 7f17cb200640 1 Processor -- start 2024-09-16T15:44:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.094+0000 7f17cb200640 1 -- start start 2024-09-16T15:44:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.095+0000 7f17cb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc074ca0 0x7f17cc0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.095+0000 7f17cb200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17cc0afde0 0x7f17cc1549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.095+0000 7f17cb200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17cc0ce100 0x7f17cc159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.095+0000 7f17cb200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc0b1010 con 0x7f17cc0afde0 2024-09-16T15:44:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.095+0000 7f17cb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc0b0e90 con 0x7f17cc074ca0 2024-09-16T15:44:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.095+0000 7f17cb200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc1662b0 con 0x7f17cc0ce100 2024-09-16T15:44:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.095+0000 7f17c1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17cc0afde0 0x7f17cc1549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17cc0afde0 0x7f17cc1549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49170/0 (socket says 172.21.15.60:49170) 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c1400640 1 -- 172.21.15.60:0/1856024710 learned_addr learned my addr 172.21.15.60:0/1856024710 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c9e00640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc074ca0 0x7f17cc0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17ca800640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17cc0ce100 0x7f17cc159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c1400640 1 -- 172.21.15.60:0/1856024710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17cc0ce100 msgr2=0x7f17cc159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c1400640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17cc0ce100 0x7f17cc159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c1400640 1 -- 172.21.15.60:0/1856024710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc074ca0 msgr2=0x7f17cc0adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c1400640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc074ca0 0x7f17cc0adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c1400640 1 -- 172.21.15.60:0/1856024710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc164e90 con 0x7f17cc0afde0 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c1400640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17cc0afde0 0x7f17cc1549c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f17b400c3a0 tx=0x7f17b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.096+0000 7f17c2a00640 1 -- 172.21.15.60:0/1856024710 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4017070 con 0x7f17cc0afde0 2024-09-16T15:44:03.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.097+0000 7f17c2a00640 1 -- 172.21.15.60:0/1856024710 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17b40053c0 con 0x7f17cc0afde0 2024-09-16T15:44:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.097+0000 7f17c2a00640 1 -- 172.21.15.60:0/1856024710 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b4003830 con 0x7f17cc0afde0 2024-09-16T15:44:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.097+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc165170 con 0x7f17cc0afde0 2024-09-16T15:44:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.097+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc0b2740 con 0x7f17cc0afde0 2024-09-16T15:44:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.098+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc0cd020 con 0x7f17cc0afde0 2024-09-16T15:44:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.099+0000 7f17c2a00640 1 -- 172.21.15.60:0/1856024710 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f17b401a070 con 0x7f17cc0afde0 2024-09-16T15:44:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.100+0000 7f17c2a00640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17a80789a0 0x7f17a807ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.100+0000 7f17c2a00640 1 -- 172.21.15.60:0/1856024710 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f17b4099e70 con 0x7f17cc0afde0 2024-09-16T15:44:03.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.100+0000 7f17c9e00640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17a80789a0 0x7f17a807ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.101+0000 7f17c9e00640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17a80789a0 0x7f17a807ae60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f17bc034000 tx=0x7f17bc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:03.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.104+0000 7f17c2a00640 1 -- 172.21.15.60:0/1856024710 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f17b40622b0 con 0x7f17cc0afde0 2024-09-16T15:44:03.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.303+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17cc0ae3c0 con 0x7f17cc0afde0 2024-09-16T15:44:03.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.304+0000 7f17c2a00640 1 -- 172.21.15.60:0/1856024710 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16687 (secure 0 0 0) 0x7f17b4061bb0 con 0x7f17cc0afde0 2024-09-16T15:44:03.308 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:44:03.309 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":107,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:44:00.461978+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:56.694179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:57.861894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17a80789a0 msgr2=0x7f17a807ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17a80789a0 0x7f17a807ae60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f17bc034000 tx=0x7f17bc004730 comp rx=0 tx=0).stop 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17cc0afde0 msgr2=0x7f17cc1549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17cc0afde0 0x7f17cc1549c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f17b400c3a0 tx=0x7f17b400c860 comp rx=0 tx=0).stop 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 shutdown_connections 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17a80789a0 0x7f17a807ae60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17cc0ce100 0x7f17cc159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17cc0afde0 0x7f17cc1549c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 --2- 172.21.15.60:0/1856024710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17cc074ca0 0x7f17cc0adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 >> 172.21.15.60:0/1856024710 conn(0x7f17cc0b1e90 msgr2=0x7f17cc0c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 shutdown_connections 2024-09-16T15:44:03.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.307+0000 7f17cb200640 1 -- 172.21.15.60:0/1856024710 wait complete. 2024-09-16T15:44:03.324 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:44:03.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:44:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.483+0000 7fc363200640 1 -- 172.21.15.60:0/2764798053 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc36415bc30 msgr2=0x7fc3640b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.483+0000 7fc363200640 1 --2- 172.21.15.60:0/2764798053 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc36415bc30 0x7fc3640b28f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc354005fd0 tx=0x7fc35402a8b0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.483+0000 7fc363200640 1 -- 172.21.15.60:0/2764798053 shutdown_connections 2024-09-16T15:44:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.483+0000 7fc363200640 1 --2- 172.21.15.60:0/2764798053 >> v2:172.21.15.100:3300/0 conn(0x7fc3640d0540 0x7fc36406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.483+0000 7fc363200640 1 --2- 172.21.15.60:0/2764798053 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc36415bc30 0x7fc3640b28f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.483+0000 7fc363200640 1 -- 172.21.15.60:0/2764798053 >> 172.21.15.60:0/2764798053 conn(0x7fc3640cfa20 msgr2=0x7fc3640cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.483+0000 7fc363200640 1 -- 172.21.15.60:0/2764798053 shutdown_connections 2024-09-16T15:44:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.483+0000 7fc363200640 1 -- 172.21.15.60:0/2764798053 wait complete. 2024-09-16T15:44:03.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.485+0000 7fc363200640 1 Processor -- start 2024-09-16T15:44:03.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.485+0000 7fc363200640 1 -- start start 2024-09-16T15:44:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.486+0000 7fc363200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3640d0540 0x7fc3640b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.486+0000 7fc363200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc36415bc30 0x7fc3640c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.486+0000 7fc363200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3640c8650 0x7fc3640c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.486+0000 7fc363200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc364074660 con 0x7fc3640d0540 2024-09-16T15:44:03.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.486+0000 7fc363200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc36415c200 con 0x7fc3640c8650 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.486+0000 7fc363200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc3640c9930 con 0x7fc36415bc30 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3640d0540 0x7fc3640b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3640d0540 0x7fc3640b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49178/0 (socket says 172.21.15.60:49178) 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 -- 172.21.15.60:0/182992093 learned_addr learned my addr 172.21.15.60:0/182992093 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc359400640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc36415bc30 0x7fc3640c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc362800640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3640c8650 0x7fc3640c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 -- 172.21.15.60:0/182992093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc36415bc30 msgr2=0x7fc3640c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc36415bc30 0x7fc3640c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 -- 172.21.15.60:0/182992093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3640c8650 msgr2=0x7fc3640c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3640c8650 0x7fc3640c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 -- 172.21.15.60:0/182992093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3640c9320 con 0x7fc3640d0540 2024-09-16T15:44:03.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.487+0000 7fc361e00640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3640d0540 0x7fc3640b4af0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc354000f80 tx=0x7fc3540316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:03.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.488+0000 7fc35aa00640 1 -- 172.21.15.60:0/182992093 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354031990 con 0x7fc3640d0540 2024-09-16T15:44:03.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.488+0000 7fc35aa00640 1 -- 172.21.15.60:0/182992093 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc354031c20 con 0x7fc3640d0540 2024-09-16T15:44:03.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.488+0000 7fc35aa00640 1 -- 172.21.15.60:0/182992093 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540332d0 con 0x7fc3640d0540 2024-09-16T15:44:03.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.488+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36415ad90 con 0x7fc3640d0540 2024-09-16T15:44:03.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.489+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3640c9010 con 0x7fc3640d0540 2024-09-16T15:44:03.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.490+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc36406fc40 con 0x7fc3640d0540 2024-09-16T15:44:03.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.491+0000 7fc35aa00640 1 -- 172.21.15.60:0/182992093 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fc354033470 con 0x7fc3640d0540 2024-09-16T15:44:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.492+0000 7fc35aa00640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc3400789a0 0x7fc34007ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.492+0000 7fc35aa00640 1 -- 172.21.15.60:0/182992093 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fc3540bbb90 con 0x7fc3640d0540 2024-09-16T15:44:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.492+0000 7fc359400640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc3400789a0 0x7fc34007ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.493+0000 7fc359400640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc3400789a0 0x7fc34007ae60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc34c007fd0 tx=0x7fc34c001430 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.496+0000 7fc35aa00640 1 -- 172.21.15.60:0/182992093 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc35404a030 con 0x7fc3640d0540 2024-09-16T15:44:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.687+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc364070fe0 con 0x7fc3400789a0 2024-09-16T15:44:03.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.692+0000 7fc35aa00640 1 -- 172.21.15.60:0/182992093 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7fc364070fe0 con 0x7fc3400789a0 2024-09-16T15:44:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:44:03.697 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:44:03.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc3400789a0 msgr2=0x7fc34007ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc3400789a0 0x7fc34007ae60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc34c007fd0 tx=0x7fc34c001430 comp rx=0 tx=0).stop 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3640d0540 msgr2=0x7fc3640b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3640d0540 0x7fc3640b4af0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc354000f80 tx=0x7fc3540316f0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 shutdown_connections 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc3400789a0 0x7fc34007ae60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3640c8650 0x7fc3640c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc36415bc30 0x7fc3640c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 --2- 172.21.15.60:0/182992093 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3640d0540 0x7fc3640b4af0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.695+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 >> 172.21.15.60:0/182992093 conn(0x7fc3640cfa20 msgr2=0x7fc3640cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.696+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 shutdown_connections 2024-09-16T15:44:03.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.696+0000 7fc363200640 1 -- 172.21.15.60:0/182992093 wait complete. 2024-09-16T15:44:03.713 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-16T15:44:02.172454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269392,"kb_used_data":11536,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754705328,"statfs":{"total":773094113280,"available":772818255872,"internally_reserved":0,"allocated":11812864,"data_stored":5809372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.729927"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556105+0000","last_change":"2024-09-16T15:43:59.488963+0000","last_active":"2024-09-16T15:44:01.556105+0000","last_peered":"2024-09-16T15:44:01.556105+0000","last_clean":"2024-09-16T15:44:01.556105+0000","last_became_active":"2024-09-16T15:43:59.484176+0000","last_became_peered":"2024-09-16T15:43:59.484176+0000","last_unstale":"2024-09-16T15:44:01.556105+0000","last_undegraded":"2024-09-16T15:44:01.556105+0000","last_fullsized":"2024-09-16T15:44:01.556105+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:11:55.436021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473764+0000","last_change":"2024-09-16T15:43:59.468062+0000","last_active":"2024-09-16T15:44:00.473764+0000","last_peered":"2024-09-16T15:44:00.473764+0000","last_clean":"2024-09-16T15:44:00.473764+0000","last_became_active":"2024-09-16T15:43:59.467771+0000","last_became_peered":"2024-09-16T15:43:59.467771+0000","last_unstale":"2024-09-16T15:44:00.473764+0000","last_undegraded":"2024-09-16T15:44:00.473764+0000","last_fullsized":"2024-09-16T15:44:00.473764+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.262997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556580+0000","last_change":"2024-09-16T15:43:59.485078+0000","last_active":"2024-09-16T15:44:01.556580+0000","last_peered":"2024-09-16T15:44:01.556580+0000","last_clean":"2024-09-16T15:44:01.556580+0000","last_became_active":"2024-09-16T15:43:59.484600+0000","last_became_peered":"2024-09-16T15:43:59.484600+0000","last_unstale":"2024-09-16T15:44:01.556580+0000","last_undegraded":"2024-09-16T15:44:01.556580+0000","last_fullsized":"2024-09-16T15:44:01.556580+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:33.214101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555822+0000","last_change":"2024-09-16T15:43:59.480908+0000","last_active":"2024-09-16T15:44:01.555822+0000","last_peered":"2024-09-16T15:44:01.555822+0000","last_clean":"2024-09-16T15:44:01.555822+0000","last_became_active":"2024-09-16T15:43:59.474701+0000","last_became_peered":"2024-09-16T15:43:59.474701+0000","last_unstale":"2024-09-16T15:44:01.555822+0000","last_undegraded":"2024-09-16T15:44:01.555822+0000","last_fullsized":"2024-09-16T15:44:01.555822+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:00.220826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041312+0000","last_change":"2024-09-16T15:43:59.482363+0000","last_active":"2024-09-16T15:44:01.041312+0000","last_peered":"2024-09-16T15:44:01.041312+0000","last_clean":"2024-09-16T15:44:01.041312+0000","last_became_active":"2024-09-16T15:43:59.482024+0000","last_became_peered":"2024-09-16T15:43:59.482024+0000","last_unstale":"2024-09-16T15:44:01.041312+0000","last_undegraded":"2024-09-16T15:44:01.041312+0000","last_fullsized":"2024-09-16T15:44:01.041312+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:59.706734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055074+0000","last_change":"2024-09-16T15:43:59.472680+0000","last_active":"2024-09-16T15:44:01.055074+0000","last_peered":"2024-09-16T15:44:01.055074+0000","last_clean":"2024-09-16T15:44:01.055074+0000","last_became_active":"2024-09-16T15:43:59.472359+0000","last_became_peered":"2024-09-16T15:43:59.472359+0000","last_unstale":"2024-09-16T15:44:01.055074+0000","last_undegraded":"2024-09-16T15:44:01.055074+0000","last_fullsized":"2024-09-16T15:44:01.055074+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:16.620530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.477022+0000","last_change":"2024-09-16T15:43:59.475220+0000","last_active":"2024-09-16T15:44:00.477022+0000","last_peered":"2024-09-16T15:44:00.477022+0000","last_clean":"2024-09-16T15:44:00.477022+0000","last_became_active":"2024-09-16T15:43:59.474814+0000","last_became_peered":"2024-09-16T15:43:59.474814+0000","last_unstale":"2024-09-16T15:44:00.477022+0000","last_undegraded":"2024-09-16T15:44:00.477022+0000","last_fullsized":"2024-09-16T15:44:00.477022+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:02:55.837680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556528+0000","last_change":"2024-09-16T15:43:59.483487+0000","last_active":"2024-09-16T15:44:01.556528+0000","last_peered":"2024-09-16T15:44:01.556528+0000","last_clean":"2024-09-16T15:44:01.556528+0000","last_became_active":"2024-09-16T15:43:59.483210+0000","last_became_peered":"2024-09-16T15:43:59.483210+0000","last_unstale":"2024-09-16T15:44:01.556528+0000","last_undegraded":"2024-09-16T15:44:01.556528+0000","last_fullsized":"2024-09-16T15:44:01.556528+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:40.325491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556158+0000","last_change":"2024-09-16T15:43:59.477160+0000","last_active":"2024-09-16T15:44:01.556158+0000","last_peered":"2024-09-16T15:44:01.556158+0000","last_clean":"2024-09-16T15:44:01.556158+0000","last_became_active":"2024-09-16T15:43:59.476881+0000","last_became_peered":"2024-09-16T15:43:59.476881+0000","last_unstale":"2024-09-16T15:44:01.556158+0000","last_undegraded":"2024-09-16T15:44:01.556158+0000","last_fullsized":"2024-09-16T15:44:01.556158+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:58.744809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556140+0000","last_change":"2024-09-16T15:43:59.462943+0000","last_active":"2024-09-16T15:44:01.556140+0000","last_peered":"2024-09-16T15:44:01.556140+0000","last_clean":"2024-09-16T15:44:01.556140+0000","last_became_active":"2024-09-16T15:43:59.462645+0000","last_became_peered":"2024-09-16T15:43:59.462645+0000","last_unstale":"2024-09-16T15:44:01.556140+0000","last_undegraded":"2024-09-16T15:44:01.556140+0000","last_fullsized":"2024-09-16T15:44:01.556140+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:05.976384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473597+0000","last_change":"2024-09-16T15:43:59.470360+0000","last_active":"2024-09-16T15:44:00.473597+0000","last_peered":"2024-09-16T15:44:00.473597+0000","last_clean":"2024-09-16T15:44:00.473597+0000","last_became_active":"2024-09-16T15:43:59.469917+0000","last_became_peered":"2024-09-16T15:43:59.469917+0000","last_unstale":"2024-09-16T15:44:00.473597+0000","last_undegraded":"2024-09-16T15:44:00.473597+0000","last_fullsized":"2024-09-16T15:44:00.473597+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:51.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555892+0000","last_change":"2024-09-16T15:43:59.479921+0000","last_active":"2024-09-16T15:44:01.555892+0000","last_peered":"2024-09-16T15:44:01.555892+0000","last_clean":"2024-09-16T15:44:01.555892+0000","last_became_active":"2024-09-16T15:43:59.471142+0000","last_became_peered":"2024-09-16T15:43:59.471142+0000","last_unstale":"2024-09-16T15:44:01.555892+0000","last_undegraded":"2024-09-16T15:44:01.555892+0000","last_fullsized":"2024-09-16T15:44:01.555892+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:51.837430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556701+0000","last_change":"2024-09-16T15:43:59.483135+0000","last_active":"2024-09-16T15:44:01.556701+0000","last_peered":"2024-09-16T15:44:01.556701+0000","last_clean":"2024-09-16T15:44:01.556701+0000","last_became_active":"2024-09-16T15:43:59.482501+0000","last_became_peered":"2024-09-16T15:43:59.482501+0000","last_unstale":"2024-09-16T15:44:01.556701+0000","last_undegraded":"2024-09-16T15:44:01.556701+0000","last_fullsized":"2024-09-16T15:44:01.556701+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:37.747156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556402+0000","last_change":"2024-09-16T15:43:59.474635+0000","last_active":"2024-09-16T15:44:01.556402+0000","last_peered":"2024-09-16T15:44:01.556402+0000","last_clean":"2024-09-16T15:44:01.556402+0000","last_became_active":"2024-09-16T15:43:59.474344+0000","last_became_peered":"2024-09-16T15:43:59.474344+0000","last_unstale":"2024-09-16T15:44:01.556402+0000","last_undegraded":"2024-09-16T15:44:01.556402+0000","last_fullsized":"2024-09-16T15:44:01.556402+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:25.374701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055230+0000","last_change":"2024-09-16T15:43:59.476480+0000","last_active":"2024-09-16T15:44:01.055230+0000","last_peered":"2024-09-16T15:44:01.055230+0000","last_clean":"2024-09-16T15:44:01.055230+0000","last_became_active":"2024-09-16T15:43:59.476198+0000","last_became_peered":"2024-09-16T15:43:59.476198+0000","last_unstale":"2024-09-16T15:44:01.055230+0000","last_undegraded":"2024-09-16T15:44:01.055230+0000","last_fullsized":"2024-09-16T15:44:01.055230+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:08:16.725732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556090+0000","last_change":"2024-09-16T15:43:59.477129+0000","last_active":"2024-09-16T15:44:01.556090+0000","last_peered":"2024-09-16T15:44:01.556090+0000","last_clean":"2024-09-16T15:44:01.556090+0000","last_became_active":"2024-09-16T15:43:59.476846+0000","last_became_peered":"2024-09-16T15:43:59.476846+0000","last_unstale":"2024-09-16T15:44:01.556090+0000","last_undegraded":"2024-09-16T15:44:01.556090+0000","last_fullsized":"2024-09-16T15:44:01.556090+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:44.221463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.474163+0000","last_change":"2024-09-16T15:43:59.468565+0000","last_active":"2024-09-16T15:44:00.474163+0000","last_peered":"2024-09-16T15:44:00.474163+0000","last_clean":"2024-09-16T15:44:00.474163+0000","last_became_active":"2024-09-16T15:43:59.466265+0000","last_became_peered":"2024-09-16T15:43:59.466265+0000","last_unstale":"2024-09-16T15:44:00.474163+0000","last_undegraded":"2024-09-16T15:44:00.474163+0000","last_fullsized":"2024-09-16T15:44:00.474163+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:42.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473005+0000","last_change":"2024-09-16T15:43:59.468595+0000","last_active":"2024-09-16T15:44:00.473005+0000","last_peered":"2024-09-16T15:44:00.473005+0000","last_clean":"2024-09-16T15:44:00.473005+0000","last_became_active":"2024-09-16T15:43:59.468317+0000","last_became_peered":"2024-09-16T15:43:59.468317+0000","last_unstale":"2024-09-16T15:44:00.473005+0000","last_undegraded":"2024-09-16T15:44:00.473005+0000","last_fullsized":"2024-09-16T15:44:00.473005+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:26.995398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556485+0000","last_change":"2024-09-16T15:43:59.486695+0000","last_active":"2024-09-16T15:44:01.556485+0000","last_peered":"2024-09-16T15:44:01.556485+0000","last_clean":"2024-09-16T15:44:01.556485+0000","last_became_active":"2024-09-16T15:43:59.486362+0000","last_became_peered":"2024-09-16T15:43:59.486362+0000","last_unstale":"2024-09-16T15:44:01.556485+0000","last_undegraded":"2024-09-16T15:44:01.556485+0000","last_fullsized":"2024-09-16T15:44:01.556485+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.643638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473346+0000","last_change":"2024-09-16T15:43:59.478374+0000","last_active":"2024-09-16T15:44:00.473346+0000","last_peered":"2024-09-16T15:44:00.473346+0000","last_clean":"2024-09-16T15:44:00.473346+0000","last_became_active":"2024-09-16T15:43:59.478094+0000","last_became_peered":"2024-09-16T15:43:59.478094+0000","last_unstale":"2024-09-16T15:44:00.473346+0000","last_undegraded":"2024-09-16T15:44:00.473346+0000","last_fullsized":"2024-09-16T15:44:00.473346+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:38.591497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473746+0000","last_change":"2024-09-16T15:43:59.471433+0000","last_active":"2024-09-16T15:44:00.473746+0000","last_peered":"2024-09-16T15:44:00.473746+0000","last_clean":"2024-09-16T15:44:00.473746+0000","last_became_active":"2024-09-16T15:43:59.471191+0000","last_became_peered":"2024-09-16T15:43:59.471191+0000","last_unstale":"2024-09-16T15:44:00.473746+0000","last_undegraded":"2024-09-16T15:44:00.473746+0000","last_fullsized":"2024-09-16T15:44:00.473746+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:49.043738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471042+0000","last_change":"2024-09-16T15:43:59.482370+0000","last_active":"2024-09-16T15:44:00.471042+0000","last_peered":"2024-09-16T15:44:00.471042+0000","last_clean":"2024-09-16T15:44:00.471042+0000","last_became_active":"2024-09-16T15:43:59.481993+0000","last_became_peered":"2024-09-16T15:43:59.481993+0000","last_unstale":"2024-09-16T15:44:00.471042+0000","last_undegraded":"2024-09-16T15:44:00.471042+0000","last_fullsized":"2024-09-16T15:44:00.471042+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:49.043738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472349+0000","last_change":"2024-09-16T15:43:59.476641+0000","last_active":"2024-09-16T15:44:00.472349+0000","last_peered":"2024-09-16T15:44:00.472349+0000","last_clean":"2024-09-16T15:44:00.472349+0000","last_became_active":"2024-09-16T15:43:59.475959+0000","last_became_peered":"2024-09-16T15:43:59.475959+0000","last_unstale":"2024-09-16T15:44:00.472349+0000","last_undegraded":"2024-09-16T15:44:00.472349+0000","last_fullsized":"2024-09-16T15:44:00.472349+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:40.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470573+0000","last_change":"2024-09-16T15:43:59.484050+0000","last_active":"2024-09-16T15:44:00.470573+0000","last_peered":"2024-09-16T15:44:00.470573+0000","last_clean":"2024-09-16T15:44:00.470573+0000","last_became_active":"2024-09-16T15:43:59.483416+0000","last_became_peered":"2024-09-16T15:43:59.483416+0000","last_unstale":"2024-09-16T15:44:00.470573+0000","last_undegraded":"2024-09-16T15:44:00.470573+0000","last_fullsized":"2024-09-16T15:44:00.470573+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:00.175329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.474292+0000","last_change":"2024-09-16T15:43:59.469434+0000","last_active":"2024-09-16T15:44:00.474292+0000","last_peered":"2024-09-16T15:44:00.474292+0000","last_clean":"2024-09-16T15:44:00.474292+0000","last_became_active":"2024-09-16T15:43:59.469160+0000","last_became_peered":"2024-09-16T15:43:59.469160+0000","last_unstale":"2024-09-16T15:44:00.474292+0000","last_undegraded":"2024-09-16T15:44:00.474292+0000","last_fullsized":"2024-09-16T15:44:00.474292+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:41.766268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556265+0000","last_change":"2024-09-16T15:43:59.474675+0000","last_active":"2024-09-16T15:44:01.556265+0000","last_peered":"2024-09-16T15:44:01.556265+0000","last_clean":"2024-09-16T15:44:01.556265+0000","last_became_active":"2024-09-16T15:43:59.468310+0000","last_became_peered":"2024-09-16T15:43:59.468310+0000","last_unstale":"2024-09-16T15:44:01.556265+0000","last_undegraded":"2024-09-16T15:44:01.556265+0000","last_fullsized":"2024-09-16T15:44:01.556265+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:13.346744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055123+0000","last_change":"2024-09-16T15:43:59.475927+0000","last_active":"2024-09-16T15:44:01.055123+0000","last_peered":"2024-09-16T15:44:01.055123+0000","last_clean":"2024-09-16T15:44:01.055123+0000","last_became_active":"2024-09-16T15:43:59.475640+0000","last_became_peered":"2024-09-16T15:43:59.475640+0000","last_unstale":"2024-09-16T15:44:01.055123+0000","last_undegraded":"2024-09-16T15:44:01.055123+0000","last_fullsized":"2024-09-16T15:44:01.055123+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:36.222941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470919+0000","last_change":"2024-09-16T15:43:59.494535+0000","last_active":"2024-09-16T15:44:00.470919+0000","last_peered":"2024-09-16T15:44:00.470919+0000","last_clean":"2024-09-16T15:44:00.470919+0000","last_became_active":"2024-09-16T15:43:59.494247+0000","last_became_peered":"2024-09-16T15:43:59.494247+0000","last_unstale":"2024-09-16T15:44:00.470919+0000","last_undegraded":"2024-09-16T15:44:00.470919+0000","last_fullsized":"2024-09-16T15:44:00.470919+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:51.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473012+0000","last_change":"2024-09-16T15:43:59.478603+0000","last_active":"2024-09-16T15:44:00.473012+0000","last_peered":"2024-09-16T15:44:00.473012+0000","last_clean":"2024-09-16T15:44:00.473012+0000","last_became_active":"2024-09-16T15:43:59.478236+0000","last_became_peered":"2024-09-16T15:43:59.478236+0000","last_unstale":"2024-09-16T15:44:00.473012+0000","last_undegraded":"2024-09-16T15:44:00.473012+0000","last_fullsized":"2024-09-16T15:44:00.473012+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:05:54.993800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476674+0000","last_change":"2024-09-16T15:43:59.476546+0000","last_active":"2024-09-16T15:44:00.476674+0000","last_peered":"2024-09-16T15:44:00.476674+0000","last_clean":"2024-09-16T15:44:00.476674+0000","last_became_active":"2024-09-16T15:43:59.476163+0000","last_became_peered":"2024-09-16T15:43:59.476163+0000","last_unstale":"2024-09-16T15:44:00.476674+0000","last_undegraded":"2024-09-16T15:44:00.476674+0000","last_fullsized":"2024-09-16T15:44:00.476674+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:41.491204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555566+0000","last_change":"2024-09-16T15:43:59.469532+0000","last_active":"2024-09-16T15:44:01.555566+0000","last_peered":"2024-09-16T15:44:01.555566+0000","last_clean":"2024-09-16T15:44:01.555566+0000","last_became_active":"2024-09-16T15:43:59.469274+0000","last_became_peered":"2024-09-16T15:43:59.469274+0000","last_unstale":"2024-09-16T15:44:01.555566+0000","last_undegraded":"2024-09-16T15:44:01.555566+0000","last_fullsized":"2024-09-16T15:44:01.555566+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:04:47.732366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.054867+0000","last_change":"2024-09-16T15:43:59.473551+0000","last_active":"2024-09-16T15:44:01.054867+0000","last_peered":"2024-09-16T15:44:01.054867+0000","last_clean":"2024-09-16T15:44:01.054867+0000","last_became_active":"2024-09-16T15:43:59.473238+0000","last_became_peered":"2024-09-16T15:43:59.473238+0000","last_unstale":"2024-09-16T15:44:01.054867+0000","last_undegraded":"2024-09-16T15:44:01.054867+0000","last_fullsized":"2024-09-16T15:44:01.054867+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:38.096575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055181+0000","last_change":"2024-09-16T15:43:59.488715+0000","last_active":"2024-09-16T15:44:01.055181+0000","last_peered":"2024-09-16T15:44:01.055181+0000","last_clean":"2024-09-16T15:44:01.055181+0000","last_became_active":"2024-09-16T15:43:59.476293+0000","last_became_peered":"2024-09-16T15:43:59.476293+0000","last_unstale":"2024-09-16T15:44:01.055181+0000","last_undegraded":"2024-09-16T15:44:01.055181+0000","last_fullsized":"2024-09-16T15:44:01.055181+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:34.270315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"106'4","reported_seq":19,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476811+0000","last_change":"2024-09-16T15:43:58.469234+0000","last_active":"2024-09-16T15:44:00.476811+0000","last_peered":"2024-09-16T15:44:00.476811+0000","last_clean":"2024-09-16T15:44:00.476811+0000","last_became_active":"2024-09-16T15:43:58.468952+0000","last_became_peered":"2024-09-16T15:43:58.468952+0000","last_unstale":"2024-09-16T15:44:00.476811+0000","last_undegraded":"2024-09-16T15:44:00.476811+0000","last_fullsized":"2024-09-16T15:44:00.476811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:08.539293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.474080+0000","last_change":"2024-09-16T15:43:59.480972+0000","last_active":"2024-09-16T15:44:00.474080+0000","last_peered":"2024-09-16T15:44:00.474080+0000","last_clean":"2024-09-16T15:44:00.474080+0000","last_became_active":"2024-09-16T15:43:59.479755+0000","last_became_peered":"2024-09-16T15:43:59.479755+0000","last_unstale":"2024-09-16T15:44:00.474080+0000","last_undegraded":"2024-09-16T15:44:00.474080+0000","last_fullsized":"2024-09-16T15:44:00.474080+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:28.033223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556549+0000","last_change":"2024-09-16T15:43:58.472137+0000","last_active":"2024-09-16T15:44:01.556549+0000","last_peered":"2024-09-16T15:44:01.556549+0000","last_clean":"2024-09-16T15:44:01.556549+0000","last_became_active":"2024-09-16T15:43:58.471742+0000","last_became_peered":"2024-09-16T15:43:58.471742+0000","last_unstale":"2024-09-16T15:44:01.556549+0000","last_undegraded":"2024-09-16T15:44:01.556549+0000","last_fullsized":"2024-09-16T15:44:01.556549+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:53.209805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055171+0000","last_change":"2024-09-16T15:43:59.475949+0000","last_active":"2024-09-16T15:44:01.055171+0000","last_peered":"2024-09-16T15:44:01.055171+0000","last_clean":"2024-09-16T15:44:01.055171+0000","last_became_active":"2024-09-16T15:43:59.475534+0000","last_became_peered":"2024-09-16T15:43:59.475534+0000","last_unstale":"2024-09-16T15:44:01.055171+0000","last_undegraded":"2024-09-16T15:44:01.055171+0000","last_fullsized":"2024-09-16T15:44:01.055171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:53.068227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041238+0000","last_change":"2024-09-16T15:43:58.471976+0000","last_active":"2024-09-16T15:44:01.041238+0000","last_peered":"2024-09-16T15:44:01.041238+0000","last_clean":"2024-09-16T15:44:01.041238+0000","last_became_active":"2024-09-16T15:43:58.471657+0000","last_became_peered":"2024-09-16T15:43:58.471657+0000","last_unstale":"2024-09-16T15:44:01.041238+0000","last_undegraded":"2024-09-16T15:44:01.041238+0000","last_fullsized":"2024-09-16T15:44:01.041238+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:47.177858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556748+0000","last_change":"2024-09-16T15:43:59.482562+0000","last_active":"2024-09-16T15:44:01.556748+0000","last_peered":"2024-09-16T15:44:01.556748+0000","last_clean":"2024-09-16T15:44:01.556748+0000","last_became_active":"2024-09-16T15:43:59.482299+0000","last_became_peered":"2024-09-16T15:43:59.482299+0000","last_unstale":"2024-09-16T15:44:01.556748+0000","last_undegraded":"2024-09-16T15:44:01.556748+0000","last_fullsized":"2024-09-16T15:44:01.556748+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:41:39.546585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041134+0000","last_change":"2024-09-16T15:43:58.472911+0000","last_active":"2024-09-16T15:44:01.041134+0000","last_peered":"2024-09-16T15:44:01.041134+0000","last_clean":"2024-09-16T15:44:01.041134+0000","last_became_active":"2024-09-16T15:43:58.472592+0000","last_became_peered":"2024-09-16T15:43:58.472592+0000","last_unstale":"2024-09-16T15:44:01.041134+0000","last_undegraded":"2024-09-16T15:44:01.041134+0000","last_fullsized":"2024-09-16T15:44:01.041134+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:43.817847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555485+0000","last_change":"2024-09-16T15:43:59.489826+0000","last_active":"2024-09-16T15:44:01.555485+0000","last_peered":"2024-09-16T15:44:01.555485+0000","last_clean":"2024-09-16T15:44:01.555485+0000","last_became_active":"2024-09-16T15:43:59.485191+0000","last_became_peered":"2024-09-16T15:43:59.485191+0000","last_unstale":"2024-09-16T15:44:01.555485+0000","last_undegraded":"2024-09-16T15:44:01.555485+0000","last_fullsized":"2024-09-16T15:44:01.555485+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:04.016419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476635+0000","last_change":"2024-09-16T15:43:58.471884+0000","last_active":"2024-09-16T15:44:00.476635+0000","last_peered":"2024-09-16T15:44:00.476635+0000","last_clean":"2024-09-16T15:44:00.476635+0000","last_became_active":"2024-09-16T15:43:58.471622+0000","last_became_peered":"2024-09-16T15:43:58.471622+0000","last_unstale":"2024-09-16T15:44:00.476635+0000","last_undegraded":"2024-09-16T15:44:00.476635+0000","last_fullsized":"2024-09-16T15:44:00.476635+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:37.836892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473241+0000","last_change":"2024-09-16T15:43:59.474923+0000","last_active":"2024-09-16T15:44:00.473241+0000","last_peered":"2024-09-16T15:44:00.473241+0000","last_clean":"2024-09-16T15:44:00.473241+0000","last_became_active":"2024-09-16T15:43:59.471817+0000","last_became_peered":"2024-09-16T15:43:59.471817+0000","last_unstale":"2024-09-16T15:44:00.473241+0000","last_undegraded":"2024-09-16T15:44:00.473241+0000","last_fullsized":"2024-09-16T15:44:00.473241+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:25.374701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471118+0000","last_change":"2024-09-16T15:43:58.479022+0000","last_active":"2024-09-16T15:44:00.471118+0000","last_peered":"2024-09-16T15:44:00.471118+0000","last_clean":"2024-09-16T15:44:00.471118+0000","last_became_active":"2024-09-16T15:43:58.478370+0000","last_became_peered":"2024-09-16T15:43:58.478370+0000","last_unstale":"2024-09-16T15:44:00.471118+0000","last_undegraded":"2024-09-16T15:44:00.471118+0000","last_fullsized":"2024-09-16T15:44:00.471118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:50:27.703032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555347+0000","last_change":"2024-09-16T15:43:59.464394+0000","last_active":"2024-09-16T15:44:01.555347+0000","last_peered":"2024-09-16T15:44:01.555347+0000","last_clean":"2024-09-16T15:44:01.555347+0000","last_became_active":"2024-09-16T15:43:59.464118+0000","last_became_peered":"2024-09-16T15:43:59.464118+0000","last_unstale":"2024-09-16T15:44:01.555347+0000","last_undegraded":"2024-09-16T15:44:01.555347+0000","last_fullsized":"2024-09-16T15:44:01.555347+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:14.250184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472752+0000","last_change":"2024-09-16T15:43:58.479166+0000","last_active":"2024-09-16T15:44:00.472752+0000","last_peered":"2024-09-16T15:44:00.472752+0000","last_clean":"2024-09-16T15:44:00.472752+0000","last_became_active":"2024-09-16T15:43:58.478789+0000","last_became_peered":"2024-09-16T15:43:58.478789+0000","last_unstale":"2024-09-16T15:44:00.472752+0000","last_undegraded":"2024-09-16T15:44:00.472752+0000","last_fullsized":"2024-09-16T15:44:00.472752+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:33.447055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476488+0000","last_change":"2024-09-16T15:43:59.476842+0000","last_active":"2024-09-16T15:44:00.476488+0000","last_peered":"2024-09-16T15:44:00.476488+0000","last_clean":"2024-09-16T15:44:00.476488+0000","last_became_active":"2024-09-16T15:43:59.476572+0000","last_became_peered":"2024-09-16T15:43:59.476572+0000","last_unstale":"2024-09-16T15:44:00.476488+0000","last_undegraded":"2024-09-16T15:44:00.476488+0000","last_fullsized":"2024-09-16T15:44:00.476488+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:54.217998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555467+0000","last_change":"2024-09-16T15:43:58.480976+0000","last_active":"2024-09-16T15:44:01.555467+0000","last_peered":"2024-09-16T15:44:01.555467+0000","last_clean":"2024-09-16T15:44:01.555467+0000","last_became_active":"2024-09-16T15:43:58.480667+0000","last_became_peered":"2024-09-16T15:43:58.480667+0000","last_unstale":"2024-09-16T15:44:01.555467+0000","last_undegraded":"2024-09-16T15:44:01.555467+0000","last_fullsized":"2024-09-16T15:44:01.555467+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:10.731783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.477062+0000","last_change":"2024-09-16T15:43:59.489293+0000","last_active":"2024-09-16T15:44:00.477062+0000","last_peered":"2024-09-16T15:44:00.477062+0000","last_clean":"2024-09-16T15:44:00.477062+0000","last_became_active":"2024-09-16T15:43:59.488916+0000","last_became_peered":"2024-09-16T15:43:59.488916+0000","last_unstale":"2024-09-16T15:44:00.477062+0000","last_undegraded":"2024-09-16T15:44:00.477062+0000","last_fullsized":"2024-09-16T15:44:00.477062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:53.281824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041284+0000","last_change":"2024-09-16T15:43:58.472206+0000","last_active":"2024-09-16T15:44:01.041284+0000","last_peered":"2024-09-16T15:44:01.041284+0000","last_clean":"2024-09-16T15:44:01.041284+0000","last_became_active":"2024-09-16T15:43:58.470989+0000","last_became_peered":"2024-09-16T15:43:58.470989+0000","last_unstale":"2024-09-16T15:44:01.041284+0000","last_undegraded":"2024-09-16T15:44:01.041284+0000","last_fullsized":"2024-09-16T15:44:01.041284+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:47.884966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.474043+0000","last_change":"2024-09-16T15:43:59.481271+0000","last_active":"2024-09-16T15:44:00.474043+0000","last_peered":"2024-09-16T15:44:00.474043+0000","last_clean":"2024-09-16T15:44:00.474043+0000","last_became_active":"2024-09-16T15:43:59.480235+0000","last_became_peered":"2024-09-16T15:43:59.480235+0000","last_unstale":"2024-09-16T15:44:00.474043+0000","last_undegraded":"2024-09-16T15:44:00.474043+0000","last_fullsized":"2024-09-16T15:44:00.474043+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:37.467453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470966+0000","last_change":"2024-09-16T15:43:58.475650+0000","last_active":"2024-09-16T15:44:00.470966+0000","last_peered":"2024-09-16T15:44:00.470966+0000","last_clean":"2024-09-16T15:44:00.470966+0000","last_became_active":"2024-09-16T15:43:58.475334+0000","last_became_peered":"2024-09-16T15:43:58.475334+0000","last_unstale":"2024-09-16T15:44:00.470966+0000","last_undegraded":"2024-09-16T15:44:00.470966+0000","last_fullsized":"2024-09-16T15:44:00.470966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:09.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471100+0000","last_change":"2024-09-16T15:43:59.486249+0000","last_active":"2024-09-16T15:44:00.471100+0000","last_peered":"2024-09-16T15:44:00.471100+0000","last_clean":"2024-09-16T15:44:00.471100+0000","last_became_active":"2024-09-16T15:43:59.485928+0000","last_became_peered":"2024-09-16T15:43:59.485928+0000","last_unstale":"2024-09-16T15:44:00.471100+0000","last_undegraded":"2024-09-16T15:44:00.471100+0000","last_fullsized":"2024-09-16T15:44:00.471100+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:42.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"106'3","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555803+0000","last_change":"2024-09-16T15:43:58.472242+0000","last_active":"2024-09-16T15:44:01.555803+0000","last_peered":"2024-09-16T15:44:01.555803+0000","last_clean":"2024-09-16T15:44:01.555803+0000","last_became_active":"2024-09-16T15:43:58.469338+0000","last_became_peered":"2024-09-16T15:43:58.469338+0000","last_unstale":"2024-09-16T15:44:01.555803+0000","last_undegraded":"2024-09-16T15:44:01.555803+0000","last_fullsized":"2024-09-16T15:44:01.555803+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:37.583304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476907+0000","last_change":"2024-09-16T15:43:59.483462+0000","last_active":"2024-09-16T15:44:00.476907+0000","last_peered":"2024-09-16T15:44:00.476907+0000","last_clean":"2024-09-16T15:44:00.476907+0000","last_became_active":"2024-09-16T15:43:59.482529+0000","last_became_peered":"2024-09-16T15:43:59.482529+0000","last_unstale":"2024-09-16T15:44:00.476907+0000","last_undegraded":"2024-09-16T15:44:00.476907+0000","last_fullsized":"2024-09-16T15:44:00.476907+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:38.047472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"106'6","reported_seq":21,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556016+0000","last_change":"2024-09-16T15:43:58.485208+0000","last_active":"2024-09-16T15:44:01.556016+0000","last_peered":"2024-09-16T15:44:01.556016+0000","last_clean":"2024-09-16T15:44:01.556016+0000","last_became_active":"2024-09-16T15:43:58.484737+0000","last_became_peered":"2024-09-16T15:43:58.484737+0000","last_unstale":"2024-09-16T15:44:01.556016+0000","last_undegraded":"2024-09-16T15:44:01.556016+0000","last_fullsized":"2024-09-16T15:44:01.556016+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:41.749653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471083+0000","last_change":"2024-09-16T15:43:59.476615+0000","last_active":"2024-09-16T15:44:00.471083+0000","last_peered":"2024-09-16T15:44:00.471083+0000","last_clean":"2024-09-16T15:44:00.471083+0000","last_became_active":"2024-09-16T15:43:59.476353+0000","last_became_peered":"2024-09-16T15:43:59.476353+0000","last_unstale":"2024-09-16T15:44:00.471083+0000","last_undegraded":"2024-09-16T15:44:00.471083+0000","last_fullsized":"2024-09-16T15:44:00.471083+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.262997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055070+0000","last_change":"2024-09-16T15:43:58.468614+0000","last_active":"2024-09-16T15:44:01.055070+0000","last_peered":"2024-09-16T15:44:01.055070+0000","last_clean":"2024-09-16T15:44:01.055070+0000","last_became_active":"2024-09-16T15:43:58.468318+0000","last_became_peered":"2024-09-16T15:43:58.468318+0000","last_unstale":"2024-09-16T15:44:01.055070+0000","last_undegraded":"2024-09-16T15:44:01.055070+0000","last_fullsized":"2024-09-16T15:44:01.055070+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:52.034867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470850+0000","last_change":"2024-09-16T15:43:59.489726+0000","last_active":"2024-09-16T15:44:00.470850+0000","last_peered":"2024-09-16T15:44:00.470850+0000","last_clean":"2024-09-16T15:44:00.470850+0000","last_became_active":"2024-09-16T15:43:59.489352+0000","last_became_peered":"2024-09-16T15:43:59.489352+0000","last_unstale":"2024-09-16T15:44:00.470850+0000","last_undegraded":"2024-09-16T15:44:00.470850+0000","last_fullsized":"2024-09-16T15:44:00.470850+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:11.739976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472667+0000","last_change":"2024-09-16T15:43:58.470226+0000","last_active":"2024-09-16T15:44:00.472667+0000","last_peered":"2024-09-16T15:44:00.472667+0000","last_clean":"2024-09-16T15:44:00.472667+0000","last_became_active":"2024-09-16T15:43:58.469820+0000","last_became_peered":"2024-09-16T15:43:58.469820+0000","last_unstale":"2024-09-16T15:44:00.472667+0000","last_undegraded":"2024-09-16T15:44:00.472667+0000","last_fullsized":"2024-09-16T15:44:00.472667+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:59.554215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470612+0000","last_change":"2024-09-16T15:43:59.472271+0000","last_active":"2024-09-16T15:44:00.470612+0000","last_peered":"2024-09-16T15:44:00.470612+0000","last_clean":"2024-09-16T15:44:00.470612+0000","last_became_active":"2024-09-16T15:43:59.472007+0000","last_became_peered":"2024-09-16T15:43:59.472007+0000","last_unstale":"2024-09-16T15:44:00.470612+0000","last_undegraded":"2024-09-16T15:44:00.470612+0000","last_fullsized":"2024-09-16T15:44:00.470612+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:25.374701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"106'8","reported_seq":23,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.040999+0000","last_change":"2024-09-16T15:43:58.480371+0000","last_active":"2024-09-16T15:44:01.040999+0000","last_peered":"2024-09-16T15:44:01.040999+0000","last_clean":"2024-09-16T15:44:01.040999+0000","last_became_active":"2024-09-16T15:43:58.476416+0000","last_became_peered":"2024-09-16T15:43:58.476416+0000","last_unstale":"2024-09-16T15:44:01.040999+0000","last_undegraded":"2024-09-16T15:44:01.040999+0000","last_fullsized":"2024-09-16T15:44:01.040999+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:51.830489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041342+0000","last_change":"2024-09-16T15:43:59.478862+0000","last_active":"2024-09-16T15:44:01.041342+0000","last_peered":"2024-09-16T15:44:01.041342+0000","last_clean":"2024-09-16T15:44:01.041342+0000","last_became_active":"2024-09-16T15:43:59.478414+0000","last_became_peered":"2024-09-16T15:43:59.478414+0000","last_unstale":"2024-09-16T15:44:01.041342+0000","last_undegraded":"2024-09-16T15:44:01.041342+0000","last_fullsized":"2024-09-16T15:44:01.041342+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:47.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556268+0000","last_change":"2024-09-16T15:43:58.471757+0000","last_active":"2024-09-16T15:44:01.556268+0000","last_peered":"2024-09-16T15:44:01.556268+0000","last_clean":"2024-09-16T15:44:01.556268+0000","last_became_active":"2024-09-16T15:43:58.471430+0000","last_became_peered":"2024-09-16T15:43:58.471430+0000","last_unstale":"2024-09-16T15:44:01.556268+0000","last_undegraded":"2024-09-16T15:44:01.556268+0000","last_fullsized":"2024-09-16T15:44:01.556268+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:15.254804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055156+0000","last_change":"2024-09-16T15:43:59.477287+0000","last_active":"2024-09-16T15:44:01.055156+0000","last_peered":"2024-09-16T15:44:01.055156+0000","last_clean":"2024-09-16T15:44:01.055156+0000","last_became_active":"2024-09-16T15:43:59.477036+0000","last_became_peered":"2024-09-16T15:43:59.477036+0000","last_unstale":"2024-09-16T15:44:01.055156+0000","last_undegraded":"2024-09-16T15:44:01.055156+0000","last_fullsized":"2024-09-16T15:44:01.055156+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:44.659898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041366+0000","last_change":"2024-09-16T15:43:58.464191+0000","last_active":"2024-09-16T15:44:01.041366+0000","last_peered":"2024-09-16T15:44:01.041366+0000","last_clean":"2024-09-16T15:44:01.041366+0000","last_became_active":"2024-09-16T15:43:58.463657+0000","last_became_peered":"2024-09-16T15:43:58.463657+0000","last_unstale":"2024-09-16T15:44:01.041366+0000","last_undegraded":"2024-09-16T15:44:01.041366+0000","last_fullsized":"2024-09-16T15:44:01.041366+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:59.290604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556463+0000","last_change":"2024-09-16T15:43:59.493233+0000","last_active":"2024-09-16T15:44:01.556463+0000","last_peered":"2024-09-16T15:44:01.556463+0000","last_clean":"2024-09-16T15:44:01.556463+0000","last_became_active":"2024-09-16T15:43:59.492958+0000","last_became_peered":"2024-09-16T15:43:59.492958+0000","last_unstale":"2024-09-16T15:44:01.556463+0000","last_undegraded":"2024-09-16T15:44:01.556463+0000","last_fullsized":"2024-09-16T15:44:01.556463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:38.591497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472895+0000","last_change":"2024-09-16T15:43:58.482245+0000","last_active":"2024-09-16T15:44:00.472895+0000","last_peered":"2024-09-16T15:44:00.472895+0000","last_clean":"2024-09-16T15:44:00.472895+0000","last_became_active":"2024-09-16T15:43:58.481889+0000","last_became_peered":"2024-09-16T15:43:58.481889+0000","last_unstale":"2024-09-16T15:44:00.472895+0000","last_undegraded":"2024-09-16T15:44:00.472895+0000","last_fullsized":"2024-09-16T15:44:00.472895+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:34:22.689014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556859+0000","last_change":"2024-09-16T15:43:59.479501+0000","last_active":"2024-09-16T15:44:01.556859+0000","last_peered":"2024-09-16T15:44:01.556859+0000","last_clean":"2024-09-16T15:44:01.556859+0000","last_became_active":"2024-09-16T15:43:59.479244+0000","last_became_peered":"2024-09-16T15:43:59.479244+0000","last_unstale":"2024-09-16T15:44:01.556859+0000","last_undegraded":"2024-09-16T15:44:01.556859+0000","last_fullsized":"2024-09-16T15:44:01.556859+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:43.013977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476547+0000","last_change":"2024-09-16T15:43:58.474662+0000","last_active":"2024-09-16T15:44:00.476547+0000","last_peered":"2024-09-16T15:44:00.476547+0000","last_clean":"2024-09-16T15:44:00.476547+0000","last_became_active":"2024-09-16T15:43:58.474360+0000","last_became_peered":"2024-09-16T15:43:58.474360+0000","last_unstale":"2024-09-16T15:44:00.476547+0000","last_undegraded":"2024-09-16T15:44:00.476547+0000","last_fullsized":"2024-09-16T15:44:00.476547+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:14.617651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556640+0000","last_change":"2024-09-16T15:43:59.486160+0000","last_active":"2024-09-16T15:44:01.556640+0000","last_peered":"2024-09-16T15:44:01.556640+0000","last_clean":"2024-09-16T15:44:01.556640+0000","last_became_active":"2024-09-16T15:43:59.484340+0000","last_became_peered":"2024-09-16T15:43:59.484340+0000","last_unstale":"2024-09-16T15:44:01.556640+0000","last_undegraded":"2024-09-16T15:44:01.556640+0000","last_fullsized":"2024-09-16T15:44:01.556640+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.262997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055176+0000","last_change":"2024-09-16T15:43:58.475912+0000","last_active":"2024-09-16T15:44:01.055176+0000","last_peered":"2024-09-16T15:44:01.055176+0000","last_clean":"2024-09-16T15:44:01.055176+0000","last_became_active":"2024-09-16T15:43:58.474401+0000","last_became_peered":"2024-09-16T15:43:58.474401+0000","last_unstale":"2024-09-16T15:44:01.055176+0000","last_undegraded":"2024-09-16T15:44:01.055176+0000","last_fullsized":"2024-09-16T15:44:01.055176+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:11.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555503+0000","last_change":"2024-09-16T15:43:59.481759+0000","last_active":"2024-09-16T15:44:01.555503+0000","last_peered":"2024-09-16T15:44:01.555503+0000","last_clean":"2024-09-16T15:44:01.555503+0000","last_became_active":"2024-09-16T15:43:59.481489+0000","last_became_peered":"2024-09-16T15:43:59.481489+0000","last_unstale":"2024-09-16T15:44:01.555503+0000","last_undegraded":"2024-09-16T15:44:01.555503+0000","last_fullsized":"2024-09-16T15:44:01.555503+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:54.738880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.040969+0000","last_change":"2024-09-16T15:43:58.472063+0000","last_active":"2024-09-16T15:44:01.040969+0000","last_peered":"2024-09-16T15:44:01.040969+0000","last_clean":"2024-09-16T15:44:01.040969+0000","last_became_active":"2024-09-16T15:43:58.471795+0000","last_became_peered":"2024-09-16T15:43:58.471795+0000","last_unstale":"2024-09-16T15:44:01.040969+0000","last_undegraded":"2024-09-16T15:44:01.040969+0000","last_fullsized":"2024-09-16T15:44:01.040969+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:26.162850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556666+0000","last_change":"2024-09-16T15:43:59.480668+0000","last_active":"2024-09-16T15:44:01.556666+0000","last_peered":"2024-09-16T15:44:01.556666+0000","last_clean":"2024-09-16T15:44:01.556666+0000","last_became_active":"2024-09-16T15:43:59.480333+0000","last_became_peered":"2024-09-16T15:43:59.480333+0000","last_unstale":"2024-09-16T15:44:01.556666+0000","last_undegraded":"2024-09-16T15:44:01.556666+0000","last_fullsized":"2024-09-16T15:44:01.556666+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:03.152682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.054818+0000","last_change":"2024-09-16T15:43:58.476976+0000","last_active":"2024-09-16T15:44:01.054818+0000","last_peered":"2024-09-16T15:44:01.054818+0000","last_clean":"2024-09-16T15:44:01.054818+0000","last_became_active":"2024-09-16T15:43:58.473880+0000","last_became_peered":"2024-09-16T15:43:58.473880+0000","last_unstale":"2024-09-16T15:44:01.054818+0000","last_undegraded":"2024-09-16T15:44:01.054818+0000","last_fullsized":"2024-09-16T15:44:01.054818+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:03.231111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470713+0000","last_change":"2024-09-16T15:43:59.477977+0000","last_active":"2024-09-16T15:44:00.470713+0000","last_peered":"2024-09-16T15:44:00.470713+0000","last_clean":"2024-09-16T15:44:00.470713+0000","last_became_active":"2024-09-16T15:43:59.477704+0000","last_became_peered":"2024-09-16T15:43:59.477704+0000","last_unstale":"2024-09-16T15:44:00.470713+0000","last_undegraded":"2024-09-16T15:44:00.470713+0000","last_fullsized":"2024-09-16T15:44:00.470713+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:38.591497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555562+0000","last_change":"2024-09-16T15:43:58.461681+0000","last_active":"2024-09-16T15:44:01.555562+0000","last_peered":"2024-09-16T15:44:01.555562+0000","last_clean":"2024-09-16T15:44:01.555562+0000","last_became_active":"2024-09-16T15:43:58.461347+0000","last_became_peered":"2024-09-16T15:43:58.461347+0000","last_unstale":"2024-09-16T15:44:01.555562+0000","last_undegraded":"2024-09-16T15:44:01.555562+0000","last_fullsized":"2024-09-16T15:44:01.555562+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:48.035545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473200+0000","last_change":"2024-09-16T15:43:59.480291+0000","last_active":"2024-09-16T15:44:00.473200+0000","last_peered":"2024-09-16T15:44:00.473200+0000","last_clean":"2024-09-16T15:44:00.473200+0000","last_became_active":"2024-09-16T15:43:59.480012+0000","last_became_peered":"2024-09-16T15:43:59.480012+0000","last_unstale":"2024-09-16T15:44:00.473200+0000","last_undegraded":"2024-09-16T15:44:00.473200+0000","last_fullsized":"2024-09-16T15:44:00.473200+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:22.295512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471030+0000","last_change":"2024-09-16T15:43:58.484782+0000","last_active":"2024-09-16T15:44:00.471030+0000","last_peered":"2024-09-16T15:44:00.471030+0000","last_clean":"2024-09-16T15:44:00.471030+0000","last_became_active":"2024-09-16T15:43:58.484452+0000","last_became_peered":"2024-09-16T15:43:58.484452+0000","last_unstale":"2024-09-16T15:44:00.471030+0000","last_undegraded":"2024-09-16T15:44:00.471030+0000","last_fullsized":"2024-09-16T15:44:00.471030+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:47:18.908240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556310+0000","last_change":"2024-09-16T15:43:59.473425+0000","last_active":"2024-09-16T15:44:01.556310+0000","last_peered":"2024-09-16T15:44:01.556310+0000","last_clean":"2024-09-16T15:44:01.556310+0000","last_became_active":"2024-09-16T15:43:59.473031+0000","last_became_peered":"2024-09-16T15:43:59.473031+0000","last_unstale":"2024-09-16T15:44:01.556310+0000","last_undegraded":"2024-09-16T15:44:01.556310+0000","last_fullsized":"2024-09-16T15:44:01.556310+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:32.494753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"107'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.475868+0000","last_change":"2024-09-16T15:43:58.466587+0000","last_active":"2024-09-16T15:44:00.475868+0000","last_peered":"2024-09-16T15:44:00.475868+0000","last_clean":"2024-09-16T15:44:00.475868+0000","last_became_active":"2024-09-16T15:43:58.466253+0000","last_became_peered":"2024-09-16T15:43:58.466253+0000","last_unstale":"2024-09-16T15:44:00.475868+0000","last_undegraded":"2024-09-16T15:44:00.475868+0000","last_fullsized":"2024-09-16T15:44:00.475868+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:36.207836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556043+0000","last_change":"2024-09-16T15:43:59.473821+0000","last_active":"2024-09-16T15:44:01.556043+0000","last_peered":"2024-09-16T15:44:01.556043+0000","last_clean":"2024-09-16T15:44:01.556043+0000","last_became_active":"2024-09-16T15:43:59.470840+0000","last_became_peered":"2024-09-16T15:43:59.470840+0000","last_unstale":"2024-09-16T15:44:01.556043+0000","last_undegraded":"2024-09-16T15:44:01.556043+0000","last_fullsized":"2024-09-16T15:44:01.556043+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:56.620811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555822+0000","last_change":"2024-09-16T15:43:58.465818+0000","last_active":"2024-09-16T15:44:01.555822+0000","last_peered":"2024-09-16T15:44:01.555822+0000","last_clean":"2024-09-16T15:44:01.555822+0000","last_became_active":"2024-09-16T15:43:58.462810+0000","last_became_peered":"2024-09-16T15:43:58.462810+0000","last_unstale":"2024-09-16T15:44:01.555822+0000","last_undegraded":"2024-09-16T15:44:01.555822+0000","last_fullsized":"2024-09-16T15:44:01.555822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:50.640173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472670+0000","last_change":"2024-09-16T15:43:59.481142+0000","last_active":"2024-09-16T15:44:00.472670+0000","last_peered":"2024-09-16T15:44:00.472670+0000","last_clean":"2024-09-16T15:44:00.472670+0000","last_became_active":"2024-09-16T15:43:59.480054+0000","last_became_peered":"2024-09-16T15:43:59.480054+0000","last_unstale":"2024-09-16T15:44:00.472670+0000","last_undegraded":"2024-09-16T15:44:00.472670+0000","last_fullsized":"2024-09-16T15:44:00.472670+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:57.811881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471198+0000","last_change":"2024-09-16T15:43:58.485091+0000","last_active":"2024-09-16T15:44:00.471198+0000","last_peered":"2024-09-16T15:44:00.471198+0000","last_clean":"2024-09-16T15:44:00.471198+0000","last_became_active":"2024-09-16T15:43:58.484831+0000","last_became_peered":"2024-09-16T15:43:58.484831+0000","last_unstale":"2024-09-16T15:44:00.471198+0000","last_undegraded":"2024-09-16T15:44:00.471198+0000","last_fullsized":"2024-09-16T15:44:00.471198+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:10.490944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473015+0000","last_change":"2024-09-16T15:43:59.477443+0000","last_active":"2024-09-16T15:44:00.473015+0000","last_peered":"2024-09-16T15:44:00.473015+0000","last_clean":"2024-09-16T15:44:00.473015+0000","last_became_active":"2024-09-16T15:43:59.477086+0000","last_became_peered":"2024-09-16T15:43:59.477086+0000","last_unstale":"2024-09-16T15:44:00.473015+0000","last_undegraded":"2024-09-16T15:44:00.473015+0000","last_fullsized":"2024-09-16T15:44:00.473015+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:36.761246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476662+0000","last_change":"2024-09-16T15:43:58.465408+0000","last_active":"2024-09-16T15:44:00.476662+0000","last_peered":"2024-09-16T15:44:00.476662+0000","last_clean":"2024-09-16T15:44:00.476662+0000","last_became_active":"2024-09-16T15:43:58.465136+0000","last_became_peered":"2024-09-16T15:43:58.465136+0000","last_unstale":"2024-09-16T15:44:00.476662+0000","last_undegraded":"2024-09-16T15:44:00.476662+0000","last_fullsized":"2024-09-16T15:44:00.476662+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:05:34.449120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473463+0000","last_change":"2024-09-16T15:43:59.466982+0000","last_active":"2024-09-16T15:44:00.473463+0000","last_peered":"2024-09-16T15:44:00.473463+0000","last_clean":"2024-09-16T15:44:00.473463+0000","last_became_active":"2024-09-16T15:43:59.466686+0000","last_became_peered":"2024-09-16T15:43:59.466686+0000","last_unstale":"2024-09-16T15:44:00.473463+0000","last_undegraded":"2024-09-16T15:44:00.473463+0000","last_fullsized":"2024-09-16T15:44:00.473463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:11.739976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473141+0000","last_change":"2024-09-16T15:43:58.470733+0000","last_active":"2024-09-16T15:44:00.473141+0000","last_peered":"2024-09-16T15:44:00.473141+0000","last_clean":"2024-09-16T15:44:00.473141+0000","last_became_active":"2024-09-16T15:43:58.470387+0000","last_became_peered":"2024-09-16T15:43:58.470387+0000","last_unstale":"2024-09-16T15:44:00.473141+0000","last_undegraded":"2024-09-16T15:44:00.473141+0000","last_fullsized":"2024-09-16T15:44:00.473141+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:42.765787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556839+0000","last_change":"2024-09-16T15:43:59.471899+0000","last_active":"2024-09-16T15:44:01.556839+0000","last_peered":"2024-09-16T15:44:01.556839+0000","last_clean":"2024-09-16T15:44:01.556839+0000","last_became_active":"2024-09-16T15:43:59.471564+0000","last_became_peered":"2024-09-16T15:43:59.471564+0000","last_unstale":"2024-09-16T15:44:01.556839+0000","last_undegraded":"2024-09-16T15:44:01.556839+0000","last_fullsized":"2024-09-16T15:44:01.556839+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:17.008581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.040991+0000","last_change":"2024-09-16T15:43:58.479731+0000","last_active":"2024-09-16T15:44:01.040991+0000","last_peered":"2024-09-16T15:44:01.040991+0000","last_clean":"2024-09-16T15:44:01.040991+0000","last_became_active":"2024-09-16T15:43:58.479210+0000","last_became_peered":"2024-09-16T15:43:58.479210+0000","last_unstale":"2024-09-16T15:44:01.040991+0000","last_undegraded":"2024-09-16T15:44:01.040991+0000","last_fullsized":"2024-09-16T15:44:01.040991+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:15.672087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555540+0000","last_change":"2024-09-16T15:43:59.472511+0000","last_active":"2024-09-16T15:44:01.555540+0000","last_peered":"2024-09-16T15:44:01.555540+0000","last_clean":"2024-09-16T15:44:01.555540+0000","last_became_active":"2024-09-16T15:43:59.470197+0000","last_became_peered":"2024-09-16T15:43:59.470197+0000","last_unstale":"2024-09-16T15:44:01.555540+0000","last_undegraded":"2024-09-16T15:44:01.555540+0000","last_fullsized":"2024-09-16T15:44:01.555540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:13.426429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555679+0000","last_change":"2024-09-16T15:43:58.463450+0000","last_active":"2024-09-16T15:44:01.555679+0000","last_peered":"2024-09-16T15:44:01.555679+0000","last_clean":"2024-09-16T15:44:01.555679+0000","last_became_active":"2024-09-16T15:43:58.462871+0000","last_became_peered":"2024-09-16T15:43:58.462871+0000","last_unstale":"2024-09-16T15:44:01.555679+0000","last_undegraded":"2024-09-16T15:44:01.555679+0000","last_fullsized":"2024-09-16T15:44:01.555679+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:40.758075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471243+0000","last_change":"2024-09-16T15:43:59.484283+0000","last_active":"2024-09-16T15:44:00.471243+0000","last_peered":"2024-09-16T15:44:00.471243+0000","last_clean":"2024-09-16T15:44:00.471243+0000","last_became_active":"2024-09-16T15:43:59.483785+0000","last_became_peered":"2024-09-16T15:43:59.483785+0000","last_unstale":"2024-09-16T15:44:00.471243+0000","last_undegraded":"2024-09-16T15:44:00.471243+0000","last_fullsized":"2024-09-16T15:44:00.471243+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:41.766268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476741+0000","last_change":"2024-09-16T15:43:58.476543+0000","last_active":"2024-09-16T15:44:00.476741+0000","last_peered":"2024-09-16T15:44:00.476741+0000","last_clean":"2024-09-16T15:44:00.476741+0000","last_became_active":"2024-09-16T15:43:58.476263+0000","last_became_peered":"2024-09-16T15:43:58.476263+0000","last_unstale":"2024-09-16T15:44:00.476741+0000","last_undegraded":"2024-09-16T15:44:00.476741+0000","last_fullsized":"2024-09-16T15:44:00.476741+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:22.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":293,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556806+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:44:01.556806+0000","last_peered":"2024-09-16T15:44:01.556806+0000","last_clean":"2024-09-16T15:44:01.556806+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:44:01.556806+0000","last_undegraded":"2024-09-16T15:44:01.556806+0000","last_fullsized":"2024-09-16T15:44:01.556806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1912832,"data_stored":1164236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1327104,"data_stored":579622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1331200,"data_stored":580156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":579552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509511,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1339392,"data_stored":581234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":579552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542216,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1331200,"data_stored":579676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1925120,"data_stored":1165344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:44:03.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:44:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.911+0000 7f20fe000640 1 -- 172.21.15.60:0/2859170034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 msgr2=0x7f20f80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.911+0000 7f20fe000640 1 --2- 172.21.15.60:0/2859170034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 0x7f20f80b2440 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f20f806dac0 tx=0x7f20e802a560 comp rx=0 tx=0).stop 2024-09-16T15:44:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.911+0000 7f20fe000640 1 -- 172.21.15.60:0/2859170034 shutdown_connections 2024-09-16T15:44:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.911+0000 7f20fe000640 1 --2- 172.21.15.60:0/2859170034 >> v2:172.21.15.100:3300/0 conn(0x7f20f8073fb0 0x7f20f8074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.911+0000 7f20fe000640 1 --2- 172.21.15.60:0/2859170034 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 0x7f20f80b2440 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.911+0000 7f20fe000640 1 -- 172.21.15.60:0/2859170034 >> 172.21.15.60:0/2859170034 conn(0x7f20f80b1950 msgr2=0x7f20f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:03.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.911+0000 7f20fe000640 1 -- 172.21.15.60:0/2859170034 shutdown_connections 2024-09-16T15:44:03.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.911+0000 7f20fe000640 1 -- 172.21.15.60:0/2859170034 wait complete. 2024-09-16T15:44:03.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.913+0000 7f20fe000640 1 Processor -- start 2024-09-16T15:44:03.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.913+0000 7f20fe000640 1 -- start start 2024-09-16T15:44:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20fe000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20f8073fb0 0x7f20f80c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20fe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f80ccf00 0x7f20f81549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20fe000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 0x7f20f809cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20fe000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20f806f2e0 con 0x7f20f815f5a0 2024-09-16T15:44:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20fe000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f20f806f160 con 0x7f20f80ccf00 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20fe000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f20f806f460 con 0x7f20f8073fb0 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20f7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 0x7f20f809cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20f7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 0x7f20f809cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49188/0 (socket says 172.21.15.60:49188) 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20f7e00640 1 -- 172.21.15.60:0/991529390 learned_addr learned my addr 172.21.15.60:0/991529390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20f7400640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20f8073fb0 0x7f20f80c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.914+0000 7f20eea00640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f80ccf00 0x7f20f81549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20f7e00640 1 -- 172.21.15.60:0/991529390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20f8073fb0 msgr2=0x7f20f80c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20f7e00640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20f8073fb0 0x7f20f80c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20f7e00640 1 -- 172.21.15.60:0/991529390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f80ccf00 msgr2=0x7f20f81549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20f7e00640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f80ccf00 0x7f20f81549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20f7e00640 1 -- 172.21.15.60:0/991529390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f81f5280 con 0x7f20f815f5a0 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20f7e00640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 0x7f20f809cbd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f20f000eaa0 tx=0x7f20f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20efe00640 1 -- 172.21.15.60:0/991529390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f000cdb0 con 0x7f20f815f5a0 2024-09-16T15:44:03.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20efe00640 1 -- 172.21.15.60:0/991529390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20f0004c10 con 0x7f20f815f5a0 2024-09-16T15:44:03.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20efe00640 1 -- 172.21.15.60:0/991529390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0005020 con 0x7f20f815f5a0 2024-09-16T15:44:03.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f81f5560 con 0x7f20f815f5a0 2024-09-16T15:44:03.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.915+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f80c92b0 con 0x7f20f815f5a0 2024-09-16T15:44:03.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.916+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f806f9f0 con 0x7f20f815f5a0 2024-09-16T15:44:03.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.918+0000 7f20efe00640 1 -- 172.21.15.60:0/991529390 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f20f00051c0 con 0x7f20f815f5a0 2024-09-16T15:44:03.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.919+0000 7f20efe00640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20d4078970 0x7f20d407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:03.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.919+0000 7f20efe00640 1 -- 172.21.15.60:0/991529390 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f20f009b5b0 con 0x7f20f815f5a0 2024-09-16T15:44:03.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.919+0000 7f20f7400640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20d4078970 0x7f20d407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:03.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.919+0000 7f20f7400640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20d4078970 0x7f20d407ae30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f20e8033000 tx=0x7f20e8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:03.923+0000 7f20efe00640 1 -- 172.21.15.60:0/991529390 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f20f0016030 con 0x7f20f815f5a0 2024-09-16T15:44:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.116+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f20f80cf590 con 0x7f20d4078970 2024-09-16T15:44:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.121+0000 7f20efe00640 1 -- 172.21.15.60:0/991529390 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7f20f80cf590 con 0x7f20d4078970 2024-09-16T15:44:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:44:04.126 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20d4078970 msgr2=0x7f20d407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20d4078970 0x7f20d407ae30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f20e8033000 tx=0x7f20e8004730 comp rx=0 tx=0).stop 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 msgr2=0x7f20f809cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 0x7f20f809cbd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f20f000eaa0 tx=0x7f20f000ef60 comp rx=0 tx=0).stop 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 shutdown_connections 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20d4078970 0x7f20d407ae30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20f815f5a0 0x7f20f809cbd0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20f80ccf00 0x7f20f81549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 --2- 172.21.15.60:0/991529390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20f8073fb0 0x7f20f80c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 >> 172.21.15.60:0/991529390 conn(0x7f20f80b1950 msgr2=0x7f20f80cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.125+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 shutdown_connections 2024-09-16T15:44:04.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.126+0000 7f20fe000640 1 -- 172.21.15.60:0/991529390 wait complete. 2024-09-16T15:44:04.142 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-16T15:44:02.172454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269392,"kb_used_data":11536,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754705328,"statfs":{"total":773094113280,"available":772818255872,"internally_reserved":0,"allocated":11812864,"data_stored":5809372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.729927"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556105+0000","last_change":"2024-09-16T15:43:59.488963+0000","last_active":"2024-09-16T15:44:01.556105+0000","last_peered":"2024-09-16T15:44:01.556105+0000","last_clean":"2024-09-16T15:44:01.556105+0000","last_became_active":"2024-09-16T15:43:59.484176+0000","last_became_peered":"2024-09-16T15:43:59.484176+0000","last_unstale":"2024-09-16T15:44:01.556105+0000","last_undegraded":"2024-09-16T15:44:01.556105+0000","last_fullsized":"2024-09-16T15:44:01.556105+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:11:55.436021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473764+0000","last_change":"2024-09-16T15:43:59.468062+0000","last_active":"2024-09-16T15:44:00.473764+0000","last_peered":"2024-09-16T15:44:00.473764+0000","last_clean":"2024-09-16T15:44:00.473764+0000","last_became_active":"2024-09-16T15:43:59.467771+0000","last_became_peered":"2024-09-16T15:43:59.467771+0000","last_unstale":"2024-09-16T15:44:00.473764+0000","last_undegraded":"2024-09-16T15:44:00.473764+0000","last_fullsized":"2024-09-16T15:44:00.473764+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.262997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556580+0000","last_change":"2024-09-16T15:43:59.485078+0000","last_active":"2024-09-16T15:44:01.556580+0000","last_peered":"2024-09-16T15:44:01.556580+0000","last_clean":"2024-09-16T15:44:01.556580+0000","last_became_active":"2024-09-16T15:43:59.484600+0000","last_became_peered":"2024-09-16T15:43:59.484600+0000","last_unstale":"2024-09-16T15:44:01.556580+0000","last_undegraded":"2024-09-16T15:44:01.556580+0000","last_fullsized":"2024-09-16T15:44:01.556580+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:33.214101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555822+0000","last_change":"2024-09-16T15:43:59.480908+0000","last_active":"2024-09-16T15:44:01.555822+0000","last_peered":"2024-09-16T15:44:01.555822+0000","last_clean":"2024-09-16T15:44:01.555822+0000","last_became_active":"2024-09-16T15:43:59.474701+0000","last_became_peered":"2024-09-16T15:43:59.474701+0000","last_unstale":"2024-09-16T15:44:01.555822+0000","last_undegraded":"2024-09-16T15:44:01.555822+0000","last_fullsized":"2024-09-16T15:44:01.555822+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:00.220826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041312+0000","last_change":"2024-09-16T15:43:59.482363+0000","last_active":"2024-09-16T15:44:01.041312+0000","last_peered":"2024-09-16T15:44:01.041312+0000","last_clean":"2024-09-16T15:44:01.041312+0000","last_became_active":"2024-09-16T15:43:59.482024+0000","last_became_peered":"2024-09-16T15:43:59.482024+0000","last_unstale":"2024-09-16T15:44:01.041312+0000","last_undegraded":"2024-09-16T15:44:01.041312+0000","last_fullsized":"2024-09-16T15:44:01.041312+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:59.706734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055074+0000","last_change":"2024-09-16T15:43:59.472680+0000","last_active":"2024-09-16T15:44:01.055074+0000","last_peered":"2024-09-16T15:44:01.055074+0000","last_clean":"2024-09-16T15:44:01.055074+0000","last_became_active":"2024-09-16T15:43:59.472359+0000","last_became_peered":"2024-09-16T15:43:59.472359+0000","last_unstale":"2024-09-16T15:44:01.055074+0000","last_undegraded":"2024-09-16T15:44:01.055074+0000","last_fullsized":"2024-09-16T15:44:01.055074+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:16.620530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.477022+0000","last_change":"2024-09-16T15:43:59.475220+0000","last_active":"2024-09-16T15:44:00.477022+0000","last_peered":"2024-09-16T15:44:00.477022+0000","last_clean":"2024-09-16T15:44:00.477022+0000","last_became_active":"2024-09-16T15:43:59.474814+0000","last_became_peered":"2024-09-16T15:43:59.474814+0000","last_unstale":"2024-09-16T15:44:00.477022+0000","last_undegraded":"2024-09-16T15:44:00.477022+0000","last_fullsized":"2024-09-16T15:44:00.477022+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:02:55.837680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556528+0000","last_change":"2024-09-16T15:43:59.483487+0000","last_active":"2024-09-16T15:44:01.556528+0000","last_peered":"2024-09-16T15:44:01.556528+0000","last_clean":"2024-09-16T15:44:01.556528+0000","last_became_active":"2024-09-16T15:43:59.483210+0000","last_became_peered":"2024-09-16T15:43:59.483210+0000","last_unstale":"2024-09-16T15:44:01.556528+0000","last_undegraded":"2024-09-16T15:44:01.556528+0000","last_fullsized":"2024-09-16T15:44:01.556528+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:40.325491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556158+0000","last_change":"2024-09-16T15:43:59.477160+0000","last_active":"2024-09-16T15:44:01.556158+0000","last_peered":"2024-09-16T15:44:01.556158+0000","last_clean":"2024-09-16T15:44:01.556158+0000","last_became_active":"2024-09-16T15:43:59.476881+0000","last_became_peered":"2024-09-16T15:43:59.476881+0000","last_unstale":"2024-09-16T15:44:01.556158+0000","last_undegraded":"2024-09-16T15:44:01.556158+0000","last_fullsized":"2024-09-16T15:44:01.556158+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:58.744809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556140+0000","last_change":"2024-09-16T15:43:59.462943+0000","last_active":"2024-09-16T15:44:01.556140+0000","last_peered":"2024-09-16T15:44:01.556140+0000","last_clean":"2024-09-16T15:44:01.556140+0000","last_became_active":"2024-09-16T15:43:59.462645+0000","last_became_peered":"2024-09-16T15:43:59.462645+0000","last_unstale":"2024-09-16T15:44:01.556140+0000","last_undegraded":"2024-09-16T15:44:01.556140+0000","last_fullsized":"2024-09-16T15:44:01.556140+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:05.976384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473597+0000","last_change":"2024-09-16T15:43:59.470360+0000","last_active":"2024-09-16T15:44:00.473597+0000","last_peered":"2024-09-16T15:44:00.473597+0000","last_clean":"2024-09-16T15:44:00.473597+0000","last_became_active":"2024-09-16T15:43:59.469917+0000","last_became_peered":"2024-09-16T15:43:59.469917+0000","last_unstale":"2024-09-16T15:44:00.473597+0000","last_undegraded":"2024-09-16T15:44:00.473597+0000","last_fullsized":"2024-09-16T15:44:00.473597+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:51.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555892+0000","last_change":"2024-09-16T15:43:59.479921+0000","last_active":"2024-09-16T15:44:01.555892+0000","last_peered":"2024-09-16T15:44:01.555892+0000","last_clean":"2024-09-16T15:44:01.555892+0000","last_became_active":"2024-09-16T15:43:59.471142+0000","last_became_peered":"2024-09-16T15:43:59.471142+0000","last_unstale":"2024-09-16T15:44:01.555892+0000","last_undegraded":"2024-09-16T15:44:01.555892+0000","last_fullsized":"2024-09-16T15:44:01.555892+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:51.837430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556701+0000","last_change":"2024-09-16T15:43:59.483135+0000","last_active":"2024-09-16T15:44:01.556701+0000","last_peered":"2024-09-16T15:44:01.556701+0000","last_clean":"2024-09-16T15:44:01.556701+0000","last_became_active":"2024-09-16T15:43:59.482501+0000","last_became_peered":"2024-09-16T15:43:59.482501+0000","last_unstale":"2024-09-16T15:44:01.556701+0000","last_undegraded":"2024-09-16T15:44:01.556701+0000","last_fullsized":"2024-09-16T15:44:01.556701+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:37.747156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556402+0000","last_change":"2024-09-16T15:43:59.474635+0000","last_active":"2024-09-16T15:44:01.556402+0000","last_peered":"2024-09-16T15:44:01.556402+0000","last_clean":"2024-09-16T15:44:01.556402+0000","last_became_active":"2024-09-16T15:43:59.474344+0000","last_became_peered":"2024-09-16T15:43:59.474344+0000","last_unstale":"2024-09-16T15:44:01.556402+0000","last_undegraded":"2024-09-16T15:44:01.556402+0000","last_fullsized":"2024-09-16T15:44:01.556402+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:25.374701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055230+0000","last_change":"2024-09-16T15:43:59.476480+0000","last_active":"2024-09-16T15:44:01.055230+0000","last_peered":"2024-09-16T15:44:01.055230+0000","last_clean":"2024-09-16T15:44:01.055230+0000","last_became_active":"2024-09-16T15:43:59.476198+0000","last_became_peered":"2024-09-16T15:43:59.476198+0000","last_unstale":"2024-09-16T15:44:01.055230+0000","last_undegraded":"2024-09-16T15:44:01.055230+0000","last_fullsized":"2024-09-16T15:44:01.055230+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:08:16.725732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556090+0000","last_change":"2024-09-16T15:43:59.477129+0000","last_active":"2024-09-16T15:44:01.556090+0000","last_peered":"2024-09-16T15:44:01.556090+0000","last_clean":"2024-09-16T15:44:01.556090+0000","last_became_active":"2024-09-16T15:43:59.476846+0000","last_became_peered":"2024-09-16T15:43:59.476846+0000","last_unstale":"2024-09-16T15:44:01.556090+0000","last_undegraded":"2024-09-16T15:44:01.556090+0000","last_fullsized":"2024-09-16T15:44:01.556090+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:44.221463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.474163+0000","last_change":"2024-09-16T15:43:59.468565+0000","last_active":"2024-09-16T15:44:00.474163+0000","last_peered":"2024-09-16T15:44:00.474163+0000","last_clean":"2024-09-16T15:44:00.474163+0000","last_became_active":"2024-09-16T15:43:59.466265+0000","last_became_peered":"2024-09-16T15:43:59.466265+0000","last_unstale":"2024-09-16T15:44:00.474163+0000","last_undegraded":"2024-09-16T15:44:00.474163+0000","last_fullsized":"2024-09-16T15:44:00.474163+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:42.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473005+0000","last_change":"2024-09-16T15:43:59.468595+0000","last_active":"2024-09-16T15:44:00.473005+0000","last_peered":"2024-09-16T15:44:00.473005+0000","last_clean":"2024-09-16T15:44:00.473005+0000","last_became_active":"2024-09-16T15:43:59.468317+0000","last_became_peered":"2024-09-16T15:43:59.468317+0000","last_unstale":"2024-09-16T15:44:00.473005+0000","last_undegraded":"2024-09-16T15:44:00.473005+0000","last_fullsized":"2024-09-16T15:44:00.473005+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:26.995398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556485+0000","last_change":"2024-09-16T15:43:59.486695+0000","last_active":"2024-09-16T15:44:01.556485+0000","last_peered":"2024-09-16T15:44:01.556485+0000","last_clean":"2024-09-16T15:44:01.556485+0000","last_became_active":"2024-09-16T15:43:59.486362+0000","last_became_peered":"2024-09-16T15:43:59.486362+0000","last_unstale":"2024-09-16T15:44:01.556485+0000","last_undegraded":"2024-09-16T15:44:01.556485+0000","last_fullsized":"2024-09-16T15:44:01.556485+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:55.643638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473346+0000","last_change":"2024-09-16T15:43:59.478374+0000","last_active":"2024-09-16T15:44:00.473346+0000","last_peered":"2024-09-16T15:44:00.473346+0000","last_clean":"2024-09-16T15:44:00.473346+0000","last_became_active":"2024-09-16T15:43:59.478094+0000","last_became_peered":"2024-09-16T15:43:59.478094+0000","last_unstale":"2024-09-16T15:44:00.473346+0000","last_undegraded":"2024-09-16T15:44:00.473346+0000","last_fullsized":"2024-09-16T15:44:00.473346+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:38.591497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473746+0000","last_change":"2024-09-16T15:43:59.471433+0000","last_active":"2024-09-16T15:44:00.473746+0000","last_peered":"2024-09-16T15:44:00.473746+0000","last_clean":"2024-09-16T15:44:00.473746+0000","last_became_active":"2024-09-16T15:43:59.471191+0000","last_became_peered":"2024-09-16T15:43:59.471191+0000","last_unstale":"2024-09-16T15:44:00.473746+0000","last_undegraded":"2024-09-16T15:44:00.473746+0000","last_fullsized":"2024-09-16T15:44:00.473746+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:49.043738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471042+0000","last_change":"2024-09-16T15:43:59.482370+0000","last_active":"2024-09-16T15:44:00.471042+0000","last_peered":"2024-09-16T15:44:00.471042+0000","last_clean":"2024-09-16T15:44:00.471042+0000","last_became_active":"2024-09-16T15:43:59.481993+0000","last_became_peered":"2024-09-16T15:43:59.481993+0000","last_unstale":"2024-09-16T15:44:00.471042+0000","last_undegraded":"2024-09-16T15:44:00.471042+0000","last_fullsized":"2024-09-16T15:44:00.471042+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:49.043738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472349+0000","last_change":"2024-09-16T15:43:59.476641+0000","last_active":"2024-09-16T15:44:00.472349+0000","last_peered":"2024-09-16T15:44:00.472349+0000","last_clean":"2024-09-16T15:44:00.472349+0000","last_became_active":"2024-09-16T15:43:59.475959+0000","last_became_peered":"2024-09-16T15:43:59.475959+0000","last_unstale":"2024-09-16T15:44:00.472349+0000","last_undegraded":"2024-09-16T15:44:00.472349+0000","last_fullsized":"2024-09-16T15:44:00.472349+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:40.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470573+0000","last_change":"2024-09-16T15:43:59.484050+0000","last_active":"2024-09-16T15:44:00.470573+0000","last_peered":"2024-09-16T15:44:00.470573+0000","last_clean":"2024-09-16T15:44:00.470573+0000","last_became_active":"2024-09-16T15:43:59.483416+0000","last_became_peered":"2024-09-16T15:43:59.483416+0000","last_unstale":"2024-09-16T15:44:00.470573+0000","last_undegraded":"2024-09-16T15:44:00.470573+0000","last_fullsized":"2024-09-16T15:44:00.470573+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:00.175329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.474292+0000","last_change":"2024-09-16T15:43:59.469434+0000","last_active":"2024-09-16T15:44:00.474292+0000","last_peered":"2024-09-16T15:44:00.474292+0000","last_clean":"2024-09-16T15:44:00.474292+0000","last_became_active":"2024-09-16T15:43:59.469160+0000","last_became_peered":"2024-09-16T15:43:59.469160+0000","last_unstale":"2024-09-16T15:44:00.474292+0000","last_undegraded":"2024-09-16T15:44:00.474292+0000","last_fullsized":"2024-09-16T15:44:00.474292+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:41.766268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556265+0000","last_change":"2024-09-16T15:43:59.474675+0000","last_active":"2024-09-16T15:44:01.556265+0000","last_peered":"2024-09-16T15:44:01.556265+0000","last_clean":"2024-09-16T15:44:01.556265+0000","last_became_active":"2024-09-16T15:43:59.468310+0000","last_became_peered":"2024-09-16T15:43:59.468310+0000","last_unstale":"2024-09-16T15:44:01.556265+0000","last_undegraded":"2024-09-16T15:44:01.556265+0000","last_fullsized":"2024-09-16T15:44:01.556265+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:13.346744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055123+0000","last_change":"2024-09-16T15:43:59.475927+0000","last_active":"2024-09-16T15:44:01.055123+0000","last_peered":"2024-09-16T15:44:01.055123+0000","last_clean":"2024-09-16T15:44:01.055123+0000","last_became_active":"2024-09-16T15:43:59.475640+0000","last_became_peered":"2024-09-16T15:43:59.475640+0000","last_unstale":"2024-09-16T15:44:01.055123+0000","last_undegraded":"2024-09-16T15:44:01.055123+0000","last_fullsized":"2024-09-16T15:44:01.055123+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:36.222941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470919+0000","last_change":"2024-09-16T15:43:59.494535+0000","last_active":"2024-09-16T15:44:00.470919+0000","last_peered":"2024-09-16T15:44:00.470919+0000","last_clean":"2024-09-16T15:44:00.470919+0000","last_became_active":"2024-09-16T15:43:59.494247+0000","last_became_peered":"2024-09-16T15:43:59.494247+0000","last_unstale":"2024-09-16T15:44:00.470919+0000","last_undegraded":"2024-09-16T15:44:00.470919+0000","last_fullsized":"2024-09-16T15:44:00.470919+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:51.419328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473012+0000","last_change":"2024-09-16T15:43:59.478603+0000","last_active":"2024-09-16T15:44:00.473012+0000","last_peered":"2024-09-16T15:44:00.473012+0000","last_clean":"2024-09-16T15:44:00.473012+0000","last_became_active":"2024-09-16T15:43:59.478236+0000","last_became_peered":"2024-09-16T15:43:59.478236+0000","last_unstale":"2024-09-16T15:44:00.473012+0000","last_undegraded":"2024-09-16T15:44:00.473012+0000","last_fullsized":"2024-09-16T15:44:00.473012+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:05:54.993800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476674+0000","last_change":"2024-09-16T15:43:59.476546+0000","last_active":"2024-09-16T15:44:00.476674+0000","last_peered":"2024-09-16T15:44:00.476674+0000","last_clean":"2024-09-16T15:44:00.476674+0000","last_became_active":"2024-09-16T15:43:59.476163+0000","last_became_peered":"2024-09-16T15:43:59.476163+0000","last_unstale":"2024-09-16T15:44:00.476674+0000","last_undegraded":"2024-09-16T15:44:00.476674+0000","last_fullsized":"2024-09-16T15:44:00.476674+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:41.491204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555566+0000","last_change":"2024-09-16T15:43:59.469532+0000","last_active":"2024-09-16T15:44:01.555566+0000","last_peered":"2024-09-16T15:44:01.555566+0000","last_clean":"2024-09-16T15:44:01.555566+0000","last_became_active":"2024-09-16T15:43:59.469274+0000","last_became_peered":"2024-09-16T15:43:59.469274+0000","last_unstale":"2024-09-16T15:44:01.555566+0000","last_undegraded":"2024-09-16T15:44:01.555566+0000","last_fullsized":"2024-09-16T15:44:01.555566+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:04:47.732366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.054867+0000","last_change":"2024-09-16T15:43:59.473551+0000","last_active":"2024-09-16T15:44:01.054867+0000","last_peered":"2024-09-16T15:44:01.054867+0000","last_clean":"2024-09-16T15:44:01.054867+0000","last_became_active":"2024-09-16T15:43:59.473238+0000","last_became_peered":"2024-09-16T15:43:59.473238+0000","last_unstale":"2024-09-16T15:44:01.054867+0000","last_undegraded":"2024-09-16T15:44:01.054867+0000","last_fullsized":"2024-09-16T15:44:01.054867+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:38.096575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055181+0000","last_change":"2024-09-16T15:43:59.488715+0000","last_active":"2024-09-16T15:44:01.055181+0000","last_peered":"2024-09-16T15:44:01.055181+0000","last_clean":"2024-09-16T15:44:01.055181+0000","last_became_active":"2024-09-16T15:43:59.476293+0000","last_became_peered":"2024-09-16T15:43:59.476293+0000","last_unstale":"2024-09-16T15:44:01.055181+0000","last_undegraded":"2024-09-16T15:44:01.055181+0000","last_fullsized":"2024-09-16T15:44:01.055181+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:34.270315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"106'4","reported_seq":19,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476811+0000","last_change":"2024-09-16T15:43:58.469234+0000","last_active":"2024-09-16T15:44:00.476811+0000","last_peered":"2024-09-16T15:44:00.476811+0000","last_clean":"2024-09-16T15:44:00.476811+0000","last_became_active":"2024-09-16T15:43:58.468952+0000","last_became_peered":"2024-09-16T15:43:58.468952+0000","last_unstale":"2024-09-16T15:44:00.476811+0000","last_undegraded":"2024-09-16T15:44:00.476811+0000","last_fullsized":"2024-09-16T15:44:00.476811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:08.539293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.474080+0000","last_change":"2024-09-16T15:43:59.480972+0000","last_active":"2024-09-16T15:44:00.474080+0000","last_peered":"2024-09-16T15:44:00.474080+0000","last_clean":"2024-09-16T15:44:00.474080+0000","last_became_active":"2024-09-16T15:43:59.479755+0000","last_became_peered":"2024-09-16T15:43:59.479755+0000","last_unstale":"2024-09-16T15:44:00.474080+0000","last_undegraded":"2024-09-16T15:44:00.474080+0000","last_fullsized":"2024-09-16T15:44:00.474080+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:28.033223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556549+0000","last_change":"2024-09-16T15:43:58.472137+0000","last_active":"2024-09-16T15:44:01.556549+0000","last_peered":"2024-09-16T15:44:01.556549+0000","last_clean":"2024-09-16T15:44:01.556549+0000","last_became_active":"2024-09-16T15:43:58.471742+0000","last_became_peered":"2024-09-16T15:43:58.471742+0000","last_unstale":"2024-09-16T15:44:01.556549+0000","last_undegraded":"2024-09-16T15:44:01.556549+0000","last_fullsized":"2024-09-16T15:44:01.556549+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:53.209805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055171+0000","last_change":"2024-09-16T15:43:59.475949+0000","last_active":"2024-09-16T15:44:01.055171+0000","last_peered":"2024-09-16T15:44:01.055171+0000","last_clean":"2024-09-16T15:44:01.055171+0000","last_became_active":"2024-09-16T15:43:59.475534+0000","last_became_peered":"2024-09-16T15:43:59.475534+0000","last_unstale":"2024-09-16T15:44:01.055171+0000","last_undegraded":"2024-09-16T15:44:01.055171+0000","last_fullsized":"2024-09-16T15:44:01.055171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:53.068227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041238+0000","last_change":"2024-09-16T15:43:58.471976+0000","last_active":"2024-09-16T15:44:01.041238+0000","last_peered":"2024-09-16T15:44:01.041238+0000","last_clean":"2024-09-16T15:44:01.041238+0000","last_became_active":"2024-09-16T15:43:58.471657+0000","last_became_peered":"2024-09-16T15:43:58.471657+0000","last_unstale":"2024-09-16T15:44:01.041238+0000","last_undegraded":"2024-09-16T15:44:01.041238+0000","last_fullsized":"2024-09-16T15:44:01.041238+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:41:47.177858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556748+0000","last_change":"2024-09-16T15:43:59.482562+0000","last_active":"2024-09-16T15:44:01.556748+0000","last_peered":"2024-09-16T15:44:01.556748+0000","last_clean":"2024-09-16T15:44:01.556748+0000","last_became_active":"2024-09-16T15:43:59.482299+0000","last_became_peered":"2024-09-16T15:43:59.482299+0000","last_unstale":"2024-09-16T15:44:01.556748+0000","last_undegraded":"2024-09-16T15:44:01.556748+0000","last_fullsized":"2024-09-16T15:44:01.556748+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:41:39.546585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041134+0000","last_change":"2024-09-16T15:43:58.472911+0000","last_active":"2024-09-16T15:44:01.041134+0000","last_peered":"2024-09-16T15:44:01.041134+0000","last_clean":"2024-09-16T15:44:01.041134+0000","last_became_active":"2024-09-16T15:43:58.472592+0000","last_became_peered":"2024-09-16T15:43:58.472592+0000","last_unstale":"2024-09-16T15:44:01.041134+0000","last_undegraded":"2024-09-16T15:44:01.041134+0000","last_fullsized":"2024-09-16T15:44:01.041134+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:43.817847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555485+0000","last_change":"2024-09-16T15:43:59.489826+0000","last_active":"2024-09-16T15:44:01.555485+0000","last_peered":"2024-09-16T15:44:01.555485+0000","last_clean":"2024-09-16T15:44:01.555485+0000","last_became_active":"2024-09-16T15:43:59.485191+0000","last_became_peered":"2024-09-16T15:43:59.485191+0000","last_unstale":"2024-09-16T15:44:01.555485+0000","last_undegraded":"2024-09-16T15:44:01.555485+0000","last_fullsized":"2024-09-16T15:44:01.555485+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:04.016419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476635+0000","last_change":"2024-09-16T15:43:58.471884+0000","last_active":"2024-09-16T15:44:00.476635+0000","last_peered":"2024-09-16T15:44:00.476635+0000","last_clean":"2024-09-16T15:44:00.476635+0000","last_became_active":"2024-09-16T15:43:58.471622+0000","last_became_peered":"2024-09-16T15:43:58.471622+0000","last_unstale":"2024-09-16T15:44:00.476635+0000","last_undegraded":"2024-09-16T15:44:00.476635+0000","last_fullsized":"2024-09-16T15:44:00.476635+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:37.836892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473241+0000","last_change":"2024-09-16T15:43:59.474923+0000","last_active":"2024-09-16T15:44:00.473241+0000","last_peered":"2024-09-16T15:44:00.473241+0000","last_clean":"2024-09-16T15:44:00.473241+0000","last_became_active":"2024-09-16T15:43:59.471817+0000","last_became_peered":"2024-09-16T15:43:59.471817+0000","last_unstale":"2024-09-16T15:44:00.473241+0000","last_undegraded":"2024-09-16T15:44:00.473241+0000","last_fullsized":"2024-09-16T15:44:00.473241+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:25.374701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471118+0000","last_change":"2024-09-16T15:43:58.479022+0000","last_active":"2024-09-16T15:44:00.471118+0000","last_peered":"2024-09-16T15:44:00.471118+0000","last_clean":"2024-09-16T15:44:00.471118+0000","last_became_active":"2024-09-16T15:43:58.478370+0000","last_became_peered":"2024-09-16T15:43:58.478370+0000","last_unstale":"2024-09-16T15:44:00.471118+0000","last_undegraded":"2024-09-16T15:44:00.471118+0000","last_fullsized":"2024-09-16T15:44:00.471118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:50:27.703032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555347+0000","last_change":"2024-09-16T15:43:59.464394+0000","last_active":"2024-09-16T15:44:01.555347+0000","last_peered":"2024-09-16T15:44:01.555347+0000","last_clean":"2024-09-16T15:44:01.555347+0000","last_became_active":"2024-09-16T15:43:59.464118+0000","last_became_peered":"2024-09-16T15:43:59.464118+0000","last_unstale":"2024-09-16T15:44:01.555347+0000","last_undegraded":"2024-09-16T15:44:01.555347+0000","last_fullsized":"2024-09-16T15:44:01.555347+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:14.250184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472752+0000","last_change":"2024-09-16T15:43:58.479166+0000","last_active":"2024-09-16T15:44:00.472752+0000","last_peered":"2024-09-16T15:44:00.472752+0000","last_clean":"2024-09-16T15:44:00.472752+0000","last_became_active":"2024-09-16T15:43:58.478789+0000","last_became_peered":"2024-09-16T15:43:58.478789+0000","last_unstale":"2024-09-16T15:44:00.472752+0000","last_undegraded":"2024-09-16T15:44:00.472752+0000","last_fullsized":"2024-09-16T15:44:00.472752+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:33.447055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476488+0000","last_change":"2024-09-16T15:43:59.476842+0000","last_active":"2024-09-16T15:44:00.476488+0000","last_peered":"2024-09-16T15:44:00.476488+0000","last_clean":"2024-09-16T15:44:00.476488+0000","last_became_active":"2024-09-16T15:43:59.476572+0000","last_became_peered":"2024-09-16T15:43:59.476572+0000","last_unstale":"2024-09-16T15:44:00.476488+0000","last_undegraded":"2024-09-16T15:44:00.476488+0000","last_fullsized":"2024-09-16T15:44:00.476488+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:54.217998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555467+0000","last_change":"2024-09-16T15:43:58.480976+0000","last_active":"2024-09-16T15:44:01.555467+0000","last_peered":"2024-09-16T15:44:01.555467+0000","last_clean":"2024-09-16T15:44:01.555467+0000","last_became_active":"2024-09-16T15:43:58.480667+0000","last_became_peered":"2024-09-16T15:43:58.480667+0000","last_unstale":"2024-09-16T15:44:01.555467+0000","last_undegraded":"2024-09-16T15:44:01.555467+0000","last_fullsized":"2024-09-16T15:44:01.555467+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:10.731783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.477062+0000","last_change":"2024-09-16T15:43:59.489293+0000","last_active":"2024-09-16T15:44:00.477062+0000","last_peered":"2024-09-16T15:44:00.477062+0000","last_clean":"2024-09-16T15:44:00.477062+0000","last_became_active":"2024-09-16T15:43:59.488916+0000","last_became_peered":"2024-09-16T15:43:59.488916+0000","last_unstale":"2024-09-16T15:44:00.477062+0000","last_undegraded":"2024-09-16T15:44:00.477062+0000","last_fullsized":"2024-09-16T15:44:00.477062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:53.281824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041284+0000","last_change":"2024-09-16T15:43:58.472206+0000","last_active":"2024-09-16T15:44:01.041284+0000","last_peered":"2024-09-16T15:44:01.041284+0000","last_clean":"2024-09-16T15:44:01.041284+0000","last_became_active":"2024-09-16T15:43:58.470989+0000","last_became_peered":"2024-09-16T15:43:58.470989+0000","last_unstale":"2024-09-16T15:44:01.041284+0000","last_undegraded":"2024-09-16T15:44:01.041284+0000","last_fullsized":"2024-09-16T15:44:01.041284+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:47.884966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.474043+0000","last_change":"2024-09-16T15:43:59.481271+0000","last_active":"2024-09-16T15:44:00.474043+0000","last_peered":"2024-09-16T15:44:00.474043+0000","last_clean":"2024-09-16T15:44:00.474043+0000","last_became_active":"2024-09-16T15:43:59.480235+0000","last_became_peered":"2024-09-16T15:43:59.480235+0000","last_unstale":"2024-09-16T15:44:00.474043+0000","last_undegraded":"2024-09-16T15:44:00.474043+0000","last_fullsized":"2024-09-16T15:44:00.474043+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:37.467453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470966+0000","last_change":"2024-09-16T15:43:58.475650+0000","last_active":"2024-09-16T15:44:00.470966+0000","last_peered":"2024-09-16T15:44:00.470966+0000","last_clean":"2024-09-16T15:44:00.470966+0000","last_became_active":"2024-09-16T15:43:58.475334+0000","last_became_peered":"2024-09-16T15:43:58.475334+0000","last_unstale":"2024-09-16T15:44:00.470966+0000","last_undegraded":"2024-09-16T15:44:00.470966+0000","last_fullsized":"2024-09-16T15:44:00.470966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:09.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471100+0000","last_change":"2024-09-16T15:43:59.486249+0000","last_active":"2024-09-16T15:44:00.471100+0000","last_peered":"2024-09-16T15:44:00.471100+0000","last_clean":"2024-09-16T15:44:00.471100+0000","last_became_active":"2024-09-16T15:43:59.485928+0000","last_became_peered":"2024-09-16T15:43:59.485928+0000","last_unstale":"2024-09-16T15:44:00.471100+0000","last_undegraded":"2024-09-16T15:44:00.471100+0000","last_fullsized":"2024-09-16T15:44:00.471100+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:42.757846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"106'3","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555803+0000","last_change":"2024-09-16T15:43:58.472242+0000","last_active":"2024-09-16T15:44:01.555803+0000","last_peered":"2024-09-16T15:44:01.555803+0000","last_clean":"2024-09-16T15:44:01.555803+0000","last_became_active":"2024-09-16T15:43:58.469338+0000","last_became_peered":"2024-09-16T15:43:58.469338+0000","last_unstale":"2024-09-16T15:44:01.555803+0000","last_undegraded":"2024-09-16T15:44:01.555803+0000","last_fullsized":"2024-09-16T15:44:01.555803+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:37.583304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476907+0000","last_change":"2024-09-16T15:43:59.483462+0000","last_active":"2024-09-16T15:44:00.476907+0000","last_peered":"2024-09-16T15:44:00.476907+0000","last_clean":"2024-09-16T15:44:00.476907+0000","last_became_active":"2024-09-16T15:43:59.482529+0000","last_became_peered":"2024-09-16T15:43:59.482529+0000","last_unstale":"2024-09-16T15:44:00.476907+0000","last_undegraded":"2024-09-16T15:44:00.476907+0000","last_fullsized":"2024-09-16T15:44:00.476907+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:38.047472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"106'6","reported_seq":21,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556016+0000","last_change":"2024-09-16T15:43:58.485208+0000","last_active":"2024-09-16T15:44:01.556016+0000","last_peered":"2024-09-16T15:44:01.556016+0000","last_clean":"2024-09-16T15:44:01.556016+0000","last_became_active":"2024-09-16T15:43:58.484737+0000","last_became_peered":"2024-09-16T15:43:58.484737+0000","last_unstale":"2024-09-16T15:44:01.556016+0000","last_undegraded":"2024-09-16T15:44:01.556016+0000","last_fullsized":"2024-09-16T15:44:01.556016+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:41.749653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471083+0000","last_change":"2024-09-16T15:43:59.476615+0000","last_active":"2024-09-16T15:44:00.471083+0000","last_peered":"2024-09-16T15:44:00.471083+0000","last_clean":"2024-09-16T15:44:00.471083+0000","last_became_active":"2024-09-16T15:43:59.476353+0000","last_became_peered":"2024-09-16T15:43:59.476353+0000","last_unstale":"2024-09-16T15:44:00.471083+0000","last_undegraded":"2024-09-16T15:44:00.471083+0000","last_fullsized":"2024-09-16T15:44:00.471083+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.262997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055070+0000","last_change":"2024-09-16T15:43:58.468614+0000","last_active":"2024-09-16T15:44:01.055070+0000","last_peered":"2024-09-16T15:44:01.055070+0000","last_clean":"2024-09-16T15:44:01.055070+0000","last_became_active":"2024-09-16T15:43:58.468318+0000","last_became_peered":"2024-09-16T15:43:58.468318+0000","last_unstale":"2024-09-16T15:44:01.055070+0000","last_undegraded":"2024-09-16T15:44:01.055070+0000","last_fullsized":"2024-09-16T15:44:01.055070+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:52.034867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470850+0000","last_change":"2024-09-16T15:43:59.489726+0000","last_active":"2024-09-16T15:44:00.470850+0000","last_peered":"2024-09-16T15:44:00.470850+0000","last_clean":"2024-09-16T15:44:00.470850+0000","last_became_active":"2024-09-16T15:43:59.489352+0000","last_became_peered":"2024-09-16T15:43:59.489352+0000","last_unstale":"2024-09-16T15:44:00.470850+0000","last_undegraded":"2024-09-16T15:44:00.470850+0000","last_fullsized":"2024-09-16T15:44:00.470850+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:11.739976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472667+0000","last_change":"2024-09-16T15:43:58.470226+0000","last_active":"2024-09-16T15:44:00.472667+0000","last_peered":"2024-09-16T15:44:00.472667+0000","last_clean":"2024-09-16T15:44:00.472667+0000","last_became_active":"2024-09-16T15:43:58.469820+0000","last_became_peered":"2024-09-16T15:43:58.469820+0000","last_unstale":"2024-09-16T15:44:00.472667+0000","last_undegraded":"2024-09-16T15:44:00.472667+0000","last_fullsized":"2024-09-16T15:44:00.472667+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:59.554215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470612+0000","last_change":"2024-09-16T15:43:59.472271+0000","last_active":"2024-09-16T15:44:00.470612+0000","last_peered":"2024-09-16T15:44:00.470612+0000","last_clean":"2024-09-16T15:44:00.470612+0000","last_became_active":"2024-09-16T15:43:59.472007+0000","last_became_peered":"2024-09-16T15:43:59.472007+0000","last_unstale":"2024-09-16T15:44:00.470612+0000","last_undegraded":"2024-09-16T15:44:00.470612+0000","last_fullsized":"2024-09-16T15:44:00.470612+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:25.374701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"106'8","reported_seq":23,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.040999+0000","last_change":"2024-09-16T15:43:58.480371+0000","last_active":"2024-09-16T15:44:01.040999+0000","last_peered":"2024-09-16T15:44:01.040999+0000","last_clean":"2024-09-16T15:44:01.040999+0000","last_became_active":"2024-09-16T15:43:58.476416+0000","last_became_peered":"2024-09-16T15:43:58.476416+0000","last_unstale":"2024-09-16T15:44:01.040999+0000","last_undegraded":"2024-09-16T15:44:01.040999+0000","last_fullsized":"2024-09-16T15:44:01.040999+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:51.830489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041342+0000","last_change":"2024-09-16T15:43:59.478862+0000","last_active":"2024-09-16T15:44:01.041342+0000","last_peered":"2024-09-16T15:44:01.041342+0000","last_clean":"2024-09-16T15:44:01.041342+0000","last_became_active":"2024-09-16T15:43:59.478414+0000","last_became_peered":"2024-09-16T15:43:59.478414+0000","last_unstale":"2024-09-16T15:44:01.041342+0000","last_undegraded":"2024-09-16T15:44:01.041342+0000","last_fullsized":"2024-09-16T15:44:01.041342+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:47.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556268+0000","last_change":"2024-09-16T15:43:58.471757+0000","last_active":"2024-09-16T15:44:01.556268+0000","last_peered":"2024-09-16T15:44:01.556268+0000","last_clean":"2024-09-16T15:44:01.556268+0000","last_became_active":"2024-09-16T15:43:58.471430+0000","last_became_peered":"2024-09-16T15:43:58.471430+0000","last_unstale":"2024-09-16T15:44:01.556268+0000","last_undegraded":"2024-09-16T15:44:01.556268+0000","last_fullsized":"2024-09-16T15:44:01.556268+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:15.254804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055156+0000","last_change":"2024-09-16T15:43:59.477287+0000","last_active":"2024-09-16T15:44:01.055156+0000","last_peered":"2024-09-16T15:44:01.055156+0000","last_clean":"2024-09-16T15:44:01.055156+0000","last_became_active":"2024-09-16T15:43:59.477036+0000","last_became_peered":"2024-09-16T15:43:59.477036+0000","last_unstale":"2024-09-16T15:44:01.055156+0000","last_undegraded":"2024-09-16T15:44:01.055156+0000","last_fullsized":"2024-09-16T15:44:01.055156+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:50:44.659898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.041366+0000","last_change":"2024-09-16T15:43:58.464191+0000","last_active":"2024-09-16T15:44:01.041366+0000","last_peered":"2024-09-16T15:44:01.041366+0000","last_clean":"2024-09-16T15:44:01.041366+0000","last_became_active":"2024-09-16T15:43:58.463657+0000","last_became_peered":"2024-09-16T15:43:58.463657+0000","last_unstale":"2024-09-16T15:44:01.041366+0000","last_undegraded":"2024-09-16T15:44:01.041366+0000","last_fullsized":"2024-09-16T15:44:01.041366+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:54:59.290604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556463+0000","last_change":"2024-09-16T15:43:59.493233+0000","last_active":"2024-09-16T15:44:01.556463+0000","last_peered":"2024-09-16T15:44:01.556463+0000","last_clean":"2024-09-16T15:44:01.556463+0000","last_became_active":"2024-09-16T15:43:59.492958+0000","last_became_peered":"2024-09-16T15:43:59.492958+0000","last_unstale":"2024-09-16T15:44:01.556463+0000","last_undegraded":"2024-09-16T15:44:01.556463+0000","last_fullsized":"2024-09-16T15:44:01.556463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:38.591497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472895+0000","last_change":"2024-09-16T15:43:58.482245+0000","last_active":"2024-09-16T15:44:00.472895+0000","last_peered":"2024-09-16T15:44:00.472895+0000","last_clean":"2024-09-16T15:44:00.472895+0000","last_became_active":"2024-09-16T15:43:58.481889+0000","last_became_peered":"2024-09-16T15:43:58.481889+0000","last_unstale":"2024-09-16T15:44:00.472895+0000","last_undegraded":"2024-09-16T15:44:00.472895+0000","last_fullsized":"2024-09-16T15:44:00.472895+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:34:22.689014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556859+0000","last_change":"2024-09-16T15:43:59.479501+0000","last_active":"2024-09-16T15:44:01.556859+0000","last_peered":"2024-09-16T15:44:01.556859+0000","last_clean":"2024-09-16T15:44:01.556859+0000","last_became_active":"2024-09-16T15:43:59.479244+0000","last_became_peered":"2024-09-16T15:43:59.479244+0000","last_unstale":"2024-09-16T15:44:01.556859+0000","last_undegraded":"2024-09-16T15:44:01.556859+0000","last_fullsized":"2024-09-16T15:44:01.556859+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:43.013977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476547+0000","last_change":"2024-09-16T15:43:58.474662+0000","last_active":"2024-09-16T15:44:00.476547+0000","last_peered":"2024-09-16T15:44:00.476547+0000","last_clean":"2024-09-16T15:44:00.476547+0000","last_became_active":"2024-09-16T15:43:58.474360+0000","last_became_peered":"2024-09-16T15:43:58.474360+0000","last_unstale":"2024-09-16T15:44:00.476547+0000","last_undegraded":"2024-09-16T15:44:00.476547+0000","last_fullsized":"2024-09-16T15:44:00.476547+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:14.617651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556640+0000","last_change":"2024-09-16T15:43:59.486160+0000","last_active":"2024-09-16T15:44:01.556640+0000","last_peered":"2024-09-16T15:44:01.556640+0000","last_clean":"2024-09-16T15:44:01.556640+0000","last_became_active":"2024-09-16T15:43:59.484340+0000","last_became_peered":"2024-09-16T15:43:59.484340+0000","last_unstale":"2024-09-16T15:44:01.556640+0000","last_undegraded":"2024-09-16T15:44:01.556640+0000","last_fullsized":"2024-09-16T15:44:01.556640+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:16.262997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.055176+0000","last_change":"2024-09-16T15:43:58.475912+0000","last_active":"2024-09-16T15:44:01.055176+0000","last_peered":"2024-09-16T15:44:01.055176+0000","last_clean":"2024-09-16T15:44:01.055176+0000","last_became_active":"2024-09-16T15:43:58.474401+0000","last_became_peered":"2024-09-16T15:43:58.474401+0000","last_unstale":"2024-09-16T15:44:01.055176+0000","last_undegraded":"2024-09-16T15:44:01.055176+0000","last_fullsized":"2024-09-16T15:44:01.055176+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:11.156767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555503+0000","last_change":"2024-09-16T15:43:59.481759+0000","last_active":"2024-09-16T15:44:01.555503+0000","last_peered":"2024-09-16T15:44:01.555503+0000","last_clean":"2024-09-16T15:44:01.555503+0000","last_became_active":"2024-09-16T15:43:59.481489+0000","last_became_peered":"2024-09-16T15:43:59.481489+0000","last_unstale":"2024-09-16T15:44:01.555503+0000","last_undegraded":"2024-09-16T15:44:01.555503+0000","last_fullsized":"2024-09-16T15:44:01.555503+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:54.738880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.040969+0000","last_change":"2024-09-16T15:43:58.472063+0000","last_active":"2024-09-16T15:44:01.040969+0000","last_peered":"2024-09-16T15:44:01.040969+0000","last_clean":"2024-09-16T15:44:01.040969+0000","last_became_active":"2024-09-16T15:43:58.471795+0000","last_became_peered":"2024-09-16T15:43:58.471795+0000","last_unstale":"2024-09-16T15:44:01.040969+0000","last_undegraded":"2024-09-16T15:44:01.040969+0000","last_fullsized":"2024-09-16T15:44:01.040969+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:26.162850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556666+0000","last_change":"2024-09-16T15:43:59.480668+0000","last_active":"2024-09-16T15:44:01.556666+0000","last_peered":"2024-09-16T15:44:01.556666+0000","last_clean":"2024-09-16T15:44:01.556666+0000","last_became_active":"2024-09-16T15:43:59.480333+0000","last_became_peered":"2024-09-16T15:43:59.480333+0000","last_unstale":"2024-09-16T15:44:01.556666+0000","last_undegraded":"2024-09-16T15:44:01.556666+0000","last_fullsized":"2024-09-16T15:44:01.556666+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:03.152682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.054818+0000","last_change":"2024-09-16T15:43:58.476976+0000","last_active":"2024-09-16T15:44:01.054818+0000","last_peered":"2024-09-16T15:44:01.054818+0000","last_clean":"2024-09-16T15:44:01.054818+0000","last_became_active":"2024-09-16T15:43:58.473880+0000","last_became_peered":"2024-09-16T15:43:58.473880+0000","last_unstale":"2024-09-16T15:44:01.054818+0000","last_undegraded":"2024-09-16T15:44:01.054818+0000","last_fullsized":"2024-09-16T15:44:01.054818+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:03.231111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.470713+0000","last_change":"2024-09-16T15:43:59.477977+0000","last_active":"2024-09-16T15:44:00.470713+0000","last_peered":"2024-09-16T15:44:00.470713+0000","last_clean":"2024-09-16T15:44:00.470713+0000","last_became_active":"2024-09-16T15:43:59.477704+0000","last_became_peered":"2024-09-16T15:43:59.477704+0000","last_unstale":"2024-09-16T15:44:00.470713+0000","last_undegraded":"2024-09-16T15:44:00.470713+0000","last_fullsized":"2024-09-16T15:44:00.470713+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:17:38.591497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555562+0000","last_change":"2024-09-16T15:43:58.461681+0000","last_active":"2024-09-16T15:44:01.555562+0000","last_peered":"2024-09-16T15:44:01.555562+0000","last_clean":"2024-09-16T15:44:01.555562+0000","last_became_active":"2024-09-16T15:43:58.461347+0000","last_became_peered":"2024-09-16T15:43:58.461347+0000","last_unstale":"2024-09-16T15:44:01.555562+0000","last_undegraded":"2024-09-16T15:44:01.555562+0000","last_fullsized":"2024-09-16T15:44:01.555562+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:48.035545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473200+0000","last_change":"2024-09-16T15:43:59.480291+0000","last_active":"2024-09-16T15:44:00.473200+0000","last_peered":"2024-09-16T15:44:00.473200+0000","last_clean":"2024-09-16T15:44:00.473200+0000","last_became_active":"2024-09-16T15:43:59.480012+0000","last_became_peered":"2024-09-16T15:43:59.480012+0000","last_unstale":"2024-09-16T15:44:00.473200+0000","last_undegraded":"2024-09-16T15:44:00.473200+0000","last_fullsized":"2024-09-16T15:44:00.473200+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:22.295512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471030+0000","last_change":"2024-09-16T15:43:58.484782+0000","last_active":"2024-09-16T15:44:00.471030+0000","last_peered":"2024-09-16T15:44:00.471030+0000","last_clean":"2024-09-16T15:44:00.471030+0000","last_became_active":"2024-09-16T15:43:58.484452+0000","last_became_peered":"2024-09-16T15:43:58.484452+0000","last_unstale":"2024-09-16T15:44:00.471030+0000","last_undegraded":"2024-09-16T15:44:00.471030+0000","last_fullsized":"2024-09-16T15:44:00.471030+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:47:18.908240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556310+0000","last_change":"2024-09-16T15:43:59.473425+0000","last_active":"2024-09-16T15:44:01.556310+0000","last_peered":"2024-09-16T15:44:01.556310+0000","last_clean":"2024-09-16T15:44:01.556310+0000","last_became_active":"2024-09-16T15:43:59.473031+0000","last_became_peered":"2024-09-16T15:43:59.473031+0000","last_unstale":"2024-09-16T15:44:01.556310+0000","last_undegraded":"2024-09-16T15:44:01.556310+0000","last_fullsized":"2024-09-16T15:44:01.556310+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:32.494753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"107'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.475868+0000","last_change":"2024-09-16T15:43:58.466587+0000","last_active":"2024-09-16T15:44:00.475868+0000","last_peered":"2024-09-16T15:44:00.475868+0000","last_clean":"2024-09-16T15:44:00.475868+0000","last_became_active":"2024-09-16T15:43:58.466253+0000","last_became_peered":"2024-09-16T15:43:58.466253+0000","last_unstale":"2024-09-16T15:44:00.475868+0000","last_undegraded":"2024-09-16T15:44:00.475868+0000","last_fullsized":"2024-09-16T15:44:00.475868+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:36.207836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556043+0000","last_change":"2024-09-16T15:43:59.473821+0000","last_active":"2024-09-16T15:44:01.556043+0000","last_peered":"2024-09-16T15:44:01.556043+0000","last_clean":"2024-09-16T15:44:01.556043+0000","last_became_active":"2024-09-16T15:43:59.470840+0000","last_became_peered":"2024-09-16T15:43:59.470840+0000","last_unstale":"2024-09-16T15:44:01.556043+0000","last_undegraded":"2024-09-16T15:44:01.556043+0000","last_fullsized":"2024-09-16T15:44:01.556043+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:56.620811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555822+0000","last_change":"2024-09-16T15:43:58.465818+0000","last_active":"2024-09-16T15:44:01.555822+0000","last_peered":"2024-09-16T15:44:01.555822+0000","last_clean":"2024-09-16T15:44:01.555822+0000","last_became_active":"2024-09-16T15:43:58.462810+0000","last_became_peered":"2024-09-16T15:43:58.462810+0000","last_unstale":"2024-09-16T15:44:01.555822+0000","last_undegraded":"2024-09-16T15:44:01.555822+0000","last_fullsized":"2024-09-16T15:44:01.555822+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:50.640173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.472670+0000","last_change":"2024-09-16T15:43:59.481142+0000","last_active":"2024-09-16T15:44:00.472670+0000","last_peered":"2024-09-16T15:44:00.472670+0000","last_clean":"2024-09-16T15:44:00.472670+0000","last_became_active":"2024-09-16T15:43:59.480054+0000","last_became_peered":"2024-09-16T15:43:59.480054+0000","last_unstale":"2024-09-16T15:44:00.472670+0000","last_undegraded":"2024-09-16T15:44:00.472670+0000","last_fullsized":"2024-09-16T15:44:00.472670+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:57.811881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471198+0000","last_change":"2024-09-16T15:43:58.485091+0000","last_active":"2024-09-16T15:44:00.471198+0000","last_peered":"2024-09-16T15:44:00.471198+0000","last_clean":"2024-09-16T15:44:00.471198+0000","last_became_active":"2024-09-16T15:43:58.484831+0000","last_became_peered":"2024-09-16T15:43:58.484831+0000","last_unstale":"2024-09-16T15:44:00.471198+0000","last_undegraded":"2024-09-16T15:44:00.471198+0000","last_fullsized":"2024-09-16T15:44:00.471198+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:10.490944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473015+0000","last_change":"2024-09-16T15:43:59.477443+0000","last_active":"2024-09-16T15:44:00.473015+0000","last_peered":"2024-09-16T15:44:00.473015+0000","last_clean":"2024-09-16T15:44:00.473015+0000","last_became_active":"2024-09-16T15:43:59.477086+0000","last_became_peered":"2024-09-16T15:43:59.477086+0000","last_unstale":"2024-09-16T15:44:00.473015+0000","last_undegraded":"2024-09-16T15:44:00.473015+0000","last_fullsized":"2024-09-16T15:44:00.473015+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:36.761246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476662+0000","last_change":"2024-09-16T15:43:58.465408+0000","last_active":"2024-09-16T15:44:00.476662+0000","last_peered":"2024-09-16T15:44:00.476662+0000","last_clean":"2024-09-16T15:44:00.476662+0000","last_became_active":"2024-09-16T15:43:58.465136+0000","last_became_peered":"2024-09-16T15:43:58.465136+0000","last_unstale":"2024-09-16T15:44:00.476662+0000","last_undegraded":"2024-09-16T15:44:00.476662+0000","last_fullsized":"2024-09-16T15:44:00.476662+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:05:34.449120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473463+0000","last_change":"2024-09-16T15:43:59.466982+0000","last_active":"2024-09-16T15:44:00.473463+0000","last_peered":"2024-09-16T15:44:00.473463+0000","last_clean":"2024-09-16T15:44:00.473463+0000","last_became_active":"2024-09-16T15:43:59.466686+0000","last_became_peered":"2024-09-16T15:43:59.466686+0000","last_unstale":"2024-09-16T15:44:00.473463+0000","last_undegraded":"2024-09-16T15:44:00.473463+0000","last_fullsized":"2024-09-16T15:44:00.473463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:11.739976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.473141+0000","last_change":"2024-09-16T15:43:58.470733+0000","last_active":"2024-09-16T15:44:00.473141+0000","last_peered":"2024-09-16T15:44:00.473141+0000","last_clean":"2024-09-16T15:44:00.473141+0000","last_became_active":"2024-09-16T15:43:58.470387+0000","last_became_peered":"2024-09-16T15:43:58.470387+0000","last_unstale":"2024-09-16T15:44:00.473141+0000","last_undegraded":"2024-09-16T15:44:00.473141+0000","last_fullsized":"2024-09-16T15:44:00.473141+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:42.765787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556839+0000","last_change":"2024-09-16T15:43:59.471899+0000","last_active":"2024-09-16T15:44:01.556839+0000","last_peered":"2024-09-16T15:44:01.556839+0000","last_clean":"2024-09-16T15:44:01.556839+0000","last_became_active":"2024-09-16T15:43:59.471564+0000","last_became_peered":"2024-09-16T15:43:59.471564+0000","last_unstale":"2024-09-16T15:44:01.556839+0000","last_undegraded":"2024-09-16T15:44:01.556839+0000","last_fullsized":"2024-09-16T15:44:01.556839+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:17.008581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.040991+0000","last_change":"2024-09-16T15:43:58.479731+0000","last_active":"2024-09-16T15:44:01.040991+0000","last_peered":"2024-09-16T15:44:01.040991+0000","last_clean":"2024-09-16T15:44:01.040991+0000","last_became_active":"2024-09-16T15:43:58.479210+0000","last_became_peered":"2024-09-16T15:43:58.479210+0000","last_unstale":"2024-09-16T15:44:01.040991+0000","last_undegraded":"2024-09-16T15:44:01.040991+0000","last_fullsized":"2024-09-16T15:44:01.040991+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:15.672087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555540+0000","last_change":"2024-09-16T15:43:59.472511+0000","last_active":"2024-09-16T15:44:01.555540+0000","last_peered":"2024-09-16T15:44:01.555540+0000","last_clean":"2024-09-16T15:44:01.555540+0000","last_became_active":"2024-09-16T15:43:59.470197+0000","last_became_peered":"2024-09-16T15:43:59.470197+0000","last_unstale":"2024-09-16T15:44:01.555540+0000","last_undegraded":"2024-09-16T15:44:01.555540+0000","last_fullsized":"2024-09-16T15:44:01.555540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:13.426429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.555679+0000","last_change":"2024-09-16T15:43:58.463450+0000","last_active":"2024-09-16T15:44:01.555679+0000","last_peered":"2024-09-16T15:44:01.555679+0000","last_clean":"2024-09-16T15:44:01.555679+0000","last_became_active":"2024-09-16T15:43:58.462871+0000","last_became_peered":"2024-09-16T15:43:58.462871+0000","last_unstale":"2024-09-16T15:44:01.555679+0000","last_undegraded":"2024-09-16T15:44:01.555679+0000","last_fullsized":"2024-09-16T15:44:01.555679+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:40.758075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.471243+0000","last_change":"2024-09-16T15:43:59.484283+0000","last_active":"2024-09-16T15:44:00.471243+0000","last_peered":"2024-09-16T15:44:00.471243+0000","last_clean":"2024-09-16T15:44:00.471243+0000","last_became_active":"2024-09-16T15:43:59.483785+0000","last_became_peered":"2024-09-16T15:43:59.483785+0000","last_unstale":"2024-09-16T15:44:00.471243+0000","last_undegraded":"2024-09-16T15:44:00.471243+0000","last_fullsized":"2024-09-16T15:44:00.471243+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:58.442288+0000","last_clean_scrub_stamp":"2024-09-16T15:43:58.442288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:41.766268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:00.476741+0000","last_change":"2024-09-16T15:43:58.476543+0000","last_active":"2024-09-16T15:44:00.476741+0000","last_peered":"2024-09-16T15:44:00.476741+0000","last_clean":"2024-09-16T15:44:00.476741+0000","last_became_active":"2024-09-16T15:43:58.476263+0000","last_became_peered":"2024-09-16T15:43:58.476263+0000","last_unstale":"2024-09-16T15:44:00.476741+0000","last_undegraded":"2024-09-16T15:44:00.476741+0000","last_fullsized":"2024-09-16T15:44:00.476741+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:43:57.434095+0000","last_clean_scrub_stamp":"2024-09-16T15:43:57.434095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:22.645019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":293,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T15:44:01.556806+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:44:01.556806+0000","last_peered":"2024-09-16T15:44:01.556806+0000","last_clean":"2024-09-16T15:44:01.556806+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:44:01.556806+0000","last_undegraded":"2024-09-16T15:44:01.556806+0000","last_fullsized":"2024-09-16T15:44:01.556806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1912832,"data_stored":1164236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1327104,"data_stored":579622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1331200,"data_stored":580156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":579552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509511,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1339392,"data_stored":581234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":579552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542216,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1331200,"data_stored":579676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37236,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334604,"statfs":{"total":96636764160,"available":96598634496,"internally_reserved":0,"allocated":1925120,"data_stored":1165344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:44:04.146 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:44:04.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:44:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.349+0000 7f1e91000640 1 -- 172.21.15.60:0/3818938463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4bd0 msgr2=0x7f1e8c0b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.349+0000 7f1e91000640 1 --2- 172.21.15.60:0/3818938463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4bd0 0x7f1e8c0b3ec0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0cd280 tx=0x7f1e7402a540 comp rx=0 tx=0).stop 2024-09-16T15:44:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.349+0000 7f1e91000640 1 -- 172.21.15.60:0/3818938463 shutdown_connections 2024-09-16T15:44:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.349+0000 7f1e91000640 1 --2- 172.21.15.60:0/3818938463 >> v2:172.21.15.100:3300/0 conn(0x7f1e8c0b4400 0x7f1e8c06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.349+0000 7f1e91000640 1 --2- 172.21.15.60:0/3818938463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4bd0 0x7f1e8c0b3ec0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.350+0000 7f1e91000640 1 -- 172.21.15.60:0/3818938463 >> 172.21.15.60:0/3818938463 conn(0x7f1e8c0b1840 msgr2=0x7f1e8c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.350+0000 7f1e91000640 1 -- 172.21.15.60:0/3818938463 shutdown_connections 2024-09-16T15:44:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.350+0000 7f1e91000640 1 -- 172.21.15.60:0/3818938463 wait complete. 2024-09-16T15:44:04.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.351+0000 7f1e91000640 1 Processor -- start 2024-09-16T15:44:04.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.351+0000 7f1e91000640 1 -- start start 2024-09-16T15:44:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e91000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4400 0x7f1e8c1f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e91000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e8c0b4bd0 0x7f1e8c1f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e91000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e8c1437c0 0x7f1e8c1f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e91000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c06ef20 con 0x7f1e8c0b4400 2024-09-16T15:44:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e91000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c06eda0 con 0x7f1e8c1437c0 2024-09-16T15:44:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e91000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8c06f0a0 con 0x7f1e8c0b4bd0 2024-09-16T15:44:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e89e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4400 0x7f1e8c1f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:04.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e89e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4400 0x7f1e8c1f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49208/0 (socket says 172.21.15.60:49208) 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.352+0000 7f1e89e00640 1 -- 172.21.15.60:0/4121334678 learned_addr learned my addr 172.21.15.60:0/4121334678 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e81400640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e8c0b4bd0 0x7f1e8c1f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e8a800640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e8c1437c0 0x7f1e8c1f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e89e00640 1 -- 172.21.15.60:0/4121334678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e8c0b4bd0 msgr2=0x7f1e8c1f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e89e00640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e8c0b4bd0 0x7f1e8c1f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e89e00640 1 -- 172.21.15.60:0/4121334678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e8c1437c0 msgr2=0x7f1e8c1f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e89e00640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e8c1437c0 0x7f1e8c1f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e89e00640 1 -- 172.21.15.60:0/4121334678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c15aab0 con 0x7f1e8c0b4400 2024-09-16T15:44:04.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e89e00640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4400 0x7f1e8c1f7f90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00c3a0 tx=0x7f1e7c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e82a00640 1 -- 172.21.15.60:0/4121334678 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c011790 con 0x7f1e8c0b4400 2024-09-16T15:44:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e82a00640 1 -- 172.21.15.60:0/4121334678 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e7c017070 con 0x7f1e8c0b4400 2024-09-16T15:44:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.353+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c159f30 con 0x7f1e8c0b4400 2024-09-16T15:44:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.354+0000 7f1e82a00640 1 -- 172.21.15.60:0/4121334678 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c011cc0 con 0x7f1e8c0b4400 2024-09-16T15:44:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.354+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c15a7d0 con 0x7f1e8c0b4400 2024-09-16T15:44:04.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.355+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c15a320 con 0x7f1e8c0b4400 2024-09-16T15:44:04.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.356+0000 7f1e82a00640 1 -- 172.21.15.60:0/4121334678 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f1e7c01c460 con 0x7f1e8c0b4400 2024-09-16T15:44:04.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.357+0000 7f1e82a00640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1e68078970 0x7f1e6807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:04.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.358+0000 7f1e82a00640 1 -- 172.21.15.60:0/4121334678 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f1e7c09a7b0 con 0x7f1e8c0b4400 2024-09-16T15:44:04.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.358+0000 7f1e81400640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1e68078970 0x7f1e6807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:04.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.358+0000 7f1e81400640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1e68078970 0x7f1e6807ae30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e74034000 tx=0x7f1e74004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.362+0000 7f1e82a00640 1 -- 172.21.15.60:0/4121334678 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1e7c0625a0 con 0x7f1e8c0b4400 2024-09-16T15:44:04.414 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-09-16T15:44:04.412+0000 7f0b16000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:44:04.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.630+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1e8c06e860 con 0x7f1e8c0b4400 2024-09-16T15:44:04.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.633+0000 7f1e82a00640 1 -- 172.21.15.60:0/4121334678 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f1e7c066fd0 con 0x7f1e8c0b4400 2024-09-16T15:44:04.634 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:44:04.635 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:44:04.635 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:44:04.635 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:44:04.635 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:44:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:44:04.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1e68078970 msgr2=0x7f1e6807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1e68078970 0x7f1e6807ae30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1e74034000 tx=0x7f1e74004730 comp rx=0 tx=0).stop 2024-09-16T15:44:04.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4400 msgr2=0x7f1e8c1f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4400 0x7f1e8c1f7f90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00c3a0 tx=0x7f1e7c00c860 comp rx=0 tx=0).stop 2024-09-16T15:44:04.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 shutdown_connections 2024-09-16T15:44:04.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1e68078970 0x7f1e6807ae30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e8c1437c0 0x7f1e8c1f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e8c0b4bd0 0x7f1e8c1f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.635+0000 7f1e91000640 1 --2- 172.21.15.60:0/4121334678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e8c0b4400 0x7f1e8c1f7f90 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.636+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 >> 172.21.15.60:0/4121334678 conn(0x7f1e8c0b1840 msgr2=0x7f1e8c0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.636+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 shutdown_connections 2024-09-16T15:44:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.636+0000 7f1e91000640 1 -- 172.21.15.60:0/4121334678 wait complete. 2024-09-16T15:44:04.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.813+0000 7fb324800640 1 -- 172.21.15.60:0/1358490401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ccf00 msgr2=0x7fb31c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.813+0000 7fb324800640 1 --2- 172.21.15.60:0/1358490401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ccf00 0x7fb31c0acd30 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb31c075130 tx=0x7fb30c02a560 comp rx=0 tx=0).stop 2024-09-16T15:44:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.813+0000 7fb324800640 1 -- 172.21.15.60:0/1358490401 shutdown_connections 2024-09-16T15:44:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.813+0000 7fb324800640 1 --2- 172.21.15.60:0/1358490401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ccf00 0x7fb31c0acd30 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.813+0000 7fb324800640 1 --2- 172.21.15.60:0/1358490401 >> v2:172.21.15.100:3300/0 conn(0x7fb31c15f5a0 0x7fb31c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.813+0000 7fb324800640 1 -- 172.21.15.60:0/1358490401 >> 172.21.15.60:0/1358490401 conn(0x7fb31c0b1950 msgr2=0x7fb31c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:04.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.813+0000 7fb324800640 1 -- 172.21.15.60:0/1358490401 shutdown_connections 2024-09-16T15:44:04.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.813+0000 7fb324800640 1 -- 172.21.15.60:0/1358490401 wait complete. 2024-09-16T15:44:04.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.815+0000 7fb324800640 1 Processor -- start 2024-09-16T15:44:04.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.815+0000 7fb324800640 1 -- start start 2024-09-16T15:44:04.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb324800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ad270 0x7fb31c1f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb324800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0ccf00 0x7fb31c14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb324800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c15f5a0 0x7fb31c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb324800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c06f160 con 0x7fb31c0ad270 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb324800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c06f2e0 con 0x7fb31c15f5a0 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb324800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c06f460 con 0x7fb31c0ccf00 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb321e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ad270 0x7fb31c1f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb321e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ad270 0x7fb31c1f8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49230/0 (socket says 172.21.15.60:49230) 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb321e00640 1 -- 172.21.15.60:0/985593491 learned_addr learned my addr 172.21.15.60:0/985593491 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb321400640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0ccf00 0x7fb31c14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.816+0000 7fb322800640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c15f5a0 0x7fb31c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb321e00640 1 -- 172.21.15.60:0/985593491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0ccf00 msgr2=0x7fb31c14d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb321e00640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0ccf00 0x7fb31c14d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb321e00640 1 -- 172.21.15.60:0/985593491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c15f5a0 msgr2=0x7fb31c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:04.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb321e00640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c15f5a0 0x7fb31c159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:04.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb321e00640 1 -- 172.21.15.60:0/985593491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c1539d0 con 0x7fb31c0ad270 2024-09-16T15:44:04.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb321e00640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ad270 0x7fb31c1f8310 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb31000d750 tx=0x7fb31000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb316000640 1 -- 172.21.15.60:0/985593491 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310017070 con 0x7fb31c0ad270 2024-09-16T15:44:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb316000640 1 -- 172.21.15.60:0/985593491 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb310004310 con 0x7fb31c0ad270 2024-09-16T15:44:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c1f89a0 con 0x7fb31c0ad270 2024-09-16T15:44:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb316000640 1 -- 172.21.15.60:0/985593491 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31000de60 con 0x7fb31c0ad270 2024-09-16T15:44:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.817+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c1f8f60 con 0x7fb31c0ad270 2024-09-16T15:44:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.818+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c1f4cf0 con 0x7fb31c0ad270 2024-09-16T15:44:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.820+0000 7fb316000640 1 -- 172.21.15.60:0/985593491 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fb310009c30 con 0x7fb31c0ad270 2024-09-16T15:44:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.820+0000 7fb316000640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb300078970 0x7fb30007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.821+0000 7fb316000640 1 -- 172.21.15.60:0/985593491 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fb31009ae70 con 0x7fb31c0ad270 2024-09-16T15:44:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.821+0000 7fb321400640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb300078970 0x7fb30007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:04.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.821+0000 7fb321400640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb300078970 0x7fb30007ae30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb30c033000 tx=0x7fb30c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:04.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:04.825+0000 7fb316000640 1 -- 172.21.15.60:0/985593491 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb310020020 con 0x7fb31c0ad270 2024-09-16T15:44:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.059+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb31c06f9f0 con 0x7fb31c0ad270 2024-09-16T15:44:05.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.059+0000 7fb316000640 1 -- 172.21.15.60:0/985593491 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7fb310062c60 con 0x7fb31c0ad270 2024-09-16T15:44:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:44:05.060 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"btime":"2024-09-16T15:44:02:480020+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/586556953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":586556953},{"type":"v1","addr":"172.21.15.100:6835","nonce":586556953}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2314857147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2314857147},{"type":"v1","addr":"172.21.15.60:6834","nonce":2314857147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1301436173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1301436173},{"type":"v1","addr":"172.21.15.60:6835","nonce":1301436173}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:58.903833+0000","modified":"2024-09-16T15:44:01.910153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6688":{"gid":6688,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6801/515865896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":515865896},{"type":"v1","addr":"172.21.15.100:6801","nonce":515865896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6688,"qdb_cluster":[6688]},"id":8}]} 2024-09-16T15:44:05.061 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb300078970 msgr2=0x7fb30007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb300078970 0x7fb30007ae30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb30c033000 tx=0x7fb30c004730 comp rx=0 tx=0).stop 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ad270 msgr2=0x7fb31c1f8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ad270 0x7fb31c1f8310 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb31000d750 tx=0x7fb31000dc10 comp rx=0 tx=0).stop 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 shutdown_connections 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb300078970 0x7fb30007ae30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c15f5a0 0x7fb31c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0ccf00 0x7fb31c14d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 --2- 172.21.15.60:0/985593491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c0ad270 0x7fb31c1f8310 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.061+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 >> 172.21.15.60:0/985593491 conn(0x7fb31c0b1950 msgr2=0x7fb31c15e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.062+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 shutdown_connections 2024-09-16T15:44:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.062+0000 7fb324800640 1 -- 172.21.15.60:0/985593491 wait complete. 2024-09-16T15:44:05.480 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:43:58.903833+0000', 'modified': '2024-09-16T15:44:01.910153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6688}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6688': {'gid': 6688, 'name': 'b', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6801/515865896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6800', 'nonce': 515865896}, {'type': 'v1', 'addr': '172.21.15.100:6801', 'nonce': 515865896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6688, 'qdb_cluster': [6688]} 2024-09-16T15:44:05.480 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:44:05.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-16T15:44:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.645+0000 7ff1c9200640 1 -- 172.21.15.60:0/148259925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 msgr2=0x7ff1a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.645+0000 7ff1c9200640 1 --2- 172.21.15.60:0/148259925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 0x7ff1a8004c80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00016c0 tx=0x7ff1a8005830 comp rx=0 tx=0).stop 2024-09-16T15:44:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.646+0000 7ff1c9200640 1 -- 172.21.15.60:0/148259925 shutdown_connections 2024-09-16T15:44:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.646+0000 7ff1c9200640 1 --2- 172.21.15.60:0/148259925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 0x7ff1a8004c80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.646+0000 7ff1c9200640 1 -- 172.21.15.60:0/148259925 >> 172.21.15.60:0/148259925 conn(0x7ff1c40d3df0 msgr2=0x7ff1c40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.646+0000 7ff1c9200640 1 -- 172.21.15.60:0/148259925 shutdown_connections 2024-09-16T15:44:05.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.646+0000 7ff1c9200640 1 -- 172.21.15.60:0/148259925 wait complete. 2024-09-16T15:44:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.647+0000 7ff1c9200640 1 Processor -- start 2024-09-16T15:44:05.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.647+0000 7ff1c9200640 1 -- start start 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.648+0000 7ff1c9200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 0x7ff1c40b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.648+0000 7ff1c9200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c40b77b0 0x7ff1c40ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.648+0000 7ff1c9200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1c40cd0c0 0x7ff1c40d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.648+0000 7ff1c9200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c40b1b00 con 0x7ff1a8004800 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.648+0000 7ff1c9200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c40b1980 con 0x7ff1c40b77b0 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.648+0000 7ff1c9200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c40b1c80 con 0x7ff1c40cd0c0 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.648+0000 7ff1c3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 0x7ff1c40b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 0x7ff1c40b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49244/0 (socket says 172.21.15.60:49244) 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3400640 1 -- 172.21.15.60:0/3969004949 learned_addr learned my addr 172.21.15.60:0/3969004949 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c40b77b0 0x7ff1c40ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1c40cd0c0 0x7ff1c40d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3400640 1 -- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1c40cd0c0 msgr2=0x7ff1c40d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3400640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1c40cd0c0 0x7ff1c40d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3400640 1 -- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c40b77b0 msgr2=0x7ff1c40ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3400640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c40b77b0 0x7ff1c40ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3400640 1 -- 172.21.15.60:0/3969004949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c41f7fe0 con 0x7ff1a8004800 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c40b77b0 0x7ff1c40ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1c40cd0c0 0x7ff1c40d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1c3400640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 0x7ff1c40b6c20 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006630 tx=0x7ff1b40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.649+0000 7ff1bb400640 1 -- 172.21.15.60:0/3969004949 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4015070 con 0x7ff1a8004800 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.650+0000 7ff1bb400640 1 -- 172.21.15.60:0/3969004949 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1b40119c0 con 0x7ff1a8004800 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.650+0000 7ff1c9200640 1 -- 172.21.15.60:0/3969004949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c41f82c0 con 0x7ff1a8004800 2024-09-16T15:44:05.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.650+0000 7ff1bb400640 1 -- 172.21.15.60:0/3969004949 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4011e10 con 0x7ff1a8004800 2024-09-16T15:44:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.650+0000 7ff1c9200640 1 -- 172.21.15.60:0/3969004949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c41f8880 con 0x7ff1a8004800 2024-09-16T15:44:05.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.652+0000 7ff1bb400640 1 -- 172.21.15.60:0/3969004949 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7ff1b4018070 con 0x7ff1a8004800 2024-09-16T15:44:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.653+0000 7ff1bb400640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff1a40789a0 0x7ff1a407ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.653+0000 7ff1bb400640 1 -- 172.21.15.60:0/3969004949 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7ff1b409b9c0 con 0x7ff1a8004800 2024-09-16T15:44:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.653+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff1a40789a0 0x7ff1a407ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.653+0000 7ff1cd685740 1 Processor -- start 2024-09-16T15:44:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.653+0000 7ff1cd685740 1 -- start start 2024-09-16T15:44:05.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.654+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff1a40789a0 0x7ff1a407ae60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0032000 tx=0x7ff1b0001190 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.654+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad1b0aa0 0x5564ad1b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.654+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16bc10 0x5564ad1af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.654+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad1afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.654+0000 7ff1cd685740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5564ad0ee430 con 0x5564ad16bc10 2024-09-16T15:44:05.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.654+0000 7ff1cd685740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5564ad0de5d0 con 0x5564ad1b0aa0 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.654+0000 7ff1cd685740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5564ad0cda00 con 0x5564ad16c190 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.654+0000 7ff1c2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16bc10 0x5564ad1af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16bc10 0x5564ad1af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49250/0 (socket says 172.21.15.60:49250) 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c2a00640 1 -- 172.21.15.60:0/558272376 learned_addr learned my addr 172.21.15.60:0/558272376 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c3400640 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad1b0aa0 0x5564ad1b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad1afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c2a00640 1 -- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 msgr2=0x5564ad1afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad1afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c2a00640 1 -- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad1b0aa0 msgr2=0x5564ad1b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad1b0aa0 0x5564ad1b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c2a00640 1 -- 172.21.15.60:0/558272376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ad096520 con 0x5564ad16bc10 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad1afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:05.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16bc10 0x5564ad1af580 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006e10 tx=0x7ff1b0039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46630ff989d0996b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.655+0000 7ff1afe00640 1 -- 172.21.15.60:0/558272376 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0003c40 con 0x5564ad16bc10 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1afe00640 1 -- 172.21.15.60:0/558272376 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1b0039070 con 0x5564ad16bc10 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1afe00640 1 -- 172.21.15.60:0/558272376 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b003d690 con 0x5564ad16bc10 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 -- 172.21.15.60:0/558272376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16bc10 msgr2=0x5564ad1af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16bc10 0x5564ad1af580 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006e10 tx=0x7ff1b0039000 comp rx=0 tx=0).stop 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 -- 172.21.15.60:0/558272376 shutdown_connections 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad1afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16bc10 0x5564ad1af580 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 --2- 172.21.15.60:0/558272376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad1b0aa0 0x5564ad1b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 -- 172.21.15.60:0/558272376 >> 172.21.15.60:0/558272376 conn(0x5564ad1671e0 msgr2=0x5564ad16ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 -- 172.21.15.60:0/558272376 shutdown_connections 2024-09-16T15:44:05.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.656+0000 7ff1cd685740 1 -- 172.21.15.60:0/558272376 wait complete. 2024-09-16T15:44:05.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.657+0000 7ff1cd685740 1 -- start start 2024-09-16T15:44:05.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.657+0000 7ff1cd685740 1 Processor -- start 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.658+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad16c190 0x5564ad1945e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.658+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad1af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.658+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad194b20 0x5564ad1afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.658+0000 7ff1cd685740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5564ad074330 con 0x5564ad194b20 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.658+0000 7ff1cd685740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5564ad0d1810 con 0x5564ad16c190 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.658+0000 7ff1cd685740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5564ace3e620 con 0x5564ad1b0aa0 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.658+0000 7ff1cd685740 10 client.0 _ll_register_callbacks cb 0x5564ad079420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad194b20 0x5564ad1afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad194b20 0x5564ad1afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49258/0 (socket says 172.21.15.60:49258) 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 -- 172.21.15.60:0/2065003583 learned_addr learned my addr 172.21.15.60:0/2065003583 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad1af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3400640 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad16c190 0x5564ad1945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 -- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 msgr2=0x5564ad1af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:05.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad1af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 -- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad16c190 msgr2=0x5564ad1945e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad16c190 0x5564ad1945e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 -- 172.21.15.60:0/2065003583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ace2f850 con 0x5564ad194b20 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3400640 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad16c190 0x5564ad1945e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad194b20 0x5564ad1afd70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc005fd0 tx=0x7ff1bc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c977f6fca03928b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1acc00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1acc00640 1 -- 172.21.15.60:0/2065003583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc004d20 con 0x5564ad194b20 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.659+0000 7ff1acc00640 1 -- 172.21.15.60:0/2065003583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7ff1bc0137f0 con 0x5564ad194b20 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 1 -- 172.21.15.60:0/2065003583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b0039070 con 0x5564ad194b20 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1acc00640 1 -- 172.21.15.60:0/2065003583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7ff1bc023050 con 0x5564ad194b20 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 10 client.7044 fetch_fsmap learned FSMap version 71 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 1 -- 172.21.15.60:0/2065003583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x5564acea66b0 con 0x5564ad194b20 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1acc00640 1 -- 172.21.15.60:0/2065003583 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1b0039070 con 0x5564ad194b20 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1acc00640 1 -- 172.21.15.60:0/2065003583 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7ff1bc026020 con 0x5564ad194b20 2024-09-16T15:44:05.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 10 client.7044 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T15:44:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 10 client.7044 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:44:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 20 client.7044 populate_metadata read hostname 'smithi060' 2024-09-16T15:44:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] conn(0x5564ad180920 0x5564ad180d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 4 client.7044 mds_command: new command op to 6688 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:44:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.660+0000 7ff1cd685740 1 -- 172.21.15.60:0/2065003583 --> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564ad0e0520 con 0x5564ad180920 2024-09-16T15:44:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.661+0000 7ff1c3400640 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] conn(0x5564ad180920 0x5564ad180d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.661+0000 7ff1c3400640 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] conn(0x5564ad180920 0x5564ad180d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2325606649233a3 server_cookie=5037ac5dad4d0468 in_seq=0 out_seq=0 2024-09-16T15:44:05.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.661+0000 7ff1acc00640 10 client.7044 ms_handle_connect on v2:172.21.15.100:6800/515865896 2024-09-16T15:44:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.662+0000 7ff1acc00640 1 -- 172.21.15.60:0/2065003583 <== mds.0 v2:172.21.15.100:6800/515865896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5564ad0e0520 con 0x5564ad180920 2024-09-16T15:44:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.662+0000 7ff1acc00640 10 client.7044 handle_command_reply: tid=0 2024-09-16T15:44:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.662+0000 7ff1cd685740 1 client.7044 shutdown 2024-09-16T15:44:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 -- 172.21.15.60:0/2065003583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad194b20 msgr2=0x5564ad1afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad194b20 0x5564ad1afd70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc005fd0 tx=0x7ff1bc003f90 comp rx=0 tx=0).stop 2024-09-16T15:44:05.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 -- 172.21.15.60:0/2065003583 shutdown_connections 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] conn(0x5564ad180920 0x5564ad180d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad194b20 0x5564ad1afd70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad1af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2065003583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad16c190 0x5564ad1945e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 -- 172.21.15.60:0/2065003583 >> 172.21.15.60:0/2065003583 conn(0x5564ad16bc10 msgr2=0x5564ad1640f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1acc00640 0 client.7044 ms_handle_reset on v2:172.21.15.100:6800/515865896 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 -- 172.21.15.60:0/2065003583 shutdown_connections 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 1 -- 172.21.15.60:0/2065003583 wait complete. 2024-09-16T15:44:05.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.663+0000 7ff1cd685740 20 client.7044 trim_cache size 0 max 16384 2024-09-16T15:44:05.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.682+0000 7ff1cd685740 1 Processor -- start 2024-09-16T15:44:05.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.682+0000 7ff1cd685740 1 -- start start 2024-09-16T15:44:05.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad165390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad1d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad1b0aa0 0x5564ad1af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1cd685740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4201630 con 0x5564ad1b0aa0 2024-09-16T15:44:05.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1cd685740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c42006c0 con 0x5564ad194b20 2024-09-16T15:44:05.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1cd685740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c40cb8d0 con 0x5564ad16c190 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad1b0aa0 0x5564ad1af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad1b0aa0 0x5564ad1af780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49274/0 (socket says 172.21.15.60:49274) 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3e00640 1 -- 172.21.15.60:0/2071328870 learned_addr learned my addr 172.21.15.60:0/2071328870 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3e00640 1 -- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 msgr2=0x5564ad165390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad1d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3400640 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad165390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad165390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3e00640 1 -- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 msgr2=0x5564ad1d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad1d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3e00640 1 -- 172.21.15.60:0/2071328870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ad1a9910 con 0x5564ad1b0aa0 2024-09-16T15:44:05.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c3400640 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad165390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.683+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad1d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.684+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad1b0aa0 0x5564ad1af780 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc013cc0 tx=0x7ff1bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6b00f9ce70b1a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.684+0000 7ff1af400640 1 -- 172.21.15.60:0/2071328870 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc02b020 con 0x5564ad1b0aa0 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.684+0000 7ff1af400640 1 -- 172.21.15.60:0/2071328870 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1bc0098e0 con 0x5564ad1b0aa0 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.684+0000 7ff1af400640 1 -- 172.21.15.60:0/2071328870 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc020070 con 0x5564ad1b0aa0 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.684+0000 7ff1cd685740 1 -- 172.21.15.60:0/2071328870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad1b0aa0 msgr2=0x5564ad1af780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.684+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad1b0aa0 0x5564ad1af780 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc013cc0 tx=0x7ff1bc004a40 comp rx=0 tx=0).stop 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 -- 172.21.15.60:0/2071328870 shutdown_connections 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad1b0aa0 0x5564ad1af780 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad1d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 --2- 172.21.15.60:0/2071328870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad16c190 0x5564ad165390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 -- 172.21.15.60:0/2071328870 >> 172.21.15.60:0/2071328870 conn(0x5564ad16bc10 msgr2=0x5564ad169400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 -- 172.21.15.60:0/2071328870 shutdown_connections 2024-09-16T15:44:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 -- 172.21.15.60:0/2071328870 wait complete. 2024-09-16T15:44:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 -- start start 2024-09-16T15:44:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.685+0000 7ff1cd685740 1 Processor -- start 2024-09-16T15:44:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.686+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16c190 0x5564ad22e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.686+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad169400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.686+0000 7ff1cd685740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad169940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.686+0000 7ff1cd685740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5564ad181500 con 0x5564ad16c190 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.686+0000 7ff1cd685740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4200070 con 0x5564ad194b20 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.686+0000 7ff1cd685740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5564ad1adc40 con 0x5564ad1b0aa0 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.686+0000 7ff1cd685740 10 client.0 _ll_register_callbacks cb 0x7ff1bc003e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16c190 0x5564ad22e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16c190 0x5564ad22e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49280/0 (socket says 172.21.15.60:49280) 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 -- 172.21.15.60:0/1357576621 learned_addr learned my addr 172.21.15.60:0/1357576621 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad169400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 -- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 msgr2=0x5564ad169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 -- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 msgr2=0x5564ad169400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad169400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 -- 172.21.15.60:0/1357576621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ad1653f0 con 0x5564ad16c190 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3e00640 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1c3400640 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16c190 0x5564ad22e180 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4099cb0 tx=0x7ff1b40632d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36df01f9facdd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.687+0000 7ff1acc00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1acc00640 1 -- 172.21.15.60:0/1357576621 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b401f040 con 0x5564ad16c190 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1acc00640 1 -- 172.21.15.60:0/1357576621 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7ff1b406cd80 con 0x5564ad16c190 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 1 -- 172.21.15.60:0/1357576621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1bc0098e0 con 0x5564ad16c190 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1acc00640 1 -- 172.21.15.60:0/1357576621 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7ff1b406d010 con 0x5564ad16c190 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 10 client.7050 fetch_fsmap learned FSMap version 71 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 1 -- 172.21.15.60:0/1357576621 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x5564ad2318d0 con 0x5564ad16c190 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1acc00640 1 -- 172.21.15.60:0/1357576621 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1bc0098e0 con 0x5564ad16c190 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1acc00640 1 -- 172.21.15.60:0/1357576621 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7ff1b406a040 con 0x5564ad16c190 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 10 client.7050 fetch_fsmap finished waiting for FSMap version 71 2024-09-16T15:44:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 10 client.7050 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 20 client.7050 populate_metadata read hostname 'smithi060' 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] conn(0x5564ad234bf0 0x5564ad232430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 4 client.7050 mds_command: new command op to 6688 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.688+0000 7ff1cd685740 1 -- 172.21.15.60:0/1357576621 --> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] -- command(tid 0: {"prefix": "status"}) -- 0x5564ad0dbfd0 con 0x5564ad234bf0 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.689+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] conn(0x5564ad234bf0 0x5564ad232430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.689+0000 7ff1c2a00640 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] conn(0x5564ad234bf0 0x5564ad232430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3467241bed4b7d25 server_cookie=cf8208dc8622ca94 in_seq=0 out_seq=0 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.689+0000 7ff1acc00640 10 client.7050 ms_handle_connect on v2:172.21.15.100:6800/515865896 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1acc00640 1 -- 172.21.15.60:0/1357576621 <== mds.0 v2:172.21.15.100:6800/515865896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5564ad0dbfd0 con 0x5564ad234bf0 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1acc00640 10 client.7050 handle_command_reply: tid=0 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 client.7050 shutdown 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 -- 172.21.15.60:0/1357576621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16c190 msgr2=0x5564ad22e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16c190 0x5564ad22e180 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4099cb0 tx=0x7ff1b40632d0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 -- 172.21.15.60:0/1357576621 shutdown_connections 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:6800/515865896,v1:172.21.15.100:6801/515865896] conn(0x5564ad234bf0 0x5564ad232430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5564ad1b0aa0 0x5564ad169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5564ad194b20 0x5564ad169400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 --2- 172.21.15.60:0/1357576621 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5564ad16c190 0x5564ad22e180 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1acc00640 0 client.7050 ms_handle_reset on v2:172.21.15.100:6800/515865896 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 -- 172.21.15.60:0/1357576621 >> 172.21.15.60:0/1357576621 conn(0x5564ad16bc10 msgr2=0x5564ad194120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 -- 172.21.15.60:0/1357576621 shutdown_connections 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 1 -- 172.21.15.60:0/1357576621 wait complete. 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.690+0000 7ff1cd685740 20 client.7050 trim_cache size 0 max 16384 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:05.695 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6688, 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2374074909999999, 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 71, 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 107, 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 107, 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.536616136999999 2024-09-16T15:44:05.696 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:05.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 -- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff1a40789a0 msgr2=0x7ff1a407ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff1a40789a0 0x7ff1a407ae60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0032000 tx=0x7ff1b0001190 comp rx=0 tx=0).stop 2024-09-16T15:44:05.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 -- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 msgr2=0x7ff1c40b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 0x7ff1c40b6c20 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006630 tx=0x7ff1b40039c0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 -- 172.21.15.60:0/3969004949 shutdown_connections 2024-09-16T15:44:05.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff1a40789a0 0x7ff1a407ae60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1c40cd0c0 0x7ff1c40d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1c40b77b0 0x7ff1c40ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 --2- 172.21.15.60:0/3969004949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1a8004800 0x7ff1c40b6c20 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 -- 172.21.15.60:0/3969004949 >> 172.21.15.60:0/3969004949 conn(0x7ff1c40d3df0 msgr2=0x7ff1c415e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:05.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 -- 172.21.15.60:0/3969004949 shutdown_connections 2024-09-16T15:44:05.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.691+0000 7ff1c9200640 1 -- 172.21.15.60:0/3969004949 wait complete. 2024-09-16T15:44:05.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:44:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.866+0000 7f36c3400640 1 -- 172.21.15.60:0/4215168008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 msgr2=0x7f36bc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.866+0000 7f36c3400640 1 --2- 172.21.15.60:0/4215168008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 0x7f36bc0b2440 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f36ac005fd0 tx=0x7f36ac02a540 comp rx=0 tx=0).stop 2024-09-16T15:44:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.866+0000 7f36c3400640 1 -- 172.21.15.60:0/4215168008 shutdown_connections 2024-09-16T15:44:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.866+0000 7f36c3400640 1 --2- 172.21.15.60:0/4215168008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36bc0ccf00 0x7f36bc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.866+0000 7f36c3400640 1 --2- 172.21.15.60:0/4215168008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 0x7f36bc0b2440 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.866+0000 7f36c3400640 1 -- 172.21.15.60:0/4215168008 >> 172.21.15.60:0/4215168008 conn(0x7f36bc0b1950 msgr2=0x7f36bc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.866+0000 7f36c3400640 1 -- 172.21.15.60:0/4215168008 shutdown_connections 2024-09-16T15:44:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.866+0000 7f36c3400640 1 -- 172.21.15.60:0/4215168008 wait complete. 2024-09-16T15:44:05.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.868+0000 7f36c3400640 1 Processor -- start 2024-09-16T15:44:05.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.868+0000 7f36c3400640 1 -- start start 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c3400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36bc0ad270 0x7f36bc153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36bc0ccf00 0x7f36bc0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 0x7f36bc0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c3400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc06f2e0 con 0x7f36bc15f5a0 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c3400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc06f160 con 0x7f36bc0ccf00 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c3400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f36bc06f460 con 0x7f36bc0ad270 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 0x7f36bc0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 0x7f36bc0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49294/0 (socket says 172.21.15.60:49294) 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c1400640 1 -- 172.21.15.60:0/3701773321 learned_addr learned my addr 172.21.15.60:0/3701773321 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36b3e00640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36bc0ccf00 0x7f36bc0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.869+0000 7f36c0a00640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36bc0ad270 0x7f36bc153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36c1400640 1 -- 172.21.15.60:0/3701773321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36bc0ad270 msgr2=0x7f36bc153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36c1400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36bc0ad270 0x7f36bc153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36c1400640 1 -- 172.21.15.60:0/3701773321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36bc0ccf00 msgr2=0x7f36bc0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36c1400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36bc0ccf00 0x7f36bc0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36c1400640 1 -- 172.21.15.60:0/3701773321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc153280 con 0x7f36bc15f5a0 2024-09-16T15:44:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36c1400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 0x7f36bc0b2d50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c7a0 tx=0x7f36b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36b9600640 1 -- 172.21.15.60:0/3701773321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4003b70 con 0x7f36bc15f5a0 2024-09-16T15:44:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36b9600640 1 -- 172.21.15.60:0/3701773321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36b4003d10 con 0x7f36bc15f5a0 2024-09-16T15:44:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36b9600640 1 -- 172.21.15.60:0/3701773321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b400aab0 con 0x7f36bc15f5a0 2024-09-16T15:44:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc150b70 con 0x7f36bc15f5a0 2024-09-16T15:44:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.870+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc153520 con 0x7f36bc15f5a0 2024-09-16T15:44:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.871+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc06f9f0 con 0x7f36bc15f5a0 2024-09-16T15:44:05.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.873+0000 7f36b9600640 1 -- 172.21.15.60:0/3701773321 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f36b400ac50 con 0x7f36bc15f5a0 2024-09-16T15:44:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.874+0000 7f36b9600640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3698078970 0x7f369807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.874+0000 7f36b9600640 1 -- 172.21.15.60:0/3701773321 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f36b409a960 con 0x7f36bc15f5a0 2024-09-16T15:44:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.874+0000 7f36c0a00640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3698078970 0x7f369807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.875+0000 7f36c0a00640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3698078970 0x7f369807ae30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f36ac033000 tx=0x7f36ac004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:05.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:05.878+0000 7f36b9600640 1 -- 172.21.15.60:0/3701773321 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f36b40626c0 con 0x7f36bc15f5a0 2024-09-16T15:44:06.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.117+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36bc1536f0 con 0x7f36bc15f5a0 2024-09-16T15:44:06.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.118+0000 7f36b9600640 1 -- 172.21.15.60:0/3701773321 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f36b40670f0 con 0x7f36bc15f5a0 2024-09-16T15:44:06.119 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3698078970 msgr2=0x7f369807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3698078970 0x7f369807ae30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f36ac033000 tx=0x7f36ac004730 comp rx=0 tx=0).stop 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 msgr2=0x7f36bc0b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 0x7f36bc0b2d50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c7a0 tx=0x7f36b400cc60 comp rx=0 tx=0).stop 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 shutdown_connections 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3698078970 0x7f369807ae30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36bc15f5a0 0x7f36bc0b2d50 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36bc0ccf00 0x7f36bc0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:06.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 --2- 172.21.15.60:0/3701773321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36bc0ad270 0x7f36bc153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:06.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 >> 172.21.15.60:0/3701773321 conn(0x7f36bc0b1950 msgr2=0x7f36bc09c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:06.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 shutdown_connections 2024-09-16T15:44:06.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06.121+0000 7f36c3400640 1 -- 172.21.15.60:0/3701773321 wait complete. 2024-09-16T15:44:06.137 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:44:06.137 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:44:06.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:44:06.138 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:44:06.138 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:06.138 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:44:06.138 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:44:06.138 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:44:06.138 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:44:06.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:06.138 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:44:06.174 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:44:06.174 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:44:06.174 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:44:06.174 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:44:06.175 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42403sec preferred_lft 42403sec 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:44:06.176 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:44:06.177 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:44:06.177 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:06.177 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:44:06.177 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:44:06.177 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:44:06.177 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:06.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:06.177 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:44:06.268 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:44:06.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:06.272 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:44:06.307 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:44:06.307 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:44:06.308 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:44:06.308 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:44:06.308 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:44:06.308 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:44:06.308 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:44:06.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:44:06.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:44:06.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:44:06.308 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:44:06.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:44:06.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:44:06.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:06.510 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:44:06.545 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:44:06.545 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:44:06.546 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:44:06.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:06.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:44:06.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:44:06.641 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:06.676 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:44:06.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:44:06.677 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:06.752 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:44:06.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:06.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:44:06.881 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:44:06.881 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:44:06.908 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:44:06.908 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:44:06.908 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:44:06.908 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:44:06.909 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:44:06.910 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:44:06.910 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:44:06.910 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:44:06.910 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:44:06.910 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:44:06.913 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:44:06.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:06.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:06.969 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:44:07.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:44:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:44:07.069 INFO:teuthology.orchestra.run.smithi060.stdout:client7059 2024-09-16T15:44:07.069 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:07.069 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:07.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:44:07.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.306+0000 7f9846a00640 1 -- 172.21.15.60:0/2260453780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 msgr2=0x7f98400b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.306+0000 7f9846a00640 1 --2- 172.21.15.60:0/2260453780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 0x7f98400b2440 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f982c005fd0 tx=0x7f982c02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.306+0000 7f9846a00640 1 -- 172.21.15.60:0/2260453780 shutdown_connections 2024-09-16T15:44:07.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.306+0000 7f9846a00640 1 --2- 172.21.15.60:0/2260453780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 0x7f98400b2440 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.306+0000 7f9846a00640 1 -- 172.21.15.60:0/2260453780 >> 172.21.15.60:0/2260453780 conn(0x7f98400b1950 msgr2=0x7f98400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:07.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.306+0000 7f9846a00640 1 -- 172.21.15.60:0/2260453780 shutdown_connections 2024-09-16T15:44:07.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.306+0000 7f9846a00640 1 -- 172.21.15.60:0/2260453780 wait complete. 2024-09-16T15:44:07.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.308+0000 7f9846a00640 1 Processor -- start 2024-09-16T15:44:07.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.308+0000 7f9846a00640 1 -- start start 2024-09-16T15:44:07.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f9846a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840073fb0 0x7f984015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f9846a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 0x7f98401f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f9846a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984015f5a0 0x7f984015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f9846a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f984006f9f0 con 0x7f98400ccf00 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f9846a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f984006f870 con 0x7f984015f5a0 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f9846a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9840074420 con 0x7f9840073fb0 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 0x7f98401f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 0x7f98401f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49314/0 (socket says 172.21.15.60:49314) 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983f400640 1 -- 172.21.15.60:0/3123823143 learned_addr learned my addr 172.21.15.60:0/3123823143 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f9844a00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984015f5a0 0x7f984015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983fe00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840073fb0 0x7f984015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983f400640 1 -- 172.21.15.60:0/3123823143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840073fb0 msgr2=0x7f984015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983f400640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840073fb0 0x7f984015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983f400640 1 -- 172.21.15.60:0/3123823143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984015f5a0 msgr2=0x7f984015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983f400640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984015f5a0 0x7f984015bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.309+0000 7f983f400640 1 -- 172.21.15.60:0/3123823143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840159610 con 0x7f98400ccf00 2024-09-16T15:44:07.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.310+0000 7f983f400640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 0x7f98401f61a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f982c009850 tx=0x7f982c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:07.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.310+0000 7f9837e00640 1 -- 172.21.15.60:0/3123823143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c03f070 con 0x7f98400ccf00 2024-09-16T15:44:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.310+0000 7f9837e00640 1 -- 172.21.15.60:0/3123823143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f982c004890 con 0x7f98400ccf00 2024-09-16T15:44:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.310+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98401598f0 con 0x7f98400ccf00 2024-09-16T15:44:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.310+0000 7f9837e00640 1 -- 172.21.15.60:0/3123823143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c035030 con 0x7f98400ccf00 2024-09-16T15:44:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.310+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984015a190 con 0x7f98400ccf00 2024-09-16T15:44:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.311+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98400b2440 con 0x7f98400ccf00 2024-09-16T15:44:07.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.312+0000 7f9837e00640 1 -- 172.21.15.60:0/3123823143 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f982c042070 con 0x7f98400ccf00 2024-09-16T15:44:07.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.313+0000 7f9837e00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.313+0000 7f9837e00640 1 -- 172.21.15.60:0/3123823143 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5767+0+0 (secure 0 0 0) 0x7f982c0bbb30 con 0x7f98400ccf00 2024-09-16T15:44:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.314+0000 7f983fe00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:07.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.314+0000 7f983fe00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98400cbfa0 tx=0x7f9830009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:07.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.318+0000 7f9837e00640 1 -- 172.21.15.60:0/3123823143 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f982c083850 con 0x7f98400ccf00 2024-09-16T15:44:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.558+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f98400acd30 con 0x7f98400ccf00 2024-09-16T15:44:07.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.558+0000 7f9837e00640 1 -- 172.21.15.60:0/3123823143 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f982c088280 con 0x7f98400ccf00 2024-09-16T15:44:07.559 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 msgr2=0x7f982007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98400cbfa0 tx=0x7f9830009040 comp rx=0 tx=0).stop 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 msgr2=0x7f98401f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 0x7f98401f61a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f982c009850 tx=0x7f982c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 shutdown_connections 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9820078970 0x7f982007ae30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984015f5a0 0x7f984015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98400ccf00 0x7f98401f61a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 --2- 172.21.15.60:0/3123823143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9840073fb0 0x7f984015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.560+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 >> 172.21.15.60:0/3123823143 conn(0x7f98400b1950 msgr2=0x7f984015cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.561+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 shutdown_connections 2024-09-16T15:44:07.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:07.561+0000 7f9846a00640 1 -- 172.21.15.60:0/3123823143 wait complete. 2024-09-16T15:44:07.577 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:44:07.577 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:44:07.577 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:44:07.577 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:44:07.577 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:44:07.577 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:44:07.577 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:44:07.578 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:44:07.578 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:44:07.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:07.578 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:44:07.617 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:44:07.617 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:07.618 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42401sec preferred_lft 42401sec 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:44:07.619 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:44:07.620 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:07.620 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:44:07.620 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:44:07.620 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:44:07.620 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:07.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:07.620 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:44:07.713 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:44:07.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:07.717 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:44:07.753 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:44:07.754 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:44:07.754 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:44:07.754 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:44:07.754 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:44:07.754 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:44:07.754 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:44:07.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:44:07.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:44:07.755 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:44:07.755 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:44:07.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:44:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:44:07.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:44:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:44:07.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:07.951 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:44:07.986 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:44:07.986 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:44:07.986 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:44:07.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:07.987 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:44:08.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:44:08.081 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:44:08.116 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:44:08.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:44:08.116 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:44:08.191 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:44:08.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:08.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:44:08.319 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:44:08.319 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:44:08.347 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:44:08.347 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:44:08.348 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:44:08.348 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:44:08.348 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:44:08.349 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:44:08.349 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:44:08.349 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:44:08.349 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:44:08.349 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:44:08.353 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:44:08.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:08.353 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:44:08.411 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:44:08.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:44:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:44:08.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:44:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:44:08.501 INFO:teuthology.orchestra.run.smithi100.stdout:client6179 2024-09-16T15:44:08.501 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:44:08.501 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:08.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:44:08.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.711+0000 7ff44ae00640 1 -- 172.21.15.60:0/3115170548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff444074280 msgr2=0x7ff44406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:08.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.711+0000 7ff44ae00640 1 --2- 172.21.15.60:0/3115170548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff444074280 0x7ff44406ed60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff4440cd0e0 tx=0x7ff43c02a820 comp rx=0 tx=0).stop 2024-09-16T15:44:08.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.711+0000 7ff44ae00640 1 -- 172.21.15.60:0/3115170548 shutdown_connections 2024-09-16T15:44:08.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.711+0000 7ff44ae00640 1 --2- 172.21.15.60:0/3115170548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff444074280 0x7ff44406ed60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.711+0000 7ff44ae00640 1 --2- 172.21.15.60:0/3115170548 >> v2:172.21.15.100:3300/0 conn(0x7ff4440b2810 0x7ff444073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.711+0000 7ff44ae00640 1 -- 172.21.15.60:0/3115170548 >> 172.21.15.60:0/3115170548 conn(0x7ff4440b1510 msgr2=0x7ff4440b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:08.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.711+0000 7ff44ae00640 1 -- 172.21.15.60:0/3115170548 shutdown_connections 2024-09-16T15:44:08.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.711+0000 7ff44ae00640 1 -- 172.21.15.60:0/3115170548 wait complete. 2024-09-16T15:44:08.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.713+0000 7ff44ae00640 1 Processor -- start 2024-09-16T15:44:08.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.713+0000 7ff44ae00640 1 -- start start 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff44ae00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff444074280 0x7ff444152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff44ae00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4440b2810 0x7ff4441f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff44ae00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff44415bb90 0x7ff4440c3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff44ae00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff44406f2a0 con 0x7ff4440b2810 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff44ae00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff44415bf70 con 0x7ff44415bb90 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff44ae00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff44406f420 con 0x7ff444074280 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff449000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4440b2810 0x7ff4441f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff449000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4440b2810 0x7ff4441f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49318/0 (socket says 172.21.15.60:49318) 2024-09-16T15:44:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff449000640 1 -- 172.21.15.60:0/421642603 learned_addr learned my addr 172.21.15.60:0/421642603 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff449a00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff444074280 0x7ff444152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.714+0000 7ff44a400640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff44415bb90 0x7ff4440c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff449000640 1 -- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff444074280 msgr2=0x7ff444152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff449000640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff444074280 0x7ff444152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff449000640 1 -- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff44415bb90 msgr2=0x7ff4440c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff449000640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff44415bb90 0x7ff4440c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff449000640 1 -- 172.21.15.60:0/421642603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4441f82b0 con 0x7ff4440b2810 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff44a400640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff44415bb90 0x7ff4440c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:44:08.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff449000640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4440b2810 0x7ff4441f7a20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff43000e9f0 tx=0x7ff43000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff441600640 1 -- 172.21.15.60:0/421642603 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43000cd80 con 0x7ff4440b2810 2024-09-16T15:44:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff441600640 1 -- 172.21.15.60:0/421642603 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff430010430 con 0x7ff4440b2810 2024-09-16T15:44:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4441f8590 con 0x7ff4440b2810 2024-09-16T15:44:08.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff441600640 1 -- 172.21.15.60:0/421642603 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430010840 con 0x7ff4440b2810 2024-09-16T15:44:08.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.715+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff444153500 con 0x7ff4440b2810 2024-09-16T15:44:08.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.716+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4440b2c60 con 0x7ff4440b2810 2024-09-16T15:44:08.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.718+0000 7ff441600640 1 -- 172.21.15.60:0/421642603 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7ff4300109e0 con 0x7ff4440b2810 2024-09-16T15:44:08.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.718+0000 7ff441600640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff424078970 0x7ff42407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:08.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.719+0000 7ff441600640 1 -- 172.21.15.60:0/421642603 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7ff43009b6d0 con 0x7ff4440b2810 2024-09-16T15:44:08.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.719+0000 7ff449a00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff424078970 0x7ff42407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:08.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.719+0000 7ff449a00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff424078970 0x7ff42407ae30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff438007fd0 tx=0x7ff438001390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:08.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.723+0000 7ff441600640 1 -- 172.21.15.60:0/421642603 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff430063540 con 0x7ff4440b2810 2024-09-16T15:44:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:08.960+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff4440b1a70 con 0x7ff4440b2810 2024-09-16T15:44:09.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.511+0000 7ff441600640 1 -- 172.21.15.60:0/421642603 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v72) ==== 136+0+0 (secure 0 0 0) 0x7ff430067f70 con 0x7ff4440b2810 2024-09-16T15:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff424078970 msgr2=0x7ff42407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff424078970 0x7ff42407ae30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff438007fd0 tx=0x7ff438001390 comp rx=0 tx=0).stop 2024-09-16T15:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4440b2810 msgr2=0x7ff4441f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4440b2810 0x7ff4441f7a20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff43000e9f0 tx=0x7ff43000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 shutdown_connections 2024-09-16T15:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff424078970 0x7ff42407ae30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff44415bb90 0x7ff4440c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4440b2810 0x7ff4441f7a20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 --2- 172.21.15.60:0/421642603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff444074280 0x7ff444152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.514+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 >> 172.21.15.60:0/421642603 conn(0x7ff4440b1510 msgr2=0x7ff4440ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.515+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 shutdown_connections 2024-09-16T15:44:09.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.515+0000 7ff44ae00640 1 -- 172.21.15.60:0/421642603 wait complete. 2024-09-16T15:44:09.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:44:09.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.698+0000 7f1836800640 1 -- 172.21.15.60:0/2433226815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 msgr2=0x7f18300732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.698+0000 7f1836800640 1 --2- 172.21.15.60:0/2433226815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 0x7f18300732a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f183006d9e0 tx=0x7f182802aa30 comp rx=0 tx=0).stop 2024-09-16T15:44:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.698+0000 7f1836800640 1 -- 172.21.15.60:0/2433226815 shutdown_connections 2024-09-16T15:44:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.698+0000 7f1836800640 1 --2- 172.21.15.60:0/2433226815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 0x7f18300732a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.698+0000 7f1836800640 1 --2- 172.21.15.60:0/2433226815 >> v2:172.21.15.100:3300/0 conn(0x7f1830161550 0x7f183006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.698+0000 7f1836800640 1 -- 172.21.15.60:0/2433226815 >> 172.21.15.60:0/2433226815 conn(0x7f18300b1950 msgr2=0x7f18300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.698+0000 7f1836800640 1 -- 172.21.15.60:0/2433226815 shutdown_connections 2024-09-16T15:44:09.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.698+0000 7f1836800640 1 -- 172.21.15.60:0/2433226815 wait complete. 2024-09-16T15:44:09.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.700+0000 7f1836800640 1 Processor -- start 2024-09-16T15:44:09.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.700+0000 7f1836800640 1 -- start start 2024-09-16T15:44:09.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f1836800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 0x7f18301540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f1836800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18300af3d0 0x7f18300cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f1836800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1830161550 0x7f183015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f1836800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f18300737e0 con 0x7f1830072df0 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f1836800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f183006fdb0 con 0x7f1830161550 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f1836800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1830073960 con 0x7f18300af3d0 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f182f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 0x7f18301540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f182f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 0x7f18301540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43730/0 (socket says 172.21.15.60:43730) 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f182f400640 1 -- 172.21.15.60:0/2025495847 learned_addr learned my addr 172.21.15.60:0/2025495847 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f182fe00640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1830161550 0x7f183015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.701+0000 7f1826a00640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18300af3d0 0x7f18300cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f182f400640 1 -- 172.21.15.60:0/2025495847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18300af3d0 msgr2=0x7f18300cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f182f400640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18300af3d0 0x7f18300cfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f182f400640 1 -- 172.21.15.60:0/2025495847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1830161550 msgr2=0x7f183015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f182f400640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1830161550 0x7f183015c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f182f400640 1 -- 172.21.15.60:0/2025495847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18301f4d20 con 0x7f1830072df0 2024-09-16T15:44:09.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f182f400640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 0x7f18301540a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f182000d770 tx=0x7f182000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:09.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f1827e00640 1 -- 172.21.15.60:0/2025495847 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820017070 con 0x7f1830072df0 2024-09-16T15:44:09.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f1827e00640 1 -- 172.21.15.60:0/2025495847 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1820009d20 con 0x7f1830072df0 2024-09-16T15:44:09.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f1827e00640 1 -- 172.21.15.60:0/2025495847 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18200106c0 con 0x7f1830072df0 2024-09-16T15:44:09.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.702+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18301f5000 con 0x7f1830072df0 2024-09-16T15:44:09.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.703+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18301f55c0 con 0x7f1830072df0 2024-09-16T15:44:09.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.703+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1830073de0 con 0x7f1830072df0 2024-09-16T15:44:09.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.705+0000 7f1827e00640 1 -- 172.21.15.60:0/2025495847 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f182001a070 con 0x7f1830072df0 2024-09-16T15:44:09.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.706+0000 7f1827e00640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f180c0789a0 0x7f180c07ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:09.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.706+0000 7f1827e00640 1 -- 172.21.15.60:0/2025495847 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f182009aca0 con 0x7f1830072df0 2024-09-16T15:44:09.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.706+0000 7f1826a00640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f180c0789a0 0x7f180c07ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:09.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.706+0000 7f1826a00640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f180c0789a0 0x7f180c07ae60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18300addb0 tx=0x7f1818009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:09.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.710+0000 7f1827e00640 1 -- 172.21.15.60:0/2025495847 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f18200629c0 con 0x7f1830072df0 2024-09-16T15:44:09.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.925+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1830073480 con 0x7f180c0789a0 2024-09-16T15:44:09.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.927+0000 7f1827e00640 1 -- 172.21.15.60:0/2025495847 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1830073480 con 0x7f180c0789a0 2024-09-16T15:44:09.929 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:44:09.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:44:09.929 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:44:09.929 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:09.929 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:44:09.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f180c0789a0 msgr2=0x7f180c07ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f180c0789a0 0x7f180c07ae60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f18300addb0 tx=0x7f1818009040 comp rx=0 tx=0).stop 2024-09-16T15:44:09.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 msgr2=0x7f18301540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 0x7f18301540a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f182000d770 tx=0x7f182000dc30 comp rx=0 tx=0).stop 2024-09-16T15:44:09.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 shutdown_connections 2024-09-16T15:44:09.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f180c0789a0 0x7f180c07ae60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1830161550 0x7f183015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18300af3d0 0x7f18300cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 --2- 172.21.15.60:0/2025495847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1830072df0 0x7f18301540a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.929+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 >> 172.21.15.60:0/2025495847 conn(0x7f18300b1950 msgr2=0x7f183015ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:09.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.930+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 shutdown_connections 2024-09-16T15:44:09.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:09.930+0000 7f1836800640 1 -- 172.21.15.60:0/2025495847 wait complete. 2024-09-16T15:44:09.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:44:10.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.110+0000 7f9c50600640 1 -- 172.21.15.60:0/853311851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 msgr2=0x7f9c30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:10.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.110+0000 7f9c50600640 1 --2- 172.21.15.60:0/853311851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 0x7f9c30004c80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009b50 tx=0x7f9c30005830 comp rx=0 tx=0).stop 2024-09-16T15:44:10.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.111+0000 7f9c4e600640 1 -- 172.21.15.60:0/853311851 reap_dead start 2024-09-16T15:44:10.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.111+0000 7f9c50600640 1 -- 172.21.15.60:0/853311851 shutdown_connections 2024-09-16T15:44:10.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.111+0000 7f9c50600640 1 -- 172.21.15.60:0/853311851 >> 172.21.15.60:0/853311851 conn(0x7f9c480b1840 msgr2=0x7f9c480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:10.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.111+0000 7f9c50600640 1 -- 172.21.15.60:0/853311851 shutdown_connections 2024-09-16T15:44:10.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.111+0000 7f9c50600640 1 -- 172.21.15.60:0/853311851 wait complete. 2024-09-16T15:44:10.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.112+0000 7f9c50600640 1 Processor -- start 2024-09-16T15:44:10.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.113+0000 7f9c50600640 1 -- start start 2024-09-16T15:44:10.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.113+0000 7f9c50600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 0x7f9c480af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.113+0000 7f9c50600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c480afba0 0x7f9c480c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.113+0000 7f9c50600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c481f5390 0x7f9c480c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c50600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48162c30 con 0x7f9c30004800 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c50600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c48162ab0 con 0x7f9c480afba0 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c50600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9c480cf0b0 con 0x7f9c481f5390 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 0x7f9c480af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 0x7f9c480af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43756/0 (socket says 172.21.15.60:43756) 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 -- 172.21.15.60:0/3283976400 learned_addr learned my addr 172.21.15.60:0/3283976400 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4d200640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c480afba0 0x7f9c480c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4e600640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c481f5390 0x7f9c480c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 -- 172.21.15.60:0/3283976400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c481f5390 msgr2=0x7f9c480c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:10.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c481f5390 0x7f9c480c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 -- 172.21.15.60:0/3283976400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c480afba0 msgr2=0x7f9c480c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c480afba0 0x7f9c480c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 -- 172.21.15.60:0/3283976400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c480b0040 con 0x7f9c30004800 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.114+0000 7f9c4dc00640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 0x7f9c480af660 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c011ba0 tx=0x7f9c3c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.115+0000 7f9c45600640 1 -- 172.21.15.60:0/3283976400 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c01e070 con 0x7f9c30004800 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.115+0000 7f9c45600640 1 -- 172.21.15.60:0/3283976400 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c3c00e7c0 con 0x7f9c30004800 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.115+0000 7f9c45600640 1 -- 172.21.15.60:0/3283976400 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00ebd0 con 0x7f9c30004800 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.115+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4814d2d0 con 0x7f9c30004800 2024-09-16T15:44:10.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.115+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4814d890 con 0x7f9c30004800 2024-09-16T15:44:10.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.116+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c480ce4a0 con 0x7f9c30004800 2024-09-16T15:44:10.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.117+0000 7f9c45600640 1 -- 172.21.15.60:0/3283976400 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f9c3c021070 con 0x7f9c30004800 2024-09-16T15:44:10.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.118+0000 7f9c45600640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c28078970 0x7f9c2807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:10.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.118+0000 7f9c45600640 1 -- 172.21.15.60:0/3283976400 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f9c3c09bc60 con 0x7f9c30004800 2024-09-16T15:44:10.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.118+0000 7f9c4d200640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c28078970 0x7f9c2807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:10.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.119+0000 7f9c4d200640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c28078970 0x7f9c2807ae30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009b50 tx=0x7f9c3402d040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:10.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.122+0000 7f9c45600640 1 -- 172.21.15.60:0/3283976400 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c3c063980 con 0x7f9c30004800 2024-09-16T15:44:10.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.318+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9c48073310 con 0x7f9c30004800 2024-09-16T15:44:10.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.318+0000 7f9c45600640 1 -- 172.21.15.60:0/3283976400 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f9c3c0683b0 con 0x7f9c30004800 2024-09-16T15:44:10.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c28078970 msgr2=0x7f9c2807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:10.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c28078970 0x7f9c2807ae30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009b50 tx=0x7f9c3402d040 comp rx=0 tx=0).stop 2024-09-16T15:44:10.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 msgr2=0x7f9c480af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:10.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 0x7f9c480af660 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c011ba0 tx=0x7f9c3c00f790 comp rx=0 tx=0).stop 2024-09-16T15:44:10.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 shutdown_connections 2024-09-16T15:44:10.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c28078970 0x7f9c2807ae30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c481f5390 0x7f9c480c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c480afba0 0x7f9c480c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 --2- 172.21.15.60:0/3283976400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c30004800 0x7f9c480af660 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 >> 172.21.15.60:0/3283976400 conn(0x7f9c480b1840 msgr2=0x7f9c48073530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:10.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 shutdown_connections 2024-09-16T15:44:10.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.321+0000 7f9c50600640 1 -- 172.21.15.60:0/3283976400 wait complete. 2024-09-16T15:44:10.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:10.339 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177) 2024-09-16T15:44:10.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:10.407 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/fake) 2024-09-16T15:44:10.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:10.493 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:44:10.531 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:44:10.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:10.531 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177) 2024-09-16T15:44:10.634 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:10.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol62870/.meta 2024-09-16T15:44:10.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol62870 2024-09-16T15:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.897+0000 7fcf19e00640 1 -- 172.21.15.60:0/2940583847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140b4bd0 msgr2=0x7fcf140b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.897+0000 7fcf19e00640 1 --2- 172.21.15.60:0/2940583847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140b4bd0 0x7fcf140b3ec0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fcf140cd280 tx=0x7fcefc02a600 comp rx=0 tx=0).stop 2024-09-16T15:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.898+0000 7fcf19e00640 1 -- 172.21.15.60:0/2940583847 shutdown_connections 2024-09-16T15:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.898+0000 7fcf19e00640 1 --2- 172.21.15.60:0/2940583847 >> v2:172.21.15.100:3300/0 conn(0x7fcf140b4400 0x7fcf1406e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.898+0000 7fcf19e00640 1 --2- 172.21.15.60:0/2940583847 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf140b4bd0 0x7fcf140b3ec0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.898+0000 7fcf19e00640 1 -- 172.21.15.60:0/2940583847 >> 172.21.15.60:0/2940583847 conn(0x7fcf140b1840 msgr2=0x7fcf140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.898+0000 7fcf19e00640 1 -- 172.21.15.60:0/2940583847 shutdown_connections 2024-09-16T15:44:10.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.898+0000 7fcf19e00640 1 -- 172.21.15.60:0/2940583847 wait complete. 2024-09-16T15:44:10.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.899+0000 7fcf19e00640 1 Processor -- start 2024-09-16T15:44:10.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.899+0000 7fcf19e00640 1 -- start start 2024-09-16T15:44:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.900+0000 7fcf19e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf140b4400 0x7fcf1413fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.900+0000 7fcf19e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140b4bd0 0x7fcf141e5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.900+0000 7fcf19e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf141437c0 0x7fcf141e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.900+0000 7fcf19e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1406ef20 con 0x7fcf141437c0 2024-09-16T15:44:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.900+0000 7fcf19e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1406eda0 con 0x7fcf140b4bd0 2024-09-16T15:44:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.900+0000 7fcf19e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1406f0a0 con 0x7fcf140b4400 2024-09-16T15:44:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.900+0000 7fcf13400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf141437c0 0x7fcf141e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf13400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf141437c0 0x7fcf141e3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43764/0 (socket says 172.21.15.60:43764) 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf13400640 1 -- 172.21.15.60:0/1856946382 learned_addr learned my addr 172.21.15.60:0/1856946382 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf0a000640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140b4bd0 0x7fcf141e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf12a00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf140b4400 0x7fcf1413fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf13400640 1 -- 172.21.15.60:0/1856946382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf140b4400 msgr2=0x7fcf1413fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf13400640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf140b4400 0x7fcf1413fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf13400640 1 -- 172.21.15.60:0/1856946382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140b4bd0 msgr2=0x7fcf141e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf13400640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140b4bd0 0x7fcf141e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf13400640 1 -- 172.21.15.60:0/1856946382 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf141f8900 con 0x7fcf141437c0 2024-09-16T15:44:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf13400640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf141437c0 0x7fcf141e3e30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00dda0 tx=0x7fcf0c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.901+0000 7fcf0be00640 1 -- 172.21.15.60:0/1856946382 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c004d20 con 0x7fcf141437c0 2024-09-16T15:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.902+0000 7fcf0be00640 1 -- 172.21.15.60:0/1856946382 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf0c00be40 con 0x7fcf141437c0 2024-09-16T15:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.902+0000 7fcf0be00640 1 -- 172.21.15.60:0/1856946382 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00e8a0 con 0x7fcf141437c0 2024-09-16T15:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.902+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf141e60b0 con 0x7fcf141437c0 2024-09-16T15:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.902+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141e6670 con 0x7fcf141437c0 2024-09-16T15:44:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.903+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf140b34e0 con 0x7fcf141437c0 2024-09-16T15:44:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.904+0000 7fcf0be00640 1 -- 172.21.15.60:0/1856946382 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fcf0c00ea40 con 0x7fcf141437c0 2024-09-16T15:44:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.905+0000 7fcf0be00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcef0078970 0x7fcef007ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.906+0000 7fcf0be00640 1 -- 172.21.15.60:0/1856946382 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fcf0c09b7d0 con 0x7fcf141437c0 2024-09-16T15:44:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.906+0000 7fcf12a00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcef0078970 0x7fcef007ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.906+0000 7fcf12a00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcef0078970 0x7fcef007ae30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf040015e0 tx=0x7fcf04009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:10.910+0000 7fcf0be00640 1 -- 172.21.15.60:0/1856946382 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcf140b34e0 con 0x7fcf141437c0 2024-09-16T15:44:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.133+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol62870", "target": ["mon-mgr", ""]}) -- 0x7fcf1406e860 con 0x7fcef0078970 2024-09-16T15:44:11.141 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:11.139+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.141 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:11.139+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.141 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:11.139+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.141 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:11.139+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.141 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:11.139+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.185+0000 7fcf0be00640 1 -- 172.21.15.60:0/1856946382 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fcf1406e860 con 0x7fcef0078970 2024-09-16T15:44:11.188 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177 2024-09-16T15:44:11.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcef0078970 msgr2=0x7fcef007ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcef0078970 0x7fcef007ae30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf040015e0 tx=0x7fcf04009040 comp rx=0 tx=0).stop 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf141437c0 msgr2=0x7fcf141e3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf141437c0 0x7fcf141e3e30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00dda0 tx=0x7fcf0c00b730 comp rx=0 tx=0).stop 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 shutdown_connections 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcef0078970 0x7fcef007ae30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf141437c0 0x7fcf141e3e30 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf140b4bd0 0x7fcf141e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 --2- 172.21.15.60:0/1856946382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf140b4400 0x7fcf1413fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 >> 172.21.15.60:0/1856946382 conn(0x7fcf140b1840 msgr2=0x7fcf141429f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 shutdown_connections 2024-09-16T15:44:11.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.188+0000 7fcf19e00640 1 -- 172.21.15.60:0/1856946382 wait complete. 2024-09-16T15:44:11.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-16T15:44:11.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.367+0000 7fd9cce00640 1 -- 172.21.15.60:0/3723408079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80b3450 msgr2=0x7fd9c80b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.367+0000 7fd9cce00640 1 --2- 172.21.15.60:0/3723408079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80b3450 0x7fd9c80b3890 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80cf1a0 tx=0x7fd9b402a560 comp rx=0 tx=0).stop 2024-09-16T15:44:11.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.367+0000 7fd9cce00640 1 -- 172.21.15.60:0/3723408079 shutdown_connections 2024-09-16T15:44:11.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.367+0000 7fd9cce00640 1 --2- 172.21.15.60:0/3723408079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80b3450 0x7fd9c80b3890 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.367+0000 7fd9cce00640 1 --2- 172.21.15.60:0/3723408079 >> v2:172.21.15.100:3300/0 conn(0x7fd9c815df60 0x7fd9c80d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.367+0000 7fd9cce00640 1 -- 172.21.15.60:0/3723408079 >> 172.21.15.60:0/3723408079 conn(0x7fd9c80d3d50 msgr2=0x7fd9c80d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:11.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.367+0000 7fd9cce00640 1 -- 172.21.15.60:0/3723408079 shutdown_connections 2024-09-16T15:44:11.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.367+0000 7fd9cce00640 1 -- 172.21.15.60:0/3723408079 wait complete. 2024-09-16T15:44:11.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.368+0000 7fd9cce00640 1 Processor -- start 2024-09-16T15:44:11.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.369+0000 7fd9cce00640 1 -- start start 2024-09-16T15:44:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.369+0000 7fd9cce00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9c80b3450 0x7fd9c81f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.369+0000 7fd9cce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80cf4b0 0x7fd9c80b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9cce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9c815df60 0x7fd9c80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9cce00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c80d1a30 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9cce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c80d2a40 con 0x7fd9c815df60 2024-09-16T15:44:11.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9cce00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c80d2bc0 con 0x7fd9c80b3450 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80cf4b0 0x7fd9c80b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80cf4b0 0x7fd9c80b8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43794/0 (socket says 172.21.15.60:43794) 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c6a00640 1 -- 172.21.15.60:0/2521067299 learned_addr learned my addr 172.21.15.60:0/2521067299 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c7e00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9c815df60 0x7fd9c80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c7400640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9c80b3450 0x7fd9c81f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c6a00640 1 -- 172.21.15.60:0/2521067299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9c80b3450 msgr2=0x7fd9c81f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c6a00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9c80b3450 0x7fd9c81f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c6a00640 1 -- 172.21.15.60:0/2521067299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9c815df60 msgr2=0x7fd9c80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c6a00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9c815df60 0x7fd9c80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.370+0000 7fd9c6a00640 1 -- 172.21.15.60:0/2521067299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c80c9bb0 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.371+0000 7fd9c6a00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80cf4b0 0x7fd9c80b8d40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009850 tx=0x7fd9b4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:11.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.371+0000 7fd9bf400640 1 -- 172.21.15.60:0/2521067299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b403f070 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.371+0000 7fd9bf400640 1 -- 172.21.15.60:0/2521067299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9b40047a0 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.371+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c81f8610 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.371+0000 7fd9bf400640 1 -- 172.21.15.60:0/2521067299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4035030 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.371+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81f8bd0 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.372+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c81f4990 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.373+0000 7fd9bf400640 1 -- 172.21.15.60:0/2521067299 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7fd9b4042070 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.375+0000 7fd9bf400640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd9a8078970 0x7fd9a807ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:11.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.375+0000 7fd9bf400640 1 -- 172.21.15.60:0/2521067299 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fd9b40bbad0 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.375+0000 7fd9c7400640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd9a8078970 0x7fd9a807ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:11.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.376+0000 7fd9c7400640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd9a8078970 0x7fd9a807ae30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8005fd0 tx=0x7fd9b8005850 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:11.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.379+0000 7fd9bf400640 1 -- 172.21.15.60:0/2521067299 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd9b4083e40 con 0x7fd9c80cf4b0 2024-09-16T15:44:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.600+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fd9c80cfb10 con 0x7fd9a8078970 2024-09-16T15:44:11.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.605+0000 7fd9bf400640 1 -- 172.21.15.60:0/2521067299 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fd9c80cfb10 con 0x7fd9a8078970 2024-09-16T15:44:11.607 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:44:11.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:44:11.607 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subvol62870" 2024-09-16T15:44:11.607 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:11.607 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:44:11.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd9a8078970 msgr2=0x7fd9a807ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd9a8078970 0x7fd9a807ae30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8005fd0 tx=0x7fd9b8005850 comp rx=0 tx=0).stop 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80cf4b0 msgr2=0x7fd9c80b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80cf4b0 0x7fd9c80b8d40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009850 tx=0x7fd9b4003910 comp rx=0 tx=0).stop 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 shutdown_connections 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd9a8078970 0x7fd9a807ae30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9c815df60 0x7fd9c80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9c80cf4b0 0x7fd9c80b8d40 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 --2- 172.21.15.60:0/2521067299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9c80b3450 0x7fd9c81f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 >> 172.21.15.60:0/2521067299 conn(0x7fd9c80d3d50 msgr2=0x7fd9c806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 shutdown_connections 2024-09-16T15:44:11.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.607+0000 7fd9cce00640 1 -- 172.21.15.60:0/2521067299 wait complete. 2024-09-16T15:44:11.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol62870 2024-09-16T15:44:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.786+0000 7f6edca00640 1 -- 172.21.15.60:0/4215551626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ed8072560 msgr2=0x7f6ed8073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.786+0000 7f6edca00640 1 --2- 172.21.15.60:0/4215551626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ed8072560 0x7f6ed8073ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0005fd0 tx=0x7f6ed002aa90 comp rx=0 tx=0).stop 2024-09-16T15:44:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.786+0000 7f6edca00640 1 -- 172.21.15.60:0/4215551626 shutdown_connections 2024-09-16T15:44:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.786+0000 7f6edca00640 1 --2- 172.21.15.60:0/4215551626 >> v2:172.21.15.100:3300/0 conn(0x7f6ed80cbc00 0x7f6ed806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.786+0000 7f6edca00640 1 --2- 172.21.15.60:0/4215551626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ed8072560 0x7f6ed8073ef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.786+0000 7f6edca00640 1 -- 172.21.15.60:0/4215551626 >> 172.21.15.60:0/4215551626 conn(0x7f6ed80cf9f0 msgr2=0x7f6ed80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.787+0000 7f6edca00640 1 -- 172.21.15.60:0/4215551626 shutdown_connections 2024-09-16T15:44:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.787+0000 7f6edca00640 1 -- 172.21.15.60:0/4215551626 wait complete. 2024-09-16T15:44:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.788+0000 7f6edca00640 1 Processor -- start 2024-09-16T15:44:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.788+0000 7f6edca00640 1 -- start start 2024-09-16T15:44:11.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.789+0000 7f6edca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed8072560 0x7f6ed80c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:11.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.789+0000 7f6edca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ed80cbc00 0x7f6ed8155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:11.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.789+0000 7f6edca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ed815bec0 0x7f6ed80ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:11.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.789+0000 7f6edca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed80ae9c0 con 0x7f6ed80cbc00 2024-09-16T15:44:11.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.789+0000 7f6edca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed80ae840 con 0x7f6ed8072560 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.789+0000 7f6edca00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed80aeb40 con 0x7f6ed815bec0 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ed80cbc00 0x7f6ed8155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ed80cbc00 0x7f6ed8155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43804/0 (socket says 172.21.15.60:43804) 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ecfe00640 1 -- 172.21.15.60:0/2344962772 learned_addr learned my addr 172.21.15.60:0/2344962772 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ed6800640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed8072560 0x7f6ed80c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ed7200640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ed815bec0 0x7f6ed80ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ecfe00640 1 -- 172.21.15.60:0/2344962772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ed815bec0 msgr2=0x7f6ed80ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ecfe00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ed815bec0 0x7f6ed80ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ecfe00640 1 -- 172.21.15.60:0/2344962772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed8072560 msgr2=0x7f6ed80c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ecfe00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed8072560 0x7f6ed80c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.790+0000 7f6ecfe00640 1 -- 172.21.15.60:0/2344962772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed81f4960 con 0x7f6ed80cbc00 2024-09-16T15:44:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.791+0000 7f6ecfe00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ed80cbc00 0x7f6ed8155ec0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000ea20 tx=0x7f6ec000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.791+0000 7f6ecea00640 1 -- 172.21.15.60:0/2344962772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000cd80 con 0x7f6ed80cbc00 2024-09-16T15:44:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.791+0000 7f6ecea00640 1 -- 172.21.15.60:0/2344962772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ec0010430 con 0x7f6ed80cbc00 2024-09-16T15:44:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.791+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed81f4be0 con 0x7f6ed80cbc00 2024-09-16T15:44:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.791+0000 7f6ecea00640 1 -- 172.21.15.60:0/2344962772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0010840 con 0x7f6ed80cbc00 2024-09-16T15:44:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.791+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81f51a0 con 0x7f6ed80cbc00 2024-09-16T15:44:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.792+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed80aeff0 con 0x7f6ed80cbc00 2024-09-16T15:44:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.793+0000 7f6ecea00640 1 -- 172.21.15.60:0/2344962772 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f6ec00109e0 con 0x7f6ed80cbc00 2024-09-16T15:44:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.794+0000 7f6ecea00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6eb4078970 0x7f6eb407ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.794+0000 7f6ecea00640 1 -- 172.21.15.60:0/2344962772 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f6ec009b7e0 con 0x7f6ed80cbc00 2024-09-16T15:44:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.794+0000 7f6ed6800640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6eb4078970 0x7f6eb407ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.795+0000 7f6ed6800640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6eb4078970 0x7f6eb407ae30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80cc410 tx=0x7f6ec8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:11.798+0000 7f6ecea00640 1 -- 172.21.15.60:0/2344962772 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6ed81f4960 con 0x7f6ed80cbc00 2024-09-16T15:44:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.022+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol62870", "target": ["mon-mgr", ""]}) -- 0x7f6ed80b4dd0 con 0x7f6eb4078970 2024-09-16T15:44:12.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.051+0000 7f6ecea00640 1 -- 172.21.15.60:0/2344962772 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+705 (secure 0 0 0) 0x7f6ed80b4dd0 con 0x7f6eb4078970 2024-09-16T15:44:12.053 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "atime": "2024-09-16 15:44:10", 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_pcent": "undefined", 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_quota": "infinite", 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_used": 0, 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "created_at": "2024-09-16 15:44:10", 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "ctime": "2024-09-16 15:44:11", 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "data_pool": "cephfs_data", 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "earmark": "", 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "features": [ 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot-clone", 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot-autoprotect" 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flavor": 1, 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 0, 2024-09-16T15:44:12.054 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": 16877, 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_addrs": [ 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6789", 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.100:6789", 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.100:6790" 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "mtime": "2024-09-16 15:44:10", 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177", 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_namespace": "", 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "complete", 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "subvolume", 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout: "uid": 0 2024-09-16T15:44:12.055 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6eb4078970 msgr2=0x7f6eb407ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6eb4078970 0x7f6eb407ae30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80cc410 tx=0x7f6ec8009040 comp rx=0 tx=0).stop 2024-09-16T15:44:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ed80cbc00 msgr2=0x7f6ed8155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ed80cbc00 0x7f6ed8155ec0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000ea20 tx=0x7f6ec000eee0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 shutdown_connections 2024-09-16T15:44:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6eb4078970 0x7f6eb407ae30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ed815bec0 0x7f6ed80ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ed80cbc00 0x7f6ed8155ec0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 --2- 172.21.15.60:0/2344962772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ed8072560 0x7f6ed80c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 >> 172.21.15.60:0/2344962772 conn(0x7f6ed80cf9f0 msgr2=0x7f6ed80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 shutdown_connections 2024-09-16T15:44:12.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.054+0000 7f6edca00640 1 -- 172.21.15.60:0/2344962772 wait complete. 2024-09-16T15:44:12.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol62870 10485760 2024-09-16T15:44:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.237+0000 7f6356800640 1 -- 172.21.15.60:0/2166851876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 msgr2=0x7f635006f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.237+0000 7f6356800640 1 --2- 172.21.15.60:0/2166851876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 0x7f635006f810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6338009830 tx=0x7f633802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.237+0000 7f6356800640 1 -- 172.21.15.60:0/2166851876 shutdown_connections 2024-09-16T15:44:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.237+0000 7f6356800640 1 --2- 172.21.15.60:0/2166851876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 0x7f635006f810 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.237+0000 7f6356800640 1 -- 172.21.15.60:0/2166851876 >> 172.21.15.60:0/2166851876 conn(0x7f63500b18e0 msgr2=0x7f63500b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.237+0000 7f6356800640 1 -- 172.21.15.60:0/2166851876 shutdown_connections 2024-09-16T15:44:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.237+0000 7f6356800640 1 -- 172.21.15.60:0/2166851876 wait complete. 2024-09-16T15:44:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.238+0000 7f6356800640 1 Processor -- start 2024-09-16T15:44:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.239+0000 7f6356800640 1 -- start start 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.239+0000 7f6356800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63500ccb00 0x7f6350142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.239+0000 7f6356800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 0x7f63501512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.239+0000 7f6356800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f635015f870 0x7f635009d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f6356800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f63500cd3b0 con 0x7f635015f870 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f6356800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f63500cd230 con 0x7f63501565b0 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f6356800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f63500739a0 con 0x7f63500ccb00 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f635015f870 0x7f635009d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f635015f870 0x7f635009d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43810/0 (socket says 172.21.15.60:43810) 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634fe00640 1 -- 172.21.15.60:0/3537069263 learned_addr learned my addr 172.21.15.60:0/3537069263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f6346a00640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 0x7f63501512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634f400640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63500ccb00 0x7f6350142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634fe00640 1 -- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63500ccb00 msgr2=0x7f6350142710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634fe00640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63500ccb00 0x7f6350142710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634fe00640 1 -- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 msgr2=0x7f63501512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634fe00640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 0x7f63501512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634fe00640 1 -- 172.21.15.60:0/3537069263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6350151d70 con 0x7f635015f870 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f634f400640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63500ccb00 0x7f6350142710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.240+0000 7f6346a00640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 0x7f63501512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.241+0000 7f634fe00640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f635015f870 0x7f635009d010 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f634800cc40 tx=0x7f634800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.241+0000 7f6347e00640 1 -- 172.21.15.60:0/3537069263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348016020 con 0x7f635015f870 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.241+0000 7f6347e00640 1 -- 172.21.15.60:0/3537069263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6348012420 con 0x7f635015f870 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.241+0000 7f6347e00640 1 -- 172.21.15.60:0/3537069263 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348012850 con 0x7f635015f870 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.241+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6350152050 con 0x7f635015f870 2024-09-16T15:44:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.241+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6350152610 con 0x7f635015f870 2024-09-16T15:44:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.242+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6350069360 con 0x7f635015f870 2024-09-16T15:44:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.243+0000 7f6347e00640 1 -- 172.21.15.60:0/3537069263 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) ==== 101389+0+0 (secure 0 0 0) 0x7f6348019020 con 0x7f635015f870 2024-09-16T15:44:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.244+0000 7f6347e00640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f632c078970 0x7f632c07ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.244+0000 7f6347e00640 1 -- 172.21.15.60:0/3537069263 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f634809b210 con 0x7f635015f870 2024-09-16T15:44:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.245+0000 7f634f400640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f632c078970 0x7f632c07ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.245+0000 7f634f400640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f632c078970 0x7f632c07ae30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6340007fd0 tx=0x7f6340009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:12.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.248+0000 7f6347e00640 1 -- 172.21.15.60:0/3537069263 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6348020030 con 0x7f635015f870 2024-09-16T15:44:12.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.478+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol62870", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7f63500cbcc0 con 0x7f632c078970 2024-09-16T15:44:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.514+0000 7f6347e00640 1 -- 172.21.15.60:0/3537069263 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7f63500cbcc0 con 0x7f632c078970 2024-09-16T15:44:12.516 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:44:12.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:44:12.516 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_used": 0 2024-09-16T15:44:12.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:12.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:44:12.517 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_quota": 10485760 2024-09-16T15:44:12.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:12.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:44:12.517 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_pcent": "0.00" 2024-09-16T15:44:12.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:12.517 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f632c078970 msgr2=0x7f632c07ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f632c078970 0x7f632c07ae30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6340007fd0 tx=0x7f6340009040 comp rx=0 tx=0).stop 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f635015f870 msgr2=0x7f635009d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f635015f870 0x7f635009d010 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f634800cc40 tx=0x7f634800a430 comp rx=0 tx=0).stop 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 shutdown_connections 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f632c078970 0x7f632c07ae30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f635015f870 0x7f635009d010 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63501565b0 0x7f63501512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 --2- 172.21.15.60:0/3537069263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63500ccb00 0x7f6350142710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.516+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 >> 172.21.15.60:0/3537069263 conn(0x7f63500b18e0 msgr2=0x7f635015d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.517+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 shutdown_connections 2024-09-16T15:44:12.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.517+0000 7f6356800640 1 -- 172.21.15.60:0/3537069263 wait complete. 2024-09-16T15:44:12.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol62870 2024-09-16T15:44:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.698+0000 7f146d400640 1 -- 172.21.15.60:0/3854715747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680cec60 msgr2=0x7f14680cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.698+0000 7f146d400640 1 --2- 172.21.15.60:0/3854715747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680cec60 0x7f14680cf0b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1460005fd0 tx=0x7f146002aa10 comp rx=0 tx=0).stop 2024-09-16T15:44:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.698+0000 7f146d400640 1 -- 172.21.15.60:0/3854715747 shutdown_connections 2024-09-16T15:44:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.698+0000 7f146d400640 1 --2- 172.21.15.60:0/3854715747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680cec60 0x7f14680cf0b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.698+0000 7f146d400640 1 --2- 172.21.15.60:0/3854715747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14680af190 0x7f14680b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.698+0000 7f146d400640 1 -- 172.21.15.60:0/3854715747 >> 172.21.15.60:0/3854715747 conn(0x7f14680b1840 msgr2=0x7f14680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.698+0000 7f146d400640 1 -- 172.21.15.60:0/3854715747 shutdown_connections 2024-09-16T15:44:12.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.698+0000 7f146d400640 1 -- 172.21.15.60:0/3854715747 wait complete. 2024-09-16T15:44:12.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.700+0000 7f146d400640 1 Processor -- start 2024-09-16T15:44:12.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.700+0000 7f146d400640 1 -- start start 2024-09-16T15:44:12.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f146d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680af190 0x7f14681f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f146d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14680cec60 0x7f146814a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f146d400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1468161760 0x7f14680cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f146d400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1468162310 con 0x7f14680cec60 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f146d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1468162190 con 0x7f14680af190 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f146d400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1468162490 con 0x7f1468161760 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f145de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14680cec60 0x7f146814a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f145de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14680cec60 0x7f146814a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43824/0 (socket says 172.21.15.60:43824) 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f145de00640 1 -- 172.21.15.60:0/30280435 learned_addr learned my addr 172.21.15.60:0/30280435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f1467200640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1468161760 0x7f14680cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f1466800640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680af190 0x7f14681f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f145de00640 1 -- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1468161760 msgr2=0x7f14680cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f145de00640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1468161760 0x7f14680cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f145de00640 1 -- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680af190 msgr2=0x7f14681f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.701+0000 7f145de00640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680af190 0x7f14681f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.702+0000 7f145de00640 1 -- 172.21.15.60:0/30280435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14681e8ac0 con 0x7f14680cec60 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.702+0000 7f1466800640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680af190 0x7f14681f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.702+0000 7f145de00640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14680cec60 0x7f146814a570 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f145000e9f0 tx=0x7f145000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.702+0000 7f145f400640 1 -- 172.21.15.60:0/30280435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145000cd20 con 0x7f14680cec60 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.702+0000 7f145f400640 1 -- 172.21.15.60:0/30280435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1450005020 con 0x7f14680cec60 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.702+0000 7f145f400640 1 -- 172.21.15.60:0/30280435 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450005450 con 0x7f14680cec60 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.702+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14681e87e0 con 0x7f14680cec60 2024-09-16T15:44:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.702+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1468159d20 con 0x7f14680cec60 2024-09-16T15:44:12.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.703+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1468162940 con 0x7f14680cec60 2024-09-16T15:44:12.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.705+0000 7f145f400640 1 -- 172.21.15.60:0/30280435 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 101440+0+0 (secure 0 0 0) 0x7f14500047a0 con 0x7f14680cec60 2024-09-16T15:44:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.706+0000 7f145f400640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1444078a30 0x7f144407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.706+0000 7f145f400640 1 -- 172.21.15.60:0/30280435 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f145009b7c0 con 0x7f14680cec60 2024-09-16T15:44:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.706+0000 7f1466800640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1444078a30 0x7f144407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.707+0000 7f1466800640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1444078a30 0x7f144407aef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f14580016a0 tx=0x7f1458009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:12.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.710+0000 7f145f400640 1 -- 172.21.15.60:0/30280435 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1450063b10 con 0x7f14680cec60 2024-09-16T15:44:12.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.934+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol62870", "target": ["mon-mgr", ""]}) -- 0x7f1468073ab0 con 0x7f1444078a30 2024-09-16T15:44:12.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.963+0000 7f145f400640 1 -- 172.21.15.60:0/30280435 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+698 (secure 0 0 0) 0x7f1468073ab0 con 0x7f1444078a30 2024-09-16T15:44:12.965 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:12.965 INFO:teuthology.orchestra.run.smithi060.stdout: "atime": "2024-09-16 15:44:10", 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_pcent": "0.00", 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_quota": 10485760, 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_used": 0, 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "created_at": "2024-09-16 15:44:10", 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "ctime": "2024-09-16 15:44:12", 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "data_pool": "cephfs_data", 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "earmark": "", 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "features": [ 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot-clone", 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot-autoprotect" 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flavor": 1, 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 0, 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": 16877, 2024-09-16T15:44:12.966 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_addrs": [ 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6789", 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.100:6789", 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.100:6790" 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "mtime": "2024-09-16 15:44:10", 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177", 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_namespace": "", 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "complete", 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "subvolume", 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout: "uid": 0 2024-09-16T15:44:12.967 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.965+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1444078a30 msgr2=0x7f144407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.965+0000 7f146d400640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1444078a30 0x7f144407aef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f14580016a0 tx=0x7f1458009040 comp rx=0 tx=0).stop 2024-09-16T15:44:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14680cec60 msgr2=0x7f146814a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14680cec60 0x7f146814a570 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f145000e9f0 tx=0x7f145000edf0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 shutdown_connections 2024-09-16T15:44:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1444078a30 0x7f144407aef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1468161760 0x7f14680cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14680cec60 0x7f146814a570 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 --2- 172.21.15.60:0/30280435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14680af190 0x7f14681f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 >> 172.21.15.60:0/30280435 conn(0x7f14680b1840 msgr2=0x7f1468072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 shutdown_connections 2024-09-16T15:44:12.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:12.966+0000 7f146d400640 1 -- 172.21.15.60:0/30280435 wait complete. 2024-09-16T15:44:12.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol62870 --mode 777 --uid 1000 --gid 1000 2024-09-16T15:44:13.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.151+0000 7fed5d200640 1 -- 172.21.15.60:0/2767253041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580aeef0 msgr2=0x7fed58074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.151+0000 7fed5d200640 1 --2- 172.21.15.60:0/2767253041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580aeef0 0x7fed58074530 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fed40005fd0 tx=0x7fed4002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.151+0000 7fed5d200640 1 -- 172.21.15.60:0/2767253041 shutdown_connections 2024-09-16T15:44:13.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.151+0000 7fed5d200640 1 --2- 172.21.15.60:0/2767253041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed580b5520 0x7fed581603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.151+0000 7fed5d200640 1 --2- 172.21.15.60:0/2767253041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580aeef0 0x7fed58074530 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.151+0000 7fed5d200640 1 -- 172.21.15.60:0/2767253041 >> 172.21.15.60:0/2767253041 conn(0x7fed580cfa10 msgr2=0x7fed580cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:13.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.151+0000 7fed5d200640 1 -- 172.21.15.60:0/2767253041 shutdown_connections 2024-09-16T15:44:13.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.151+0000 7fed5d200640 1 -- 172.21.15.60:0/2767253041 wait complete. 2024-09-16T15:44:13.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.153+0000 7fed5d200640 1 Processor -- start 2024-09-16T15:44:13.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.153+0000 7fed5d200640 1 -- start start 2024-09-16T15:44:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed5d200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed580aeef0 0x7fed5815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed5d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed580b5520 0x7fed5815e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed5d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580ceef0 0x7fed580ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed5d200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fed5815b800 con 0x7fed580ceef0 2024-09-16T15:44:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed5d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed5815b680 con 0x7fed580b5520 2024-09-16T15:44:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed5d200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fed5815b980 con 0x7fed580aeef0 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed57e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580ceef0 0x7fed580ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed57e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580ceef0 0x7fed580ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43846/0 (socket says 172.21.15.60:43846) 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed57e00640 1 -- 172.21.15.60:0/2458060436 learned_addr learned my addr 172.21.15.60:0/2458060436 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.154+0000 7fed57400640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed580aeef0 0x7fed5815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed4ea00640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed580b5520 0x7fed5815e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed57e00640 1 -- 172.21.15.60:0/2458060436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed580aeef0 msgr2=0x7fed5815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed57e00640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed580aeef0 0x7fed5815d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed57e00640 1 -- 172.21.15.60:0/2458060436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed580b5520 msgr2=0x7fed5815e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed57e00640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed580b5520 0x7fed5815e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed57e00640 1 -- 172.21.15.60:0/2458060436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed58158520 con 0x7fed580ceef0 2024-09-16T15:44:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed57e00640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580ceef0 0x7fed580ca060 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fed5000b9e0 tx=0x7fed5000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed4fe00640 1 -- 172.21.15.60:0/2458060436 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50019070 con 0x7fed580ceef0 2024-09-16T15:44:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed4fe00640 1 -- 172.21.15.60:0/2458060436 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed5000cbb0 con 0x7fed580ceef0 2024-09-16T15:44:13.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed4fe00640 1 -- 172.21.15.60:0/2458060436 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50014440 con 0x7fed580ceef0 2024-09-16T15:44:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.155+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed580c7bd0 con 0x7fed580ceef0 2024-09-16T15:44:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.156+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed580c8190 con 0x7fed580ceef0 2024-09-16T15:44:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.157+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed581603d0 con 0x7fed580ceef0 2024-09-16T15:44:13.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.158+0000 7fed4fe00640 1 -- 172.21.15.60:0/2458060436 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 101440+0+0 (secure 0 0 0) 0x7fed5001c070 con 0x7fed580ceef0 2024-09-16T15:44:13.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.159+0000 7fed4fe00640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fed34078a00 0x7fed3407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:13.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.160+0000 7fed4fe00640 1 -- 172.21.15.60:0/2458060436 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fed5009a990 con 0x7fed580ceef0 2024-09-16T15:44:13.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.160+0000 7fed57400640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fed34078a00 0x7fed3407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:13.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.160+0000 7fed57400640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fed34078a00 0x7fed3407aec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fed48001720 tx=0x7fed48009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:13.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.164+0000 7fed4fe00640 1 -- 172.21.15.60:0/2458060436 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fed50062680 con 0x7fed580ceef0 2024-09-16T15:44:13.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.388+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol62870", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7fed580706c0 con 0x7fed34078a00 2024-09-16T15:44:13.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.425+0000 7fed4fe00640 1 -- 172.21.15.60:0/2458060436 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fed580706c0 con 0x7fed34078a00 2024-09-16T15:44:13.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fed34078a00 msgr2=0x7fed3407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fed34078a00 0x7fed3407aec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fed48001720 tx=0x7fed48009040 comp rx=0 tx=0).stop 2024-09-16T15:44:13.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580ceef0 msgr2=0x7fed580ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580ceef0 0x7fed580ca060 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fed5000b9e0 tx=0x7fed5000bea0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 shutdown_connections 2024-09-16T15:44:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fed34078a00 0x7fed3407aec0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed580ceef0 0x7fed580ca060 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed580b5520 0x7fed5815e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 --2- 172.21.15.60:0/2458060436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed580aeef0 0x7fed5815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 >> 172.21.15.60:0/2458060436 conn(0x7fed580cfa10 msgr2=0x7fed580cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 shutdown_connections 2024-09-16T15:44:13.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.428+0000 7fed5d200640 1 -- 172.21.15.60:0/2458060436 wait complete. 2024-09-16T15:44:13.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol62870 2024-09-16T15:44:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.609+0000 7fc32a000640 1 -- 172.21.15.60:0/3590379171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc32415bec0 msgr2=0x7fc32406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.609+0000 7fc32a000640 1 --2- 172.21.15.60:0/3590379171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc32415bec0 0x7fc32406fce0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc314009830 tx=0x7fc31402a510 comp rx=0 tx=0).stop 2024-09-16T15:44:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.609+0000 7fc32a000640 1 -- 172.21.15.60:0/3590379171 shutdown_connections 2024-09-16T15:44:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.609+0000 7fc32a000640 1 --2- 172.21.15.60:0/3590379171 >> v2:172.21.15.100:3300/0 conn(0x7fc3240cbc00 0x7fc324073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.609+0000 7fc32a000640 1 --2- 172.21.15.60:0/3590379171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc32415bec0 0x7fc32406fce0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.609+0000 7fc32a000640 1 -- 172.21.15.60:0/3590379171 >> 172.21.15.60:0/3590379171 conn(0x7fc3240cf9f0 msgr2=0x7fc3240cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.609+0000 7fc32a000640 1 -- 172.21.15.60:0/3590379171 shutdown_connections 2024-09-16T15:44:13.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.609+0000 7fc32a000640 1 -- 172.21.15.60:0/3590379171 wait complete. 2024-09-16T15:44:13.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.611+0000 7fc32a000640 1 Processor -- start 2024-09-16T15:44:13.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.611+0000 7fc32a000640 1 -- start start 2024-09-16T15:44:13.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc32a000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324072560 0x7fc3240abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:13.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc32a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240cbc00 0x7fc3240ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:13.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc32a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32415bec0 0x7fc32414be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:13.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc32a000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240ae9e0 con 0x7fc3240cbc00 2024-09-16T15:44:13.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc32a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240ae860 con 0x7fc32415bec0 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc32a000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc3240aeb60 con 0x7fc324072560 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc31be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240cbc00 0x7fc3240ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc31be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240cbc00 0x7fc3240ac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43848/0 (socket says 172.21.15.60:43848) 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc31be00640 1 -- 172.21.15.60:0/608532463 learned_addr learned my addr 172.21.15.60:0/608532463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc328c00640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324072560 0x7fc3240abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc329600640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32415bec0 0x7fc32414be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc31be00640 1 -- 172.21.15.60:0/608532463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324072560 msgr2=0x7fc3240abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc31be00640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324072560 0x7fc3240abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc31be00640 1 -- 172.21.15.60:0/608532463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32415bec0 msgr2=0x7fc32414be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc31be00640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32415bec0 0x7fc32414be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.612+0000 7fc31be00640 1 -- 172.21.15.60:0/608532463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32415dae0 con 0x7fc3240cbc00 2024-09-16T15:44:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.613+0000 7fc31be00640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240cbc00 0x7fc3240ac3d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00c430 tx=0x7fc30c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.613+0000 7fc321600640 1 -- 172.21.15.60:0/608532463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c017070 con 0x7fc3240cbc00 2024-09-16T15:44:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.613+0000 7fc321600640 1 -- 172.21.15.60:0/608532463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc30c0035f0 con 0x7fc3240cbc00 2024-09-16T15:44:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.613+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32415ddc0 con 0x7fc3240cbc00 2024-09-16T15:44:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.613+0000 7fc321600640 1 -- 172.21.15.60:0/608532463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c003ac0 con 0x7fc3240cbc00 2024-09-16T15:44:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.613+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32415e380 con 0x7fc3240cbc00 2024-09-16T15:44:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.614+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3240af010 con 0x7fc3240cbc00 2024-09-16T15:44:13.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.615+0000 7fc321600640 1 -- 172.21.15.60:0/608532463 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 101440+0+0 (secure 0 0 0) 0x7fc30c01a070 con 0x7fc3240cbc00 2024-09-16T15:44:13.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.616+0000 7fc321600640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078a00 0x7fc30007aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:13.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.617+0000 7fc321600640 1 -- 172.21.15.60:0/608532463 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fc30c099f80 con 0x7fc3240cbc00 2024-09-16T15:44:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.617+0000 7fc328c00640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078a00 0x7fc30007aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:13.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.617+0000 7fc328c00640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078a00 0x7fc30007aec0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc314034000 tx=0x7fc314004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:13.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.621+0000 7fc321600640 1 -- 172.21.15.60:0/608532463 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc30c061c70 con 0x7fc3240cbc00 2024-09-16T15:44:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.847+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol62870", "target": ["mon-mgr", ""]}) -- 0x7fc32415b6e0 con 0x7fc300078a00 2024-09-16T15:44:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.876+0000 7fc321600640 1 -- 172.21.15.60:0/608532463 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fc32415b6e0 con 0x7fc300078a00 2024-09-16T15:44:13.878 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.878+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078a00 msgr2=0x7fc30007aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.878+0000 7fc32a000640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078a00 0x7fc30007aec0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc314034000 tx=0x7fc314004730 comp rx=0 tx=0).stop 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.878+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240cbc00 msgr2=0x7fc3240ac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.878+0000 7fc32a000640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240cbc00 0x7fc3240ac3d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00c430 tx=0x7fc30c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.879+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 shutdown_connections 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.879+0000 7fc32a000640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078a00 0x7fc30007aec0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.879+0000 7fc32a000640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc32415bec0 0x7fc32414be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.879+0000 7fc32a000640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240cbc00 0x7fc3240ac3d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.879+0000 7fc32a000640 1 --2- 172.21.15.60:0/608532463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324072560 0x7fc3240abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.879+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 >> 172.21.15.60:0/608532463 conn(0x7fc3240cf9f0 msgr2=0x7fc3240af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:13.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.879+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 shutdown_connections 2024-09-16T15:44:13.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:13.879+0000 7fc32a000640 1 -- 172.21.15.60:0/608532463 wait complete. 2024-09-16T15:44:13.897 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol62870 with 8 files each 1MB size under directory volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177 2024-09-16T15:44:13.897 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:13.897 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:13.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:13.933 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-16T15:44:14.028 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:44:14.028 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:44:14.028 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176845 s, 59.3 MB/s 2024-09-16T15:44:14.029 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:14.029 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:14.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:14.064 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-16T15:44:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:44:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:44:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227722 s, 46.0 MB/s 2024-09-16T15:44:14.164 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:14.164 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:14.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:14.199 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-16T15:44:14.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:44:14.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:44:14.296 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190151 s, 55.1 MB/s 2024-09-16T15:44:14.297 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:14.297 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:14.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:14.332 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-16T15:44:14.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:44:14.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:44:14.424 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156259 s, 67.1 MB/s 2024-09-16T15:44:14.425 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:14.425 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:14.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:14.461 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-16T15:44:14.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:44:14.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:44:14.557 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177138 s, 59.2 MB/s 2024-09-16T15:44:14.558 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:14.558 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:14.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:14.593 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-16T15:44:14.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:44:14.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:44:14.687 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167034 s, 62.8 MB/s 2024-09-16T15:44:14.688 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:14.688 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:14.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:14.724 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-16T15:44:14.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:44:14.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:44:14.818 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174749 s, 60.0 MB/s 2024-09-16T15:44:14.819 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:44:14.819 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:14.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:14.855 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-16T15:44:14.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-16T15:44:14.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-16T15:44:14.948 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170815 s, 61.4 MB/s 2024-09-16T15:44:14.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol62870 subvol_snap57910 2024-09-16T15:44:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.114+0000 7f2e1f600640 1 -- 172.21.15.60:0/1169193834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e18073de0 msgr2=0x7f2e18074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.114+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1169193834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e18073de0 0x7f2e18074240 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009830 tx=0x7f2e0002a480 comp rx=0 tx=0).stop 2024-09-16T15:44:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.114+0000 7f2e1f600640 1 -- 172.21.15.60:0/1169193834 shutdown_connections 2024-09-16T15:44:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.114+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1169193834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e18073de0 0x7f2e18074240 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.114+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1169193834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e1815f9a0 0x7f2e1815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.114+0000 7f2e1f600640 1 -- 172.21.15.60:0/1169193834 >> 172.21.15.60:0/1169193834 conn(0x7f2e180b18e0 msgr2=0x7f2e180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.114+0000 7f2e1f600640 1 -- 172.21.15.60:0/1169193834 shutdown_connections 2024-09-16T15:44:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.114+0000 7f2e1f600640 1 -- 172.21.15.60:0/1169193834 wait complete. 2024-09-16T15:44:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.115+0000 7f2e1f600640 1 Processor -- start 2024-09-16T15:44:15.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.115+0000 7f2e1f600640 1 -- start start 2024-09-16T15:44:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.116+0000 7f2e1f600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e18073de0 0x7f2e181f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.116+0000 7f2e1f600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e1815c3f0 0x7f2e1809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.116+0000 7f2e1f600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e1815f9a0 0x7f2e1809af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.116+0000 7f2e1f600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2e180cd3d0 con 0x7f2e1815c3f0 2024-09-16T15:44:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.116+0000 7f2e1f600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e180ccbd0 con 0x7f2e1815f9a0 2024-09-16T15:44:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.116+0000 7f2e1f600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2e180cd550 con 0x7f2e18073de0 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e1815c3f0 0x7f2e1809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e1815c3f0 0x7f2e1809aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43864/0 (socket says 172.21.15.60:43864) 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 -- 172.21.15.60:0/1771989711 learned_addr learned my addr 172.21.15.60:0/1771989711 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e1cc00640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e18073de0 0x7f2e181f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e1d600640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e1815f9a0 0x7f2e1809af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 -- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e18073de0 msgr2=0x7f2e181f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e18073de0 0x7f2e181f7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 -- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e1815f9a0 msgr2=0x7f2e1809af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e1815f9a0 0x7f2e1809af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 -- 172.21.15.60:0/1771989711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1809d860 con 0x7f2e1815c3f0 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e1cc00640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e18073de0 0x7f2e181f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.117+0000 7f2e0fe00640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e1815c3f0 0x7f2e1809aa00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009950 tx=0x7f2e00032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:15.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.118+0000 7f2e15600640 1 -- 172.21.15.60:0/1771989711 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00040070 con 0x7f2e1815c3f0 2024-09-16T15:44:15.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.118+0000 7f2e15600640 1 -- 172.21.15.60:0/1771989711 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e00005310 con 0x7f2e1815c3f0 2024-09-16T15:44:15.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.118+0000 7f2e15600640 1 -- 172.21.15.60:0/1771989711 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000047a0 con 0x7f2e1815c3f0 2024-09-16T15:44:15.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.118+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e180b2e80 con 0x7f2e1815c3f0 2024-09-16T15:44:15.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.118+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e181f5c80 con 0x7f2e1815c3f0 2024-09-16T15:44:15.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.119+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e18069360 con 0x7f2e1815c3f0 2024-09-16T15:44:15.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.120+0000 7f2e15600640 1 -- 172.21.15.60:0/1771989711 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 101440+0+0 (secure 0 0 0) 0x7f2e00039040 con 0x7f2e1815c3f0 2024-09-16T15:44:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.121+0000 7f2e15600640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2df4078a30 0x7f2df407aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.121+0000 7f2e15600640 1 -- 172.21.15.60:0/1771989711 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f2e000bc240 con 0x7f2e1815c3f0 2024-09-16T15:44:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.121+0000 7f2e1cc00640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2df4078a30 0x7f2df407aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:15.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.122+0000 7f2e1cc00640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2df4078a30 0x7f2df407aef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e08005fd0 tx=0x7f2e0800f360 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.125+0000 7f2e15600640 1 -- 172.21.15.60:0/1771989711 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2e00084440 con 0x7f2e1815c3f0 2024-09-16T15:44:15.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.352+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol62870", "snap_name": "subvol_snap57910", "target": ["mon-mgr", ""]}) -- 0x7f2e181f7e80 con 0x7f2df4078a30 2024-09-16T15:44:15.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.386+0000 7f2e15600640 1 -- 172.21.15.60:0/1771989711 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2e0800fb80 con 0x7f2df4078a30 2024-09-16T15:44:15.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2df4078a30 msgr2=0x7f2df407aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2df4078a30 0x7f2df407aef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e08005fd0 tx=0x7f2e0800f360 comp rx=0 tx=0).stop 2024-09-16T15:44:15.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e1815c3f0 msgr2=0x7f2e1809aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e1815c3f0 0x7f2e1809aa00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009950 tx=0x7f2e00032c10 comp rx=0 tx=0).stop 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 shutdown_connections 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2df4078a30 0x7f2df407aef0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e1815f9a0 0x7f2e1809af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e1815c3f0 0x7f2e1809aa00 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 --2- 172.21.15.60:0/1771989711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e18073de0 0x7f2e181f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 >> 172.21.15.60:0/1771989711 conn(0x7f2e180b18e0 msgr2=0x7f2e180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 shutdown_connections 2024-09-16T15:44:15.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.388+0000 7f2e1f600640 1 -- 172.21.15.60:0/1771989711 wait complete. 2024-09-16T15:44:15.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol62870 subvol_snap57910 subvol_clone94430 2024-09-16T15:44:15.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.576+0000 7fa4be200640 1 -- 172.21.15.60:0/3152317246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b80cccd0 msgr2=0x7fa4b8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.576+0000 7fa4be200640 1 --2- 172.21.15.60:0/3152317246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b80cccd0 0x7fa4b8073440 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0005fd0 tx=0x7fa4a002a540 comp rx=0 tx=0).stop 2024-09-16T15:44:15.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.576+0000 7fa4be200640 1 -- 172.21.15.60:0/3152317246 shutdown_connections 2024-09-16T15:44:15.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.576+0000 7fa4be200640 1 --2- 172.21.15.60:0/3152317246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4b806efa0 0x7fa4b80cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.576+0000 7fa4be200640 1 --2- 172.21.15.60:0/3152317246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b80cccd0 0x7fa4b8073440 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.576+0000 7fa4be200640 1 -- 172.21.15.60:0/3152317246 >> 172.21.15.60:0/3152317246 conn(0x7fa4b80b1840 msgr2=0x7fa4b80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:15.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.576+0000 7fa4be200640 1 -- 172.21.15.60:0/3152317246 shutdown_connections 2024-09-16T15:44:15.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.576+0000 7fa4be200640 1 -- 172.21.15.60:0/3152317246 wait complete. 2024-09-16T15:44:15.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.577+0000 7fa4be200640 1 Processor -- start 2024-09-16T15:44:15.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.578+0000 7fa4be200640 1 -- start start 2024-09-16T15:44:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.578+0000 7fa4be200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b806efa0 0x7fa4b815e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.578+0000 7fa4be200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4b80cccd0 0x7fa4b81f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4be200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b8145750 0x7fa4b80b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4be200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80ce480 con 0x7fa4b806efa0 2024-09-16T15:44:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4be200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80ce300 con 0x7fa4b8145750 2024-09-16T15:44:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4be200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b80ce600 con 0x7fa4b80cccd0 2024-09-16T15:44:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b806efa0 0x7fa4b815e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:15.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b806efa0 0x7fa4b815e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43884/0 (socket says 172.21.15.60:43884) 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 -- 172.21.15.60:0/3798332919 learned_addr learned my addr 172.21.15.60:0/3798332919 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4afe00640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4b80cccd0 0x7fa4b81f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7e00640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b8145750 0x7fa4b80b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 -- 172.21.15.60:0/3798332919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4b80cccd0 msgr2=0x7fa4b81f8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4b80cccd0 0x7fa4b81f8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 -- 172.21.15.60:0/3798332919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b8145750 msgr2=0x7fa4b80b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b8145750 0x7fa4b80b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 -- 172.21.15.60:0/3798332919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b81f9080 con 0x7fa4b806efa0 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.579+0000 7fa4b7400640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b806efa0 0x7fa4b815e370 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800c3c0 tx=0x7fa4a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.580+0000 7fa4af400640 1 -- 172.21.15.60:0/3798332919 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8017070 con 0x7fa4b806efa0 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.580+0000 7fa4af400640 1 -- 172.21.15.60:0/3798332919 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4a80035f0 con 0x7fa4b806efa0 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.580+0000 7fa4af400640 1 -- 172.21.15.60:0/3798332919 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8003a80 con 0x7fa4b806efa0 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.580+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b81f4de0 con 0x7fa4b806efa0 2024-09-16T15:44:15.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.580+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81f53a0 con 0x7fa4b806efa0 2024-09-16T15:44:15.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.581+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b80cdbb0 con 0x7fa4b806efa0 2024-09-16T15:44:15.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.582+0000 7fa4af400640 1 -- 172.21.15.60:0/3798332919 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 101440+0+0 (secure 0 0 0) 0x7fa4a801a070 con 0x7fa4b806efa0 2024-09-16T15:44:15.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.583+0000 7fa4af400640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa494078a00 0x7fa49407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:15.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.583+0000 7fa4af400640 1 -- 172.21.15.60:0/3798332919 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fa4a8099f90 con 0x7fa4b806efa0 2024-09-16T15:44:15.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.583+0000 7fa4afe00640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa494078a00 0x7fa49407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:15.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.584+0000 7fa4afe00640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa494078a00 0x7fa49407aec0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0033000 tx=0x7fa4a0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:15.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.587+0000 7fa4af400640 1 -- 172.21.15.60:0/3798332919 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa4a80622c0 con 0x7fa4b806efa0 2024-09-16T15:44:15.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:15.819+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol62870", "snap_name": "subvol_snap57910", "target_sub_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7fa4b80af3a0 con 0x7fa494078a00 2024-09-16T15:44:19.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.252+0000 7fa4af400640 1 -- 172.21.15.60:0/3798332919 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa4b80af3a0 con 0x7fa494078a00 2024-09-16T15:44:19.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa494078a00 msgr2=0x7fa49407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa494078a00 0x7fa49407aec0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0033000 tx=0x7fa4a0004730 comp rx=0 tx=0).stop 2024-09-16T15:44:19.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b806efa0 msgr2=0x7fa4b815e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b806efa0 0x7fa4b815e370 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800c3c0 tx=0x7fa4a800c880 comp rx=0 tx=0).stop 2024-09-16T15:44:19.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 shutdown_connections 2024-09-16T15:44:19.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa494078a00 0x7fa49407aec0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4b8145750 0x7fa4b80b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4b80cccd0 0x7fa4b81f8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 --2- 172.21.15.60:0/3798332919 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4b806efa0 0x7fa4b815e370 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 >> 172.21.15.60:0/3798332919 conn(0x7fa4b80b1840 msgr2=0x7fa4b81ea750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:19.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 shutdown_connections 2024-09-16T15:44:19.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.256+0000 7fa4be200640 1 -- 172.21.15.60:0/3798332919 wait complete. 2024-09-16T15:44:19.260 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.258+0000 7fcb8d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.260 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.258+0000 7fcb8d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.260 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.258+0000 7fcb8d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.260 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.258+0000 7fcb8d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.260 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.258+0000 7fcb8d200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:19.284 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.279+0000 7fcb8b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.284 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.279+0000 7fcb8b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.284 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.279+0000 7fcb8b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.284 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.279+0000 7fcb8b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.284 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:19.279+0000 7fcb8b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.452+0000 7f321ba00640 1 -- 172.21.15.60:0/398775562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3214161760 msgr2=0x7f32140ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.452+0000 7f321ba00640 1 --2- 172.21.15.60:0/398775562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3214161760 0x7f32140ce4a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3204009830 tx=0x7f320402a500 comp rx=0 tx=0).stop 2024-09-16T15:44:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.452+0000 7f321ba00640 1 -- 172.21.15.60:0/398775562 shutdown_connections 2024-09-16T15:44:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.452+0000 7f321ba00640 1 --2- 172.21.15.60:0/398775562 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32140af190 0x7f32140b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.452+0000 7f321ba00640 1 --2- 172.21.15.60:0/398775562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3214161760 0x7f32140ce4a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.452+0000 7f321ba00640 1 -- 172.21.15.60:0/398775562 >> 172.21.15.60:0/398775562 conn(0x7f32140b1840 msgr2=0x7f32140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.452+0000 7f321ba00640 1 -- 172.21.15.60:0/398775562 shutdown_connections 2024-09-16T15:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.452+0000 7f321ba00640 1 -- 172.21.15.60:0/398775562 wait complete. 2024-09-16T15:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.454+0000 7f321ba00640 1 Processor -- start 2024-09-16T15:44:19.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.454+0000 7f321ba00640 1 -- start start 2024-09-16T15:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f321ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32140af190 0x7f3214148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f321ba00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32140cec60 0x7f32141f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f321ba00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214161760 0x7f32141601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f321ba00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3214162a20 con 0x7f3214161760 2024-09-16T15:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f321ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f32141628a0 con 0x7f32140af190 2024-09-16T15:44:19.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f321ba00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3214162ba0 con 0x7f32140cec60 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f3219a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214161760 0x7f32141601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f3219a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214161760 0x7f32141601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37224/0 (socket says 172.21.15.60:37224) 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f3219000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32140af190 0x7f3214148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f3219a00640 1 -- 172.21.15.60:0/3167288242 learned_addr learned my addr 172.21.15.60:0/3167288242 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.455+0000 7f3213e00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32140cec60 0x7f32141f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3219a00640 1 -- 172.21.15.60:0/3167288242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32140cec60 msgr2=0x7f32141f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3219a00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32140cec60 0x7f32141f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3219a00640 1 -- 172.21.15.60:0/3167288242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32140af190 msgr2=0x7f3214148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3219a00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32140af190 0x7f3214148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3219a00640 1 -- 172.21.15.60:0/3167288242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32141582f0 con 0x7f3214161760 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3219a00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214161760 0x7f32141601d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f320c00cbc0 tx=0x7f320c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3210c00640 1 -- 172.21.15.60:0/3167288242 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c016020 con 0x7f3214161760 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3210c00640 1 -- 172.21.15.60:0/3167288242 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f320c00add0 con 0x7f3214161760 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f3210c00640 1 -- 172.21.15.60:0/3167288242 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c012680 con 0x7f3214161760 2024-09-16T15:44:19.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32141585d0 con 0x7f3214161760 2024-09-16T15:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.456+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3214158b90 con 0x7f3214161760 2024-09-16T15:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.458+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32140b4d70 con 0x7f3214161760 2024-09-16T15:44:19.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.459+0000 7f3210c00640 1 -- 172.21.15.60:0/3167288242 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) ==== 101440+0+0 (secure 0 0 0) 0x7f320c019020 con 0x7f3214161760 2024-09-16T15:44:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.460+0000 7f3210c00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f31f4078a00 0x7f31f407aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.460+0000 7f3210c00640 1 -- 172.21.15.60:0/3167288242 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f320c09af40 con 0x7f3214161760 2024-09-16T15:44:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.460+0000 7f3219000640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f31f4078a00 0x7f31f407aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:19.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.461+0000 7f3219000640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f31f4078a00 0x7f31f407aec0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3204034000 tx=0x7f3204004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:19.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.465+0000 7f3210c00640 1 -- 172.21.15.60:0/3167288242 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f320c020030 con 0x7f3214161760 2024-09-16T15:44:19.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.701+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f32141494d0 con 0x7f31f4078a00 2024-09-16T15:44:19.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.735+0000 7f3210c00640 1 -- 172.21.15.60:0/3167288242 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7f32141494d0 con 0x7f31f4078a00 2024-09-16T15:44:19.737 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:19.737 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:19.737 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:19.737 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:19.737 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:19.737 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "0.003%", 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "213/8.0M", 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "3/9" 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:19.738 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:19.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.737+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f31f4078a00 msgr2=0x7f31f407aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.737+0000 7f321ba00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f31f4078a00 0x7f31f407aec0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3204034000 tx=0x7f3204004730 comp rx=0 tx=0).stop 2024-09-16T15:44:19.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.737+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214161760 msgr2=0x7f32141601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.737+0000 7f321ba00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214161760 0x7f32141601d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f320c00cbc0 tx=0x7f320c00a430 comp rx=0 tx=0).stop 2024-09-16T15:44:19.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.738+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 shutdown_connections 2024-09-16T15:44:19.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.738+0000 7f321ba00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f31f4078a00 0x7f31f407aec0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.738+0000 7f321ba00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3214161760 0x7f32141601d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.738+0000 7f321ba00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32140cec60 0x7f32141f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.738+0000 7f321ba00640 1 --2- 172.21.15.60:0/3167288242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32140af190 0x7f3214148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.738+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 >> 172.21.15.60:0/3167288242 conn(0x7f32140b1840 msgr2=0x7f32140addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:19.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.738+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 shutdown_connections 2024-09-16T15:44:19.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:19.738+0000 7f321ba00640 1 -- 172.21.15.60:0/3167288242 wait complete. 2024-09-16T15:44:19.755 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:20.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:20.261+0000 7fcb6de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:20.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:20.261+0000 7fcb6de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:20.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:20.261+0000 7fcb6de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:20.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:20.261+0000 7fcb6de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:20.263 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:44:20.261+0000 7fcb6de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:20.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:20.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.930+0000 7f7483200640 1 -- 172.21.15.60:0/2744854984 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74840cf830 msgr2=0x7f74840b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.930+0000 7f7483200640 1 --2- 172.21.15.60:0/2744854984 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74840cf830 0x7f74840b33b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f746c009860 tx=0x7f746c02a4b0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.930+0000 7f7483200640 1 -- 172.21.15.60:0/2744854984 shutdown_connections 2024-09-16T15:44:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.930+0000 7f7483200640 1 --2- 172.21.15.60:0/2744854984 >> v2:172.21.15.100:3300/0 conn(0x7f74840d2a00 0x7f74840b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.930+0000 7f7483200640 1 --2- 172.21.15.60:0/2744854984 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74840cf830 0x7f74840b33b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.930+0000 7f7483200640 1 -- 172.21.15.60:0/2744854984 >> 172.21.15.60:0/2744854984 conn(0x7f748409d0b0 msgr2=0x7f74840d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.930+0000 7f7483200640 1 -- 172.21.15.60:0/2744854984 shutdown_connections 2024-09-16T15:44:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.930+0000 7f7483200640 1 -- 172.21.15.60:0/2744854984 wait complete. 2024-09-16T15:44:20.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.932+0000 7f7483200640 1 Processor -- start 2024-09-16T15:44:20.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.932+0000 7f7483200640 1 -- start start 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7483200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74840cf830 0x7f74840a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7483200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74840d2a00 0x7f7484156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7483200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484161460 0x7f74840cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7483200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74840b56d0 con 0x7f7484161460 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7483200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74840b5150 con 0x7f74840cf830 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7483200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74840b3e30 con 0x7f74840d2a00 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7482800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484161460 0x7f74840cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7482800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484161460 0x7f74840cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37264/0 (socket says 172.21.15.60:37264) 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7482800640 1 -- 172.21.15.60:0/493669858 learned_addr learned my addr 172.21.15.60:0/493669858 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7481e00640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74840cf830 0x7f74840a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.933+0000 7f7479400640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74840d2a00 0x7f7484156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f7482800640 1 -- 172.21.15.60:0/493669858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74840d2a00 msgr2=0x7f7484156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f7482800640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74840d2a00 0x7f7484156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f7482800640 1 -- 172.21.15.60:0/493669858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74840cf830 msgr2=0x7f74840a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f7482800640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74840cf830 0x7f74840a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f7482800640 1 -- 172.21.15.60:0/493669858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74840cc8e0 con 0x7f7484161460 2024-09-16T15:44:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f7482800640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484161460 0x7f74840cd260 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f747c00ef60 tx=0x7f747c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f747aa00640 1 -- 172.21.15.60:0/493669858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c004b20 con 0x7f7484161460 2024-09-16T15:44:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f747aa00640 1 -- 172.21.15.60:0/493669858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f747c004db0 con 0x7f7484161460 2024-09-16T15:44:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74840cb880 con 0x7f7484161460 2024-09-16T15:44:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f747aa00640 1 -- 172.21.15.60:0/493669858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c003bd0 con 0x7f7484161460 2024-09-16T15:44:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.934+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74840cbb20 con 0x7f7484161460 2024-09-16T15:44:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.935+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74840b42f0 con 0x7f7484161460 2024-09-16T15:44:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.937+0000 7f747aa00640 1 -- 172.21.15.60:0/493669858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f747c01d020 con 0x7f7484161460 2024-09-16T15:44:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.938+0000 7f747aa00640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7460078be0 0x7f746007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.938+0000 7f747aa00640 1 -- 172.21.15.60:0/493669858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f747c09b550 con 0x7f7484161460 2024-09-16T15:44:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.938+0000 7f7481e00640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7460078be0 0x7f746007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.939+0000 7f7481e00640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7460078be0 0x7f746007b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7474001720 tx=0x7f7474009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:20.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:20.942+0000 7f747aa00640 1 -- 172.21.15.60:0/493669858 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f747c062fd0 con 0x7f7484161460 2024-09-16T15:44:21.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.176+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f7484069d80 con 0x7f7460078be0 2024-09-16T15:44:21.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.205+0000 7f747aa00640 1 -- 172.21.15.60:0/493669858 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7f74740013c0 con 0x7f7460078be0 2024-09-16T15:44:21.207 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:21.207 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:21.207 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:21.207 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "0.003%", 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "213/8.0M", 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "3/9" 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:21.208 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:21.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7460078be0 msgr2=0x7f746007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:21.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7460078be0 0x7f746007b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7474001720 tx=0x7f7474009040 comp rx=0 tx=0).stop 2024-09-16T15:44:21.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484161460 msgr2=0x7f74840cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:21.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484161460 0x7f74840cd260 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f747c00ef60 tx=0x7f747c00ef90 comp rx=0 tx=0).stop 2024-09-16T15:44:21.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 shutdown_connections 2024-09-16T15:44:21.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7460078be0 0x7f746007b0a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:21.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7484161460 0x7f74840cd260 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:21.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74840d2a00 0x7f7484156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:21.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 --2- 172.21.15.60:0/493669858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74840cf830 0x7f74840a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:21.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 >> 172.21.15.60:0/493669858 conn(0x7f748409d0b0 msgr2=0x7f748406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:21.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 shutdown_connections 2024-09-16T15:44:21.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:21.208+0000 7f7483200640 1 -- 172.21.15.60:0/493669858 wait complete. 2024-09-16T15:44:21.225 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:22.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:22.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.402+0000 7f3775800640 1 -- 172.21.15.60:0/1013809078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3770145750 msgr2=0x7f3770073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:22.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.402+0000 7f3775800640 1 --2- 172.21.15.60:0/1013809078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3770145750 0x7f3770073440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3760009860 tx=0x7f376002a510 comp rx=0 tx=0).stop 2024-09-16T15:44:22.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.402+0000 7f3775800640 1 -- 172.21.15.60:0/1013809078 shutdown_connections 2024-09-16T15:44:22.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.402+0000 7f3775800640 1 --2- 172.21.15.60:0/1013809078 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3770145750 0x7f3770073440 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:22.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.402+0000 7f3775800640 1 -- 172.21.15.60:0/1013809078 >> 172.21.15.60:0/1013809078 conn(0x7f37700b1840 msgr2=0x7f37700b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:22.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.402+0000 7f3775800640 1 -- 172.21.15.60:0/1013809078 shutdown_connections 2024-09-16T15:44:22.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.402+0000 7f3775800640 1 -- 172.21.15.60:0/1013809078 wait complete. 2024-09-16T15:44:22.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.403+0000 7f3775800640 1 Processor -- start 2024-09-16T15:44:22.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.404+0000 7f3775800640 1 -- start start 2024-09-16T15:44:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.404+0000 7f3775800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377006efa0 0x7f3770142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f3775800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37700cccd0 0x7f3770139580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f3775800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770145750 0x7f377015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f3775800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f37700ce6d0 con 0x7f377006efa0 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f3775800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37700ce550 con 0x7f3770145750 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f3775800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f37700ce850 con 0x7f37700cccd0 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377006efa0 0x7f3770142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377006efa0 0x7f3770142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37278/0 (socket says 172.21.15.60:37278) 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376ea00640 1 -- 172.21.15.60:0/811087436 learned_addr learned my addr 172.21.15.60:0/811087436 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f3767e00640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37700cccd0 0x7f3770139580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376f400640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770145750 0x7f377015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376ea00640 1 -- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37700cccd0 msgr2=0x7f3770139580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376ea00640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37700cccd0 0x7f3770139580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376ea00640 1 -- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770145750 msgr2=0x7f377015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376ea00640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770145750 0x7f377015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:22.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f376ea00640 1 -- 172.21.15.60:0/811087436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377015b6c0 con 0x7f377006efa0 2024-09-16T15:44:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.405+0000 7f3767e00640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37700cccd0 0x7f3770139580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.406+0000 7f376ea00640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377006efa0 0x7f3770142b90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f376002ab00 tx=0x7f3760033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.406+0000 7f3766a00640 1 -- 172.21.15.60:0/811087436 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760005c90 con 0x7f377006efa0 2024-09-16T15:44:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.406+0000 7f3766a00640 1 -- 172.21.15.60:0/811087436 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3760031e50 con 0x7f377006efa0 2024-09-16T15:44:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.406+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377015ae20 con 0x7f377006efa0 2024-09-16T15:44:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.406+0000 7f3766a00640 1 -- 172.21.15.60:0/811087436 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760005260 con 0x7f377006efa0 2024-09-16T15:44:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.406+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377015b3e0 con 0x7f377006efa0 2024-09-16T15:44:22.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.407+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37700ced00 con 0x7f377006efa0 2024-09-16T15:44:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.408+0000 7f3766a00640 1 -- 172.21.15.60:0/811087436 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f3760005400 con 0x7f377006efa0 2024-09-16T15:44:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.409+0000 7f3766a00640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f374c078be0 0x7f374c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.409+0000 7f3766a00640 1 -- 172.21.15.60:0/811087436 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f37600bc000 con 0x7f377006efa0 2024-09-16T15:44:22.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.409+0000 7f3767e00640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f374c078be0 0x7f374c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:22.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.410+0000 7f3767e00640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f374c078be0 0x7f374c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f37700b3370 tx=0x7f3758009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:22.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.413+0000 7f3766a00640 1 -- 172.21.15.60:0/811087436 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f376003f030 con 0x7f377006efa0 2024-09-16T15:44:22.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.651+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f3770139ac0 con 0x7f374c078be0 2024-09-16T15:44:22.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.671+0000 7f3766a00640 1 -- 172.21.15.60:0/811087436 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7f3758007d70 con 0x7f374c078be0 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "0.003%", 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "213/8.0M", 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "3/9" 2024-09-16T15:44:22.674 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:22.675 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:22.675 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:22.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f374c078be0 msgr2=0x7f374c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:22.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f374c078be0 0x7f374c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f37700b3370 tx=0x7f3758009040 comp rx=0 tx=0).stop 2024-09-16T15:44:22.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377006efa0 msgr2=0x7f3770142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:22.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377006efa0 0x7f3770142b90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f376002ab00 tx=0x7f3760033c20 comp rx=0 tx=0).stop 2024-09-16T15:44:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 shutdown_connections 2024-09-16T15:44:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f374c078be0 0x7f374c07b0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3770145750 0x7f377015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37700cccd0 0x7f3770139580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 --2- 172.21.15.60:0/811087436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f377006efa0 0x7f3770142b90 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 >> 172.21.15.60:0/811087436 conn(0x7f37700b1840 msgr2=0x7f37700cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 shutdown_connections 2024-09-16T15:44:22.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:22.674+0000 7f3775800640 1 -- 172.21.15.60:0/811087436 wait complete. 2024-09-16T15:44:22.692 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:23.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.868+0000 7fd7d1800640 1 -- 172.21.15.60:0/3609967649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc06efa0 msgr2=0x7fd7cc0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.868+0000 7fd7d1800640 1 --2- 172.21.15.60:0/3609967649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc06efa0 0x7fd7cc0cc5c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4005fd0 tx=0x7fd7b402a540 comp rx=0 tx=0).stop 2024-09-16T15:44:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.868+0000 7fd7d1800640 1 -- 172.21.15.60:0/3609967649 shutdown_connections 2024-09-16T15:44:23.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.868+0000 7fd7d1800640 1 --2- 172.21.15.60:0/3609967649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc06efa0 0x7fd7cc0cc5c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.868+0000 7fd7d1800640 1 --2- 172.21.15.60:0/3609967649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7cc145750 0x7fd7cc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.868+0000 7fd7d1800640 1 -- 172.21.15.60:0/3609967649 >> 172.21.15.60:0/3609967649 conn(0x7fd7cc0b1840 msgr2=0x7fd7cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.868+0000 7fd7d1800640 1 -- 172.21.15.60:0/3609967649 shutdown_connections 2024-09-16T15:44:23.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.868+0000 7fd7d1800640 1 -- 172.21.15.60:0/3609967649 wait complete. 2024-09-16T15:44:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.870+0000 7fd7d1800640 1 Processor -- start 2024-09-16T15:44:23.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.870+0000 7fd7d1800640 1 -- start start 2024-09-16T15:44:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7d1800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7cc06efa0 0x7fd7cc0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7d1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7cc0ccb00 0x7fd7cc0cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7d1800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc145750 0x7fd7cc1f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7d1800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc0b2650 con 0x7fd7cc145750 2024-09-16T15:44:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7d1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc0cedd0 con 0x7fd7cc0ccb00 2024-09-16T15:44:23.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7d1800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd7cc160100 con 0x7fd7cc06efa0 2024-09-16T15:44:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7cb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc145750 0x7fd7cc1f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7cb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc145750 0x7fd7cc1f2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37296/0 (socket says 172.21.15.60:37296) 2024-09-16T15:44:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7cb400640 1 -- 172.21.15.60:0/2056896590 learned_addr learned my addr 172.21.15.60:0/2056896590 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7caa00640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7cc06efa0 0x7fd7cc0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.871+0000 7fd7c2000640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7cc0ccb00 0x7fd7cc0cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7cb400640 1 -- 172.21.15.60:0/2056896590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7cc06efa0 msgr2=0x7fd7cc0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7cb400640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7cc06efa0 0x7fd7cc0cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7cb400640 1 -- 172.21.15.60:0/2056896590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7cc0ccb00 msgr2=0x7fd7cc0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7cb400640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7cc0ccb00 0x7fd7cc0cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7cb400640 1 -- 172.21.15.60:0/2056896590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc1365f0 con 0x7fd7cc145750 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7cb400640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc145750 0x7fd7cc1f2b30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c7a0 tx=0x7fd7c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7c3400640 1 -- 172.21.15.60:0/2056896590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4003b70 con 0x7fd7cc145750 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7c3400640 1 -- 172.21.15.60:0/2056896590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7c4003d30 con 0x7fd7cc145750 2024-09-16T15:44:23.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc1368d0 con 0x7fd7cc145750 2024-09-16T15:44:23.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7c3400640 1 -- 172.21.15.60:0/2056896590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400aab0 con 0x7fd7cc145750 2024-09-16T15:44:23.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.872+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc136e90 con 0x7fd7cc145750 2024-09-16T15:44:23.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.873+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7cc1605b0 con 0x7fd7cc145750 2024-09-16T15:44:23.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.875+0000 7fd7c3400640 1 -- 172.21.15.60:0/2056896590 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fd7c400ac50 con 0x7fd7cc145750 2024-09-16T15:44:23.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.876+0000 7fd7c3400640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd7a8078bb0 0x7fd7a807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:23.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.876+0000 7fd7c3400640 1 -- 172.21.15.60:0/2056896590 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fd7c409adf0 con 0x7fd7cc145750 2024-09-16T15:44:23.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.876+0000 7fd7caa00640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd7a8078bb0 0x7fd7a807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:23.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.877+0000 7fd7caa00640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd7a8078bb0 0x7fd7a807b070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00bb50 tx=0x7fd7bc00b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:23.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:23.881+0000 7fd7c3400640 1 -- 172.21.15.60:0/2056896590 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd7c4062770 con 0x7fd7cc145750 2024-09-16T15:44:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.114+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7fd7cc0c4ab0 con 0x7fd7a8078bb0 2024-09-16T15:44:24.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.137+0000 7fd7c3400640 1 -- 172.21.15.60:0/2056896590 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7fd7cc0c4ab0 con 0x7fd7a8078bb0 2024-09-16T15:44:24.139 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:24.139 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:24.139 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "0.003%", 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "213/8.0M", 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "3/9" 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:24.140 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd7a8078bb0 msgr2=0x7fd7a807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd7a8078bb0 0x7fd7a807b070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00bb50 tx=0x7fd7bc00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc145750 msgr2=0x7fd7cc1f2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc145750 0x7fd7cc1f2b30 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c7a0 tx=0x7fd7c400cc60 comp rx=0 tx=0).stop 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 shutdown_connections 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd7a8078bb0 0x7fd7a807b070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7cc145750 0x7fd7cc1f2b30 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7cc0ccb00 0x7fd7cc0cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 --2- 172.21.15.60:0/2056896590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7cc06efa0 0x7fd7cc0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.139+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 >> 172.21.15.60:0/2056896590 conn(0x7fd7cc0b1840 msgr2=0x7fd7cc0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:24.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.140+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 shutdown_connections 2024-09-16T15:44:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:24.140+0000 7fd7d1800640 1 -- 172.21.15.60:0/2056896590 wait complete. 2024-09-16T15:44:24.157 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:25.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:25.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.333+0000 7fde43200640 1 -- 172.21.15.60:0/2398120614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde20004d40 msgr2=0x7fde200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.333+0000 7fde43200640 1 --2- 172.21.15.60:0/2398120614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde20004d40 0x7fde200051c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fde34009ed0 tx=0x7fde20005830 comp rx=0 tx=0).stop 2024-09-16T15:44:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.333+0000 7fde41200640 1 -- 172.21.15.60:0/2398120614 reap_dead start 2024-09-16T15:44:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.333+0000 7fde43200640 1 -- 172.21.15.60:0/2398120614 shutdown_connections 2024-09-16T15:44:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.333+0000 7fde43200640 1 -- 172.21.15.60:0/2398120614 >> 172.21.15.60:0/2398120614 conn(0x7fde3c0b1840 msgr2=0x7fde3c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.333+0000 7fde43200640 1 -- 172.21.15.60:0/2398120614 shutdown_connections 2024-09-16T15:44:25.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.333+0000 7fde43200640 1 -- 172.21.15.60:0/2398120614 wait complete. 2024-09-16T15:44:25.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.335+0000 7fde43200640 1 Processor -- start 2024-09-16T15:44:25.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.335+0000 7fde43200640 1 -- start start 2024-09-16T15:44:25.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde43200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde20004d40 0x7fde3c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:25.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde43200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde3c1585d0 0x7fde3c0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde43200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde3c158a50 0x7fde3c0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde43200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c0737d0 con 0x7fde3c158a50 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde43200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c073650 con 0x7fde3c1585d0 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde43200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fde3c0ce0f0 con 0x7fde20004d40 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde41200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde3c158a50 0x7fde3c0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde41200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde3c158a50 0x7fde3c0c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37328/0 (socket says 172.21.15.60:37328) 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde41200640 1 -- 172.21.15.60:0/2012632293 learned_addr learned my addr 172.21.15.60:0/2012632293 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde3be00640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde20004d40 0x7fde3c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde3b400640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde3c1585d0 0x7fde3c0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde41200640 1 -- 172.21.15.60:0/2012632293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde20004d40 msgr2=0x7fde3c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde41200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde20004d40 0x7fde3c0c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde41200640 1 -- 172.21.15.60:0/2012632293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde3c1585d0 msgr2=0x7fde3c0c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde41200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde3c1585d0 0x7fde3c0c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.336+0000 7fde41200640 1 -- 172.21.15.60:0/2012632293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c141c00 con 0x7fde3c158a50 2024-09-16T15:44:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.337+0000 7fde41200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde3c158a50 0x7fde3c0c68f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fde34009ed0 tx=0x7fde34033de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:25.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.337+0000 7fde33e00640 1 -- 172.21.15.60:0/2012632293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34042070 con 0x7fde3c158a50 2024-09-16T15:44:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.337+0000 7fde33e00640 1 -- 172.21.15.60:0/2012632293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde34003c60 con 0x7fde3c158a50 2024-09-16T15:44:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.337+0000 7fde33e00640 1 -- 172.21.15.60:0/2012632293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34038040 con 0x7fde3c158a50 2024-09-16T15:44:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.337+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c141920 con 0x7fde3c158a50 2024-09-16T15:44:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.337+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c0c4e30 con 0x7fde3c158a50 2024-09-16T15:44:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.338+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3c0cdbb0 con 0x7fde3c158a50 2024-09-16T15:44:25.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.339+0000 7fde33e00640 1 -- 172.21.15.60:0/2012632293 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fde34004290 con 0x7fde3c158a50 2024-09-16T15:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.340+0000 7fde33e00640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fde1c078bb0 0x7fde1c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.340+0000 7fde33e00640 1 -- 172.21.15.60:0/2012632293 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fde340bb350 con 0x7fde3c158a50 2024-09-16T15:44:25.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.341+0000 7fde3be00640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fde1c078bb0 0x7fde1c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:25.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.341+0000 7fde3be00640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fde1c078bb0 0x7fde1c07b070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009aa0 tx=0x7fde2c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:25.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.345+0000 7fde33e00640 1 -- 172.21.15.60:0/2012632293 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fde34082d40 con 0x7fde3c158a50 2024-09-16T15:44:25.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.582+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7fde3c0ae610 con 0x7fde1c078bb0 2024-09-16T15:44:25.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.601+0000 7fde33e00640 1 -- 172.21.15.60:0/2012632293 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7fde3c0ae610 con 0x7fde1c078bb0 2024-09-16T15:44:25.603 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "25.003%", 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "2.0M/8.0M", 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "6/9" 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:25.604 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:25.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fde1c078bb0 msgr2=0x7fde1c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fde1c078bb0 0x7fde1c07b070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009aa0 tx=0x7fde2c005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:25.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde3c158a50 msgr2=0x7fde3c0c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde3c158a50 0x7fde3c0c68f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fde34009ed0 tx=0x7fde34033de0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 shutdown_connections 2024-09-16T15:44:25.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fde1c078bb0 0x7fde1c07b070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde3c158a50 0x7fde3c0c68f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde3c1585d0 0x7fde3c0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 --2- 172.21.15.60:0/2012632293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde20004d40 0x7fde3c0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 >> 172.21.15.60:0/2012632293 conn(0x7fde3c0b1840 msgr2=0x7fde3c06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:25.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 shutdown_connections 2024-09-16T15:44:25.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:25.604+0000 7fde43200640 1 -- 172.21.15.60:0/2012632293 wait complete. 2024-09-16T15:44:25.621 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:26.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.794+0000 7fa43fc00640 1 -- 172.21.15.60:0/1657914208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 msgr2=0x7fa438165480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.794+0000 7fa43fc00640 1 --2- 172.21.15.60:0/1657914208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 0x7fa438165480 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa430005fd0 tx=0x7fa43002bc70 comp rx=0 tx=0).stop 2024-09-16T15:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.794+0000 7fa43fc00640 1 -- 172.21.15.60:0/1657914208 shutdown_connections 2024-09-16T15:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.794+0000 7fa43fc00640 1 --2- 172.21.15.60:0/1657914208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 0x7fa438165480 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.794+0000 7fa43fc00640 1 -- 172.21.15.60:0/1657914208 >> 172.21.15.60:0/1657914208 conn(0x7fa4380b1840 msgr2=0x7fa4380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.794+0000 7fa43fc00640 1 -- 172.21.15.60:0/1657914208 shutdown_connections 2024-09-16T15:44:26.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.794+0000 7fa43fc00640 1 -- 172.21.15.60:0/1657914208 wait complete. 2024-09-16T15:44:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.796+0000 7fa43fc00640 1 Processor -- start 2024-09-16T15:44:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.796+0000 7fa43fc00640 1 -- start start 2024-09-16T15:44:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa43fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4380cd490 0x7fa43815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa43fc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 0x7fa4381f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa43fc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4381517a0 0x7fa438142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa43fc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa4380cc9e0 con 0x7fa4380cdad0 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa43fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa4380cc860 con 0x7fa4380cd490 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa43fc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa4380ccb60 con 0x7fa4381517a0 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa42fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 0x7fa4381f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa42fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 0x7fa4381f8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37352/0 (socket says 172.21.15.60:37352) 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa42fe00640 1 -- 172.21.15.60:0/3761339338 learned_addr learned my addr 172.21.15.60:0/3761339338 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa43d200640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4380cd490 0x7fa43815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.797+0000 7fa43dc00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4381517a0 0x7fa438142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa42fe00640 1 -- 172.21.15.60:0/3761339338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4381517a0 msgr2=0x7fa438142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa42fe00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4381517a0 0x7fa438142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa42fe00640 1 -- 172.21.15.60:0/3761339338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4380cd490 msgr2=0x7fa43815c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa42fe00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4380cd490 0x7fa43815c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa42fe00640 1 -- 172.21.15.60:0/3761339338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43815cb60 con 0x7fa4380cdad0 2024-09-16T15:44:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa42fe00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 0x7fa4381f8460 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c500 tx=0x7fa42000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa435600640 1 -- 172.21.15.60:0/3761339338 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420016020 con 0x7fa4380cdad0 2024-09-16T15:44:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa435600640 1 -- 172.21.15.60:0/3761339338 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa420012920 con 0x7fa4380cdad0 2024-09-16T15:44:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43815ce40 con 0x7fa4380cdad0 2024-09-16T15:44:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa435600640 1 -- 172.21.15.60:0/3761339338 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420012d50 con 0x7fa4380cdad0 2024-09-16T15:44:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.798+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43815d400 con 0x7fa4380cdad0 2024-09-16T15:44:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.800+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4380cd190 con 0x7fa4380cdad0 2024-09-16T15:44:26.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.801+0000 7fa435600640 1 -- 172.21.15.60:0/3761339338 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fa420019020 con 0x7fa4380cdad0 2024-09-16T15:44:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.802+0000 7fa435600640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa414078bb0 0x7fa41407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.802+0000 7fa435600640 1 -- 172.21.15.60:0/3761339338 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fa42009aea0 con 0x7fa4380cdad0 2024-09-16T15:44:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.802+0000 7fa43d200640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa414078bb0 0x7fa41407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.803+0000 7fa43d200640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa414078bb0 0x7fa41407b070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa428001720 tx=0x7fa428009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:26.806+0000 7fa435600640 1 -- 172.21.15.60:0/3761339338 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa420020020 con 0x7fa4380cdad0 2024-09-16T15:44:27.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.040+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7fa4380b4860 con 0x7fa414078bb0 2024-09-16T15:44:27.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.060+0000 7fa435600640 1 -- 172.21.15.60:0/3761339338 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7fa4380b4860 con 0x7fa414078bb0 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "25.003%", 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "2.0M/8.0M", 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "6/9" 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:27.062 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:27.063 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa414078bb0 msgr2=0x7fa41407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa414078bb0 0x7fa41407b070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa428001720 tx=0x7fa428009040 comp rx=0 tx=0).stop 2024-09-16T15:44:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 msgr2=0x7fa4381f8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 0x7fa4381f8460 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c500 tx=0x7fa42000c9c0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 shutdown_connections 2024-09-16T15:44:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa414078bb0 0x7fa41407b070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4381517a0 0x7fa438142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4380cdad0 0x7fa4381f8460 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 --2- 172.21.15.60:0/3761339338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4380cd490 0x7fa43815c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 >> 172.21.15.60:0/3761339338 conn(0x7fa4380b1840 msgr2=0x7fa4380cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:27.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 shutdown_connections 2024-09-16T15:44:27.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:27.062+0000 7fa43fc00640 1 -- 172.21.15.60:0/3761339338 wait complete. 2024-09-16T15:44:27.080 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:28.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:28.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.254+0000 7f4b8a200640 1 -- 172.21.15.60:0/2576148731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b84145750 msgr2=0x7f4b84073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.254+0000 7f4b8a200640 1 --2- 172.21.15.60:0/2576148731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b84145750 0x7f4b84073440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009830 tx=0x7f4b7402a490 comp rx=0 tx=0).stop 2024-09-16T15:44:28.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.255+0000 7f4b8a200640 1 -- 172.21.15.60:0/2576148731 shutdown_connections 2024-09-16T15:44:28.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.255+0000 7f4b8a200640 1 --2- 172.21.15.60:0/2576148731 >> v2:172.21.15.100:3300/0 conn(0x7f4b840cccd0 0x7f4b840cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.255+0000 7f4b8a200640 1 --2- 172.21.15.60:0/2576148731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b84145750 0x7f4b84073440 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.255+0000 7f4b8a200640 1 -- 172.21.15.60:0/2576148731 >> 172.21.15.60:0/2576148731 conn(0x7f4b840b1840 msgr2=0x7f4b840b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:28.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.255+0000 7f4b8a200640 1 -- 172.21.15.60:0/2576148731 shutdown_connections 2024-09-16T15:44:28.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.255+0000 7f4b8a200640 1 -- 172.21.15.60:0/2576148731 wait complete. 2024-09-16T15:44:28.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.257+0000 7f4b8a200640 1 Processor -- start 2024-09-16T15:44:28.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.257+0000 7f4b8a200640 1 -- start start 2024-09-16T15:44:28.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b8a200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b8406efa0 0x7f4b841f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b8a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b840cccd0 0x7f4b841f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b8a200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b84145750 0x7f4b841f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b8a200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4b840ce480 con 0x7f4b8406efa0 2024-09-16T15:44:28.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b8a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b840ce300 con 0x7f4b840cccd0 2024-09-16T15:44:28.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b8a200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4b840ce600 con 0x7f4b84145750 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b83400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b8406efa0 0x7f4b841f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b83400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b8406efa0 0x7f4b841f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37362/0 (socket says 172.21.15.60:37362) 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b83400640 1 -- 172.21.15.60:0/3956345883 learned_addr learned my addr 172.21.15.60:0/3956345883 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b83e00640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b84145750 0x7f4b841f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b7aa00640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b840cccd0 0x7f4b841f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b83400640 1 -- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b84145750 msgr2=0x7f4b841f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b83400640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b84145750 0x7f4b841f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b83400640 1 -- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b840cccd0 msgr2=0x7f4b841f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.258+0000 7f4b83400640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b840cccd0 0x7f4b841f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.259+0000 7f4b83400640 1 -- 172.21.15.60:0/3956345883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b841f4670 con 0x7f4b8406efa0 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.259+0000 7f4b7aa00640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b840cccd0 0x7f4b841f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.259+0000 7f4b83400640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b8406efa0 0x7f4b841f1a60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4b7402aa80 tx=0x7f4b74033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.259+0000 7f4b7be00640 1 -- 172.21.15.60:0/3956345883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74005c90 con 0x7f4b8406efa0 2024-09-16T15:44:28.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.259+0000 7f4b7be00640 1 -- 172.21.15.60:0/3956345883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b74031e50 con 0x7f4b8406efa0 2024-09-16T15:44:28.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.259+0000 7f4b7be00640 1 -- 172.21.15.60:0/3956345883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74005260 con 0x7f4b8406efa0 2024-09-16T15:44:28.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.259+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b841f84c0 con 0x7f4b8406efa0 2024-09-16T15:44:28.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.259+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b841f5390 con 0x7f4b8406efa0 2024-09-16T15:44:28.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.260+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b840cdbb0 con 0x7f4b8406efa0 2024-09-16T15:44:28.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.261+0000 7f4b7be00640 1 -- 172.21.15.60:0/3956345883 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f4b74005400 con 0x7f4b8406efa0 2024-09-16T15:44:28.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.262+0000 7f4b7be00640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4b60078bb0 0x7f4b6007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.263+0000 7f4b7be00640 1 -- 172.21.15.60:0/3956345883 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f4b740bbfb0 con 0x7f4b8406efa0 2024-09-16T15:44:28.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.263+0000 7f4b7aa00640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4b60078bb0 0x7f4b6007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.263+0000 7f4b7aa00640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4b60078bb0 0x7f4b6007b070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c007a50 tx=0x7f4b6c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:28.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.267+0000 7f4b7be00640 1 -- 172.21.15.60:0/3956345883 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4b7403f030 con 0x7f4b8406efa0 2024-09-16T15:44:28.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.500+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f4b841f4430 con 0x7f4b60078bb0 2024-09-16T15:44:28.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.525+0000 7f4b7be00640 1 -- 172.21.15.60:0/3956345883 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f4b841f4430 con 0x7f4b60078bb0 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "25.003%", 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "2.0M/8.0M", 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "6/9" 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:28.528 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:28.529 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4b60078bb0 msgr2=0x7f4b6007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4b60078bb0 0x7f4b6007b070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c007a50 tx=0x7f4b6c009040 comp rx=0 tx=0).stop 2024-09-16T15:44:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b8406efa0 msgr2=0x7f4b841f1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b8406efa0 0x7f4b841f1a60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4b7402aa80 tx=0x7f4b74033c20 comp rx=0 tx=0).stop 2024-09-16T15:44:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 shutdown_connections 2024-09-16T15:44:28.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4b60078bb0 0x7f4b6007b070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b84145750 0x7f4b841f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b840cccd0 0x7f4b841f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 --2- 172.21.15.60:0/3956345883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b8406efa0 0x7f4b841f1a60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 >> 172.21.15.60:0/3956345883 conn(0x7f4b840b1840 msgr2=0x7f4b8406f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 shutdown_connections 2024-09-16T15:44:28.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:28.528+0000 7f4b8a200640 1 -- 172.21.15.60:0/3956345883 wait complete. 2024-09-16T15:44:28.546 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:29.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.722+0000 7f4110400640 1 -- 172.21.15.60:0/445224027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 msgr2=0x7f4108073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.722+0000 7f4110400640 1 --2- 172.21.15.60:0/445224027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 0x7f4108073440 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f40fc005fd0 tx=0x7f40fc02a540 comp rx=0 tx=0).stop 2024-09-16T15:44:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.722+0000 7f4110400640 1 -- 172.21.15.60:0/445224027 shutdown_connections 2024-09-16T15:44:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.722+0000 7f4110400640 1 --2- 172.21.15.60:0/445224027 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f410806efa0 0x7f41080cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.722+0000 7f4110400640 1 --2- 172.21.15.60:0/445224027 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 0x7f4108073440 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.722+0000 7f4110400640 1 -- 172.21.15.60:0/445224027 >> 172.21.15.60:0/445224027 conn(0x7f41080b1840 msgr2=0x7f41080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.722+0000 7f4110400640 1 -- 172.21.15.60:0/445224027 shutdown_connections 2024-09-16T15:44:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.722+0000 7f4110400640 1 -- 172.21.15.60:0/445224027 wait complete. 2024-09-16T15:44:29.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.724+0000 7f4110400640 1 Processor -- start 2024-09-16T15:44:29.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.724+0000 7f4110400640 1 -- start start 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f4110400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f410806efa0 0x7f41081ea270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f4110400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41080ccb00 0x7f41081ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f4110400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 0x7f4108130620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f4110400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f41080b26b0 con 0x7f4108145750 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f4110400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41080cedd0 con 0x7f41080ccb00 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f4110400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4108160100 con 0x7f410806efa0 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f410e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 0x7f4108130620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f410e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 0x7f4108130620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49498/0 (socket says 172.21.15.60:49498) 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f410e400640 1 -- 172.21.15.60:0/1700443412 learned_addr learned my addr 172.21.15.60:0/1700443412 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.725+0000 7f410da00640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f410806efa0 0x7f41081ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f410d000640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41080ccb00 0x7f41081ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f410e400640 1 -- 172.21.15.60:0/1700443412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f410806efa0 msgr2=0x7f41081ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f410e400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f410806efa0 0x7f41081ea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f410e400640 1 -- 172.21.15.60:0/1700443412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41080ccb00 msgr2=0x7f41081ea7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f410e400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41080ccb00 0x7f41081ea7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f410e400640 1 -- 172.21.15.60:0/1700443412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41081e6f20 con 0x7f4108145750 2024-09-16T15:44:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f410e400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 0x7f4108130620 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f410000c7a0 tx=0x7f410000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f4105600640 1 -- 172.21.15.60:0/1700443412 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100003b70 con 0x7f4108145750 2024-09-16T15:44:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f4105600640 1 -- 172.21.15.60:0/1700443412 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4100003d10 con 0x7f4108145750 2024-09-16T15:44:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f4105600640 1 -- 172.21.15.60:0/1700443412 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410000aab0 con 0x7f4108145750 2024-09-16T15:44:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.726+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410812e630 con 0x7f4108145750 2024-09-16T15:44:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.727+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f410812eed0 con 0x7f4108145750 2024-09-16T15:44:29.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.728+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41081605b0 con 0x7f4108145750 2024-09-16T15:44:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.729+0000 7f4105600640 1 -- 172.21.15.60:0/1700443412 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f410000ac50 con 0x7f4108145750 2024-09-16T15:44:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.730+0000 7f4105600640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40e8078be0 0x7f40e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.730+0000 7f4105600640 1 -- 172.21.15.60:0/1700443412 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f410009ae10 con 0x7f4108145750 2024-09-16T15:44:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.730+0000 7f410da00640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40e8078be0 0x7f40e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.731+0000 7f410da00640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40e8078be0 0x7f40e807b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f40fc033000 tx=0x7f40fc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.734+0000 7f4105600640 1 -- 172.21.15.60:0/1700443412 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f410009e030 con 0x7f4108145750 2024-09-16T15:44:29.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.966+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f41080cf980 con 0x7f40e8078be0 2024-09-16T15:44:29.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.986+0000 7f4105600640 1 -- 172.21.15.60:0/1700443412 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f41080cf980 con 0x7f40e8078be0 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "50.003%", 2024-09-16T15:44:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "4.0M/8.0M", 2024-09-16T15:44:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "7/9" 2024-09-16T15:44:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:29.989 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:29.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40e8078be0 msgr2=0x7f40e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40e8078be0 0x7f40e807b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f40fc033000 tx=0x7f40fc004730 comp rx=0 tx=0).stop 2024-09-16T15:44:29.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 msgr2=0x7f4108130620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 0x7f4108130620 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f410000c7a0 tx=0x7f410000cc60 comp rx=0 tx=0).stop 2024-09-16T15:44:29.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 shutdown_connections 2024-09-16T15:44:29.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40e8078be0 0x7f40e807b0a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4108145750 0x7f4108130620 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41080ccb00 0x7f41081ea7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 --2- 172.21.15.60:0/1700443412 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f410806efa0 0x7f41081ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 >> 172.21.15.60:0/1700443412 conn(0x7f41080b1840 msgr2=0x7f41080cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:29.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 shutdown_connections 2024-09-16T15:44:29.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:29.988+0000 7f4110400640 1 -- 172.21.15.60:0/1700443412 wait complete. 2024-09-16T15:44:30.006 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:31.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:31.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:44:31.181 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:44:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.180+0000 7f9a67a00640 1 -- 172.21.15.60:0/64949198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a60145750 msgr2=0x7f9a60073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.180+0000 7f9a67a00640 1 --2- 172.21.15.60:0/64949198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a60145750 0x7f9a60073440 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009830 tx=0x7f9a5002a480 comp rx=0 tx=0).stop 2024-09-16T15:44:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.180+0000 7f9a67a00640 1 -- 172.21.15.60:0/64949198 shutdown_connections 2024-09-16T15:44:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.180+0000 7f9a67a00640 1 --2- 172.21.15.60:0/64949198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a6006efa0 0x7f9a600cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.180+0000 7f9a67a00640 1 --2- 172.21.15.60:0/64949198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a60145750 0x7f9a60073440 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.180+0000 7f9a67a00640 1 -- 172.21.15.60:0/64949198 >> 172.21.15.60:0/64949198 conn(0x7f9a600b1840 msgr2=0x7f9a600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.180+0000 7f9a67a00640 1 -- 172.21.15.60:0/64949198 shutdown_connections 2024-09-16T15:44:31.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.180+0000 7f9a67a00640 1 -- 172.21.15.60:0/64949198 wait complete. 2024-09-16T15:44:31.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.181+0000 7f9a67a00640 1 Processor -- start 2024-09-16T15:44:31.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.182+0000 7f9a67a00640 1 -- start start 2024-09-16T15:44:31.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a67a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a6006efa0 0x7f9a60140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a67a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a600ccb00 0x7f9a600ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a67a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a60145750 0x7f9a600addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a67a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600b26b0 con 0x7f9a6006efa0 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a67a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600cedd0 con 0x7f9a60145750 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a67a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9a60160100 con 0x7f9a600ccb00 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a6006efa0 0x7f9a60140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a6006efa0 0x7f9a60140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49506/0 (socket says 172.21.15.60:49506) 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65000640 1 -- 172.21.15.60:0/1987482566 learned_addr learned my addr 172.21.15.60:0/1987482566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a5fe00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a600ccb00 0x7f9a600ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65a00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a60145750 0x7f9a600addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:31.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65000640 1 -- 172.21.15.60:0/1987482566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a600ccb00 msgr2=0x7f9a600ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65000640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a600ccb00 0x7f9a600ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65000640 1 -- 172.21.15.60:0/1987482566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a60145750 msgr2=0x7f9a600addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65000640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a60145750 0x7f9a600addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.183+0000 7f9a65000640 1 -- 172.21.15.60:0/1987482566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a60153470 con 0x7f9a6006efa0 2024-09-16T15:44:31.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.184+0000 7f9a65000640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a6006efa0 0x7f9a60140500 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009950 tx=0x7f9a50033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:31.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.184+0000 7f9a5cc00640 1 -- 172.21.15.60:0/1987482566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50031d60 con 0x7f9a6006efa0 2024-09-16T15:44:31.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.184+0000 7f9a5cc00640 1 -- 172.21.15.60:0/1987482566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a50005020 con 0x7f9a6006efa0 2024-09-16T15:44:31.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.184+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a60153710 con 0x7f9a6006efa0 2024-09-16T15:44:31.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.184+0000 7f9a5cc00640 1 -- 172.21.15.60:0/1987482566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50005470 con 0x7f9a6006efa0 2024-09-16T15:44:31.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.184+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a60153cd0 con 0x7f9a6006efa0 2024-09-16T15:44:31.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.185+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a601605b0 con 0x7f9a6006efa0 2024-09-16T15:44:31.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.187+0000 7f9a5cc00640 1 -- 172.21.15.60:0/1987482566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f9a500047a0 con 0x7f9a6006efa0 2024-09-16T15:44:31.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.188+0000 7f9a5cc00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9a40078bb0 0x7f9a4007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:31.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.188+0000 7f9a5cc00640 1 -- 172.21.15.60:0/1987482566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f9a500bc040 con 0x7f9a6006efa0 2024-09-16T15:44:31.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.188+0000 7f9a5fe00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9a40078bb0 0x7f9a4007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:31.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.189+0000 7f9a5fe00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9a40078bb0 0x7f9a4007b070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9a600b07f0 tx=0x7f9a4c00a2d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:31.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.192+0000 7f9a5cc00640 1 -- 172.21.15.60:0/1987482566 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9a5003f030 con 0x7f9a6006efa0 2024-09-16T15:44:31.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.428+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f9a600735e0 con 0x7f9a40078bb0 2024-09-16T15:44:31.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.448+0000 7f9a5cc00640 1 -- 172.21.15.60:0/1987482566 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f9a600735e0 con 0x7f9a40078bb0 2024-09-16T15:44:31.450 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "50.003%", 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "4.0M/8.0M", 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "7/9" 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:31.452 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9a40078bb0 msgr2=0x7f9a4007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9a40078bb0 0x7f9a4007b070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9a600b07f0 tx=0x7f9a4c00a2d0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a6006efa0 msgr2=0x7f9a60140500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a6006efa0 0x7f9a60140500 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009950 tx=0x7f9a50033ce0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 shutdown_connections 2024-09-16T15:44:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9a40078bb0 0x7f9a4007b070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9a60145750 0x7f9a600addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9a600ccb00 0x7f9a600ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 --2- 172.21.15.60:0/1987482566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9a6006efa0 0x7f9a60140500 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 >> 172.21.15.60:0/1987482566 conn(0x7f9a600b1840 msgr2=0x7f9a600b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 shutdown_connections 2024-09-16T15:44:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:31.451+0000 7f9a67a00640 1 -- 172.21.15.60:0/1987482566 wait complete. 2024-09-16T15:44:31.469 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:32.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:32.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.645+0000 7f965b000640 1 -- 172.21.15.60:0/3255813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96540d2a00 msgr2=0x7f96540b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.645+0000 7f965b000640 1 --2- 172.21.15.60:0/3255813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96540d2a00 0x7f96540b3ac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f964c005fd0 tx=0x7f964c02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.645+0000 7f965b000640 1 -- 172.21.15.60:0/3255813 shutdown_connections 2024-09-16T15:44:32.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.645+0000 7f965b000640 1 --2- 172.21.15.60:0/3255813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96540d2a00 0x7f96540b3ac0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.645+0000 7f965b000640 1 -- 172.21.15.60:0/3255813 >> 172.21.15.60:0/3255813 conn(0x7f965409d0b0 msgr2=0x7f96540d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:32.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.645+0000 7f965b000640 1 -- 172.21.15.60:0/3255813 shutdown_connections 2024-09-16T15:44:32.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.645+0000 7f965b000640 1 -- 172.21.15.60:0/3255813 wait complete. 2024-09-16T15:44:32.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.647+0000 7f965b000640 1 Processor -- start 2024-09-16T15:44:32.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.647+0000 7f965b000640 1 -- start start 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f965b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96540cf830 0x7f96540b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f965b000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96540d2a00 0x7f96540b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f965b000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9654161460 0x7f96540b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f965b000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f96540b50c0 con 0x7f96540d2a00 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f965b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96540b4f40 con 0x7f96540cf830 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f965b000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f96540b4000 con 0x7f9654161460 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f9659200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96540d2a00 0x7f96540b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f9659200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96540d2a00 0x7f96540b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49514/0 (socket says 172.21.15.60:49514) 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f9659200640 1 -- 172.21.15.60:0/3978954711 learned_addr learned my addr 172.21.15.60:0/3978954711 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f9659c00640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96540cf830 0x7f96540b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.649+0000 7f965a600640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9654161460 0x7f96540b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9659200640 1 -- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9654161460 msgr2=0x7f96540b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9659200640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9654161460 0x7f96540b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9659200640 1 -- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96540cf830 msgr2=0x7f96540b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9659200640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96540cf830 0x7f96540b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9659200640 1 -- 172.21.15.60:0/3978954711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96540b3df0 con 0x7f96540d2a00 2024-09-16T15:44:32.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f965a600640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9654161460 0x7f96540b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9659200640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96540d2a00 0x7f96540b38b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f964000c530 tx=0x7f964000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9651600640 1 -- 172.21.15.60:0/3978954711 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96400052d0 con 0x7f96540d2a00 2024-09-16T15:44:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9651600640 1 -- 172.21.15.60:0/3978954711 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9640005470 con 0x7f96540d2a00 2024-09-16T15:44:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96540c79c0 con 0x7f96540d2a00 2024-09-16T15:44:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.650+0000 7f9651600640 1 -- 172.21.15.60:0/3978954711 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9640012870 con 0x7f96540d2a00 2024-09-16T15:44:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.651+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96540c7f80 con 0x7f96540d2a00 2024-09-16T15:44:32.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.652+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96540a1de0 con 0x7f96540d2a00 2024-09-16T15:44:32.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.653+0000 7f9651600640 1 -- 172.21.15.60:0/3978954711 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f96400035f0 con 0x7f96540d2a00 2024-09-16T15:44:32.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.654+0000 7f9651600640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9634078bb0 0x7f963407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.654+0000 7f9651600640 1 -- 172.21.15.60:0/3978954711 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f964009ab70 con 0x7f96540d2a00 2024-09-16T15:44:32.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.654+0000 7f9659c00640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9634078bb0 0x7f963407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.655+0000 7f9659c00640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9634078bb0 0x7f963407b070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96480046b0 tx=0x7f9648007cc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:32.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.658+0000 7f9651600640 1 -- 172.21.15.60:0/3978954711 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f96540a1de0 con 0x7f96540d2a00 2024-09-16T15:44:32.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.894+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f96540b4590 con 0x7f9634078bb0 2024-09-16T15:44:32.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.919+0000 7f9651600640 1 -- 172.21.15.60:0/3978954711 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f96540b4590 con 0x7f9634078bb0 2024-09-16T15:44:32.921 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:32.921 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:32.921 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:32.921 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:32.921 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:32.921 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "50.003%", 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "4.0M/8.0M", 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "7/9" 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:32.922 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9634078bb0 msgr2=0x7f963407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9634078bb0 0x7f963407b070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96480046b0 tx=0x7f9648007cc0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96540d2a00 msgr2=0x7f96540b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96540d2a00 0x7f96540b38b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f964000c530 tx=0x7f964000c9f0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 shutdown_connections 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9634078bb0 0x7f963407b070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9654161460 0x7f96540b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96540d2a00 0x7f96540b38b0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 --2- 172.21.15.60:0/3978954711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96540cf830 0x7f96540b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 >> 172.21.15.60:0/3978954711 conn(0x7f965409d0b0 msgr2=0x7f96540b61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 shutdown_connections 2024-09-16T15:44:32.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:32.921+0000 7f965b000640 1 -- 172.21.15.60:0/3978954711 wait complete. 2024-09-16T15:44:32.939 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:33.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.113+0000 7f7258800640 1 -- 172.21.15.60:0/2139320085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72500af190 msgr2=0x7f72500b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.113+0000 7f7258800640 1 --2- 172.21.15.60:0/2139320085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72500af190 0x7f72500b2b40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7240009830 tx=0x7f724002a500 comp rx=0 tx=0).stop 2024-09-16T15:44:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.113+0000 7f7258800640 1 -- 172.21.15.60:0/2139320085 shutdown_connections 2024-09-16T15:44:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.113+0000 7f7258800640 1 --2- 172.21.15.60:0/2139320085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72500cec60 0x7f72500cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.113+0000 7f7258800640 1 --2- 172.21.15.60:0/2139320085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72500af190 0x7f72500b2b40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.113+0000 7f7258800640 1 -- 172.21.15.60:0/2139320085 >> 172.21.15.60:0/2139320085 conn(0x7f72500b1840 msgr2=0x7f72500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.113+0000 7f7258800640 1 -- 172.21.15.60:0/2139320085 shutdown_connections 2024-09-16T15:44:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.113+0000 7f7258800640 1 -- 172.21.15.60:0/2139320085 wait complete. 2024-09-16T15:44:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.114+0000 7f7258800640 1 Processor -- start 2024-09-16T15:44:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.114+0000 7f7258800640 1 -- start start 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.115+0000 7f7258800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72500af190 0x7f725015ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.115+0000 7f7258800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72500cec60 0x7f72501f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.115+0000 7f7258800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7250161760 0x7f72501f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.115+0000 7f7258800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7250162310 con 0x7f7250161760 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.115+0000 7f7258800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7250162190 con 0x7f72500af190 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.115+0000 7f7258800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7250162490 con 0x7f72500cec60 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7250161760 0x7f72501f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7250161760 0x7f72501f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49528/0 (socket says 172.21.15.60:49528) 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 -- 172.21.15.60:0/4072587593 learned_addr learned my addr 172.21.15.60:0/4072587593 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7255400640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72500cec60 0x7f72501f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 -- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72500cec60 msgr2=0x7f72501f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72500cec60 0x7f72501f7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7255e00640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72500af190 0x7f725015ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 -- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72500af190 msgr2=0x7f725015ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72500af190 0x7f725015ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 -- 172.21.15.60:0/4072587593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725015ef10 con 0x7f7250161760 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7255400640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72500cec60 0x7f72501f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7255e00640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72500af190 0x7f725015ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.116+0000 7f7256800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7250161760 0x7f72501f8490 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f724c00eaa0 tx=0x7f724c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.117+0000 7f724a000640 1 -- 172.21.15.60:0/4072587593 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c00cdb0 con 0x7f7250161760 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.117+0000 7f724a000640 1 -- 172.21.15.60:0/4072587593 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f724c004c10 con 0x7f7250161760 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.117+0000 7f724a000640 1 -- 172.21.15.60:0/4072587593 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c005020 con 0x7f7250161760 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.117+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725015f1f0 con 0x7f7250161760 2024-09-16T15:44:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.117+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725015f7b0 con 0x7f7250161760 2024-09-16T15:44:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.118+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7250162920 con 0x7f7250161760 2024-09-16T15:44:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.119+0000 7f724a000640 1 -- 172.21.15.60:0/4072587593 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f724c0051c0 con 0x7f7250161760 2024-09-16T15:44:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.120+0000 7f724a000640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7234078bb0 0x7f723407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.120+0000 7f724a000640 1 -- 172.21.15.60:0/4072587593 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f724c09bb10 con 0x7f7250161760 2024-09-16T15:44:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.120+0000 7f7255e00640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7234078bb0 0x7f723407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.121+0000 7f7255e00640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7234078bb0 0x7f723407b070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7250152340 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.125+0000 7f724a000640 1 -- 172.21.15.60:0/4072587593 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f724c016030 con 0x7f7250161760 2024-09-16T15:44:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.361+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f725015d750 con 0x7f7234078bb0 2024-09-16T15:44:34.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.386+0000 7f724a000640 1 -- 172.21.15.60:0/4072587593 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f725015d750 con 0x7f7234078bb0 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:34.388 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "50.003%", 2024-09-16T15:44:34.389 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "4.0M/8.0M", 2024-09-16T15:44:34.389 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "8/9" 2024-09-16T15:44:34.389 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:34.389 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:34.389 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:34.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7234078bb0 msgr2=0x7f723407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7234078bb0 0x7f723407b070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7250152340 tx=0x7f7244009040 comp rx=0 tx=0).stop 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7250161760 msgr2=0x7f72501f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7250161760 0x7f72501f8490 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f724c00eaa0 tx=0x7f724c00ef60 comp rx=0 tx=0).stop 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 shutdown_connections 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7234078bb0 0x7f723407b070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7250161760 0x7f72501f8490 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72500cec60 0x7f72501f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 --2- 172.21.15.60:0/4072587593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72500af190 0x7f725015ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 >> 172.21.15.60:0/4072587593 conn(0x7f72500b1840 msgr2=0x7f7250072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 shutdown_connections 2024-09-16T15:44:34.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:34.388+0000 7f7258800640 1 -- 172.21.15.60:0/4072587593 wait complete. 2024-09-16T15:44:34.406 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:35.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:35.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.581+0000 7f0b39000640 1 -- 172.21.15.60:0/3393413961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b18004800 msgr2=0x7f0b18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.581+0000 7f0b39000640 1 --2- 172.21.15.60:0/3393413961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b18004800 0x7f0b18004c80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0b20006f80 tx=0x7f0b18005830 comp rx=0 tx=0).stop 2024-09-16T15:44:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.581+0000 7f0b39000640 1 -- 172.21.15.60:0/3393413961 shutdown_connections 2024-09-16T15:44:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.581+0000 7f0b39000640 1 --2- 172.21.15.60:0/3393413961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b18004800 0x7f0b18004c80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.581+0000 7f0b39000640 1 -- 172.21.15.60:0/3393413961 >> 172.21.15.60:0/3393413961 conn(0x7f0b3409d0b0 msgr2=0x7f0b340d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.581+0000 7f0b39000640 1 -- 172.21.15.60:0/3393413961 shutdown_connections 2024-09-16T15:44:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.581+0000 7f0b39000640 1 -- 172.21.15.60:0/3393413961 wait complete. 2024-09-16T15:44:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.583+0000 7f0b39000640 1 Processor -- start 2024-09-16T15:44:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.583+0000 7f0b39000640 1 -- start start 2024-09-16T15:44:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b39000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b18004800 0x7f0b3414f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b39000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b340cd110 0x7f0b340cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b39000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b340b8790 0x7f0b340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:35.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b39000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0b340d22f0 con 0x7f0b18004800 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b39000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b340b5210 con 0x7f0b340b8790 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b39000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0b340d2470 con 0x7f0b340cd110 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b33400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b18004800 0x7f0b3414f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b33400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b18004800 0x7f0b3414f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49538/0 (socket says 172.21.15.60:49538) 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b33400640 1 -- 172.21.15.60:0/1785242408 learned_addr learned my addr 172.21.15.60:0/1785242408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b32a00640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b340cd110 0x7f0b340cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.584+0000 7f0b33e00640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b340b8790 0x7f0b340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b33400640 1 -- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b340cd110 msgr2=0x7f0b340cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b33400640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b340cd110 0x7f0b340cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b33400640 1 -- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b340b8790 msgr2=0x7f0b340b8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b33400640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b340b8790 0x7f0b340b8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b33400640 1 -- 172.21.15.60:0/1785242408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b340b6c40 con 0x7f0b18004800 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b33e00640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b340b8790 0x7f0b340b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b33400640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b18004800 0x7f0b3414f110 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0b24011b90 tx=0x7f0b2400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b2b400640 1 -- 172.21.15.60:0/1785242408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b240098d0 con 0x7f0b18004800 2024-09-16T15:44:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b2b400640 1 -- 172.21.15.60:0/1785242408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b240035f0 con 0x7f0b18004800 2024-09-16T15:44:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b2b400640 1 -- 172.21.15.60:0/1785242408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24003a00 con 0x7f0b18004800 2024-09-16T15:44:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.585+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b340b6f20 con 0x7f0b18004800 2024-09-16T15:44:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.586+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b340b74e0 con 0x7f0b18004800 2024-09-16T15:44:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.586+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b340d29d0 con 0x7f0b18004800 2024-09-16T15:44:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.588+0000 7f0b2b400640 1 -- 172.21.15.60:0/1785242408 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f0b24003ba0 con 0x7f0b18004800 2024-09-16T15:44:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.588+0000 7f0b2b400640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0b14078bb0 0x7f0b1407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.589+0000 7f0b2b400640 1 -- 172.21.15.60:0/1785242408 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f0b2409c5b0 con 0x7f0b18004800 2024-09-16T15:44:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.589+0000 7f0b32a00640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0b14078bb0 0x7f0b1407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.590+0000 7f0b32a00640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0b14078bb0 0x7f0b1407b070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0b20033000 tx=0x7f0b2003b040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.593+0000 7f0b2b400640 1 -- 172.21.15.60:0/1785242408 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0b24064610 con 0x7f0b18004800 2024-09-16T15:44:35.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.839+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f0b34069d80 con 0x7f0b14078bb0 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.859+0000 7f0b2b400640 1 -- 172.21.15.60:0/1785242408 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f0b20001530 con 0x7f0b14078bb0 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "50.003%", 2024-09-16T15:44:35.861 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "4.0M/8.0M", 2024-09-16T15:44:35.862 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "8/9" 2024-09-16T15:44:35.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:35.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:35.862 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0b14078bb0 msgr2=0x7f0b1407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0b14078bb0 0x7f0b1407b070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0b20033000 tx=0x7f0b2003b040 comp rx=0 tx=0).stop 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b18004800 msgr2=0x7f0b3414f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b18004800 0x7f0b3414f110 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0b24011b90 tx=0x7f0b2400f790 comp rx=0 tx=0).stop 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 shutdown_connections 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0b14078bb0 0x7f0b1407b070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b340b8790 0x7f0b340b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b340cd110 0x7f0b340cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 --2- 172.21.15.60:0/1785242408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b18004800 0x7f0b3414f110 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 >> 172.21.15.60:0/1785242408 conn(0x7f0b3409d0b0 msgr2=0x7f0b340cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:35.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 shutdown_connections 2024-09-16T15:44:35.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:35.861+0000 7f0b39000640 1 -- 172.21.15.60:0/1785242408 wait complete. 2024-09-16T15:44:35.879 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:36.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:37.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.049+0000 7f495be00640 1 -- 172.21.15.60:0/2119157076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 msgr2=0x7f495c073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.049+0000 7f495be00640 1 --2- 172.21.15.60:0/2119157076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 0x7f495c073ef0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f4954005fd0 tx=0x7f495402a520 comp rx=0 tx=0).stop 2024-09-16T15:44:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.049+0000 7f495be00640 1 -- 172.21.15.60:0/2119157076 shutdown_connections 2024-09-16T15:44:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.049+0000 7f495be00640 1 --2- 172.21.15.60:0/2119157076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 0x7f495c073ef0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.049+0000 7f495be00640 1 --2- 172.21.15.60:0/2119157076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bec0 0x7f495c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.049+0000 7f495be00640 1 -- 172.21.15.60:0/2119157076 >> 172.21.15.60:0/2119157076 conn(0x7f495c0cf9f0 msgr2=0x7f495c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.049+0000 7f495be00640 1 -- 172.21.15.60:0/2119157076 shutdown_connections 2024-09-16T15:44:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.049+0000 7f495be00640 1 -- 172.21.15.60:0/2119157076 wait complete. 2024-09-16T15:44:37.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.051+0000 7f495be00640 1 Processor -- start 2024-09-16T15:44:37.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.051+0000 7f495be00640 1 -- start start 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f495be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c072560 0x7f495c0a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f495be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 0x7f495c14a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f495be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bec0 0x7f495c0abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f495be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f495c0ae990 con 0x7f495c0cbc00 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f495be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f495c0ae810 con 0x7f495c072560 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f495be00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f495c0aeb10 con 0x7f495c15bec0 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f4952000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 0x7f495c14a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f495aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c072560 0x7f495c0a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f4952000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 0x7f495c14a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49558/0 (socket says 172.21.15.60:49558) 2024-09-16T15:44:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f4952000640 1 -- 172.21.15.60:0/1415691203 learned_addr learned my addr 172.21.15.60:0/1415691203 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.052+0000 7f495b400640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bec0 0x7f495c0abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4952000640 1 -- 172.21.15.60:0/1415691203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bec0 msgr2=0x7f495c0abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4952000640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bec0 0x7f495c0abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4952000640 1 -- 172.21.15.60:0/1415691203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c072560 msgr2=0x7f495c0a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4952000640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c072560 0x7f495c0a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4952000640 1 -- 172.21.15.60:0/1415691203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c14d5f0 con 0x7f495c0cbc00 2024-09-16T15:44:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4952000640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 0x7f495c14a840 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f494400c430 tx=0x7f494400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4953400640 1 -- 172.21.15.60:0/1415691203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944011790 con 0x7f495c0cbc00 2024-09-16T15:44:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4953400640 1 -- 172.21.15.60:0/1415691203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4944017070 con 0x7f495c0cbc00 2024-09-16T15:44:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f4953400640 1 -- 172.21.15.60:0/1415691203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944011cc0 con 0x7f495c0cbc00 2024-09-16T15:44:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c14d8d0 con 0x7f495c0cbc00 2024-09-16T15:44:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.053+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c14de90 con 0x7f495c0cbc00 2024-09-16T15:44:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.054+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c0aefc0 con 0x7f495c0cbc00 2024-09-16T15:44:37.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.056+0000 7f4953400640 1 -- 172.21.15.60:0/1415691203 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f494401c460 con 0x7f495c0cbc00 2024-09-16T15:44:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.057+0000 7f4953400640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4938078be0 0x7f493807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.057+0000 7f4953400640 1 -- 172.21.15.60:0/1415691203 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f494409abf0 con 0x7f495c0cbc00 2024-09-16T15:44:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.057+0000 7f495aa00640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4938078be0 0x7f493807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.058+0000 7f495aa00640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4938078be0 0x7f493807b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f495c15ae60 tx=0x7f494c0095e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.061+0000 7f4953400640 1 -- 172.21.15.60:0/1415691203 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4944062700 con 0x7f495c0cbc00 2024-09-16T15:44:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.294+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f495c0c7ba0 con 0x7f4938078be0 2024-09-16T15:44:37.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.315+0000 7f4953400640 1 -- 172.21.15.60:0/1415691203 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f495c0c7ba0 con 0x7f4938078be0 2024-09-16T15:44:37.317 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:37.317 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "50.003%", 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "4.0M/8.0M", 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "8/9" 2024-09-16T15:44:37.318 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:37.319 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:37.319 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4938078be0 msgr2=0x7f493807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4938078be0 0x7f493807b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f495c15ae60 tx=0x7f494c0095e0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 msgr2=0x7f495c14a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 0x7f495c14a840 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f494400c430 tx=0x7f494400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 shutdown_connections 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4938078be0 0x7f493807b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bec0 0x7f495c0abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c0cbc00 0x7f495c14a840 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 --2- 172.21.15.60:0/1415691203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c072560 0x7f495c0a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 >> 172.21.15.60:0/1415691203 conn(0x7f495c0cf9f0 msgr2=0x7f495c0b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 shutdown_connections 2024-09-16T15:44:37.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:37.318+0000 7f495be00640 1 -- 172.21.15.60:0/1415691203 wait complete. 2024-09-16T15:44:37.335 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:38.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.510+0000 7f0e8be00640 1 -- 172.21.15.60:0/179012940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 msgr2=0x7f0e8c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.510+0000 7f0e8be00640 1 --2- 172.21.15.60:0/179012940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 0x7f0e8c0b3890 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0e78005fd0 tx=0x7f0e7802a540 comp rx=0 tx=0).stop 2024-09-16T15:44:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.510+0000 7f0e8be00640 1 -- 172.21.15.60:0/179012940 shutdown_connections 2024-09-16T15:44:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.510+0000 7f0e8be00640 1 --2- 172.21.15.60:0/179012940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c0cf4b0 0x7f0e8c0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.510+0000 7f0e8be00640 1 --2- 172.21.15.60:0/179012940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 0x7f0e8c0b3890 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.510+0000 7f0e8be00640 1 -- 172.21.15.60:0/179012940 >> 172.21.15.60:0/179012940 conn(0x7f0e8c0d3d50 msgr2=0x7f0e8c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:38.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.510+0000 7f0e8be00640 1 -- 172.21.15.60:0/179012940 shutdown_connections 2024-09-16T15:44:38.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.510+0000 7f0e8be00640 1 -- 172.21.15.60:0/179012940 wait complete. 2024-09-16T15:44:38.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.511+0000 7f0e8be00640 1 Processor -- start 2024-09-16T15:44:38.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.512+0000 7f0e8be00640 1 -- start start 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.512+0000 7f0e8be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 0x7f0e8c156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.512+0000 7f0e8be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c0cf4b0 0x7f0e8c15e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.512+0000 7f0e8be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c15df60 0x7f0e8c0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.512+0000 7f0e8be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c0d24b0 con 0x7f0e8c0b3450 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.512+0000 7f0e8be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c0d2330 con 0x7f0e8c0cf4b0 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.512+0000 7f0e8be00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8c0d2630 con 0x7f0e8c15df60 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 0x7f0e8c156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 0x7f0e8c156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49576/0 (socket says 172.21.15.60:49576) 2024-09-16T15:44:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 -- 172.21.15.60:0/822295045 learned_addr learned my addr 172.21.15.60:0/822295045 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8a000640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c0cf4b0 0x7f0e8c15e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8b400640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c15df60 0x7f0e8c0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 -- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c15df60 msgr2=0x7f0e8c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c15df60 0x7f0e8c0cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 -- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c0cf4b0 msgr2=0x7f0e8c15e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c0cf4b0 0x7f0e8c15e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 -- 172.21.15.60:0/822295045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c0cc650 con 0x7f0e8c0b3450 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8a000640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c0cf4b0 0x7f0e8c15e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8b400640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c15df60 0x7f0e8c0cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.513+0000 7f0e8aa00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 0x7f0e8c156760 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c3a0 tx=0x7f0e7c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.514+0000 7f0e82a00640 1 -- 172.21.15.60:0/822295045 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c017070 con 0x7f0e8c0b3450 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.514+0000 7f0e82a00640 1 -- 172.21.15.60:0/822295045 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e7c0053c0 con 0x7f0e8c0b3450 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.514+0000 7f0e82a00640 1 -- 172.21.15.60:0/822295045 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c003850 con 0x7f0e8c0b3450 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.514+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c0cc930 con 0x7f0e8c0b3450 2024-09-16T15:44:38.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.514+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c0ccef0 con 0x7f0e8c0b3450 2024-09-16T15:44:38.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.515+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c0d2bc0 con 0x7f0e8c0b3450 2024-09-16T15:44:38.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.516+0000 7f0e82a00640 1 -- 172.21.15.60:0/822295045 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f0e7c01a070 con 0x7f0e8c0b3450 2024-09-16T15:44:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.517+0000 7f0e82a00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e6c078c00 0x7f0e6c07b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.517+0000 7f0e82a00640 1 -- 172.21.15.60:0/822295045 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f0e7c09a260 con 0x7f0e8c0b3450 2024-09-16T15:44:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.518+0000 7f0e8a000640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e6c078c00 0x7f0e6c07b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.518+0000 7f0e8a000640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e6c078c00 0x7f0e6c07b0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e78033000 tx=0x7f0e78004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:38.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.521+0000 7f0e82a00640 1 -- 172.21.15.60:0/822295045 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0e7c062440 con 0x7f0e8c0b3450 2024-09-16T15:44:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.757+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f0e8c15cb80 con 0x7f0e6c078c00 2024-09-16T15:44:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.779+0000 7f0e82a00640 1 -- 172.21.15.60:0/822295045 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f0e8c15cb80 con 0x7f0e6c078c00 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "50.003%", 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "4.0M/8.0M", 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "8/9" 2024-09-16T15:44:38.782 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:38.783 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:38.783 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:38.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e6c078c00 msgr2=0x7f0e6c07b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e6c078c00 0x7f0e6c07b0c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e78033000 tx=0x7f0e78004730 comp rx=0 tx=0).stop 2024-09-16T15:44:38.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 msgr2=0x7f0e8c156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 0x7f0e8c156760 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c3a0 tx=0x7f0e7c00c860 comp rx=0 tx=0).stop 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 shutdown_connections 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0e6c078c00 0x7f0e6c07b0c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0e8c15df60 0x7f0e8c0cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0e8c0cf4b0 0x7f0e8c15e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 --2- 172.21.15.60:0/822295045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e8c0b3450 0x7f0e8c156760 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 >> 172.21.15.60:0/822295045 conn(0x7f0e8c0d3d50 msgr2=0x7f0e8c0b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 shutdown_connections 2024-09-16T15:44:38.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:38.782+0000 7f0e8be00640 1 -- 172.21.15.60:0/822295045 wait complete. 2024-09-16T15:44:38.800 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:39.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:39.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.978+0000 7f388ac00640 1 -- 172.21.15.60:0/824535508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f386c004800 msgr2=0x7f386c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:39.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.978+0000 7f388ac00640 1 --2- 172.21.15.60:0/824535508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f386c004800 0x7f386c004c80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3870006f90 tx=0x7f386c005830 comp rx=0 tx=0).stop 2024-09-16T15:44:39.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.978+0000 7f388ac00640 1 -- 172.21.15.60:0/824535508 shutdown_connections 2024-09-16T15:44:39.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.978+0000 7f388ac00640 1 --2- 172.21.15.60:0/824535508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f386c004800 0x7f386c004c80 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.978+0000 7f388ac00640 1 -- 172.21.15.60:0/824535508 >> 172.21.15.60:0/824535508 conn(0x7f38840d1bf0 msgr2=0x7f38840d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:39.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.978+0000 7f388ac00640 1 -- 172.21.15.60:0/824535508 shutdown_connections 2024-09-16T15:44:39.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.978+0000 7f388ac00640 1 -- 172.21.15.60:0/824535508 wait complete. 2024-09-16T15:44:39.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.979+0000 7f388ac00640 1 Processor -- start 2024-09-16T15:44:39.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.980+0000 7f388ac00640 1 -- start start 2024-09-16T15:44:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.980+0000 7f388ac00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f386c004800 0x7f388414cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.980+0000 7f388ac00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f388414f550 0x7f388409e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.980+0000 7f388ac00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f388409ec40 0x7f38840c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.980+0000 7f388ac00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f38840724c0 con 0x7f388414f550 2024-09-16T15:44:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.980+0000 7f388ac00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3884072340 con 0x7f388409ec40 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.980+0000 7f388ac00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3884072640 con 0x7f386c004800 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f388414f550 0x7f388409e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f388414f550 0x7f388409e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60456/0 (socket says 172.21.15.60:60456) 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 -- 172.21.15.60:0/3591397440 learned_addr learned my addr 172.21.15.60:0/3591397440 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3889800640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f386c004800 0x7f388414cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f388a200640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f388409ec40 0x7f38840c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 -- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f386c004800 msgr2=0x7f388414cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f386c004800 0x7f388414cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 -- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f388409ec40 msgr2=0x7f38840c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f388409ec40 0x7f38840c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 -- 172.21.15.60:0/3591397440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388409f0c0 con 0x7f388414f550 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f388a200640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f388409ec40 0x7f38840c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.981+0000 7f3888e00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f388414f550 0x7f388409e700 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f387002b070 tx=0x7f38700064a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.982+0000 7f3881600640 1 -- 172.21.15.60:0/3591397440 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870003a00 con 0x7f388414f550 2024-09-16T15:44:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.982+0000 7f3881600640 1 -- 172.21.15.60:0/3591397440 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3870003c90 con 0x7f388414f550 2024-09-16T15:44:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.982+0000 7f3881600640 1 -- 172.21.15.60:0/3591397440 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870038690 con 0x7f388414f550 2024-09-16T15:44:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.982+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38840caa40 con 0x7f388414f550 2024-09-16T15:44:39.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.982+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38840cb000 con 0x7f388414f550 2024-09-16T15:44:39.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.983+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38840cfeb0 con 0x7f388414f550 2024-09-16T15:44:39.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.984+0000 7f3881600640 1 -- 172.21.15.60:0/3591397440 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f3870035020 con 0x7f388414f550 2024-09-16T15:44:39.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.985+0000 7f3881600640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3864078be0 0x7f386407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:39.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.986+0000 7f3881600640 1 -- 172.21.15.60:0/3591397440 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f38700bb5b0 con 0x7f388414f550 2024-09-16T15:44:39.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.986+0000 7f3889800640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3864078be0 0x7f386407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:39.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.986+0000 7f3889800640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3864078be0 0x7f386407b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f38841f85e0 tx=0x7f3878006b70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:39.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:39.990+0000 7f3881600640 1 -- 172.21.15.60:0/3591397440 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3870047020 con 0x7f388414f550 2024-09-16T15:44:40.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.222+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f38840cf8c0 con 0x7f3864078be0 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.242+0000 7f3881600640 1 -- 172.21.15.60:0/3591397440 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7f38840cf8c0 con 0x7f3864078be0 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:40.244 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "62.503%", 2024-09-16T15:44:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "5.0M/8.0M", 2024-09-16T15:44:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "9/9" 2024-09-16T15:44:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:40.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:40.245 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.244+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3864078be0 msgr2=0x7f386407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.244+0000 7f388ac00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3864078be0 0x7f386407b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f38841f85e0 tx=0x7f3878006b70 comp rx=0 tx=0).stop 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.244+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f388414f550 msgr2=0x7f388409e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.244+0000 7f388ac00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f388414f550 0x7f388409e700 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f387002b070 tx=0x7f38700064a0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.244+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 shutdown_connections 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.244+0000 7f388ac00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3864078be0 0x7f386407b0a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.244+0000 7f388ac00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f388409ec40 0x7f38840c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.245+0000 7f388ac00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f388414f550 0x7f388409e700 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.245+0000 7f388ac00640 1 --2- 172.21.15.60:0/3591397440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f386c004800 0x7f388414cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.245+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 >> 172.21.15.60:0/3591397440 conn(0x7f38840d1bf0 msgr2=0x7f38840b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:40.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.245+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 shutdown_connections 2024-09-16T15:44:40.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:40.245+0000 7f388ac00640 1 -- 172.21.15.60:0/3591397440 wait complete. 2024-09-16T15:44:40.262 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:41.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:41.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.436+0000 7feb20c00640 1 -- 172.21.15.60:0/4126860875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 msgr2=0x7feb1c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.436+0000 7feb20c00640 1 --2- 172.21.15.60:0/4126860875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 0x7feb1c06fce0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7feb08009830 tx=0x7feb0802a510 comp rx=0 tx=0).stop 2024-09-16T15:44:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.436+0000 7feb20c00640 1 -- 172.21.15.60:0/4126860875 shutdown_connections 2024-09-16T15:44:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.436+0000 7feb20c00640 1 --2- 172.21.15.60:0/4126860875 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb1c072560 0x7feb1c073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.436+0000 7feb20c00640 1 --2- 172.21.15.60:0/4126860875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 0x7feb1c06fce0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.436+0000 7feb20c00640 1 -- 172.21.15.60:0/4126860875 >> 172.21.15.60:0/4126860875 conn(0x7feb1c0cf9f0 msgr2=0x7feb1c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.436+0000 7feb20c00640 1 -- 172.21.15.60:0/4126860875 shutdown_connections 2024-09-16T15:44:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.436+0000 7feb20c00640 1 -- 172.21.15.60:0/4126860875 wait complete. 2024-09-16T15:44:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.438+0000 7feb20c00640 1 Processor -- start 2024-09-16T15:44:41.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.438+0000 7feb20c00640 1 -- start start 2024-09-16T15:44:41.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb20c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb1c072560 0x7feb1c155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb20c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 0x7feb1c15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb20c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c15bec0 0x7feb1c1f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb20c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c0ae960 con 0x7feb1c0cbc00 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb20c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c0ae7e0 con 0x7feb1c15bec0 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb20c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7feb1c0aeae0 con 0x7feb1c072560 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 0x7feb1c15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 0x7feb1c15dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60478/0 (socket says 172.21.15.60:60478) 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1aa00640 1 -- 172.21.15.60:0/2332232646 learned_addr learned my addr 172.21.15.60:0/2332232646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1aa00640 1 -- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb1c072560 msgr2=0x7feb1c155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1be00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c15bec0 0x7feb1c1f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1b400640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb1c072560 0x7feb1c155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1aa00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb1c072560 0x7feb1c155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1aa00640 1 -- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c15bec0 msgr2=0x7feb1c1f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1aa00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c15bec0 0x7feb1c1f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1aa00640 1 -- 172.21.15.60:0/2332232646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c0c8230 con 0x7feb1c0cbc00 2024-09-16T15:44:41.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1b400640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb1c072560 0x7feb1c155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:41.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.439+0000 7feb1be00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c15bec0 0x7feb1c1f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:41.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.440+0000 7feb1aa00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 0x7feb1c15dab0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7feb08009950 tx=0x7feb08032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:41.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.440+0000 7feb13400640 1 -- 172.21.15.60:0/2332232646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08040070 con 0x7feb1c0cbc00 2024-09-16T15:44:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.440+0000 7feb13400640 1 -- 172.21.15.60:0/2332232646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb08005310 con 0x7feb1c0cbc00 2024-09-16T15:44:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.440+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c0c8510 con 0x7feb1c0cbc00 2024-09-16T15:44:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.440+0000 7feb13400640 1 -- 172.21.15.60:0/2332232646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb080047a0 con 0x7feb1c0cbc00 2024-09-16T15:44:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.440+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c156130 con 0x7feb1c0cbc00 2024-09-16T15:44:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.441+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1c0af020 con 0x7feb1c0cbc00 2024-09-16T15:44:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.442+0000 7feb13400640 1 -- 172.21.15.60:0/2332232646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7feb08039040 con 0x7feb1c0cbc00 2024-09-16T15:44:41.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.443+0000 7feb13400640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7feafc078be0 0x7feafc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.443+0000 7feb13400640 1 -- 172.21.15.60:0/2332232646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7feb080bc300 con 0x7feb1c0cbc00 2024-09-16T15:44:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.444+0000 7feb1b400640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7feafc078be0 0x7feafc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.444+0000 7feb1b400640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7feafc078be0 0x7feafc07b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0046b0 tx=0x7feb0c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:41.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.448+0000 7feb13400640 1 -- 172.21.15.60:0/2332232646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feb08084460 con 0x7feb1c0cbc00 2024-09-16T15:44:41.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.685+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7feb1c0c7fe0 con 0x7feafc078be0 2024-09-16T15:44:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.705+0000 7feb13400640 1 -- 172.21.15.60:0/2332232646 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7feb1c0c7fe0 con 0x7feafc078be0 2024-09-16T15:44:41.707 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "62.503%", 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "5.0M/8.0M", 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "9/9" 2024-09-16T15:44:41.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:41.709 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:41.709 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:41.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.707+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7feafc078be0 msgr2=0x7feafc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.707+0000 7feb20c00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7feafc078be0 0x7feafc07b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0046b0 tx=0x7feb0c009040 comp rx=0 tx=0).stop 2024-09-16T15:44:41.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.707+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 msgr2=0x7feb1c15dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.707+0000 7feb20c00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 0x7feb1c15dab0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7feb08009950 tx=0x7feb08032c10 comp rx=0 tx=0).stop 2024-09-16T15:44:41.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.708+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 shutdown_connections 2024-09-16T15:44:41.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.708+0000 7feb20c00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7feafc078be0 0x7feafc07b0a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.708+0000 7feb20c00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c15bec0 0x7feb1c1f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.708+0000 7feb20c00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb1c0cbc00 0x7feb1c15dab0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.708+0000 7feb20c00640 1 --2- 172.21.15.60:0/2332232646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb1c072560 0x7feb1c155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.708+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 >> 172.21.15.60:0/2332232646 conn(0x7feb1c0cf9f0 msgr2=0x7feb1c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.708+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 shutdown_connections 2024-09-16T15:44:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:41.708+0000 7feb20c00640 1 -- 172.21.15.60:0/2332232646 wait complete. 2024-09-16T15:44:41.725 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:42.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.899+0000 7fcbe0000640 1 -- 172.21.15.60:0/3033451076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd80ccb00 msgr2=0x7fcbd815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.899+0000 7fcbe0000640 1 --2- 172.21.15.60:0/3033451076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd80ccb00 0x7fcbd815f9f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80d0cd0 tx=0x7fcbd002a9a0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.899+0000 7fcbe0000640 1 -- 172.21.15.60:0/3033451076 shutdown_connections 2024-09-16T15:44:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.899+0000 7fcbe0000640 1 --2- 172.21.15.60:0/3033451076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd80ccb00 0x7fcbd815f9f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.899+0000 7fcbe0000640 1 --2- 172.21.15.60:0/3033451076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbd806efa0 0x7fcbd80cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.899+0000 7fcbe0000640 1 --2- 172.21.15.60:0/3033451076 >> v2:172.21.15.100:3300/0 conn(0x7fcbd8145750 0x7fcbd8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.900+0000 7fcbe0000640 1 -- 172.21.15.60:0/3033451076 >> 172.21.15.60:0/3033451076 conn(0x7fcbd80b1840 msgr2=0x7fcbd80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.900+0000 7fcbe0000640 1 -- 172.21.15.60:0/3033451076 shutdown_connections 2024-09-16T15:44:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.900+0000 7fcbe0000640 1 -- 172.21.15.60:0/3033451076 wait complete. 2024-09-16T15:44:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.901+0000 7fcbe0000640 1 Processor -- start 2024-09-16T15:44:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.901+0000 7fcbe0000640 1 -- start start 2024-09-16T15:44:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.902+0000 7fcbe0000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbd806efa0 0x7fcbd81f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.902+0000 7fcbe0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbd80ccb00 0x7fcbd8142230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.902+0000 7fcbe0000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd8145750 0x7fcbd8142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.902+0000 7fcbe0000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81600b0 con 0x7fcbd8145750 2024-09-16T15:44:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.902+0000 7fcbe0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd815ff30 con 0x7fcbd80ccb00 2024-09-16T15:44:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.902+0000 7fcbe0000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd8160230 con 0x7fcbd806efa0 2024-09-16T15:44:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbde000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd8145750 0x7fcbd8142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbde000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd8145750 0x7fcbd8142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60488/0 (socket says 172.21.15.60:60488) 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbde000640 1 -- 172.21.15.60:0/665226857 learned_addr learned my addr 172.21.15.60:0/665226857 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbdd600640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbd806efa0 0x7fcbd81f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbdcc00640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbd80ccb00 0x7fcbd8142230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbde000640 1 -- 172.21.15.60:0/665226857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbd806efa0 msgr2=0x7fcbd81f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbde000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbd806efa0 0x7fcbd81f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbde000640 1 -- 172.21.15.60:0/665226857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbd80ccb00 msgr2=0x7fcbd8142230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbde000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbd80ccb00 0x7fcbd8142230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.903+0000 7fcbde000640 1 -- 172.21.15.60:0/665226857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd80c4ab0 con 0x7fcbd8145750 2024-09-16T15:44:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.904+0000 7fcbde000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd8145750 0x7fcbd8142770 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009d60 tx=0x7fcbd0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.904+0000 7fcbd5600640 1 -- 172.21.15.60:0/665226857 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0047070 con 0x7fcbd8145750 2024-09-16T15:44:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.904+0000 7fcbd5600640 1 -- 172.21.15.60:0/665226857 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbd002f600 con 0x7fcbd8145750 2024-09-16T15:44:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.904+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd80c4d50 con 0x7fcbd8145750 2024-09-16T15:44:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.904+0000 7fcbd5600640 1 -- 172.21.15.60:0/665226857 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0033050 con 0x7fcbd8145750 2024-09-16T15:44:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.904+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd80c5310 con 0x7fcbd8145750 2024-09-16T15:44:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.905+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd8160710 con 0x7fcbd8145750 2024-09-16T15:44:42.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.906+0000 7fcbd5600640 1 -- 172.21.15.60:0/665226857 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fcbd004a070 con 0x7fcbd8145750 2024-09-16T15:44:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.907+0000 7fcbd5600640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcbb8078bb0 0x7fcbb807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.907+0000 7fcbd5600640 1 -- 172.21.15.60:0/665226857 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fcbd00bbda0 con 0x7fcbd8145750 2024-09-16T15:44:42.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.907+0000 7fcbdd600640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcbb8078bb0 0x7fcbb807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.908+0000 7fcbdd600640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcbb8078bb0 0x7fcbb807b070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009b30 tx=0x7fcbcc005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:42.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:42.911+0000 7fcbd5600640 1 -- 172.21.15.60:0/665226857 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcbd0083820 con 0x7fcbd8145750 2024-09-16T15:44:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.147+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7fcbd80cc580 con 0x7fcbb8078bb0 2024-09-16T15:44:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.167+0000 7fcbd5600640 1 -- 172.21.15.60:0/665226857 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+303 (secure 0 0 0) 0x7fcbd80cc580 con 0x7fcbb8078bb0 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "62.503%", 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "5.0M/8.0M", 2024-09-16T15:44:43.170 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "9/9" 2024-09-16T15:44:43.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:43.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:43.171 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:43.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcbb8078bb0 msgr2=0x7fcbb807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcbb8078bb0 0x7fcbb807b070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009b30 tx=0x7fcbcc005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd8145750 msgr2=0x7fcbd8142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd8145750 0x7fcbd8142770 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009d60 tx=0x7fcbd0003f10 comp rx=0 tx=0).stop 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 shutdown_connections 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcbb8078bb0 0x7fcbb807b070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcbd8145750 0x7fcbd8142770 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbd80ccb00 0x7fcbd8142230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 --2- 172.21.15.60:0/665226857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbd806efa0 0x7fcbd81f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 >> 172.21.15.60:0/665226857 conn(0x7fcbd80b1840 msgr2=0x7fcbd80cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 shutdown_connections 2024-09-16T15:44:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:43.170+0000 7fcbe0000640 1 -- 172.21.15.60:0/665226857 wait complete. 2024-09-16T15:44:43.188 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:44.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.360+0000 7fd1bdc00640 1 -- 172.21.15.60:0/3145013821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1b8145750 msgr2=0x7fd1b8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.360+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/3145013821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1b8145750 0x7fd1b8073440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005fd0 tx=0x7fd1a802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.360+0000 7fd1bdc00640 1 -- 172.21.15.60:0/3145013821 shutdown_connections 2024-09-16T15:44:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.360+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/3145013821 >> v2:172.21.15.100:3300/0 conn(0x7fd1b80cccd0 0x7fd1b80cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.360+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/3145013821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1b8145750 0x7fd1b8073440 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.360+0000 7fd1bdc00640 1 -- 172.21.15.60:0/3145013821 >> 172.21.15.60:0/3145013821 conn(0x7fd1b80b1840 msgr2=0x7fd1b80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:44.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.360+0000 7fd1bdc00640 1 -- 172.21.15.60:0/3145013821 shutdown_connections 2024-09-16T15:44:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.360+0000 7fd1bdc00640 1 -- 172.21.15.60:0/3145013821 wait complete. 2024-09-16T15:44:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.362+0000 7fd1bdc00640 1 Processor -- start 2024-09-16T15:44:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.362+0000 7fd1bdc00640 1 -- start start 2024-09-16T15:44:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1bdc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b806efa0 0x7fd1b815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1bdc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b80cccd0 0x7fd1b80c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1bdc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1b8145750 0x7fd1b80c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1bdc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b80ce480 con 0x7fd1b80cccd0 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1bdc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b80ce300 con 0x7fd1b806efa0 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1bdc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b80ce600 con 0x7fd1b8145750 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1b6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b80cccd0 0x7fd1b80c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1b6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b80cccd0 0x7fd1b80c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60496/0 (socket says 172.21.15.60:60496) 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1b6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b806efa0 0x7fd1b815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1b6000640 1 -- 172.21.15.60:0/553452911 learned_addr learned my addr 172.21.15.60:0/553452911 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.363+0000 7fd1b7400640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1b8145750 0x7fd1b80c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:44.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1b6000640 1 -- 172.21.15.60:0/553452911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1b8145750 msgr2=0x7fd1b80c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1b6000640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1b8145750 0x7fd1b80c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1b6000640 1 -- 172.21.15.60:0/553452911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b806efa0 msgr2=0x7fd1b815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1b6000640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b806efa0 0x7fd1b815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1b6000640 1 -- 172.21.15.60:0/553452911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b80c65d0 con 0x7fd1b80cccd0 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1b6000640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b80cccd0 0x7fd1b80c5070 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c430 tx=0x7fd1a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1aea00640 1 -- 172.21.15.60:0/553452911 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4011790 con 0x7fd1b80cccd0 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1aea00640 1 -- 172.21.15.60:0/553452911 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1a4017070 con 0x7fd1b80cccd0 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1aea00640 1 -- 172.21.15.60:0/553452911 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4011d00 con 0x7fd1b80cccd0 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.364+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b80c6010 con 0x7fd1b80cccd0 2024-09-16T15:44:44.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.365+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b813ef30 con 0x7fd1b80cccd0 2024-09-16T15:44:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.366+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b80cdbb0 con 0x7fd1b80cccd0 2024-09-16T15:44:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.367+0000 7fd1aea00640 1 -- 172.21.15.60:0/553452911 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fd1a401c460 con 0x7fd1b80cccd0 2024-09-16T15:44:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.368+0000 7fd1aea00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd198078bb0 0x7fd19807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.368+0000 7fd1aea00640 1 -- 172.21.15.60:0/553452911 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fd1a409acb0 con 0x7fd1b80cccd0 2024-09-16T15:44:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.368+0000 7fd1b6a00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd198078bb0 0x7fd19807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:44.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.369+0000 7fd1b6a00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd198078bb0 0x7fd19807b070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8033000 tx=0x7fd1a8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:44.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.372+0000 7fd1aea00640 1 -- 172.21.15.60:0/553452911 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd1b80c65d0 con 0x7fd1b80cccd0 2024-09-16T15:44:44.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.607+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7fd1b80af630 con 0x7fd198078bb0 2024-09-16T15:44:44.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.630+0000 7fd1aea00640 1 -- 172.21.15.60:0/553452911 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7fd1b80af630 con 0x7fd198078bb0 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "75.003%", 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "6.0M/8.0M", 2024-09-16T15:44:44.633 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "10/9" 2024-09-16T15:44:44.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:44.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:44.634 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd198078bb0 msgr2=0x7fd19807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd198078bb0 0x7fd19807b070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8033000 tx=0x7fd1a8004730 comp rx=0 tx=0).stop 2024-09-16T15:44:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b80cccd0 msgr2=0x7fd1b80c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b80cccd0 0x7fd1b80c5070 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c430 tx=0x7fd1a400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 shutdown_connections 2024-09-16T15:44:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd198078bb0 0x7fd19807b070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1b8145750 0x7fd1b80c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1b80cccd0 0x7fd1b80c5070 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 --2- 172.21.15.60:0/553452911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b806efa0 0x7fd1b815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 >> 172.21.15.60:0/553452911 conn(0x7fd1b80b1840 msgr2=0x7fd1b80b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 shutdown_connections 2024-09-16T15:44:44.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:44.633+0000 7fd1bdc00640 1 -- 172.21.15.60:0/553452911 wait complete. 2024-09-16T15:44:44.652 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:45.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:45.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 -- 172.21.15.60:0/1229444181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815bec0 msgr2=0x7fc38806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:45.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 --2- 172.21.15.60:0/1229444181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815bec0 0x7fc38806fce0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc388072b20 tx=0x7fc37c02a560 comp rx=0 tx=0).stop 2024-09-16T15:44:45.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 -- 172.21.15.60:0/1229444181 shutdown_connections 2024-09-16T15:44:45.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 --2- 172.21.15.60:0/1229444181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3880cb4f0 0x7fc3880cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 --2- 172.21.15.60:0/1229444181 >> v2:172.21.15.100:3300/0 conn(0x7fc388072560 0x7fc388073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 --2- 172.21.15.60:0/1229444181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc38815bec0 0x7fc38806fce0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 -- 172.21.15.60:0/1229444181 >> 172.21.15.60:0/1229444181 conn(0x7fc3880cf9f0 msgr2=0x7fc3880cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:45.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 -- 172.21.15.60:0/1229444181 shutdown_connections 2024-09-16T15:44:45.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.826+0000 7fc38e800640 1 -- 172.21.15.60:0/1229444181 wait complete. 2024-09-16T15:44:45.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.828+0000 7fc38e800640 1 Processor -- start 2024-09-16T15:44:45.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.828+0000 7fc38e800640 1 -- start start 2024-09-16T15:44:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.828+0000 7fc38e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc388072560 0x7fc3880c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.828+0000 7fc38e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3880cb4f0 0x7fc3881f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38e800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815bec0 0x7fc38814dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38e800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880ae850 con 0x7fc388072560 2024-09-16T15:44:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880ae6d0 con 0x7fc3880cb4f0 2024-09-16T15:44:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38e800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc3880ae9d0 con 0x7fc38815bec0 2024-09-16T15:44:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc388072560 0x7fc3880c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc388072560 0x7fc3880c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60524/0 (socket says 172.21.15.60:60524) 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38d400640 1 -- 172.21.15.60:0/2038521459 learned_addr learned my addr 172.21.15.60:0/2038521459 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38de00640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815bec0 0x7fc38814dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38ca00640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3880cb4f0 0x7fc3881f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38d400640 1 -- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815bec0 msgr2=0x7fc38814dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38d400640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815bec0 0x7fc38814dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38d400640 1 -- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3880cb4f0 msgr2=0x7fc3881f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38d400640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3880cb4f0 0x7fc3881f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38d400640 1 -- 172.21.15.60:0/2038521459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3880c89d0 con 0x7fc388072560 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.829+0000 7fc38ca00640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3880cb4f0 0x7fc3881f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.830+0000 7fc38d400640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc388072560 0x7fc3880c8490 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc37c000c00 tx=0x7fc37c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.830+0000 7fc385600640 1 -- 172.21.15.60:0/2038521459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c03e030 con 0x7fc388072560 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.830+0000 7fc385600640 1 -- 172.21.15.60:0/2038521459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc37c0047a0 con 0x7fc388072560 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.830+0000 7fc385600640 1 -- 172.21.15.60:0/2038521459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c035040 con 0x7fc388072560 2024-09-16T15:44:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.830+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3880c8ca0 con 0x7fc388072560 2024-09-16T15:44:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.830+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3881f7da0 con 0x7fc388072560 2024-09-16T15:44:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.831+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3880aef00 con 0x7fc388072560 2024-09-16T15:44:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.832+0000 7fc385600640 1 -- 172.21.15.60:0/2038521459 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fc37c041030 con 0x7fc388072560 2024-09-16T15:44:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.833+0000 7fc385600640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc368078bb0 0x7fc36807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.833+0000 7fc385600640 1 -- 172.21.15.60:0/2038521459 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fc37c0bb700 con 0x7fc388072560 2024-09-16T15:44:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.833+0000 7fc38ca00640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc368078bb0 0x7fc36807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:45.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.834+0000 7fc38ca00640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc368078bb0 0x7fc36807b070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc3880a9a20 tx=0x7fc374008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:45.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:45.838+0000 7fc385600640 1 -- 172.21.15.60:0/2038521459 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc37c0837f0 con 0x7fc388072560 2024-09-16T15:44:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.069+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7fc38806fef0 con 0x7fc368078bb0 2024-09-16T15:44:46.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.091+0000 7fc385600640 1 -- 172.21.15.60:0/2038521459 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7fc38806fef0 con 0x7fc368078bb0 2024-09-16T15:44:46.093 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:46.093 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "75.003%", 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "6.0M/8.0M", 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "10/9" 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:46.094 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:46.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc368078bb0 msgr2=0x7fc36807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc368078bb0 0x7fc36807b070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc3880a9a20 tx=0x7fc374008040 comp rx=0 tx=0).stop 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc388072560 msgr2=0x7fc3880c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc388072560 0x7fc3880c8490 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc37c000c00 tx=0x7fc37c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 shutdown_connections 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc368078bb0 0x7fc36807b070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc38815bec0 0x7fc38814dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3880cb4f0 0x7fc3881f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 --2- 172.21.15.60:0/2038521459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc388072560 0x7fc3880c8490 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 >> 172.21.15.60:0/2038521459 conn(0x7fc3880cf9f0 msgr2=0x7fc3880b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 shutdown_connections 2024-09-16T15:44:46.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:46.095+0000 7fc38e800640 1 -- 172.21.15.60:0/2038521459 wait complete. 2024-09-16T15:44:46.112 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:47.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:47.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.286+0000 7f3ad8200640 1 -- 172.21.15.60:0/2570297436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 msgr2=0x7f3ad00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.286+0000 7f3ad8200640 1 --2- 172.21.15.60:0/2570297436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 0x7f3ad00cdbb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005fd0 tx=0x7f3ac802aa10 comp rx=0 tx=0).stop 2024-09-16T15:44:47.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.286+0000 7f3ad8200640 1 -- 172.21.15.60:0/2570297436 shutdown_connections 2024-09-16T15:44:47.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.286+0000 7f3ad8200640 1 --2- 172.21.15.60:0/2570297436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 0x7f3ad00cdbb0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.286+0000 7f3ad8200640 1 --2- 172.21.15.60:0/2570297436 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ad00cccd0 0x7f3ad0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.286+0000 7f3ad8200640 1 -- 172.21.15.60:0/2570297436 >> 172.21.15.60:0/2570297436 conn(0x7f3ad00b1840 msgr2=0x7f3ad00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:47.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.286+0000 7f3ad8200640 1 -- 172.21.15.60:0/2570297436 shutdown_connections 2024-09-16T15:44:47.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.286+0000 7f3ad8200640 1 -- 172.21.15.60:0/2570297436 wait complete. 2024-09-16T15:44:47.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.288+0000 7f3ad8200640 1 Processor -- start 2024-09-16T15:44:47.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.288+0000 7f3ad8200640 1 -- start start 2024-09-16T15:44:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad8200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 0x7f3ad01f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:47.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad8200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ad00cccd0 0x7f3ad01f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad8200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ad0145750 0x7f3ad015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad8200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad00ce480 con 0x7f3ad006efa0 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad8200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad00ce300 con 0x7f3ad00cccd0 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad8200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad00ce600 con 0x7f3ad0145750 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad5800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 0x7f3ad01f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad5800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 0x7f3ad01f7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60556/0 (socket says 172.21.15.60:60556) 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad5800640 1 -- 172.21.15.60:0/4196589464 learned_addr learned my addr 172.21.15.60:0/4196589464 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad6200640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ad0145750 0x7f3ad015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.289+0000 7f3ad4e00640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ad00cccd0 0x7f3ad01f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3ad5800640 1 -- 172.21.15.60:0/4196589464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ad0145750 msgr2=0x7f3ad015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3ad5800640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ad0145750 0x7f3ad015dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3ad5800640 1 -- 172.21.15.60:0/4196589464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ad00cccd0 msgr2=0x7f3ad01f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3ad5800640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ad00cccd0 0x7f3ad01f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3ad5800640 1 -- 172.21.15.60:0/4196589464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad0146640 con 0x7f3ad006efa0 2024-09-16T15:44:47.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3ad5800640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 0x7f3ad01f7e00 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c3a0 tx=0x7f3ac400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:47.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3acd600640 1 -- 172.21.15.60:0/4196589464 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4017070 con 0x7f3ad006efa0 2024-09-16T15:44:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3acd600640 1 -- 172.21.15.60:0/4196589464 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ac40053c0 con 0x7f3ad006efa0 2024-09-16T15:44:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3acd600640 1 -- 172.21.15.60:0/4196589464 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4003850 con 0x7f3ad006efa0 2024-09-16T15:44:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad013a070 con 0x7f3ad006efa0 2024-09-16T15:44:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.290+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad00b30c0 con 0x7f3ad006efa0 2024-09-16T15:44:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.291+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad00cdbb0 con 0x7f3ad006efa0 2024-09-16T15:44:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.292+0000 7f3acd600640 1 -- 172.21.15.60:0/4196589464 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f3ac401a070 con 0x7f3ad006efa0 2024-09-16T15:44:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.294+0000 7f3acd600640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ab0078bb0 0x7f3ab007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.294+0000 7f3acd600640 1 -- 172.21.15.60:0/4196589464 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f3ac409a230 con 0x7f3ad006efa0 2024-09-16T15:44:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.294+0000 7f3ad4e00640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ab0078bb0 0x7f3ab007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.295+0000 7f3ad4e00640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ab0078bb0 0x7f3ab007b070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00c9fc0 tx=0x7f3abc00b390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:47.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.298+0000 7f3acd600640 1 -- 172.21.15.60:0/4196589464 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3ac4061d40 con 0x7f3ad006efa0 2024-09-16T15:44:47.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.535+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f3ad013a400 con 0x7f3ab0078bb0 2024-09-16T15:44:47.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.557+0000 7f3acd600640 1 -- 172.21.15.60:0/4196589464 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7f3ad013a400 con 0x7f3ab0078bb0 2024-09-16T15:44:47.558 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:47.558 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:47.558 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:47.558 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:47.558 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:47.558 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:47.558 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:47.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:47.559 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:47.559 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "75.003%", 2024-09-16T15:44:47.559 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "6.0M/8.0M", 2024-09-16T15:44:47.559 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "10/9" 2024-09-16T15:44:47.559 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:47.559 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:47.559 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ab0078bb0 msgr2=0x7f3ab007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ab0078bb0 0x7f3ab007b070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00c9fc0 tx=0x7f3abc00b390 comp rx=0 tx=0).stop 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 msgr2=0x7f3ad01f7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 0x7f3ad01f7e00 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c3a0 tx=0x7f3ac400c860 comp rx=0 tx=0).stop 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 shutdown_connections 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ab0078bb0 0x7f3ab007b070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ad0145750 0x7f3ad015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ad00cccd0 0x7f3ad01f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 --2- 172.21.15.60:0/4196589464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ad006efa0 0x7f3ad01f7e00 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 >> 172.21.15.60:0/4196589464 conn(0x7f3ad00b1840 msgr2=0x7f3ad00aa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 shutdown_connections 2024-09-16T15:44:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:47.559+0000 7f3ad8200640 1 -- 172.21.15.60:0/4196589464 wait complete. 2024-09-16T15:44:47.576 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:48.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:48.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.752+0000 7fa257200640 1 -- 172.21.15.60:0/3384405501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580d4980 msgr2=0x7fa2580cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.752+0000 7fa257200640 1 --2- 172.21.15.60:0/3384405501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580d4980 0x7fa2580cee70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa244009900 tx=0x7fa24402a5f0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.752+0000 7fa257200640 1 -- 172.21.15.60:0/3384405501 shutdown_connections 2024-09-16T15:44:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.752+0000 7fa257200640 1 --2- 172.21.15.60:0/3384405501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580d4980 0x7fa2580cee70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.752+0000 7fa257200640 1 --2- 172.21.15.60:0/3384405501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa25815df50 0x7fa2580ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.752+0000 7fa257200640 1 -- 172.21.15.60:0/3384405501 >> 172.21.15.60:0/3384405501 conn(0x7fa2580d3e60 msgr2=0x7fa2580d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.752+0000 7fa257200640 1 -- 172.21.15.60:0/3384405501 shutdown_connections 2024-09-16T15:44:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.752+0000 7fa257200640 1 -- 172.21.15.60:0/3384405501 wait complete. 2024-09-16T15:44:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.754+0000 7fa257200640 1 Processor -- start 2024-09-16T15:44:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.754+0000 7fa257200640 1 -- start start 2024-09-16T15:44:48.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa257200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580cf3b0 0x7fa2580a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa257200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2580d4980 0x7fa2580ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa257200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa25815df50 0x7fa2580cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa257200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa2580d0ce0 con 0x7fa25815df50 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa257200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2580c9030 con 0x7fa2580cf3b0 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa257200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa2580d0e60 con 0x7fa2580d4980 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa256800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa25815df50 0x7fa2580cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa256800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa25815df50 0x7fa2580cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60568/0 (socket says 172.21.15.60:60568) 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa256800640 1 -- 172.21.15.60:0/2555245501 learned_addr learned my addr 172.21.15.60:0/2555245501 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa255400640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2580d4980 0x7fa2580ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.755+0000 7fa255e00640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580cf3b0 0x7fa2580a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa256800640 1 -- 172.21.15.60:0/2555245501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2580d4980 msgr2=0x7fa2580ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:48.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa256800640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2580d4980 0x7fa2580ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa256800640 1 -- 172.21.15.60:0/2555245501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580cf3b0 msgr2=0x7fa2580a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:48.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa256800640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580cf3b0 0x7fa2580a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa256800640 1 -- 172.21.15.60:0/2555245501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2581f8bc0 con 0x7fa25815df50 2024-09-16T15:44:48.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa256800640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa25815df50 0x7fa2580cd430 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa25000c710 tx=0x7fa25000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:48.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa24e000640 1 -- 172.21.15.60:0/2555245501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250016020 con 0x7fa25815df50 2024-09-16T15:44:48.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa24e000640 1 -- 172.21.15.60:0/2555245501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa25000ce20 con 0x7fa25815df50 2024-09-16T15:44:48.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2581f7e00 con 0x7fa25815df50 2024-09-16T15:44:48.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa24e000640 1 -- 172.21.15.60:0/2555245501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250006e60 con 0x7fa25815df50 2024-09-16T15:44:48.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.756+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581f83c0 con 0x7fa25815df50 2024-09-16T15:44:48.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.757+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2580d3010 con 0x7fa25815df50 2024-09-16T15:44:48.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.758+0000 7fa24e000640 1 -- 172.21.15.60:0/2555245501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fa250003bd0 con 0x7fa25815df50 2024-09-16T15:44:48.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.759+0000 7fa24e000640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa238078bb0 0x7fa23807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:48.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.760+0000 7fa24e000640 1 -- 172.21.15.60:0/2555245501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fa25009ab00 con 0x7fa25815df50 2024-09-16T15:44:48.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.760+0000 7fa255e00640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa238078bb0 0x7fa23807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:48.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.760+0000 7fa255e00640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa238078bb0 0x7fa23807b070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa248009b30 tx=0x7fa248005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:48.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.764+0000 7fa24e000640 1 -- 172.21.15.60:0/2555245501 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa2580d3010 con 0x7fa25815df50 2024-09-16T15:44:48.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:48.998+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7fa2580cda60 con 0x7fa238078bb0 2024-09-16T15:44:49.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.019+0000 7fa24e000640 1 -- 172.21.15.60:0/2555245501 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7fa248011be0 con 0x7fa238078bb0 2024-09-16T15:44:49.020 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "in-progress", 2024-09-16T15:44:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:44:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:44:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "subvol_snap57910" 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "progress_report": { 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "percentage cloned": "75.003%", 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "amount cloned": "6.0M/8.0M", 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "files cloned": "10/9" 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:49.021 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa238078bb0 msgr2=0x7fa23807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa238078bb0 0x7fa23807b070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa248009b30 tx=0x7fa248005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa25815df50 msgr2=0x7fa2580cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa25815df50 0x7fa2580cd430 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa25000c710 tx=0x7fa25000cbd0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 shutdown_connections 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa238078bb0 0x7fa23807b070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa25815df50 0x7fa2580cd430 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2580d4980 0x7fa2580ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 --2- 172.21.15.60:0/2555245501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2580cf3b0 0x7fa2580a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 >> 172.21.15.60:0/2555245501 conn(0x7fa2580d3e60 msgr2=0x7fa25806d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:49.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.021+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 shutdown_connections 2024-09-16T15:44:49.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:49.022+0000 7fa257200640 1 -- 172.21.15.60:0/2555245501 wait complete. 2024-09-16T15:44:49.038 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:44:50.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94430 2024-09-16T15:44:50.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.213+0000 7f4bbf200640 1 -- 172.21.15.60:0/3908025634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 msgr2=0x7f4bc006e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.213+0000 7f4bbf200640 1 --2- 172.21.15.60:0/3908025634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 0x7f4bc006e880 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009be0 tx=0x7f4bb802a970 comp rx=0 tx=0).stop 2024-09-16T15:44:50.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.213+0000 7f4bbf200640 1 -- 172.21.15.60:0/3908025634 shutdown_connections 2024-09-16T15:44:50.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.213+0000 7f4bbf200640 1 --2- 172.21.15.60:0/3908025634 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 0x7f4bc006e880 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.213+0000 7f4bbf200640 1 -- 172.21.15.60:0/3908025634 >> 172.21.15.60:0/3908025634 conn(0x7f4bc00cf9f0 msgr2=0x7f4bc00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:50.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.213+0000 7f4bbf200640 1 -- 172.21.15.60:0/3908025634 shutdown_connections 2024-09-16T15:44:50.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.213+0000 7f4bbf200640 1 -- 172.21.15.60:0/3908025634 wait complete. 2024-09-16T15:44:50.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.215+0000 7f4bbf200640 1 Processor -- start 2024-09-16T15:44:50.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.215+0000 7f4bbf200640 1 -- start start 2024-09-16T15:44:50.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.215+0000 7f4bbf200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 0x7f4bc00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbf200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc015e870 0x7f4bc01f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015f360 0x7f4bc0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbf200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0074440 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc00742c0 con 0x7f4bc015f360 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbf200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc00745c0 con 0x7f4bc015e870 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbde00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 0x7f4bc00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbde00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 0x7f4bc00b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36054/0 (socket says 172.21.15.60:36054) 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbde00640 1 -- 172.21.15.60:0/2749157945 learned_addr learned my addr 172.21.15.60:0/2749157945 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbe800640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015f360 0x7f4bc0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbd400640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc015e870 0x7f4bc01f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbde00640 1 -- 172.21.15.60:0/2749157945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc015e870 msgr2=0x7f4bc01f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbde00640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc015e870 0x7f4bc01f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbde00640 1 -- 172.21.15.60:0/2749157945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015f360 msgr2=0x7f4bc0154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbde00640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015f360 0x7f4bc0154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.216+0000 7f4bbde00640 1 -- 172.21.15.60:0/2749157945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc00b4460 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.217+0000 7f4bbde00640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 0x7f4bc00b3dd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000c4d0 tx=0x7f4bb000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:50.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.217+0000 7f4bb6000640 1 -- 172.21.15.60:0/2749157945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0016020 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.217+0000 7f4bb6000640 1 -- 172.21.15.60:0/2749157945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bb0012920 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.217+0000 7f4bb6000640 1 -- 172.21.15.60:0/2749157945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0012d30 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.217+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc00b4740 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.217+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc01f5370 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.218+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc006e880 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.219+0000 7f4bb6000640 1 -- 172.21.15.60:0/2749157945 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f4bb0019020 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.221+0000 7f4bb6000640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ba0078bb0 0x7f4ba007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.221+0000 7f4bb6000640 1 -- 172.21.15.60:0/2749157945 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f4bb009af10 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.221+0000 7f4bbd400640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ba0078bb0 0x7f4ba007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.222+0000 7f4bbd400640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ba0078bb0 0x7f4ba007b070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00c9350 tx=0x7f4bac009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:50.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.225+0000 7f4bb6000640 1 -- 172.21.15.60:0/2749157945 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4bb0020020 con 0x7f4bc00ae5e0 2024-09-16T15:44:50.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:50.460+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f4bc00d0710 con 0x7f4ba0078bb0 2024-09-16T15:44:54.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.169+0000 7f4bb6000640 1 -- 172.21.15.60:0/2749157945 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f4bc00d0710 con 0x7f4ba0078bb0 2024-09-16T15:44:54.170 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:54.170 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:44:54.170 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "complete" 2024-09-16T15:44:54.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:44:54.171 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ba0078bb0 msgr2=0x7f4ba007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ba0078bb0 0x7f4ba007b070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4bc00c9350 tx=0x7f4bac009040 comp rx=0 tx=0).stop 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 msgr2=0x7f4bc00b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 0x7f4bc00b3dd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000c4d0 tx=0x7f4bb000c990 comp rx=0 tx=0).stop 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 shutdown_connections 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ba0078bb0 0x7f4ba007b070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015f360 0x7f4bc0154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc015e870 0x7f4bc01f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 --2- 172.21.15.60:0/2749157945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bc00ae5e0 0x7f4bc00b3dd0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 >> 172.21.15.60:0/2749157945 conn(0x7f4bc00cf9f0 msgr2=0x7f4bc00cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:54.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 shutdown_connections 2024-09-16T15:44:54.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.172+0000 7f4bbf200640 1 -- 172.21.15.60:0/2749157945 wait complete. 2024-09-16T15:44:54.192 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-16T15:44:54.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:54.193 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone94430/.meta) 2024-09-16T15:44:54.253 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:44:54.254 INFO:teuthology.orchestra.run.smithi060.stdout:version = 2 2024-09-16T15:44:54.254 INFO:teuthology.orchestra.run.smithi060.stdout:type = clone 2024-09-16T15:44:54.254 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5 2024-09-16T15:44:54.254 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:44:54.254 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:44:54.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol62870 2024-09-16T15:44:54.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.420+0000 7f698ea00640 1 -- 172.21.15.60:0/4169018814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698815bec0 msgr2=0x7f698806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.420+0000 7f698ea00640 1 --2- 172.21.15.60:0/4169018814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698815bec0 0x7f698806fce0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f6988072b20 tx=0x7f697c02a560 comp rx=0 tx=0).stop 2024-09-16T15:44:54.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.420+0000 7f698ea00640 1 -- 172.21.15.60:0/4169018814 shutdown_connections 2024-09-16T15:44:54.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.420+0000 7f698ea00640 1 --2- 172.21.15.60:0/4169018814 >> v2:172.21.15.100:3300/0 conn(0x7f6988072560 0x7f6988073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.420+0000 7f698ea00640 1 --2- 172.21.15.60:0/4169018814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f698815bec0 0x7f698806fce0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.420+0000 7f698ea00640 1 -- 172.21.15.60:0/4169018814 >> 172.21.15.60:0/4169018814 conn(0x7f69880cf9f0 msgr2=0x7f69880cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:54.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.420+0000 7f698ea00640 1 -- 172.21.15.60:0/4169018814 shutdown_connections 2024-09-16T15:44:54.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.420+0000 7f698ea00640 1 -- 172.21.15.60:0/4169018814 wait complete. 2024-09-16T15:44:54.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.422+0000 7f698ea00640 1 Processor -- start 2024-09-16T15:44:54.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.422+0000 7f698ea00640 1 -- start start 2024-09-16T15:44:54.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6988072560 0x7f698814de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69880cb4f0 0x7f69881508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698ea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698815bec0 0x7f698814d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698ea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f69880cbdf0 con 0x7f6988072560 2024-09-16T15:44:54.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f69880cbc70 con 0x7f69880cb4f0 2024-09-16T15:44:54.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698ea00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f69880aebd0 con 0x7f698815bec0 2024-09-16T15:44:54.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6988072560 0x7f698814de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6988072560 0x7f698814de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36070/0 (socket says 172.21.15.60:36070) 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.423+0000 7f698d600640 1 -- 172.21.15.60:0/4129201614 learned_addr learned my addr 172.21.15.60:0/4129201614 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698cc00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69880cb4f0 0x7f69881508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698e000640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698815bec0 0x7f698814d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698d600640 1 -- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698815bec0 msgr2=0x7f698814d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698d600640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698815bec0 0x7f698814d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698d600640 1 -- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69880cb4f0 msgr2=0x7f69881508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698d600640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69880cb4f0 0x7f69881508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698d600640 1 -- 172.21.15.60:0/4129201614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6988150e00 con 0x7f6988072560 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698e000640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698815bec0 0x7f698814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.424+0000 7f698d600640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6988072560 0x7f698814de20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f697c034040 tx=0x7f697c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.425+0000 7f6985600640 1 -- 172.21.15.60:0/4129201614 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c03e030 con 0x7f6988072560 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.425+0000 7f6985600640 1 -- 172.21.15.60:0/4129201614 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f697c003e00 con 0x7f6988072560 2024-09-16T15:44:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.425+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881f8a50 con 0x7f6988072560 2024-09-16T15:44:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.425+0000 7f6985600640 1 -- 172.21.15.60:0/4129201614 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c035040 con 0x7f6988072560 2024-09-16T15:44:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.425+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69880a9470 con 0x7f6988072560 2024-09-16T15:44:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.426+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69880af050 con 0x7f6988072560 2024-09-16T15:44:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.427+0000 7f6985600640 1 -- 172.21.15.60:0/4129201614 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f697c041030 con 0x7f6988072560 2024-09-16T15:44:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.428+0000 7f6985600640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6968078bb0 0x7f696807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.428+0000 7f6985600640 1 -- 172.21.15.60:0/4129201614 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f697c0bb600 con 0x7f6988072560 2024-09-16T15:44:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.428+0000 7f698cc00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6968078bb0 0x7f696807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.429+0000 7f698cc00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6968078bb0 0x7f696807b070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f698815f620 tx=0x7f697400b390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:54.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.432+0000 7f6985600640 1 -- 172.21.15.60:0/4129201614 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6988150e00 con 0x7f6988072560 2024-09-16T15:44:54.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.661+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol62870", "target": ["mon-mgr", ""]}) -- 0x7f69880af570 con 0x7f6968078bb0 2024-09-16T15:44:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.680+0000 7f6985600640 1 -- 172.21.15.60:0/4129201614 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f69880af570 con 0x7f6968078bb0 2024-09-16T15:44:54.681 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177 2024-09-16T15:44:54.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6968078bb0 msgr2=0x7f696807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6968078bb0 0x7f696807b070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f698815f620 tx=0x7f697400b390 comp rx=0 tx=0).stop 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6988072560 msgr2=0x7f698814de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6988072560 0x7f698814de20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f697c034040 tx=0x7f697c003910 comp rx=0 tx=0).stop 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 shutdown_connections 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6968078bb0 0x7f696807b070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698815bec0 0x7f698814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69880cb4f0 0x7f69881508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 --2- 172.21.15.60:0/4129201614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6988072560 0x7f698814de20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 >> 172.21.15.60:0/4129201614 conn(0x7f69880cf9f0 msgr2=0x7f69880ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.682+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 shutdown_connections 2024-09-16T15:44:54.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.683+0000 7f698ea00640 1 -- 172.21.15.60:0/4129201614 wait complete. 2024-09-16T15:44:54.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone94430 2024-09-16T15:44:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.861+0000 7f3b65000640 1 -- 172.21.15.60:0/1537603411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 msgr2=0x7f3b600cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.861+0000 7f3b65000640 1 --2- 172.21.15.60:0/1537603411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 0x7f3b600cf070 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3b600af120 tx=0x7f3b5802a9e0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.861+0000 7f3b65000640 1 -- 172.21.15.60:0/1537603411 shutdown_connections 2024-09-16T15:44:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.861+0000 7f3b65000640 1 --2- 172.21.15.60:0/1537603411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 0x7f3b600cf070 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.861+0000 7f3b65000640 1 --2- 172.21.15.60:0/1537603411 >> v2:172.21.15.100:3300/0 conn(0x7f3b600af190 0x7f3b600b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.861+0000 7f3b65000640 1 -- 172.21.15.60:0/1537603411 >> 172.21.15.60:0/1537603411 conn(0x7f3b600b1840 msgr2=0x7f3b600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.861+0000 7f3b65000640 1 -- 172.21.15.60:0/1537603411 shutdown_connections 2024-09-16T15:44:54.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.861+0000 7f3b65000640 1 -- 172.21.15.60:0/1537603411 wait complete. 2024-09-16T15:44:54.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.863+0000 7f3b65000640 1 Processor -- start 2024-09-16T15:44:54.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.863+0000 7f3b65000640 1 -- start start 2024-09-16T15:44:54.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b65000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b600af190 0x7f3b601f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b65000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 0x7f3b60144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b65000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b60161760 0x7f3b600cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b65000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b60162310 con 0x7f3b600cec60 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b65000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3b60162190 con 0x7f3b600af190 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b65000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3b60162490 con 0x7f3b60161760 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b55400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 0x7f3b60144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b55400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 0x7f3b60144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36098/0 (socket says 172.21.15.60:36098) 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b55400640 1 -- 172.21.15.60:0/2266146913 learned_addr learned my addr 172.21.15.60:0/2266146913 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b5de00640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b600af190 0x7f3b601f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.864+0000 7f3b5e800640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b60161760 0x7f3b600cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b55400640 1 -- 172.21.15.60:0/2266146913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b60161760 msgr2=0x7f3b600cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b55400640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b60161760 0x7f3b600cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b55400640 1 -- 172.21.15.60:0/2266146913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b600af190 msgr2=0x7f3b601f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b55400640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b600af190 0x7f3b601f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b55400640 1 -- 172.21.15.60:0/2266146913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b60144ff0 con 0x7f3b600cec60 2024-09-16T15:44:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b55400640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 0x7f3b60144ab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800d950 tx=0x7f3b4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b56a00640 1 -- 172.21.15.60:0/2266146913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48017070 con 0x7f3b600cec60 2024-09-16T15:44:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b56a00640 1 -- 172.21.15.60:0/2266146913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b48009d20 con 0x7f3b600cec60 2024-09-16T15:44:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b56a00640 1 -- 172.21.15.60:0/2266146913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b480106c0 con 0x7f3b600cec60 2024-09-16T15:44:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b601f4d40 con 0x7f3b600cec60 2024-09-16T15:44:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.865+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b601f5300 con 0x7f3b600cec60 2024-09-16T15:44:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.866+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b60162980 con 0x7f3b600cec60 2024-09-16T15:44:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.868+0000 7f3b56a00640 1 -- 172.21.15.60:0/2266146913 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f3b48010860 con 0x7f3b600cec60 2024-09-16T15:44:54.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.869+0000 7f3b56a00640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b3c078bb0 0x7f3b3c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.869+0000 7f3b56a00640 1 -- 172.21.15.60:0/2266146913 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f3b4809b3b0 con 0x7f3b600cec60 2024-09-16T15:44:54.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.869+0000 7f3b5de00640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b3c078bb0 0x7f3b3c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.869+0000 7f3b5de00640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b3c078bb0 0x7f3b3c07b070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b50006fd0 tx=0x7f3b50009060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:54.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:54.873+0000 7f3b56a00640 1 -- 172.21.15.60:0/2266146913 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b48062dc0 con 0x7f3b600cec60 2024-09-16T15:44:55.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.097+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f3b60146b00 con 0x7f3b3c078bb0 2024-09-16T15:44:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.104+0000 7f3b56a00640 1 -- 172.21.15.60:0/2266146913 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7f3b60146b00 con 0x7f3b3c078bb0 2024-09-16T15:44:55.105 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5 2024-09-16T15:44:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b3c078bb0 msgr2=0x7f3b3c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b3c078bb0 0x7f3b3c07b070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b50006fd0 tx=0x7f3b50009060 comp rx=0 tx=0).stop 2024-09-16T15:44:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 msgr2=0x7f3b60144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 0x7f3b60144ab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800d950 tx=0x7f3b4800de10 comp rx=0 tx=0).stop 2024-09-16T15:44:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 shutdown_connections 2024-09-16T15:44:55.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b3c078bb0 0x7f3b3c07b070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b60161760 0x7f3b600cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b600cec60 0x7f3b60144ab0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 --2- 172.21.15.60:0/2266146913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b600af190 0x7f3b601f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 >> 172.21.15.60:0/2266146913 conn(0x7f3b600b1840 msgr2=0x7f3b600b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:55.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 shutdown_connections 2024-09-16T15:44:55.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.107+0000 7f3b65000640 1 -- 172.21.15.60:0/2266146913 wait complete. 2024-09-16T15:44:55.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:55.124 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:55.163 INFO:teuthology.orchestra.run.smithi060.stdout:9 2024-09-16T15:44:55.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:55.164 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5) 2024-09-16T15:44:55.241 INFO:teuthology.orchestra.run.smithi060.stdout:9 2024-09-16T15:44:55.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone94430 2024-09-16T15:44:55.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.445+0000 7f6953000640 1 -- 172.21.15.60:0/4059749550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6930003a50 msgr2=0x7f6930003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.445+0000 7f6953000640 1 --2- 172.21.15.60:0/4059749550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6930003a50 0x7f6930003e70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f693c00c290 tx=0x7f6930005830 comp rx=0 tx=0).stop 2024-09-16T15:44:55.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.445+0000 7f6951000640 1 -- 172.21.15.60:0/4059749550 reap_dead start 2024-09-16T15:44:55.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.445+0000 7f6953000640 1 -- 172.21.15.60:0/4059749550 shutdown_connections 2024-09-16T15:44:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.445+0000 7f6953000640 1 -- 172.21.15.60:0/4059749550 >> 172.21.15.60:0/4059749550 conn(0x7f694c0b1840 msgr2=0x7f694c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.445+0000 7f6953000640 1 -- 172.21.15.60:0/4059749550 shutdown_connections 2024-09-16T15:44:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.445+0000 7f6953000640 1 -- 172.21.15.60:0/4059749550 wait complete. 2024-09-16T15:44:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.446+0000 7f6953000640 1 Processor -- start 2024-09-16T15:44:55.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.447+0000 7f6953000640 1 -- start start 2024-09-16T15:44:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.447+0000 7f6953000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6930003a50 0x7f694c155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.447+0000 7f6953000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c148660 0x7f694c15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.447+0000 7f6953000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f694c145200 0x7f694c073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.447+0000 7f6953000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f694c0b2ed0 con 0x7f694c148660 2024-09-16T15:44:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.447+0000 7f6953000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f694c0b2d50 con 0x7f694c145200 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.447+0000 7f6953000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f694c0b3050 con 0x7f6930003a50 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c148660 0x7f694c15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c148660 0x7f694c15dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36118/0 (socket says 172.21.15.60:36118) 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 -- 172.21.15.60:0/1749171309 learned_addr learned my addr 172.21.15.60:0/1749171309 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f6951000640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f694c145200 0x7f694c073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694be00640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6930003a50 0x7f694c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 -- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6930003a50 msgr2=0x7f694c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6930003a50 0x7f694c155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 -- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f694c145200 msgr2=0x7f694c073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f694c145200 0x7f694c073990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 -- 172.21.15.60:0/1749171309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c148ae0 con 0x7f694c148660 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f6951000640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f694c145200 0x7f694c073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.448+0000 7f694b400640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c148660 0x7f694c15dc20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6938011870 tx=0x7f6938011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.449+0000 7f6943e00640 1 -- 172.21.15.60:0/1749171309 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693801f070 con 0x7f694c148660 2024-09-16T15:44:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.449+0000 7f6943e00640 1 -- 172.21.15.60:0/1749171309 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f693800fe40 con 0x7f694c148660 2024-09-16T15:44:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.449+0000 7f6943e00640 1 -- 172.21.15.60:0/1749171309 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69380126b0 con 0x7f694c148660 2024-09-16T15:44:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.449+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694c15e450 con 0x7f694c148660 2024-09-16T15:44:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.449+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694c15ea10 con 0x7f694c148660 2024-09-16T15:44:55.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.450+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694c15f770 con 0x7f694c148660 2024-09-16T15:44:55.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.451+0000 7f6943e00640 1 -- 172.21.15.60:0/1749171309 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f6938022070 con 0x7f694c148660 2024-09-16T15:44:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.452+0000 7f6943e00640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f692c078be0 0x7f692c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.452+0000 7f6943e00640 1 -- 172.21.15.60:0/1749171309 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f693809ba20 con 0x7f694c148660 2024-09-16T15:44:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.452+0000 7f694be00640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f692c078be0 0x7f692c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.453+0000 7f694be00640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f692c078be0 0x7f692c07b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f693c033000 tx=0x7f694c0750b0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:55.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.456+0000 7f6943e00640 1 -- 172.21.15.60:0/1749171309 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f694c15f770 con 0x7f694c148660 2024-09-16T15:44:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.684+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f694c0ae6e0 con 0x7f692c078be0 2024-09-16T15:44:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.694+0000 7f6943e00640 1 -- 172.21.15.60:0/1749171309 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+743 (secure 0 0 0) 0x7f694c0ae6e0 con 0x7f692c078be0 2024-09-16T15:44:55.695 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "atime": "2024-09-16 15:44:10", 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_pcent": "80.00", 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_quota": 10485760, 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "bytes_used": 8388608, 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "created_at": "2024-09-16 15:44:19", 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "ctime": "2024-09-16 15:44:49", 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "data_pool": "cephfs_data", 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "earmark": "", 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "features": [ 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot-clone", 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot-autoprotect", 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot-retention" 2024-09-16T15:44:55.696 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "flavor": 2, 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "gid": 1000, 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": 16895, 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_addrs": [ 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6789", 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.100:6789", 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.100:6790" 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "mtime": "2024-09-16 15:44:14", 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5", 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_namespace": "", 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "complete", 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "clone", 2024-09-16T15:44:55.697 INFO:teuthology.orchestra.run.smithi060.stdout: "uid": 1000 2024-09-16T15:44:55.698 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:44:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.696+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f692c078be0 msgr2=0x7f692c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.696+0000 7f6953000640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f692c078be0 0x7f692c07b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f693c033000 tx=0x7f694c0750b0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.696+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c148660 msgr2=0x7f694c15dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.696+0000 7f6953000640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c148660 0x7f694c15dc20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6938011870 tx=0x7f6938011d30 comp rx=0 tx=0).stop 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.697+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 shutdown_connections 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.697+0000 7f6953000640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f692c078be0 0x7f692c07b0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.697+0000 7f6953000640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f694c145200 0x7f694c073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.697+0000 7f6953000640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f694c148660 0x7f694c15dc20 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.697+0000 7f6953000640 1 --2- 172.21.15.60:0/1749171309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6930003a50 0x7f694c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.697+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 >> 172.21.15.60:0/1749171309 conn(0x7f694c0b1840 msgr2=0x7f694c0727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.697+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 shutdown_connections 2024-09-16T15:44:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:44:55.697+0000 7f6953000640 1 -- 172.21.15.60:0/1749171309 wait complete. 2024-09-16T15:44:55.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:55.713 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:55.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:44:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910: ceph.quota.max_bytes: No such attribute 2024-09-16T15:44:55.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:55.750 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:55.827 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:44:55.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:55.827 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:55.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:55.903 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.2 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.5 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.4 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.7 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.6 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.0 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.3 2024-09-16T15:44:55.982 INFO:teuthology.orchestra.run.smithi060.stdout:volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.1 2024-09-16T15:44:55.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:55.983 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:56.058 INFO:teuthology.orchestra.run.smithi060.stdout:41ff 2024-09-16T15:44:56.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.059 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/) 2024-09-16T15:44:56.134 INFO:teuthology.orchestra.run.smithi060.stdout:41ff 2024-09-16T15:44:56.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.135 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:56.210 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:56.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.211 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/) 2024-09-16T15:44:56.286 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:56.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.288 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:56.362 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:56.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.363 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/) 2024-09-16T15:44:56.440 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:56.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.441 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910) 2024-09-16T15:44:56.516 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:56.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.517 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/) 2024-09-16T15:44:56.592 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:56.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.593 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.2) 2024-09-16T15:44:56.667 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:56.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.668 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.2) 2024-09-16T15:44:56.745 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:56.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.746 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.2) 2024-09-16T15:44:56.822 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:56.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.823 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.2) 2024-09-16T15:44:56.898 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:56.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.899 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.2) 2024-09-16T15:44:56.974 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:56.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.975 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.2) 2024-09-16T15:44:57.050 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:57.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.051 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.2) 2024-09-16T15:44:57.126 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:57.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.127 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.2) 2024-09-16T15:44:57.203 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:57.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.204 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.5) 2024-09-16T15:44:57.280 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:57.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.281 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.5) 2024-09-16T15:44:57.356 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:57.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.358 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.5) 2024-09-16T15:44:57.432 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:57.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.433 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.5) 2024-09-16T15:44:57.508 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:57.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.509 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.5) 2024-09-16T15:44:57.586 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:57.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.587 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.5) 2024-09-16T15:44:57.661 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:57.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.662 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.5) 2024-09-16T15:44:57.737 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:57.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.739 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.5) 2024-09-16T15:44:57.814 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:57.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.815 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.4) 2024-09-16T15:44:57.889 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:57.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.890 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.4) 2024-09-16T15:44:57.967 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:57.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.968 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.4) 2024-09-16T15:44:58.042 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:58.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.043 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.4) 2024-09-16T15:44:58.119 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:58.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.120 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.4) 2024-09-16T15:44:58.195 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:58.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.196 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.4) 2024-09-16T15:44:58.272 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:58.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.273 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.4) 2024-09-16T15:44:58.348 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:58.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.350 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.4) 2024-09-16T15:44:58.424 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:58.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.425 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.7) 2024-09-16T15:44:58.500 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:58.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.502 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.7) 2024-09-16T15:44:58.577 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:58.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.578 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.7) 2024-09-16T15:44:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:58.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.654 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.7) 2024-09-16T15:44:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:58.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.731 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.7) 2024-09-16T15:44:58.805 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:58.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.806 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.7) 2024-09-16T15:44:58.881 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:58.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.882 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.7) 2024-09-16T15:44:58.956 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:58.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:58.957 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.7) 2024-09-16T15:44:59.033 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:59.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.034 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.6) 2024-09-16T15:44:59.108 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:59.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.109 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.6) 2024-09-16T15:44:59.189 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:59.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.190 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.6) 2024-09-16T15:44:59.265 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:59.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.266 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.6) 2024-09-16T15:44:59.341 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:59.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.342 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.6) 2024-09-16T15:44:59.416 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:59.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.417 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.6) 2024-09-16T15:44:59.493 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:44:59.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.494 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.6) 2024-09-16T15:44:59.568 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:59.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.569 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.6) 2024-09-16T15:44:59.645 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:44:59.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.646 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.0) 2024-09-16T15:44:59.721 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.722 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.0) 2024-09-16T15:44:59.798 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:44:59.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.799 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.0) 2024-09-16T15:44:59.874 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:59.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.875 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.0) 2024-09-16T15:44:59.950 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:44:59.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:59.951 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.0) 2024-09-16T15:45:00.026 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:45:00.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.027 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.0) 2024-09-16T15:45:00.102 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.103 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.0) 2024-09-16T15:45:00.178 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:45:00.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.179 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.0) 2024-09-16T15:45:00.254 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:45:00.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.255 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.3) 2024-09-16T15:45:00.330 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:45:00.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.331 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.3) 2024-09-16T15:45:00.407 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:45:00.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.408 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.3) 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:45:00.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.485 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.3) 2024-09-16T15:45:00.560 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:45:00.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.561 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.3) 2024-09-16T15:45:00.636 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:45:00.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.637 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.3) 2024-09-16T15:45:00.712 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:45:00.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.713 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.3) 2024-09-16T15:45:00.787 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:45:00.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.788 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.3) 2024-09-16T15:45:00.863 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:45:00.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.865 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.1) 2024-09-16T15:45:00.939 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:45:00.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:00.940 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.1) 2024-09-16T15:45:01.016 INFO:teuthology.orchestra.run.smithi060.stdout:81a4 2024-09-16T15:45:01.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:01.018 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.1) 2024-09-16T15:45:01.092 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:45:01.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:01.093 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.1) 2024-09-16T15:45:01.168 INFO:teuthology.orchestra.run.smithi060.stdout:1000 2024-09-16T15:45:01.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:01.169 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.1) 2024-09-16T15:45:01.243 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:45:01.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:01.244 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.1) 2024-09-16T15:45:01.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:45:01.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:45:01.320 INFO:teuthology.orchestra.run.smithi060.stdout:1333 2024-09-16T15:45:01.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:01.321 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177/.snap/subvol_snap57910/subvolume_file.1) 2024-09-16T15:45:01.395 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:45:01.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:01.396 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94430/9c3cbc9d-1068-4cc5-856f-31528cce55b5/subvolume_file.1) 2024-09-16T15:45:01.472 INFO:teuthology.orchestra.run.smithi060.stdout:1726501454 2024-09-16T15:45:01.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol62870 fake subvol_clone86371 2024-09-16T15:45:01.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.679+0000 7fe4af600640 1 -- 172.21.15.60:0/2594125582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 msgr2=0x7fe4a8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:01.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.679+0000 7fe4af600640 1 --2- 172.21.15.60:0/2594125582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 0x7fe4a8073440 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe490009830 tx=0x7fe49002a480 comp rx=0 tx=0).stop 2024-09-16T15:45:01.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.679+0000 7fe4af600640 1 -- 172.21.15.60:0/2594125582 shutdown_connections 2024-09-16T15:45:01.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.679+0000 7fe4af600640 1 --2- 172.21.15.60:0/2594125582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4a806efa0 0x7fe4a80cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.679+0000 7fe4af600640 1 --2- 172.21.15.60:0/2594125582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 0x7fe4a8073440 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.679+0000 7fe4af600640 1 -- 172.21.15.60:0/2594125582 >> 172.21.15.60:0/2594125582 conn(0x7fe4a80b1840 msgr2=0x7fe4a80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:01.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.679+0000 7fe4af600640 1 -- 172.21.15.60:0/2594125582 shutdown_connections 2024-09-16T15:45:01.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.679+0000 7fe4af600640 1 -- 172.21.15.60:0/2594125582 wait complete. 2024-09-16T15:45:01.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.680+0000 7fe4af600640 1 Processor -- start 2024-09-16T15:45:01.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.681+0000 7fe4af600640 1 -- start start 2024-09-16T15:45:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.681+0000 7fe4af600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4a806efa0 0x7fe4a8142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.681+0000 7fe4af600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 0x7fe4a80c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.681+0000 7fe4af600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a8145750 0x7fe4a80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.681+0000 7fe4af600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a80ce480 con 0x7fe4a80cccd0 2024-09-16T15:45:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe4af600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a80ce300 con 0x7fe4a8145750 2024-09-16T15:45:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe4af600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a80ce600 con 0x7fe4a806efa0 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe49fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 0x7fe4a80c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe49fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 0x7fe4a80c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48082/0 (socket says 172.21.15.60:48082) 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe49fe00640 1 -- 172.21.15.60:0/1784273496 learned_addr learned my addr 172.21.15.60:0/1784273496 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe4ad600640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a8145750 0x7fe4a80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe4acc00640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4a806efa0 0x7fe4a8142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe49fe00640 1 -- 172.21.15.60:0/1784273496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4a806efa0 msgr2=0x7fe4a8142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe49fe00640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4a806efa0 0x7fe4a8142bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe49fe00640 1 -- 172.21.15.60:0/1784273496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a8145750 msgr2=0x7fe4a80b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe49fe00640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a8145750 0x7fe4a80b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.682+0000 7fe49fe00640 1 -- 172.21.15.60:0/1784273496 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a80b3a20 con 0x7fe4a80cccd0 2024-09-16T15:45:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.683+0000 7fe49fe00640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 0x7fe4a80c5690 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe490009950 tx=0x7fe490032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.683+0000 7fe4a5600640 1 -- 172.21.15.60:0/1784273496 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490040070 con 0x7fe4a80cccd0 2024-09-16T15:45:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.683+0000 7fe4a5600640 1 -- 172.21.15.60:0/1784273496 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe490005310 con 0x7fe4a80cccd0 2024-09-16T15:45:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.683+0000 7fe4a5600640 1 -- 172.21.15.60:0/1784273496 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4900047a0 con 0x7fe4a80cccd0 2024-09-16T15:45:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.683+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a80b3cc0 con 0x7fe4a80cccd0 2024-09-16T15:45:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.683+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a80b4280 con 0x7fe4a80cccd0 2024-09-16T15:45:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.684+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a80cdbb0 con 0x7fe4a80cccd0 2024-09-16T15:45:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.685+0000 7fe4a5600640 1 -- 172.21.15.60:0/1784273496 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fe490039040 con 0x7fe4a80cccd0 2024-09-16T15:45:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.686+0000 7fe4a5600640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe484078bb0 0x7fe48407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.686+0000 7fe4a5600640 1 -- 172.21.15.60:0/1784273496 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fe4900bc0a0 con 0x7fe4a80cccd0 2024-09-16T15:45:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.687+0000 7fe4acc00640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe484078bb0 0x7fe48407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:01.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.687+0000 7fe4acc00640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe484078bb0 0x7fe48407b070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe498001720 tx=0x7fe498007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.691+0000 7fe4a5600640 1 -- 172.21.15.60:0/1784273496 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe490083bb0 con 0x7fe4a80cccd0 2024-09-16T15:45:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:01.923+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol62870", "snap_name": "fake", "target_sub_name": "subvol_clone86371", "target": ["mon-mgr", ""]}) -- 0x7fe4a80cfde0 con 0x7fe484078bb0 2024-09-16T15:45:02.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.006+0000 7fe4a5600640 1 -- 172.21.15.60:0/1784273496 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4a80cfde0 con 0x7fe484078bb0 2024-09-16T15:45:02.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe484078bb0 msgr2=0x7fe48407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:02.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe484078bb0 0x7fe48407b070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe498001720 tx=0x7fe498007e90 comp rx=0 tx=0).stop 2024-09-16T15:45:02.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 msgr2=0x7fe4a80c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:02.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 0x7fe4a80c5690 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe490009950 tx=0x7fe490032c10 comp rx=0 tx=0).stop 2024-09-16T15:45:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 shutdown_connections 2024-09-16T15:45:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe484078bb0 0x7fe48407b070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a8145750 0x7fe4a80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4a80cccd0 0x7fe4a80c5690 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 --2- 172.21.15.60:0/1784273496 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4a806efa0 0x7fe4a8142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 >> 172.21.15.60:0/1784273496 conn(0x7fe4a80b1840 msgr2=0x7fe4a815f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 shutdown_connections 2024-09-16T15:45:02.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.009+0000 7fe4af600640 1 -- 172.21.15.60:0/1784273496 wait complete. 2024-09-16T15:45:02.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone86371 2024-09-16T15:45:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.191+0000 7f4c3fc00640 1 -- 172.21.15.60:0/1130723332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c38162330 msgr2=0x7f4c38168f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.191+0000 7f4c3fc00640 1 --2- 172.21.15.60:0/1130723332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c38162330 0x7f4c38168f50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009be0 tx=0x7f4c3002a920 comp rx=0 tx=0).stop 2024-09-16T15:45:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.191+0000 7f4c3fc00640 1 -- 172.21.15.60:0/1130723332 shutdown_connections 2024-09-16T15:45:02.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.191+0000 7f4c3fc00640 1 --2- 172.21.15.60:0/1130723332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c38162330 0x7f4c38168f50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.191+0000 7f4c3fc00640 1 -- 172.21.15.60:0/1130723332 >> 172.21.15.60:0/1130723332 conn(0x7f4c380d1e20 msgr2=0x7f4c380d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.191+0000 7f4c3fc00640 1 -- 172.21.15.60:0/1130723332 shutdown_connections 2024-09-16T15:45:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.191+0000 7f4c3fc00640 1 -- 172.21.15.60:0/1130723332 wait complete. 2024-09-16T15:45:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.192+0000 7f4c3fc00640 1 Processor -- start 2024-09-16T15:45:02.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.193+0000 7f4c3fc00640 1 -- start start 2024-09-16T15:45:02.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.193+0000 7f4c3fc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c3806ed00 0x7f4c38150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:02.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.193+0000 7f4c3fc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c3815e250 0x7f4c380b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:02.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.193+0000 7f4c3fc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c38162330 0x7f4c380b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:02.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.193+0000 7f4c3fc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38169610 con 0x7f4c38162330 2024-09-16T15:45:02.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.193+0000 7f4c3fc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38169490 con 0x7f4c3806ed00 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.193+0000 7f4c3fc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4c38169790 con 0x7f4c3815e250 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c38162330 0x7f4c380b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c38162330 0x7f4c380b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48084/0 (socket says 172.21.15.60:48084) 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3dc00640 1 -- 172.21.15.60:0/2267033194 learned_addr learned my addr 172.21.15.60:0/2267033194 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3dc00640 1 -- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c3815e250 msgr2=0x7f4c380b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c37e00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c3815e250 0x7f4c380b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3dc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c3815e250 0x7f4c380b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3dc00640 1 -- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c3806ed00 msgr2=0x7f4c38150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3d200640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c3806ed00 0x7f4c38150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3dc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c3806ed00 0x7f4c38150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3dc00640 1 -- 172.21.15.60:0/2267033194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c380c83a0 con 0x7f4c38162330 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c37e00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c3815e250 0x7f4c380b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:45:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.194+0000 7f4c3d200640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c3806ed00 0x7f4c38150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:45:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.195+0000 7f4c3dc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c38162330 0x7f4c380b18f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009be0 tx=0x7f4c30003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.195+0000 7f4c34c00640 1 -- 172.21.15.60:0/2267033194 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30030c50 con 0x7f4c38162330 2024-09-16T15:45:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.195+0000 7f4c34c00640 1 -- 172.21.15.60:0/2267033194 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c3003e070 con 0x7f4c38162330 2024-09-16T15:45:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.195+0000 7f4c34c00640 1 -- 172.21.15.60:0/2267033194 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c300042a0 con 0x7f4c38162330 2024-09-16T15:45:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.195+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c380c8680 con 0x7f4c38162330 2024-09-16T15:45:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.195+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c38151170 con 0x7f4c38162330 2024-09-16T15:45:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.196+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c38169250 con 0x7f4c38162330 2024-09-16T15:45:02.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.197+0000 7f4c34c00640 1 -- 172.21.15.60:0/2267033194 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f4c30004440 con 0x7f4c38162330 2024-09-16T15:45:02.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.199+0000 7f4c34c00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4c18078bb0 0x7f4c1807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:02.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.199+0000 7f4c34c00640 1 -- 172.21.15.60:0/2267033194 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f4c300bc800 con 0x7f4c38162330 2024-09-16T15:45:02.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.199+0000 7f4c3d200640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4c18078bb0 0x7f4c1807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:02.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.199+0000 7f4c3d200640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4c18078bb0 0x7f4c1807b070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c001220 tx=0x7f4c2c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:02.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.203+0000 7f4c34c00640 1 -- 172.21.15.60:0/2267033194 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4c3002d070 con 0x7f4c38162330 2024-09-16T15:45:02.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:02.438+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone86371", "target": ["mon-mgr", ""]}) -- 0x7f4c38069630 con 0x7f4c18078bb0 2024-09-16T15:45:04.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.182+0000 7f4c34c00640 1 -- 172.21.15.60:0/2267033194 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7f4c38069630 con 0x7f4c18078bb0 2024-09-16T15:45:04.184 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:45:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:45:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "pending", 2024-09-16T15:45:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:45:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "cephfs", 2024-09-16T15:45:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "subvol62870", 2024-09-16T15:45:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "fake" 2024-09-16T15:45:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:04.185 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.185+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4c18078bb0 msgr2=0x7f4c1807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.185+0000 7f4c3fc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4c18078bb0 0x7f4c1807b070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c001220 tx=0x7f4c2c009040 comp rx=0 tx=0).stop 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.185+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c38162330 msgr2=0x7f4c380b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.185+0000 7f4c3fc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c38162330 0x7f4c380b18f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009be0 tx=0x7f4c30003ba0 comp rx=0 tx=0).stop 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.186+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 shutdown_connections 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.186+0000 7f4c3fc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4c18078bb0 0x7f4c1807b070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.186+0000 7f4c3fc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c38162330 0x7f4c380b18f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.186+0000 7f4c3fc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c3815e250 0x7f4c380b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:04.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.186+0000 7f4c3fc00640 1 --2- 172.21.15.60:0/2267033194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c3806ed00 0x7f4c38150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.186+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 >> 172.21.15.60:0/2267033194 conn(0x7f4c380d1e20 msgr2=0x7f4c38072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.186+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 shutdown_connections 2024-09-16T15:45:04.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:04.186+0000 7f4c3fc00640 1 -- 172.21.15.60:0/2267033194 wait complete. 2024-09-16T15:45:04.210 DEBUG:tasks.cephfs.test_volumes:current clone state = pending 2024-09-16T15:45:05.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone86371 2024-09-16T15:45:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.387+0000 7f9d59400640 1 -- 172.21.15.60:0/2317043199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d540af190 msgr2=0x7f9d540b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.387+0000 7f9d59400640 1 --2- 172.21.15.60:0/2317043199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d540af190 0x7f9d540b2b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009830 tx=0x7f9d4002a410 comp rx=0 tx=0).stop 2024-09-16T15:45:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.387+0000 7f9d59400640 1 -- 172.21.15.60:0/2317043199 shutdown_connections 2024-09-16T15:45:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.387+0000 7f9d59400640 1 --2- 172.21.15.60:0/2317043199 >> v2:172.21.15.100:3300/0 conn(0x7f9d540cec60 0x7f9d540cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.387+0000 7f9d59400640 1 --2- 172.21.15.60:0/2317043199 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d540af190 0x7f9d540b2b40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.387+0000 7f9d59400640 1 -- 172.21.15.60:0/2317043199 >> 172.21.15.60:0/2317043199 conn(0x7f9d540b1840 msgr2=0x7f9d540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.387+0000 7f9d59400640 1 -- 172.21.15.60:0/2317043199 shutdown_connections 2024-09-16T15:45:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.387+0000 7f9d59400640 1 -- 172.21.15.60:0/2317043199 wait complete. 2024-09-16T15:45:05.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.389+0000 7f9d59400640 1 Processor -- start 2024-09-16T15:45:05.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.389+0000 7f9d59400640 1 -- start start 2024-09-16T15:45:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d59400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d540af190 0x7f9d5415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d59400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d540cec60 0x7f9d54072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d59400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d54161760 0x7f9d540b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d59400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54162310 con 0x7f9d540af190 2024-09-16T15:45:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d59400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54162190 con 0x7f9d540cec60 2024-09-16T15:45:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d59400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9d54162490 con 0x7f9d54161760 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d52800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d540af190 0x7f9d5415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d52800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d540af190 0x7f9d5415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48096/0 (socket says 172.21.15.60:48096) 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d52800640 1 -- 172.21.15.60:0/2885511425 learned_addr learned my addr 172.21.15.60:0/2885511425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d51e00640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d540cec60 0x7f9d54072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.390+0000 7f9d53200640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d54161760 0x7f9d540b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d52800640 1 -- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d54161760 msgr2=0x7f9d540b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d52800640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d54161760 0x7f9d540b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d52800640 1 -- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d540cec60 msgr2=0x7f9d54072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d52800640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d540cec60 0x7f9d54072d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d52800640 1 -- 172.21.15.60:0/2885511425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d541f49a0 con 0x7f9d540af190 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d51e00640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d540cec60 0x7f9d54072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d52800640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d540af190 0x7f9d5415ad80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c500 tx=0x7f9d4400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d4aa00640 1 -- 172.21.15.60:0/2885511425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44016020 con 0x7f9d540af190 2024-09-16T15:45:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d4aa00640 1 -- 172.21.15.60:0/2885511425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d44012940 con 0x7f9d540af190 2024-09-16T15:45:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d541f4c80 con 0x7f9d540af190 2024-09-16T15:45:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d4aa00640 1 -- 172.21.15.60:0/2885511425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44012d90 con 0x7f9d540af190 2024-09-16T15:45:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.391+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541f5240 con 0x7f9d540af190 2024-09-16T15:45:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.392+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d54162970 con 0x7f9d540af190 2024-09-16T15:45:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.394+0000 7f9d4aa00640 1 -- 172.21.15.60:0/2885511425 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f9d44019020 con 0x7f9d540af190 2024-09-16T15:45:05.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.394+0000 7f9d4aa00640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d34078be0 0x7f9d3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:05.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.395+0000 7f9d4aa00640 1 -- 172.21.15.60:0/2885511425 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f9d4409b0a0 con 0x7f9d540af190 2024-09-16T15:45:05.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.395+0000 7f9d51e00640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d34078be0 0x7f9d3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:05.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.396+0000 7f9d51e00640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d34078be0 0x7f9d3407b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d40034000 tx=0x7f9d4002c040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:05.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.399+0000 7f9d4aa00640 1 -- 172.21.15.60:0/2885511425 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d44020020 con 0x7f9d540af190 2024-09-16T15:45:05.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:05.636+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone86371", "target": ["mon-mgr", ""]}) -- 0x7f9d541f5480 con 0x7f9d34078be0 2024-09-16T15:45:09.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.169+0000 7f9d4aa00640 1 -- 172.21.15.60:0/2885511425 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f9d4003d470 con 0x7f9d34078be0 2024-09-16T15:45:09.170 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:45:09.170 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:45:09.170 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "complete" 2024-09-16T15:45:09.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:09.171 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:45:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d34078be0 msgr2=0x7f9d3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d34078be0 0x7f9d3407b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9d40034000 tx=0x7f9d4002c040 comp rx=0 tx=0).stop 2024-09-16T15:45:09.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d540af190 msgr2=0x7f9d5415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d540af190 0x7f9d5415ad80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c500 tx=0x7f9d4400c9c0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 shutdown_connections 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d34078be0 0x7f9d3407b0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d54161760 0x7f9d540b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d540cec60 0x7f9d54072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 --2- 172.21.15.60:0/2885511425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d540af190 0x7f9d5415ad80 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 >> 172.21.15.60:0/2885511425 conn(0x7f9d540b1840 msgr2=0x7f9d540b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 shutdown_connections 2024-09-16T15:45:09.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.172+0000 7f9d59400640 1 -- 172.21.15.60:0/2885511425 wait complete. 2024-09-16T15:45:09.192 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-16T15:45:09.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:09.192 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone86371/.meta) 2024-09-16T15:45:09.253 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:45:09.253 INFO:teuthology.orchestra.run.smithi060.stdout:version = 2 2024-09-16T15:45:09.254 INFO:teuthology.orchestra.run.smithi060.stdout:type = clone 2024-09-16T15:45:09.254 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/_nogroup/subvol_clone86371/af3d1985-acd8-4555-abe3-927c40b962e8 2024-09-16T15:45:09.254 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:45:09.254 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:09.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol62870 subvol_snap57910 2024-09-16T15:45:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.423+0000 7f4ecb800640 1 -- 172.21.15.60:0/1380832018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec4158900 msgr2=0x7f4ec415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.423+0000 7f4ecb800640 1 --2- 172.21.15.60:0/1380832018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec4158900 0x7f4ec415b1c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4005fd0 tx=0x7f4eb402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.423+0000 7f4ecb800640 1 -- 172.21.15.60:0/1380832018 shutdown_connections 2024-09-16T15:45:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.423+0000 7f4ecb800640 1 --2- 172.21.15.60:0/1380832018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ec406e6f0 0x7f4ec406eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.423+0000 7f4ecb800640 1 --2- 172.21.15.60:0/1380832018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec4158900 0x7f4ec415b1c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.423+0000 7f4ecb800640 1 -- 172.21.15.60:0/1380832018 >> 172.21.15.60:0/1380832018 conn(0x7f4ec40b18e0 msgr2=0x7f4ec40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.423+0000 7f4ecb800640 1 -- 172.21.15.60:0/1380832018 shutdown_connections 2024-09-16T15:45:09.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.423+0000 7f4ecb800640 1 -- 172.21.15.60:0/1380832018 wait complete. 2024-09-16T15:45:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.425+0000 7f4ecb800640 1 Processor -- start 2024-09-16T15:45:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.425+0000 7f4ecb800640 1 -- start start 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ecb800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec406e6f0 0x7f4ec414c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ecb800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4158900 0x7f4ec409d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ecb800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ec415b700 0x7f4ec409d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ecb800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40ced60 con 0x7f4ec406e6f0 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ecb800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40cebe0 con 0x7f4ec4158900 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ecb800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec40ceee0 con 0x7f4ec415b700 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ec8e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec406e6f0 0x7f4ec414c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ec8e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec406e6f0 0x7f4ec414c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42982/0 (socket says 172.21.15.60:42982) 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ec8e00640 1 -- 172.21.15.60:0/600508188 learned_addr learned my addr 172.21.15.60:0/600508188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:09.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ebbe00640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4158900 0x7f4ec409d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.426+0000 7f4ec9800640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ec415b700 0x7f4ec409d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec8e00640 1 -- 172.21.15.60:0/600508188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ec415b700 msgr2=0x7f4ec409d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec8e00640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ec415b700 0x7f4ec409d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec8e00640 1 -- 172.21.15.60:0/600508188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4158900 msgr2=0x7f4ec409d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec8e00640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4158900 0x7f4ec409d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec8e00640 1 -- 172.21.15.60:0/600508188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec414caf0 con 0x7f4ec406e6f0 2024-09-16T15:45:09.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec8e00640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec406e6f0 0x7f4ec414c5b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009850 tx=0x7f4eb40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec1600640 1 -- 172.21.15.60:0/600508188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb403f070 con 0x7f4ec406e6f0 2024-09-16T15:45:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec1600640 1 -- 172.21.15.60:0/600508188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4eb4005600 con 0x7f4ec406e6f0 2024-09-16T15:45:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ec1600640 1 -- 172.21.15.60:0/600508188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4035030 con 0x7f4ec406e6f0 2024-09-16T15:45:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec409def0 con 0x7f4ec406e6f0 2024-09-16T15:45:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.427+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec41f84c0 con 0x7f4ec406e6f0 2024-09-16T15:45:09.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.428+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec4069360 con 0x7f4ec406e6f0 2024-09-16T15:45:09.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.429+0000 7f4ec1600640 1 -- 172.21.15.60:0/600508188 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f4eb4042070 con 0x7f4ec406e6f0 2024-09-16T15:45:09.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.431+0000 7f4ec1600640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ea0078bb0 0x7f4ea007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:09.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.431+0000 7f4ec1600640 1 -- 172.21.15.60:0/600508188 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f4eb40bbc50 con 0x7f4ec406e6f0 2024-09-16T15:45:09.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.431+0000 7f4ebbe00640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ea0078bb0 0x7f4ea007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.432+0000 7f4ebbe00640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ea0078bb0 0x7f4ea007b070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ec409ac20 tx=0x7f4eac009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.436+0000 7f4ec1600640 1 -- 172.21.15.60:0/600508188 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4eb4083db0 con 0x7f4ec406e6f0 2024-09-16T15:45:09.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.666+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol62870", "snap_name": "subvol_snap57910", "target": ["mon-mgr", ""]}) -- 0x7f4ec415a710 con 0x7f4ea0078bb0 2024-09-16T15:45:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.687+0000 7f4ec1600640 1 -- 172.21.15.60:0/600508188 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7f4ec415a710 con 0x7f4ea0078bb0 2024-09-16T15:45:09.688 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:45:09.688 INFO:teuthology.orchestra.run.smithi060.stdout: "created_at": "2024-09-16 15:44:15.381924", 2024-09-16T15:45:09.688 INFO:teuthology.orchestra.run.smithi060.stdout: "data_pool": "cephfs_data", 2024-09-16T15:45:09.688 INFO:teuthology.orchestra.run.smithi060.stdout: "has_pending_clones": "no" 2024-09-16T15:45:09.688 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ea0078bb0 msgr2=0x7f4ea007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ea0078bb0 0x7f4ea007b070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ec409ac20 tx=0x7f4eac009040 comp rx=0 tx=0).stop 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec406e6f0 msgr2=0x7f4ec414c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec406e6f0 0x7f4ec414c5b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009850 tx=0x7f4eb40038a0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 shutdown_connections 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4ea0078bb0 0x7f4ea007b070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4ec415b700 0x7f4ec409d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ec4158900 0x7f4ec409d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 --2- 172.21.15.60:0/600508188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4ec406e6f0 0x7f4ec414c5b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 >> 172.21.15.60:0/600508188 conn(0x7f4ec40b18e0 msgr2=0x7f4ec415d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 shutdown_connections 2024-09-16T15:45:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.689+0000 7f4ecb800640 1 -- 172.21.15.60:0/600508188 wait complete. 2024-09-16T15:45:09.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol62870 2024-09-16T15:45:09.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.871+0000 7f60ca000640 1 -- 172.21.15.60:0/1765859859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c4070550 msgr2=0x7f60c40cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.871+0000 7f60ca000640 1 --2- 172.21.15.60:0/1765859859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c4070550 0x7f60c40cb1c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f60bc005fd0 tx=0x7f60bc02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.871+0000 7f60ca000640 1 -- 172.21.15.60:0/1765859859 shutdown_connections 2024-09-16T15:45:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.871+0000 7f60ca000640 1 --2- 172.21.15.60:0/1765859859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c4070550 0x7f60c40cb1c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.871+0000 7f60ca000640 1 --2- 172.21.15.60:0/1765859859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60c415bda0 0x7f60c4073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.871+0000 7f60ca000640 1 -- 172.21.15.60:0/1765859859 >> 172.21.15.60:0/1765859859 conn(0x7f60c40cfa70 msgr2=0x7f60c40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.871+0000 7f60ca000640 1 -- 172.21.15.60:0/1765859859 shutdown_connections 2024-09-16T15:45:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.871+0000 7f60ca000640 1 -- 172.21.15.60:0/1765859859 wait complete. 2024-09-16T15:45:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.873+0000 7f60ca000640 1 Processor -- start 2024-09-16T15:45:09.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.873+0000 7f60ca000640 1 -- start start 2024-09-16T15:45:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60ca000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c406e590 0x7f60c415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60ca000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60c4070550 0x7f60c40adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60ca000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c415bda0 0x7f60c4152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60ca000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f60c40cba90 con 0x7f60c406e590 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60ca000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60c40cb910 con 0x7f60c415bda0 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60ca000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f60c40cbc10 con 0x7f60c4070550 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c406e590 0x7f60c415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60c8c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c406e590 0x7f60c415d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43008/0 (socket says 172.21.15.60:43008) 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.874+0000 7f60c8c00640 1 -- 172.21.15.60:0/156667284 learned_addr learned my addr 172.21.15.60:0/156667284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60bbe00640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60c4070550 0x7f60c40adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c9600640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c415bda0 0x7f60c4152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c8c00640 1 -- 172.21.15.60:0/156667284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60c4070550 msgr2=0x7f60c40adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c8c00640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60c4070550 0x7f60c40adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c8c00640 1 -- 172.21.15.60:0/156667284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c415bda0 msgr2=0x7f60c4152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c8c00640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c415bda0 0x7f60c4152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c8c00640 1 -- 172.21.15.60:0/156667284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c41f7d50 con 0x7f60c406e590 2024-09-16T15:45:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c8c00640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c406e590 0x7f60c415d350 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f60b400d770 tx=0x7f60b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c1600640 1 -- 172.21.15.60:0/156667284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4017070 con 0x7f60c406e590 2024-09-16T15:45:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c1600640 1 -- 172.21.15.60:0/156667284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60b4009d20 con 0x7f60c406e590 2024-09-16T15:45:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60c1600640 1 -- 172.21.15.60:0/156667284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b40106c0 con 0x7f60c406e590 2024-09-16T15:45:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c41f8030 con 0x7f60c406e590 2024-09-16T15:45:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.875+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c41f85f0 con 0x7f60c406e590 2024-09-16T15:45:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.876+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c40cc1a0 con 0x7f60c406e590 2024-09-16T15:45:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.878+0000 7f60c1600640 1 -- 172.21.15.60:0/156667284 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f60b401a070 con 0x7f60c406e590 2024-09-16T15:45:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.879+0000 7f60c1600640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60a0078bb0 0x7f60a007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.879+0000 7f60c1600640 1 -- 172.21.15.60:0/156667284 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f60b409b1b0 con 0x7f60c406e590 2024-09-16T15:45:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.879+0000 7f60bbe00640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60a0078bb0 0x7f60a007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.879+0000 7f60bbe00640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60a0078bb0 0x7f60a007b070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60c40ae470 tx=0x7f60ac009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:09.883+0000 7f60c1600640 1 -- 172.21.15.60:0/156667284 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f60b4063210 con 0x7f60c406e590 2024-09-16T15:45:10.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.109+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol62870", "target": ["mon-mgr", ""]}) -- 0x7f60c4073c80 con 0x7f60a0078bb0 2024-09-16T15:45:10.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.124+0000 7f60c1600640 1 -- 172.21.15.60:0/156667284 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7f60c4073c80 con 0x7f60a0078bb0 2024-09-16T15:45:10.125 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:45:10.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:10.126 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "fake" 2024-09-16T15:45:10.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:10.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:10.126 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "subvol_snap57910" 2024-09-16T15:45:10.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:10.126 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:45:10.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60a0078bb0 msgr2=0x7f60a007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60a0078bb0 0x7f60a007b070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60c40ae470 tx=0x7f60ac009040 comp rx=0 tx=0).stop 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c406e590 msgr2=0x7f60c415d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c406e590 0x7f60c415d350 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f60b400d770 tx=0x7f60b400dc30 comp rx=0 tx=0).stop 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 shutdown_connections 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60a0078bb0 0x7f60a007b070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60c415bda0 0x7f60c4152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60c4070550 0x7f60c40adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 --2- 172.21.15.60:0/156667284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60c406e590 0x7f60c415d350 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 >> 172.21.15.60:0/156667284 conn(0x7f60c40cfa70 msgr2=0x7f60c40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.126+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 shutdown_connections 2024-09-16T15:45:10.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.127+0000 7f60ca000640 1 -- 172.21.15.60:0/156667284 wait complete. 2024-09-16T15:45:10.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol62870 subvol_snap57910 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 -- 172.21.15.60:0/1723803210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0cfdf0 msgr2=0x7f86bc0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 --2- 172.21.15.60:0/1723803210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0cfdf0 0x7f86bc0d0280 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f86bc051470 tx=0x7f86b402aa30 comp rx=0 tx=0).stop 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 -- 172.21.15.60:0/1723803210 shutdown_connections 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 --2- 172.21.15.60:0/1723803210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0cfdf0 0x7f86bc0d0280 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 --2- 172.21.15.60:0/1723803210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86bc0cce90 0x7f86bc06cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 --2- 172.21.15.60:0/1723803210 >> v2:172.21.15.100:3300/0 conn(0x7f86bc0ccab0 0x7f86bc162390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 -- 172.21.15.60:0/1723803210 >> 172.21.15.60:0/1723803210 conn(0x7f86bc0d0b80 msgr2=0x7f86bc0d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 -- 172.21.15.60:0/1723803210 shutdown_connections 2024-09-16T15:45:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.308+0000 7f86bbe00640 1 -- 172.21.15.60:0/1723803210 wait complete. 2024-09-16T15:45:10.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.310+0000 7f86bbe00640 1 Processor -- start 2024-09-16T15:45:10.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.310+0000 7f86bbe00640 1 -- start start 2024-09-16T15:45:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86bbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0ccab0 0x7f86bc0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86bbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc0cce90 0x7f86bc0c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86bbe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86bc0cfdf0 0x7f86bc1f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:10.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86bbe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc073aa0 con 0x7f86bc0ccab0 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86bbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc073920 con 0x7f86bc0cce90 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86bbe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc073c20 con 0x7f86bc0cfdf0 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86baa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0ccab0 0x7f86bc0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86baa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0ccab0 0x7f86bc0b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43030/0 (socket says 172.21.15.60:43030) 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86baa00640 1 -- 172.21.15.60:0/771484782 learned_addr learned my addr 172.21.15.60:0/771484782 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86b2000640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc0cce90 0x7f86bc0c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.311+0000 7f86bb400640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86bc0cfdf0 0x7f86bc1f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86baa00640 1 -- 172.21.15.60:0/771484782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86bc0cfdf0 msgr2=0x7f86bc1f8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86baa00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86bc0cfdf0 0x7f86bc1f8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86baa00640 1 -- 172.21.15.60:0/771484782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc0cce90 msgr2=0x7f86bc0c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86baa00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc0cce90 0x7f86bc0c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86baa00640 1 -- 172.21.15.60:0/771484782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc0b46a0 con 0x7f86bc0ccab0 2024-09-16T15:45:10.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86baa00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0ccab0 0x7f86bc0b5530 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9f0 tx=0x7f86ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:10.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86b3e00640 1 -- 172.21.15.60:0/771484782 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00ccc0 con 0x7f86bc0ccab0 2024-09-16T15:45:10.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86b3e00640 1 -- 172.21.15.60:0/771484782 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86ac010430 con 0x7f86bc0ccab0 2024-09-16T15:45:10.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86b3e00640 1 -- 172.21.15.60:0/771484782 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac010840 con 0x7f86bc0ccab0 2024-09-16T15:45:10.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.312+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc0b4980 con 0x7f86bc0ccab0 2024-09-16T15:45:10.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.313+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc0b4370 con 0x7f86bc0ccab0 2024-09-16T15:45:10.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.313+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc0cfbb0 con 0x7f86bc0ccab0 2024-09-16T15:45:10.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.315+0000 7f86b3e00640 1 -- 172.21.15.60:0/771484782 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f86ac0109e0 con 0x7f86bc0ccab0 2024-09-16T15:45:10.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.316+0000 7f86b3e00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8698078bb0 0x7f869807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:10.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.316+0000 7f86b3e00640 1 -- 172.21.15.60:0/771484782 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f86ac09ba10 con 0x7f86bc0ccab0 2024-09-16T15:45:10.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.316+0000 7f86b2000640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8698078bb0 0x7f869807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:10.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.316+0000 7f86b2000640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8698078bb0 0x7f869807b070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0b0590 tx=0x7f86a4005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:10.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.320+0000 7f86b3e00640 1 -- 172.21.15.60:0/771484782 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f86ac063490 con 0x7f86bc0ccab0 2024-09-16T15:45:10.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.551+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol62870", "snap_name": "subvol_snap57910", "target": ["mon-mgr", ""]}) -- 0x7f86bc0692a0 con 0x7f8698078bb0 2024-09-16T15:45:10.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.591+0000 7f86b3e00640 1 -- 172.21.15.60:0/771484782 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86bc0692a0 con 0x7f8698078bb0 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8698078bb0 msgr2=0x7f869807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8698078bb0 0x7f869807b070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0b0590 tx=0x7f86a4005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0ccab0 msgr2=0x7f86bc0b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0ccab0 0x7f86bc0b5530 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e9f0 tx=0x7f86ac00eeb0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 shutdown_connections 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8698078bb0 0x7f869807b070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86bc0cfdf0 0x7f86bc1f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86bc0cce90 0x7f86bc0c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 --2- 172.21.15.60:0/771484782 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86bc0ccab0 0x7f86bc0b5530 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.593+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 >> 172.21.15.60:0/771484782 conn(0x7f86bc0d0b80 msgr2=0x7f86bc0a0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.594+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 shutdown_connections 2024-09-16T15:45:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.594+0000 7f86bbe00640 1 -- 172.21.15.60:0/771484782 wait complete. 2024-09-16T15:45:10.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol62870 fake 2024-09-16T15:45:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.778+0000 7fc9ff800640 1 -- 172.21.15.60:0/8269319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9dc004800 msgr2=0x7fc9dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.778+0000 7fc9ff800640 1 --2- 172.21.15.60:0/8269319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9dc004800 0x7fc9dc004c80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009b20 tx=0x7fc9dc005830 comp rx=0 tx=0).stop 2024-09-16T15:45:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.779+0000 7fc9ff800640 1 -- 172.21.15.60:0/8269319 shutdown_connections 2024-09-16T15:45:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.779+0000 7fc9ff800640 1 --2- 172.21.15.60:0/8269319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9dc004800 0x7fc9dc004c80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.779+0000 7fc9ff800640 1 -- 172.21.15.60:0/8269319 >> 172.21.15.60:0/8269319 conn(0x7fc9f80b18e0 msgr2=0x7fc9f80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.779+0000 7fc9ff800640 1 -- 172.21.15.60:0/8269319 shutdown_connections 2024-09-16T15:45:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.779+0000 7fc9ff800640 1 -- 172.21.15.60:0/8269319 wait complete. 2024-09-16T15:45:10.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.780+0000 7fc9ff800640 1 Processor -- start 2024-09-16T15:45:10.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.781+0000 7fc9ff800640 1 -- start start 2024-09-16T15:45:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.781+0000 7fc9ff800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9dc004800 0x7fc9f80cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.781+0000 7fc9ff800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9f81f9770 0x7fc9f81f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.781+0000 7fc9ff800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9f815aaf0 0x7fc9f815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9ff800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80cd5e0 con 0x7fc9f815aaf0 2024-09-16T15:45:10.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9ff800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8074240 con 0x7fc9dc004800 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9ff800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f80743c0 con 0x7fc9f81f9770 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fd800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9f815aaf0 0x7fc9f815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fd800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9f815aaf0 0x7fc9f815af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43044/0 (socket says 172.21.15.60:43044) 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fd800640 1 -- 172.21.15.60:0/1389165543 learned_addr learned my addr 172.21.15.60:0/1389165543 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fce00640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9dc004800 0x7fc9f80cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9efe00640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9f81f9770 0x7fc9f81f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fd800640 1 -- 172.21.15.60:0/1389165543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9f81f9770 msgr2=0x7fc9f81f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fd800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9f81f9770 0x7fc9f81f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fd800640 1 -- 172.21.15.60:0/1389165543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9dc004800 msgr2=0x7fc9f80cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fd800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9dc004800 0x7fc9f80cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:10.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.782+0000 7fc9fd800640 1 -- 172.21.15.60:0/1389165543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f8141fe0 con 0x7fc9f815aaf0 2024-09-16T15:45:10.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.783+0000 7fc9fd800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9f815aaf0 0x7fc9f815af80 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8070600 tx=0x7fc9f00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:10.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.783+0000 7fc9f5600640 1 -- 172.21.15.60:0/1389165543 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0015020 con 0x7fc9f815aaf0 2024-09-16T15:45:10.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.783+0000 7fc9f5600640 1 -- 172.21.15.60:0/1389165543 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9f0011960 con 0x7fc9f815aaf0 2024-09-16T15:45:10.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.783+0000 7fc9f5600640 1 -- 172.21.15.60:0/1389165543 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0011d70 con 0x7fc9f815aaf0 2024-09-16T15:45:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.783+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f81422c0 con 0x7fc9f815aaf0 2024-09-16T15:45:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.783+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f8142880 con 0x7fc9f815aaf0 2024-09-16T15:45:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.784+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f8069360 con 0x7fc9f815aaf0 2024-09-16T15:45:10.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.785+0000 7fc9f5600640 1 -- 172.21.15.60:0/1389165543 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7fc9f0018020 con 0x7fc9f815aaf0 2024-09-16T15:45:10.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.786+0000 7fc9f5600640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9d4078bb0 0x7fc9d407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:10.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.786+0000 7fc9f5600640 1 -- 172.21.15.60:0/1389165543 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fc9f009b8b0 con 0x7fc9f815aaf0 2024-09-16T15:45:10.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.787+0000 7fc9fce00640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9d4078bb0 0x7fc9d407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:10.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.787+0000 7fc9fce00640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9d4078bb0 0x7fc9d407b070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8001860 tx=0x7fc9e8006040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:10.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:10.791+0000 7fc9f5600640 1 -- 172.21.15.60:0/1389165543 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc9f001f020 con 0x7fc9f815aaf0 2024-09-16T15:45:11.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.020+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol62870", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7fc9f815ff40 con 0x7fc9d4078bb0 2024-09-16T15:45:11.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.061+0000 7fc9f5600640 1 -- 172.21.15.60:0/1389165543 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9f815ff40 con 0x7fc9d4078bb0 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9d4078bb0 msgr2=0x7fc9d407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9d4078bb0 0x7fc9d407b070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8001860 tx=0x7fc9e8006040 comp rx=0 tx=0).stop 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9f815aaf0 msgr2=0x7fc9f815af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9f815aaf0 0x7fc9f815af80 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8070600 tx=0x7fc9f00019b0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 shutdown_connections 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9d4078bb0 0x7fc9d407b070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9f815aaf0 0x7fc9f815af80 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9f81f9770 0x7fc9f81f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 --2- 172.21.15.60:0/1389165543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9dc004800 0x7fc9f80cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.063+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 >> 172.21.15.60:0/1389165543 conn(0x7fc9f80b18e0 msgr2=0x7fc9f815b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.064+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 shutdown_connections 2024-09-16T15:45:11.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.064+0000 7fc9ff800640 1 -- 172.21.15.60:0/1389165543 wait complete. 2024-09-16T15:45:11.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:11.081 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol62870/.meta) 2024-09-16T15:45:11.138 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:45:11.138 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:45:11.138 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:45:11.138 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/_nogroup/subvol62870/3f5b069b-578d-4015-ad9d-5669822f1177 2024-09-16T15:45:11.138 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:45:11.138 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:11.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol62870 2024-09-16T15:45:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.305+0000 7f1062c00640 1 -- 172.21.15.60:0/1376155757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15f360 msgr2=0x7f105c0ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.305+0000 7f1062c00640 1 --2- 172.21.15.60:0/1376155757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15f360 0x7f105c0ae5e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1048009860 tx=0x7f104802a540 comp rx=0 tx=0).stop 2024-09-16T15:45:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.305+0000 7f1062c00640 1 -- 172.21.15.60:0/1376155757 shutdown_connections 2024-09-16T15:45:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.305+0000 7f1062c00640 1 --2- 172.21.15.60:0/1376155757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c06e880 0x7f105c0739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.305+0000 7f1062c00640 1 --2- 172.21.15.60:0/1376155757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15f360 0x7f105c0ae5e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.305+0000 7f1062c00640 1 -- 172.21.15.60:0/1376155757 >> 172.21.15.60:0/1376155757 conn(0x7f105c0cf9f0 msgr2=0x7f105c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.305+0000 7f1062c00640 1 -- 172.21.15.60:0/1376155757 shutdown_connections 2024-09-16T15:45:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.305+0000 7f1062c00640 1 -- 172.21.15.60:0/1376155757 wait complete. 2024-09-16T15:45:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.306+0000 7f1062c00640 1 Processor -- start 2024-09-16T15:45:11.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.307+0000 7f1062c00640 1 -- start start 2024-09-16T15:45:11.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.307+0000 7f1062c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c06e880 0x7f105c0c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:11.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.307+0000 7f1062c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15e870 0x7f105c15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:11.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1062c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f360 0x7f105c14fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:11.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1062c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f105c074060 con 0x7f105c15e870 2024-09-16T15:45:11.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1062c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f105c073ee0 con 0x7f105c06e880 2024-09-16T15:45:11.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1062c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f105c0741e0 con 0x7f105c15f360 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1060e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15e870 0x7f105c15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1060e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15e870 0x7f105c15d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43064/0 (socket says 172.21.15.60:43064) 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1060e00640 1 -- 172.21.15.60:0/3107411273 learned_addr learned my addr 172.21.15.60:0/3107411273 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1062200640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f360 0x7f105c14fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1061800640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c06e880 0x7f105c0c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1060e00640 1 -- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f360 msgr2=0x7f105c14fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1060e00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f360 0x7f105c14fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1060e00640 1 -- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c06e880 msgr2=0x7f105c0c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1060e00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c06e880 0x7f105c0c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1060e00640 1 -- 172.21.15.60:0/3107411273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105c0c8400 con 0x7f105c15e870 2024-09-16T15:45:11.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.308+0000 7f1062200640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f360 0x7f105c14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.309+0000 7f1060e00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15e870 0x7f105c15d1a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1048009780 tx=0x7f1048033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.309+0000 7f1059600640 1 -- 172.21.15.60:0/3107411273 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048031d60 con 0x7f105c15e870 2024-09-16T15:45:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.309+0000 7f1059600640 1 -- 172.21.15.60:0/3107411273 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1048033e10 con 0x7f105c15e870 2024-09-16T15:45:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.309+0000 7f1059600640 1 -- 172.21.15.60:0/3107411273 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048005260 con 0x7f105c15e870 2024-09-16T15:45:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.309+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105c0c7a20 con 0x7f105c15e870 2024-09-16T15:45:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.309+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105c0c6f40 con 0x7f105c15e870 2024-09-16T15:45:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.310+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105c0739a0 con 0x7f105c15e870 2024-09-16T15:45:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.311+0000 7f1059600640 1 -- 172.21.15.60:0/3107411273 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f1048005400 con 0x7f105c15e870 2024-09-16T15:45:11.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.312+0000 7f1059600640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c078bb0 0x7f103c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:11.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.312+0000 7f1059600640 1 -- 172.21.15.60:0/3107411273 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f10480bc100 con 0x7f105c15e870 2024-09-16T15:45:11.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.312+0000 7f1061800640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c078bb0 0x7f103c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:11.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.313+0000 7f1061800640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c078bb0 0x7f103c07b070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f105c0b59c0 tx=0x7f1050007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.317+0000 7f1059600640 1 -- 172.21.15.60:0/3107411273 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f105c0c8400 con 0x7f105c15e870 2024-09-16T15:45:11.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.543+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol62870", "target": ["mon-mgr", ""]}) -- 0x7f105c0d0710 con 0x7f103c078bb0 2024-09-16T15:45:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.597+0000 7f1059600640 1 -- 172.21.15.60:0/3107411273 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f105c0d0710 con 0x7f103c078bb0 2024-09-16T15:45:11.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c078bb0 msgr2=0x7f103c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c078bb0 0x7f103c07b070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f105c0b59c0 tx=0x7f1050007e90 comp rx=0 tx=0).stop 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15e870 msgr2=0x7f105c15d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15e870 0x7f105c15d1a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f1048009780 tx=0x7f1048033c20 comp rx=0 tx=0).stop 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 shutdown_connections 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c078bb0 0x7f103c07b070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f360 0x7f105c14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c15e870 0x7f105c15d1a0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 --2- 172.21.15.60:0/3107411273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c06e880 0x7f105c0c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 >> 172.21.15.60:0/3107411273 conn(0x7f105c0cf9f0 msgr2=0x7f105c072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 shutdown_connections 2024-09-16T15:45:11.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.600+0000 7f1062c00640 1 -- 172.21.15.60:0/3107411273 wait complete. 2024-09-16T15:45:11.603 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.602+0000 7fcb88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.603 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.602+0000 7fcb88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.603 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.602+0000 7fcb88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.603 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.602+0000 7fcb88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.603 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.602+0000 7fcb88200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.618 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone94430 2024-09-16T15:45:11.623 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.620+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.623 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.620+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.623 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.620+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.624 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.620+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.624 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:11.620+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.789+0000 7f6520000640 1 -- 172.21.15.60:0/3808961870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65181517a0 msgr2=0x7f65180cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.789+0000 7f6520000640 1 --2- 172.21.15.60:0/3808961870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65181517a0 0x7f65180cdad0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f650c005fd0 tx=0x7f650c02a540 comp rx=0 tx=0).stop 2024-09-16T15:45:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.789+0000 7f6520000640 1 -- 172.21.15.60:0/3808961870 shutdown_connections 2024-09-16T15:45:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.789+0000 7f6520000640 1 --2- 172.21.15.60:0/3808961870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65180cd490 0x7f65180732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.789+0000 7f6520000640 1 --2- 172.21.15.60:0/3808961870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65181517a0 0x7f65180cdad0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.789+0000 7f6520000640 1 -- 172.21.15.60:0/3808961870 >> 172.21.15.60:0/3808961870 conn(0x7f65180b1840 msgr2=0x7f65180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.789+0000 7f6520000640 1 -- 172.21.15.60:0/3808961870 shutdown_connections 2024-09-16T15:45:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.789+0000 7f6520000640 1 -- 172.21.15.60:0/3808961870 wait complete. 2024-09-16T15:45:11.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.791+0000 7f6520000640 1 Processor -- start 2024-09-16T15:45:11.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.791+0000 7f6520000640 1 -- start start 2024-09-16T15:45:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f6520000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518073800 0x7f651814f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f6520000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65180cd490 0x7f651806fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:11.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f6520000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65181517a0 0x7f65181e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f6520000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f65180cc650 con 0x7f65181517a0 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f6520000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6518073fc0 con 0x7f6518073800 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f6520000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f65180cc7d0 con 0x7f65180cd490 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f651d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518073800 0x7f651814f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f651d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518073800 0x7f651814f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:35170/0 (socket says 172.21.15.60:35170) 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f651d600640 1 -- 172.21.15.60:0/1925608809 learned_addr learned my addr 172.21.15.60:0/1925608809 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f651e000640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65181517a0 0x7f65181e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.792+0000 7f651cc00640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65180cd490 0x7f651806fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f651d600640 1 -- 172.21.15.60:0/1925608809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65180cd490 msgr2=0x7f651806fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f651d600640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65180cd490 0x7f651806fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f651d600640 1 -- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65181517a0 msgr2=0x7f65181e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f651d600640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65181517a0 0x7f65181e3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f651d600640 1 -- 172.21.15.60:0/1925608809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651806f260 con 0x7f6518073800 2024-09-16T15:45:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f651d600640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518073800 0x7f651814f330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f650c034040 tx=0x7f650c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f6515600640 1 -- 172.21.15.60:0/1925608809 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c03e030 con 0x7f6518073800 2024-09-16T15:45:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f6515600640 1 -- 172.21.15.60:0/1925608809 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f650c003d90 con 0x7f6518073800 2024-09-16T15:45:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651806f540 con 0x7f6518073800 2024-09-16T15:45:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f6515600640 1 -- 172.21.15.60:0/1925608809 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c035040 con 0x7f6518073800 2024-09-16T15:45:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.793+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65180b03d0 con 0x7f6518073800 2024-09-16T15:45:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.794+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65180ccc80 con 0x7f6518073800 2024-09-16T15:45:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.796+0000 7f6515600640 1 -- 172.21.15.60:0/1925608809 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f650c041030 con 0x7f6518073800 2024-09-16T15:45:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.796+0000 7f6515600640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f64f8078bb0 0x7f64f807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.797+0000 7f6515600640 1 -- 172.21.15.60:0/1925608809 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f650c0bb940 con 0x7f6518073800 2024-09-16T15:45:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.797+0000 7f651cc00640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f64f8078bb0 0x7f64f807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:11.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.797+0000 7f651cc00640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f64f8078bb0 0x7f64f807b070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65180d0290 tx=0x7f650400b490 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:11.801+0000 7f6515600640 1 -- 172.21.15.60:0/1925608809 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f650c0833c0 con 0x7f6518073800 2024-09-16T15:45:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.024+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone94430", "target": ["mon-mgr", ""]}) -- 0x7f65180cd190 con 0x7f64f8078bb0 2024-09-16T15:45:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.046+0000 7f6515600640 1 -- 172.21.15.60:0/1925608809 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f65180cd190 con 0x7f64f8078bb0 2024-09-16T15:45:12.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f64f8078bb0 msgr2=0x7f64f807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f64f8078bb0 0x7f64f807b070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65180d0290 tx=0x7f650400b490 comp rx=0 tx=0).stop 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518073800 msgr2=0x7f651814f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518073800 0x7f651814f330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f650c034040 tx=0x7f650c0038a0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 shutdown_connections 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f64f8078bb0 0x7f64f807b070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65181517a0 0x7f65181e3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65180cd490 0x7f651806fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 --2- 172.21.15.60:0/1925608809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6518073800 0x7f651814f330 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 >> 172.21.15.60:0/1925608809 conn(0x7f65180b1840 msgr2=0x7f65180d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 shutdown_connections 2024-09-16T15:45:12.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.048+0000 7f6520000640 1 -- 172.21.15.60:0/1925608809 wait complete. 2024-09-16T15:45:12.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone86371 2024-09-16T15:45:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.237+0000 7f9d75000640 1 -- 172.21.15.60:0/1698710434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d700b2480 msgr2=0x7f9d700b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.237+0000 7f9d75000640 1 --2- 172.21.15.60:0/1698710434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d700b2480 0x7f9d700b3120 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f9d60005fd0 tx=0x7f9d6002a5c0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.237+0000 7f9d75000640 1 -- 172.21.15.60:0/1698710434 shutdown_connections 2024-09-16T15:45:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.237+0000 7f9d75000640 1 --2- 172.21.15.60:0/1698710434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d7015bb80 0x7f9d700b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.237+0000 7f9d75000640 1 --2- 172.21.15.60:0/1698710434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d700b2480 0x7f9d700b3120 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.237+0000 7f9d75000640 1 -- 172.21.15.60:0/1698710434 >> 172.21.15.60:0/1698710434 conn(0x7f9d700b5680 msgr2=0x7f9d700b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.237+0000 7f9d75000640 1 -- 172.21.15.60:0/1698710434 shutdown_connections 2024-09-16T15:45:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.237+0000 7f9d75000640 1 -- 172.21.15.60:0/1698710434 wait complete. 2024-09-16T15:45:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.238+0000 7f9d75000640 1 Processor -- start 2024-09-16T15:45:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.238+0000 7f9d75000640 1 -- start start 2024-09-16T15:45:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.239+0000 7f9d75000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d700b2480 0x7f9d701e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.239+0000 7f9d75000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d700b4100 0x7f9d701486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.239+0000 7f9d75000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7015bb80 0x7f9d70150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.239+0000 7f9d75000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7015e240 con 0x7f9d7015bb80 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.239+0000 7f9d75000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7015e0c0 con 0x7f9d700b2480 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.239+0000 7f9d75000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7015e3c0 con 0x7f9d700b4100 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7015bb80 0x7f9d70150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7015bb80 0x7f9d70150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43138/0 (socket says 172.21.15.60:43138) 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6fe00640 1 -- 172.21.15.60:0/520939819 learned_addr learned my addr 172.21.15.60:0/520939819 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6f400640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d700b2480 0x7f9d701e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6ea00640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d700b4100 0x7f9d701486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6fe00640 1 -- 172.21.15.60:0/520939819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d700b4100 msgr2=0x7f9d701486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6fe00640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d700b4100 0x7f9d701486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6fe00640 1 -- 172.21.15.60:0/520939819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d700b2480 msgr2=0x7f9d701e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6fe00640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d700b2480 0x7f9d701e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.240+0000 7f9d6fe00640 1 -- 172.21.15.60:0/520939819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d70148c20 con 0x7f9d7015bb80 2024-09-16T15:45:12.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.241+0000 7f9d6fe00640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7015bb80 0x7f9d70150e10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c7a0 tx=0x7f9d6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.241+0000 7f9d67400640 1 -- 172.21.15.60:0/520939819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68003b70 con 0x7f9d7015bb80 2024-09-16T15:45:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.241+0000 7f9d67400640 1 -- 172.21.15.60:0/520939819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d68003d30 con 0x7f9d7015bb80 2024-09-16T15:45:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.241+0000 7f9d67400640 1 -- 172.21.15.60:0/520939819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6800aab0 con 0x7f9d7015bb80 2024-09-16T15:45:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.241+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701f8960 con 0x7f9d7015bb80 2024-09-16T15:45:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.241+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701f8f20 con 0x7f9d7015bb80 2024-09-16T15:45:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.242+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d70070780 con 0x7f9d7015bb80 2024-09-16T15:45:12.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.243+0000 7f9d67400640 1 -- 172.21.15.60:0/520939819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) ==== 101593+0+0 (secure 0 0 0) 0x7f9d6800ac50 con 0x7f9d7015bb80 2024-09-16T15:45:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.244+0000 7f9d67400640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d50078bb0 0x7f9d5007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.244+0000 7f9d67400640 1 -- 172.21.15.60:0/520939819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f9d6809adf0 con 0x7f9d7015bb80 2024-09-16T15:45:12.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.245+0000 7f9d6f400640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d50078bb0 0x7f9d5007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:12.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.245+0000 7f9d6f400640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d50078bb0 0x7f9d5007b070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d60033000 tx=0x7f9d60004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:12.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.248+0000 7f9d67400640 1 -- 172.21.15.60:0/520939819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d70070780 con 0x7f9d7015bb80 2024-09-16T15:45:12.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.472+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone86371", "target": ["mon-mgr", ""]}) -- 0x7f9d70069fa0 con 0x7f9d50078bb0 2024-09-16T15:45:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.490+0000 7f9d67400640 1 -- 172.21.15.60:0/520939819 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d70069fa0 con 0x7f9d50078bb0 2024-09-16T15:45:12.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d50078bb0 msgr2=0x7f9d5007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d50078bb0 0x7f9d5007b070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d60033000 tx=0x7f9d60004730 comp rx=0 tx=0).stop 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7015bb80 msgr2=0x7f9d70150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7015bb80 0x7f9d70150e10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c7a0 tx=0x7f9d6800cc60 comp rx=0 tx=0).stop 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 shutdown_connections 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9d50078bb0 0x7f9d5007b070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d7015bb80 0x7f9d70150e10 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9d700b4100 0x7f9d701486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 --2- 172.21.15.60:0/520939819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d700b2480 0x7f9d701e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 >> 172.21.15.60:0/520939819 conn(0x7f9d700b5680 msgr2=0x7f9d7014b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.492+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 shutdown_connections 2024-09-16T15:45:12.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12.493+0000 7f9d75000640 1 -- 172.21.15.60:0/520939819 wait complete. 2024-09-16T15:45:12.510 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:45:12.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:45:12.598 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-16T15:45:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:45:12.607 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:45:12.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:45:12.699 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:45:12.798 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:45:12.798 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:45:12.799 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:12.799 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:12.834 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:12.834 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:12.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:45:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 -- 172.21.15.60:0/3474438994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 msgr2=0x7f1b4c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 --2- 172.21.15.60:0/3474438994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 0x7f1b4c0b28f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c070640 tx=0x7f1b3c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:45:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 -- 172.21.15.60:0/3474438994 shutdown_connections 2024-09-16T15:45:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 --2- 172.21.15.60:0/3474438994 >> v2:172.21.15.100:3300/0 conn(0x7f1b4c0d0540 0x7f1b4c074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 --2- 172.21.15.60:0/3474438994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b4c0725d0 0x7f1b4c073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 --2- 172.21.15.60:0/3474438994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 0x7f1b4c0b28f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 -- 172.21.15.60:0/3474438994 >> 172.21.15.60:0/3474438994 conn(0x7f1b4c0cfa20 msgr2=0x7f1b4c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:13.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 -- 172.21.15.60:0/3474438994 shutdown_connections 2024-09-16T15:45:13.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.110+0000 7f1b4b200640 1 -- 172.21.15.60:0/3474438994 wait complete. 2024-09-16T15:45:13.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.111+0000 7f1b4b200640 1 Processor -- start 2024-09-16T15:45:13.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.112+0000 7f1b4b200640 1 -- start start 2024-09-16T15:45:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b4b200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b4c0d0540 0x7f1b4c145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b4b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 0x7f1b4c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b4b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b4c0cebf0 0x7f1b4c1f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b4b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c06f320 con 0x7f1b4c15bc30 2024-09-16T15:45:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b4b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c06f4a0 con 0x7f1b4c0cebf0 2024-09-16T15:45:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b4b200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1b4c06f620 con 0x7f1b4c0d0540 2024-09-16T15:45:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b41400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 0x7f1b4c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b49e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b4c0d0540 0x7f1b4c145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b41400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 0x7f1b4c0ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43150/0 (socket says 172.21.15.60:43150) 2024-09-16T15:45:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b41400640 1 -- 172.21.15.60:0/422406979 learned_addr learned my addr 172.21.15.60:0/422406979 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b4a800640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b4c0cebf0 0x7f1b4c1f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:13.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b41400640 1 -- 172.21.15.60:0/422406979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b4c0d0540 msgr2=0x7f1b4c145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b41400640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b4c0d0540 0x7f1b4c145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b41400640 1 -- 172.21.15.60:0/422406979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b4c0cebf0 msgr2=0x7f1b4c1f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b41400640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b4c0cebf0 0x7f1b4c1f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.113+0000 7f1b41400640 1 -- 172.21.15.60:0/422406979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c148f70 con 0x7f1b4c15bc30 2024-09-16T15:45:13.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.114+0000 7f1b41400640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 0x7f1b4c0ce6b0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400d720 tx=0x7f1b3400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:13.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.114+0000 7f1b42a00640 1 -- 172.21.15.60:0/422406979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34017070 con 0x7f1b4c15bc30 2024-09-16T15:45:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.114+0000 7f1b42a00640 1 -- 172.21.15.60:0/422406979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b34009d20 con 0x7f1b4c15bc30 2024-09-16T15:45:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.114+0000 7f1b42a00640 1 -- 172.21.15.60:0/422406979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34004280 con 0x7f1b4c15bc30 2024-09-16T15:45:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.114+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c1483f0 con 0x7f1b4c15bc30 2024-09-16T15:45:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.114+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c1489b0 con 0x7f1b4c15bc30 2024-09-16T15:45:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.115+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c0c60d0 con 0x7f1b4c15bc30 2024-09-16T15:45:13.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.116+0000 7f1b42a00640 1 -- 172.21.15.60:0/422406979 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f1b34005020 con 0x7f1b4c15bc30 2024-09-16T15:45:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.117+0000 7f1b42a00640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b28078cd0 0x7f1b2807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.117+0000 7f1b42a00640 1 -- 172.21.15.60:0/422406979 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f1b3409b5b0 con 0x7f1b4c15bc30 2024-09-16T15:45:13.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.118+0000 7f1b49e00640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b28078cd0 0x7f1b2807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:13.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.118+0000 7f1b49e00640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b28078cd0 0x7f1b2807b190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c035000 tx=0x7f1b3c03b040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:13.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.122+0000 7f1b42a00640 1 -- 172.21.15.60:0/422406979 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1b4c0c60d0 con 0x7f1b4c15bc30 2024-09-16T15:45:13.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.331+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1b4c0b28f0 con 0x7f1b4c15bc30 2024-09-16T15:45:13.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.331+0000 7f1b42a00640 1 -- 172.21.15.60:0/422406979 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7f1b34020020 con 0x7f1b4c15bc30 2024-09-16T15:45:13.332 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b28078cd0 msgr2=0x7f1b2807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b28078cd0 0x7f1b2807b190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c035000 tx=0x7f1b3c03b040 comp rx=0 tx=0).stop 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 msgr2=0x7f1b4c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 0x7f1b4c0ce6b0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400d720 tx=0x7f1b3400dbe0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 shutdown_connections 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b28078cd0 0x7f1b2807b190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b4c0cebf0 0x7f1b4c1f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b4c15bc30 0x7f1b4c0ce6b0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 --2- 172.21.15.60:0/422406979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b4c0d0540 0x7f1b4c145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 >> 172.21.15.60:0/422406979 conn(0x7f1b4c0cfa20 msgr2=0x7f1b4c0b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.333+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 shutdown_connections 2024-09-16T15:45:13.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.334+0000 7f1b4b200640 1 -- 172.21.15.60:0/422406979 wait complete. 2024-09-16T15:45:13.350 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:45:13.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:13.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:45:13.435 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:45:13.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:13.435 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:45:13.482 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:45:13.482 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:13.482 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:13.518 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:13.518 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:13.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:45:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.758+0000 7f0589e00640 1 -- 172.21.15.60:0/896763342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ccf00 msgr2=0x7f05840acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.758+0000 7f0589e00640 1 --2- 172.21.15.60:0/896763342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ccf00 0x7f05840acd30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f0584075130 tx=0x7f056c02a560 comp rx=0 tx=0).stop 2024-09-16T15:45:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.758+0000 7f0589e00640 1 -- 172.21.15.60:0/896763342 shutdown_connections 2024-09-16T15:45:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.758+0000 7f0589e00640 1 --2- 172.21.15.60:0/896763342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ccf00 0x7f05840acd30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.758+0000 7f0589e00640 1 --2- 172.21.15.60:0/896763342 >> v2:172.21.15.100:3300/0 conn(0x7f058415f5a0 0x7f05840b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.758+0000 7f0589e00640 1 -- 172.21.15.60:0/896763342 >> 172.21.15.60:0/896763342 conn(0x7f05840b1950 msgr2=0x7f05840b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.758+0000 7f0589e00640 1 -- 172.21.15.60:0/896763342 shutdown_connections 2024-09-16T15:45:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.759+0000 7f0589e00640 1 -- 172.21.15.60:0/896763342 wait complete. 2024-09-16T15:45:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.760+0000 7f0589e00640 1 Processor -- start 2024-09-16T15:45:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.760+0000 7f0589e00640 1 -- start start 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.761+0000 7f0589e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ad270 0x7f0584146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.761+0000 7f0589e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840ccf00 0x7f05840d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.761+0000 7f0589e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415f5a0 0x7f0584155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.761+0000 7f0589e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f058406f160 con 0x7f05840ad270 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.761+0000 7f0589e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f058406f2e0 con 0x7f05840ccf00 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.761+0000 7f0589e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f058406f460 con 0x7f058415f5a0 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.761+0000 7f0582a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ad270 0x7f0584146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0582a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ad270 0x7f0584146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43162/0 (socket says 172.21.15.60:43162) 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0582a00640 1 -- 172.21.15.60:0/877357775 learned_addr learned my addr 172.21.15.60:0/877357775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f057a000640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840ccf00 0x7f05840d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0583400640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415f5a0 0x7f0584155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:13.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0582a00640 1 -- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415f5a0 msgr2=0x7f0584155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0582a00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415f5a0 0x7f0584155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0582a00640 1 -- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840ccf00 msgr2=0x7f05840d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0582a00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840ccf00 0x7f05840d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0582a00640 1 -- 172.21.15.60:0/877357775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0584146b80 con 0x7f05840ad270 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f057a000640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840ccf00 0x7f05840d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0582a00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ad270 0x7f0584146640 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f057400d770 tx=0x7f057400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f057be00640 1 -- 172.21.15.60:0/877357775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574017070 con 0x7f05840ad270 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f057be00640 1 -- 172.21.15.60:0/877357775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0574009d20 con 0x7f05840ad270 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.762+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05841566d0 con 0x7f05840ad270 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.763+0000 7f057be00640 1 -- 172.21.15.60:0/877357775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05740106c0 con 0x7f05840ad270 2024-09-16T15:45:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.763+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0584156c90 con 0x7f05840ad270 2024-09-16T15:45:13.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.764+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058406f9f0 con 0x7f05840ad270 2024-09-16T15:45:13.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.765+0000 7f057be00640 1 -- 172.21.15.60:0/877357775 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f057401a070 con 0x7f05840ad270 2024-09-16T15:45:13.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.766+0000 7f057be00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0560078cd0 0x7f056007b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:13.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.766+0000 7f057be00640 1 -- 172.21.15.60:0/877357775 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7f057409b3c0 con 0x7f05840ad270 2024-09-16T15:45:13.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.766+0000 7f057a000640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0560078cd0 0x7f056007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:13.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.767+0000 7f057a000640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0560078cd0 0x7f056007b190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f056c033000 tx=0x7f056c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:13.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.771+0000 7f057be00640 1 -- 172.21.15.60:0/877357775 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0574063270 con 0x7f05840ad270 2024-09-16T15:45:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.979+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f05840741e0 con 0x7f05840ad270 2024-09-16T15:45:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.979+0000 7f057be00640 1 -- 172.21.15.60:0/877357775 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) ==== 80+0+0 (secure 0 0 0) 0x7f0574062b70 con 0x7f05840ad270 2024-09-16T15:45:13.981 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.981+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0560078cd0 msgr2=0x7f056007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.981+0000 7f0589e00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0560078cd0 0x7f056007b190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f056c033000 tx=0x7f056c004730 comp rx=0 tx=0).stop 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ad270 msgr2=0x7f0584146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ad270 0x7f0584146640 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f057400d770 tx=0x7f057400dc30 comp rx=0 tx=0).stop 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 shutdown_connections 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0560078cd0 0x7f056007b190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415f5a0 0x7f0584155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840ccf00 0x7f05840d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 --2- 172.21.15.60:0/877357775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840ad270 0x7f0584146640 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:13.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 >> 172.21.15.60:0/877357775 conn(0x7f05840b1950 msgr2=0x7f058415e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:13.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 shutdown_connections 2024-09-16T15:45:13.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:13.982+0000 7f0589e00640 1 -- 172.21.15.60:0/877357775 wait complete. 2024-09-16T15:45:13.998 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:45:13.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:13.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:45:14.061 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:45:14.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:14.062 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:45:14.098 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:45:14.098 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:14.098 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:14.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:45:14.135 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:45:14.135 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:14.135 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:14.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:45:14.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.335+0000 7fef6b400640 1 -- 172.21.15.60:0/3531478351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 msgr2=0x7fef6406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.335+0000 7fef6b400640 1 --2- 172.21.15.60:0/3531478351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 0x7fef6406ef50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fef58005fd0 tx=0x7fef5802a540 comp rx=0 tx=0).stop 2024-09-16T15:45:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.335+0000 7fef6b400640 1 -- 172.21.15.60:0/3531478351 shutdown_connections 2024-09-16T15:45:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.335+0000 7fef6b400640 1 --2- 172.21.15.60:0/3531478351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640af3d0 0x7fef6406f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.335+0000 7fef6b400640 1 --2- 172.21.15.60:0/3531478351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 0x7fef6406ef50 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.335+0000 7fef6b400640 1 -- 172.21.15.60:0/3531478351 >> 172.21.15.60:0/3531478351 conn(0x7fef640b1950 msgr2=0x7fef640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.335+0000 7fef6b400640 1 -- 172.21.15.60:0/3531478351 shutdown_connections 2024-09-16T15:45:14.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.336+0000 7fef6b400640 1 -- 172.21.15.60:0/3531478351 wait complete. 2024-09-16T15:45:14.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.337+0000 7fef6b400640 1 Processor -- start 2024-09-16T15:45:14.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.337+0000 7fef6b400640 1 -- start start 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef6b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6406f9d0 0x7fef641568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef6b400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640af3d0 0x7fef640ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef6b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 0x7fef641f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef6b400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fef641623a0 con 0x7fef64161550 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef6b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef64074010 con 0x7fef6406f9d0 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef6b400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fef64162520 con 0x7fef640af3d0 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef69400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 0x7fef641f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef69400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 0x7fef641f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43184/0 (socket says 172.21.15.60:43184) 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef69400640 1 -- 172.21.15.60:0/1500033891 learned_addr learned my addr 172.21.15.60:0/1500033891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef68a00640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6406f9d0 0x7fef641568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.338+0000 7fef63e00640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640af3d0 0x7fef640ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef69400640 1 -- 172.21.15.60:0/1500033891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640af3d0 msgr2=0x7fef640ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef69400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640af3d0 0x7fef640ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef69400640 1 -- 172.21.15.60:0/1500033891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6406f9d0 msgr2=0x7fef641568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef69400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6406f9d0 0x7fef641568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef69400640 1 -- 172.21.15.60:0/1500033891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef641576d0 con 0x7fef64161550 2024-09-16T15:45:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef69400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 0x7fef641f84d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c7a0 tx=0x7fef5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef60c00640 1 -- 172.21.15.60:0/1500033891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c003b70 con 0x7fef64161550 2024-09-16T15:45:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef60c00640 1 -- 172.21.15.60:0/1500033891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef5c003d30 con 0x7fef64161550 2024-09-16T15:45:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef60c00640 1 -- 172.21.15.60:0/1500033891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00aab0 con 0x7fef64161550 2024-09-16T15:45:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef641e8f40 con 0x7fef64161550 2024-09-16T15:45:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.339+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef641e9500 con 0x7fef64161550 2024-09-16T15:45:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.340+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef640760b0 con 0x7fef64161550 2024-09-16T15:45:14.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.342+0000 7fef60c00640 1 -- 172.21.15.60:0/1500033891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fef5c00ac50 con 0x7fef64161550 2024-09-16T15:45:14.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.342+0000 7fef60c00640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44078cd0 0x7fef4407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.343+0000 7fef60c00640 1 -- 172.21.15.60:0/1500033891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5767+0+0 (secure 0 0 0) 0x7fef5c09aed0 con 0x7fef64161550 2024-09-16T15:45:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.343+0000 7fef68a00640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44078cd0 0x7fef4407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.343+0000 7fef68a00640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44078cd0 0x7fef4407b190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef58033000 tx=0x7fef58004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.347+0000 7fef60c00640 1 -- 172.21.15.60:0/1500033891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fef5c0627d0 con 0x7fef64161550 2024-09-16T15:45:14.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.588+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef64162ab0 con 0x7fef64161550 2024-09-16T15:45:14.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.588+0000 7fef60c00640 1 -- 172.21.15.60:0/1500033891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5225 (secure 0 0 0) 0x7fef5c067200 con 0x7fef64161550 2024-09-16T15:45:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:14.590 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":72,"btime":"2024-09-16T15:44:09:511165+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/586556953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":586556953},{"type":"v1","addr":"172.21.15.100:6835","nonce":586556953}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2314857147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2314857147},{"type":"v1","addr":"172.21.15.60:6834","nonce":2314857147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1301436173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1301436173},{"type":"v1","addr":"172.21.15.60:6835","nonce":1301436173}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:58.903833+0000","modified":"2024-09-16T15:44:01.910153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6688":{"gid":6688,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6801/515865896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":515865896},{"type":"v1","addr":"172.21.15.100:6801","nonce":515865896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6688,"qdb_cluster":[6688]},"id":8}]} 2024-09-16T15:45:14.590 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 72 2024-09-16T15:45:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44078cd0 msgr2=0x7fef4407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44078cd0 0x7fef4407b190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef58033000 tx=0x7fef58004730 comp rx=0 tx=0).stop 2024-09-16T15:45:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 msgr2=0x7fef641f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:14.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 0x7fef641f84d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c7a0 tx=0x7fef5c00cc60 comp rx=0 tx=0).stop 2024-09-16T15:45:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 shutdown_connections 2024-09-16T15:45:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44078cd0 0x7fef4407b190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef64161550 0x7fef641f84d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640af3d0 0x7fef640ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 --2- 172.21.15.60:0/1500033891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6406f9d0 0x7fef641568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 >> 172.21.15.60:0/1500033891 conn(0x7fef640b1950 msgr2=0x7fef640af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 shutdown_connections 2024-09-16T15:45:14.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.591+0000 7fef6b400640 1 -- 172.21.15.60:0/1500033891 wait complete. 2024-09-16T15:45:14.609 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:14.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.771+0000 7f463ce00640 1 -- 172.21.15.60:0/1004136071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 msgr2=0x7f463806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:14.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.771+0000 7f463ce00640 1 --2- 172.21.15.60:0/1004136071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 0x7f463806ef50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4628005fd0 tx=0x7f462802a540 comp rx=0 tx=0).stop 2024-09-16T15:45:14.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.771+0000 7f463ce00640 1 -- 172.21.15.60:0/1004136071 shutdown_connections 2024-09-16T15:45:14.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.771+0000 7f463ce00640 1 --2- 172.21.15.60:0/1004136071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46380af3d0 0x7f463806f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.771+0000 7f463ce00640 1 --2- 172.21.15.60:0/1004136071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 0x7f463806ef50 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.771+0000 7f463ce00640 1 -- 172.21.15.60:0/1004136071 >> 172.21.15.60:0/1004136071 conn(0x7f46380b1950 msgr2=0x7f46380b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:14.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.771+0000 7f463ce00640 1 -- 172.21.15.60:0/1004136071 shutdown_connections 2024-09-16T15:45:14.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.772+0000 7f463ce00640 1 -- 172.21.15.60:0/1004136071 wait complete. 2024-09-16T15:45:14.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.773+0000 7f463ce00640 1 Processor -- start 2024-09-16T15:45:14.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.773+0000 7f463ce00640 1 -- start start 2024-09-16T15:45:14.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.774+0000 7f463ce00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f463806f9d0 0x7f4638148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:14.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.774+0000 7f463ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46380af3d0 0x7f46381f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:14.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.774+0000 7f463ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 0x7f463809cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:14.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.774+0000 7f463ce00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f46381623a0 con 0x7f4638161550 2024-09-16T15:45:14.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.774+0000 7f463ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4638074010 con 0x7f46380af3d0 2024-09-16T15:45:14.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.774+0000 7f463ce00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4638162520 con 0x7f463806f9d0 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.774+0000 7f4636800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 0x7f463809cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.774+0000 7f4635e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f463806f9d0 0x7f4638148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4636800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 0x7f463809cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43202/0 (socket says 172.21.15.60:43202) 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4636800640 1 -- 172.21.15.60:0/3872362166 learned_addr learned my addr 172.21.15.60:0/3872362166 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f462d400640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46380af3d0 0x7f46381f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4635e00640 1 -- 172.21.15.60:0/3872362166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46380af3d0 msgr2=0x7f46381f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4635e00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46380af3d0 0x7f46381f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4635e00640 1 -- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 msgr2=0x7f463809cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4635e00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 0x7f463809cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:14.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4635e00640 1 -- 172.21.15.60:0/3872362166 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4638148900 con 0x7f463806f9d0 2024-09-16T15:45:14.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4636800640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 0x7f463809cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:45:14.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.775+0000 7f4635e00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f463806f9d0 0x7f4638148270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4628031000 tx=0x7f46280038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:14.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.776+0000 7f462ea00640 1 -- 172.21.15.60:0/3872362166 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462803e030 con 0x7f463806f9d0 2024-09-16T15:45:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.776+0000 7f462ea00640 1 -- 172.21.15.60:0/3872362166 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4628003ca0 con 0x7f463806f9d0 2024-09-16T15:45:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.776+0000 7f462ea00640 1 -- 172.21.15.60:0/3872362166 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628035040 con 0x7f463806f9d0 2024-09-16T15:45:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.776+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4638148be0 con 0x7f463806f9d0 2024-09-16T15:45:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.776+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f46380af9a0 con 0x7f463806f9d0 2024-09-16T15:45:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.777+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46380760b0 con 0x7f463806f9d0 2024-09-16T15:45:14.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.778+0000 7f462ea00640 1 -- 172.21.15.60:0/3872362166 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f4628041030 con 0x7f463806f9d0 2024-09-16T15:45:14.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.779+0000 7f462ea00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4614078cd0 0x7f461407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:14.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.779+0000 7f462ea00640 1 -- 172.21.15.60:0/3872362166 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(110..110 src has 1..110) ==== 5823+0+0 (secure 0 0 0) 0x7f4628067020 con 0x7f463806f9d0 2024-09-16T15:45:14.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.779+0000 7f462d400640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4614078cd0 0x7f461407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:14.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.780+0000 7f462d400640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4614078cd0 0x7f461407b190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4638160e70 tx=0x7f4620005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:14.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:14.782+0000 7f462ea00640 1 -- 172.21.15.60:0/3872362166 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4628083610 con 0x7f463806f9d0 2024-09-16T15:45:15.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.014+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4638162ab0 con 0x7f463806f9d0 2024-09-16T15:45:15.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.015+0000 7f462ea00640 1 -- 172.21.15.60:0/3872362166 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5225 (secure 0 0 0) 0x7f4628088040 con 0x7f463806f9d0 2024-09-16T15:45:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:15.016 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":72,"btime":"2024-09-16T15:44:09:511165+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/586556953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":586556953},{"type":"v1","addr":"172.21.15.100:6835","nonce":586556953}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2314857147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2314857147},{"type":"v1","addr":"172.21.15.60:6834","nonce":2314857147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1301436173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1301436173},{"type":"v1","addr":"172.21.15.60:6835","nonce":1301436173}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:58.903833+0000","modified":"2024-09-16T15:44:01.910153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6688":{"gid":6688,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6801/515865896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":515865896},{"type":"v1","addr":"172.21.15.100:6801","nonce":515865896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6688,"qdb_cluster":[6688]},"id":8}]} 2024-09-16T15:45:15.016 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 72 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4614078cd0 msgr2=0x7f461407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4614078cd0 0x7f461407b190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4638160e70 tx=0x7f4620005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f463806f9d0 msgr2=0x7f4638148270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f463806f9d0 0x7f4638148270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4628031000 tx=0x7f46280038a0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 shutdown_connections 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4614078cd0 0x7f461407b190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4638161550 0x7f463809cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46380af3d0 0x7f46381f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 --2- 172.21.15.60:0/3872362166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f463806f9d0 0x7f4638148270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.017+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 >> 172.21.15.60:0/3872362166 conn(0x7f46380b1950 msgr2=0x7f46380acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.018+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 shutdown_connections 2024-09-16T15:45:15.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.018+0000 7f463ce00640 1 -- 172.21.15.60:0/3872362166 wait complete. 2024-09-16T15:45:15.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:15.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.196+0000 7fdf34c00640 1 -- 172.21.15.60:0/3528699496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf30161550 msgr2=0x7fdf3006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.196+0000 7fdf34c00640 1 --2- 172.21.15.60:0/3528699496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf30161550 0x7fdf3006ef50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005fd0 tx=0x7fdf2002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.197+0000 7fdf34c00640 1 -- 172.21.15.60:0/3528699496 shutdown_connections 2024-09-16T15:45:15.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.197+0000 7fdf34c00640 1 --2- 172.21.15.60:0/3528699496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf30161550 0x7fdf3006ef50 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.197+0000 7fdf34c00640 1 -- 172.21.15.60:0/3528699496 >> 172.21.15.60:0/3528699496 conn(0x7fdf300b1950 msgr2=0x7fdf300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:15.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.197+0000 7fdf34c00640 1 -- 172.21.15.60:0/3528699496 shutdown_connections 2024-09-16T15:45:15.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.197+0000 7fdf34c00640 1 -- 172.21.15.60:0/3528699496 wait complete. 2024-09-16T15:45:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.198+0000 7fdf34c00640 1 Processor -- start 2024-09-16T15:45:15.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.198+0000 7fdf34c00640 1 -- start start 2024-09-16T15:45:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.199+0000 7fdf34c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf30072df0 0x7fdf301f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.199+0000 7fdf34c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf300af3d0 0x7fdf300cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:15.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.199+0000 7fdf34c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf30161550 0x7fdf30157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.199+0000 7fdf34c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdf300739b0 con 0x7fdf300af3d0 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.199+0000 7fdf34c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3006fdb0 con 0x7fdf30161550 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.199+0000 7fdf34c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdf30073b30 con 0x7fdf30072df0 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf25400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf300af3d0 0x7fdf300cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf25400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf300af3d0 0x7fdf300cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43216/0 (socket says 172.21.15.60:43216) 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf25400640 1 -- 172.21.15.60:0/3253107472 learned_addr learned my addr 172.21.15.60:0/3253107472 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf2de00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf30072df0 0x7fdf301f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf2e800640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf30161550 0x7fdf30157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf25400640 1 -- 172.21.15.60:0/3253107472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf30072df0 msgr2=0x7fdf301f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf25400640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf30072df0 0x7fdf301f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf25400640 1 -- 172.21.15.60:0/3253107472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf30161550 msgr2=0x7fdf30157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf25400640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf30161550 0x7fdf30157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.200+0000 7fdf25400640 1 -- 172.21.15.60:0/3253107472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3009c850 con 0x7fdf300af3d0 2024-09-16T15:45:15.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.201+0000 7fdf25400640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf300af3d0 0x7fdf300cc270 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c500 tx=0x7fdf1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:15.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.201+0000 7fdf26a00640 1 -- 172.21.15.60:0/3253107472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180052d0 con 0x7fdf300af3d0 2024-09-16T15:45:15.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.201+0000 7fdf26a00640 1 -- 172.21.15.60:0/3253107472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf18005470 con 0x7fdf300af3d0 2024-09-16T15:45:15.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.201+0000 7fdf26a00640 1 -- 172.21.15.60:0/3253107472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18012870 con 0x7fdf300af3d0 2024-09-16T15:45:15.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.201+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf300ccb40 con 0x7fdf300af3d0 2024-09-16T15:45:15.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.201+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf300cd380 con 0x7fdf300af3d0 2024-09-16T15:45:15.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.202+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf30074020 con 0x7fdf300af3d0 2024-09-16T15:45:15.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.203+0000 7fdf26a00640 1 -- 172.21.15.60:0/3253107472 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fdf180035f0 con 0x7fdf300af3d0 2024-09-16T15:45:15.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.205+0000 7fdf26a00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf0c078cd0 0x7fdf0c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:15.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.205+0000 7fdf26a00640 1 -- 172.21.15.60:0/3253107472 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5823+0+0 (secure 0 0 0) 0x7fdf1809ac70 con 0x7fdf300af3d0 2024-09-16T15:45:15.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.205+0000 7fdf2de00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf0c078cd0 0x7fdf0c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:15.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.206+0000 7fdf2de00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf0c078cd0 0x7fdf0c07b190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf20033000 tx=0x7fdf20004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:15.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.209+0000 7fdf26a00640 1 -- 172.21.15.60:0/3253107472 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdf300ccb40 con 0x7fdf300af3d0 2024-09-16T15:45:15.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.409+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf3009ca20 con 0x7fdf300af3d0 2024-09-16T15:45:15.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.410+0000 7fdf26a00640 1 -- 172.21.15.60:0/3253107472 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16950 (secure 0 0 0) 0x7fdf18062550 con 0x7fdf300af3d0 2024-09-16T15:45:15.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:15.411 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":110,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:14.716042+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:56.694179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:57.861894+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.4","mappings":[{"from":0,"to":5}]},{"pgid":"17.c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:15.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf0c078cd0 msgr2=0x7fdf0c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf0c078cd0 0x7fdf0c07b190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf20033000 tx=0x7fdf20004730 comp rx=0 tx=0).stop 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf300af3d0 msgr2=0x7fdf300cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf300af3d0 0x7fdf300cc270 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c500 tx=0x7fdf1800c9c0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 shutdown_connections 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf0c078cd0 0x7fdf0c07b190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf30161550 0x7fdf30157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf300af3d0 0x7fdf300cc270 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 --2- 172.21.15.60:0/3253107472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf30072df0 0x7fdf301f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 >> 172.21.15.60:0/3253107472 conn(0x7fdf300b1950 msgr2=0x7fdf3006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.412+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 shutdown_connections 2024-09-16T15:45:15.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.413+0000 7fdf34c00640 1 -- 172.21.15.60:0/3253107472 wait complete. 2024-09-16T15:45:15.430 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:45:15.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.593+0000 7efdae400640 1 -- 172.21.15.60:0/3464539324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 msgr2=0x7efda80d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.593+0000 7efdae400640 1 --2- 172.21.15.60:0/3464539324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 0x7efda80d2ae0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7efda808c670 tx=0x7efd9002a680 comp rx=0 tx=0).stop 2024-09-16T15:45:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.593+0000 7efdae400640 1 -- 172.21.15.60:0/3464539324 shutdown_connections 2024-09-16T15:45:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.593+0000 7efdae400640 1 --2- 172.21.15.60:0/3464539324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 0x7efda80d2ae0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.593+0000 7efdae400640 1 --2- 172.21.15.60:0/3464539324 >> v2:172.21.15.100:3300/0 conn(0x7efda80d21c0 0x7efda80d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.593+0000 7efdae400640 1 -- 172.21.15.60:0/3464539324 >> 172.21.15.60:0/3464539324 conn(0x7efda80d3fe0 msgr2=0x7efda80d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.594+0000 7efdae400640 1 -- 172.21.15.60:0/3464539324 shutdown_connections 2024-09-16T15:45:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.594+0000 7efdae400640 1 -- 172.21.15.60:0/3464539324 wait complete. 2024-09-16T15:45:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.595+0000 7efdae400640 1 Processor -- start 2024-09-16T15:45:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.595+0000 7efdae400640 1 -- start start 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efdae400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 0x7efda815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efdae400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda80d21c0 0x7efda81f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efdae400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda8161230 0x7efda806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efdae400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efda8161b60 con 0x7efda80b50c0 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efdae400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efda81619e0 con 0x7efda80d21c0 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efdae400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7efda80b4050 con 0x7efda8161230 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efda7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 0x7efda815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efda7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 0x7efda815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43236/0 (socket says 172.21.15.60:43236) 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efda7400640 1 -- 172.21.15.60:0/1936099632 learned_addr learned my addr 172.21.15.60:0/1936099632 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.596+0000 7efd9ea00640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda80d21c0 0x7efda81f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efda7e00640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda8161230 0x7efda806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efda7400640 1 -- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda8161230 msgr2=0x7efda806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efda7400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda8161230 0x7efda806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efda7400640 1 -- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda80d21c0 msgr2=0x7efda81f55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efda7400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda80d21c0 0x7efda81f55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efda7400640 1 -- 172.21.15.60:0/1936099632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda80b6e10 con 0x7efda80b50c0 2024-09-16T15:45:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efda7e00640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda8161230 0x7efda806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efda7400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 0x7efda815bb90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7efd9800ea20 tx=0x7efd9800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efd9fe00640 1 -- 172.21.15.60:0/1936099632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9800cd20 con 0x7efda80b50c0 2024-09-16T15:45:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efd9fe00640 1 -- 172.21.15.60:0/1936099632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd98005020 con 0x7efda80b50c0 2024-09-16T15:45:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda815ddb0 con 0x7efda80b50c0 2024-09-16T15:45:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efd9fe00640 1 -- 172.21.15.60:0/1936099632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98005450 con 0x7efda80b50c0 2024-09-16T15:45:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.597+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda815e370 con 0x7efda80b50c0 2024-09-16T15:45:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.598+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda8160c30 con 0x7efda80b50c0 2024-09-16T15:45:15.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.600+0000 7efd9fe00640 1 -- 172.21.15.60:0/1936099632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7efd980047a0 con 0x7efda80b50c0 2024-09-16T15:45:15.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.601+0000 7efd9fe00640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd84078cd0 0x7efd8407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:15.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.601+0000 7efd9fe00640 1 -- 172.21.15.60:0/1936099632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5823+0+0 (secure 0 0 0) 0x7efd9809bd00 con 0x7efda80b50c0 2024-09-16T15:45:15.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.601+0000 7efd9ea00640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd84078cd0 0x7efd8407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:15.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.602+0000 7efd9ea00640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd84078cd0 0x7efd8407b190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd90033000 tx=0x7efd90004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:15.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.606+0000 7efd9fe00640 1 -- 172.21.15.60:0/1936099632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efda8160c30 con 0x7efda80b50c0 2024-09-16T15:45:15.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.839+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efda80b3940 con 0x7efda80b50c0 2024-09-16T15:45:15.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.840+0000 7efd9fe00640 1 -- 172.21.15.60:0/1936099632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5225 (secure 0 0 0) 0x7efd98063af0 con 0x7efda80b50c0 2024-09-16T15:45:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:15.842 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":72,"btime":"2024-09-16T15:44:09:511165+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/586556953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":586556953},{"type":"v1","addr":"172.21.15.100:6835","nonce":586556953}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2314857147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2314857147},{"type":"v1","addr":"172.21.15.60:6834","nonce":2314857147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1301436173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1301436173},{"type":"v1","addr":"172.21.15.60:6835","nonce":1301436173}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:58.903833+0000","modified":"2024-09-16T15:44:01.910153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6688":{"gid":6688,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6801/515865896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":515865896},{"type":"v1","addr":"172.21.15.100:6801","nonce":515865896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6688,"qdb_cluster":[6688]},"id":8}]} 2024-09-16T15:45:15.842 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 72 2024-09-16T15:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd84078cd0 msgr2=0x7efd8407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd84078cd0 0x7efd8407b190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd90033000 tx=0x7efd90004730 comp rx=0 tx=0).stop 2024-09-16T15:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 msgr2=0x7efda815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 0x7efda815bb90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7efd9800ea20 tx=0x7efd9800ee20 comp rx=0 tx=0).stop 2024-09-16T15:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 shutdown_connections 2024-09-16T15:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd84078cd0 0x7efd8407b190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda8161230 0x7efda806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda80d21c0 0x7efda81f55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 --2- 172.21.15.60:0/1936099632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda80b50c0 0x7efda815bb90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:15.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 >> 172.21.15.60:0/1936099632 conn(0x7efda80d3fe0 msgr2=0x7efda8160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:15.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 shutdown_connections 2024-09-16T15:45:15.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:15.843+0000 7efdae400640 1 -- 172.21.15.60:0/1936099632 wait complete. 2024-09-16T15:45:15.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:16.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.020+0000 7ff2ddc00640 1 -- 172.21.15.60:0/2479375438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2bc003aa0 msgr2=0x7ff2bc003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.020+0000 7ff2ddc00640 1 --2- 172.21.15.60:0/2479375438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2bc003aa0 0x7ff2bc003ec0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006fb0 tx=0x7ff2bc005830 comp rx=0 tx=0).stop 2024-09-16T15:45:16.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.020+0000 7ff2ddc00640 1 -- 172.21.15.60:0/2479375438 shutdown_connections 2024-09-16T15:45:16.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.020+0000 7ff2ddc00640 1 --2- 172.21.15.60:0/2479375438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2bc003aa0 0x7ff2bc003ec0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.020+0000 7ff2ddc00640 1 -- 172.21.15.60:0/2479375438 >> 172.21.15.60:0/2479375438 conn(0x7ff2d80b1950 msgr2=0x7ff2d80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:16.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.020+0000 7ff2ddc00640 1 -- 172.21.15.60:0/2479375438 shutdown_connections 2024-09-16T15:45:16.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.021+0000 7ff2ddc00640 1 -- 172.21.15.60:0/2479375438 wait complete. 2024-09-16T15:45:16.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.022+0000 7ff2ddc00640 1 Processor -- start 2024-09-16T15:45:16.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.022+0000 7ff2ddc00640 1 -- start start 2024-09-16T15:45:16.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2ddc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2bc003aa0 0x7ff2d81f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2ddc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2d814e580 0x7ff2d814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2ddc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2d80b0d90 0x7ff2d80b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2ddc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8074420 con 0x7ff2d80b0d90 2024-09-16T15:45:16.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2ddc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d80b26a0 con 0x7ff2d814e580 2024-09-16T15:45:16.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2ddc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d80745a0 con 0x7ff2bc003aa0 2024-09-16T15:45:16.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2d7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2d80b0d90 0x7ff2d80b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2d7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2d80b0d90 0x7ff2d80b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43248/0 (socket says 172.21.15.60:43248) 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2d7400640 1 -- 172.21.15.60:0/726626426 learned_addr learned my addr 172.21.15.60:0/726626426 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.023+0000 7ff2d6a00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2bc003aa0 0x7ff2d81f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2ce000640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2d814e580 0x7ff2d814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2d7400640 1 -- 172.21.15.60:0/726626426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2bc003aa0 msgr2=0x7ff2d81f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2d7400640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2bc003aa0 0x7ff2d81f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2d7400640 1 -- 172.21.15.60:0/726626426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2d814e580 msgr2=0x7ff2d814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2d7400640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2d814e580 0x7ff2d814c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2d7400640 1 -- 172.21.15.60:0/726626426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d8150300 con 0x7ff2d80b0d90 2024-09-16T15:45:16.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2d7400640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2d80b0d90 0x7ff2d80b1220 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff2d806e130 tx=0x7ff2d0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:16.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2cf400640 1 -- 172.21.15.60:0/726626426 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0011840 con 0x7ff2d80b0d90 2024-09-16T15:45:16.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2cf400640 1 -- 172.21.15.60:0/726626426 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2d0018070 con 0x7ff2d80b0d90 2024-09-16T15:45:16.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2cf400640 1 -- 172.21.15.60:0/726626426 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0011cc0 con 0x7ff2d80b0d90 2024-09-16T15:45:16.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.024+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d814f500 con 0x7ff2d80b0d90 2024-09-16T15:45:16.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.025+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d814ef40 con 0x7ff2d80b0d90 2024-09-16T15:45:16.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.026+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d80acd30 con 0x7ff2d80b0d90 2024-09-16T15:45:16.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.027+0000 7ff2cf400640 1 -- 172.21.15.60:0/726626426 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7ff2d001d460 con 0x7ff2d80b0d90 2024-09-16T15:45:16.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.028+0000 7ff2cf400640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2b4078d00 0x7ff2b407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.028+0000 7ff2cf400640 1 -- 172.21.15.60:0/726626426 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5823+0+0 (secure 0 0 0) 0x7ff2d009b970 con 0x7ff2d80b0d90 2024-09-16T15:45:16.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.028+0000 7ff2d6a00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2b4078d00 0x7ff2b407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.029+0000 7ff2d6a00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2b4078d00 0x7ff2b407b1c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8033000 tx=0x7ff2c803b040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:16.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.032+0000 7ff2cf400640 1 -- 172.21.15.60:0/726626426 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff2d0063920 con 0x7ff2d80b0d90 2024-09-16T15:45:16.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.266+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2d8154fb0 con 0x7ff2d80b0d90 2024-09-16T15:45:16.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.267+0000 7ff2cf400640 1 -- 172.21.15.60:0/726626426 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5225 (secure 0 0 0) 0x7ff2d0063220 con 0x7ff2d80b0d90 2024-09-16T15:45:16.268 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:16.268 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":72,"btime":"2024-09-16T15:44:09:511165+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/586556953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":586556953},{"type":"v1","addr":"172.21.15.100:6835","nonce":586556953}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2314857147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2314857147},{"type":"v1","addr":"172.21.15.60:6834","nonce":2314857147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1301436173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1301436173},{"type":"v1","addr":"172.21.15.60:6835","nonce":1301436173}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:43:58.903833+0000","modified":"2024-09-16T15:44:01.910153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6688},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6688":{"gid":6688,"name":"b","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6801/515865896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":515865896},{"type":"v1","addr":"172.21.15.100:6801","nonce":515865896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6688,"qdb_cluster":[6688]},"id":8}]} 2024-09-16T15:45:16.268 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 72 2024-09-16T15:45:16.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.269+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2b4078d00 msgr2=0x7ff2b407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.269+0000 7ff2ddc00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2b4078d00 0x7ff2b407b1c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8033000 tx=0x7ff2c803b040 comp rx=0 tx=0).stop 2024-09-16T15:45:16.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.269+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2d80b0d90 msgr2=0x7ff2d80b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.269+0000 7ff2ddc00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2d80b0d90 0x7ff2d80b1220 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff2d806e130 tx=0x7ff2d0006430 comp rx=0 tx=0).stop 2024-09-16T15:45:16.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.270+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 shutdown_connections 2024-09-16T15:45:16.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.270+0000 7ff2ddc00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2b4078d00 0x7ff2b407b1c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.270+0000 7ff2ddc00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2d80b0d90 0x7ff2d80b1220 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.270+0000 7ff2ddc00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2d814e580 0x7ff2d814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.270+0000 7ff2ddc00640 1 --2- 172.21.15.60:0/726626426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2bc003aa0 0x7ff2d81f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.270+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 >> 172.21.15.60:0/726626426 conn(0x7ff2d80b1950 msgr2=0x7ff2d809ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:16.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.270+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 shutdown_connections 2024-09-16T15:45:16.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.270+0000 7ff2ddc00640 1 -- 172.21.15.60:0/726626426 wait complete. 2024-09-16T15:45:16.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:16.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.446+0000 7fcd39200640 1 -- 172.21.15.60:0/1838106035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd340d0670 msgr2=0x7fcd340d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.446+0000 7fcd39200640 1 --2- 172.21.15.60:0/1838106035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd340d0670 0x7fcd340d2760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009860 tx=0x7fcd2002a540 comp rx=0 tx=0).stop 2024-09-16T15:45:16.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.446+0000 7fcd39200640 1 -- 172.21.15.60:0/1838106035 shutdown_connections 2024-09-16T15:45:16.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.446+0000 7fcd39200640 1 --2- 172.21.15.60:0/1838106035 >> v2:172.21.15.100:3300/0 conn(0x7fcd340b1680 0x7fcd340b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.446+0000 7fcd39200640 1 --2- 172.21.15.60:0/1838106035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd340d0670 0x7fcd340d2760 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.446+0000 7fcd39200640 1 -- 172.21.15.60:0/1838106035 >> 172.21.15.60:0/1838106035 conn(0x7fcd340d1c40 msgr2=0x7fcd340d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:16.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.446+0000 7fcd39200640 1 -- 172.21.15.60:0/1838106035 shutdown_connections 2024-09-16T15:45:16.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.446+0000 7fcd39200640 1 -- 172.21.15.60:0/1838106035 wait complete. 2024-09-16T15:45:16.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.448+0000 7fcd39200640 1 Processor -- start 2024-09-16T15:45:16.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.448+0000 7fcd39200640 1 -- start start 2024-09-16T15:45:16.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd39200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd340b1680 0x7fcd341575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd39200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd340d0670 0x7fcd3406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd39200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3415bd30 0x7fcd340718a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd39200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcd340b21f0 con 0x7fcd340d0670 2024-09-16T15:45:16.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd39200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcd340b2070 con 0x7fcd3415bd30 2024-09-16T15:45:16.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd39200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcd340b2370 con 0x7fcd340b1680 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd32a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd340d0670 0x7fcd3406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd32a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd340d0670 0x7fcd3406f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43250/0 (socket says 172.21.15.60:43250) 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd32a00640 1 -- 172.21.15.60:0/3254667739 learned_addr learned my addr 172.21.15.60:0/3254667739 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd33e00640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3415bd30 0x7fcd340718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.449+0000 7fcd33400640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd340b1680 0x7fcd341575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd32a00640 1 -- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd340b1680 msgr2=0x7fcd341575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd32a00640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd340b1680 0x7fcd341575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd32a00640 1 -- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3415bd30 msgr2=0x7fcd340718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd32a00640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3415bd30 0x7fcd340718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd32a00640 1 -- 172.21.15.60:0/3254667739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd340b5aa0 con 0x7fcd340d0670 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd33e00640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3415bd30 0x7fcd340718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd32a00640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd340d0670 0x7fcd3406f580 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009980 tx=0x7fcd20032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:16.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd2b400640 1 -- 172.21.15.60:0/3254667739 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20040070 con 0x7fcd340d0670 2024-09-16T15:45:16.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd2b400640 1 -- 172.21.15.60:0/3254667739 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd20005330 con 0x7fcd340d0670 2024-09-16T15:45:16.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd2b400640 1 -- 172.21.15.60:0/3254667739 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200047a0 con 0x7fcd340d0670 2024-09-16T15:45:16.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd340b5d80 con 0x7fcd340d0670 2024-09-16T15:45:16.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.450+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd340b6340 con 0x7fcd340d0670 2024-09-16T15:45:16.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.451+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd340b2900 con 0x7fcd340d0670 2024-09-16T15:45:16.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.453+0000 7fcd2b400640 1 -- 172.21.15.60:0/3254667739 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fcd20039040 con 0x7fcd340d0670 2024-09-16T15:45:16.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.454+0000 7fcd2b400640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcd14078d00 0x7fcd1407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.454+0000 7fcd2b400640 1 -- 172.21.15.60:0/3254667739 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5823+0+0 (secure 0 0 0) 0x7fcd200bc3a0 con 0x7fcd340d0670 2024-09-16T15:45:16.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.454+0000 7fcd33400640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcd14078d00 0x7fcd1407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.454+0000 7fcd33400640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcd14078d00 0x7fcd1407b1c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd341f7d90 tx=0x7fcd24009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:16.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.458+0000 7fcd2b400640 1 -- 172.21.15.60:0/3254667739 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcd20083ac0 con 0x7fcd340d0670 2024-09-16T15:45:16.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.653+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd340cde20 con 0x7fcd340d0670 2024-09-16T15:45:16.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.654+0000 7fcd2b400640 1 -- 172.21.15.60:0/3254667739 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16950 (secure 0 0 0) 0x7fcd200884f0 con 0x7fcd340d0670 2024-09-16T15:45:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:16.656 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":110,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:14.716042+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:43:56.694179+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:43:57.861894+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.4","mappings":[{"from":0,"to":5}]},{"pgid":"17.c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcd14078d00 msgr2=0x7fcd1407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcd14078d00 0x7fcd1407b1c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd341f7d90 tx=0x7fcd24009040 comp rx=0 tx=0).stop 2024-09-16T15:45:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd340d0670 msgr2=0x7fcd3406f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd340d0670 0x7fcd3406f580 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009980 tx=0x7fcd20032b90 comp rx=0 tx=0).stop 2024-09-16T15:45:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 shutdown_connections 2024-09-16T15:45:16.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcd14078d00 0x7fcd1407b1c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd3415bd30 0x7fcd340718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd340d0670 0x7fcd3406f580 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 --2- 172.21.15.60:0/3254667739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd340b1680 0x7fcd341575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 >> 172.21.15.60:0/3254667739 conn(0x7fcd340d1c40 msgr2=0x7fcd34157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:16.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 shutdown_connections 2024-09-16T15:45:16.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.657+0000 7fcd39200640 1 -- 172.21.15.60:0/3254667739 wait complete. 2024-09-16T15:45:16.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:45:16.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.836+0000 7f5577e00640 1 -- 172.21.15.60:0/35449031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f555c004800 msgr2=0x7f555c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.836+0000 7f5577e00640 1 --2- 172.21.15.60:0/35449031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f555c004800 0x7f555c004c80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5578072580 tx=0x7f555c005830 comp rx=0 tx=0).stop 2024-09-16T15:45:16.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.836+0000 7f5577e00640 1 -- 172.21.15.60:0/35449031 shutdown_connections 2024-09-16T15:45:16.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.836+0000 7f5577e00640 1 --2- 172.21.15.60:0/35449031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f555c004800 0x7f555c004c80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.836+0000 7f5577e00640 1 -- 172.21.15.60:0/35449031 >> 172.21.15.60:0/35449031 conn(0x7f55780cfa20 msgr2=0x7f55780cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:16.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.836+0000 7f5577e00640 1 -- 172.21.15.60:0/35449031 shutdown_connections 2024-09-16T15:45:16.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.836+0000 7f5577e00640 1 -- 172.21.15.60:0/35449031 wait complete. 2024-09-16T15:45:16.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.838+0000 7f5577e00640 1 Processor -- start 2024-09-16T15:45:16.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.838+0000 7f5577e00640 1 -- start start 2024-09-16T15:45:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f5577e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555c004800 0x7f557814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f5577e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557814dae0 0x7f557814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f5577e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55781f5ab0 0x7f55781f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f5577e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f55780b2c80 con 0x7f557814dae0 2024-09-16T15:45:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f5577e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55780b2b00 con 0x7f55781f5ab0 2024-09-16T15:45:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f5577e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5578073d40 con 0x7f555c004800 2024-09-16T15:45:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557814dae0 0x7f557814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557814dae0 0x7f557814f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43270/0 (socket says 172.21.15.60:43270) 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 -- 172.21.15.60:0/3545435725 learned_addr learned my addr 172.21.15.60:0/3545435725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 -- 172.21.15.60:0/3545435725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555c004800 msgr2=0x7f557814c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555c004800 0x7f557814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 -- 172.21.15.60:0/3545435725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55781f5ab0 msgr2=0x7f55781f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55781f5ab0 0x7f55781f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 -- 172.21.15.60:0/3545435725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557814ca40 con 0x7f557814dae0 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.839+0000 7f556e000640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557814dae0 0x7f557814f2c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f5560036040 tx=0x7f5560033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.840+0000 7f556f400640 1 -- 172.21.15.60:0/3545435725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560038040 con 0x7f557814dae0 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.840+0000 7f556f400640 1 -- 172.21.15.60:0/3545435725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5560041070 con 0x7f557814dae0 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.840+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557814cd20 con 0x7f557814dae0 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.840+0000 7f556f400640 1 -- 172.21.15.60:0/3545435725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560033c10 con 0x7f557814dae0 2024-09-16T15:45:16.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.840+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557814d2e0 con 0x7f557814dae0 2024-09-16T15:45:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.841+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5578073320 con 0x7f557814dae0 2024-09-16T15:45:16.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.842+0000 7f556f400640 1 -- 172.21.15.60:0/3545435725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f5560046460 con 0x7f557814dae0 2024-09-16T15:45:16.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.843+0000 7f556f400640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5554078cd0 0x7f555407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:16.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.843+0000 7f556f400640 1 -- 172.21.15.60:0/3545435725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5823+0+0 (secure 0 0 0) 0x7f55600bb980 con 0x7f557814dae0 2024-09-16T15:45:16.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.843+0000 7f5576a00640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5554078cd0 0x7f555407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:16.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.844+0000 7f5576a00640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5554078cd0 0x7f555407b190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5568000fb0 tx=0x7f55680014b0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:16.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:16.847+0000 7f556f400640 1 -- 172.21.15.60:0/3545435725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5578073320 con 0x7f557814dae0 2024-09-16T15:45:17.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.083+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f55780742a0 con 0x7f557814dae0 2024-09-16T15:45:17.085 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:45:17.084+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:45:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.086+0000 7f556f400640 1 -- 172.21.15.60:0/3545435725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) ==== 161+0+0 (secure 0 0 0) 0x7f55600838a0 con 0x7f557814dae0 2024-09-16T15:45:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.089+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5554078cd0 msgr2=0x7f555407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.089+0000 7f5577e00640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5554078cd0 0x7f555407b190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5568000fb0 tx=0x7f55680014b0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.089+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557814dae0 msgr2=0x7f557814f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.089+0000 7f5577e00640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557814dae0 0x7f557814f2c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f5560036040 tx=0x7f5560033670 comp rx=0 tx=0).stop 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.090+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 shutdown_connections 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.090+0000 7f5577e00640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5554078cd0 0x7f555407b190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.090+0000 7f5577e00640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55781f5ab0 0x7f55781f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.090+0000 7f5577e00640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f557814dae0 0x7f557814f2c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.090+0000 7f5577e00640 1 --2- 172.21.15.60:0/3545435725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555c004800 0x7f557814c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.090+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 >> 172.21.15.60:0/3545435725 conn(0x7f55780cfa20 msgr2=0x7f5578152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.090+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 shutdown_connections 2024-09-16T15:45:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.090+0000 7f5577e00640 1 -- 172.21.15.60:0/3545435725 wait complete. 2024-09-16T15:45:17.106 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:45:17.157 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:45:17.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.272+0000 7fbad0600640 1 -- 172.21.15.60:0/2327857710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 msgr2=0x7fbac80cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.272+0000 7fbad0600640 1 --2- 172.21.15.60:0/2327857710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 0x7fbac80cebe0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbac806db10 tx=0x7fbabc02a530 comp rx=0 tx=0).stop 2024-09-16T15:45:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.272+0000 7fbad0600640 1 -- 172.21.15.60:0/2327857710 shutdown_connections 2024-09-16T15:45:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.272+0000 7fbad0600640 1 --2- 172.21.15.60:0/2327857710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbac815f090 0x7fbac80cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.272+0000 7fbad0600640 1 --2- 172.21.15.60:0/2327857710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 0x7fbac80cebe0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.272+0000 7fbad0600640 1 -- 172.21.15.60:0/2327857710 >> 172.21.15.60:0/2327857710 conn(0x7fbac80b1840 msgr2=0x7fbac80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.272+0000 7fbad0600640 1 -- 172.21.15.60:0/2327857710 shutdown_connections 2024-09-16T15:45:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.272+0000 7fbad0600640 1 -- 172.21.15.60:0/2327857710 wait complete. 2024-09-16T15:45:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.274+0000 7fbad0600640 1 Processor -- start 2024-09-16T15:45:17.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.274+0000 7fbad0600640 1 -- start start 2024-09-16T15:45:17.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbad0600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 0x7fbac8153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:17.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbad0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbac815c080 0x7fbac8159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbad0600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbac815f090 0x7fbac81f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbad0600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbac8164e50 con 0x7fbac80b3370 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbad0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbac8164cd0 con 0x7fbac815c080 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbad0600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbac8164fd0 con 0x7fbac815f090 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbacdc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 0x7fbac8153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbacdc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 0x7fbac8153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43284/0 (socket says 172.21.15.60:43284) 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbacdc00640 1 -- 172.21.15.60:0/3124825552 learned_addr learned my addr 172.21.15.60:0/3124825552 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbacd200640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbac815c080 0x7fbac8159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.275+0000 7fbace600640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbac815f090 0x7fbac81f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbacdc00640 1 -- 172.21.15.60:0/3124825552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbac815f090 msgr2=0x7fbac81f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbacdc00640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbac815f090 0x7fbac81f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbacdc00640 1 -- 172.21.15.60:0/3124825552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbac815c080 msgr2=0x7fbac8159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:17.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbacdc00640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbac815c080 0x7fbac8159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:17.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbacdc00640 1 -- 172.21.15.60:0/3124825552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac8155580 con 0x7fbac80b3370 2024-09-16T15:45:17.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbacdc00640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 0x7fbac8153310 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009850 tx=0x7fbabc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:17.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbac5600640 1 -- 172.21.15.60:0/3124825552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc03f070 con 0x7fbac80b3370 2024-09-16T15:45:17.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbac5600640 1 -- 172.21.15.60:0/3124825552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbabc0047a0 con 0x7fbac80b3370 2024-09-16T15:45:17.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbac5600640 1 -- 172.21.15.60:0/3124825552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc035030 con 0x7fbac80b3370 2024-09-16T15:45:17.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.276+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac815d510 con 0x7fbac80b3370 2024-09-16T15:45:17.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.277+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac815dad0 con 0x7fbac80b3370 2024-09-16T15:45:17.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.278+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac815e980 con 0x7fbac80b3370 2024-09-16T15:45:17.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.279+0000 7fbac5600640 1 -- 172.21.15.60:0/3124825552 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fbabc042070 con 0x7fbac80b3370 2024-09-16T15:45:17.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.280+0000 7fbac5600640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbaa8078cd0 0x7fbaa807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:17.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.280+0000 7fbac5600640 1 -- 172.21.15.60:0/3124825552 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7fbabc0bc060 con 0x7fbac80b3370 2024-09-16T15:45:17.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.280+0000 7fbacd200640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbaa8078cd0 0x7fbaa807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:17.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.281+0000 7fbacd200640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbaa8078cd0 0x7fbaa807b190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbac8164650 tx=0x7fbab4007e50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:17.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.284+0000 7fbac5600640 1 -- 172.21.15.60:0/3124825552 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbac815b320 con 0x7fbac80b3370 2024-09-16T15:45:17.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:17.520+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbac8155750 con 0x7fbac80b3370 2024-09-16T15:45:18.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.087+0000 7fbac5600640 1 -- 172.21.15.60:0/3124825552 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) ==== 104+0+0 (secure 0 0 0) 0x7fbabc083850 con 0x7fbac80b3370 2024-09-16T15:45:18.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.090+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbaa8078cd0 msgr2=0x7fbaa807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:18.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.090+0000 7fbad0600640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbaa8078cd0 0x7fbaa807b190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbac8164650 tx=0x7fbab4007e50 comp rx=0 tx=0).stop 2024-09-16T15:45:18.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.090+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 msgr2=0x7fbac8153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:18.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.090+0000 7fbad0600640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 0x7fbac8153310 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009850 tx=0x7fbabc003910 comp rx=0 tx=0).stop 2024-09-16T15:45:18.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.091+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 shutdown_connections 2024-09-16T15:45:18.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.091+0000 7fbad0600640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbaa8078cd0 0x7fbaa807b190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:18.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.091+0000 7fbad0600640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbac815f090 0x7fbac81f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:18.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.091+0000 7fbad0600640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbac815c080 0x7fbac8159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:18.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.091+0000 7fbad0600640 1 --2- 172.21.15.60:0/3124825552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbac80b3370 0x7fbac8153310 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:18.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.091+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 >> 172.21.15.60:0/3124825552 conn(0x7fbac80b1840 msgr2=0x7fbac80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:18.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.091+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 shutdown_connections 2024-09-16T15:45:18.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.091+0000 7fbad0600640 1 -- 172.21.15.60:0/3124825552 wait complete. 2024-09-16T15:45:18.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:45:18.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.275+0000 7f5f0c400640 1 -- 172.21.15.60:0/1603627134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 msgr2=0x7f5f04073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:18.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.275+0000 7f5f0c400640 1 --2- 172.21.15.60:0/1603627134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 0x7f5f04073d50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f5f0406aa00 tx=0x7f5ef002a560 comp rx=0 tx=0).stop 2024-09-16T15:45:18.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.276+0000 7f5f0c400640 1 -- 172.21.15.60:0/1603627134 shutdown_connections 2024-09-16T15:45:18.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.276+0000 7f5f0c400640 1 --2- 172.21.15.60:0/1603627134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 0x7f5f04073d50 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:18.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.276+0000 7f5f0c400640 1 --2- 172.21.15.60:0/1603627134 >> v2:172.21.15.100:3300/0 conn(0x7f5f0415bd60 0x7f5f040cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:18.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.276+0000 7f5f0c400640 1 -- 172.21.15.60:0/1603627134 >> 172.21.15.60:0/1603627134 conn(0x7f5f040b18e0 msgr2=0x7f5f040b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:18.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.276+0000 7f5f0c400640 1 -- 172.21.15.60:0/1603627134 shutdown_connections 2024-09-16T15:45:18.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.276+0000 7f5f0c400640 1 -- 172.21.15.60:0/1603627134 wait complete. 2024-09-16T15:45:18.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.277+0000 7f5f0c400640 1 Processor -- start 2024-09-16T15:45:18.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.277+0000 7f5f0c400640 1 -- start start 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.278+0000 7f5f0c400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 0x7f5f041f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.278+0000 7f5f0c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f04074290 0x7f5f04158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.278+0000 7f5f0c400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0415bd60 0x7f5f04156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.278+0000 7f5f0c400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0415ff70 con 0x7f5f0406e910 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.278+0000 7f5f0c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5f040749c0 con 0x7f5f04074290 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.278+0000 7f5f0c400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5f041600f0 con 0x7f5f0415bd60 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 0x7f5f041f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 0x7f5f041f5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43300/0 (socket says 172.21.15.60:43300) 2024-09-16T15:45:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 -- 172.21.15.60:0/3770030126 learned_addr learned my addr 172.21.15.60:0/3770030126 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09000640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f04074290 0x7f5f04158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f0a400640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0415bd60 0x7f5f04156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 -- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0415bd60 msgr2=0x7f5f04156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0415bd60 0x7f5f04156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 -- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f04074290 msgr2=0x7f5f04158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f04074290 0x7f5f04158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 -- 172.21.15.60:0/3770030126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f04159710 con 0x7f5f0406e910 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09000640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f04074290 0x7f5f04158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.279+0000 7f5f09a00640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 0x7f5f041f5e00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800d6d0 tx=0x7f5ef800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.280+0000 7f5f01600640 1 -- 172.21.15.60:0/3770030126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8017070 con 0x7f5f0406e910 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.280+0000 7f5f01600640 1 -- 172.21.15.60:0/3770030126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ef8005330 con 0x7f5f0406e910 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.280+0000 7f5f01600640 1 -- 172.21.15.60:0/3770030126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80097c0 con 0x7f5f0406e910 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.280+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f041f6340 con 0x7f5f0406e910 2024-09-16T15:45:18.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.280+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f04159430 con 0x7f5f0406e910 2024-09-16T15:45:18.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.281+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f04069360 con 0x7f5f0406e910 2024-09-16T15:45:18.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.282+0000 7f5f01600640 1 -- 172.21.15.60:0/3770030126 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f5ef8009b70 con 0x7f5f0406e910 2024-09-16T15:45:18.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.283+0000 7f5f01600640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ee4078d00 0x7f5ee407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:18.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.283+0000 7f5f01600640 1 -- 172.21.15.60:0/3770030126 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5881+0+0 (secure 0 0 0) 0x7f5ef809b530 con 0x7f5f0406e910 2024-09-16T15:45:18.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.283+0000 7f5f09000640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ee4078d00 0x7f5ee407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:18.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.284+0000 7f5f09000640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ee4078d00 0x7f5ee407b1c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0033000 tx=0x7f5ef0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:18.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.287+0000 7f5f01600640 1 -- 172.21.15.60:0/3770030126 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5f04159710 con 0x7f5f0406e910 2024-09-16T15:45:18.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:18.485+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5f040cbab0 con 0x7f5f0406e910 2024-09-16T15:45:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.094+0000 7f5f01600640 1 -- 172.21.15.60:0/3770030126 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7f5f040cbab0 con 0x7f5f0406e910 2024-09-16T15:45:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.098+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ee4078d00 msgr2=0x7f5ee407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.098+0000 7f5f0c400640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ee4078d00 0x7f5ee407b1c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0033000 tx=0x7f5ef0004730 comp rx=0 tx=0).stop 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.098+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 msgr2=0x7f5f041f5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.098+0000 7f5f0c400640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 0x7f5f041f5e00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800d6d0 tx=0x7f5ef800db90 comp rx=0 tx=0).stop 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.099+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 shutdown_connections 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.099+0000 7f5f0c400640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ee4078d00 0x7f5ee407b1c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.099+0000 7f5f0c400640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0415bd60 0x7f5f04156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.099+0000 7f5f0c400640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f04074290 0x7f5f04158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.099+0000 7f5f0c400640 1 --2- 172.21.15.60:0/3770030126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0406e910 0x7f5f041f5e00 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.099+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 >> 172.21.15.60:0/3770030126 conn(0x7f5f040b18e0 msgr2=0x7f5f040b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:19.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.099+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 shutdown_connections 2024-09-16T15:45:19.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.099+0000 7f5f0c400640 1 -- 172.21.15.60:0/3770030126 wait complete. 2024-09-16T15:45:19.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:45:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.290+0000 7fa01c400640 1 -- 172.21.15.60:0/1368963312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc004800 msgr2=0x7f9ffc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.290+0000 7fa01c400640 1 --2- 172.21.15.60:0/1368963312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ffc004800 0x7f9ffc004c80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa000009b50 tx=0x7f9ffc005830 comp rx=0 tx=0).stop 2024-09-16T15:45:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.290+0000 7fa01a400640 1 -- 172.21.15.60:0/1368963312 reap_dead start 2024-09-16T15:45:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.291+0000 7fa01c400640 1 -- 172.21.15.60:0/1368963312 shutdown_connections 2024-09-16T15:45:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.291+0000 7fa01c400640 1 -- 172.21.15.60:0/1368963312 >> 172.21.15.60:0/1368963312 conn(0x7fa0140b18e0 msgr2=0x7fa0140b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:19.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.291+0000 7fa01c400640 1 -- 172.21.15.60:0/1368963312 shutdown_connections 2024-09-16T15:45:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.291+0000 7fa01c400640 1 -- 172.21.15.60:0/1368963312 wait complete. 2024-09-16T15:45:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.292+0000 7fa01c400640 1 Processor -- start 2024-09-16T15:45:19.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.292+0000 7fa01c400640 1 -- start start 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.293+0000 7fa01c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc004800 0x7fa0141f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.293+0000 7fa01c400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0140d04d0 0x7fa0140ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.293+0000 7fa01c400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa014154240 0x7fa01414adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.293+0000 7fa01c400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa01406faf0 con 0x7fa014154240 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.293+0000 7fa01c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa01406f970 con 0x7f9ffc004800 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.293+0000 7fa01c400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa0140cc190 con 0x7fa0140d04d0 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa014154240 0x7fa01414adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa014154240 0x7fa01414adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46946/0 (socket says 172.21.15.60:46946) 2024-09-16T15:45:19.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 -- 172.21.15.60:0/4087322528 learned_addr learned my addr 172.21.15.60:0/4087322528 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa019000640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0140d04d0 0x7fa0140ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa019a00640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc004800 0x7fa0141f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 -- 172.21.15.60:0/4087322528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0140d04d0 msgr2=0x7fa0140ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0140d04d0 0x7fa0140ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 -- 172.21.15.60:0/4087322528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc004800 msgr2=0x7fa0141f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc004800 0x7fa0141f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 -- 172.21.15.60:0/4087322528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0140cdad0 con 0x7fa014154240 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.294+0000 7fa01a400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa014154240 0x7fa01414adb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa0140af550 tx=0x7fa00c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.295+0000 7fa011600640 1 -- 172.21.15.60:0/4087322528 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c004b20 con 0x7fa014154240 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.295+0000 7fa011600640 1 -- 172.21.15.60:0/4087322528 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa00c004cc0 con 0x7fa014154240 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.295+0000 7fa011600640 1 -- 172.21.15.60:0/4087322528 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c003bd0 con 0x7fa014154240 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.295+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0140cddb0 con 0x7fa014154240 2024-09-16T15:45:19.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.295+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0140ce370 con 0x7fa014154240 2024-09-16T15:45:19.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.296+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa014069360 con 0x7fa014154240 2024-09-16T15:45:19.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.297+0000 7fa011600640 1 -- 172.21.15.60:0/4087322528 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fa00c004030 con 0x7fa014154240 2024-09-16T15:45:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.299+0000 7fa011600640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9ff4078d00 0x7f9ff407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.299+0000 7fa011600640 1 -- 172.21.15.60:0/4087322528 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5443+0+0 (secure 0 0 0) 0x7fa00c09b580 con 0x7fa014154240 2024-09-16T15:45:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.299+0000 7fa019a00640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9ff4078d00 0x7f9ff407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:19.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.300+0000 7fa019a00640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9ff4078d00 0x7f9ff407b1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa008001860 tx=0x7fa008001200 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:19.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.303+0000 7fa011600640 1 -- 172.21.15.60:0/4087322528 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa00c063520 con 0x7fa014154240 2024-09-16T15:45:19.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:19.504+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa014152d90 con 0x7fa014154240 2024-09-16T15:45:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.096+0000 7fa011600640 1 -- 172.21.15.60:0/4087322528 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7fa00c062e20 con 0x7fa014154240 2024-09-16T15:45:20.097 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.100+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9ff4078d00 msgr2=0x7f9ff407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.100+0000 7fa01c400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9ff4078d00 0x7f9ff407b1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa008001860 tx=0x7fa008001200 comp rx=0 tx=0).stop 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.100+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa014154240 msgr2=0x7fa01414adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.100+0000 7fa01c400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa014154240 0x7fa01414adb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa0140af550 tx=0x7fa00c00f3c0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.101+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 shutdown_connections 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.101+0000 7fa01c400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9ff4078d00 0x7f9ff407b1c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.101+0000 7fa01c400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa014154240 0x7fa01414adb0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.101+0000 7fa01c400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0140d04d0 0x7fa0140ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.101+0000 7fa01c400640 1 --2- 172.21.15.60:0/4087322528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc004800 0x7fa0141f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.101+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 >> 172.21.15.60:0/4087322528 conn(0x7fa0140b18e0 msgr2=0x7fa0140ad340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:20.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.101+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 shutdown_connections 2024-09-16T15:45:20.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.101+0000 7fa01c400640 1 -- 172.21.15.60:0/4087322528 wait complete. 2024-09-16T15:45:20.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:45:20.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.285+0000 7f4ea3e00640 1 -- 172.21.15.60:0/1153776498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 msgr2=0x7f4e84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.285+0000 7f4ea3e00640 1 --2- 172.21.15.60:0/1153776498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 0x7f4e84004c80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4e88001830 tx=0x7f4e84005830 comp rx=0 tx=0).stop 2024-09-16T15:45:20.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.285+0000 7f4ea1e00640 1 -- 172.21.15.60:0/1153776498 reap_dead start 2024-09-16T15:45:20.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.286+0000 7f4ea3e00640 1 -- 172.21.15.60:0/1153776498 shutdown_connections 2024-09-16T15:45:20.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.286+0000 7f4ea3e00640 1 -- 172.21.15.60:0/1153776498 >> 172.21.15.60:0/1153776498 conn(0x7f4e9c0b1950 msgr2=0x7f4e9c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:20.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.286+0000 7f4ea3e00640 1 -- 172.21.15.60:0/1153776498 shutdown_connections 2024-09-16T15:45:20.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.286+0000 7f4ea3e00640 1 -- 172.21.15.60:0/1153776498 wait complete. 2024-09-16T15:45:20.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.287+0000 7f4ea3e00640 1 Processor -- start 2024-09-16T15:45:20.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.288+0000 7f4ea3e00640 1 -- start start 2024-09-16T15:45:20.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.288+0000 7f4ea3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 0x7f4e9c09c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:20.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.288+0000 7f4ea3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e9c0cc700 0x7f4e9c15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:20.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea3e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e9c15e640 0x7f4e9c1f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:20.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea3e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c074840 con 0x7f4e84004800 2024-09-16T15:45:20.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c0746c0 con 0x7f4e9c0cc700 2024-09-16T15:45:20.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea3e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9c06f160 con 0x7f4e9c15e640 2024-09-16T15:45:20.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 0x7f4e9c09c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 0x7f4e9c09c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46970/0 (socket says 172.21.15.60:46970) 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1400640 1 -- 172.21.15.60:0/3512514560 learned_addr learned my addr 172.21.15.60:0/3512514560 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea0a00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e9c0cc700 0x7f4e9c15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1e00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e9c15e640 0x7f4e9c1f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1400640 1 -- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e9c15e640 msgr2=0x7f4e9c1f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1400640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e9c15e640 0x7f4e9c1f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1400640 1 -- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e9c0cc700 msgr2=0x7f4e9c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1400640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e9c0cc700 0x7f4e9c15e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1400640 1 -- 172.21.15.60:0/3512514560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c09cbf0 con 0x7f4e84004800 2024-09-16T15:45:20.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.289+0000 7f4ea1e00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e9c15e640 0x7f4e9c1f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:20.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.290+0000 7f4ea1400640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 0x7f4e9c09c6b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000e610 tx=0x7f4e9000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:20.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.290+0000 7f4e99600640 1 -- 172.21.15.60:0/3512514560 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90015070 con 0x7f4e84004800 2024-09-16T15:45:20.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.290+0000 7f4e99600640 1 -- 172.21.15.60:0/3512514560 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e90010980 con 0x7f4e84004800 2024-09-16T15:45:20.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.290+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c09ad30 con 0x7f4e84004800 2024-09-16T15:45:20.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.290+0000 7f4e99600640 1 -- 172.21.15.60:0/3512514560 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90010dd0 con 0x7f4e84004800 2024-09-16T15:45:20.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.290+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c09b2f0 con 0x7f4e84004800 2024-09-16T15:45:20.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.291+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9c06f650 con 0x7f4e84004800 2024-09-16T15:45:20.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.292+0000 7f4e99600640 1 -- 172.21.15.60:0/3512514560 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f4e90018070 con 0x7f4e84004800 2024-09-16T15:45:20.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.294+0000 7f4e99600640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e7c078cd0 0x7f4e7c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:20.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.294+0000 7f4e99600640 1 -- 172.21.15.60:0/3512514560 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7f4e9009b2a0 con 0x7f4e84004800 2024-09-16T15:45:20.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.294+0000 7f4ea0a00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e7c078cd0 0x7f4e7c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:20.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.295+0000 7f4ea0a00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e7c078cd0 0x7f4e7c07b190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4e88032000 tx=0x7f4e88001360 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:20.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.298+0000 7f4e99600640 1 -- 172.21.15.60:0/3512514560 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e90062e20 con 0x7f4e84004800 2024-09-16T15:45:20.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.495+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f4e9c15cb90 con 0x7f4e84004800 2024-09-16T15:45:20.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.498+0000 7f4e99600640 1 -- 172.21.15.60:0/3512514560 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7f4e90067850 con 0x7f4e84004800 2024-09-16T15:45:20.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e7c078cd0 msgr2=0x7f4e7c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e7c078cd0 0x7f4e7c07b190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4e88032000 tx=0x7f4e88001360 comp rx=0 tx=0).stop 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 msgr2=0x7f4e9c09c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 0x7f4e9c09c6b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000e610 tx=0x7f4e9000ead0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 shutdown_connections 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e7c078cd0 0x7f4e7c07b190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e9c15e640 0x7f4e9c1f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e9c0cc700 0x7f4e9c15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 --2- 172.21.15.60:0/3512514560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e84004800 0x7f4e9c09c6b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 >> 172.21.15.60:0/3512514560 conn(0x7f4e9c0b1950 msgr2=0x7f4e9c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 shutdown_connections 2024-09-16T15:45:20.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.501+0000 7f4ea3e00640 1 -- 172.21.15.60:0/3512514560 wait complete. 2024-09-16T15:45:20.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-16T15:45:20.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.678+0000 7f43c2e00640 1 -- 172.21.15.60:0/1487038512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 msgr2=0x7f43a4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.678+0000 7f43c2e00640 1 --2- 172.21.15.60:0/1487038512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 0x7f43a4003e70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f43b0006fb0 tx=0x7f43a4005830 comp rx=0 tx=0).stop 2024-09-16T15:45:20.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.678+0000 7f43c2400640 1 -- 172.21.15.60:0/1487038512 reap_dead start 2024-09-16T15:45:20.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.678+0000 7f43c2e00640 1 -- 172.21.15.60:0/1487038512 shutdown_connections 2024-09-16T15:45:20.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.678+0000 7f43c2e00640 1 -- 172.21.15.60:0/1487038512 >> 172.21.15.60:0/1487038512 conn(0x7f43bc0cfa20 msgr2=0x7f43bc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:20.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.678+0000 7f43c2e00640 1 -- 172.21.15.60:0/1487038512 shutdown_connections 2024-09-16T15:45:20.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.678+0000 7f43c2e00640 1 -- 172.21.15.60:0/1487038512 wait complete. 2024-09-16T15:45:20.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.680+0000 7f43c2e00640 1 Processor -- start 2024-09-16T15:45:20.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.680+0000 7f43c2e00640 1 -- start start 2024-09-16T15:45:20.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c2e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 0x7f43bc0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:20.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c2e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43bc0b0940 0x7f43bc1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:20.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c2e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43bc1f8310 0x7f43bc15a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:20.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c2e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc074660 con 0x7f43a4003a50 2024-09-16T15:45:20.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c2e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc06f320 con 0x7f43bc1f8310 2024-09-16T15:45:20.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c2e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f43bc0abb60 con 0x7f43bc0b0940 2024-09-16T15:45:20.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c1a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 0x7f43bc0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:20.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c1a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 0x7f43bc0b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46982/0 (socket says 172.21.15.60:46982) 2024-09-16T15:45:20.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.681+0000 7f43c1a00640 1 -- 172.21.15.60:0/3049705017 learned_addr learned my addr 172.21.15.60:0/3049705017 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c1000640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43bc0b0940 0x7f43bc1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c2400640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43bc1f8310 0x7f43bc15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c1a00640 1 -- 172.21.15.60:0/3049705017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43bc0b0940 msgr2=0x7f43bc1511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c1a00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43bc0b0940 0x7f43bc1511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c1a00640 1 -- 172.21.15.60:0/3049705017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43bc1f8310 msgr2=0x7f43bc15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c1a00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43bc1f8310 0x7f43bc15a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c1a00640 1 -- 172.21.15.60:0/3049705017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc0c9460 con 0x7f43a4003a50 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c1a00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 0x7f43bc0b1080 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c350 tx=0x7f43b0001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43b9600640 1 -- 172.21.15.60:0/3049705017 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0031040 con 0x7f43a4003a50 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43b9600640 1 -- 172.21.15.60:0/3049705017 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43b0003800 con 0x7f43a4003a50 2024-09-16T15:45:20.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc0c9700 con 0x7f43a4003a50 2024-09-16T15:45:20.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.682+0000 7f43b9600640 1 -- 172.21.15.60:0/3049705017 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0003c30 con 0x7f43a4003a50 2024-09-16T15:45:20.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.683+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc1f7b10 con 0x7f43a4003a50 2024-09-16T15:45:20.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.684+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43bc159dd0 con 0x7f43a4003a50 2024-09-16T15:45:20.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.685+0000 7f43b9600640 1 -- 172.21.15.60:0/3049705017 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f43b0034070 con 0x7f43a4003a50 2024-09-16T15:45:20.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.685+0000 7f43b9600640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f439c078cd0 0x7f439c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:20.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.686+0000 7f43b9600640 1 -- 172.21.15.60:0/3049705017 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7f43b00bb1c0 con 0x7f43a4003a50 2024-09-16T15:45:20.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.686+0000 7f43c1000640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f439c078cd0 0x7f439c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:20.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.686+0000 7f43c1000640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f439c078cd0 0x7f439c07b190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f43bc1f2070 tx=0x7f43a8005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:20.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.690+0000 7f43b9600640 1 -- 172.21.15.60:0/3049705017 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f43b0083410 con 0x7f43a4003a50 2024-09-16T15:45:20.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:20.964+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f43bc06fcc0 con 0x7f43a4003a50 2024-09-16T15:45:21.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.101+0000 7f43b9600640 1 -- 172.21.15.60:0/3049705017 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v372) ==== 149+0+0 (secure 0 0 0) 0x7f43b0082d10 con 0x7f43a4003a50 2024-09-16T15:45:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f439c078cd0 msgr2=0x7f439c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f439c078cd0 0x7f439c07b190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f43bc1f2070 tx=0x7f43a8005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 msgr2=0x7f43bc0b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 0x7f43bc0b1080 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c350 tx=0x7f43b0001cc0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 shutdown_connections 2024-09-16T15:45:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f439c078cd0 0x7f439c07b190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43bc1f8310 0x7f43bc15a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43bc0b0940 0x7f43bc1511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 --2- 172.21.15.60:0/3049705017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43a4003a50 0x7f43bc0b1080 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 >> 172.21.15.60:0/3049705017 conn(0x7f43bc0cfa20 msgr2=0x7f43bc158460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 shutdown_connections 2024-09-16T15:45:21.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.104+0000 7f43c2e00640 1 -- 172.21.15.60:0/3049705017 wait complete. 2024-09-16T15:45:21.122 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-16T15:45:21.122 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-16T15:45:21.122 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:45:21.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.281+0000 7fd42f200640 1 -- 172.21.15.60:0/1831741142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b2780 msgr2=0x7fd430168c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.281+0000 7fd42f200640 1 --2- 172.21.15.60:0/1831741142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b2780 0x7fd430168c30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd4300d4ea0 tx=0x7fd42802aaa0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.282+0000 7fd42f200640 1 -- 172.21.15.60:0/1831741142 shutdown_connections 2024-09-16T15:45:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.282+0000 7fd42f200640 1 --2- 172.21.15.60:0/1831741142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b2780 0x7fd430168c30 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.282+0000 7fd42f200640 1 -- 172.21.15.60:0/1831741142 >> 172.21.15.60:0/1831741142 conn(0x7fd4300d3df0 msgr2=0x7fd4300d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.282+0000 7fd42f200640 1 -- 172.21.15.60:0/1831741142 shutdown_connections 2024-09-16T15:45:21.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.282+0000 7fd42f200640 1 -- 172.21.15.60:0/1831741142 wait complete. 2024-09-16T15:45:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.283+0000 7fd42f200640 1 Processor -- start 2024-09-16T15:45:21.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.283+0000 7fd42f200640 1 -- start start 2024-09-16T15:45:21.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.284+0000 7fd42f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4300b2780 0x7fd430154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.284+0000 7fd42f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b5370 0x7fd430150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.284+0000 7fd42f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd430161440 0x7fd430150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.284+0000 7fd42f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301692f0 con 0x7fd4300b5370 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.284+0000 7fd42f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd430169170 con 0x7fd430161440 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.284+0000 7fd42f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd430169470 con 0x7fd4300b2780 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.284+0000 7fd427e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b5370 0x7fd430150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd427e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b5370 0x7fd430150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46996/0 (socket says 172.21.15.60:46996) 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd427e00640 1 -- 172.21.15.60:0/1051345407 learned_addr learned my addr 172.21.15.60:0/1051345407 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd42e800640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd430161440 0x7fd430150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd42de00640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4300b2780 0x7fd430154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd427e00640 1 -- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4300b2780 msgr2=0x7fd430154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd427e00640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4300b2780 0x7fd430154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd427e00640 1 -- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd430161440 msgr2=0x7fd430150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd427e00640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd430161440 0x7fd430150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd427e00640 1 -- 172.21.15.60:0/1051345407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43014b8e0 con 0x7fd4300b5370 2024-09-16T15:45:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd42e800640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd430161440 0x7fd430150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd427e00640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b5370 0x7fd430150110 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd41800c530 tx=0x7fd41800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd426000640 1 -- 172.21.15.60:0/1051345407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4180052d0 con 0x7fd4300b5370 2024-09-16T15:45:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd426000640 1 -- 172.21.15.60:0/1051345407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd418005470 con 0x7fd4300b5370 2024-09-16T15:45:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.285+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4301533c0 con 0x7fd4300b5370 2024-09-16T15:45:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.286+0000 7fd426000640 1 -- 172.21.15.60:0/1051345407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418012870 con 0x7fd4300b5370 2024-09-16T15:45:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.286+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd430153980 con 0x7fd4300b5370 2024-09-16T15:45:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.287+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4300b34b0 con 0x7fd4300b5370 2024-09-16T15:45:21.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.288+0000 7fd426000640 1 -- 172.21.15.60:0/1051345407 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fd4180035f0 con 0x7fd4300b5370 2024-09-16T15:45:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.289+0000 7fd426000640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd40c078cd0 0x7fd40c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.289+0000 7fd426000640 1 -- 172.21.15.60:0/1051345407 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7fd41809aaa0 con 0x7fd4300b5370 2024-09-16T15:45:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.289+0000 7fd42de00640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd40c078cd0 0x7fd40c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.290+0000 7fd42de00640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd40c078cd0 0x7fd40c07b190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd420000f60 tx=0x7fd420009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:21.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.293+0000 7fd426000640 1 -- 172.21.15.60:0/1051345407 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd418062d10 con 0x7fd4300b5370 2024-09-16T15:45:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.485+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4301557e0 con 0x7fd4300b5370 2024-09-16T15:45:21.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.486+0000 7fd426000640 1 -- 172.21.15.60:0/1051345407 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+13086 (secure 0 0 0) 0x7fd418062610 con 0x7fd4300b5370 2024-09-16T15:45:21.487 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:21.487 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":113,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:20.094728+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6801/515865896":"2024-09-17T15:45:17.084892+0000","172.21.15.100:6800/515865896":"2024-09-17T15:45:17.084892+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd40c078cd0 msgr2=0x7fd40c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd40c078cd0 0x7fd40c07b190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd420000f60 tx=0x7fd420009040 comp rx=0 tx=0).stop 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b5370 msgr2=0x7fd430150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b5370 0x7fd430150110 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd41800c530 tx=0x7fd41800c9f0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 shutdown_connections 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd40c078cd0 0x7fd40c07b190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd430161440 0x7fd430150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4300b5370 0x7fd430150110 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 --2- 172.21.15.60:0/1051345407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4300b2780 0x7fd430154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 >> 172.21.15.60:0/1051345407 conn(0x7fd4300d3df0 msgr2=0x7fd43006dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 shutdown_connections 2024-09-16T15:45:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.489+0000 7fd42f200640 1 -- 172.21.15.60:0/1051345407 wait complete. 2024-09-16T15:45:21.506 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:45:21.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:45:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.670+0000 7f88a4a00640 1 -- 172.21.15.60:0/773524135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a00cccd0 msgr2=0x7f88a0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.670+0000 7f88a4a00640 1 --2- 172.21.15.60:0/773524135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a00cccd0 0x7f88a0073440 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f888c009830 tx=0x7f888c02a480 comp rx=0 tx=0).stop 2024-09-16T15:45:21.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.670+0000 7f88a4a00640 1 -- 172.21.15.60:0/773524135 shutdown_connections 2024-09-16T15:45:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.670+0000 7f88a4a00640 1 --2- 172.21.15.60:0/773524135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a006efa0 0x7f88a00cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.670+0000 7f88a4a00640 1 --2- 172.21.15.60:0/773524135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a00cccd0 0x7f88a0073440 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.670+0000 7f88a4a00640 1 -- 172.21.15.60:0/773524135 >> 172.21.15.60:0/773524135 conn(0x7f88a00b1840 msgr2=0x7f88a00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.670+0000 7f88a4a00640 1 -- 172.21.15.60:0/773524135 shutdown_connections 2024-09-16T15:45:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.670+0000 7f88a4a00640 1 -- 172.21.15.60:0/773524135 wait complete. 2024-09-16T15:45:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.672+0000 7f88a4a00640 1 Processor -- start 2024-09-16T15:45:21.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.672+0000 7f88a4a00640 1 -- start start 2024-09-16T15:45:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f88a4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a006efa0 0x7f88a015e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f88a4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a00cccd0 0x7f88a015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f88a4a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0145750 0x7f88a013fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:21.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f88a4a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00ce480 con 0x7f88a006efa0 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f88a4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88a00ce300 con 0x7f88a00cccd0 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f88a4a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88a00ce600 con 0x7f88a0145750 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a006efa0 0x7f88a015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a006efa0 0x7f88a015e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47004/0 (socket says 172.21.15.60:47004) 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889de00640 1 -- 172.21.15.60:0/599114469 learned_addr learned my addr 172.21.15.60:0/599114469 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889e800640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0145750 0x7f88a013fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889d400640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a00cccd0 0x7f88a015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889de00640 1 -- 172.21.15.60:0/599114469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0145750 msgr2=0x7f88a013fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889de00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0145750 0x7f88a013fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889de00640 1 -- 172.21.15.60:0/599114469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a00cccd0 msgr2=0x7f88a015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889de00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a00cccd0 0x7f88a015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.673+0000 7f889de00640 1 -- 172.21.15.60:0/599114469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a0142d70 con 0x7f88a006efa0 2024-09-16T15:45:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.674+0000 7f889de00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a006efa0 0x7f88a015e670 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f889000c430 tx=0x7f889000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.674+0000 7f8896000640 1 -- 172.21.15.60:0/599114469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890011790 con 0x7f88a006efa0 2024-09-16T15:45:21.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.674+0000 7f8896000640 1 -- 172.21.15.60:0/599114469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8890017070 con 0x7f88a006efa0 2024-09-16T15:45:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.674+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a00aef30 con 0x7f88a006efa0 2024-09-16T15:45:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.674+0000 7f8896000640 1 -- 172.21.15.60:0/599114469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890011ce0 con 0x7f88a006efa0 2024-09-16T15:45:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.674+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a0142a90 con 0x7f88a006efa0 2024-09-16T15:45:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.675+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a00cdbb0 con 0x7f88a006efa0 2024-09-16T15:45:21.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.676+0000 7f8896000640 1 -- 172.21.15.60:0/599114469 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f889001c460 con 0x7f88a006efa0 2024-09-16T15:45:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.677+0000 7f8896000640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8880078cd0 0x7f888007b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.677+0000 7f8896000640 1 -- 172.21.15.60:0/599114469 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7f889009ad10 con 0x7f88a006efa0 2024-09-16T15:45:21.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.677+0000 7f889d400640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8880078cd0 0x7f888007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:21.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.678+0000 7f889d400640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8880078cd0 0x7f888007b190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f888c034000 tx=0x7f888c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:21.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.681+0000 7f8896000640 1 -- 172.21.15.60:0/599114469 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8890062ef0 con 0x7f88a006efa0 2024-09-16T15:45:21.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.873+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f88a00cf640 con 0x7f88a006efa0 2024-09-16T15:45:21.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.874+0000 7f8896000640 1 -- 172.21.15.60:0/599114469 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f88900627f0 con 0x7f88a006efa0 2024-09-16T15:45:21.875 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:45:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8880078cd0 msgr2=0x7f888007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8880078cd0 0x7f888007b190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f888c034000 tx=0x7f888c004730 comp rx=0 tx=0).stop 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a006efa0 msgr2=0x7f88a015e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a006efa0 0x7f88a015e670 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f889000c430 tx=0x7f889000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 shutdown_connections 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8880078cd0 0x7f888007b190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0145750 0x7f88a013fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a00cccd0 0x7f88a015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 --2- 172.21.15.60:0/599114469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a006efa0 0x7f88a015e670 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 >> 172.21.15.60:0/599114469 conn(0x7f88a00b1840 msgr2=0x7f88a00b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 shutdown_connections 2024-09-16T15:45:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:21.876+0000 7f88a4a00640 1 -- 172.21.15.60:0/599114469 wait complete. 2024-09-16T15:45:21.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-16T15:45:22.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.053+0000 7f6cf8000640 1 -- 172.21.15.60:0/1397638888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf00b73c0 msgr2=0x7f6cf0071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.053+0000 7f6cf8000640 1 --2- 172.21.15.60:0/1397638888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf00b73c0 0x7f6cf0071bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009be0 tx=0x7f6ce402a820 comp rx=0 tx=0).stop 2024-09-16T15:45:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.053+0000 7f6cf8000640 1 -- 172.21.15.60:0/1397638888 shutdown_connections 2024-09-16T15:45:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.053+0000 7f6cf8000640 1 --2- 172.21.15.60:0/1397638888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf00b73c0 0x7f6cf0071bf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.053+0000 7f6cf8000640 1 -- 172.21.15.60:0/1397638888 >> 172.21.15.60:0/1397638888 conn(0x7f6cf006d230 msgr2=0x7f6cf00688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.053+0000 7f6cf8000640 1 -- 172.21.15.60:0/1397638888 shutdown_connections 2024-09-16T15:45:22.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.053+0000 7f6cf8000640 1 -- 172.21.15.60:0/1397638888 wait complete. 2024-09-16T15:45:22.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.055+0000 7f6cf8000640 1 Processor -- start 2024-09-16T15:45:22.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.055+0000 7f6cf8000640 1 -- start start 2024-09-16T15:45:22.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf8000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0071260 0x7f6cf0152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:22.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf8000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf00b73c0 0x7f6cf00c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf8000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00cf170 0x7f6cf00c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf8000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf00d2350 con 0x7f6cf00cf170 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf8000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf00b79b0 con 0x7f6cf00b73c0 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf8000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf00d24d0 con 0x7f6cf0071260 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00cf170 0x7f6cf00c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00cf170 0x7f6cf00c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47016/0 (socket says 172.21.15.60:47016) 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.056+0000 7f6cf6000640 1 -- 172.21.15.60:0/1649991824 learned_addr learned my addr 172.21.15.60:0/1649991824 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6cf5600640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0071260 0x7f6cf0152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:22.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6cf4c00640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf00b73c0 0x7f6cf00c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6cf6000640 1 -- 172.21.15.60:0/1649991824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0071260 msgr2=0x7f6cf0152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6cf6000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0071260 0x7f6cf0152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6cf6000640 1 -- 172.21.15.60:0/1649991824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf00b73c0 msgr2=0x7f6cf00c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6cf6000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf00b73c0 0x7f6cf00c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6cf6000640 1 -- 172.21.15.60:0/1649991824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf01f4880 con 0x7f6cf00cf170 2024-09-16T15:45:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6cf6000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00cf170 0x7f6cf00c79e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800c500 tx=0x7f6ce800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:22.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.057+0000 7f6ced600640 1 -- 172.21.15.60:0/1649991824 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8016020 con 0x7f6cf00cf170 2024-09-16T15:45:22.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.058+0000 7f6ced600640 1 -- 172.21.15.60:0/1649991824 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ce8012940 con 0x7f6cf00cf170 2024-09-16T15:45:22.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.058+0000 7f6ced600640 1 -- 172.21.15.60:0/1649991824 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8012d90 con 0x7f6cf00cf170 2024-09-16T15:45:22.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.058+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf01f4b40 con 0x7f6cf00cf170 2024-09-16T15:45:22.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.058+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf01f5100 con 0x7f6cf00cf170 2024-09-16T15:45:22.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.059+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf00c7280 con 0x7f6cf00cf170 2024-09-16T15:45:22.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.060+0000 7f6ced600640 1 -- 172.21.15.60:0/1649991824 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f6ce8019020 con 0x7f6cf00cf170 2024-09-16T15:45:22.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.061+0000 7f6ced600640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6cd0078cd0 0x7f6cd007b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:22.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.061+0000 7f6ced600640 1 -- 172.21.15.60:0/1649991824 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7f6ce809ac00 con 0x7f6cf00cf170 2024-09-16T15:45:22.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.061+0000 7f6cf5600640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6cd0078cd0 0x7f6cd007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:22.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.062+0000 7f6cf5600640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6cd0078cd0 0x7f6cd007b190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4033000 tx=0x7f6ce4004ab0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:22.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.065+0000 7f6ced600640 1 -- 172.21.15.60:0/1649991824 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6ce8020030 con 0x7f6cf00cf170 2024-09-16T15:45:22.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:22.335+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f6cf01607b0 con 0x7f6cf00cf170 2024-09-16T15:45:23.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.108+0000 7f6ced600640 1 -- 172.21.15.60:0/1649991824 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v374) ==== 151+0+0 (secure 0 0 0) 0x7f6ce8062700 con 0x7f6cf00cf170 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6cd0078cd0 msgr2=0x7f6cd007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6cd0078cd0 0x7f6cd007b190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4033000 tx=0x7f6ce4004ab0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00cf170 msgr2=0x7f6cf00c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00cf170 0x7f6cf00c79e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800c500 tx=0x7f6ce800c9c0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 shutdown_connections 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6cd0078cd0 0x7f6cd007b190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cf00cf170 0x7f6cf00c79e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cf00b73c0 0x7f6cf00c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 --2- 172.21.15.60:0/1649991824 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cf0071260 0x7f6cf0152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 >> 172.21.15.60:0/1649991824 conn(0x7f6cf006d230 msgr2=0x7f6cf00b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:23.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 shutdown_connections 2024-09-16T15:45:23.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.113+0000 7f6cf8000640 1 -- 172.21.15.60:0/1649991824 wait complete. 2024-09-16T15:45:23.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:45:23.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.296+0000 7f8697c00640 1 -- 172.21.15.60:0/525500401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8678004800 msgr2=0x7f8678004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.296+0000 7f8697c00640 1 --2- 172.21.15.60:0/525500401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8678004800 0x7f8678004c80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c290 tx=0x7f8678005830 comp rx=0 tx=0).stop 2024-09-16T15:45:23.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.296+0000 7f8695c00640 1 -- 172.21.15.60:0/525500401 reap_dead start 2024-09-16T15:45:23.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.296+0000 7f8697c00640 1 -- 172.21.15.60:0/525500401 shutdown_connections 2024-09-16T15:45:23.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.296+0000 7f8697c00640 1 -- 172.21.15.60:0/525500401 >> 172.21.15.60:0/525500401 conn(0x7f86900b2230 msgr2=0x7f86900b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:23.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.296+0000 7f8697c00640 1 -- 172.21.15.60:0/525500401 shutdown_connections 2024-09-16T15:45:23.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.296+0000 7f8697c00640 1 -- 172.21.15.60:0/525500401 wait complete. 2024-09-16T15:45:23.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.297+0000 7f8697c00640 1 Processor -- start 2024-09-16T15:45:23.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.298+0000 7f8697c00640 1 -- start start 2024-09-16T15:45:23.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.298+0000 7f8697c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8678004800 0x7f869013d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:23.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.298+0000 7f8697c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86901f50f0 0x7f8690141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:23.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.298+0000 7f8697c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8690142020 0x7f86900b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:23.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8697c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f86900737d0 con 0x7f8690142020 2024-09-16T15:45:23.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8697c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8690068c10 con 0x7f86901f50f0 2024-09-16T15:45:23.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8697c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8690073950 con 0x7f8678004800 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8690142020 0x7f86900b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8690142020 0x7f86900b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47028/0 (socket says 172.21.15.60:47028) 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 -- 172.21.15.60:0/4251876573 learned_addr learned my addr 172.21.15.60:0/4251876573 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695200640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8678004800 0x7f869013d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f868fe00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86901f50f0 0x7f8690141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 -- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8678004800 msgr2=0x7f869013d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8678004800 0x7f869013d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 -- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86901f50f0 msgr2=0x7f8690141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86901f50f0 0x7f8690141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 -- 172.21.15.60:0/4251876573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86901f8070 con 0x7f8690142020 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695200640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8678004800 0x7f869013d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:23.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.299+0000 7f8695c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8690142020 0x7f86900b1020 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8688010c80 tx=0x7f868800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:23.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.300+0000 7f868cc00640 1 -- 172.21.15.60:0/4251876573 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688003b70 con 0x7f8690142020 2024-09-16T15:45:23.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.300+0000 7f868cc00640 1 -- 172.21.15.60:0/4251876573 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8688015070 con 0x7f8690142020 2024-09-16T15:45:23.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.300+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86901f8350 con 0x7f8690142020 2024-09-16T15:45:23.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.300+0000 7f868cc00640 1 -- 172.21.15.60:0/4251876573 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688010e90 con 0x7f8690142020 2024-09-16T15:45:23.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.300+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901f8910 con 0x7f8690142020 2024-09-16T15:45:23.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.301+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f869006cfd0 con 0x7f8690142020 2024-09-16T15:45:23.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.302+0000 7f868cc00640 1 -- 172.21.15.60:0/4251876573 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f8688017020 con 0x7f8690142020 2024-09-16T15:45:23.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.303+0000 7f868cc00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8670078e70 0x7f867007b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:23.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.304+0000 7f868cc00640 1 -- 172.21.15.60:0/4251876573 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7f868809b7c0 con 0x7f8690142020 2024-09-16T15:45:23.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.304+0000 7f8695200640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8670078e70 0x7f867007b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:23.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.304+0000 7f8695200640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8670078e70 0x7f867007b330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86900763d0 tx=0x7f8684005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:23.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.308+0000 7f868cc00640 1 -- 172.21.15.60:0/4251876573 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8688063870 con 0x7f8690142020 2024-09-16T15:45:23.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.506+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f86900af9b0 con 0x7f8690142020 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.507+0000 7f868cc00640 1 -- 172.21.15.60:0/4251876573 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+1998 (secure 0 0 0) 0x7f8688063170 con 0x7f8690142020 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:45:23.508 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:45:23.509 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:45:23.510 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:45:23.510 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:45:23.510 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:45:23.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8670078e70 msgr2=0x7f867007b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8670078e70 0x7f867007b330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f86900763d0 tx=0x7f8684005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8690142020 msgr2=0x7f86900b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8690142020 0x7f86900b1020 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8688010c80 tx=0x7f868800e720 comp rx=0 tx=0).stop 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 shutdown_connections 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8670078e70 0x7f867007b330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8690142020 0x7f86900b1020 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86901f50f0 0x7f8690141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 --2- 172.21.15.60:0/4251876573 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8678004800 0x7f869013d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 >> 172.21.15.60:0/4251876573 conn(0x7f86900b2230 msgr2=0x7f86900cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 shutdown_connections 2024-09-16T15:45:23.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.509+0000 7f8697c00640 1 -- 172.21.15.60:0/4251876573 wait complete. 2024-09-16T15:45:23.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:45:23.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.690+0000 7fc323e00640 1 -- 172.21.15.60:0/1301962760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 msgr2=0x7fc3240d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.690+0000 7fc323e00640 1 --2- 172.21.15.60:0/1301962760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 0x7fc3240d2830 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc31c005fd0 tx=0x7fc31c02aa30 comp rx=0 tx=0).stop 2024-09-16T15:45:23.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.690+0000 7fc323e00640 1 -- 172.21.15.60:0/1301962760 shutdown_connections 2024-09-16T15:45:23.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.690+0000 7fc323e00640 1 --2- 172.21.15.60:0/1301962760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 0x7fc3240d2830 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.690+0000 7fc323e00640 1 --2- 172.21.15.60:0/1301962760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324161460 0x7fc3240d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.690+0000 7fc323e00640 1 -- 172.21.15.60:0/1301962760 >> 172.21.15.60:0/1301962760 conn(0x7fc32409d0b0 msgr2=0x7fc3240d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:23.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.690+0000 7fc323e00640 1 -- 172.21.15.60:0/1301962760 shutdown_connections 2024-09-16T15:45:23.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.690+0000 7fc323e00640 1 -- 172.21.15.60:0/1301962760 wait complete. 2024-09-16T15:45:23.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.692+0000 7fc323e00640 1 Processor -- start 2024-09-16T15:45:23.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.692+0000 7fc323e00640 1 -- start start 2024-09-16T15:45:23.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc323e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 0x7fc32414e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:23.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc323e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240b5480 0x7fc32409db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:23.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc323e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324161460 0x7fc32409e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:23.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc323e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240cf0b0 con 0x7fc3240b43c0 2024-09-16T15:45:23.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc323e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240cef30 con 0x7fc3240b5480 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc323e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc3240cf230 con 0x7fc324161460 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc322a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 0x7fc32414e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc322a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 0x7fc32414e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47058/0 (socket says 172.21.15.60:47058) 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.693+0000 7fc322a00640 1 -- 172.21.15.60:0/428812751 learned_addr learned my addr 172.21.15.60:0/428812751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc323400640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324161460 0x7fc32409e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc31a000640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240b5480 0x7fc32409db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc322a00640 1 -- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324161460 msgr2=0x7fc32409e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc322a00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324161460 0x7fc32409e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc322a00640 1 -- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240b5480 msgr2=0x7fc32409db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc322a00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240b5480 0x7fc32409db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc322a00640 1 -- 172.21.15.60:0/428812751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc324150f40 con 0x7fc3240b43c0 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc323400640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324161460 0x7fc32409e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc31a000640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240b5480 0x7fc32409db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:23.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc322a00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 0x7fc32414e450 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc31400d740 tx=0x7fc31400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc31b400640 1 -- 172.21.15.60:0/428812751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314017070 con 0x7fc3240b43c0 2024-09-16T15:45:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc31b400640 1 -- 172.21.15.60:0/428812751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc314009d20 con 0x7fc3240b43c0 2024-09-16T15:45:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc31b400640 1 -- 172.21.15.60:0/428812751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140106c0 con 0x7fc3240b43c0 2024-09-16T15:45:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc324151cc0 con 0x7fc3240b43c0 2024-09-16T15:45:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.694+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3241514c0 con 0x7fc3240b43c0 2024-09-16T15:45:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.695+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3240cf750 con 0x7fc3240b43c0 2024-09-16T15:45:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.697+0000 7fc31b400640 1 -- 172.21.15.60:0/428812751 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fc31401a070 con 0x7fc3240b43c0 2024-09-16T15:45:23.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.698+0000 7fc31b400640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078cd0 0x7fc30007b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:23.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.698+0000 7fc31b400640 1 -- 172.21.15.60:0/428812751 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7fc31409b160 con 0x7fc3240b43c0 2024-09-16T15:45:23.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.698+0000 7fc31a000640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078cd0 0x7fc30007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:23.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.699+0000 7fc31a000640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078cd0 0x7fc30007b190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc32415dca0 tx=0x7fc30c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:23.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.702+0000 7fc31b400640 1 -- 172.21.15.60:0/428812751 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc3140632a0 con 0x7fc3240b43c0 2024-09-16T15:45:23.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.896+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc324050f70 con 0x7fc3240b43c0 2024-09-16T15:45:23.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.900+0000 7fc31b400640 1 -- 172.21.15.60:0/428812751 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7fc314062ba0 con 0x7fc3240b43c0 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.902+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078cd0 msgr2=0x7fc30007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.902+0000 7fc323e00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078cd0 0x7fc30007b190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc32415dca0 tx=0x7fc30c009040 comp rx=0 tx=0).stop 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.902+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 msgr2=0x7fc32414e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.902+0000 7fc323e00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 0x7fc32414e450 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc31400d740 tx=0x7fc31400dc00 comp rx=0 tx=0).stop 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.903+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 shutdown_connections 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.903+0000 7fc323e00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc300078cd0 0x7fc30007b190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.903+0000 7fc323e00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc324161460 0x7fc32409e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.903+0000 7fc323e00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3240b5480 0x7fc32409db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.903+0000 7fc323e00640 1 --2- 172.21.15.60:0/428812751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3240b43c0 0x7fc32414e450 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:23.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.903+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 >> 172.21.15.60:0/428812751 conn(0x7fc32409d0b0 msgr2=0x7fc3240cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:23.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.903+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 shutdown_connections 2024-09-16T15:45:23.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:23.903+0000 7fc323e00640 1 -- 172.21.15.60:0/428812751 wait complete. 2024-09-16T15:45:23.920 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:45:23.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:45:24.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:45:24.114 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:45:24.115 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:24.115 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:24.150 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:45:24.150 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:24.150 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:24.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:45:24.188 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:24.188 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:24.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:45:24.226 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:24.226 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:24.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:45:24.264 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:45:24.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:24.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.425+0000 7f7af3e00640 1 -- 172.21.15.60:0/3507025602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af40725d0 msgr2=0x7f7af40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.425+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3507025602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af40725d0 0x7f7af40b28f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7adc005fd0 tx=0x7f7adc02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.425+0000 7f7af3e00640 1 -- 172.21.15.60:0/3507025602 shutdown_connections 2024-09-16T15:45:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.425+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3507025602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af40725d0 0x7f7af40b28f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.425+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3507025602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7af40d0540 0x7f7af406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.425+0000 7f7af3e00640 1 -- 172.21.15.60:0/3507025602 >> 172.21.15.60:0/3507025602 conn(0x7f7af40cfa20 msgr2=0x7f7af40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.425+0000 7f7af3e00640 1 -- 172.21.15.60:0/3507025602 shutdown_connections 2024-09-16T15:45:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.425+0000 7f7af3e00640 1 -- 172.21.15.60:0/3507025602 wait complete. 2024-09-16T15:45:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.427+0000 7f7af3e00640 1 Processor -- start 2024-09-16T15:45:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.427+0000 7f7af3e00640 1 -- start start 2024-09-16T15:45:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af40d0540 0x7f7af414e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7af415bc30 0x7f7af40b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af4157b40 0x7f7af4157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4074660 con 0x7f7af4157b40 2024-09-16T15:45:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7af406f860 con 0x7f7af40d0540 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7af41584f0 con 0x7f7af415bc30 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af4157b40 0x7f7af4157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af4157b40 0x7f7af4157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47076/0 (socket says 172.21.15.60:47076) 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af3400640 1 -- 172.21.15.60:0/3508519751 learned_addr learned my addr 172.21.15.60:0/3508519751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7aea000640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7af415bc30 0x7f7af40b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.428+0000 7f7af2a00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af40d0540 0x7f7af414e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7af3400640 1 -- 172.21.15.60:0/3508519751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7af415bc30 msgr2=0x7f7af40b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7af3400640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7af415bc30 0x7f7af40b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7af3400640 1 -- 172.21.15.60:0/3508519751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af40d0540 msgr2=0x7f7af414e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7af3400640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af40d0540 0x7f7af414e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7af3400640 1 -- 172.21.15.60:0/3508519751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af41f7e10 con 0x7f7af4157b40 2024-09-16T15:45:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7af3400640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af4157b40 0x7f7af4157fb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00eb10 tx=0x7f7aec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7aeb400640 1 -- 172.21.15.60:0/3508519751 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004b20 con 0x7f7af4157b40 2024-09-16T15:45:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7aeb400640 1 -- 172.21.15.60:0/3508519751 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7aec004ce0 con 0x7f7af4157b40 2024-09-16T15:45:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7aeb400640 1 -- 172.21.15.60:0/3508519751 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec005280 con 0x7f7af4157b40 2024-09-16T15:45:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af41f80f0 con 0x7f7af4157b40 2024-09-16T15:45:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.429+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af41f86b0 con 0x7f7af4157b40 2024-09-16T15:45:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.430+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af406fdf0 con 0x7f7af4157b40 2024-09-16T15:45:24.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.431+0000 7f7aeb400640 1 -- 172.21.15.60:0/3508519751 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f7aec005420 con 0x7f7af4157b40 2024-09-16T15:45:24.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.432+0000 7f7aeb400640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ad0078cd0 0x7f7ad007b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:24.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.433+0000 7f7aeb400640 1 -- 172.21.15.60:0/3508519751 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7f7aec09bd60 con 0x7f7af4157b40 2024-09-16T15:45:24.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.433+0000 7f7af2a00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ad0078cd0 0x7f7ad007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:24.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.433+0000 7f7af2a00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ad0078cd0 0x7f7ad007b190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40015e0 tx=0x7f7ae4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:24.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.436+0000 7f7aeb400640 1 -- 172.21.15.60:0/3508519751 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7aec063ea0 con 0x7f7af4157b40 2024-09-16T15:45:24.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.670+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7af40cffa0 con 0x7f7af4157b40 2024-09-16T15:45:24.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.671+0000 7f7aeb400640 1 -- 172.21.15.60:0/3508519751 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+3914 (secure 0 0 0) 0x7f7aec0637a0 con 0x7f7af4157b40 2024-09-16T15:45:24.672 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:24.672 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":74,"btime":"2024-09-16T15:45:18:087015+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/586556953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":586556953},{"type":"v1","addr":"172.21.15.100:6835","nonce":586556953}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2314857147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2314857147},{"type":"v1","addr":"172.21.15.60:6834","nonce":2314857147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1301436173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1301436173},{"type":"v1","addr":"172.21.15.60:6835","nonce":1301436173}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6464,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/3011062349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3011062349},{"type":"v1","addr":"172.21.15.100:6801","nonce":3011062349}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[]} 2024-09-16T15:45:24.672 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 74 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.673+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ad0078cd0 msgr2=0x7f7ad007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.673+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ad0078cd0 0x7f7ad007b190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40015e0 tx=0x7f7ae4009040 comp rx=0 tx=0).stop 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.673+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af4157b40 msgr2=0x7f7af4157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.673+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af4157b40 0x7f7af4157fb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00eb10 tx=0x7f7aec00c2d0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.674+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 shutdown_connections 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.674+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ad0078cd0 0x7f7ad007b190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.674+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7af4157b40 0x7f7af4157fb0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.674+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7af415bc30 0x7f7af40b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.674+0000 7f7af3e00640 1 --2- 172.21.15.60:0/3508519751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7af40d0540 0x7f7af414e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.674+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 >> 172.21.15.60:0/3508519751 conn(0x7f7af40cfa20 msgr2=0x7f7af40caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.674+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 shutdown_connections 2024-09-16T15:45:24.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.674+0000 7f7af3e00640 1 -- 172.21.15.60:0/3508519751 wait complete. 2024-09-16T15:45:24.691 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:45:24.691 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:45:24.691 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:45:24.691 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:45:24.691 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:45:24.691 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:45:24.692 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:45:24.692 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:45:24.692 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:45:24.692 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:45:24.692 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:45:24.692 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:45:24.692 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:45:24.692 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:45:24.693 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:45:24.693 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:45:24.707 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:45:24.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:45:24.708 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:45:24.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:45:24.709 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:45:24.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:45:24.710 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:45:24.710 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:45:24.750 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:45:24.751 DEBUG:teuthology.parallel:result is None 2024-09-16T15:45:24.753 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:45:24.753 DEBUG:teuthology.parallel:result is None 2024-09-16T15:45:24.753 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:45:24.754 DEBUG:teuthology.parallel:result is None 2024-09-16T15:45:24.756 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:45:24.756 DEBUG:teuthology.parallel:result is None 2024-09-16T15:45:24.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:45:24.883 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:45:24.886 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:45:24.887 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:45:24.887 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:45:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.924+0000 7f3c21800640 1 -- 172.21.15.60:0/1100002071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0b2440 msgr2=0x7f3c1c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.924+0000 7f3c21800640 1 --2- 172.21.15.60:0/1100002071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0b2440 0x7f3c1c06f160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009be0 tx=0x7f3c1402a820 comp rx=0 tx=0).stop 2024-09-16T15:45:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.925+0000 7f3c21800640 1 -- 172.21.15.60:0/1100002071 shutdown_connections 2024-09-16T15:45:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.925+0000 7f3c21800640 1 --2- 172.21.15.60:0/1100002071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0b2440 0x7f3c1c06f160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.925+0000 7f3c21800640 1 -- 172.21.15.60:0/1100002071 >> 172.21.15.60:0/1100002071 conn(0x7f3c1c0b1950 msgr2=0x7f3c1c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.925+0000 7f3c21800640 1 -- 172.21.15.60:0/1100002071 shutdown_connections 2024-09-16T15:45:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.925+0000 7f3c21800640 1 -- 172.21.15.60:0/1100002071 wait complete. 2024-09-16T15:45:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.926+0000 7f3c21800640 1 Processor -- start 2024-09-16T15:45:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.926+0000 7f3c21800640 1 -- start start 2024-09-16T15:45:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.927+0000 7f3c21800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c1c0b2440 0x7f3c1c1f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.927+0000 7f3c21800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0ccf00 0x7f3c1c0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:24.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.927+0000 7f3c21800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c15f5a0 0x7f3c1c1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.927+0000 7f3c21800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c074840 con 0x7f3c1c15f5a0 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.927+0000 7f3c21800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c0746c0 con 0x7f3c1c0b2440 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.927+0000 7f3c21800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c06f6a0 con 0x7f3c1c0ccf00 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c1c0b2440 0x7f3c1c1f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1a000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0ccf00 0x7f3c1c0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c1c0b2440 0x7f3c1c1f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:49082/0 (socket says 172.21.15.60:49082) 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1aa00640 1 -- 172.21.15.60:0/3114473274 learned_addr learned my addr 172.21.15.60:0/3114473274 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1a000640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0ccf00 0x7f3c1c0b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59596/0 (socket says 172.21.15.60:59596) 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1b400640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c15f5a0 0x7f3c1c1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1aa00640 1 -- 172.21.15.60:0/3114473274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0ccf00 msgr2=0x7f3c1c0b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1aa00640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0ccf00 0x7f3c1c0b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1aa00640 1 -- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c15f5a0 msgr2=0x7f3c1c1539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:24.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1aa00640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c15f5a0 0x7f3c1c1539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.928+0000 7f3c1aa00640 1 -- 172.21.15.60:0/3114473274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c0aff90 con 0x7f3c1c0b2440 2024-09-16T15:45:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.929+0000 7f3c1aa00640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c1c0b2440 0x7f3c1c1f52b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00c4f0 tx=0x7f3c0c00c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.929+0000 7f3c12a00640 1 -- 172.21.15.60:0/3114473274 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c016020 con 0x7f3c1c0b2440 2024-09-16T15:45:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.929+0000 7f3c12a00640 1 -- 172.21.15.60:0/3114473274 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c0c012920 con 0x7f3c1c0b2440 2024-09-16T15:45:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.929+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c0b0270 con 0x7f3c1c0b2440 2024-09-16T15:45:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.929+0000 7f3c12a00640 1 -- 172.21.15.60:0/3114473274 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c012d50 con 0x7f3c1c0b2440 2024-09-16T15:45:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.929+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c0b0830 con 0x7f3c1c0b2440 2024-09-16T15:45:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.930+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c0b1570 con 0x7f3c1c0b2440 2024-09-16T15:45:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.931+0000 7f3c12a00640 1 -- 172.21.15.60:0/3114473274 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f3c0c019020 con 0x7f3c1c0b2440 2024-09-16T15:45:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.932+0000 7f3c12a00640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3bfc078d00 0x7f3bfc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.932+0000 7f3c12a00640 1 -- 172.21.15.60:0/3114473274 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4971+0+0 (secure 0 0 0) 0x7f3c0c09af60 con 0x7f3c1c0b2440 2024-09-16T15:45:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.932+0000 7f3c1a000640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3bfc078d00 0x7f3bfc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.933+0000 7f3c1a000640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3bfc078d00 0x7f3bfc07b1c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c1f7d80 tx=0x7f3c08009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:24.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:24.937+0000 7f3c12a00640 1 -- 172.21.15.60:0/3114473274 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3c0c020020 con 0x7f3c1c0b2440 2024-09-16T15:45:25.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:25.149+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3c1c06f160 con 0x7f3c1c0b2440 2024-09-16T15:45:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.139+0000 7f3c12a00640 1 -- 172.21.15.60:0/3114473274 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v118) ==== 97+0+0 (secure 0 0 0) 0x7f3c0c062a50 con 0x7f3c1c0b2440 2024-09-16T15:45:26.140 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3bfc078d00 msgr2=0x7f3bfc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3bfc078d00 0x7f3bfc07b1c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c1f7d80 tx=0x7f3c08009040 comp rx=0 tx=0).stop 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c1c0b2440 msgr2=0x7f3c1c1f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c1c0b2440 0x7f3c1c1f52b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00c4f0 tx=0x7f3c0c00c9b0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 shutdown_connections 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3bfc078d00 0x7f3bfc07b1c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c1c15f5a0 0x7f3c1c1539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c1c0ccf00 0x7f3c1c0b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 --2- 172.21.15.60:0/3114473274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c1c0b2440 0x7f3c1c1f52b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 >> 172.21.15.60:0/3114473274 conn(0x7f3c1c0b1950 msgr2=0x7f3c1c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.142+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 shutdown_connections 2024-09-16T15:45:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.143+0000 7f3c21800640 1 -- 172.21.15.60:0/3114473274 wait complete. 2024-09-16T15:45:26.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:45:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.334+0000 7f6b29600640 1 -- 172.21.15.60:0/2593852294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 msgr2=0x7f6b24073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.334+0000 7f6b29600640 1 --2- 172.21.15.60:0/2593852294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 0x7f6b24073fb0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6b24075130 tx=0x7f6b1c02a980 comp rx=0 tx=0).stop 2024-09-16T15:45:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.334+0000 7f6b29600640 1 -- 172.21.15.60:0/2593852294 shutdown_connections 2024-09-16T15:45:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.334+0000 7f6b29600640 1 --2- 172.21.15.60:0/2593852294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 0x7f6b24073fb0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.334+0000 7f6b29600640 1 --2- 172.21.15.60:0/2593852294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b240ccf00 0x7f6b240acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.334+0000 7f6b29600640 1 --2- 172.21.15.60:0/2593852294 >> v2:172.21.15.100:3300/0 conn(0x7f6b2415f5a0 0x7f6b240b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.334+0000 7f6b29600640 1 -- 172.21.15.60:0/2593852294 >> 172.21.15.60:0/2593852294 conn(0x7f6b240b1950 msgr2=0x7f6b240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.335+0000 7f6b29600640 1 -- 172.21.15.60:0/2593852294 shutdown_connections 2024-09-16T15:45:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.335+0000 7f6b29600640 1 -- 172.21.15.60:0/2593852294 wait complete. 2024-09-16T15:45:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.336+0000 7f6b29600640 1 Processor -- start 2024-09-16T15:45:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.336+0000 7f6b29600640 1 -- start start 2024-09-16T15:45:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.337+0000 7f6b29600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 0x7f6b24154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.337+0000 7f6b29600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b240ccf00 0x7f6b241f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.337+0000 7f6b29600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b2415f5a0 0x7f6b2415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.337+0000 7f6b29600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2406f160 con 0x7f6b240ad270 2024-09-16T15:45:26.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.337+0000 7f6b29600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6b24074780 con 0x7f6b2415f5a0 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.337+0000 7f6b29600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2406f2e0 con 0x7f6b240ccf00 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 0x7f6b24154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 0x7f6b24154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47138/0 (socket says 172.21.15.60:47138) 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 -- 172.21.15.60:0/2817014862 learned_addr learned my addr 172.21.15.60:0/2817014862 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b1a000640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b240ccf00 0x7f6b241f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b23400640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b2415f5a0 0x7f6b2415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 -- 172.21.15.60:0/2817014862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b240ccf00 msgr2=0x7f6b241f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b240ccf00 0x7f6b241f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 -- 172.21.15.60:0/2817014862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b2415f5a0 msgr2=0x7f6b2415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b2415f5a0 0x7f6b2415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 -- 172.21.15.60:0/2817014862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b241569e0 con 0x7f6b240ad270 2024-09-16T15:45:26.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.338+0000 7f6b22a00640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 0x7f6b24154280 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d770 tx=0x7f6b1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.339+0000 7f6b1b400640 1 -- 172.21.15.60:0/2817014862 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14017070 con 0x7f6b240ad270 2024-09-16T15:45:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.339+0000 7f6b1b400640 1 -- 172.21.15.60:0/2817014862 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b14009d20 con 0x7f6b240ad270 2024-09-16T15:45:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.339+0000 7f6b1b400640 1 -- 172.21.15.60:0/2817014862 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b140106c0 con 0x7f6b240ad270 2024-09-16T15:45:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.339+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b24156cc0 con 0x7f6b240ad270 2024-09-16T15:45:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.339+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b24157280 con 0x7f6b240ad270 2024-09-16T15:45:26.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.340+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2406f870 con 0x7f6b240ad270 2024-09-16T15:45:26.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.341+0000 7f6b1b400640 1 -- 172.21.15.60:0/2817014862 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f6b1401a070 con 0x7f6b240ad270 2024-09-16T15:45:26.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.342+0000 7f6b1b400640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6b00078cd0 0x7f6b0007b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:26.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.342+0000 7f6b1b400640 1 -- 172.21.15.60:0/2817014862 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4885+0+0 (secure 0 0 0) 0x7f6b1409b0e0 con 0x7f6b240ad270 2024-09-16T15:45:26.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.342+0000 7f6b1a000640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6b00078cd0 0x7f6b0007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:26.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.343+0000 7f6b1a000640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6b00078cd0 0x7f6b0007b190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6b240cced0 tx=0x7f6b0c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:26.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.346+0000 7f6b1b400640 1 -- 172.21.15.60:0/2817014862 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6b14063270 con 0x7f6b240ad270 2024-09-16T15:45:26.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.618+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6b240b03f0 con 0x7f6b240ad270 2024-09-16T15:45:26.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.618+0000 7f6b1b400640 1 -- 172.21.15.60:0/2817014862 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f6b14062b70 con 0x7f6b240ad270 2024-09-16T15:45:26.619 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:26.619 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.620 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:45:26.621 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.622 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.623 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:45:26.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:45:26.626 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.627 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:45:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:45:26.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:45:26.630 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.631 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:45:26.632 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:45:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6b00078cd0 msgr2=0x7f6b0007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6b00078cd0 0x7f6b0007b190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6b240cced0 tx=0x7f6b0c005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:26.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 msgr2=0x7f6b24154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 0x7f6b24154280 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400d770 tx=0x7f6b1400dc30 comp rx=0 tx=0).stop 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 shutdown_connections 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6b00078cd0 0x7f6b0007b190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b2415f5a0 0x7f6b2415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b240ccf00 0x7f6b241f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 --2- 172.21.15.60:0/2817014862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b240ad270 0x7f6b24154280 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 >> 172.21.15.60:0/2817014862 conn(0x7f6b240b1950 msgr2=0x7f6b2409cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 shutdown_connections 2024-09-16T15:45:26.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.621+0000 7f6b29600640 1 -- 172.21.15.60:0/2817014862 wait complete. 2024-09-16T15:45:26.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:45:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.799+0000 7fb4d4a00640 1 -- 172.21.15.60:0/673463041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4b4003a50 msgr2=0x7fb4b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.799+0000 7fb4d4a00640 1 --2- 172.21.15.60:0/673463041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4b4003a50 0x7fb4b4003e70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0006fb0 tx=0x7fb4b4005830 comp rx=0 tx=0).stop 2024-09-16T15:45:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.799+0000 7fb4ce800640 1 -- 172.21.15.60:0/673463041 reap_dead start 2024-09-16T15:45:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.799+0000 7fb4d4a00640 1 -- 172.21.15.60:0/673463041 shutdown_connections 2024-09-16T15:45:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.799+0000 7fb4d4a00640 1 -- 172.21.15.60:0/673463041 >> 172.21.15.60:0/673463041 conn(0x7fb4d00b1950 msgr2=0x7fb4d00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.799+0000 7fb4d4a00640 1 -- 172.21.15.60:0/673463041 shutdown_connections 2024-09-16T15:45:26.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.799+0000 7fb4d4a00640 1 -- 172.21.15.60:0/673463041 wait complete. 2024-09-16T15:45:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.801+0000 7fb4d4a00640 1 Processor -- start 2024-09-16T15:45:26.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.801+0000 7fb4d4a00640 1 -- start start 2024-09-16T15:45:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b4003a50 0x7fb4d014c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4d01e56c0 0x7fb4d01e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4d01f4ba0 0x7fb4d01f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4d4a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d00745c0 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4d4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0074440 con 0x7fb4b4003a50 2024-09-16T15:45:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4d4a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d0074740 con 0x7fb4d01e56c0 2024-09-16T15:45:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4ce800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4d01f4ba0 0x7fb4d01f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:26.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4ce800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4d01f4ba0 0x7fb4d01f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47148/0 (socket says 172.21.15.60:47148) 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4ce800640 1 -- 172.21.15.60:0/2702356242 learned_addr learned my addr 172.21.15.60:0/2702356242 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.802+0000 7fb4cde00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b4003a50 0x7fb4d014c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4c5400640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4d01e56c0 0x7fb4d01e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4ce800640 1 -- 172.21.15.60:0/2702356242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4d01e56c0 msgr2=0x7fb4d01e5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4ce800640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4d01e56c0 0x7fb4d01e5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4ce800640 1 -- 172.21.15.60:0/2702356242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b4003a50 msgr2=0x7fb4d014c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4ce800640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b4003a50 0x7fb4d014c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4ce800640 1 -- 172.21.15.60:0/2702356242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d014f010 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4ce800640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4d01f4ba0 0x7fb4d01f5030 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb4d006e130 tx=0x7fb4c8011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4c6a00640 1 -- 172.21.15.60:0/2702356242 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800fd40 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4c6a00640 1 -- 172.21.15.60:0/2702356242 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4c8004b20 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4c6a00640 1 -- 172.21.15.60:0/2702356242 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8003970 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.803+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d014f2f0 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.804+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d014f8b0 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.804+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d014fab0 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.806+0000 7fb4c6a00640 1 -- 172.21.15.60:0/2702356242 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fb4c8020020 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.807+0000 7fb4c6a00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4ac078cd0 0x7fb4ac07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.807+0000 7fb4c6a00640 1 -- 172.21.15.60:0/2702356242 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4885+0+0 (secure 0 0 0) 0x7fb4c809b620 con 0x7fb4d01f4ba0 2024-09-16T15:45:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.807+0000 7fb4cde00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4ac078cd0 0x7fb4ac07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.807+0000 7fb4cde00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4ac078cd0 0x7fb4ac07b190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0033000 tx=0x7fb4c0038040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:26.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:26.811+0000 7fb4c6a00640 1 -- 172.21.15.60:0/2702356242 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb4d014fab0 con 0x7fb4d01f4ba0 2024-09-16T15:45:27.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.009+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb4d00b2440 con 0x7fb4d01f4ba0 2024-09-16T15:45:27.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.009+0000 7fb4c6a00640 1 -- 172.21.15.60:0/2702356242 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v118) ==== 84+0+65 (secure 0 0 0) 0x7fb4c80631f0 con 0x7fb4d01f4ba0 2024-09-16T15:45:27.010 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:27.011 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:45:27.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:27.011 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:45:27.011 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:45:27.011 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:27.011 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4ac078cd0 msgr2=0x7fb4ac07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4ac078cd0 0x7fb4ac07b190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0033000 tx=0x7fb4c0038040 comp rx=0 tx=0).stop 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4d01f4ba0 msgr2=0x7fb4d01f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4d01f4ba0 0x7fb4d01f5030 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb4d006e130 tx=0x7fb4c8011f90 comp rx=0 tx=0).stop 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 shutdown_connections 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb4ac078cd0 0x7fb4ac07b190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4d01f4ba0 0x7fb4d01f5030 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4d01e56c0 0x7fb4d01e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 --2- 172.21.15.60:0/2702356242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4b4003a50 0x7fb4d014c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 >> 172.21.15.60:0/2702356242 conn(0x7fb4d00b1950 msgr2=0x7fb4d00b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:27.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 shutdown_connections 2024-09-16T15:45:27.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.012+0000 7fb4d4a00640 1 -- 172.21.15.60:0/2702356242 wait complete. 2024-09-16T15:45:27.029 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:45:27.030 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:45:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.194+0000 7f74c4000640 1 -- 172.21.15.60:0/3606791441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 msgr2=0x7f74bc06eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.194+0000 7f74c4000640 1 --2- 172.21.15.60:0/3606791441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 0x7f74bc06eba0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f74b4005fd0 tx=0x7f74b402aa60 comp rx=0 tx=0).stop 2024-09-16T15:45:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.195+0000 7f74c4000640 1 -- 172.21.15.60:0/3606791441 shutdown_connections 2024-09-16T15:45:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.195+0000 7f74c4000640 1 --2- 172.21.15.60:0/3606791441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 0x7f74bc06eba0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.195+0000 7f74c4000640 1 --2- 172.21.15.60:0/3606791441 >> v2:172.21.15.100:3300/0 conn(0x7f74bc15f9a0 0x7f74bc15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.195+0000 7f74c4000640 1 -- 172.21.15.60:0/3606791441 >> 172.21.15.60:0/3606791441 conn(0x7f74bc0b18e0 msgr2=0x7f74bc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.195+0000 7f74c4000640 1 -- 172.21.15.60:0/3606791441 shutdown_connections 2024-09-16T15:45:27.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.195+0000 7f74c4000640 1 -- 172.21.15.60:0/3606791441 wait complete. 2024-09-16T15:45:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.196+0000 7f74c4000640 1 Processor -- start 2024-09-16T15:45:27.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.196+0000 7f74c4000640 1 -- start start 2024-09-16T15:45:27.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.197+0000 7f74c4000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 0x7f74bc1f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:27.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.197+0000 7f74c4000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74bc073de0 0x7f74bc1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.197+0000 7f74c4000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74bc15f9a0 0x7f74bc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.197+0000 7f74c4000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0cd550 con 0x7f74bc06e6f0 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.197+0000 7f74c4000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0cd3d0 con 0x7f74bc15f9a0 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.197+0000 7f74c4000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc06f0e0 con 0x7f74bc073de0 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.197+0000 7f74c1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 0x7f74bc1f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 0x7f74bc1f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47176/0 (socket says 172.21.15.60:47176) 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c1600640 1 -- 172.21.15.60:0/667953225 learned_addr learned my addr 172.21.15.60:0/667953225 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c2000640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74bc15f9a0 0x7f74bc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c0c00640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74bc073de0 0x7f74bc1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c1600640 1 -- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74bc073de0 msgr2=0x7f74bc1f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:27.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c1600640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74bc073de0 0x7f74bc1f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c1600640 1 -- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74bc15f9a0 msgr2=0x7f74bc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c1600640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74bc15f9a0 0x7f74bc0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c1600640 1 -- 172.21.15.60:0/667953225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc0adec0 con 0x7f74bc06e6f0 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c2000640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74bc15f9a0 0x7f74bc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c1600640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 0x7f74bc1f6300 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f74b000d780 tx=0x7f74b000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74b9600640 1 -- 172.21.15.60:0/667953225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0017070 con 0x7f74bc06e6f0 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74b9600640 1 -- 172.21.15.60:0/667953225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74b000de90 con 0x7f74bc06e6f0 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc0cfa10 con 0x7f74bc06e6f0 2024-09-16T15:45:27.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.198+0000 7f74b9600640 1 -- 172.21.15.60:0/667953225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b00099d0 con 0x7f74bc06e6f0 2024-09-16T15:45:27.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.199+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc1f6bc0 con 0x7f74bc06e6f0 2024-09-16T15:45:27.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.199+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc06f560 con 0x7f74bc06e6f0 2024-09-16T15:45:27.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.201+0000 7f74b9600640 1 -- 172.21.15.60:0/667953225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f74b0009b70 con 0x7f74bc06e6f0 2024-09-16T15:45:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.202+0000 7f74b9600640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f749c078cd0 0x7f749c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.202+0000 7f74b9600640 1 -- 172.21.15.60:0/667953225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4885+0+0 (secure 0 0 0) 0x7f74b009b370 con 0x7f74bc06e6f0 2024-09-16T15:45:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.202+0000 7f74c0c00640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f749c078cd0 0x7f749c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:27.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.203+0000 7f74c0c00640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f749c078cd0 0x7f749c07b190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74bc15bcf0 tx=0x7f74a8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:27.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.206+0000 7f74b9600640 1 -- 172.21.15.60:0/667953225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f74b0020020 con 0x7f74bc06e6f0 2024-09-16T15:45:27.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:27.402+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f74bc0cd7b0 con 0x7f74bc06e6f0 2024-09-16T15:45:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.146+0000 7f74b9600640 1 -- 172.21.15.60:0/667953225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v119) ==== 138+0+0 (secure 0 0 0) 0x7f74b0063560 con 0x7f74bc06e6f0 2024-09-16T15:45:28.147 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:45:28.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.150+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f749c078cd0 msgr2=0x7f749c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.150+0000 7f74c4000640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f749c078cd0 0x7f749c07b190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74bc15bcf0 tx=0x7f74a8009040 comp rx=0 tx=0).stop 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.150+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 msgr2=0x7f74bc1f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.150+0000 7f74c4000640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 0x7f74bc1f6300 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f74b000d780 tx=0x7f74b000dc40 comp rx=0 tx=0).stop 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.151+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 shutdown_connections 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.151+0000 7f74c4000640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f749c078cd0 0x7f749c07b190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.151+0000 7f74c4000640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74bc15f9a0 0x7f74bc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.151+0000 7f74c4000640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74bc073de0 0x7f74bc1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.151+0000 7f74c4000640 1 --2- 172.21.15.60:0/667953225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74bc06e6f0 0x7f74bc1f6300 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.151+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 >> 172.21.15.60:0/667953225 conn(0x7f74bc0b18e0 msgr2=0x7f74bc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:28.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.151+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 shutdown_connections 2024-09-16T15:45:28.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.151+0000 7f74c4000640 1 -- 172.21.15.60:0/667953225 wait complete. 2024-09-16T15:45:28.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:45:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.349+0000 7f836e800640 1 -- 172.21.15.60:0/871378657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83680b3430 msgr2=0x7f83680b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.349+0000 7f836e800640 1 --2- 172.21.15.60:0/871378657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83680b3430 0x7f83680b3870 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8350005fd0 tx=0x7f835002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.349+0000 7f836e800640 1 -- 172.21.15.60:0/871378657 shutdown_connections 2024-09-16T15:45:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.349+0000 7f836e800640 1 --2- 172.21.15.60:0/871378657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83680b3430 0x7f83680b3870 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.349+0000 7f836e800640 1 --2- 172.21.15.60:0/871378657 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8368154bc0 0x7f8368154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.349+0000 7f836e800640 1 -- 172.21.15.60:0/871378657 >> 172.21.15.60:0/871378657 conn(0x7f83680b2130 msgr2=0x7f83680b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:28.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.349+0000 7f836e800640 1 -- 172.21.15.60:0/871378657 shutdown_connections 2024-09-16T15:45:28.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.349+0000 7f836e800640 1 -- 172.21.15.60:0/871378657 wait complete. 2024-09-16T15:45:28.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.350+0000 7f836e800640 1 Processor -- start 2024-09-16T15:45:28.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.351+0000 7f836e800640 1 -- start start 2024-09-16T15:45:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.351+0000 7f836e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8368070490 0x7f83681561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.351+0000 7f836e800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83680b3430 0x7f83681f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:28.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.351+0000 7f836e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8368154bc0 0x7f8368153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.351+0000 7f836e800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f836815e9f0 con 0x7f8368154bc0 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.351+0000 7f836e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f836815e870 con 0x7f8368070490 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.351+0000 7f836e800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f836815eb70 con 0x7f83680b3430 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8368154bc0 0x7f8368153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8368154bc0 0x7f8368153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47206/0 (socket says 172.21.15.60:47206) 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367e00640 1 -- 172.21.15.60:0/736003675 learned_addr learned my addr 172.21.15.60:0/736003675 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f835ea00640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83680b3430 0x7f83681f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367400640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8368070490 0x7f83681561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367e00640 1 -- 172.21.15.60:0/736003675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83680b3430 msgr2=0x7f83681f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367e00640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83680b3430 0x7f83681f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367e00640 1 -- 172.21.15.60:0/736003675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8368070490 msgr2=0x7f83681561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367e00640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8368070490 0x7f83681561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.352+0000 7f8367e00640 1 -- 172.21.15.60:0/736003675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681f56d0 con 0x7f8368154bc0 2024-09-16T15:45:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.353+0000 7f8367e00640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8368154bc0 0x7f8368153f90 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f836000c3c0 tx=0x7f836000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:28.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.353+0000 7f835fe00640 1 -- 172.21.15.60:0/736003675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360017070 con 0x7f8368154bc0 2024-09-16T15:45:28.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.353+0000 7f835fe00640 1 -- 172.21.15.60:0/736003675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83600053c0 con 0x7f8368154bc0 2024-09-16T15:45:28.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.353+0000 7f835fe00640 1 -- 172.21.15.60:0/736003675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360003850 con 0x7f8368154bc0 2024-09-16T15:45:28.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.353+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8368159910 con 0x7f8368154bc0 2024-09-16T15:45:28.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.353+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8368158b20 con 0x7f8368154bc0 2024-09-16T15:45:28.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.354+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836815f090 con 0x7f8368154bc0 2024-09-16T15:45:28.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.357+0000 7f835fe00640 1 -- 172.21.15.60:0/736003675 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f836001a070 con 0x7f8368154bc0 2024-09-16T15:45:28.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.358+0000 7f835fe00640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8344078cd0 0x7f834407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:28.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.358+0000 7f835fe00640 1 -- 172.21.15.60:0/736003675 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5255+0+0 (secure 0 0 0) 0x7f836009a510 con 0x7f8368154bc0 2024-09-16T15:45:28.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.358+0000 7f8367400640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8344078cd0 0x7f834407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:28.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.359+0000 7f8367400640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8344078cd0 0x7f834407b190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f83681f8df0 tx=0x7f83580095e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:28.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.361+0000 7f835fe00640 1 -- 172.21.15.60:0/736003675 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8360021080 con 0x7f8368154bc0 2024-09-16T15:45:28.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:28.558+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8368075a30 con 0x7f8368154bc0 2024-09-16T15:45:29.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.154+0000 7f835fe00640 1 -- 172.21.15.60:0/736003675 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v120) ==== 170+0+0 (secure 0 0 0) 0x7f8360061fc0 con 0x7f8368154bc0 2024-09-16T15:45:29.155 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:45:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.157+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8344078cd0 msgr2=0x7f834407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.157+0000 7f836e800640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8344078cd0 0x7f834407b190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f83681f8df0 tx=0x7f83580095e0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.157+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8368154bc0 msgr2=0x7f8368153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.157+0000 7f836e800640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8368154bc0 0x7f8368153f90 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f836000c3c0 tx=0x7f836000c880 comp rx=0 tx=0).stop 2024-09-16T15:45:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.158+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 shutdown_connections 2024-09-16T15:45:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.158+0000 7f836e800640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8344078cd0 0x7f834407b190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.158+0000 7f836e800640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8368154bc0 0x7f8368153f90 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.158+0000 7f836e800640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83680b3430 0x7f83681f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.158+0000 7f836e800640 1 --2- 172.21.15.60:0/736003675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8368070490 0x7f83681561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.158+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 >> 172.21.15.60:0/736003675 conn(0x7f83680b2130 msgr2=0x7f83680d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.158+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 shutdown_connections 2024-09-16T15:45:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.158+0000 7f836e800640 1 -- 172.21.15.60:0/736003675 wait complete. 2024-09-16T15:45:29.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:45:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.358+0000 7f56d4400640 1 -- 172.21.15.60:0/3060068299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc06efa0 msgr2=0x7f56cc0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.358+0000 7f56d4400640 1 --2- 172.21.15.60:0/3060068299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc06efa0 0x7f56cc0cc5c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0cc550 tx=0x7f56b802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.358+0000 7f56d4400640 1 -- 172.21.15.60:0/3060068299 shutdown_connections 2024-09-16T15:45:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.358+0000 7f56d4400640 1 --2- 172.21.15.60:0/3060068299 >> v2:172.21.15.100:3300/0 conn(0x7f56cc0ccb00 0x7f56cc15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.358+0000 7f56d4400640 1 --2- 172.21.15.60:0/3060068299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc06efa0 0x7f56cc0cc5c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.358+0000 7f56d4400640 1 --2- 172.21.15.60:0/3060068299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc145750 0x7f56cc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.358+0000 7f56d4400640 1 -- 172.21.15.60:0/3060068299 >> 172.21.15.60:0/3060068299 conn(0x7f56cc0b1840 msgr2=0x7f56cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.359+0000 7f56d4400640 1 -- 172.21.15.60:0/3060068299 shutdown_connections 2024-09-16T15:45:29.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.359+0000 7f56d4400640 1 -- 172.21.15.60:0/3060068299 wait complete. 2024-09-16T15:45:29.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.360+0000 7f56d4400640 1 Processor -- start 2024-09-16T15:45:29.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.360+0000 7f56d4400640 1 -- start start 2024-09-16T15:45:29.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.361+0000 7f56d4400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc06efa0 0x7f56cc1f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:29.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.361+0000 7f56d4400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccb00 0x7f56cc0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:29.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.361+0000 7f56d4400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc145750 0x7f56cc1f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:29.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.361+0000 7f56d4400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc1600b0 con 0x7f56cc145750 2024-09-16T15:45:29.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.361+0000 7f56d4400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc15ff30 con 0x7f56cc06efa0 2024-09-16T15:45:29.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.361+0000 7f56d4400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f56cc160230 con 0x7f56cc0ccb00 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc145750 0x7f56cc1f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc145750 0x7f56cc1f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46692/0 (socket says 172.21.15.60:46692) 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 -- 172.21.15.60:0/427520191 learned_addr learned my addr 172.21.15.60:0/427520191 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d1a00640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc06efa0 0x7f56cc1f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d1000640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccb00 0x7f56cc0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 -- 172.21.15.60:0/427520191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccb00 msgr2=0x7f56cc0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccb00 0x7f56cc0b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 -- 172.21.15.60:0/427520191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc06efa0 msgr2=0x7f56cc1f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc06efa0 0x7f56cc1f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 -- 172.21.15.60:0/427520191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc13ec70 con 0x7f56cc145750 2024-09-16T15:45:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.362+0000 7f56d2400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc145750 0x7f56cc1f8630 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f56c400eaa0 tx=0x7f56c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.363+0000 7f56c9600640 1 -- 172.21.15.60:0/427520191 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004b20 con 0x7f56cc145750 2024-09-16T15:45:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.363+0000 7f56c9600640 1 -- 172.21.15.60:0/427520191 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56c4004db0 con 0x7f56cc145750 2024-09-16T15:45:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.363+0000 7f56c9600640 1 -- 172.21.15.60:0/427520191 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4005280 con 0x7f56cc145750 2024-09-16T15:45:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.363+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc13ef50 con 0x7f56cc145750 2024-09-16T15:45:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.363+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc13f510 con 0x7f56cc145750 2024-09-16T15:45:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.364+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc160710 con 0x7f56cc145750 2024-09-16T15:45:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.365+0000 7f56c9600640 1 -- 172.21.15.60:0/427520191 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f56c4005420 con 0x7f56cc145750 2024-09-16T15:45:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.366+0000 7f56c9600640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f56ac078cd0 0x7f56ac07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.366+0000 7f56c9600640 1 -- 172.21.15.60:0/427520191 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5637+0+0 (secure 0 0 0) 0x7f56c409c3b0 con 0x7f56cc145750 2024-09-16T15:45:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.367+0000 7f56d1a00640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f56ac078cd0 0x7f56ac07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:29.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.367+0000 7f56d1a00640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f56ac078cd0 0x7f56ac07b190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009b30 tx=0x7f56c0005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.370+0000 7f56c9600640 1 -- 172.21.15.60:0/427520191 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f56c4027080 con 0x7f56cc145750 2024-09-16T15:45:29.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:29.602+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f56cc0cf740 con 0x7f56cc145750 2024-09-16T15:45:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.156+0000 7f56c9600640 1 -- 172.21.15.60:0/427520191 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v77) ==== 174+0+0 (secure 0 0 0) 0x7f56c4063d10 con 0x7f56cc145750 2024-09-16T15:45:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 18 and data pool 19 2024-09-16T15:45:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.163+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f56ac078cd0 msgr2=0x7f56ac07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.163+0000 7f56d4400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f56ac078cd0 0x7f56ac07b190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009b30 tx=0x7f56c0005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.163+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc145750 msgr2=0x7f56cc1f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.163+0000 7f56d4400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc145750 0x7f56cc1f8630 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f56c400eaa0 tx=0x7f56c400ef60 comp rx=0 tx=0).stop 2024-09-16T15:45:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.164+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 shutdown_connections 2024-09-16T15:45:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.164+0000 7f56d4400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f56ac078cd0 0x7f56ac07b190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.164+0000 7f56d4400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56cc145750 0x7f56cc1f8630 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.164+0000 7f56d4400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56cc0ccb00 0x7f56cc0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.164+0000 7f56d4400640 1 --2- 172.21.15.60:0/427520191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56cc06efa0 0x7f56cc1f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.164+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 >> 172.21.15.60:0/427520191 conn(0x7f56cc0b1840 msgr2=0x7f56cc0b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.167+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 shutdown_connections 2024-09-16T15:45:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.167+0000 7f56d4400640 1 -- 172.21.15.60:0/427520191 wait complete. 2024-09-16T15:45:30.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:30.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.370+0000 7fa410a00640 1 -- 172.21.15.60:0/3924458597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa40c15f5a0 msgr2=0x7fa40c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.370+0000 7fa410a00640 1 --2- 172.21.15.60:0/3924458597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa40c15f5a0 0x7fa40c0b2440 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009830 tx=0x7fa3fc02a520 comp rx=0 tx=0).stop 2024-09-16T15:45:30.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.371+0000 7fa410a00640 1 -- 172.21.15.60:0/3924458597 shutdown_connections 2024-09-16T15:45:30.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.371+0000 7fa410a00640 1 --2- 172.21.15.60:0/3924458597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa40c15f5a0 0x7fa40c0b2440 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.371+0000 7fa410a00640 1 -- 172.21.15.60:0/3924458597 >> 172.21.15.60:0/3924458597 conn(0x7fa40c0b1950 msgr2=0x7fa40c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:30.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.371+0000 7fa410a00640 1 -- 172.21.15.60:0/3924458597 shutdown_connections 2024-09-16T15:45:30.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.371+0000 7fa410a00640 1 -- 172.21.15.60:0/3924458597 wait complete. 2024-09-16T15:45:30.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.372+0000 7fa410a00640 1 Processor -- start 2024-09-16T15:45:30.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.372+0000 7fa410a00640 1 -- start start 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.373+0000 7fa410a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa40c073fb0 0x7fa40c1f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.373+0000 7fa410a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c0ccf00 0x7fa40c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.373+0000 7fa410a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa40c15f5a0 0x7fa40c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.373+0000 7fa410a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c06f9f0 con 0x7fa40c0ccf00 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.373+0000 7fa410a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c06f870 con 0x7fa40c15f5a0 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.373+0000 7fa410a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c074420 con 0x7fa40c073fb0 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.373+0000 7fa401400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c0ccf00 0x7fa40c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa401400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c0ccf00 0x7fa40c156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46698/0 (socket says 172.21.15.60:46698) 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa401400640 1 -- 172.21.15.60:0/629584716 learned_addr learned my addr 172.21.15.60:0/629584716 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa40a800640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa40c15f5a0 0x7fa40c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:30.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa409e00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa40c073fb0 0x7fa40c1f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa401400640 1 -- 172.21.15.60:0/629584716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa40c073fb0 msgr2=0x7fa40c1f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa401400640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa40c073fb0 0x7fa40c1f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa401400640 1 -- 172.21.15.60:0/629584716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa40c15f5a0 msgr2=0x7fa40c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa401400640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa40c15f5a0 0x7fa40c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa401400640 1 -- 172.21.15.60:0/629584716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1f4940 con 0x7fa40c0ccf00 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.374+0000 7fa401400640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c0ccf00 0x7fa40c156950 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400c500 tx=0x7fa3f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.375+0000 7fa402a00640 1 -- 172.21.15.60:0/629584716 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4017070 con 0x7fa40c0ccf00 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.375+0000 7fa402a00640 1 -- 172.21.15.60:0/629584716 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3f40035f0 con 0x7fa40c0ccf00 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.375+0000 7fa402a00640 1 -- 172.21.15.60:0/629584716 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4003a40 con 0x7fa40c0ccf00 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.375+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1f4c20 con 0x7fa40c0ccf00 2024-09-16T15:45:30.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.375+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1f51e0 con 0x7fa40c0ccf00 2024-09-16T15:45:30.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.376+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c1f8980 con 0x7fa40c0ccf00 2024-09-16T15:45:30.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.377+0000 7fa402a00640 1 -- 172.21.15.60:0/629584716 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fa3f401a070 con 0x7fa40c0ccf00 2024-09-16T15:45:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.378+0000 7fa402a00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa3e8078cd0 0x7fa3e807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.378+0000 7fa402a00640 1 -- 172.21.15.60:0/629584716 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5737+0+0 (secure 0 0 0) 0x7fa3f409a690 con 0x7fa40c0ccf00 2024-09-16T15:45:30.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.378+0000 7fa409e00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa3e8078cd0 0x7fa3e807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:30.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.379+0000 7fa409e00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa3e8078cd0 0x7fa3e807b190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc034000 tx=0x7fa3fc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:30.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.382+0000 7fa402a00640 1 -- 172.21.15.60:0/629584716 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa3f4061f10 con 0x7fa40c0ccf00 2024-09-16T15:45:30.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.580+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa40c1f55e0 con 0x7fa40c0ccf00 2024-09-16T15:45:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.581+0000 7fa402a00640 1 -- 172.21.15.60:0/629584716 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16796 (secure 0 0 0) 0x7fa3f4066940 con 0x7fa40c0ccf00 2024-09-16T15:45:30.583 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:30.583 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":121,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:29.603267+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:45:27.412685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:45:28.568791+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa3e8078cd0 msgr2=0x7fa3e807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa3e8078cd0 0x7fa3e807b190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc034000 tx=0x7fa3fc004730 comp rx=0 tx=0).stop 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c0ccf00 msgr2=0x7fa40c156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c0ccf00 0x7fa40c156950 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400c500 tx=0x7fa3f400c9c0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 shutdown_connections 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa3e8078cd0 0x7fa3e807b190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa40c15f5a0 0x7fa40c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa40c0ccf00 0x7fa40c156950 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 --2- 172.21.15.60:0/629584716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa40c073fb0 0x7fa40c1f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.583+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 >> 172.21.15.60:0/629584716 conn(0x7fa40c0b1950 msgr2=0x7fa40c0b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:30.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.584+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 shutdown_connections 2024-09-16T15:45:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.584+0000 7fa410a00640 1 -- 172.21.15.60:0/629584716 wait complete. 2024-09-16T15:45:30.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.760+0000 7f45d2000640 1 -- 172.21.15.60:0/681023222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 msgr2=0x7f45cc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.760+0000 7f45d2000640 1 --2- 172.21.15.60:0/681023222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 0x7f45cc0b2440 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f45bc005fd0 tx=0x7f45bc02a540 comp rx=0 tx=0).stop 2024-09-16T15:45:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.760+0000 7f45d2000640 1 -- 172.21.15.60:0/681023222 shutdown_connections 2024-09-16T15:45:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.760+0000 7f45d2000640 1 --2- 172.21.15.60:0/681023222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45cc0ccf00 0x7f45cc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.760+0000 7f45d2000640 1 --2- 172.21.15.60:0/681023222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 0x7f45cc0b2440 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.760+0000 7f45d2000640 1 -- 172.21.15.60:0/681023222 >> 172.21.15.60:0/681023222 conn(0x7f45cc0b1950 msgr2=0x7f45cc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.760+0000 7f45d2000640 1 -- 172.21.15.60:0/681023222 shutdown_connections 2024-09-16T15:45:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.760+0000 7f45d2000640 1 -- 172.21.15.60:0/681023222 wait complete. 2024-09-16T15:45:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.762+0000 7f45d2000640 1 Processor -- start 2024-09-16T15:45:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.762+0000 7f45d2000640 1 -- start start 2024-09-16T15:45:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45d2000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45cc0ad270 0x7f45cc1f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45d2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45cc0ccf00 0x7f45cc148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45d2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 0x7f45cc09c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45d2000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc06f2e0 con 0x7f45cc15f5a0 2024-09-16T15:45:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45d2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc06f160 con 0x7f45cc0ccf00 2024-09-16T15:45:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45d2000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f45cc06f460 con 0x7f45cc0ad270 2024-09-16T15:45:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 0x7f45cc09c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 0x7f45cc09c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46726/0 (socket says 172.21.15.60:46726) 2024-09-16T15:45:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.763+0000 7f45cbe00640 1 -- 172.21.15.60:0/3324256829 learned_addr learned my addr 172.21.15.60:0/3324256829 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45c2a00640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45cc0ccf00 0x7f45cc148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45cb400640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45cc0ad270 0x7f45cc1f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45cbe00640 1 -- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45cc0ad270 msgr2=0x7f45cc1f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45cbe00640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45cc0ad270 0x7f45cc1f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45cbe00640 1 -- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45cc0ccf00 msgr2=0x7f45cc148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45cbe00640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45cc0ccf00 0x7f45cc148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45cbe00640 1 -- 172.21.15.60:0/3324256829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc1f9090 con 0x7f45cc15f5a0 2024-09-16T15:45:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45c2a00640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45cc0ccf00 0x7f45cc148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45cbe00640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 0x7f45cc09c7d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f45c400c7d0 tx=0x7f45c400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.764+0000 7f45c3e00640 1 -- 172.21.15.60:0/3324256829 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4003b70 con 0x7f45cc15f5a0 2024-09-16T15:45:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.765+0000 7f45c3e00640 1 -- 172.21.15.60:0/3324256829 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45c4003d10 con 0x7f45cc15f5a0 2024-09-16T15:45:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.765+0000 7f45c3e00640 1 -- 172.21.15.60:0/3324256829 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c400aab0 con 0x7f45cc15f5a0 2024-09-16T15:45:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.765+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc148540 con 0x7f45cc15f5a0 2024-09-16T15:45:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.765+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc14af60 con 0x7f45cc15f5a0 2024-09-16T15:45:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.766+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc06f9f0 con 0x7f45cc15f5a0 2024-09-16T15:45:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.767+0000 7f45c3e00640 1 -- 172.21.15.60:0/3324256829 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f45c400ac50 con 0x7f45cc15f5a0 2024-09-16T15:45:30.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.768+0000 7f45c3e00640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f45a8078cd0 0x7f45a807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:30.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.768+0000 7f45c3e00640 1 -- 172.21.15.60:0/3324256829 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5737+0+0 (secure 0 0 0) 0x7f45c409aff0 con 0x7f45cc15f5a0 2024-09-16T15:45:30.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.768+0000 7f45cb400640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f45a8078cd0 0x7f45a807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.769+0000 7f45cb400640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f45a8078cd0 0x7f45a807b190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45bc033000 tx=0x7f45bc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:30.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.772+0000 7f45c3e00640 1 -- 172.21.15.60:0/3324256829 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f45c4062ff0 con 0x7f45cc15f5a0 2024-09-16T15:45:30.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.968+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45cc09d8a0 con 0x7f45cc15f5a0 2024-09-16T15:45:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.969+0000 7f45c3e00640 1 -- 172.21.15.60:0/3324256829 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16796 (secure 0 0 0) 0x7f45c40628f0 con 0x7f45cc15f5a0 2024-09-16T15:45:30.971 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:30.971 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":121,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:29.603267+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:45:27.412685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:45:28.568791+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:30.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f45a8078cd0 msgr2=0x7f45a807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f45a8078cd0 0x7f45a807b190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45bc033000 tx=0x7f45bc004730 comp rx=0 tx=0).stop 2024-09-16T15:45:30.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 msgr2=0x7f45cc09c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:30.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 0x7f45cc09c7d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f45c400c7d0 tx=0x7f45c400cc90 comp rx=0 tx=0).stop 2024-09-16T15:45:30.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 shutdown_connections 2024-09-16T15:45:30.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f45a8078cd0 0x7f45a807b190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45cc15f5a0 0x7f45cc09c7d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45cc0ccf00 0x7f45cc148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 --2- 172.21.15.60:0/3324256829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45cc0ad270 0x7f45cc1f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 >> 172.21.15.60:0/3324256829 conn(0x7f45cc0b1950 msgr2=0x7f45cc0b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 shutdown_connections 2024-09-16T15:45:30.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:30.972+0000 7f45d2000640 1 -- 172.21.15.60:0/3324256829 wait complete. 2024-09-16T15:45:30.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.151+0000 7faffc200640 1 -- 172.21.15.60:0/2448964263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff406efa0 msgr2=0x7faff40cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.151+0000 7faffc200640 1 --2- 172.21.15.60:0/2448964263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff406efa0 0x7faff40cdbb0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7faff40af780 tx=0x7fafec02a4b0 comp rx=0 tx=0).stop 2024-09-16T15:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.152+0000 7faffc200640 1 -- 172.21.15.60:0/2448964263 shutdown_connections 2024-09-16T15:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.152+0000 7faffc200640 1 --2- 172.21.15.60:0/2448964263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff406efa0 0x7faff40cdbb0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.152+0000 7faffc200640 1 --2- 172.21.15.60:0/2448964263 >> v2:172.21.15.100:3300/0 conn(0x7faff40cccd0 0x7faff4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.152+0000 7faffc200640 1 -- 172.21.15.60:0/2448964263 >> 172.21.15.60:0/2448964263 conn(0x7faff40b1840 msgr2=0x7faff40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.152+0000 7faffc200640 1 -- 172.21.15.60:0/2448964263 shutdown_connections 2024-09-16T15:45:31.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.152+0000 7faffc200640 1 -- 172.21.15.60:0/2448964263 wait complete. 2024-09-16T15:45:31.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.153+0000 7faffc200640 1 Processor -- start 2024-09-16T15:45:31.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.153+0000 7faffc200640 1 -- start start 2024-09-16T15:45:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.154+0000 7faffc200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faff406efa0 0x7faff41421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.154+0000 7faffc200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff40cccd0 0x7faff415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.154+0000 7faffc200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff4145750 0x7faff413ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.154+0000 7faffc200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faff40ce480 con 0x7faff4145750 2024-09-16T15:45:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.154+0000 7faffc200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faff40ce300 con 0x7faff40cccd0 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.154+0000 7faffc200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7faff40ce600 con 0x7faff406efa0 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff4145750 0x7faff413ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff4145750 0x7faff413ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46732/0 (socket says 172.21.15.60:46732) 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 -- 172.21.15.60:0/1452941376 learned_addr learned my addr 172.21.15.60:0/1452941376 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faff8e00640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff40cccd0 0x7faff415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faff9800640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faff406efa0 0x7faff41421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 -- 172.21.15.60:0/1452941376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faff406efa0 msgr2=0x7faff41421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faff406efa0 0x7faff41421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 -- 172.21.15.60:0/1452941376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff40cccd0 msgr2=0x7faff415deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff40cccd0 0x7faff415deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 -- 172.21.15.60:0/1452941376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff41426f0 con 0x7faff4145750 2024-09-16T15:45:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.155+0000 7faffa200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff4145750 0x7faff413ca90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fafec009920 tx=0x7fafec032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:31.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.156+0000 7faff1600640 1 -- 172.21.15.60:0/1452941376 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec040070 con 0x7faff4145750 2024-09-16T15:45:31.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.156+0000 7faff1600640 1 -- 172.21.15.60:0/1452941376 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafec005310 con 0x7faff4145750 2024-09-16T15:45:31.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.156+0000 7faff1600640 1 -- 172.21.15.60:0/1452941376 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec0047a0 con 0x7faff4145750 2024-09-16T15:45:31.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.156+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff415e3f0 con 0x7faff4145750 2024-09-16T15:45:31.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.156+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff413eb20 con 0x7faff4145750 2024-09-16T15:45:31.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.157+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff40cdbb0 con 0x7faff4145750 2024-09-16T15:45:31.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.158+0000 7faff1600640 1 -- 172.21.15.60:0/1452941376 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fafec039040 con 0x7faff4145750 2024-09-16T15:45:31.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.159+0000 7faff1600640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fafd4078cd0 0x7fafd407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:31.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.159+0000 7faff1600640 1 -- 172.21.15.60:0/1452941376 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5737+0+0 (secure 0 0 0) 0x7fafec0bc2a0 con 0x7faff4145750 2024-09-16T15:45:31.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.159+0000 7faff9800640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fafd4078cd0 0x7fafd407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:31.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.160+0000 7faff9800640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fafd4078cd0 0x7fafd407b190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fafe80012e0 tx=0x7fafe8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:31.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.163+0000 7faff1600640 1 -- 172.21.15.60:0/1452941376 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faff41426f0 con 0x7faff4145750 2024-09-16T15:45:31.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:45:31.392 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:45:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:31.401+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faff4144a60 con 0x7faff4145750 2024-09-16T15:45:32.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.180+0000 7faff1600640 1 -- 172.21.15.60:0/1452941376 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) ==== 118+0+0 (secure 0 0 0) 0x7fafec083b20 con 0x7faff4145750 2024-09-16T15:45:32.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fafd4078cd0 msgr2=0x7fafd407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:32.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fafd4078cd0 0x7fafd407b190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fafe80012e0 tx=0x7fafe8009040 comp rx=0 tx=0).stop 2024-09-16T15:45:32.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff4145750 msgr2=0x7faff413ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:32.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff4145750 0x7faff413ca90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fafec009920 tx=0x7fafec032c00 comp rx=0 tx=0).stop 2024-09-16T15:45:32.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 shutdown_connections 2024-09-16T15:45:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fafd4078cd0 0x7fafd407b190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faff4145750 0x7faff413ca90 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faff40cccd0 0x7faff415deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 --2- 172.21.15.60:0/1452941376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faff406efa0 0x7faff41421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 >> 172.21.15.60:0/1452941376 conn(0x7faff40b1840 msgr2=0x7faff40a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 shutdown_connections 2024-09-16T15:45:32.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.183+0000 7faffc200640 1 -- 172.21.15.60:0/1452941376 wait complete. 2024-09-16T15:45:32.204 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:45:32.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:45:32.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.373+0000 7ff260400640 1 -- 172.21.15.60:0/1371470909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff240005290 msgr2=0x7ff24000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:32.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.373+0000 7ff260400640 1 --2- 172.21.15.60:0/1371470909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff240005290 0x7ff24000b9c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff250009e20 tx=0x7ff240005830 comp rx=0 tx=0).stop 2024-09-16T15:45:32.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.373+0000 7ff260400640 1 -- 172.21.15.60:0/1371470909 shutdown_connections 2024-09-16T15:45:32.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.373+0000 7ff260400640 1 --2- 172.21.15.60:0/1371470909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff240005290 0x7ff24000b9c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:32.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.373+0000 7ff260400640 1 -- 172.21.15.60:0/1371470909 >> 172.21.15.60:0/1371470909 conn(0x7ff2580b1840 msgr2=0x7ff2580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:32.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.373+0000 7ff260400640 1 -- 172.21.15.60:0/1371470909 shutdown_connections 2024-09-16T15:45:32.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.373+0000 7ff260400640 1 -- 172.21.15.60:0/1371470909 wait complete. 2024-09-16T15:45:32.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.375+0000 7ff260400640 1 Processor -- start 2024-09-16T15:45:32.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.375+0000 7ff260400640 1 -- start start 2024-09-16T15:45:32.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.375+0000 7ff260400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff240005290 0x7ff25814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:32.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff260400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff258142510 0x7ff258140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:32.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff260400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258140890 0x7ff258143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:32.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff260400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff2580737d0 con 0x7ff258140890 2024-09-16T15:45:32.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff260400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff258073650 con 0x7ff240005290 2024-09-16T15:45:32.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff260400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff2580cc5c0 con 0x7ff258142510 2024-09-16T15:45:32.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258140890 0x7ff258143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258140890 0x7ff258143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46740/0 (socket says 172.21.15.60:46740) 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25e400640 1 -- 172.21.15.60:0/3885322196 learned_addr learned my addr 172.21.15.60:0/3885322196 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25da00640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff240005290 0x7ff25814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25d000640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff258142510 0x7ff258140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25e400640 1 -- 172.21.15.60:0/3885322196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff258142510 msgr2=0x7ff258140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25e400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff258142510 0x7ff258140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25e400640 1 -- 172.21.15.60:0/3885322196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff240005290 msgr2=0x7ff25814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25e400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff240005290 0x7ff25814fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.376+0000 7ff25e400640 1 -- 172.21.15.60:0/3885322196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff258150270 con 0x7ff258140890 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.377+0000 7ff25e400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258140890 0x7ff258143c10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff250005fd0 tx=0x7ff25002cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.377+0000 7ff255600640 1 -- 172.21.15.60:0/3885322196 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250009a20 con 0x7ff258140890 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.377+0000 7ff255600640 1 -- 172.21.15.60:0/3885322196 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff250044070 con 0x7ff258140890 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.377+0000 7ff255600640 1 -- 172.21.15.60:0/3885322196 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250005580 con 0x7ff258140890 2024-09-16T15:45:32.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.377+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2581429e0 con 0x7ff258140890 2024-09-16T15:45:32.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.377+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff258142fa0 con 0x7ff258140890 2024-09-16T15:45:32.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.378+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25815f9f0 con 0x7ff258140890 2024-09-16T15:45:32.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.379+0000 7ff255600640 1 -- 172.21.15.60:0/3885322196 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7ff250005720 con 0x7ff258140890 2024-09-16T15:45:32.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.381+0000 7ff255600640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff238078cd0 0x7ff23807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:32.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.381+0000 7ff255600640 1 -- 172.21.15.60:0/3885322196 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7ff2500bc330 con 0x7ff258140890 2024-09-16T15:45:32.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.381+0000 7ff25da00640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff238078cd0 0x7ff23807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:32.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.382+0000 7ff25da00640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff238078cd0 0x7ff23807b190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006cd0 tx=0x7ff24c0067a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:32.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.385+0000 7ff255600640 1 -- 172.21.15.60:0/3885322196 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff250083c50 con 0x7ff258140890 2024-09-16T15:45:32.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:32.621+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff2580aef00 con 0x7ff258140890 2024-09-16T15:45:33.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.187+0000 7ff255600640 1 -- 172.21.15.60:0/3885322196 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) ==== 122+0+0 (secure 0 0 0) 0x7ff250088680 con 0x7ff258140890 2024-09-16T15:45:33.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff238078cd0 msgr2=0x7ff23807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff238078cd0 0x7ff23807b190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006cd0 tx=0x7ff24c0067a0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258140890 msgr2=0x7ff258143c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258140890 0x7ff258143c10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff250005fd0 tx=0x7ff25002cea0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 shutdown_connections 2024-09-16T15:45:33.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff238078cd0 0x7ff23807b190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff258140890 0x7ff258143c10 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff258142510 0x7ff258140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 --2- 172.21.15.60:0/3885322196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff240005290 0x7ff25814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 >> 172.21.15.60:0/3885322196 conn(0x7ff2580b1840 msgr2=0x7ff2580cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:33.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 shutdown_connections 2024-09-16T15:45:33.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.190+0000 7ff260400640 1 -- 172.21.15.60:0/3885322196 wait complete. 2024-09-16T15:45:33.209 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:45:33.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:33.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 -- 172.21.15.60:0/3789694643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca80b3e50 msgr2=0x7f8ca80d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 --2- 172.21.15.60:0/3789694643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca80b3e50 0x7f8ca80d1d60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8ca806a240 tx=0x7f8ca002a980 comp rx=0 tx=0).stop 2024-09-16T15:45:33.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 -- 172.21.15.60:0/3789694643 shutdown_connections 2024-09-16T15:45:33.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 --2- 172.21.15.60:0/3789694643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca80b3e50 0x7f8ca80d1d60 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 --2- 172.21.15.60:0/3789694643 >> v2:172.21.15.100:3300/0 conn(0x7f8ca80b34b0 0x7f8ca80b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 --2- 172.21.15.60:0/3789694643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ca8161440 0x7f8ca80b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 -- 172.21.15.60:0/3789694643 >> 172.21.15.60:0/3789694643 conn(0x7f8ca80d3df0 msgr2=0x7f8ca80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:33.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 -- 172.21.15.60:0/3789694643 shutdown_connections 2024-09-16T15:45:33.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.371+0000 7f8ca7200640 1 -- 172.21.15.60:0/3789694643 wait complete. 2024-09-16T15:45:33.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.372+0000 7f8ca7200640 1 Processor -- start 2024-09-16T15:45:33.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.373+0000 7f8ca7200640 1 -- start start 2024-09-16T15:45:33.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.373+0000 7f8ca7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80b34b0 0x7f8ca806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:33.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.373+0000 7f8ca7200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ca80b3e50 0x7f8ca81574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:33.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.373+0000 7f8ca7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca8161440 0x7f8ca80ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:33.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.373+0000 7f8ca7200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80d2420 con 0x7f8ca8161440 2024-09-16T15:45:33.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.373+0000 7f8ca7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80d22a0 con 0x7f8ca80b34b0 2024-09-16T15:45:33.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.373+0000 7f8ca7200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca80d25a0 con 0x7f8ca80b3e50 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca8161440 0x7f8ca80ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca8161440 0x7f8ca80ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46764/0 (socket says 172.21.15.60:46764) 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca6800640 1 -- 172.21.15.60:0/978689758 learned_addr learned my addr 172.21.15.60:0/978689758 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8c9d400640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ca80b3e50 0x7f8ca81574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca5e00640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80b34b0 0x7f8ca806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca6800640 1 -- 172.21.15.60:0/978689758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ca80b3e50 msgr2=0x7f8ca81574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca6800640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ca80b3e50 0x7f8ca81574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca6800640 1 -- 172.21.15.60:0/978689758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80b34b0 msgr2=0x7f8ca806d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca6800640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80b34b0 0x7f8ca806d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.374+0000 7f8ca6800640 1 -- 172.21.15.60:0/978689758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca80cd770 con 0x7f8ca8161440 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.375+0000 7f8ca6800640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca8161440 0x7f8ca80ca290 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8ca002af70 tx=0x7f8ca0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:33.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.375+0000 7f8c9ea00640 1 -- 172.21.15.60:0/978689758 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0034050 con 0x7f8ca8161440 2024-09-16T15:45:33.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.375+0000 7f8c9ea00640 1 -- 172.21.15.60:0/978689758 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ca0004450 con 0x7f8ca8161440 2024-09-16T15:45:33.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.375+0000 7f8c9ea00640 1 -- 172.21.15.60:0/978689758 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0004840 con 0x7f8ca8161440 2024-09-16T15:45:33.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.375+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca80cda50 con 0x7f8ca8161440 2024-09-16T15:45:33.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.375+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca80c7d40 con 0x7f8ca8161440 2024-09-16T15:45:33.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.376+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca806d9e0 con 0x7f8ca8161440 2024-09-16T15:45:33.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.377+0000 7f8c9ea00640 1 -- 172.21.15.60:0/978689758 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f8ca0037020 con 0x7f8ca8161440 2024-09-16T15:45:33.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.378+0000 7f8c9ea00640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c84078cd0 0x7f8c8407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:33.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.378+0000 7f8c9ea00640 1 -- 172.21.15.60:0/978689758 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f8ca00bc550 con 0x7f8ca8161440 2024-09-16T15:45:33.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.378+0000 7f8ca5e00640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c84078cd0 0x7f8c8407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:33.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.379+0000 7f8ca5e00640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c84078cd0 0x7f8c8407b190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c98005fd0 tx=0x7f8c9800b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:33.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.382+0000 7f8c9ea00640 1 -- 172.21.15.60:0/978689758 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ca806d9e0 con 0x7f8ca8161440 2024-09-16T15:45:33.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.618+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ca80d1430 con 0x7f8ca8161440 2024-09-16T15:45:33.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.618+0000 7f8c9ea00640 1 -- 172.21.15.60:0/978689758 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5225 (secure 0 0 0) 0x7f8ca0083d50 con 0x7f8ca8161440 2024-09-16T15:45:33.619 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:33.620 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":81,"btime":"2024-09-16T15:45:33:181165+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7225,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/900096462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":900096462},{"type":"v1","addr":"172.21.15.100:6834","nonce":900096462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2427775344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2427775344},{"type":"v1","addr":"172.21.15.100:6835","nonce":2427775344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7584,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1853423008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1853423008},{"type":"v1","addr":"172.21.15.60:6833","nonce":1853423008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:45:29.603235+0000","modified":"2024-09-16T15:45:32.622836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/678119606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":678119606},{"type":"v1","addr":"172.21.15.60:6835","nonce":678119606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7593,"qdb_cluster":[7593]},"id":9}]} 2024-09-16T15:45:33.620 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 81 2024-09-16T15:45:33.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c84078cd0 msgr2=0x7f8c8407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c84078cd0 0x7f8c8407b190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c98005fd0 tx=0x7f8c9800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca8161440 msgr2=0x7f8ca80ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca8161440 0x7f8ca80ca290 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8ca002af70 tx=0x7f8ca0003f90 comp rx=0 tx=0).stop 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 shutdown_connections 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c84078cd0 0x7f8c8407b190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ca8161440 0x7f8ca80ca290 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ca80b3e50 0x7f8ca81574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 --2- 172.21.15.60:0/978689758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ca80b34b0 0x7f8ca806d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 >> 172.21.15.60:0/978689758 conn(0x7f8ca80d3df0 msgr2=0x7f8ca80d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 shutdown_connections 2024-09-16T15:45:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.621+0000 7f8ca7200640 1 -- 172.21.15.60:0/978689758 wait complete. 2024-09-16T15:45:33.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.799+0000 7f3db2800640 1 -- 172.21.15.60:0/2745197836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac073fb0 msgr2=0x7f3dac074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.799+0000 7f3db2800640 1 --2- 172.21.15.60:0/2745197836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac073fb0 0x7f3dac074420 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0cfbb0 tx=0x7f3da402a820 comp rx=0 tx=0).stop 2024-09-16T15:45:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.799+0000 7f3db2800640 1 -- 172.21.15.60:0/2745197836 shutdown_connections 2024-09-16T15:45:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.799+0000 7f3db2800640 1 --2- 172.21.15.60:0/2745197836 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac073fb0 0x7f3dac074420 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.799+0000 7f3db2800640 1 --2- 172.21.15.60:0/2745197836 >> v2:172.21.15.100:3300/0 conn(0x7f3dac0ccf00 0x7f3dac0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.799+0000 7f3db2800640 1 -- 172.21.15.60:0/2745197836 >> 172.21.15.60:0/2745197836 conn(0x7f3dac0b1950 msgr2=0x7f3dac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.799+0000 7f3db2800640 1 -- 172.21.15.60:0/2745197836 shutdown_connections 2024-09-16T15:45:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.799+0000 7f3db2800640 1 -- 172.21.15.60:0/2745197836 wait complete. 2024-09-16T15:45:33.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.801+0000 7f3db2800640 1 Processor -- start 2024-09-16T15:45:33.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.801+0000 7f3db2800640 1 -- start start 2024-09-16T15:45:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3db2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dac073fb0 0x7f3dac14e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:33.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3db2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac0ccf00 0x7f3dac1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3db2800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dac15f5a0 0x7f3dac1f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3db2800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac06f2e0 con 0x7f3dac0ccf00 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3db2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac06f160 con 0x7f3dac073fb0 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3db2800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3dac06f460 con 0x7f3dac15f5a0 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3da3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac0ccf00 0x7f3dac1f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3da3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac0ccf00 0x7f3dac1f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46776/0 (socket says 172.21.15.60:46776) 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3da3e00640 1 -- 172.21.15.60:0/135124896 learned_addr learned my addr 172.21.15.60:0/135124896 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3dab400640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dac073fb0 0x7f3dac14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.802+0000 7f3dabe00640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dac15f5a0 0x7f3dac1f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3e00640 1 -- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dac15f5a0 msgr2=0x7f3dac1f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3e00640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dac15f5a0 0x7f3dac1f8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3e00640 1 -- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dac073fb0 msgr2=0x7f3dac14e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3e00640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dac073fb0 0x7f3dac14e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:33.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3e00640 1 -- 172.21.15.60:0/135124896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac15c300 con 0x7f3dac0ccf00 2024-09-16T15:45:33.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3dab400640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dac073fb0 0x7f3dac14e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:33.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3dabe00640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dac15f5a0 0x7f3dac1f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:33.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3e00640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac0ccf00 0x7f3dac1f8580 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400d950 tx=0x7f3d9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3400640 1 -- 172.21.15.60:0/135124896 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94017070 con 0x7f3dac0ccf00 2024-09-16T15:45:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3400640 1 -- 172.21.15.60:0/135124896 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d94009d20 con 0x7f3dac0ccf00 2024-09-16T15:45:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac15c5e0 con 0x7f3dac0ccf00 2024-09-16T15:45:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3da3400640 1 -- 172.21.15.60:0/135124896 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d940106c0 con 0x7f3dac0ccf00 2024-09-16T15:45:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.803+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac0b2df0 con 0x7f3dac0ccf00 2024-09-16T15:45:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.804+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dac0708c0 con 0x7f3dac0ccf00 2024-09-16T15:45:33.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.806+0000 7f3da3400640 1 -- 172.21.15.60:0/135124896 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f3d94010860 con 0x7f3dac0ccf00 2024-09-16T15:45:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.807+0000 7f3da3400640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3d88078cd0 0x7f3d8807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.807+0000 7f3da3400640 1 -- 172.21.15.60:0/135124896 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f3d9409b650 con 0x7f3dac0ccf00 2024-09-16T15:45:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.807+0000 7f3dab400640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3d88078cd0 0x7f3d8807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:33.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.808+0000 7f3dab400640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3d88078cd0 0x7f3d8807b190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0016a0 tx=0x7f3d9c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:33.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:33.811+0000 7f3da3400640 1 -- 172.21.15.60:0/135124896 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3dac0708c0 con 0x7f3dac0ccf00 2024-09-16T15:45:34.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.009+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dac09d8a0 con 0x7f3dac0ccf00 2024-09-16T15:45:34.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.010+0000 7f3da3400640 1 -- 172.21.15.60:0/135124896 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16783 (secure 0 0 0) 0x7f3d94063520 con 0x7f3dac0ccf00 2024-09-16T15:45:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:34.011 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":122,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:31.167860+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:45:27.412685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:45:28.568791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3d88078cd0 msgr2=0x7f3d8807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3d88078cd0 0x7f3d8807b190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0016a0 tx=0x7f3d9c009040 comp rx=0 tx=0).stop 2024-09-16T15:45:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac0ccf00 msgr2=0x7f3dac1f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac0ccf00 0x7f3dac1f8580 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400d950 tx=0x7f3d9400de10 comp rx=0 tx=0).stop 2024-09-16T15:45:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 shutdown_connections 2024-09-16T15:45:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3d88078cd0 0x7f3d8807b190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dac15f5a0 0x7f3dac1f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dac0ccf00 0x7f3dac1f8580 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 --2- 172.21.15.60:0/135124896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dac073fb0 0x7f3dac14e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 >> 172.21.15.60:0/135124896 conn(0x7f3dac0b1950 msgr2=0x7f3dac15e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 shutdown_connections 2024-09-16T15:45:34.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.013+0000 7f3db2800640 1 -- 172.21.15.60:0/135124896 wait complete. 2024-09-16T15:45:34.030 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:45:34.030 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:45:34.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.193+0000 7f85b2800640 1 -- 172.21.15.60:0/1382760793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac0af3d0 msgr2=0x7f85ac06f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.193+0000 7f85b2800640 1 --2- 172.21.15.60:0/1382760793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac0af3d0 0x7f85ac06f660 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8598009830 tx=0x7f859802a480 comp rx=0 tx=0).stop 2024-09-16T15:45:34.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.193+0000 7f85b2800640 1 -- 172.21.15.60:0/1382760793 shutdown_connections 2024-09-16T15:45:34.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.193+0000 7f85b2800640 1 --2- 172.21.15.60:0/1382760793 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85ac072df0 0x7f85ac073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.193+0000 7f85b2800640 1 --2- 172.21.15.60:0/1382760793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac0af3d0 0x7f85ac06f660 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.193+0000 7f85b2800640 1 -- 172.21.15.60:0/1382760793 >> 172.21.15.60:0/1382760793 conn(0x7f85ac0b1950 msgr2=0x7f85ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:34.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.193+0000 7f85b2800640 1 -- 172.21.15.60:0/1382760793 shutdown_connections 2024-09-16T15:45:34.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.193+0000 7f85b2800640 1 -- 172.21.15.60:0/1382760793 wait complete. 2024-09-16T15:45:34.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.194+0000 7f85b2800640 1 Processor -- start 2024-09-16T15:45:34.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.195+0000 7f85b2800640 1 -- start start 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.195+0000 7f85b2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85ac072df0 0x7f85ac14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.195+0000 7f85b2800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85ac0af3d0 0x7f85ac1f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.195+0000 7f85b2800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac161550 0x7f85ac1f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.195+0000 7f85b2800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac0737c0 con 0x7f85ac072df0 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.195+0000 7f85b2800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac06fdb0 con 0x7f85ac161550 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.195+0000 7f85b2800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f85ac073940 con 0x7f85ac0af3d0 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85ac072df0 0x7f85ac14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85ac072df0 0x7f85ac14c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46788/0 (socket says 172.21.15.60:46788) 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 -- 172.21.15.60:0/4251572479 learned_addr learned my addr 172.21.15.60:0/4251572479 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85abe00640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac161550 0x7f85ac1f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85aaa00640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85ac0af3d0 0x7f85ac1f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 -- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85ac0af3d0 msgr2=0x7f85ac1f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85ac0af3d0 0x7f85ac1f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 -- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac161550 msgr2=0x7f85ac1f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac161550 0x7f85ac1f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 -- 172.21.15.60:0/4251572479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac158a20 con 0x7f85ac072df0 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85aaa00640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85ac0af3d0 0x7f85ac1f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.196+0000 7f85ab400640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85ac072df0 0x7f85ac14c630 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f859c00c3d0 tx=0x7f859c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.197+0000 7f85a3400640 1 -- 172.21.15.60:0/4251572479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c011790 con 0x7f85ac072df0 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.197+0000 7f85a3400640 1 -- 172.21.15.60:0/4251572479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f859c017070 con 0x7f85ac072df0 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.197+0000 7f85a3400640 1 -- 172.21.15.60:0/4251572479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c011cc0 con 0x7f85ac072df0 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.197+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac0b0d60 con 0x7f85ac072df0 2024-09-16T15:45:34.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.197+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac158ca0 con 0x7f85ac072df0 2024-09-16T15:45:34.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.198+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ac073ea0 con 0x7f85ac072df0 2024-09-16T15:45:34.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.199+0000 7f85a3400640 1 -- 172.21.15.60:0/4251572479 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f859c01c460 con 0x7f85ac072df0 2024-09-16T15:45:34.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.200+0000 7f85a3400640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f858c078cd0 0x7f858c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:34.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.200+0000 7f85a3400640 1 -- 172.21.15.60:0/4251572479 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f859c09aed0 con 0x7f85ac072df0 2024-09-16T15:45:34.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.200+0000 7f85aaa00640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f858c078cd0 0x7f858c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.201+0000 7f85aaa00640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f858c078cd0 0x7f858c07b190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8598034000 tx=0x7f8598004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:34.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.204+0000 7f85a3400640 1 -- 172.21.15.60:0/4251572479 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f85ac158a20 con 0x7f85ac072df0 2024-09-16T15:45:34.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.397+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85ac160aa0 con 0x7f858c078cd0 2024-09-16T15:45:34.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.402+0000 7f85a3400640 1 -- 172.21.15.60:0/4251572479 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f85ac160aa0 con 0x7f858c078cd0 2024-09-16T15:45:34.406 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:34.408 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:45:34.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f858c078cd0 msgr2=0x7f858c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f858c078cd0 0x7f858c07b190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8598034000 tx=0x7f8598004730 comp rx=0 tx=0).stop 2024-09-16T15:45:34.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85ac072df0 msgr2=0x7f85ac14c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85ac072df0 0x7f85ac14c630 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f859c00c3d0 tx=0x7f859c00c890 comp rx=0 tx=0).stop 2024-09-16T15:45:34.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 shutdown_connections 2024-09-16T15:45:34.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f858c078cd0 0x7f858c07b190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85ac161550 0x7f85ac1f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85ac0af3d0 0x7f85ac1f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 --2- 172.21.15.60:0/4251572479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85ac072df0 0x7f85ac14c630 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 >> 172.21.15.60:0/4251572479 conn(0x7f85ac0b1950 msgr2=0x7f85ac15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:34.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 shutdown_connections 2024-09-16T15:45:34.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.406+0000 7f85b2800640 1 -- 172.21.15.60:0/4251572479 wait complete. 2024-09-16T15:45:34.423 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-16T15:45:34.193570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279180,"kb_used_data":13016,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695540,"statfs":{"total":773094113280,"available":772808232960,"internally_reserved":0,"allocated":13328384,"data_stored":6496268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001265"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696838+0000","last_change":"2024-09-16T15:45:30.195601+0000","last_active":"2024-09-16T15:45:31.696838+0000","last_peered":"2024-09-16T15:45:31.696838+0000","last_clean":"2024-09-16T15:45:31.696838+0000","last_became_active":"2024-09-16T15:45:30.191133+0000","last_became_peered":"2024-09-16T15:45:30.191133+0000","last_unstale":"2024-09-16T15:45:31.696838+0000","last_undegraded":"2024-09-16T15:45:31.696838+0000","last_fullsized":"2024-09-16T15:45:31.696838+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:11.030668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466280+0000","last_change":"2024-09-16T15:45:30.185945+0000","last_active":"2024-09-16T15:45:31.466280+0000","last_peered":"2024-09-16T15:45:31.466280+0000","last_clean":"2024-09-16T15:45:31.466280+0000","last_became_active":"2024-09-16T15:45:30.185687+0000","last_became_peered":"2024-09-16T15:45:30.185687+0000","last_unstale":"2024-09-16T15:45:31.466280+0000","last_undegraded":"2024-09-16T15:45:31.466280+0000","last_fullsized":"2024-09-16T15:45:31.466280+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:09:16.707957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179341+0000","last_change":"2024-09-16T15:45:30.193397+0000","last_active":"2024-09-16T15:45:31.179341+0000","last_peered":"2024-09-16T15:45:31.179341+0000","last_clean":"2024-09-16T15:45:31.179341+0000","last_became_active":"2024-09-16T15:45:30.193111+0000","last_became_peered":"2024-09-16T15:45:30.193111+0000","last_unstale":"2024-09-16T15:45:31.179341+0000","last_undegraded":"2024-09-16T15:45:31.179341+0000","last_fullsized":"2024-09-16T15:45:31.179341+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:58.080375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697076+0000","last_change":"2024-09-16T15:45:30.179078+0000","last_active":"2024-09-16T15:45:31.697076+0000","last_peered":"2024-09-16T15:45:31.697076+0000","last_clean":"2024-09-16T15:45:31.697076+0000","last_became_active":"2024-09-16T15:45:30.178774+0000","last_became_peered":"2024-09-16T15:45:30.178774+0000","last_unstale":"2024-09-16T15:45:31.697076+0000","last_undegraded":"2024-09-16T15:45:31.697076+0000","last_fullsized":"2024-09-16T15:45:31.697076+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:42.445153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696910+0000","last_change":"2024-09-16T15:45:30.187642+0000","last_active":"2024-09-16T15:45:31.696910+0000","last_peered":"2024-09-16T15:45:31.696910+0000","last_clean":"2024-09-16T15:45:31.696910+0000","last_became_active":"2024-09-16T15:45:30.187354+0000","last_became_peered":"2024-09-16T15:45:30.187354+0000","last_unstale":"2024-09-16T15:45:31.696910+0000","last_undegraded":"2024-09-16T15:45:31.696910+0000","last_fullsized":"2024-09-16T15:45:31.696910+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:05.626792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.178107+0000","last_change":"2024-09-16T15:45:30.190529+0000","last_active":"2024-09-16T15:45:31.178107+0000","last_peered":"2024-09-16T15:45:31.178107+0000","last_clean":"2024-09-16T15:45:31.178107+0000","last_became_active":"2024-09-16T15:45:30.190254+0000","last_became_peered":"2024-09-16T15:45:30.190254+0000","last_unstale":"2024-09-16T15:45:31.178107+0000","last_undegraded":"2024-09-16T15:45:31.178107+0000","last_fullsized":"2024-09-16T15:45:31.178107+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:48.280644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177318+0000","last_change":"2024-09-16T15:45:30.187109+0000","last_active":"2024-09-16T15:45:31.177318+0000","last_peered":"2024-09-16T15:45:31.177318+0000","last_clean":"2024-09-16T15:45:31.177318+0000","last_became_active":"2024-09-16T15:45:30.185804+0000","last_became_peered":"2024-09-16T15:45:30.185804+0000","last_unstale":"2024-09-16T15:45:31.177318+0000","last_undegraded":"2024-09-16T15:45:31.177318+0000","last_fullsized":"2024-09-16T15:45:31.177318+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:10.104750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179069+0000","last_change":"2024-09-16T15:45:30.182413+0000","last_active":"2024-09-16T15:45:31.179069+0000","last_peered":"2024-09-16T15:45:31.179069+0000","last_clean":"2024-09-16T15:45:31.179069+0000","last_became_active":"2024-09-16T15:45:30.181970+0000","last_became_peered":"2024-09-16T15:45:30.181970+0000","last_unstale":"2024-09-16T15:45:31.179069+0000","last_undegraded":"2024-09-16T15:45:31.179069+0000","last_fullsized":"2024-09-16T15:45:31.179069+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:15.746732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179683+0000","last_change":"2024-09-16T15:45:30.181367+0000","last_active":"2024-09-16T15:45:31.179683+0000","last_peered":"2024-09-16T15:45:31.179683+0000","last_clean":"2024-09-16T15:45:31.179683+0000","last_became_active":"2024-09-16T15:45:30.180972+0000","last_became_peered":"2024-09-16T15:45:30.180972+0000","last_unstale":"2024-09-16T15:45:31.179683+0000","last_undegraded":"2024-09-16T15:45:31.179683+0000","last_fullsized":"2024-09-16T15:45:31.179683+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:42.054674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466272+0000","last_change":"2024-09-16T15:45:30.185816+0000","last_active":"2024-09-16T15:45:31.466272+0000","last_peered":"2024-09-16T15:45:31.466272+0000","last_clean":"2024-09-16T15:45:31.466272+0000","last_became_active":"2024-09-16T15:45:30.185504+0000","last_became_peered":"2024-09-16T15:45:30.185504+0000","last_unstale":"2024-09-16T15:45:31.466272+0000","last_undegraded":"2024-09-16T15:45:31.466272+0000","last_fullsized":"2024-09-16T15:45:31.466272+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:04:25.595769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176492+0000","last_change":"2024-09-16T15:45:30.194520+0000","last_active":"2024-09-16T15:45:31.176492+0000","last_peered":"2024-09-16T15:45:31.176492+0000","last_clean":"2024-09-16T15:45:31.176492+0000","last_became_active":"2024-09-16T15:45:30.194209+0000","last_became_peered":"2024-09-16T15:45:30.194209+0000","last_unstale":"2024-09-16T15:45:31.176492+0000","last_undegraded":"2024-09-16T15:45:31.176492+0000","last_fullsized":"2024-09-16T15:45:31.176492+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:34.169524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179131+0000","last_change":"2024-09-16T15:45:30.183859+0000","last_active":"2024-09-16T15:45:31.179131+0000","last_peered":"2024-09-16T15:45:31.179131+0000","last_clean":"2024-09-16T15:45:31.179131+0000","last_became_active":"2024-09-16T15:45:30.183596+0000","last_became_peered":"2024-09-16T15:45:30.183596+0000","last_unstale":"2024-09-16T15:45:31.179131+0000","last_undegraded":"2024-09-16T15:45:31.179131+0000","last_fullsized":"2024-09-16T15:45:31.179131+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:36.615198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696579+0000","last_change":"2024-09-16T15:45:30.186993+0000","last_active":"2024-09-16T15:45:31.696579+0000","last_peered":"2024-09-16T15:45:31.696579+0000","last_clean":"2024-09-16T15:45:31.696579+0000","last_became_active":"2024-09-16T15:45:30.186681+0000","last_became_peered":"2024-09-16T15:45:30.186681+0000","last_unstale":"2024-09-16T15:45:31.696579+0000","last_undegraded":"2024-09-16T15:45:31.696579+0000","last_fullsized":"2024-09-16T15:45:31.696579+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:39.314286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466004+0000","last_change":"2024-09-16T15:45:30.177706+0000","last_active":"2024-09-16T15:45:31.466004+0000","last_peered":"2024-09-16T15:45:31.466004+0000","last_clean":"2024-09-16T15:45:31.466004+0000","last_became_active":"2024-09-16T15:45:30.177410+0000","last_became_peered":"2024-09-16T15:45:30.177410+0000","last_unstale":"2024-09-16T15:45:31.466004+0000","last_undegraded":"2024-09-16T15:45:31.466004+0000","last_fullsized":"2024-09-16T15:45:31.466004+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:41.205372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697075+0000","last_change":"2024-09-16T15:45:30.192350+0000","last_active":"2024-09-16T15:45:31.697075+0000","last_peered":"2024-09-16T15:45:31.697075+0000","last_clean":"2024-09-16T15:45:31.697075+0000","last_became_active":"2024-09-16T15:45:30.192056+0000","last_became_peered":"2024-09-16T15:45:30.192056+0000","last_unstale":"2024-09-16T15:45:31.697075+0000","last_undegraded":"2024-09-16T15:45:31.697075+0000","last_fullsized":"2024-09-16T15:45:31.697075+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:26.348815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696251+0000","last_change":"2024-09-16T15:45:30.187017+0000","last_active":"2024-09-16T15:45:31.696251+0000","last_peered":"2024-09-16T15:45:31.696251+0000","last_clean":"2024-09-16T15:45:31.696251+0000","last_became_active":"2024-09-16T15:45:30.186730+0000","last_became_peered":"2024-09-16T15:45:30.186730+0000","last_unstale":"2024-09-16T15:45:31.696251+0000","last_undegraded":"2024-09-16T15:45:31.696251+0000","last_fullsized":"2024-09-16T15:45:31.696251+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:47.183846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177777+0000","last_change":"2024-09-16T15:45:30.195648+0000","last_active":"2024-09-16T15:45:31.177777+0000","last_peered":"2024-09-16T15:45:31.177777+0000","last_clean":"2024-09-16T15:45:31.177777+0000","last_became_active":"2024-09-16T15:45:30.195019+0000","last_became_peered":"2024-09-16T15:45:30.195019+0000","last_unstale":"2024-09-16T15:45:31.177777+0000","last_undegraded":"2024-09-16T15:45:31.177777+0000","last_fullsized":"2024-09-16T15:45:31.177777+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:01.587071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697351+0000","last_change":"2024-09-16T15:45:30.191536+0000","last_active":"2024-09-16T15:45:31.697351+0000","last_peered":"2024-09-16T15:45:31.697351+0000","last_clean":"2024-09-16T15:45:31.697351+0000","last_became_active":"2024-09-16T15:45:30.191113+0000","last_became_peered":"2024-09-16T15:45:30.191113+0000","last_unstale":"2024-09-16T15:45:31.697351+0000","last_undegraded":"2024-09-16T15:45:31.697351+0000","last_fullsized":"2024-09-16T15:45:31.697351+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:18.212680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176565+0000","last_change":"2024-09-16T15:45:30.192761+0000","last_active":"2024-09-16T15:45:31.176565+0000","last_peered":"2024-09-16T15:45:31.176565+0000","last_clean":"2024-09-16T15:45:31.176565+0000","last_became_active":"2024-09-16T15:45:30.191719+0000","last_became_peered":"2024-09-16T15:45:30.191719+0000","last_unstale":"2024-09-16T15:45:31.176565+0000","last_undegraded":"2024-09-16T15:45:31.176565+0000","last_fullsized":"2024-09-16T15:45:31.176565+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:09:06.814030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697067+0000","last_change":"2024-09-16T15:45:30.183872+0000","last_active":"2024-09-16T15:45:31.697067+0000","last_peered":"2024-09-16T15:45:31.697067+0000","last_clean":"2024-09-16T15:45:31.697067+0000","last_became_active":"2024-09-16T15:45:30.180444+0000","last_became_peered":"2024-09-16T15:45:30.180444+0000","last_unstale":"2024-09-16T15:45:31.697067+0000","last_undegraded":"2024-09-16T15:45:31.697067+0000","last_fullsized":"2024-09-16T15:45:31.697067+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:59.745713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696494+0000","last_change":"2024-09-16T15:45:30.187090+0000","last_active":"2024-09-16T15:45:31.696494+0000","last_peered":"2024-09-16T15:45:31.696494+0000","last_clean":"2024-09-16T15:45:31.696494+0000","last_became_active":"2024-09-16T15:45:30.186737+0000","last_became_peered":"2024-09-16T15:45:30.186737+0000","last_unstale":"2024-09-16T15:45:31.696494+0000","last_undegraded":"2024-09-16T15:45:31.696494+0000","last_fullsized":"2024-09-16T15:45:31.696494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:40.145347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176391+0000","last_change":"2024-09-16T15:45:30.184695+0000","last_active":"2024-09-16T15:45:31.176391+0000","last_peered":"2024-09-16T15:45:31.176391+0000","last_clean":"2024-09-16T15:45:31.176391+0000","last_became_active":"2024-09-16T15:45:30.182589+0000","last_became_peered":"2024-09-16T15:45:30.182589+0000","last_unstale":"2024-09-16T15:45:31.176391+0000","last_undegraded":"2024-09-16T15:45:31.176391+0000","last_fullsized":"2024-09-16T15:45:31.176391+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:06.422521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.178928+0000","last_change":"2024-09-16T15:45:30.179712+0000","last_active":"2024-09-16T15:45:31.178928+0000","last_peered":"2024-09-16T15:45:31.178928+0000","last_clean":"2024-09-16T15:45:31.178928+0000","last_became_active":"2024-09-16T15:45:30.179437+0000","last_became_peered":"2024-09-16T15:45:30.179437+0000","last_unstale":"2024-09-16T15:45:31.178928+0000","last_undegraded":"2024-09-16T15:45:31.178928+0000","last_fullsized":"2024-09-16T15:45:31.178928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:50.599464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176225+0000","last_change":"2024-09-16T15:45:30.189458+0000","last_active":"2024-09-16T15:45:31.176225+0000","last_peered":"2024-09-16T15:45:31.176225+0000","last_clean":"2024-09-16T15:45:31.176225+0000","last_became_active":"2024-09-16T15:45:30.188985+0000","last_became_peered":"2024-09-16T15:45:30.188985+0000","last_unstale":"2024-09-16T15:45:31.176225+0000","last_undegraded":"2024-09-16T15:45:31.176225+0000","last_fullsized":"2024-09-16T15:45:31.176225+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:35:23.754764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466347+0000","last_change":"2024-09-16T15:45:30.180005+0000","last_active":"2024-09-16T15:45:31.466347+0000","last_peered":"2024-09-16T15:45:31.466347+0000","last_clean":"2024-09-16T15:45:31.466347+0000","last_became_active":"2024-09-16T15:45:30.179690+0000","last_became_peered":"2024-09-16T15:45:30.179690+0000","last_unstale":"2024-09-16T15:45:31.466347+0000","last_undegraded":"2024-09-16T15:45:31.466347+0000","last_fullsized":"2024-09-16T15:45:31.466347+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:55.564184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697290+0000","last_change":"2024-09-16T15:45:30.198928+0000","last_active":"2024-09-16T15:45:31.697290+0000","last_peered":"2024-09-16T15:45:31.697290+0000","last_clean":"2024-09-16T15:45:31.697290+0000","last_became_active":"2024-09-16T15:45:30.198623+0000","last_became_peered":"2024-09-16T15:45:30.198623+0000","last_unstale":"2024-09-16T15:45:31.697290+0000","last_undegraded":"2024-09-16T15:45:31.697290+0000","last_fullsized":"2024-09-16T15:45:31.697290+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:56.525396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696776+0000","last_change":"2024-09-16T15:45:30.186499+0000","last_active":"2024-09-16T15:45:31.696776+0000","last_peered":"2024-09-16T15:45:31.696776+0000","last_clean":"2024-09-16T15:45:31.696776+0000","last_became_active":"2024-09-16T15:45:30.186218+0000","last_became_peered":"2024-09-16T15:45:30.186218+0000","last_unstale":"2024-09-16T15:45:31.696776+0000","last_undegraded":"2024-09-16T15:45:31.696776+0000","last_fullsized":"2024-09-16T15:45:31.696776+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:05:16.170424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696387+0000","last_change":"2024-09-16T15:45:30.178447+0000","last_active":"2024-09-16T15:45:31.696387+0000","last_peered":"2024-09-16T15:45:31.696387+0000","last_clean":"2024-09-16T15:45:31.696387+0000","last_became_active":"2024-09-16T15:45:30.178136+0000","last_became_peered":"2024-09-16T15:45:30.178136+0000","last_unstale":"2024-09-16T15:45:31.696387+0000","last_undegraded":"2024-09-16T15:45:31.696387+0000","last_fullsized":"2024-09-16T15:45:31.696387+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:22.542607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696638+0000","last_change":"2024-09-16T15:45:30.182560+0000","last_active":"2024-09-16T15:45:31.696638+0000","last_peered":"2024-09-16T15:45:31.696638+0000","last_clean":"2024-09-16T15:45:31.696638+0000","last_became_active":"2024-09-16T15:45:30.182244+0000","last_became_peered":"2024-09-16T15:45:30.182244+0000","last_unstale":"2024-09-16T15:45:31.696638+0000","last_undegraded":"2024-09-16T15:45:31.696638+0000","last_fullsized":"2024-09-16T15:45:31.696638+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:32:08.856816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697155+0000","last_change":"2024-09-16T15:45:30.181758+0000","last_active":"2024-09-16T15:45:31.697155+0000","last_peered":"2024-09-16T15:45:31.697155+0000","last_clean":"2024-09-16T15:45:31.697155+0000","last_became_active":"2024-09-16T15:45:30.181409+0000","last_became_peered":"2024-09-16T15:45:30.181409+0000","last_unstale":"2024-09-16T15:45:31.697155+0000","last_undegraded":"2024-09-16T15:45:31.697155+0000","last_fullsized":"2024-09-16T15:45:31.697155+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:32.614877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696643+0000","last_change":"2024-09-16T15:45:30.198483+0000","last_active":"2024-09-16T15:45:31.696643+0000","last_peered":"2024-09-16T15:45:31.696643+0000","last_clean":"2024-09-16T15:45:31.696643+0000","last_became_active":"2024-09-16T15:45:30.198159+0000","last_became_peered":"2024-09-16T15:45:30.198159+0000","last_unstale":"2024-09-16T15:45:31.696643+0000","last_undegraded":"2024-09-16T15:45:31.696643+0000","last_fullsized":"2024-09-16T15:45:31.696643+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:00.316651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177368+0000","last_change":"2024-09-16T15:45:30.188321+0000","last_active":"2024-09-16T15:45:31.177368+0000","last_peered":"2024-09-16T15:45:31.177368+0000","last_clean":"2024-09-16T15:45:31.177368+0000","last_became_active":"2024-09-16T15:45:30.187822+0000","last_became_peered":"2024-09-16T15:45:30.187822+0000","last_unstale":"2024-09-16T15:45:31.177368+0000","last_undegraded":"2024-09-16T15:45:31.177368+0000","last_fullsized":"2024-09-16T15:45:31.177368+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:04.076799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697417+0000","last_change":"2024-09-16T15:45:30.192437+0000","last_active":"2024-09-16T15:45:31.697417+0000","last_peered":"2024-09-16T15:45:31.697417+0000","last_clean":"2024-09-16T15:45:31.697417+0000","last_became_active":"2024-09-16T15:45:30.192115+0000","last_became_peered":"2024-09-16T15:45:30.192115+0000","last_unstale":"2024-09-16T15:45:31.697417+0000","last_undegraded":"2024-09-16T15:45:31.697417+0000","last_fullsized":"2024-09-16T15:45:31.697417+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:36.274150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"121'4","reported_seq":19,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697261+0000","last_change":"2024-09-16T15:45:29.180725+0000","last_active":"2024-09-16T15:45:31.697261+0000","last_peered":"2024-09-16T15:45:31.697261+0000","last_clean":"2024-09-16T15:45:31.697261+0000","last_became_active":"2024-09-16T15:45:29.180416+0000","last_became_peered":"2024-09-16T15:45:29.180416+0000","last_unstale":"2024-09-16T15:45:31.697261+0000","last_undegraded":"2024-09-16T15:45:31.697261+0000","last_fullsized":"2024-09-16T15:45:31.697261+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:47:32.675896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179599+0000","last_change":"2024-09-16T15:45:30.193216+0000","last_active":"2024-09-16T15:45:31.179599+0000","last_peered":"2024-09-16T15:45:31.179599+0000","last_clean":"2024-09-16T15:45:31.179599+0000","last_became_active":"2024-09-16T15:45:30.192913+0000","last_became_peered":"2024-09-16T15:45:30.192913+0000","last_unstale":"2024-09-16T15:45:31.179599+0000","last_undegraded":"2024-09-16T15:45:31.179599+0000","last_fullsized":"2024-09-16T15:45:31.179599+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:46.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696452+0000","last_change":"2024-09-16T15:45:29.180104+0000","last_active":"2024-09-16T15:45:31.696452+0000","last_peered":"2024-09-16T15:45:31.696452+0000","last_clean":"2024-09-16T15:45:31.696452+0000","last_became_active":"2024-09-16T15:45:29.179799+0000","last_became_peered":"2024-09-16T15:45:29.179799+0000","last_unstale":"2024-09-16T15:45:31.696452+0000","last_undegraded":"2024-09-16T15:45:31.696452+0000","last_fullsized":"2024-09-16T15:45:31.696452+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:08.291680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697084+0000","last_change":"2024-09-16T15:45:30.186256+0000","last_active":"2024-09-16T15:45:31.697084+0000","last_peered":"2024-09-16T15:45:31.697084+0000","last_clean":"2024-09-16T15:45:31.697084+0000","last_became_active":"2024-09-16T15:45:30.185665+0000","last_became_peered":"2024-09-16T15:45:30.185665+0000","last_unstale":"2024-09-16T15:45:31.697084+0000","last_undegraded":"2024-09-16T15:45:31.697084+0000","last_fullsized":"2024-09-16T15:45:31.697084+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:49.353996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176716+0000","last_change":"2024-09-16T15:45:29.184056+0000","last_active":"2024-09-16T15:45:31.176716+0000","last_peered":"2024-09-16T15:45:31.176716+0000","last_clean":"2024-09-16T15:45:31.176716+0000","last_became_active":"2024-09-16T15:45:29.182998+0000","last_became_peered":"2024-09-16T15:45:29.182998+0000","last_unstale":"2024-09-16T15:45:31.176716+0000","last_undegraded":"2024-09-16T15:45:31.176716+0000","last_fullsized":"2024-09-16T15:45:31.176716+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:19.489850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696970+0000","last_change":"2024-09-16T15:45:30.181498+0000","last_active":"2024-09-16T15:45:31.696970+0000","last_peered":"2024-09-16T15:45:31.696970+0000","last_clean":"2024-09-16T15:45:31.696970+0000","last_became_active":"2024-09-16T15:45:30.180766+0000","last_became_peered":"2024-09-16T15:45:30.180766+0000","last_unstale":"2024-09-16T15:45:31.696970+0000","last_undegraded":"2024-09-16T15:45:31.696970+0000","last_fullsized":"2024-09-16T15:45:31.696970+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:59.236504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466121+0000","last_change":"2024-09-16T15:45:29.186646+0000","last_active":"2024-09-16T15:45:31.466121+0000","last_peered":"2024-09-16T15:45:31.466121+0000","last_clean":"2024-09-16T15:45:31.466121+0000","last_became_active":"2024-09-16T15:45:29.185458+0000","last_became_peered":"2024-09-16T15:45:29.185458+0000","last_unstale":"2024-09-16T15:45:31.466121+0000","last_undegraded":"2024-09-16T15:45:31.466121+0000","last_fullsized":"2024-09-16T15:45:31.466121+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:17.462421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696369+0000","last_change":"2024-09-16T15:45:30.190056+0000","last_active":"2024-09-16T15:45:31.696369+0000","last_peered":"2024-09-16T15:45:31.696369+0000","last_clean":"2024-09-16T15:45:31.696369+0000","last_became_active":"2024-09-16T15:45:30.189699+0000","last_became_peered":"2024-09-16T15:45:30.189699+0000","last_unstale":"2024-09-16T15:45:31.696369+0000","last_undegraded":"2024-09-16T15:45:31.696369+0000","last_fullsized":"2024-09-16T15:45:31.696369+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:16.174307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465873+0000","last_change":"2024-09-16T15:45:29.180962+0000","last_active":"2024-09-16T15:45:31.465873+0000","last_peered":"2024-09-16T15:45:31.465873+0000","last_clean":"2024-09-16T15:45:31.465873+0000","last_became_active":"2024-09-16T15:45:29.180678+0000","last_became_peered":"2024-09-16T15:45:29.180678+0000","last_unstale":"2024-09-16T15:45:31.465873+0000","last_undegraded":"2024-09-16T15:45:31.465873+0000","last_fullsized":"2024-09-16T15:45:31.465873+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:26.903652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177689+0000","last_change":"2024-09-16T15:45:30.192822+0000","last_active":"2024-09-16T15:45:31.177689+0000","last_peered":"2024-09-16T15:45:31.177689+0000","last_clean":"2024-09-16T15:45:31.177689+0000","last_became_active":"2024-09-16T15:45:30.192554+0000","last_became_peered":"2024-09-16T15:45:30.192554+0000","last_unstale":"2024-09-16T15:45:31.177689+0000","last_undegraded":"2024-09-16T15:45:31.177689+0000","last_fullsized":"2024-09-16T15:45:31.177689+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:46.682615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696479+0000","last_change":"2024-09-16T15:45:29.185985+0000","last_active":"2024-09-16T15:45:31.696479+0000","last_peered":"2024-09-16T15:45:31.696479+0000","last_clean":"2024-09-16T15:45:31.696479+0000","last_became_active":"2024-09-16T15:45:29.185728+0000","last_became_peered":"2024-09-16T15:45:29.185728+0000","last_unstale":"2024-09-16T15:45:31.696479+0000","last_undegraded":"2024-09-16T15:45:31.696479+0000","last_fullsized":"2024-09-16T15:45:31.696479+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:39.433170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179066+0000","last_change":"2024-09-16T15:45:30.182333+0000","last_active":"2024-09-16T15:45:31.179066+0000","last_peered":"2024-09-16T15:45:31.179066+0000","last_clean":"2024-09-16T15:45:31.179066+0000","last_became_active":"2024-09-16T15:45:30.181398+0000","last_became_peered":"2024-09-16T15:45:30.181398+0000","last_unstale":"2024-09-16T15:45:31.179066+0000","last_undegraded":"2024-09-16T15:45:31.179066+0000","last_fullsized":"2024-09-16T15:45:31.179066+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:36.098523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177213+0000","last_change":"2024-09-16T15:45:29.174336+0000","last_active":"2024-09-16T15:45:31.177213+0000","last_peered":"2024-09-16T15:45:31.177213+0000","last_clean":"2024-09-16T15:45:31.177213+0000","last_became_active":"2024-09-16T15:45:29.174064+0000","last_became_peered":"2024-09-16T15:45:29.174064+0000","last_unstale":"2024-09-16T15:45:31.177213+0000","last_undegraded":"2024-09-16T15:45:31.177213+0000","last_fullsized":"2024-09-16T15:45:31.177213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:23.625293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465953+0000","last_change":"2024-09-16T15:45:30.183317+0000","last_active":"2024-09-16T15:45:31.465953+0000","last_peered":"2024-09-16T15:45:31.465953+0000","last_clean":"2024-09-16T15:45:31.465953+0000","last_became_active":"2024-09-16T15:45:30.178662+0000","last_became_peered":"2024-09-16T15:45:30.178662+0000","last_unstale":"2024-09-16T15:45:31.465953+0000","last_undegraded":"2024-09-16T15:45:31.465953+0000","last_fullsized":"2024-09-16T15:45:31.465953+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:54.779284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465961+0000","last_change":"2024-09-16T15:45:29.175175+0000","last_active":"2024-09-16T15:45:31.465961+0000","last_peered":"2024-09-16T15:45:31.465961+0000","last_clean":"2024-09-16T15:45:31.465961+0000","last_became_active":"2024-09-16T15:45:29.174872+0000","last_became_peered":"2024-09-16T15:45:29.174872+0000","last_unstale":"2024-09-16T15:45:31.465961+0000","last_undegraded":"2024-09-16T15:45:31.465961+0000","last_fullsized":"2024-09-16T15:45:31.465961+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:41.567223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697215+0000","last_change":"2024-09-16T15:45:30.195424+0000","last_active":"2024-09-16T15:45:31.697215+0000","last_peered":"2024-09-16T15:45:31.697215+0000","last_clean":"2024-09-16T15:45:31.697215+0000","last_became_active":"2024-09-16T15:45:30.195158+0000","last_became_peered":"2024-09-16T15:45:30.195158+0000","last_unstale":"2024-09-16T15:45:31.697215+0000","last_undegraded":"2024-09-16T15:45:31.697215+0000","last_fullsized":"2024-09-16T15:45:31.697215+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:00:03.919278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697046+0000","last_change":"2024-09-16T15:45:29.181145+0000","last_active":"2024-09-16T15:45:31.697046+0000","last_peered":"2024-09-16T15:45:31.697046+0000","last_clean":"2024-09-16T15:45:31.697046+0000","last_became_active":"2024-09-16T15:45:29.180864+0000","last_became_peered":"2024-09-16T15:45:29.180864+0000","last_unstale":"2024-09-16T15:45:31.697046+0000","last_undegraded":"2024-09-16T15:45:31.697046+0000","last_fullsized":"2024-09-16T15:45:31.697046+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696863+0000","last_change":"2024-09-16T15:45:30.185081+0000","last_active":"2024-09-16T15:45:31.696863+0000","last_peered":"2024-09-16T15:45:31.696863+0000","last_clean":"2024-09-16T15:45:31.696863+0000","last_became_active":"2024-09-16T15:45:30.184778+0000","last_became_peered":"2024-09-16T15:45:30.184778+0000","last_unstale":"2024-09-16T15:45:31.696863+0000","last_undegraded":"2024-09-16T15:45:31.696863+0000","last_fullsized":"2024-09-16T15:45:31.696863+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:14.926640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697423+0000","last_change":"2024-09-16T15:45:29.181119+0000","last_active":"2024-09-16T15:45:31.697423+0000","last_peered":"2024-09-16T15:45:31.697423+0000","last_clean":"2024-09-16T15:45:31.697423+0000","last_became_active":"2024-09-16T15:45:29.180832+0000","last_became_peered":"2024-09-16T15:45:29.180832+0000","last_unstale":"2024-09-16T15:45:31.697423+0000","last_undegraded":"2024-09-16T15:45:31.697423+0000","last_fullsized":"2024-09-16T15:45:31.697423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:47:32.675896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697291+0000","last_change":"2024-09-16T15:45:30.187570+0000","last_active":"2024-09-16T15:45:31.697291+0000","last_peered":"2024-09-16T15:45:31.697291+0000","last_clean":"2024-09-16T15:45:31.697291+0000","last_became_active":"2024-09-16T15:45:30.187022+0000","last_became_peered":"2024-09-16T15:45:30.187022+0000","last_unstale":"2024-09-16T15:45:31.697291+0000","last_undegraded":"2024-09-16T15:45:31.697291+0000","last_fullsized":"2024-09-16T15:45:31.697291+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:28.685150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"121'3","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696583+0000","last_change":"2024-09-16T15:45:29.177762+0000","last_active":"2024-09-16T15:45:31.696583+0000","last_peered":"2024-09-16T15:45:31.696583+0000","last_clean":"2024-09-16T15:45:31.696583+0000","last_became_active":"2024-09-16T15:45:29.177395+0000","last_became_peered":"2024-09-16T15:45:29.177395+0000","last_unstale":"2024-09-16T15:45:31.696583+0000","last_undegraded":"2024-09-16T15:45:31.696583+0000","last_fullsized":"2024-09-16T15:45:31.696583+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:16.095883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697208+0000","last_change":"2024-09-16T15:45:30.190738+0000","last_active":"2024-09-16T15:45:31.697208+0000","last_peered":"2024-09-16T15:45:31.697208+0000","last_clean":"2024-09-16T15:45:31.697208+0000","last_became_active":"2024-09-16T15:45:30.190439+0000","last_became_peered":"2024-09-16T15:45:30.190439+0000","last_unstale":"2024-09-16T15:45:31.697208+0000","last_undegraded":"2024-09-16T15:45:31.697208+0000","last_fullsized":"2024-09-16T15:45:31.697208+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:24.782249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"121'6","reported_seq":21,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179133+0000","last_change":"2024-09-16T15:45:29.171700+0000","last_active":"2024-09-16T15:45:31.179133+0000","last_peered":"2024-09-16T15:45:31.179133+0000","last_clean":"2024-09-16T15:45:31.179133+0000","last_became_active":"2024-09-16T15:45:29.171262+0000","last_became_peered":"2024-09-16T15:45:29.171262+0000","last_unstale":"2024-09-16T15:45:31.179133+0000","last_undegraded":"2024-09-16T15:45:31.179133+0000","last_fullsized":"2024-09-16T15:45:31.179133+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:40.779854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696819+0000","last_change":"2024-09-16T15:45:30.195921+0000","last_active":"2024-09-16T15:45:31.696819+0000","last_peered":"2024-09-16T15:45:31.696819+0000","last_clean":"2024-09-16T15:45:31.696819+0000","last_became_active":"2024-09-16T15:45:30.195621+0000","last_became_peered":"2024-09-16T15:45:30.195621+0000","last_unstale":"2024-09-16T15:45:31.696819+0000","last_undegraded":"2024-09-16T15:45:31.696819+0000","last_fullsized":"2024-09-16T15:45:31.696819+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:07.466423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696749+0000","last_change":"2024-09-16T15:45:29.170591+0000","last_active":"2024-09-16T15:45:31.696749+0000","last_peered":"2024-09-16T15:45:31.696749+0000","last_clean":"2024-09-16T15:45:31.696749+0000","last_became_active":"2024-09-16T15:45:29.170281+0000","last_became_peered":"2024-09-16T15:45:29.170281+0000","last_unstale":"2024-09-16T15:45:31.696749+0000","last_undegraded":"2024-09-16T15:45:31.696749+0000","last_fullsized":"2024-09-16T15:45:31.696749+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:30.045314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465858+0000","last_change":"2024-09-16T15:45:30.167207+0000","last_active":"2024-09-16T15:45:31.465858+0000","last_peered":"2024-09-16T15:45:31.465858+0000","last_clean":"2024-09-16T15:45:31.465858+0000","last_became_active":"2024-09-16T15:45:30.166907+0000","last_became_peered":"2024-09-16T15:45:30.166907+0000","last_unstale":"2024-09-16T15:45:31.465858+0000","last_undegraded":"2024-09-16T15:45:31.465858+0000","last_fullsized":"2024-09-16T15:45:31.465858+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:38.507200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465974+0000","last_change":"2024-09-16T15:45:29.181674+0000","last_active":"2024-09-16T15:45:31.465974+0000","last_peered":"2024-09-16T15:45:31.465974+0000","last_clean":"2024-09-16T15:45:31.465974+0000","last_became_active":"2024-09-16T15:45:29.181334+0000","last_became_peered":"2024-09-16T15:45:29.181334+0000","last_unstale":"2024-09-16T15:45:31.465974+0000","last_undegraded":"2024-09-16T15:45:31.465974+0000","last_fullsized":"2024-09-16T15:45:31.465974+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:21:51.167893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.178887+0000","last_change":"2024-09-16T15:45:30.182337+0000","last_active":"2024-09-16T15:45:31.178887+0000","last_peered":"2024-09-16T15:45:31.178887+0000","last_clean":"2024-09-16T15:45:31.178887+0000","last_became_active":"2024-09-16T15:45:30.179787+0000","last_became_peered":"2024-09-16T15:45:30.179787+0000","last_unstale":"2024-09-16T15:45:31.178887+0000","last_undegraded":"2024-09-16T15:45:31.178887+0000","last_fullsized":"2024-09-16T15:45:31.178887+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:12.436952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"121'8","reported_seq":23,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179180+0000","last_change":"2024-09-16T15:45:29.178341+0000","last_active":"2024-09-16T15:45:31.179180+0000","last_peered":"2024-09-16T15:45:31.179180+0000","last_clean":"2024-09-16T15:45:31.179180+0000","last_became_active":"2024-09-16T15:45:29.177742+0000","last_became_peered":"2024-09-16T15:45:29.177742+0000","last_unstale":"2024-09-16T15:45:31.179180+0000","last_undegraded":"2024-09-16T15:45:31.179180+0000","last_fullsized":"2024-09-16T15:45:31.179180+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:55:57.521505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179512+0000","last_change":"2024-09-16T15:45:30.182824+0000","last_active":"2024-09-16T15:45:31.179512+0000","last_peered":"2024-09-16T15:45:31.179512+0000","last_clean":"2024-09-16T15:45:31.179512+0000","last_became_active":"2024-09-16T15:45:30.182533+0000","last_became_peered":"2024-09-16T15:45:30.182533+0000","last_unstale":"2024-09-16T15:45:31.179512+0000","last_undegraded":"2024-09-16T15:45:31.179512+0000","last_fullsized":"2024-09-16T15:45:31.179512+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:01.388439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697172+0000","last_change":"2024-09-16T15:45:29.178358+0000","last_active":"2024-09-16T15:45:31.697172+0000","last_peered":"2024-09-16T15:45:31.697172+0000","last_clean":"2024-09-16T15:45:31.697172+0000","last_became_active":"2024-09-16T15:45:29.177952+0000","last_became_peered":"2024-09-16T15:45:29.177952+0000","last_unstale":"2024-09-16T15:45:31.697172+0000","last_undegraded":"2024-09-16T15:45:31.697172+0000","last_fullsized":"2024-09-16T15:45:31.697172+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:33.711534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697044+0000","last_change":"2024-09-16T15:45:30.194307+0000","last_active":"2024-09-16T15:45:31.697044+0000","last_peered":"2024-09-16T15:45:31.697044+0000","last_clean":"2024-09-16T15:45:31.697044+0000","last_became_active":"2024-09-16T15:45:30.193939+0000","last_became_peered":"2024-09-16T15:45:30.193939+0000","last_unstale":"2024-09-16T15:45:31.697044+0000","last_undegraded":"2024-09-16T15:45:31.697044+0000","last_fullsized":"2024-09-16T15:45:31.697044+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:13:26.141198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176486+0000","last_change":"2024-09-16T15:45:29.172406+0000","last_active":"2024-09-16T15:45:31.176486+0000","last_peered":"2024-09-16T15:45:31.176486+0000","last_clean":"2024-09-16T15:45:31.176486+0000","last_became_active":"2024-09-16T15:45:29.171452+0000","last_became_peered":"2024-09-16T15:45:29.171452+0000","last_unstale":"2024-09-16T15:45:31.176486+0000","last_undegraded":"2024-09-16T15:45:31.176486+0000","last_fullsized":"2024-09-16T15:45:31.176486+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:40.401517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697280+0000","last_change":"2024-09-16T15:45:30.186855+0000","last_active":"2024-09-16T15:45:31.697280+0000","last_peered":"2024-09-16T15:45:31.697280+0000","last_clean":"2024-09-16T15:45:31.697280+0000","last_became_active":"2024-09-16T15:45:30.186571+0000","last_became_peered":"2024-09-16T15:45:30.186571+0000","last_unstale":"2024-09-16T15:45:31.697280+0000","last_undegraded":"2024-09-16T15:45:31.697280+0000","last_fullsized":"2024-09-16T15:45:31.697280+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:37.740173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177765+0000","last_change":"2024-09-16T15:45:29.177128+0000","last_active":"2024-09-16T15:45:31.177765+0000","last_peered":"2024-09-16T15:45:31.177765+0000","last_clean":"2024-09-16T15:45:31.177765+0000","last_became_active":"2024-09-16T15:45:29.176810+0000","last_became_peered":"2024-09-16T15:45:29.176810+0000","last_unstale":"2024-09-16T15:45:31.177765+0000","last_undegraded":"2024-09-16T15:45:31.177765+0000","last_fullsized":"2024-09-16T15:45:31.177765+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:47.665949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466316+0000","last_change":"2024-09-16T15:45:30.182507+0000","last_active":"2024-09-16T15:45:31.466316+0000","last_peered":"2024-09-16T15:45:31.466316+0000","last_clean":"2024-09-16T15:45:31.466316+0000","last_became_active":"2024-09-16T15:45:30.179003+0000","last_became_peered":"2024-09-16T15:45:30.179003+0000","last_unstale":"2024-09-16T15:45:31.466316+0000","last_undegraded":"2024-09-16T15:45:31.466316+0000","last_fullsized":"2024-09-16T15:45:31.466316+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:04.547979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696495+0000","last_change":"2024-09-16T15:45:29.176966+0000","last_active":"2024-09-16T15:45:31.696495+0000","last_peered":"2024-09-16T15:45:31.696495+0000","last_clean":"2024-09-16T15:45:31.696495+0000","last_became_active":"2024-09-16T15:45:29.176398+0000","last_became_peered":"2024-09-16T15:45:29.176398+0000","last_unstale":"2024-09-16T15:45:31.696495+0000","last_undegraded":"2024-09-16T15:45:31.696495+0000","last_fullsized":"2024-09-16T15:45:31.696495+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:55.181737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696641+0000","last_change":"2024-09-16T15:45:30.185485+0000","last_active":"2024-09-16T15:45:31.696641+0000","last_peered":"2024-09-16T15:45:31.696641+0000","last_clean":"2024-09-16T15:45:31.696641+0000","last_became_active":"2024-09-16T15:45:30.185187+0000","last_became_peered":"2024-09-16T15:45:30.185187+0000","last_unstale":"2024-09-16T15:45:31.696641+0000","last_undegraded":"2024-09-16T15:45:31.696641+0000","last_fullsized":"2024-09-16T15:45:31.696641+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:04.525210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696425+0000","last_change":"2024-09-16T15:45:29.182582+0000","last_active":"2024-09-16T15:45:31.696425+0000","last_peered":"2024-09-16T15:45:31.696425+0000","last_clean":"2024-09-16T15:45:31.696425+0000","last_became_active":"2024-09-16T15:45:29.182296+0000","last_became_peered":"2024-09-16T15:45:29.182296+0000","last_unstale":"2024-09-16T15:45:31.696425+0000","last_undegraded":"2024-09-16T15:45:31.696425+0000","last_fullsized":"2024-09-16T15:45:31.696425+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:46.819724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.178230+0000","last_change":"2024-09-16T15:45:30.193342+0000","last_active":"2024-09-16T15:45:31.178230+0000","last_peered":"2024-09-16T15:45:31.178230+0000","last_clean":"2024-09-16T15:45:31.178230+0000","last_became_active":"2024-09-16T15:45:30.193060+0000","last_became_peered":"2024-09-16T15:45:30.193060+0000","last_unstale":"2024-09-16T15:45:31.178230+0000","last_undegraded":"2024-09-16T15:45:31.178230+0000","last_fullsized":"2024-09-16T15:45:31.178230+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:30.147795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696573+0000","last_change":"2024-09-16T15:45:29.177474+0000","last_active":"2024-09-16T15:45:31.696573+0000","last_peered":"2024-09-16T15:45:31.696573+0000","last_clean":"2024-09-16T15:45:31.696573+0000","last_became_active":"2024-09-16T15:45:29.177148+0000","last_became_peered":"2024-09-16T15:45:29.177148+0000","last_unstale":"2024-09-16T15:45:31.696573+0000","last_undegraded":"2024-09-16T15:45:31.696573+0000","last_fullsized":"2024-09-16T15:45:31.696573+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:53.240059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465855+0000","last_change":"2024-09-16T15:45:30.183144+0000","last_active":"2024-09-16T15:45:31.465855+0000","last_peered":"2024-09-16T15:45:31.465855+0000","last_clean":"2024-09-16T15:45:31.465855+0000","last_became_active":"2024-09-16T15:45:30.182866+0000","last_became_peered":"2024-09-16T15:45:30.182866+0000","last_unstale":"2024-09-16T15:45:31.465855+0000","last_undegraded":"2024-09-16T15:45:31.465855+0000","last_fullsized":"2024-09-16T15:45:31.465855+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:48.399155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696753+0000","last_change":"2024-09-16T15:45:29.178091+0000","last_active":"2024-09-16T15:45:31.696753+0000","last_peered":"2024-09-16T15:45:31.696753+0000","last_clean":"2024-09-16T15:45:31.696753+0000","last_became_active":"2024-09-16T15:45:29.177797+0000","last_became_peered":"2024-09-16T15:45:29.177797+0000","last_unstale":"2024-09-16T15:45:31.696753+0000","last_undegraded":"2024-09-16T15:45:31.696753+0000","last_fullsized":"2024-09-16T15:45:31.696753+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:08.537058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696361+0000","last_change":"2024-09-16T15:45:30.186619+0000","last_active":"2024-09-16T15:45:31.696361+0000","last_peered":"2024-09-16T15:45:31.696361+0000","last_clean":"2024-09-16T15:45:31.696361+0000","last_became_active":"2024-09-16T15:45:30.186327+0000","last_became_peered":"2024-09-16T15:45:30.186327+0000","last_unstale":"2024-09-16T15:45:31.696361+0000","last_undegraded":"2024-09-16T15:45:31.696361+0000","last_fullsized":"2024-09-16T15:45:31.696361+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:41.276942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176124+0000","last_change":"2024-09-16T15:45:29.181970+0000","last_active":"2024-09-16T15:45:31.176124+0000","last_peered":"2024-09-16T15:45:31.176124+0000","last_clean":"2024-09-16T15:45:31.176124+0000","last_became_active":"2024-09-16T15:45:29.179340+0000","last_became_peered":"2024-09-16T15:45:29.179340+0000","last_unstale":"2024-09-16T15:45:31.176124+0000","last_undegraded":"2024-09-16T15:45:31.176124+0000","last_fullsized":"2024-09-16T15:45:31.176124+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:48.828545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177891+0000","last_change":"2024-09-16T15:45:30.182837+0000","last_active":"2024-09-16T15:45:31.177891+0000","last_peered":"2024-09-16T15:45:31.177891+0000","last_clean":"2024-09-16T15:45:31.177891+0000","last_became_active":"2024-09-16T15:45:30.181331+0000","last_became_peered":"2024-09-16T15:45:30.181331+0000","last_unstale":"2024-09-16T15:45:31.177891+0000","last_undegraded":"2024-09-16T15:45:31.177891+0000","last_fullsized":"2024-09-16T15:45:31.177891+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:37:56.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696958+0000","last_change":"2024-09-16T15:45:29.176979+0000","last_active":"2024-09-16T15:45:31.696958+0000","last_peered":"2024-09-16T15:45:31.696958+0000","last_clean":"2024-09-16T15:45:31.696958+0000","last_became_active":"2024-09-16T15:45:29.176699+0000","last_became_peered":"2024-09-16T15:45:29.176699+0000","last_unstale":"2024-09-16T15:45:31.696958+0000","last_undegraded":"2024-09-16T15:45:31.696958+0000","last_fullsized":"2024-09-16T15:45:31.696958+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:53.348328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696852+0000","last_change":"2024-09-16T15:45:30.191425+0000","last_active":"2024-09-16T15:45:31.696852+0000","last_peered":"2024-09-16T15:45:31.696852+0000","last_clean":"2024-09-16T15:45:31.696852+0000","last_became_active":"2024-09-16T15:45:30.190489+0000","last_became_peered":"2024-09-16T15:45:30.190489+0000","last_unstale":"2024-09-16T15:45:31.696852+0000","last_undegraded":"2024-09-16T15:45:31.696852+0000","last_fullsized":"2024-09-16T15:45:31.696852+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:06.893466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696999+0000","last_change":"2024-09-16T15:45:29.173375+0000","last_active":"2024-09-16T15:45:31.696999+0000","last_peered":"2024-09-16T15:45:31.696999+0000","last_clean":"2024-09-16T15:45:31.696999+0000","last_became_active":"2024-09-16T15:45:29.172775+0000","last_became_peered":"2024-09-16T15:45:29.172775+0000","last_unstale":"2024-09-16T15:45:31.696999+0000","last_undegraded":"2024-09-16T15:45:31.696999+0000","last_fullsized":"2024-09-16T15:45:31.696999+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:08.537058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466198+0000","last_change":"2024-09-16T15:45:30.180342+0000","last_active":"2024-09-16T15:45:31.466198+0000","last_peered":"2024-09-16T15:45:31.466198+0000","last_clean":"2024-09-16T15:45:31.466198+0000","last_became_active":"2024-09-16T15:45:30.180095+0000","last_became_peered":"2024-09-16T15:45:30.180095+0000","last_unstale":"2024-09-16T15:45:31.466198+0000","last_undegraded":"2024-09-16T15:45:31.466198+0000","last_fullsized":"2024-09-16T15:45:31.466198+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:18.944471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466239+0000","last_change":"2024-09-16T15:45:29.182547+0000","last_active":"2024-09-16T15:45:31.466239+0000","last_peered":"2024-09-16T15:45:31.466239+0000","last_clean":"2024-09-16T15:45:31.466239+0000","last_became_active":"2024-09-16T15:45:29.182066+0000","last_became_peered":"2024-09-16T15:45:29.182066+0000","last_unstale":"2024-09-16T15:45:31.466239+0000","last_undegraded":"2024-09-16T15:45:31.466239+0000","last_fullsized":"2024-09-16T15:45:31.466239+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:51.852544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177778+0000","last_change":"2024-09-16T15:45:30.190109+0000","last_active":"2024-09-16T15:45:31.177778+0000","last_peered":"2024-09-16T15:45:31.177778+0000","last_clean":"2024-09-16T15:45:31.177778+0000","last_became_active":"2024-09-16T15:45:30.188648+0000","last_became_peered":"2024-09-16T15:45:30.188648+0000","last_unstale":"2024-09-16T15:45:31.177778+0000","last_undegraded":"2024-09-16T15:45:31.177778+0000","last_fullsized":"2024-09-16T15:45:31.177778+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.131402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176451+0000","last_change":"2024-09-16T15:45:29.186287+0000","last_active":"2024-09-16T15:45:31.176451+0000","last_peered":"2024-09-16T15:45:31.176451+0000","last_clean":"2024-09-16T15:45:31.176451+0000","last_became_active":"2024-09-16T15:45:29.185838+0000","last_became_peered":"2024-09-16T15:45:29.185838+0000","last_unstale":"2024-09-16T15:45:31.176451+0000","last_undegraded":"2024-09-16T15:45:31.176451+0000","last_fullsized":"2024-09-16T15:45:31.176451+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:33:30.424655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696709+0000","last_change":"2024-09-16T15:45:30.183868+0000","last_active":"2024-09-16T15:45:31.696709+0000","last_peered":"2024-09-16T15:45:31.696709+0000","last_clean":"2024-09-16T15:45:31.696709+0000","last_became_active":"2024-09-16T15:45:30.183554+0000","last_became_peered":"2024-09-16T15:45:30.183554+0000","last_unstale":"2024-09-16T15:45:31.696709+0000","last_undegraded":"2024-09-16T15:45:31.696709+0000","last_fullsized":"2024-09-16T15:45:31.696709+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:58.058490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465930+0000","last_change":"2024-09-16T15:45:29.177657+0000","last_active":"2024-09-16T15:45:31.465930+0000","last_peered":"2024-09-16T15:45:31.465930+0000","last_clean":"2024-09-16T15:45:31.465930+0000","last_became_active":"2024-09-16T15:45:29.177113+0000","last_became_peered":"2024-09-16T15:45:29.177113+0000","last_unstale":"2024-09-16T15:45:31.465930+0000","last_undegraded":"2024-09-16T15:45:31.465930+0000","last_fullsized":"2024-09-16T15:45:31.465930+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:56.944663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696713+0000","last_change":"2024-09-16T15:45:30.180641+0000","last_active":"2024-09-16T15:45:31.696713+0000","last_peered":"2024-09-16T15:45:31.696713+0000","last_clean":"2024-09-16T15:45:31.696713+0000","last_became_active":"2024-09-16T15:45:30.180377+0000","last_became_peered":"2024-09-16T15:45:30.180377+0000","last_unstale":"2024-09-16T15:45:31.696713+0000","last_undegraded":"2024-09-16T15:45:31.696713+0000","last_fullsized":"2024-09-16T15:45:31.696713+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:27.325989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696812+0000","last_change":"2024-09-16T15:45:29.181524+0000","last_active":"2024-09-16T15:45:31.696812+0000","last_peered":"2024-09-16T15:45:31.696812+0000","last_clean":"2024-09-16T15:45:31.696812+0000","last_became_active":"2024-09-16T15:45:29.181222+0000","last_became_peered":"2024-09-16T15:45:29.181222+0000","last_unstale":"2024-09-16T15:45:31.696812+0000","last_undegraded":"2024-09-16T15:45:31.696812+0000","last_fullsized":"2024-09-16T15:45:31.696812+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:07.983234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696293+0000","last_change":"2024-09-16T15:45:30.194204+0000","last_active":"2024-09-16T15:45:31.696293+0000","last_peered":"2024-09-16T15:45:31.696293+0000","last_clean":"2024-09-16T15:45:31.696293+0000","last_became_active":"2024-09-16T15:45:30.193106+0000","last_became_peered":"2024-09-16T15:45:30.193106+0000","last_unstale":"2024-09-16T15:45:31.696293+0000","last_undegraded":"2024-09-16T15:45:31.696293+0000","last_fullsized":"2024-09-16T15:45:31.696293+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:18.852050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696842+0000","last_change":"2024-09-16T15:45:29.184118+0000","last_active":"2024-09-16T15:45:31.696842+0000","last_peered":"2024-09-16T15:45:31.696842+0000","last_clean":"2024-09-16T15:45:31.696842+0000","last_became_active":"2024-09-16T15:45:29.179258+0000","last_became_peered":"2024-09-16T15:45:29.179258+0000","last_unstale":"2024-09-16T15:45:31.696842+0000","last_undegraded":"2024-09-16T15:45:31.696842+0000","last_fullsized":"2024-09-16T15:45:31.696842+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:50.713882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465889+0000","last_change":"2024-09-16T15:45:30.177930+0000","last_active":"2024-09-16T15:45:31.465889+0000","last_peered":"2024-09-16T15:45:31.465889+0000","last_clean":"2024-09-16T15:45:31.465889+0000","last_became_active":"2024-09-16T15:45:30.177511+0000","last_became_peered":"2024-09-16T15:45:30.177511+0000","last_unstale":"2024-09-16T15:45:31.465889+0000","last_undegraded":"2024-09-16T15:45:31.465889+0000","last_fullsized":"2024-09-16T15:45:31.465889+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:54:27.832197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176222+0000","last_change":"2024-09-16T15:45:29.185510+0000","last_active":"2024-09-16T15:45:31.176222+0000","last_peered":"2024-09-16T15:45:31.176222+0000","last_clean":"2024-09-16T15:45:31.176222+0000","last_became_active":"2024-09-16T15:45:29.184491+0000","last_became_peered":"2024-09-16T15:45:29.184491+0000","last_unstale":"2024-09-16T15:45:31.176222+0000","last_undegraded":"2024-09-16T15:45:31.176222+0000","last_fullsized":"2024-09-16T15:45:31.176222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:43.426732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696698+0000","last_change":"2024-09-16T15:45:30.186476+0000","last_active":"2024-09-16T15:45:31.696698+0000","last_peered":"2024-09-16T15:45:31.696698+0000","last_clean":"2024-09-16T15:45:31.696698+0000","last_became_active":"2024-09-16T15:45:30.186138+0000","last_became_peered":"2024-09-16T15:45:30.186138+0000","last_unstale":"2024-09-16T15:45:31.696698+0000","last_undegraded":"2024-09-16T15:45:31.696698+0000","last_fullsized":"2024-09-16T15:45:31.696698+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:01.063888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696782+0000","last_change":"2024-09-16T15:45:29.180566+0000","last_active":"2024-09-16T15:45:31.696782+0000","last_peered":"2024-09-16T15:45:31.696782+0000","last_clean":"2024-09-16T15:45:31.696782+0000","last_became_active":"2024-09-16T15:45:29.180179+0000","last_became_peered":"2024-09-16T15:45:29.180179+0000","last_unstale":"2024-09-16T15:45:31.696782+0000","last_undegraded":"2024-09-16T15:45:31.696782+0000","last_fullsized":"2024-09-16T15:45:31.696782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:50.713882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":321,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696056+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:45:31.696056+0000","last_peered":"2024-09-16T15:45:31.696056+0000","last_clean":"2024-09-16T15:45:31.696056+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:45:31.696056+0000","last_undegraded":"2024-09-16T15:45:31.696056+0000","last_fullsized":"2024-09-16T15:45:31.696056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509530,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33264,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2121728,"data_stored":1253293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1527808,"data_stored":668615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509528,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1523712,"data_stored":667115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509528,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":667073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509529,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36828,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335012,"statfs":{"total":96636764160,"available":96599052288,"internally_reserved":0,"allocated":1511424,"data_stored":660835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":1511424,"data_stored":661291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":1503232,"data_stored":660675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2109440,"data_stored":1257371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:45:34.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:45:34.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.625+0000 7f8d77e00640 1 -- 172.21.15.60:0/1014505465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d5c005210 msgr2=0x7f8d5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.625+0000 7f8d77e00640 1 --2- 172.21.15.60:0/1014505465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d5c005210 0x7f8d5c00b9c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009b20 tx=0x7f8d5c005830 comp rx=0 tx=0).stop 2024-09-16T15:45:34.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.625+0000 7f8d77e00640 1 -- 172.21.15.60:0/1014505465 shutdown_connections 2024-09-16T15:45:34.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.625+0000 7f8d77e00640 1 --2- 172.21.15.60:0/1014505465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d5c005210 0x7f8d5c00b9c0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.625+0000 7f8d77e00640 1 -- 172.21.15.60:0/1014505465 >> 172.21.15.60:0/1014505465 conn(0x7f8d780cfa20 msgr2=0x7f8d780cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:34.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.625+0000 7f8d77e00640 1 -- 172.21.15.60:0/1014505465 shutdown_connections 2024-09-16T15:45:34.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.625+0000 7f8d77e00640 1 -- 172.21.15.60:0/1014505465 wait complete. 2024-09-16T15:45:34.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.627+0000 7f8d77e00640 1 Processor -- start 2024-09-16T15:45:34.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.627+0000 7f8d77e00640 1 -- start start 2024-09-16T15:45:34.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5c005210 0x7f8d780b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:34.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d78157680 0x7f8d78154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:34.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d78157b00 0x7f8d781551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d77e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8d780b2c80 con 0x7f8d78157680 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d77e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d780b2b00 con 0x7f8d78157b00 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d77e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8d78073d40 con 0x7f8d5c005210 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d6e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d78157680 0x7f8d78154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d6e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d78157680 0x7f8d78154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46808/0 (socket says 172.21.15.60:46808) 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d6e000640 1 -- 172.21.15.60:0/1990884460 learned_addr learned my addr 172.21.15.60:0/1990884460 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d77400640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d78157b00 0x7f8d781551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d76a00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5c005210 0x7f8d780b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d6e000640 1 -- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5c005210 msgr2=0x7f8d780b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d6e000640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5c005210 0x7f8d780b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d6e000640 1 -- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d78157b00 msgr2=0x7f8d781551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d6e000640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d78157b00 0x7f8d781551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d6e000640 1 -- 172.21.15.60:0/1990884460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d780b12d0 con 0x7f8d78157680 2024-09-16T15:45:34.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.628+0000 7f8d76a00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5c005210 0x7f8d780b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:34.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.629+0000 7f8d6e000640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d78157680 0x7f8d78154c80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f8d60003e10 tx=0x7f8d60001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:34.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.629+0000 7f8d6f400640 1 -- 172.21.15.60:0/1990884460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60016030 con 0x7f8d78157680 2024-09-16T15:45:34.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.629+0000 7f8d6f400640 1 -- 172.21.15.60:0/1990884460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d60011640 con 0x7f8d78157680 2024-09-16T15:45:34.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.629+0000 7f8d6f400640 1 -- 172.21.15.60:0/1990884460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011a90 con 0x7f8d78157680 2024-09-16T15:45:34.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.629+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d781580d0 con 0x7f8d78157680 2024-09-16T15:45:34.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.629+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d78158690 con 0x7f8d78157680 2024-09-16T15:45:34.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.630+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d780742a0 con 0x7f8d78157680 2024-09-16T15:45:34.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.631+0000 7f8d6f400640 1 -- 172.21.15.60:0/1990884460 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f8d60019070 con 0x7f8d78157680 2024-09-16T15:45:34.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.632+0000 7f8d6f400640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8d54078cd0 0x7f8d5407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:34.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.632+0000 7f8d6f400640 1 -- 172.21.15.60:0/1990884460 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f8d6009b830 con 0x7f8d78157680 2024-09-16T15:45:34.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.633+0000 7f8d76a00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8d54078cd0 0x7f8d5407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:34.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.633+0000 7f8d76a00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8d54078cd0 0x7f8d5407b190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8d68006fd0 tx=0x7f8d68006aa0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:34.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.636+0000 7f8d6f400640 1 -- 172.21.15.60:0/1990884460 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8d60063700 con 0x7f8d78157680 2024-09-16T15:45:34.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.832+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d780b4be0 con 0x7f8d54078cd0 2024-09-16T15:45:34.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.836+0000 7f8d6f400640 1 -- 172.21.15.60:0/1990884460 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f8d780b4be0 con 0x7f8d54078cd0 2024-09-16T15:45:34.840 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:34.841 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:45:34.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8d54078cd0 msgr2=0x7f8d5407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8d54078cd0 0x7f8d5407b190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8d68006fd0 tx=0x7f8d68006aa0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d78157680 msgr2=0x7f8d78154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:34.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d78157680 0x7f8d78154c80 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f8d60003e10 tx=0x7f8d60001630 comp rx=0 tx=0).stop 2024-09-16T15:45:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 shutdown_connections 2024-09-16T15:45:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8d54078cd0 0x7f8d5407b190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d78157b00 0x7f8d781551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d78157680 0x7f8d78154c80 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 --2- 172.21.15.60:0/1990884460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d5c005210 0x7f8d780b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.840+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 >> 172.21.15.60:0/1990884460 conn(0x7f8d780cfa20 msgr2=0x7f8d780b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.841+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 shutdown_connections 2024-09-16T15:45:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:34.841+0000 7f8d77e00640 1 -- 172.21.15.60:0/1990884460 wait complete. 2024-09-16T15:45:34.857 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-16T15:45:34.193570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279180,"kb_used_data":13016,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695540,"statfs":{"total":773094113280,"available":772808232960,"internally_reserved":0,"allocated":13328384,"data_stored":6496268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001265"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696838+0000","last_change":"2024-09-16T15:45:30.195601+0000","last_active":"2024-09-16T15:45:31.696838+0000","last_peered":"2024-09-16T15:45:31.696838+0000","last_clean":"2024-09-16T15:45:31.696838+0000","last_became_active":"2024-09-16T15:45:30.191133+0000","last_became_peered":"2024-09-16T15:45:30.191133+0000","last_unstale":"2024-09-16T15:45:31.696838+0000","last_undegraded":"2024-09-16T15:45:31.696838+0000","last_fullsized":"2024-09-16T15:45:31.696838+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:11.030668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466280+0000","last_change":"2024-09-16T15:45:30.185945+0000","last_active":"2024-09-16T15:45:31.466280+0000","last_peered":"2024-09-16T15:45:31.466280+0000","last_clean":"2024-09-16T15:45:31.466280+0000","last_became_active":"2024-09-16T15:45:30.185687+0000","last_became_peered":"2024-09-16T15:45:30.185687+0000","last_unstale":"2024-09-16T15:45:31.466280+0000","last_undegraded":"2024-09-16T15:45:31.466280+0000","last_fullsized":"2024-09-16T15:45:31.466280+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:09:16.707957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179341+0000","last_change":"2024-09-16T15:45:30.193397+0000","last_active":"2024-09-16T15:45:31.179341+0000","last_peered":"2024-09-16T15:45:31.179341+0000","last_clean":"2024-09-16T15:45:31.179341+0000","last_became_active":"2024-09-16T15:45:30.193111+0000","last_became_peered":"2024-09-16T15:45:30.193111+0000","last_unstale":"2024-09-16T15:45:31.179341+0000","last_undegraded":"2024-09-16T15:45:31.179341+0000","last_fullsized":"2024-09-16T15:45:31.179341+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:58.080375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697076+0000","last_change":"2024-09-16T15:45:30.179078+0000","last_active":"2024-09-16T15:45:31.697076+0000","last_peered":"2024-09-16T15:45:31.697076+0000","last_clean":"2024-09-16T15:45:31.697076+0000","last_became_active":"2024-09-16T15:45:30.178774+0000","last_became_peered":"2024-09-16T15:45:30.178774+0000","last_unstale":"2024-09-16T15:45:31.697076+0000","last_undegraded":"2024-09-16T15:45:31.697076+0000","last_fullsized":"2024-09-16T15:45:31.697076+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:42.445153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696910+0000","last_change":"2024-09-16T15:45:30.187642+0000","last_active":"2024-09-16T15:45:31.696910+0000","last_peered":"2024-09-16T15:45:31.696910+0000","last_clean":"2024-09-16T15:45:31.696910+0000","last_became_active":"2024-09-16T15:45:30.187354+0000","last_became_peered":"2024-09-16T15:45:30.187354+0000","last_unstale":"2024-09-16T15:45:31.696910+0000","last_undegraded":"2024-09-16T15:45:31.696910+0000","last_fullsized":"2024-09-16T15:45:31.696910+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:05.626792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.178107+0000","last_change":"2024-09-16T15:45:30.190529+0000","last_active":"2024-09-16T15:45:31.178107+0000","last_peered":"2024-09-16T15:45:31.178107+0000","last_clean":"2024-09-16T15:45:31.178107+0000","last_became_active":"2024-09-16T15:45:30.190254+0000","last_became_peered":"2024-09-16T15:45:30.190254+0000","last_unstale":"2024-09-16T15:45:31.178107+0000","last_undegraded":"2024-09-16T15:45:31.178107+0000","last_fullsized":"2024-09-16T15:45:31.178107+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:48.280644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177318+0000","last_change":"2024-09-16T15:45:30.187109+0000","last_active":"2024-09-16T15:45:31.177318+0000","last_peered":"2024-09-16T15:45:31.177318+0000","last_clean":"2024-09-16T15:45:31.177318+0000","last_became_active":"2024-09-16T15:45:30.185804+0000","last_became_peered":"2024-09-16T15:45:30.185804+0000","last_unstale":"2024-09-16T15:45:31.177318+0000","last_undegraded":"2024-09-16T15:45:31.177318+0000","last_fullsized":"2024-09-16T15:45:31.177318+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:10.104750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179069+0000","last_change":"2024-09-16T15:45:30.182413+0000","last_active":"2024-09-16T15:45:31.179069+0000","last_peered":"2024-09-16T15:45:31.179069+0000","last_clean":"2024-09-16T15:45:31.179069+0000","last_became_active":"2024-09-16T15:45:30.181970+0000","last_became_peered":"2024-09-16T15:45:30.181970+0000","last_unstale":"2024-09-16T15:45:31.179069+0000","last_undegraded":"2024-09-16T15:45:31.179069+0000","last_fullsized":"2024-09-16T15:45:31.179069+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:15.746732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179683+0000","last_change":"2024-09-16T15:45:30.181367+0000","last_active":"2024-09-16T15:45:31.179683+0000","last_peered":"2024-09-16T15:45:31.179683+0000","last_clean":"2024-09-16T15:45:31.179683+0000","last_became_active":"2024-09-16T15:45:30.180972+0000","last_became_peered":"2024-09-16T15:45:30.180972+0000","last_unstale":"2024-09-16T15:45:31.179683+0000","last_undegraded":"2024-09-16T15:45:31.179683+0000","last_fullsized":"2024-09-16T15:45:31.179683+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:42.054674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466272+0000","last_change":"2024-09-16T15:45:30.185816+0000","last_active":"2024-09-16T15:45:31.466272+0000","last_peered":"2024-09-16T15:45:31.466272+0000","last_clean":"2024-09-16T15:45:31.466272+0000","last_became_active":"2024-09-16T15:45:30.185504+0000","last_became_peered":"2024-09-16T15:45:30.185504+0000","last_unstale":"2024-09-16T15:45:31.466272+0000","last_undegraded":"2024-09-16T15:45:31.466272+0000","last_fullsized":"2024-09-16T15:45:31.466272+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:04:25.595769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176492+0000","last_change":"2024-09-16T15:45:30.194520+0000","last_active":"2024-09-16T15:45:31.176492+0000","last_peered":"2024-09-16T15:45:31.176492+0000","last_clean":"2024-09-16T15:45:31.176492+0000","last_became_active":"2024-09-16T15:45:30.194209+0000","last_became_peered":"2024-09-16T15:45:30.194209+0000","last_unstale":"2024-09-16T15:45:31.176492+0000","last_undegraded":"2024-09-16T15:45:31.176492+0000","last_fullsized":"2024-09-16T15:45:31.176492+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:34.169524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179131+0000","last_change":"2024-09-16T15:45:30.183859+0000","last_active":"2024-09-16T15:45:31.179131+0000","last_peered":"2024-09-16T15:45:31.179131+0000","last_clean":"2024-09-16T15:45:31.179131+0000","last_became_active":"2024-09-16T15:45:30.183596+0000","last_became_peered":"2024-09-16T15:45:30.183596+0000","last_unstale":"2024-09-16T15:45:31.179131+0000","last_undegraded":"2024-09-16T15:45:31.179131+0000","last_fullsized":"2024-09-16T15:45:31.179131+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:36.615198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696579+0000","last_change":"2024-09-16T15:45:30.186993+0000","last_active":"2024-09-16T15:45:31.696579+0000","last_peered":"2024-09-16T15:45:31.696579+0000","last_clean":"2024-09-16T15:45:31.696579+0000","last_became_active":"2024-09-16T15:45:30.186681+0000","last_became_peered":"2024-09-16T15:45:30.186681+0000","last_unstale":"2024-09-16T15:45:31.696579+0000","last_undegraded":"2024-09-16T15:45:31.696579+0000","last_fullsized":"2024-09-16T15:45:31.696579+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:39.314286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466004+0000","last_change":"2024-09-16T15:45:30.177706+0000","last_active":"2024-09-16T15:45:31.466004+0000","last_peered":"2024-09-16T15:45:31.466004+0000","last_clean":"2024-09-16T15:45:31.466004+0000","last_became_active":"2024-09-16T15:45:30.177410+0000","last_became_peered":"2024-09-16T15:45:30.177410+0000","last_unstale":"2024-09-16T15:45:31.466004+0000","last_undegraded":"2024-09-16T15:45:31.466004+0000","last_fullsized":"2024-09-16T15:45:31.466004+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:41.205372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697075+0000","last_change":"2024-09-16T15:45:30.192350+0000","last_active":"2024-09-16T15:45:31.697075+0000","last_peered":"2024-09-16T15:45:31.697075+0000","last_clean":"2024-09-16T15:45:31.697075+0000","last_became_active":"2024-09-16T15:45:30.192056+0000","last_became_peered":"2024-09-16T15:45:30.192056+0000","last_unstale":"2024-09-16T15:45:31.697075+0000","last_undegraded":"2024-09-16T15:45:31.697075+0000","last_fullsized":"2024-09-16T15:45:31.697075+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:26.348815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696251+0000","last_change":"2024-09-16T15:45:30.187017+0000","last_active":"2024-09-16T15:45:31.696251+0000","last_peered":"2024-09-16T15:45:31.696251+0000","last_clean":"2024-09-16T15:45:31.696251+0000","last_became_active":"2024-09-16T15:45:30.186730+0000","last_became_peered":"2024-09-16T15:45:30.186730+0000","last_unstale":"2024-09-16T15:45:31.696251+0000","last_undegraded":"2024-09-16T15:45:31.696251+0000","last_fullsized":"2024-09-16T15:45:31.696251+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:47.183846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177777+0000","last_change":"2024-09-16T15:45:30.195648+0000","last_active":"2024-09-16T15:45:31.177777+0000","last_peered":"2024-09-16T15:45:31.177777+0000","last_clean":"2024-09-16T15:45:31.177777+0000","last_became_active":"2024-09-16T15:45:30.195019+0000","last_became_peered":"2024-09-16T15:45:30.195019+0000","last_unstale":"2024-09-16T15:45:31.177777+0000","last_undegraded":"2024-09-16T15:45:31.177777+0000","last_fullsized":"2024-09-16T15:45:31.177777+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:01.587071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697351+0000","last_change":"2024-09-16T15:45:30.191536+0000","last_active":"2024-09-16T15:45:31.697351+0000","last_peered":"2024-09-16T15:45:31.697351+0000","last_clean":"2024-09-16T15:45:31.697351+0000","last_became_active":"2024-09-16T15:45:30.191113+0000","last_became_peered":"2024-09-16T15:45:30.191113+0000","last_unstale":"2024-09-16T15:45:31.697351+0000","last_undegraded":"2024-09-16T15:45:31.697351+0000","last_fullsized":"2024-09-16T15:45:31.697351+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:18.212680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176565+0000","last_change":"2024-09-16T15:45:30.192761+0000","last_active":"2024-09-16T15:45:31.176565+0000","last_peered":"2024-09-16T15:45:31.176565+0000","last_clean":"2024-09-16T15:45:31.176565+0000","last_became_active":"2024-09-16T15:45:30.191719+0000","last_became_peered":"2024-09-16T15:45:30.191719+0000","last_unstale":"2024-09-16T15:45:31.176565+0000","last_undegraded":"2024-09-16T15:45:31.176565+0000","last_fullsized":"2024-09-16T15:45:31.176565+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:09:06.814030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697067+0000","last_change":"2024-09-16T15:45:30.183872+0000","last_active":"2024-09-16T15:45:31.697067+0000","last_peered":"2024-09-16T15:45:31.697067+0000","last_clean":"2024-09-16T15:45:31.697067+0000","last_became_active":"2024-09-16T15:45:30.180444+0000","last_became_peered":"2024-09-16T15:45:30.180444+0000","last_unstale":"2024-09-16T15:45:31.697067+0000","last_undegraded":"2024-09-16T15:45:31.697067+0000","last_fullsized":"2024-09-16T15:45:31.697067+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:59.745713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696494+0000","last_change":"2024-09-16T15:45:30.187090+0000","last_active":"2024-09-16T15:45:31.696494+0000","last_peered":"2024-09-16T15:45:31.696494+0000","last_clean":"2024-09-16T15:45:31.696494+0000","last_became_active":"2024-09-16T15:45:30.186737+0000","last_became_peered":"2024-09-16T15:45:30.186737+0000","last_unstale":"2024-09-16T15:45:31.696494+0000","last_undegraded":"2024-09-16T15:45:31.696494+0000","last_fullsized":"2024-09-16T15:45:31.696494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:40.145347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176391+0000","last_change":"2024-09-16T15:45:30.184695+0000","last_active":"2024-09-16T15:45:31.176391+0000","last_peered":"2024-09-16T15:45:31.176391+0000","last_clean":"2024-09-16T15:45:31.176391+0000","last_became_active":"2024-09-16T15:45:30.182589+0000","last_became_peered":"2024-09-16T15:45:30.182589+0000","last_unstale":"2024-09-16T15:45:31.176391+0000","last_undegraded":"2024-09-16T15:45:31.176391+0000","last_fullsized":"2024-09-16T15:45:31.176391+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:06.422521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.178928+0000","last_change":"2024-09-16T15:45:30.179712+0000","last_active":"2024-09-16T15:45:31.178928+0000","last_peered":"2024-09-16T15:45:31.178928+0000","last_clean":"2024-09-16T15:45:31.178928+0000","last_became_active":"2024-09-16T15:45:30.179437+0000","last_became_peered":"2024-09-16T15:45:30.179437+0000","last_unstale":"2024-09-16T15:45:31.178928+0000","last_undegraded":"2024-09-16T15:45:31.178928+0000","last_fullsized":"2024-09-16T15:45:31.178928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:50.599464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176225+0000","last_change":"2024-09-16T15:45:30.189458+0000","last_active":"2024-09-16T15:45:31.176225+0000","last_peered":"2024-09-16T15:45:31.176225+0000","last_clean":"2024-09-16T15:45:31.176225+0000","last_became_active":"2024-09-16T15:45:30.188985+0000","last_became_peered":"2024-09-16T15:45:30.188985+0000","last_unstale":"2024-09-16T15:45:31.176225+0000","last_undegraded":"2024-09-16T15:45:31.176225+0000","last_fullsized":"2024-09-16T15:45:31.176225+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:35:23.754764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466347+0000","last_change":"2024-09-16T15:45:30.180005+0000","last_active":"2024-09-16T15:45:31.466347+0000","last_peered":"2024-09-16T15:45:31.466347+0000","last_clean":"2024-09-16T15:45:31.466347+0000","last_became_active":"2024-09-16T15:45:30.179690+0000","last_became_peered":"2024-09-16T15:45:30.179690+0000","last_unstale":"2024-09-16T15:45:31.466347+0000","last_undegraded":"2024-09-16T15:45:31.466347+0000","last_fullsized":"2024-09-16T15:45:31.466347+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:55.564184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697290+0000","last_change":"2024-09-16T15:45:30.198928+0000","last_active":"2024-09-16T15:45:31.697290+0000","last_peered":"2024-09-16T15:45:31.697290+0000","last_clean":"2024-09-16T15:45:31.697290+0000","last_became_active":"2024-09-16T15:45:30.198623+0000","last_became_peered":"2024-09-16T15:45:30.198623+0000","last_unstale":"2024-09-16T15:45:31.697290+0000","last_undegraded":"2024-09-16T15:45:31.697290+0000","last_fullsized":"2024-09-16T15:45:31.697290+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:56.525396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696776+0000","last_change":"2024-09-16T15:45:30.186499+0000","last_active":"2024-09-16T15:45:31.696776+0000","last_peered":"2024-09-16T15:45:31.696776+0000","last_clean":"2024-09-16T15:45:31.696776+0000","last_became_active":"2024-09-16T15:45:30.186218+0000","last_became_peered":"2024-09-16T15:45:30.186218+0000","last_unstale":"2024-09-16T15:45:31.696776+0000","last_undegraded":"2024-09-16T15:45:31.696776+0000","last_fullsized":"2024-09-16T15:45:31.696776+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:05:16.170424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696387+0000","last_change":"2024-09-16T15:45:30.178447+0000","last_active":"2024-09-16T15:45:31.696387+0000","last_peered":"2024-09-16T15:45:31.696387+0000","last_clean":"2024-09-16T15:45:31.696387+0000","last_became_active":"2024-09-16T15:45:30.178136+0000","last_became_peered":"2024-09-16T15:45:30.178136+0000","last_unstale":"2024-09-16T15:45:31.696387+0000","last_undegraded":"2024-09-16T15:45:31.696387+0000","last_fullsized":"2024-09-16T15:45:31.696387+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:22.542607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696638+0000","last_change":"2024-09-16T15:45:30.182560+0000","last_active":"2024-09-16T15:45:31.696638+0000","last_peered":"2024-09-16T15:45:31.696638+0000","last_clean":"2024-09-16T15:45:31.696638+0000","last_became_active":"2024-09-16T15:45:30.182244+0000","last_became_peered":"2024-09-16T15:45:30.182244+0000","last_unstale":"2024-09-16T15:45:31.696638+0000","last_undegraded":"2024-09-16T15:45:31.696638+0000","last_fullsized":"2024-09-16T15:45:31.696638+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:32:08.856816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697155+0000","last_change":"2024-09-16T15:45:30.181758+0000","last_active":"2024-09-16T15:45:31.697155+0000","last_peered":"2024-09-16T15:45:31.697155+0000","last_clean":"2024-09-16T15:45:31.697155+0000","last_became_active":"2024-09-16T15:45:30.181409+0000","last_became_peered":"2024-09-16T15:45:30.181409+0000","last_unstale":"2024-09-16T15:45:31.697155+0000","last_undegraded":"2024-09-16T15:45:31.697155+0000","last_fullsized":"2024-09-16T15:45:31.697155+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:32.614877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696643+0000","last_change":"2024-09-16T15:45:30.198483+0000","last_active":"2024-09-16T15:45:31.696643+0000","last_peered":"2024-09-16T15:45:31.696643+0000","last_clean":"2024-09-16T15:45:31.696643+0000","last_became_active":"2024-09-16T15:45:30.198159+0000","last_became_peered":"2024-09-16T15:45:30.198159+0000","last_unstale":"2024-09-16T15:45:31.696643+0000","last_undegraded":"2024-09-16T15:45:31.696643+0000","last_fullsized":"2024-09-16T15:45:31.696643+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:00.316651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177368+0000","last_change":"2024-09-16T15:45:30.188321+0000","last_active":"2024-09-16T15:45:31.177368+0000","last_peered":"2024-09-16T15:45:31.177368+0000","last_clean":"2024-09-16T15:45:31.177368+0000","last_became_active":"2024-09-16T15:45:30.187822+0000","last_became_peered":"2024-09-16T15:45:30.187822+0000","last_unstale":"2024-09-16T15:45:31.177368+0000","last_undegraded":"2024-09-16T15:45:31.177368+0000","last_fullsized":"2024-09-16T15:45:31.177368+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:04.076799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697417+0000","last_change":"2024-09-16T15:45:30.192437+0000","last_active":"2024-09-16T15:45:31.697417+0000","last_peered":"2024-09-16T15:45:31.697417+0000","last_clean":"2024-09-16T15:45:31.697417+0000","last_became_active":"2024-09-16T15:45:30.192115+0000","last_became_peered":"2024-09-16T15:45:30.192115+0000","last_unstale":"2024-09-16T15:45:31.697417+0000","last_undegraded":"2024-09-16T15:45:31.697417+0000","last_fullsized":"2024-09-16T15:45:31.697417+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:36.274150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"121'4","reported_seq":19,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697261+0000","last_change":"2024-09-16T15:45:29.180725+0000","last_active":"2024-09-16T15:45:31.697261+0000","last_peered":"2024-09-16T15:45:31.697261+0000","last_clean":"2024-09-16T15:45:31.697261+0000","last_became_active":"2024-09-16T15:45:29.180416+0000","last_became_peered":"2024-09-16T15:45:29.180416+0000","last_unstale":"2024-09-16T15:45:31.697261+0000","last_undegraded":"2024-09-16T15:45:31.697261+0000","last_fullsized":"2024-09-16T15:45:31.697261+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:47:32.675896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179599+0000","last_change":"2024-09-16T15:45:30.193216+0000","last_active":"2024-09-16T15:45:31.179599+0000","last_peered":"2024-09-16T15:45:31.179599+0000","last_clean":"2024-09-16T15:45:31.179599+0000","last_became_active":"2024-09-16T15:45:30.192913+0000","last_became_peered":"2024-09-16T15:45:30.192913+0000","last_unstale":"2024-09-16T15:45:31.179599+0000","last_undegraded":"2024-09-16T15:45:31.179599+0000","last_fullsized":"2024-09-16T15:45:31.179599+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:46.646275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696452+0000","last_change":"2024-09-16T15:45:29.180104+0000","last_active":"2024-09-16T15:45:31.696452+0000","last_peered":"2024-09-16T15:45:31.696452+0000","last_clean":"2024-09-16T15:45:31.696452+0000","last_became_active":"2024-09-16T15:45:29.179799+0000","last_became_peered":"2024-09-16T15:45:29.179799+0000","last_unstale":"2024-09-16T15:45:31.696452+0000","last_undegraded":"2024-09-16T15:45:31.696452+0000","last_fullsized":"2024-09-16T15:45:31.696452+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:42:08.291680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697084+0000","last_change":"2024-09-16T15:45:30.186256+0000","last_active":"2024-09-16T15:45:31.697084+0000","last_peered":"2024-09-16T15:45:31.697084+0000","last_clean":"2024-09-16T15:45:31.697084+0000","last_became_active":"2024-09-16T15:45:30.185665+0000","last_became_peered":"2024-09-16T15:45:30.185665+0000","last_unstale":"2024-09-16T15:45:31.697084+0000","last_undegraded":"2024-09-16T15:45:31.697084+0000","last_fullsized":"2024-09-16T15:45:31.697084+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:49.353996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176716+0000","last_change":"2024-09-16T15:45:29.184056+0000","last_active":"2024-09-16T15:45:31.176716+0000","last_peered":"2024-09-16T15:45:31.176716+0000","last_clean":"2024-09-16T15:45:31.176716+0000","last_became_active":"2024-09-16T15:45:29.182998+0000","last_became_peered":"2024-09-16T15:45:29.182998+0000","last_unstale":"2024-09-16T15:45:31.176716+0000","last_undegraded":"2024-09-16T15:45:31.176716+0000","last_fullsized":"2024-09-16T15:45:31.176716+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:19.489850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696970+0000","last_change":"2024-09-16T15:45:30.181498+0000","last_active":"2024-09-16T15:45:31.696970+0000","last_peered":"2024-09-16T15:45:31.696970+0000","last_clean":"2024-09-16T15:45:31.696970+0000","last_became_active":"2024-09-16T15:45:30.180766+0000","last_became_peered":"2024-09-16T15:45:30.180766+0000","last_unstale":"2024-09-16T15:45:31.696970+0000","last_undegraded":"2024-09-16T15:45:31.696970+0000","last_fullsized":"2024-09-16T15:45:31.696970+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:59.236504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466121+0000","last_change":"2024-09-16T15:45:29.186646+0000","last_active":"2024-09-16T15:45:31.466121+0000","last_peered":"2024-09-16T15:45:31.466121+0000","last_clean":"2024-09-16T15:45:31.466121+0000","last_became_active":"2024-09-16T15:45:29.185458+0000","last_became_peered":"2024-09-16T15:45:29.185458+0000","last_unstale":"2024-09-16T15:45:31.466121+0000","last_undegraded":"2024-09-16T15:45:31.466121+0000","last_fullsized":"2024-09-16T15:45:31.466121+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:17.462421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696369+0000","last_change":"2024-09-16T15:45:30.190056+0000","last_active":"2024-09-16T15:45:31.696369+0000","last_peered":"2024-09-16T15:45:31.696369+0000","last_clean":"2024-09-16T15:45:31.696369+0000","last_became_active":"2024-09-16T15:45:30.189699+0000","last_became_peered":"2024-09-16T15:45:30.189699+0000","last_unstale":"2024-09-16T15:45:31.696369+0000","last_undegraded":"2024-09-16T15:45:31.696369+0000","last_fullsized":"2024-09-16T15:45:31.696369+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:16.174307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465873+0000","last_change":"2024-09-16T15:45:29.180962+0000","last_active":"2024-09-16T15:45:31.465873+0000","last_peered":"2024-09-16T15:45:31.465873+0000","last_clean":"2024-09-16T15:45:31.465873+0000","last_became_active":"2024-09-16T15:45:29.180678+0000","last_became_peered":"2024-09-16T15:45:29.180678+0000","last_unstale":"2024-09-16T15:45:31.465873+0000","last_undegraded":"2024-09-16T15:45:31.465873+0000","last_fullsized":"2024-09-16T15:45:31.465873+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:35:26.903652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177689+0000","last_change":"2024-09-16T15:45:30.192822+0000","last_active":"2024-09-16T15:45:31.177689+0000","last_peered":"2024-09-16T15:45:31.177689+0000","last_clean":"2024-09-16T15:45:31.177689+0000","last_became_active":"2024-09-16T15:45:30.192554+0000","last_became_peered":"2024-09-16T15:45:30.192554+0000","last_unstale":"2024-09-16T15:45:31.177689+0000","last_undegraded":"2024-09-16T15:45:31.177689+0000","last_fullsized":"2024-09-16T15:45:31.177689+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:46.682615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696479+0000","last_change":"2024-09-16T15:45:29.185985+0000","last_active":"2024-09-16T15:45:31.696479+0000","last_peered":"2024-09-16T15:45:31.696479+0000","last_clean":"2024-09-16T15:45:31.696479+0000","last_became_active":"2024-09-16T15:45:29.185728+0000","last_became_peered":"2024-09-16T15:45:29.185728+0000","last_unstale":"2024-09-16T15:45:31.696479+0000","last_undegraded":"2024-09-16T15:45:31.696479+0000","last_fullsized":"2024-09-16T15:45:31.696479+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:39.433170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179066+0000","last_change":"2024-09-16T15:45:30.182333+0000","last_active":"2024-09-16T15:45:31.179066+0000","last_peered":"2024-09-16T15:45:31.179066+0000","last_clean":"2024-09-16T15:45:31.179066+0000","last_became_active":"2024-09-16T15:45:30.181398+0000","last_became_peered":"2024-09-16T15:45:30.181398+0000","last_unstale":"2024-09-16T15:45:31.179066+0000","last_undegraded":"2024-09-16T15:45:31.179066+0000","last_fullsized":"2024-09-16T15:45:31.179066+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:36.098523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177213+0000","last_change":"2024-09-16T15:45:29.174336+0000","last_active":"2024-09-16T15:45:31.177213+0000","last_peered":"2024-09-16T15:45:31.177213+0000","last_clean":"2024-09-16T15:45:31.177213+0000","last_became_active":"2024-09-16T15:45:29.174064+0000","last_became_peered":"2024-09-16T15:45:29.174064+0000","last_unstale":"2024-09-16T15:45:31.177213+0000","last_undegraded":"2024-09-16T15:45:31.177213+0000","last_fullsized":"2024-09-16T15:45:31.177213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:23.625293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465953+0000","last_change":"2024-09-16T15:45:30.183317+0000","last_active":"2024-09-16T15:45:31.465953+0000","last_peered":"2024-09-16T15:45:31.465953+0000","last_clean":"2024-09-16T15:45:31.465953+0000","last_became_active":"2024-09-16T15:45:30.178662+0000","last_became_peered":"2024-09-16T15:45:30.178662+0000","last_unstale":"2024-09-16T15:45:31.465953+0000","last_undegraded":"2024-09-16T15:45:31.465953+0000","last_fullsized":"2024-09-16T15:45:31.465953+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:54.779284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465961+0000","last_change":"2024-09-16T15:45:29.175175+0000","last_active":"2024-09-16T15:45:31.465961+0000","last_peered":"2024-09-16T15:45:31.465961+0000","last_clean":"2024-09-16T15:45:31.465961+0000","last_became_active":"2024-09-16T15:45:29.174872+0000","last_became_peered":"2024-09-16T15:45:29.174872+0000","last_unstale":"2024-09-16T15:45:31.465961+0000","last_undegraded":"2024-09-16T15:45:31.465961+0000","last_fullsized":"2024-09-16T15:45:31.465961+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:41.567223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697215+0000","last_change":"2024-09-16T15:45:30.195424+0000","last_active":"2024-09-16T15:45:31.697215+0000","last_peered":"2024-09-16T15:45:31.697215+0000","last_clean":"2024-09-16T15:45:31.697215+0000","last_became_active":"2024-09-16T15:45:30.195158+0000","last_became_peered":"2024-09-16T15:45:30.195158+0000","last_unstale":"2024-09-16T15:45:31.697215+0000","last_undegraded":"2024-09-16T15:45:31.697215+0000","last_fullsized":"2024-09-16T15:45:31.697215+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:00:03.919278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697046+0000","last_change":"2024-09-16T15:45:29.181145+0000","last_active":"2024-09-16T15:45:31.697046+0000","last_peered":"2024-09-16T15:45:31.697046+0000","last_clean":"2024-09-16T15:45:31.697046+0000","last_became_active":"2024-09-16T15:45:29.180864+0000","last_became_peered":"2024-09-16T15:45:29.180864+0000","last_unstale":"2024-09-16T15:45:31.697046+0000","last_undegraded":"2024-09-16T15:45:31.697046+0000","last_fullsized":"2024-09-16T15:45:31.697046+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:43.945300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696863+0000","last_change":"2024-09-16T15:45:30.185081+0000","last_active":"2024-09-16T15:45:31.696863+0000","last_peered":"2024-09-16T15:45:31.696863+0000","last_clean":"2024-09-16T15:45:31.696863+0000","last_became_active":"2024-09-16T15:45:30.184778+0000","last_became_peered":"2024-09-16T15:45:30.184778+0000","last_unstale":"2024-09-16T15:45:31.696863+0000","last_undegraded":"2024-09-16T15:45:31.696863+0000","last_fullsized":"2024-09-16T15:45:31.696863+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:14.926640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697423+0000","last_change":"2024-09-16T15:45:29.181119+0000","last_active":"2024-09-16T15:45:31.697423+0000","last_peered":"2024-09-16T15:45:31.697423+0000","last_clean":"2024-09-16T15:45:31.697423+0000","last_became_active":"2024-09-16T15:45:29.180832+0000","last_became_peered":"2024-09-16T15:45:29.180832+0000","last_unstale":"2024-09-16T15:45:31.697423+0000","last_undegraded":"2024-09-16T15:45:31.697423+0000","last_fullsized":"2024-09-16T15:45:31.697423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:47:32.675896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697291+0000","last_change":"2024-09-16T15:45:30.187570+0000","last_active":"2024-09-16T15:45:31.697291+0000","last_peered":"2024-09-16T15:45:31.697291+0000","last_clean":"2024-09-16T15:45:31.697291+0000","last_became_active":"2024-09-16T15:45:30.187022+0000","last_became_peered":"2024-09-16T15:45:30.187022+0000","last_unstale":"2024-09-16T15:45:31.697291+0000","last_undegraded":"2024-09-16T15:45:31.697291+0000","last_fullsized":"2024-09-16T15:45:31.697291+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:28.685150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"121'3","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696583+0000","last_change":"2024-09-16T15:45:29.177762+0000","last_active":"2024-09-16T15:45:31.696583+0000","last_peered":"2024-09-16T15:45:31.696583+0000","last_clean":"2024-09-16T15:45:31.696583+0000","last_became_active":"2024-09-16T15:45:29.177395+0000","last_became_peered":"2024-09-16T15:45:29.177395+0000","last_unstale":"2024-09-16T15:45:31.696583+0000","last_undegraded":"2024-09-16T15:45:31.696583+0000","last_fullsized":"2024-09-16T15:45:31.696583+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:16.095883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697208+0000","last_change":"2024-09-16T15:45:30.190738+0000","last_active":"2024-09-16T15:45:31.697208+0000","last_peered":"2024-09-16T15:45:31.697208+0000","last_clean":"2024-09-16T15:45:31.697208+0000","last_became_active":"2024-09-16T15:45:30.190439+0000","last_became_peered":"2024-09-16T15:45:30.190439+0000","last_unstale":"2024-09-16T15:45:31.697208+0000","last_undegraded":"2024-09-16T15:45:31.697208+0000","last_fullsized":"2024-09-16T15:45:31.697208+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:24.782249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"121'6","reported_seq":21,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179133+0000","last_change":"2024-09-16T15:45:29.171700+0000","last_active":"2024-09-16T15:45:31.179133+0000","last_peered":"2024-09-16T15:45:31.179133+0000","last_clean":"2024-09-16T15:45:31.179133+0000","last_became_active":"2024-09-16T15:45:29.171262+0000","last_became_peered":"2024-09-16T15:45:29.171262+0000","last_unstale":"2024-09-16T15:45:31.179133+0000","last_undegraded":"2024-09-16T15:45:31.179133+0000","last_fullsized":"2024-09-16T15:45:31.179133+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:40.779854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696819+0000","last_change":"2024-09-16T15:45:30.195921+0000","last_active":"2024-09-16T15:45:31.696819+0000","last_peered":"2024-09-16T15:45:31.696819+0000","last_clean":"2024-09-16T15:45:31.696819+0000","last_became_active":"2024-09-16T15:45:30.195621+0000","last_became_peered":"2024-09-16T15:45:30.195621+0000","last_unstale":"2024-09-16T15:45:31.696819+0000","last_undegraded":"2024-09-16T15:45:31.696819+0000","last_fullsized":"2024-09-16T15:45:31.696819+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:07.466423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696749+0000","last_change":"2024-09-16T15:45:29.170591+0000","last_active":"2024-09-16T15:45:31.696749+0000","last_peered":"2024-09-16T15:45:31.696749+0000","last_clean":"2024-09-16T15:45:31.696749+0000","last_became_active":"2024-09-16T15:45:29.170281+0000","last_became_peered":"2024-09-16T15:45:29.170281+0000","last_unstale":"2024-09-16T15:45:31.696749+0000","last_undegraded":"2024-09-16T15:45:31.696749+0000","last_fullsized":"2024-09-16T15:45:31.696749+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:30.045314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465858+0000","last_change":"2024-09-16T15:45:30.167207+0000","last_active":"2024-09-16T15:45:31.465858+0000","last_peered":"2024-09-16T15:45:31.465858+0000","last_clean":"2024-09-16T15:45:31.465858+0000","last_became_active":"2024-09-16T15:45:30.166907+0000","last_became_peered":"2024-09-16T15:45:30.166907+0000","last_unstale":"2024-09-16T15:45:31.465858+0000","last_undegraded":"2024-09-16T15:45:31.465858+0000","last_fullsized":"2024-09-16T15:45:31.465858+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:38.507200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465974+0000","last_change":"2024-09-16T15:45:29.181674+0000","last_active":"2024-09-16T15:45:31.465974+0000","last_peered":"2024-09-16T15:45:31.465974+0000","last_clean":"2024-09-16T15:45:31.465974+0000","last_became_active":"2024-09-16T15:45:29.181334+0000","last_became_peered":"2024-09-16T15:45:29.181334+0000","last_unstale":"2024-09-16T15:45:31.465974+0000","last_undegraded":"2024-09-16T15:45:31.465974+0000","last_fullsized":"2024-09-16T15:45:31.465974+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:21:51.167893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.178887+0000","last_change":"2024-09-16T15:45:30.182337+0000","last_active":"2024-09-16T15:45:31.178887+0000","last_peered":"2024-09-16T15:45:31.178887+0000","last_clean":"2024-09-16T15:45:31.178887+0000","last_became_active":"2024-09-16T15:45:30.179787+0000","last_became_peered":"2024-09-16T15:45:30.179787+0000","last_unstale":"2024-09-16T15:45:31.178887+0000","last_undegraded":"2024-09-16T15:45:31.178887+0000","last_fullsized":"2024-09-16T15:45:31.178887+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:12.436952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"121'8","reported_seq":23,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179180+0000","last_change":"2024-09-16T15:45:29.178341+0000","last_active":"2024-09-16T15:45:31.179180+0000","last_peered":"2024-09-16T15:45:31.179180+0000","last_clean":"2024-09-16T15:45:31.179180+0000","last_became_active":"2024-09-16T15:45:29.177742+0000","last_became_peered":"2024-09-16T15:45:29.177742+0000","last_unstale":"2024-09-16T15:45:31.179180+0000","last_undegraded":"2024-09-16T15:45:31.179180+0000","last_fullsized":"2024-09-16T15:45:31.179180+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:55:57.521505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.179512+0000","last_change":"2024-09-16T15:45:30.182824+0000","last_active":"2024-09-16T15:45:31.179512+0000","last_peered":"2024-09-16T15:45:31.179512+0000","last_clean":"2024-09-16T15:45:31.179512+0000","last_became_active":"2024-09-16T15:45:30.182533+0000","last_became_peered":"2024-09-16T15:45:30.182533+0000","last_unstale":"2024-09-16T15:45:31.179512+0000","last_undegraded":"2024-09-16T15:45:31.179512+0000","last_fullsized":"2024-09-16T15:45:31.179512+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:01.388439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697172+0000","last_change":"2024-09-16T15:45:29.178358+0000","last_active":"2024-09-16T15:45:31.697172+0000","last_peered":"2024-09-16T15:45:31.697172+0000","last_clean":"2024-09-16T15:45:31.697172+0000","last_became_active":"2024-09-16T15:45:29.177952+0000","last_became_peered":"2024-09-16T15:45:29.177952+0000","last_unstale":"2024-09-16T15:45:31.697172+0000","last_undegraded":"2024-09-16T15:45:31.697172+0000","last_fullsized":"2024-09-16T15:45:31.697172+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:33.711534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697044+0000","last_change":"2024-09-16T15:45:30.194307+0000","last_active":"2024-09-16T15:45:31.697044+0000","last_peered":"2024-09-16T15:45:31.697044+0000","last_clean":"2024-09-16T15:45:31.697044+0000","last_became_active":"2024-09-16T15:45:30.193939+0000","last_became_peered":"2024-09-16T15:45:30.193939+0000","last_unstale":"2024-09-16T15:45:31.697044+0000","last_undegraded":"2024-09-16T15:45:31.697044+0000","last_fullsized":"2024-09-16T15:45:31.697044+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:13:26.141198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176486+0000","last_change":"2024-09-16T15:45:29.172406+0000","last_active":"2024-09-16T15:45:31.176486+0000","last_peered":"2024-09-16T15:45:31.176486+0000","last_clean":"2024-09-16T15:45:31.176486+0000","last_became_active":"2024-09-16T15:45:29.171452+0000","last_became_peered":"2024-09-16T15:45:29.171452+0000","last_unstale":"2024-09-16T15:45:31.176486+0000","last_undegraded":"2024-09-16T15:45:31.176486+0000","last_fullsized":"2024-09-16T15:45:31.176486+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:40.401517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.697280+0000","last_change":"2024-09-16T15:45:30.186855+0000","last_active":"2024-09-16T15:45:31.697280+0000","last_peered":"2024-09-16T15:45:31.697280+0000","last_clean":"2024-09-16T15:45:31.697280+0000","last_became_active":"2024-09-16T15:45:30.186571+0000","last_became_peered":"2024-09-16T15:45:30.186571+0000","last_unstale":"2024-09-16T15:45:31.697280+0000","last_undegraded":"2024-09-16T15:45:31.697280+0000","last_fullsized":"2024-09-16T15:45:31.697280+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:37.740173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177765+0000","last_change":"2024-09-16T15:45:29.177128+0000","last_active":"2024-09-16T15:45:31.177765+0000","last_peered":"2024-09-16T15:45:31.177765+0000","last_clean":"2024-09-16T15:45:31.177765+0000","last_became_active":"2024-09-16T15:45:29.176810+0000","last_became_peered":"2024-09-16T15:45:29.176810+0000","last_unstale":"2024-09-16T15:45:31.177765+0000","last_undegraded":"2024-09-16T15:45:31.177765+0000","last_fullsized":"2024-09-16T15:45:31.177765+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:47.665949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466316+0000","last_change":"2024-09-16T15:45:30.182507+0000","last_active":"2024-09-16T15:45:31.466316+0000","last_peered":"2024-09-16T15:45:31.466316+0000","last_clean":"2024-09-16T15:45:31.466316+0000","last_became_active":"2024-09-16T15:45:30.179003+0000","last_became_peered":"2024-09-16T15:45:30.179003+0000","last_unstale":"2024-09-16T15:45:31.466316+0000","last_undegraded":"2024-09-16T15:45:31.466316+0000","last_fullsized":"2024-09-16T15:45:31.466316+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:27:04.547979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696495+0000","last_change":"2024-09-16T15:45:29.176966+0000","last_active":"2024-09-16T15:45:31.696495+0000","last_peered":"2024-09-16T15:45:31.696495+0000","last_clean":"2024-09-16T15:45:31.696495+0000","last_became_active":"2024-09-16T15:45:29.176398+0000","last_became_peered":"2024-09-16T15:45:29.176398+0000","last_unstale":"2024-09-16T15:45:31.696495+0000","last_undegraded":"2024-09-16T15:45:31.696495+0000","last_fullsized":"2024-09-16T15:45:31.696495+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:55.181737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696641+0000","last_change":"2024-09-16T15:45:30.185485+0000","last_active":"2024-09-16T15:45:31.696641+0000","last_peered":"2024-09-16T15:45:31.696641+0000","last_clean":"2024-09-16T15:45:31.696641+0000","last_became_active":"2024-09-16T15:45:30.185187+0000","last_became_peered":"2024-09-16T15:45:30.185187+0000","last_unstale":"2024-09-16T15:45:31.696641+0000","last_undegraded":"2024-09-16T15:45:31.696641+0000","last_fullsized":"2024-09-16T15:45:31.696641+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:04.525210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696425+0000","last_change":"2024-09-16T15:45:29.182582+0000","last_active":"2024-09-16T15:45:31.696425+0000","last_peered":"2024-09-16T15:45:31.696425+0000","last_clean":"2024-09-16T15:45:31.696425+0000","last_became_active":"2024-09-16T15:45:29.182296+0000","last_became_peered":"2024-09-16T15:45:29.182296+0000","last_unstale":"2024-09-16T15:45:31.696425+0000","last_undegraded":"2024-09-16T15:45:31.696425+0000","last_fullsized":"2024-09-16T15:45:31.696425+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:46.819724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.178230+0000","last_change":"2024-09-16T15:45:30.193342+0000","last_active":"2024-09-16T15:45:31.178230+0000","last_peered":"2024-09-16T15:45:31.178230+0000","last_clean":"2024-09-16T15:45:31.178230+0000","last_became_active":"2024-09-16T15:45:30.193060+0000","last_became_peered":"2024-09-16T15:45:30.193060+0000","last_unstale":"2024-09-16T15:45:31.178230+0000","last_undegraded":"2024-09-16T15:45:31.178230+0000","last_fullsized":"2024-09-16T15:45:31.178230+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:37:30.147795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696573+0000","last_change":"2024-09-16T15:45:29.177474+0000","last_active":"2024-09-16T15:45:31.696573+0000","last_peered":"2024-09-16T15:45:31.696573+0000","last_clean":"2024-09-16T15:45:31.696573+0000","last_became_active":"2024-09-16T15:45:29.177148+0000","last_became_peered":"2024-09-16T15:45:29.177148+0000","last_unstale":"2024-09-16T15:45:31.696573+0000","last_undegraded":"2024-09-16T15:45:31.696573+0000","last_fullsized":"2024-09-16T15:45:31.696573+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:53.240059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465855+0000","last_change":"2024-09-16T15:45:30.183144+0000","last_active":"2024-09-16T15:45:31.465855+0000","last_peered":"2024-09-16T15:45:31.465855+0000","last_clean":"2024-09-16T15:45:31.465855+0000","last_became_active":"2024-09-16T15:45:30.182866+0000","last_became_peered":"2024-09-16T15:45:30.182866+0000","last_unstale":"2024-09-16T15:45:31.465855+0000","last_undegraded":"2024-09-16T15:45:31.465855+0000","last_fullsized":"2024-09-16T15:45:31.465855+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:48.399155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696753+0000","last_change":"2024-09-16T15:45:29.178091+0000","last_active":"2024-09-16T15:45:31.696753+0000","last_peered":"2024-09-16T15:45:31.696753+0000","last_clean":"2024-09-16T15:45:31.696753+0000","last_became_active":"2024-09-16T15:45:29.177797+0000","last_became_peered":"2024-09-16T15:45:29.177797+0000","last_unstale":"2024-09-16T15:45:31.696753+0000","last_undegraded":"2024-09-16T15:45:31.696753+0000","last_fullsized":"2024-09-16T15:45:31.696753+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:08.537058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696361+0000","last_change":"2024-09-16T15:45:30.186619+0000","last_active":"2024-09-16T15:45:31.696361+0000","last_peered":"2024-09-16T15:45:31.696361+0000","last_clean":"2024-09-16T15:45:31.696361+0000","last_became_active":"2024-09-16T15:45:30.186327+0000","last_became_peered":"2024-09-16T15:45:30.186327+0000","last_unstale":"2024-09-16T15:45:31.696361+0000","last_undegraded":"2024-09-16T15:45:31.696361+0000","last_fullsized":"2024-09-16T15:45:31.696361+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:41.276942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176124+0000","last_change":"2024-09-16T15:45:29.181970+0000","last_active":"2024-09-16T15:45:31.176124+0000","last_peered":"2024-09-16T15:45:31.176124+0000","last_clean":"2024-09-16T15:45:31.176124+0000","last_became_active":"2024-09-16T15:45:29.179340+0000","last_became_peered":"2024-09-16T15:45:29.179340+0000","last_unstale":"2024-09-16T15:45:31.176124+0000","last_undegraded":"2024-09-16T15:45:31.176124+0000","last_fullsized":"2024-09-16T15:45:31.176124+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:48.828545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177891+0000","last_change":"2024-09-16T15:45:30.182837+0000","last_active":"2024-09-16T15:45:31.177891+0000","last_peered":"2024-09-16T15:45:31.177891+0000","last_clean":"2024-09-16T15:45:31.177891+0000","last_became_active":"2024-09-16T15:45:30.181331+0000","last_became_peered":"2024-09-16T15:45:30.181331+0000","last_unstale":"2024-09-16T15:45:31.177891+0000","last_undegraded":"2024-09-16T15:45:31.177891+0000","last_fullsized":"2024-09-16T15:45:31.177891+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:37:56.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696958+0000","last_change":"2024-09-16T15:45:29.176979+0000","last_active":"2024-09-16T15:45:31.696958+0000","last_peered":"2024-09-16T15:45:31.696958+0000","last_clean":"2024-09-16T15:45:31.696958+0000","last_became_active":"2024-09-16T15:45:29.176699+0000","last_became_peered":"2024-09-16T15:45:29.176699+0000","last_unstale":"2024-09-16T15:45:31.696958+0000","last_undegraded":"2024-09-16T15:45:31.696958+0000","last_fullsized":"2024-09-16T15:45:31.696958+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:46:53.348328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696852+0000","last_change":"2024-09-16T15:45:30.191425+0000","last_active":"2024-09-16T15:45:31.696852+0000","last_peered":"2024-09-16T15:45:31.696852+0000","last_clean":"2024-09-16T15:45:31.696852+0000","last_became_active":"2024-09-16T15:45:30.190489+0000","last_became_peered":"2024-09-16T15:45:30.190489+0000","last_unstale":"2024-09-16T15:45:31.696852+0000","last_undegraded":"2024-09-16T15:45:31.696852+0000","last_fullsized":"2024-09-16T15:45:31.696852+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:06.893466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696999+0000","last_change":"2024-09-16T15:45:29.173375+0000","last_active":"2024-09-16T15:45:31.696999+0000","last_peered":"2024-09-16T15:45:31.696999+0000","last_clean":"2024-09-16T15:45:31.696999+0000","last_became_active":"2024-09-16T15:45:29.172775+0000","last_became_peered":"2024-09-16T15:45:29.172775+0000","last_unstale":"2024-09-16T15:45:31.696999+0000","last_undegraded":"2024-09-16T15:45:31.696999+0000","last_fullsized":"2024-09-16T15:45:31.696999+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:08.537058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466198+0000","last_change":"2024-09-16T15:45:30.180342+0000","last_active":"2024-09-16T15:45:31.466198+0000","last_peered":"2024-09-16T15:45:31.466198+0000","last_clean":"2024-09-16T15:45:31.466198+0000","last_became_active":"2024-09-16T15:45:30.180095+0000","last_became_peered":"2024-09-16T15:45:30.180095+0000","last_unstale":"2024-09-16T15:45:31.466198+0000","last_undegraded":"2024-09-16T15:45:31.466198+0000","last_fullsized":"2024-09-16T15:45:31.466198+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:18.944471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.466239+0000","last_change":"2024-09-16T15:45:29.182547+0000","last_active":"2024-09-16T15:45:31.466239+0000","last_peered":"2024-09-16T15:45:31.466239+0000","last_clean":"2024-09-16T15:45:31.466239+0000","last_became_active":"2024-09-16T15:45:29.182066+0000","last_became_peered":"2024-09-16T15:45:29.182066+0000","last_unstale":"2024-09-16T15:45:31.466239+0000","last_undegraded":"2024-09-16T15:45:31.466239+0000","last_fullsized":"2024-09-16T15:45:31.466239+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:51.852544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.177778+0000","last_change":"2024-09-16T15:45:30.190109+0000","last_active":"2024-09-16T15:45:31.177778+0000","last_peered":"2024-09-16T15:45:31.177778+0000","last_clean":"2024-09-16T15:45:31.177778+0000","last_became_active":"2024-09-16T15:45:30.188648+0000","last_became_peered":"2024-09-16T15:45:30.188648+0000","last_unstale":"2024-09-16T15:45:31.177778+0000","last_undegraded":"2024-09-16T15:45:31.177778+0000","last_fullsized":"2024-09-16T15:45:31.177778+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:13.131402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176451+0000","last_change":"2024-09-16T15:45:29.186287+0000","last_active":"2024-09-16T15:45:31.176451+0000","last_peered":"2024-09-16T15:45:31.176451+0000","last_clean":"2024-09-16T15:45:31.176451+0000","last_became_active":"2024-09-16T15:45:29.185838+0000","last_became_peered":"2024-09-16T15:45:29.185838+0000","last_unstale":"2024-09-16T15:45:31.176451+0000","last_undegraded":"2024-09-16T15:45:31.176451+0000","last_fullsized":"2024-09-16T15:45:31.176451+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:33:30.424655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696709+0000","last_change":"2024-09-16T15:45:30.183868+0000","last_active":"2024-09-16T15:45:31.696709+0000","last_peered":"2024-09-16T15:45:31.696709+0000","last_clean":"2024-09-16T15:45:31.696709+0000","last_became_active":"2024-09-16T15:45:30.183554+0000","last_became_peered":"2024-09-16T15:45:30.183554+0000","last_unstale":"2024-09-16T15:45:31.696709+0000","last_undegraded":"2024-09-16T15:45:31.696709+0000","last_fullsized":"2024-09-16T15:45:31.696709+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:58.058490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465930+0000","last_change":"2024-09-16T15:45:29.177657+0000","last_active":"2024-09-16T15:45:31.465930+0000","last_peered":"2024-09-16T15:45:31.465930+0000","last_clean":"2024-09-16T15:45:31.465930+0000","last_became_active":"2024-09-16T15:45:29.177113+0000","last_became_peered":"2024-09-16T15:45:29.177113+0000","last_unstale":"2024-09-16T15:45:31.465930+0000","last_undegraded":"2024-09-16T15:45:31.465930+0000","last_fullsized":"2024-09-16T15:45:31.465930+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:56.944663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696713+0000","last_change":"2024-09-16T15:45:30.180641+0000","last_active":"2024-09-16T15:45:31.696713+0000","last_peered":"2024-09-16T15:45:31.696713+0000","last_clean":"2024-09-16T15:45:31.696713+0000","last_became_active":"2024-09-16T15:45:30.180377+0000","last_became_peered":"2024-09-16T15:45:30.180377+0000","last_unstale":"2024-09-16T15:45:31.696713+0000","last_undegraded":"2024-09-16T15:45:31.696713+0000","last_fullsized":"2024-09-16T15:45:31.696713+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:27.325989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696812+0000","last_change":"2024-09-16T15:45:29.181524+0000","last_active":"2024-09-16T15:45:31.696812+0000","last_peered":"2024-09-16T15:45:31.696812+0000","last_clean":"2024-09-16T15:45:31.696812+0000","last_became_active":"2024-09-16T15:45:29.181222+0000","last_became_peered":"2024-09-16T15:45:29.181222+0000","last_unstale":"2024-09-16T15:45:31.696812+0000","last_undegraded":"2024-09-16T15:45:31.696812+0000","last_fullsized":"2024-09-16T15:45:31.696812+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:07.983234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696293+0000","last_change":"2024-09-16T15:45:30.194204+0000","last_active":"2024-09-16T15:45:31.696293+0000","last_peered":"2024-09-16T15:45:31.696293+0000","last_clean":"2024-09-16T15:45:31.696293+0000","last_became_active":"2024-09-16T15:45:30.193106+0000","last_became_peered":"2024-09-16T15:45:30.193106+0000","last_unstale":"2024-09-16T15:45:31.696293+0000","last_undegraded":"2024-09-16T15:45:31.696293+0000","last_fullsized":"2024-09-16T15:45:31.696293+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:18.852050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696842+0000","last_change":"2024-09-16T15:45:29.184118+0000","last_active":"2024-09-16T15:45:31.696842+0000","last_peered":"2024-09-16T15:45:31.696842+0000","last_clean":"2024-09-16T15:45:31.696842+0000","last_became_active":"2024-09-16T15:45:29.179258+0000","last_became_peered":"2024-09-16T15:45:29.179258+0000","last_unstale":"2024-09-16T15:45:31.696842+0000","last_undegraded":"2024-09-16T15:45:31.696842+0000","last_fullsized":"2024-09-16T15:45:31.696842+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:50.713882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.465889+0000","last_change":"2024-09-16T15:45:30.177930+0000","last_active":"2024-09-16T15:45:31.465889+0000","last_peered":"2024-09-16T15:45:31.465889+0000","last_clean":"2024-09-16T15:45:31.465889+0000","last_became_active":"2024-09-16T15:45:30.177511+0000","last_became_peered":"2024-09-16T15:45:30.177511+0000","last_unstale":"2024-09-16T15:45:31.465889+0000","last_undegraded":"2024-09-16T15:45:31.465889+0000","last_fullsized":"2024-09-16T15:45:31.465889+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:54:27.832197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.176222+0000","last_change":"2024-09-16T15:45:29.185510+0000","last_active":"2024-09-16T15:45:31.176222+0000","last_peered":"2024-09-16T15:45:31.176222+0000","last_clean":"2024-09-16T15:45:31.176222+0000","last_became_active":"2024-09-16T15:45:29.184491+0000","last_became_peered":"2024-09-16T15:45:29.184491+0000","last_unstale":"2024-09-16T15:45:31.176222+0000","last_undegraded":"2024-09-16T15:45:31.176222+0000","last_fullsized":"2024-09-16T15:45:31.176222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:43.426732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696698+0000","last_change":"2024-09-16T15:45:30.186476+0000","last_active":"2024-09-16T15:45:31.696698+0000","last_peered":"2024-09-16T15:45:31.696698+0000","last_clean":"2024-09-16T15:45:31.696698+0000","last_became_active":"2024-09-16T15:45:30.186138+0000","last_became_peered":"2024-09-16T15:45:30.186138+0000","last_unstale":"2024-09-16T15:45:31.696698+0000","last_undegraded":"2024-09-16T15:45:31.696698+0000","last_fullsized":"2024-09-16T15:45:31.696698+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:29.147465+0000","last_clean_scrub_stamp":"2024-09-16T15:45:29.147465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:33:01.063888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696782+0000","last_change":"2024-09-16T15:45:29.180566+0000","last_active":"2024-09-16T15:45:31.696782+0000","last_peered":"2024-09-16T15:45:31.696782+0000","last_clean":"2024-09-16T15:45:31.696782+0000","last_became_active":"2024-09-16T15:45:29.180179+0000","last_became_peered":"2024-09-16T15:45:29.180179+0000","last_unstale":"2024-09-16T15:45:31.696782+0000","last_undegraded":"2024-09-16T15:45:31.696782+0000","last_fullsized":"2024-09-16T15:45:31.696782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:45:28.137404+0000","last_clean_scrub_stamp":"2024-09-16T15:45:28.137404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:50.713882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":321,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T15:45:31.696056+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:45:31.696056+0000","last_peered":"2024-09-16T15:45:31.696056+0000","last_clean":"2024-09-16T15:45:31.696056+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:45:31.696056+0000","last_undegraded":"2024-09-16T15:45:31.696056+0000","last_fullsized":"2024-09-16T15:45:31.696056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509530,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33264,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338576,"statfs":{"total":96636764160,"available":96602701824,"internally_reserved":0,"allocated":2121728,"data_stored":1253293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1527808,"data_stored":668615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509528,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1523712,"data_stored":667115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509528,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":667073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509529,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36828,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335012,"statfs":{"total":96636764160,"available":96599052288,"internally_reserved":0,"allocated":1511424,"data_stored":660835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":1511424,"data_stored":661291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542234,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":1503232,"data_stored":660675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2109440,"data_stored":1257371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:45:34.860 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:45:34.860 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:45:35.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.065+0000 7f0132200640 1 -- 172.21.15.60:0/4259750848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01100052d0 msgr2=0x7f011000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.065+0000 7f0132200640 1 --2- 172.21.15.60:0/4259750848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01100052d0 0x7f011000b9c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f012c074800 tx=0x7f0110005830 comp rx=0 tx=0).stop 2024-09-16T15:45:35.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.065+0000 7f0132200640 1 -- 172.21.15.60:0/4259750848 shutdown_connections 2024-09-16T15:45:35.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.065+0000 7f0132200640 1 --2- 172.21.15.60:0/4259750848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01100052d0 0x7f011000b9c0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.065+0000 7f0132200640 1 -- 172.21.15.60:0/4259750848 >> 172.21.15.60:0/4259750848 conn(0x7f012c0d1c20 msgr2=0x7f012c0d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:35.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.065+0000 7f0132200640 1 -- 172.21.15.60:0/4259750848 shutdown_connections 2024-09-16T15:45:35.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.065+0000 7f0132200640 1 -- 172.21.15.60:0/4259750848 wait complete. 2024-09-16T15:45:35.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.067+0000 7f0132200640 1 Processor -- start 2024-09-16T15:45:35.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.067+0000 7f0132200640 1 -- start start 2024-09-16T15:45:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0132200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01100052d0 0x7f012c0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:35.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0132200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f012c0ce7a0 0x7f012c0cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0132200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f012c1f7020 0x7f012c1f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0132200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f012c0b33c0 con 0x7f012c0ce7a0 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0132200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f012c0b3240 con 0x7f01100052d0 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0132200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f012c15f990 con 0x7f012c1f7020 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0123e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f012c0ce7a0 0x7f012c0cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0123e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f012c0ce7a0 0x7f012c0cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46820/0 (socket says 172.21.15.60:46820) 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.068+0000 7f0123e00640 1 -- 172.21.15.60:0/1421521362 learned_addr learned my addr 172.21.15.60:0/1421521362 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0130e00640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01100052d0 0x7f012c0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0131800640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f012c1f7020 0x7f012c1f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0123e00640 1 -- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f012c1f7020 msgr2=0x7f012c1f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0123e00640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f012c1f7020 0x7f012c1f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0123e00640 1 -- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01100052d0 msgr2=0x7f012c0cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0123e00640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01100052d0 0x7f012c0cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0123e00640 1 -- 172.21.15.60:0/1421521362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c0cf1e0 con 0x7f012c0ce7a0 2024-09-16T15:45:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0131800640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f012c1f7020 0x7f012c1f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0123e00640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f012c0ce7a0 0x7f012c0cec00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f0114006fd0 tx=0x7f0114001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0129600640 1 -- 172.21.15.60:0/1421521362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01140107d0 con 0x7f012c0ce7a0 2024-09-16T15:45:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0129600640 1 -- 172.21.15.60:0/1421521362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0114016070 con 0x7f012c0ce7a0 2024-09-16T15:45:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.069+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012c0cf4c0 con 0x7f012c0ce7a0 2024-09-16T15:45:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.070+0000 7f0129600640 1 -- 172.21.15.60:0/1421521362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114010ce0 con 0x7f012c0ce7a0 2024-09-16T15:45:35.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.070+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012c0b5770 con 0x7f012c0ce7a0 2024-09-16T15:45:35.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.071+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012c0cfa00 con 0x7f012c0ce7a0 2024-09-16T15:45:35.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.072+0000 7f0129600640 1 -- 172.21.15.60:0/1421521362 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f011401b460 con 0x7f012c0ce7a0 2024-09-16T15:45:35.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.073+0000 7f0129600640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0108078cd0 0x7f010807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:35.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.073+0000 7f0129600640 1 -- 172.21.15.60:0/1421521362 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f011409b9e0 con 0x7f012c0ce7a0 2024-09-16T15:45:35.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.074+0000 7f0130e00640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0108078cd0 0x7f010807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:35.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.074+0000 7f0130e00640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0108078cd0 0x7f010807b190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f011c005e00 tx=0x7f011c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:35.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.078+0000 7f0129600640 1 -- 172.21.15.60:0/1421521362 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0114063950 con 0x7f012c0ce7a0 2024-09-16T15:45:35.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.348+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f012c071ef0 con 0x7f012c0ce7a0 2024-09-16T15:45:35.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.351+0000 7f0129600640 1 -- 172.21.15.60:0/1421521362 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f0114063250 con 0x7f012c0ce7a0 2024-09-16T15:45:35.352 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:45:35.352 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:45:35.352 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:45:35.352 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:45:35.352 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:45:35.353 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:45:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.353+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0108078cd0 msgr2=0x7f010807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.353+0000 7f0132200640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0108078cd0 0x7f010807b190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f011c005e00 tx=0x7f011c005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.353+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f012c0ce7a0 msgr2=0x7f012c0cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.353+0000 7f0132200640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f012c0ce7a0 0x7f012c0cec00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f0114006fd0 tx=0x7f0114001800 comp rx=0 tx=0).stop 2024-09-16T15:45:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.354+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 shutdown_connections 2024-09-16T15:45:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.354+0000 7f0132200640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0108078cd0 0x7f010807b190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.354+0000 7f0132200640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f012c1f7020 0x7f012c1f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.354+0000 7f0132200640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f012c0ce7a0 0x7f012c0cec00 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.354+0000 7f0132200640 1 --2- 172.21.15.60:0/1421521362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01100052d0 0x7f012c0cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.354+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 >> 172.21.15.60:0/1421521362 conn(0x7f012c0d1c20 msgr2=0x7f012c15b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.354+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 shutdown_connections 2024-09-16T15:45:35.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.354+0000 7f0132200640 1 -- 172.21.15.60:0/1421521362 wait complete. 2024-09-16T15:45:35.371 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.529+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2671939884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 msgr2=0x7f9ca40adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.529+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2671939884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 0x7f9ca40adcd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c005fd0 tx=0x7f9c8c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.530+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2671939884 shutdown_connections 2024-09-16T15:45:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.530+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2671939884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 0x7f9ca40adcd0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.530+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2671939884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ca406fb90 0x7f9ca406ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.530+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2671939884 >> 172.21.15.60:0/2671939884 conn(0x7f9ca40cfa20 msgr2=0x7f9ca40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.530+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2671939884 shutdown_connections 2024-09-16T15:45:35.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.530+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2671939884 wait complete. 2024-09-16T15:45:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.531+0000 7f9ca8a00640 1 Processor -- start 2024-09-16T15:45:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.531+0000 7f9ca8a00640 1 -- start start 2024-09-16T15:45:35.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.532+0000 7f9ca8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ca406fb90 0x7f9ca4154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.532+0000 7f9ca8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 0x7f9ca40c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.532+0000 7f9ca8a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ca40cd5f0 0x7f9ca4157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.532+0000 7f9ca8a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca415ed60 con 0x7f9ca40ad880 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.532+0000 7f9ca8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca415ebe0 con 0x7f9ca406fb90 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.532+0000 7f9ca8a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca415eee0 con 0x7f9ca40cd5f0 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 0x7f9ca40c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 0x7f9ca40c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46852/0 (socket says 172.21.15.60:46852) 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 -- 172.21.15.60:0/2159930667 learned_addr learned my addr 172.21.15.60:0/2159930667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9ca3200640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ca40cd5f0 0x7f9ca4157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:35.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9ca2800640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ca406fb90 0x7f9ca4154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:35.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 -- 172.21.15.60:0/2159930667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ca40cd5f0 msgr2=0x7f9ca4157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ca40cd5f0 0x7f9ca4157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 -- 172.21.15.60:0/2159930667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ca406fb90 msgr2=0x7f9ca4154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ca406fb90 0x7f9ca4154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 -- 172.21.15.60:0/2159930667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca40c9930 con 0x7f9ca40ad880 2024-09-16T15:45:35.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.533+0000 7f9c9be00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 0x7f9ca40c8c00 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009850 tx=0x7f9c8c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:35.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.534+0000 7f9c9aa00640 1 -- 172.21.15.60:0/2159930667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c03f070 con 0x7f9ca40ad880 2024-09-16T15:45:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.534+0000 7f9c9aa00640 1 -- 172.21.15.60:0/2159930667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c8c005600 con 0x7f9ca40ad880 2024-09-16T15:45:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.534+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca40c9650 con 0x7f9ca40ad880 2024-09-16T15:45:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.534+0000 7f9c9aa00640 1 -- 172.21.15.60:0/2159930667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c035030 con 0x7f9ca40ad880 2024-09-16T15:45:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.534+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca41f82c0 con 0x7f9ca40ad880 2024-09-16T15:45:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.535+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca406ff70 con 0x7f9ca40ad880 2024-09-16T15:45:35.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.536+0000 7f9c9aa00640 1 -- 172.21.15.60:0/2159930667 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f9c8c042070 con 0x7f9ca40ad880 2024-09-16T15:45:35.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.537+0000 7f9c9aa00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c80078d00 0x7f9c8007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:35.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.537+0000 7f9c9aa00640 1 -- 172.21.15.60:0/2159930667 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f9c8c0bbd50 con 0x7f9ca40ad880 2024-09-16T15:45:35.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.537+0000 7f9ca2800640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c80078d00 0x7f9c8007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.538+0000 7f9ca2800640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c80078d00 0x7f9c8007b1c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4154760 tx=0x7f9c9400b390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:35.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.541+0000 7f9c9aa00640 1 -- 172.21.15.60:0/2159930667 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c8c0835d0 con 0x7f9ca40ad880 2024-09-16T15:45:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.774+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ca4154af0 con 0x7f9ca40ad880 2024-09-16T15:45:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.774+0000 7f9c9aa00640 1 -- 172.21.15.60:0/2159930667 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5225 (secure 0 0 0) 0x7f9c8c088000 con 0x7f9ca40ad880 2024-09-16T15:45:35.775 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:35.776 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":81,"btime":"2024-09-16T15:45:33:181165+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7225,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/900096462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":900096462},{"type":"v1","addr":"172.21.15.100:6834","nonce":900096462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2427775344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2427775344},{"type":"v1","addr":"172.21.15.100:6835","nonce":2427775344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7584,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1853423008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1853423008},{"type":"v1","addr":"172.21.15.60:6833","nonce":1853423008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:45:29.603235+0000","modified":"2024-09-16T15:45:32.622836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/678119606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":678119606},{"type":"v1","addr":"172.21.15.60:6835","nonce":678119606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7593,"qdb_cluster":[7593]},"id":9}]} 2024-09-16T15:45:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 81 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.776+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c80078d00 msgr2=0x7f9c8007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.776+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c80078d00 0x7f9c8007b1c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4154760 tx=0x7f9c9400b390 comp rx=0 tx=0).stop 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 msgr2=0x7f9ca40c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 0x7f9ca40c8c00 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009850 tx=0x7f9c8c0038a0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 shutdown_connections 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9c80078d00 0x7f9c8007b1c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ca40cd5f0 0x7f9ca4157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ca40ad880 0x7f9ca40c8c00 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 --2- 172.21.15.60:0/2159930667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ca406fb90 0x7f9ca4154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 >> 172.21.15.60:0/2159930667 conn(0x7f9ca40cfa20 msgr2=0x7f9ca406f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 shutdown_connections 2024-09-16T15:45:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:35.777+0000 7f9ca8a00640 1 -- 172.21.15.60:0/2159930667 wait complete. 2024-09-16T15:45:36.130 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-09-16T15:45:36.129+0000 7f2ca1c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:45:36.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:45:29.603235+0000', 'modified': '2024-09-16T15:45:32.622836+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7593}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7593': {'gid': 7593, 'name': 'c', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/678119606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 678119606}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 678119606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7593, 'qdb_cluster': [7593]} 2024-09-16T15:45:36.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:45:36.196 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:45:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.358+0000 7f381ca00640 1 -- 172.21.15.60:0/3872718260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180b4d10 msgr2=0x7f3818162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.358+0000 7f381ca00640 1 --2- 172.21.15.60:0/3872718260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180b4d10 0x7f3818162020 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3804005fd0 tx=0x7f380402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.358+0000 7f381ca00640 1 -- 172.21.15.60:0/3872718260 shutdown_connections 2024-09-16T15:45:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.358+0000 7f381ca00640 1 --2- 172.21.15.60:0/3872718260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180b4d10 0x7f3818162020 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.358+0000 7f381ca00640 1 --2- 172.21.15.60:0/3872718260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38180d1ec0 0x7f38180d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.358+0000 7f381ca00640 1 -- 172.21.15.60:0/3872718260 >> 172.21.15.60:0/3872718260 conn(0x7f38180d3de0 msgr2=0x7f38180d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.358+0000 7f381ca00640 1 -- 172.21.15.60:0/3872718260 shutdown_connections 2024-09-16T15:45:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.358+0000 7f381ca00640 1 -- 172.21.15.60:0/3872718260 wait complete. 2024-09-16T15:45:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.359+0000 7f381ca00640 1 Processor -- start 2024-09-16T15:45:36.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.359+0000 7f381ca00640 1 -- start start 2024-09-16T15:45:36.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.360+0000 7f381ca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180b4d10 0x7f3818160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.360+0000 7f381ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180d1ec0 0x7f3818152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.360+0000 7f381ca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3818162560 0x7f3818153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.360+0000 7f381ca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3818162b50 con 0x7f38180d1ec0 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.360+0000 7f381ca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38181629d0 con 0x7f38180b4d10 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.360+0000 7f381ca00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3818162cd0 con 0x7f3818162560 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180d1ec0 0x7f3818152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180d1ec0 0x7f3818152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46876/0 (socket says 172.21.15.60:46876) 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 -- 172.21.15.60:0/2853256750 learned_addr learned my addr 172.21.15.60:0/2853256750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3816800640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180b4d10 0x7f3818160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3817200640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3818162560 0x7f3818153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 -- 172.21.15.60:0/2853256750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3818162560 msgr2=0x7f3818153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3818162560 0x7f3818153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 -- 172.21.15.60:0/2853256750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180b4d10 msgr2=0x7f3818160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180b4d10 0x7f3818160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 -- 172.21.15.60:0/2853256750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38180cb6a0 con 0x7f38180d1ec0 2024-09-16T15:45:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f3815e00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180d1ec0 0x7f3818152ed0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3804006010 tx=0x7f38040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.361+0000 7f380ea00640 1 -- 172.21.15.60:0/2853256750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380403f070 con 0x7f38180d1ec0 2024-09-16T15:45:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.362+0000 7f380ea00640 1 -- 172.21.15.60:0/2853256750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3804005600 con 0x7f38180d1ec0 2024-09-16T15:45:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.362+0000 7f380ea00640 1 -- 172.21.15.60:0/2853256750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804035030 con 0x7f38180d1ec0 2024-09-16T15:45:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.362+0000 7f381ca00640 1 -- 172.21.15.60:0/2853256750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38180cb940 con 0x7f38180d1ec0 2024-09-16T15:45:36.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.362+0000 7f381ca00640 1 -- 172.21.15.60:0/2853256750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38180cbf00 con 0x7f38180d1ec0 2024-09-16T15:45:36.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.364+0000 7f380ea00640 1 -- 172.21.15.60:0/2853256750 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f3804042070 con 0x7f38180d1ec0 2024-09-16T15:45:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.365+0000 7f3820410740 1 Processor -- start 2024-09-16T15:45:36.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.365+0000 7f380ea00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f37f8078cd0 0x7f37f807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.365+0000 7f380ea00640 1 -- 172.21.15.60:0/2853256750 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f38040bbef0 con 0x7f38180d1ec0 2024-09-16T15:45:36.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.365+0000 7f3820410740 1 -- start start 2024-09-16T15:45:36.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.366+0000 7f3817200640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f37f8078cd0 0x7f37f807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.366+0000 7f3817200640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f37f8078cd0 0x7f37f807b190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3810007fd0 tx=0x7f3810009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.366+0000 7f3820410740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd488bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.366+0000 7f3820410740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48878c10 0x55bd488bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.366+0000 7f3820410740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488bcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.366+0000 7f3820410740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bd487fb430 con 0x55bd48878c10 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.366+0000 7f3820410740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bd487eb5d0 con 0x55bd48879190 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.366+0000 7f3820410740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55bd487daa00 con 0x55bd488bdaa0 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48878c10 0x55bd488bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48878c10 0x55bd488bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46892/0 (socket says 172.21.15.60:46892) 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 -- 172.21.15.60:0/4167122512 learned_addr learned my addr 172.21.15.60:0/4167122512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3816800640 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd488bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3817200640 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488bcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 -- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 msgr2=0x55bd488bde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd488bde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 -- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 msgr2=0x55bd488bcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488bcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 -- 172.21.15.60:0/4167122512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd487a3520 con 0x55bd48878c10 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3817200640 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.367+0000 7f3815e00640 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48878c10 0x55bd488bc580 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f38040833c0 tx=0x7f38040830d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75742b95e5e41c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3803000640 1 -- 172.21.15.60:0/4167122512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38040a9020 con 0x55bd48878c10 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3803000640 1 -- 172.21.15.60:0/4167122512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38040836c0 con 0x55bd48878c10 2024-09-16T15:45:36.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3803000640 1 -- 172.21.15.60:0/4167122512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804049040 con 0x55bd48878c10 2024-09-16T15:45:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 -- 172.21.15.60:0/4167122512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48878c10 msgr2=0x55bd488bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48878c10 0x55bd488bc580 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f38040833c0 tx=0x7f38040830d0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 -- 172.21.15.60:0/4167122512 shutdown_connections 2024-09-16T15:45:36.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488bcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48878c10 0x55bd488bc580 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 --2- 172.21.15.60:0/4167122512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd488bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 -- 172.21.15.60:0/4167122512 >> 172.21.15.60:0/4167122512 conn(0x55bd488741e0 msgr2=0x55bd48877c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 -- 172.21.15.60:0/4167122512 shutdown_connections 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.368+0000 7f3820410740 1 -- 172.21.15.60:0/4167122512 wait complete. 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.369+0000 7f3820410740 1 -- start start 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.369+0000 7f3820410740 1 Processor -- start 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.370+0000 7f3820410740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.370+0000 7f3820410740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd4888c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.370+0000 7f3820410740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd4888cd70 0x55bd48875f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.370+0000 7f3820410740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bd48781330 con 0x55bd4888cd70 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.370+0000 7f3820410740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bd487de810 con 0x55bd48879190 2024-09-16T15:45:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.370+0000 7f3820410740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55bd4854b620 con 0x55bd488bdaa0 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.370+0000 7f3820410740 10 client.0 _ll_register_callbacks cb 0x55bd48786420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.370+0000 7f3817200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd4888cd70 0x55bd48875f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3817200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd4888cd70 0x55bd48875f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46902/0 (socket says 172.21.15.60:46902) 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3817200640 1 -- 172.21.15.60:0/1562212120 learned_addr learned my addr 172.21.15.60:0/1562212120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3816800640 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3815e00640 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd4888c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3817200640 1 -- 172.21.15.60:0/1562212120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 msgr2=0x55bd4888c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3817200640 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd4888c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3817200640 1 -- 172.21.15.60:0/1562212120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 msgr2=0x55bd488b9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3817200640 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488b9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3817200640 1 -- 172.21.15.60:0/1562212120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4853c850 con 0x55bd4888cd70 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f3817200640 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd4888cd70 0x55bd48875f00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f381000e8a0 tx=0x7f3810001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145b25ed86c1d5af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.371+0000 7f37f7400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f37f7400640 1 -- 172.21.15.60:0/1562212120 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810027060 con 0x55bd4888cd70 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f37f7400640 1 -- 172.21.15.60:0/1562212120 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f3810010040 con 0x55bd4888cd70 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 1 -- 172.21.15.60:0/1562212120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38040836c0 con 0x55bd4888cd70 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f37f7400640 1 -- 172.21.15.60:0/1562212120 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3810005650 con 0x55bd4888cd70 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 10 client.7716 fetch_fsmap learned FSMap version 81 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 1 -- 172.21.15.60:0/1562212120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55bd485b36b0 con 0x55bd4888cd70 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f37f7400640 1 -- 172.21.15.60:0/1562212120 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38040836c0 con 0x55bd4888cd70 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f37f7400640 1 -- 172.21.15.60:0/1562212120 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 81) ==== 3502+0+0 (secure 0 0 0) 0x7f3810013630 con 0x55bd4888cd70 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 10 client.7716 fetch_fsmap finished waiting for FSMap version 81 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 10 client.7716 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 20 client.7716 populate_metadata read hostname 'smithi060' 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] conn(0x55bd488a15c0 0x55bd488d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 4 client.7716 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:45:36.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.372+0000 7f3820410740 1 -- 172.21.15.60:0/1562212120 --> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd487ed520 con 0x55bd488a15c0 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.373+0000 7f3816800640 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] conn(0x55bd488a15c0 0x55bd488d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.373+0000 7f3816800640 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] conn(0x55bd488a15c0 0x55bd488d00e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45eeee317b1ec69 server_cookie=3b1c9f443bc6d73c in_seq=0 out_seq=0 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.373+0000 7f37f7400640 10 client.7716 ms_handle_connect on v2:172.21.15.60:6834/678119606 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f37f7400640 1 -- 172.21.15.60:0/1562212120 <== mds.0 v2:172.21.15.60:6834/678119606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd487ed520 con 0x55bd488a15c0 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f37f7400640 10 client.7716 handle_command_reply: tid=0 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 client.7716 shutdown 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 -- 172.21.15.60:0/1562212120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd4888cd70 msgr2=0x55bd48875f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd4888cd70 0x55bd48875f00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f381000e8a0 tx=0x7f3810001250 comp rx=0 tx=0).stop 2024-09-16T15:45:36.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 -- 172.21.15.60:0/1562212120 shutdown_connections 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] conn(0x55bd488a15c0 0x55bd488d00e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd4888cd70 0x55bd48875f00 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd4888c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 --2- 172.21.15.60:0/1562212120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd48879190 0x55bd488b9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f37f7400640 0 client.7716 ms_handle_reset on v2:172.21.15.60:6834/678119606 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.374+0000 7f3820410740 1 -- 172.21.15.60:0/1562212120 >> 172.21.15.60:0/1562212120 conn(0x55bd48878c10 msgr2=0x55bd488b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.375+0000 7f3820410740 1 -- 172.21.15.60:0/1562212120 shutdown_connections 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.375+0000 7f3820410740 1 -- 172.21.15.60:0/1562212120 wait complete. 2024-09-16T15:45:36.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.375+0000 7f3820410740 20 client.7716 trim_cache size 0 max 16384 2024-09-16T15:45:36.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.394+0000 7f3820410740 1 Processor -- start 2024-09-16T15:45:36.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.394+0000 7f3820410740 1 -- start start 2024-09-16T15:45:36.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.394+0000 7f3820410740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd4889d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3820410740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd488b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3820410740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 0x55bd488e0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3820410740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f38182000f0 con 0x55bd48879190 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3820410740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38180cd5a0 con 0x55bd488b6c40 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3820410740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f38180cdbf0 con 0x55bd488bdaa0 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd4889d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd4889d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46914/0 (socket says 172.21.15.60:46914) 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 -- 172.21.15.60:0/849964727 learned_addr learned my addr 172.21.15.60:0/849964727 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3815e00640 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd488b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3817200640 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 0x55bd488e0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 -- 172.21.15.60:0/849964727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 msgr2=0x55bd488b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd488b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 -- 172.21.15.60:0/849964727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 msgr2=0x55bd488e0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 0x55bd488e0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 -- 172.21.15.60:0/849964727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd488ba340 con 0x55bd48879190 2024-09-16T15:45:36.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.395+0000 7f3816800640 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd4889d420 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f380800d370 tx=0x7f380800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e858bd9c9356cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:36.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3802600640 1 -- 172.21.15.60:0/849964727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808003b10 con 0x55bd48879190 2024-09-16T15:45:36.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3802600640 1 -- 172.21.15.60:0/849964727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3808003cb0 con 0x55bd48879190 2024-09-16T15:45:36.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3802600640 1 -- 172.21.15.60:0/849964727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380801d670 con 0x55bd48879190 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3820410740 1 -- 172.21.15.60:0/849964727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 msgr2=0x55bd4889d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3820410740 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd4889d420 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f380800d370 tx=0x7f380800dd50 comp rx=0 tx=0).stop 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3820410740 1 -- 172.21.15.60:0/849964727 shutdown_connections 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3820410740 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 0x55bd488e0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3820410740 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd488b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3820410740 1 --2- 172.21.15.60:0/849964727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd4889d420 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.396+0000 7f3820410740 1 -- 172.21.15.60:0/849964727 >> 172.21.15.60:0/849964727 conn(0x55bd48878c10 msgr2=0x55bd488b9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.397+0000 7f3820410740 1 -- 172.21.15.60:0/849964727 shutdown_connections 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.397+0000 7f3820410740 1 -- 172.21.15.60:0/849964727 wait complete. 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.397+0000 7f3820410740 1 -- start start 2024-09-16T15:45:36.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.397+0000 7f3820410740 1 Processor -- start 2024-09-16T15:45:36.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.398+0000 7f3820410740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd488df180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.398+0000 7f3820410740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 0x55bd4893ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.398+0000 7f3820410740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd4893f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.398+0000 7f3820410740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55bd488a1440 con 0x55bd48879190 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.398+0000 7f3820410740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bd488ba5c0 con 0x55bd488b6c40 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.398+0000 7f3820410740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55bd488a1a90 con 0x55bd488bdaa0 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.398+0000 7f3820410740 10 client.0 _ll_register_callbacks cb 0x7f3810003c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.398+0000 7f3816800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd488df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3816800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd488df180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46930/0 (socket says 172.21.15.60:46930) 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3816800640 1 -- 172.21.15.60:0/2520944425 learned_addr learned my addr 172.21.15.60:0/2520944425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3815e00640 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 0x55bd4893ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3817200640 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd4893f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3816800640 1 -- 172.21.15.60:0/2520944425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 msgr2=0x55bd4893f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3816800640 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd4893f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3816800640 1 -- 172.21.15.60:0/2520944425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 msgr2=0x55bd4893ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3816800640 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 0x55bd4893ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3816800640 1 -- 172.21.15.60:0/2520944425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd488b26b0 con 0x55bd48879190 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3816800640 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd488df180 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f380800d870 tx=0x7f3808005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b41b2039f3df06 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f37f7400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f37f7400640 1 -- 172.21.15.60:0/2520944425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808020040 con 0x55bd48879190 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f37f7400640 1 -- 172.21.15.60:0/2520944425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f3808005220 con 0x55bd48879190 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.399+0000 7f3820410740 1 -- 172.21.15.60:0/2520944425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3808003cb0 con 0x55bd48879190 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f37f7400640 1 -- 172.21.15.60:0/2520944425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3808005420 con 0x55bd48879190 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3820410740 10 client.7722 fetch_fsmap learned FSMap version 81 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3820410740 1 -- 172.21.15.60:0/2520944425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55bd4893fb20 con 0x55bd48879190 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f37f7400640 1 -- 172.21.15.60:0/2520944425 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3808003cb0 con 0x55bd48879190 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f37f7400640 1 -- 172.21.15.60:0/2520944425 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 81) ==== 3502+0+0 (secure 0 0 0) 0x7f3808038020 con 0x55bd48879190 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3820410740 10 client.7722 fetch_fsmap finished waiting for FSMap version 81 2024-09-16T15:45:36.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3820410740 10 client.7722 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:45:36.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3820410740 20 client.7722 populate_metadata read hostname 'smithi060' 2024-09-16T15:45:36.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3820410740 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] conn(0x55bd488cf2e0 0x55bd4888b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3820410740 4 client.7722 mds_command: new command op to 7593 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:45:36.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3820410740 1 -- 172.21.15.60:0/2520944425 --> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] -- command(tid 0: {"prefix": "status"}) -- 0x55bd487e8fd0 con 0x55bd488cf2e0 2024-09-16T15:45:36.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.400+0000 7f3815e00640 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] conn(0x55bd488cf2e0 0x55bd4888b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3815e00640 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] conn(0x55bd488cf2e0 0x55bd4888b600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2c9fdf84b65727 server_cookie=ad9e7e16ca58f6ee in_seq=0 out_seq=0 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f37f7400640 10 client.7722 ms_handle_connect on v2:172.21.15.60:6834/678119606 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f37f7400640 1 -- 172.21.15.60:0/2520944425 <== mds.0 v2:172.21.15.60:6834/678119606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bd487e8fd0 con 0x55bd488cf2e0 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f37f7400640 10 client.7722 handle_command_reply: tid=0 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 client.7722 shutdown 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 -- 172.21.15.60:0/2520944425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 msgr2=0x55bd488df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd488df180 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f380800d870 tx=0x7f3808005850 comp rx=0 tx=0).stop 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 -- 172.21.15.60:0/2520944425 shutdown_connections 2024-09-16T15:45:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.60:6834/678119606,v1:172.21.15.60:6835/678119606] conn(0x55bd488cf2e0 0x55bd4888b600 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bd488bdaa0 0x55bd4893f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bd488b6c40 0x55bd4893ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f37f7400640 0 client.7722 ms_handle_reset on v2:172.21.15.60:6834/678119606 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 --2- 172.21.15.60:0/2520944425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bd48879190 0x55bd488df180 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.401+0000 7f3820410740 1 -- 172.21.15.60:0/2520944425 >> 172.21.15.60:0/2520944425 conn(0x55bd48878c10 msgr2=0x55bd488b9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f3820410740 1 -- 172.21.15.60:0/2520944425 shutdown_connections 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f3820410740 1 -- 172.21.15.60:0/2520944425 wait complete. 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f3820410740 20 client.7722 trim_cache size 0 max 16384 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:45:36.406 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7593, 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2443500109999999, 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 81, 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 122, 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 122, 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.510880187 2024-09-16T15:45:36.407 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 -- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f37f8078cd0 msgr2=0x7f37f807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f37f8078cd0 0x7f37f807b190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3810007fd0 tx=0x7f3810009040 comp rx=0 tx=0).stop 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 -- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180d1ec0 msgr2=0x7f3818152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180d1ec0 0x7f3818152ed0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3804006010 tx=0x7f38040038a0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 -- 172.21.15.60:0/2853256750 shutdown_connections 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f37f8078cd0 0x7f37f807b190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3818162560 0x7f3818153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180d1ec0 0x7f3818152ed0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 --2- 172.21.15.60:0/2853256750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180b4d10 0x7f3818160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.402+0000 7f381ca00640 1 -- 172.21.15.60:0/2853256750 >> 172.21.15.60:0/2853256750 conn(0x7f38180d3de0 msgr2=0x7f381815e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.403+0000 7f381ca00640 1 -- 172.21.15.60:0/2853256750 shutdown_connections 2024-09-16T15:45:36.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.403+0000 7f381ca00640 1 -- 172.21.15.60:0/2853256750 wait complete. 2024-09-16T15:45:36.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:45:36.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.578+0000 7fca1c800640 1 -- 172.21.15.60:0/3502988968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca14072df0 msgr2=0x7fca140732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.578+0000 7fca1c800640 1 --2- 172.21.15.60:0/3502988968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca14072df0 0x7fca140732a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fca10005fd0 tx=0x7fca1002aa10 comp rx=0 tx=0).stop 2024-09-16T15:45:36.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.578+0000 7fca1c800640 1 -- 172.21.15.60:0/3502988968 shutdown_connections 2024-09-16T15:45:36.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.578+0000 7fca1c800640 1 --2- 172.21.15.60:0/3502988968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca14072df0 0x7fca140732a0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.578+0000 7fca1c800640 1 --2- 172.21.15.60:0/3502988968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca14161550 0x7fca1406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.578+0000 7fca1c800640 1 -- 172.21.15.60:0/3502988968 >> 172.21.15.60:0/3502988968 conn(0x7fca140b1950 msgr2=0x7fca140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:36.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.578+0000 7fca1c800640 1 -- 172.21.15.60:0/3502988968 shutdown_connections 2024-09-16T15:45:36.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.578+0000 7fca1c800640 1 -- 172.21.15.60:0/3502988968 wait complete. 2024-09-16T15:45:36.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.580+0000 7fca1c800640 1 Processor -- start 2024-09-16T15:45:36.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.580+0000 7fca1c800640 1 -- start start 2024-09-16T15:45:36.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca1c800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca14072df0 0x7fca140c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca1c800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca140af3d0 0x7fca140cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca1c800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca14161550 0x7fca14151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca1c800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fca140737e0 con 0x7fca140af3d0 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca1c800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca1406fdb0 con 0x7fca14072df0 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca1c800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fca14073960 con 0x7fca14161550 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca19400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca140af3d0 0x7fca140cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca19400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca140af3d0 0x7fca140cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46950/0 (socket says 172.21.15.60:46950) 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca19400640 1 -- 172.21.15.60:0/1645926736 learned_addr learned my addr 172.21.15.60:0/1645926736 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca19e00640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca14072df0 0x7fca140c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.581+0000 7fca1a800640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca14161550 0x7fca14151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca19400640 1 -- 172.21.15.60:0/1645926736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca14161550 msgr2=0x7fca14151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca19400640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca14161550 0x7fca14151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca19400640 1 -- 172.21.15.60:0/1645926736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca14072df0 msgr2=0x7fca140c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca19400640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca14072df0 0x7fca140c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca19400640 1 -- 172.21.15.60:0/1645926736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca140cc150 con 0x7fca140af3d0 2024-09-16T15:45:36.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca19400640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca140af3d0 0x7fca140cbb20 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c420 tx=0x7fca0400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:36.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca0e000640 1 -- 172.21.15.60:0/1645926736 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04017070 con 0x7fca140af3d0 2024-09-16T15:45:36.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca0e000640 1 -- 172.21.15.60:0/1645926736 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca040053c0 con 0x7fca140af3d0 2024-09-16T15:45:36.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca140cc430 con 0x7fca140af3d0 2024-09-16T15:45:36.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca0e000640 1 -- 172.21.15.60:0/1645926736 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04003850 con 0x7fca140af3d0 2024-09-16T15:45:36.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.582+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca140cc9f0 con 0x7fca140af3d0 2024-09-16T15:45:36.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.583+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca140b3030 con 0x7fca140af3d0 2024-09-16T15:45:36.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.584+0000 7fca0e000640 1 -- 172.21.15.60:0/1645926736 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7fca0401a070 con 0x7fca140af3d0 2024-09-16T15:45:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.585+0000 7fca0e000640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9f8078cd0 0x7fc9f807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.586+0000 7fca0e000640 1 -- 172.21.15.60:0/1645926736 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7fca0409a5b0 con 0x7fca140af3d0 2024-09-16T15:45:36.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.586+0000 7fca19e00640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9f8078cd0 0x7fc9f807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:36.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.586+0000 7fca19e00640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9f8078cd0 0x7fc9f807b190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fca0800bb50 tx=0x7fca0800b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:36.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.590+0000 7fca0e000640 1 -- 172.21.15.60:0/1645926736 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fca140b3030 con 0x7fca140af3d0 2024-09-16T15:45:36.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.823+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fca14074390 con 0x7fca140af3d0 2024-09-16T15:45:36.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.824+0000 7fca0e000640 1 -- 172.21.15.60:0/1645926736 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7fca04061da0 con 0x7fca140af3d0 2024-09-16T15:45:36.825 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:45:36.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9f8078cd0 msgr2=0x7fc9f807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9f8078cd0 0x7fc9f807b190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fca0800bb50 tx=0x7fca0800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca140af3d0 msgr2=0x7fca140cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca140af3d0 0x7fca140cbb20 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c420 tx=0x7fca0400c8e0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 shutdown_connections 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc9f8078cd0 0x7fc9f807b190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fca14161550 0x7fca14151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca140af3d0 0x7fca140cbb20 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 --2- 172.21.15.60:0/1645926736 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca14072df0 0x7fca140c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 >> 172.21.15.60:0/1645926736 conn(0x7fca140b1950 msgr2=0x7fca1406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 shutdown_connections 2024-09-16T15:45:36.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:36.826+0000 7fca1c800640 1 -- 172.21.15.60:0/1645926736 wait complete. 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:45:36.844 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:45:36.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:36.845 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:45:36.880 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:45:36.880 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:45:36.880 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:45:36.880 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:45:36.881 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42312sec preferred_lft 42312sec 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:45:36.882 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:45:36.883 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:36.883 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:45:36.883 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:45:36.883 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:45:36.883 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:36.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:36.883 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:45:36.974 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:45:36.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:36.977 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:45:37.013 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:45:37.013 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:45:37.013 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:45:37.013 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:45:37.014 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:45:37.014 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:45:37.014 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:45:37.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:45:37.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:45:37.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:45:37.014 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:45:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:45:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:45:37.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:37.214 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:45:37.249 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:45:37.249 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:45:37.249 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:45:37.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:37.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:45:37.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:45:37.345 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:45:37.381 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:45:37.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:45:37.382 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:45:37.456 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:45:37.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:37.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:45:37.585 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:45:37.585 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:45:37.612 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:45:37.612 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:45:37.612 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:45:37.612 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:45:37.612 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:45:37.614 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:45:37.614 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:45:37.614 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:45:37.614 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:45:37.614 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:45:37.617 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:45:37.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:37.617 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:45:37.673 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:45:37.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:45:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:45:37.770 INFO:teuthology.orchestra.run.smithi060.stdout:client7731 2024-09-16T15:45:37.770 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:37.770 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:37.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:45:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.005+0000 7f8437e00640 1 -- 172.21.15.60:0/551370060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f841c004800 msgr2=0x7f841c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.005+0000 7f8437e00640 1 --2- 172.21.15.60:0/551370060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f841c004800 0x7f841c004c80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8438072580 tx=0x7f841c005830 comp rx=0 tx=0).stop 2024-09-16T15:45:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.005+0000 7f8437400640 1 -- 172.21.15.60:0/551370060 reap_dead start 2024-09-16T15:45:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.005+0000 7f8437e00640 1 -- 172.21.15.60:0/551370060 shutdown_connections 2024-09-16T15:45:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.005+0000 7f8437e00640 1 -- 172.21.15.60:0/551370060 >> 172.21.15.60:0/551370060 conn(0x7f84380cfa20 msgr2=0x7f84380cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.006+0000 7f8437e00640 1 -- 172.21.15.60:0/551370060 shutdown_connections 2024-09-16T15:45:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.006+0000 7f8437e00640 1 -- 172.21.15.60:0/551370060 wait complete. 2024-09-16T15:45:38.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.007+0000 7f8437e00640 1 Processor -- start 2024-09-16T15:45:38.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.007+0000 7f8437e00640 1 -- start start 2024-09-16T15:45:38.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.008+0000 7f8437e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f841c004800 0x7f8438154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:38.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.008+0000 7f8437e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84381548b0 0x7f84381f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:38.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.008+0000 7f8437e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84381f2c50 0x7f84380cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:38.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.008+0000 7f8437e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8438074660 con 0x7f84381f2c50 2024-09-16T15:45:38.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.008+0000 7f8437e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84380b28f0 con 0x7f841c004800 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.008+0000 7f8437e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f843814e0e0 con 0x7f84381548b0 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84381f2c50 0x7f84380cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84381f2c50 0x7f84380cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46976/0 (socket says 172.21.15.60:46976) 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 -- 172.21.15.60:0/170487615 learned_addr learned my addr 172.21.15.60:0/170487615 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8436a00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f841c004800 0x7f8438154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f842fe00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84381548b0 0x7f84381f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 -- 172.21.15.60:0/170487615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84381548b0 msgr2=0x7f84381f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84381548b0 0x7f84381f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 -- 172.21.15.60:0/170487615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f841c004800 msgr2=0x7f8438154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f841c004800 0x7f8438154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 -- 172.21.15.60:0/170487615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8438154d80 con 0x7f84381f2c50 2024-09-16T15:45:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.009+0000 7f8437400640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84381f2c50 0x7f84380cc400 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8430005fd0 tx=0x7f8430011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:38.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.010+0000 7f842ea00640 1 -- 172.21.15.60:0/170487615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430004b20 con 0x7f84381f2c50 2024-09-16T15:45:38.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.010+0000 7f842ea00640 1 -- 172.21.15.60:0/170487615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8430004cc0 con 0x7f84381f2c50 2024-09-16T15:45:38.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.010+0000 7f842ea00640 1 -- 172.21.15.60:0/170487615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430003bd0 con 0x7f84381f2c50 2024-09-16T15:45:38.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.010+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843814e3a0 con 0x7f84381f2c50 2024-09-16T15:45:38.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.010+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843814e960 con 0x7f84381f2c50 2024-09-16T15:45:38.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.011+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8438073d40 con 0x7f84381f2c50 2024-09-16T15:45:38.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.012+0000 7f842ea00640 1 -- 172.21.15.60:0/170487615 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f8430004030 con 0x7f84381f2c50 2024-09-16T15:45:38.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.013+0000 7f842ea00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8414078cd0 0x7f841407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:38.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.013+0000 7f842ea00640 1 -- 172.21.15.60:0/170487615 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f843009b5d0 con 0x7f84381f2c50 2024-09-16T15:45:38.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.013+0000 7f8436a00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8414078cd0 0x7f841407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:38.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.014+0000 7f8436a00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8414078cd0 0x7f841407b190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8428006560 tx=0x7f8428006b10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:38.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.017+0000 7f842ea00640 1 -- 172.21.15.60:0/170487615 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8430062dc0 con 0x7f84381f2c50 2024-09-16T15:45:38.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.255+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84380cad00 con 0x7f84381f2c50 2024-09-16T15:45:38.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.256+0000 7f842ea00640 1 -- 172.21.15.60:0/170487615 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f84300677f0 con 0x7f84381f2c50 2024-09-16T15:45:38.257 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:45:38.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.258+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8414078cd0 msgr2=0x7f841407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:38.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.258+0000 7f8437e00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8414078cd0 0x7f841407b190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8428006560 tx=0x7f8428006b10 comp rx=0 tx=0).stop 2024-09-16T15:45:38.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.258+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84381f2c50 msgr2=0x7f84380cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:38.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.258+0000 7f8437e00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84381f2c50 0x7f84380cc400 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8430005fd0 tx=0x7f8430011ef0 comp rx=0 tx=0).stop 2024-09-16T15:45:38.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.259+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 shutdown_connections 2024-09-16T15:45:38.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.259+0000 7f8437e00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8414078cd0 0x7f841407b190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:38.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.259+0000 7f8437e00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84381f2c50 0x7f84380cc400 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.259+0000 7f8437e00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84381548b0 0x7f84381f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.259+0000 7f8437e00640 1 --2- 172.21.15.60:0/170487615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f841c004800 0x7f8438154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.259+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 >> 172.21.15.60:0/170487615 conn(0x7f84380cfa20 msgr2=0x7f84380c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.259+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 shutdown_connections 2024-09-16T15:45:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:38.259+0000 7f8437e00640 1 -- 172.21.15.60:0/170487615 wait complete. 2024-09-16T15:45:38.275 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:45:38.276 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:45:38.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:45:38.276 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:45:38.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:45:38.276 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:45:38.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:45:38.276 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:45:38.276 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:45:38.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:38.276 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:45:38.317 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:45:38.317 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:45:38.317 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:45:38.317 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:38.317 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:45:38.317 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:38.317 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42311sec preferred_lft 42311sec 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:38.318 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:45:38.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:38.320 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:45:38.415 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:45:38.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:38.419 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:45:38.456 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:45:38.456 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:45:38.456 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:45:38.456 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:45:38.456 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:45:38.457 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:45:38.457 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:45:38.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:45:38.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:45:38.457 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:45:38.457 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:45:38.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:45:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:45:38.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:45:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:45:38.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:38.652 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:45:38.688 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:45:38.688 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:45:38.688 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:45:38.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:38.688 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:45:38.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:45:38.784 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:45:38.819 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:45:38.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:45:38.819 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:45:38.894 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:45:38.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:38.894 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:45:39.021 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:45:39.022 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:45:39.049 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:45:39.049 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:45:39.049 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:45:39.049 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:45:39.049 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:45:39.051 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:45:39.051 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:45:39.051 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:45:39.051 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:45:39.051 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:45:39.054 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:45:39.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:39.054 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:45:39.110 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:45:39.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:45:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:45:39.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:45:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:45:39.199 INFO:teuthology.orchestra.run.smithi100.stdout:client7740 2024-09-16T15:45:39.200 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:39.200 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:39.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:45:39.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.412+0000 7f889e800640 1 -- 172.21.15.60:0/2267997647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889806f220 msgr2=0x7f889806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:39.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.412+0000 7f889e800640 1 --2- 172.21.15.60:0/2267997647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889806f220 0x7f889806f6b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8890009be0 tx=0x7f889002a920 comp rx=0 tx=0).stop 2024-09-16T15:45:39.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.412+0000 7f889e800640 1 -- 172.21.15.60:0/2267997647 shutdown_connections 2024-09-16T15:45:39.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.412+0000 7f889e800640 1 --2- 172.21.15.60:0/2267997647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889806f220 0x7f889806f6b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:39.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.412+0000 7f889e800640 1 -- 172.21.15.60:0/2267997647 >> 172.21.15.60:0/2267997647 conn(0x7f88980b18e0 msgr2=0x7f88980b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.412+0000 7f889e800640 1 -- 172.21.15.60:0/2267997647 shutdown_connections 2024-09-16T15:45:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.412+0000 7f889e800640 1 -- 172.21.15.60:0/2267997647 wait complete. 2024-09-16T15:45:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.413+0000 7f889e800640 1 Processor -- start 2024-09-16T15:45:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.413+0000 7f889e800640 1 -- start start 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.414+0000 7f889e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806f220 0x7f8898152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.414+0000 7f889e800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8898159aa0 0x7f88980d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.414+0000 7f889e800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889815cff0 0x7f889809b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.414+0000 7f889e800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88980cc710 con 0x7f889806f220 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.414+0000 7f889e800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88980cc590 con 0x7f8898159aa0 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.414+0000 7f889e800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88980cc890 con 0x7f889815cff0 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806f220 0x7f8898152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806f220 0x7f8898152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33758/0 (socket says 172.21.15.60:33758) 2024-09-16T15:45:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 -- 172.21.15.60:0/618049213 learned_addr learned my addr 172.21.15.60:0/618049213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 -- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889815cff0 msgr2=0x7f889809b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897e00640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889815cff0 0x7f889809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f888ea00640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8898159aa0 0x7f88980d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889815cff0 0x7f889809b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 -- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8898159aa0 msgr2=0x7f88980d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8898159aa0 0x7f88980d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 -- 172.21.15.60:0/618049213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8898154070 con 0x7f889806f220 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897e00640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889815cff0 0x7f889809b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f888ea00640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8898159aa0 0x7f88980d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.415+0000 7f8897400640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806f220 0x7f8898152410 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f888800c4c0 tx=0x7f888800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.416+0000 7f888fe00640 1 -- 172.21.15.60:0/618049213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888016020 con 0x7f889806f220 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.416+0000 7f888fe00640 1 -- 172.21.15.60:0/618049213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8888012940 con 0x7f889806f220 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.416+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88980b02b0 con 0x7f889806f220 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.416+0000 7f888fe00640 1 -- 172.21.15.60:0/618049213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888012d90 con 0x7f889806f220 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.416+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88980b0870 con 0x7f889806f220 2024-09-16T15:45:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.417+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8898069360 con 0x7f889806f220 2024-09-16T15:45:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.418+0000 7f888fe00640 1 -- 172.21.15.60:0/618049213 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f8888019020 con 0x7f889806f220 2024-09-16T15:45:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.419+0000 7f888fe00640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8874078cd0 0x7f887407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.419+0000 7f888fe00640 1 -- 172.21.15.60:0/618049213 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f888809b270 con 0x7f889806f220 2024-09-16T15:45:39.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.419+0000 7f888ea00640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8874078cd0 0x7f887407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:39.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.420+0000 7f888ea00640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8874078cd0 0x7f887407b190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8880007fd0 tx=0x7f8880009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:39.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.424+0000 7f888fe00640 1 -- 172.21.15.60:0/618049213 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8888020020 con 0x7f889806f220 2024-09-16T15:45:39.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:39.658+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f8898154f30 con 0x7f889806f220 2024-09-16T15:45:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.368+0000 7f888fe00640 1 -- 172.21.15.60:0/618049213 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v82) ==== 136+0+0 (secure 0 0 0) 0x7f8888063140 con 0x7f889806f220 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.371+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8874078cd0 msgr2=0x7f887407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.371+0000 7f889e800640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8874078cd0 0x7f887407b190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8880007fd0 tx=0x7f8880009040 comp rx=0 tx=0).stop 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.371+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806f220 msgr2=0x7f8898152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.371+0000 7f889e800640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806f220 0x7f8898152410 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f888800c4c0 tx=0x7f888800c980 comp rx=0 tx=0).stop 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.372+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 shutdown_connections 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.372+0000 7f889e800640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8874078cd0 0x7f887407b190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.372+0000 7f889e800640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f889815cff0 0x7f889809b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.372+0000 7f889e800640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8898159aa0 0x7f88980d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.372+0000 7f889e800640 1 --2- 172.21.15.60:0/618049213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f889806f220 0x7f8898152410 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.372+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 >> 172.21.15.60:0/618049213 conn(0x7f88980b18e0 msgr2=0x7f889815e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.372+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 shutdown_connections 2024-09-16T15:45:40.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.372+0000 7f889e800640 1 -- 172.21.15.60:0/618049213 wait complete. 2024-09-16T15:45:40.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:45:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.556+0000 7f74a8e00640 1 -- 172.21.15.60:0/2774232976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a40725d0 msgr2=0x7f74a40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.556+0000 7f74a8e00640 1 --2- 172.21.15.60:0/2774232976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a40725d0 0x7f74a40b28f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f748c005fd0 tx=0x7f748c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.556+0000 7f74a8e00640 1 -- 172.21.15.60:0/2774232976 shutdown_connections 2024-09-16T15:45:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.556+0000 7f74a8e00640 1 --2- 172.21.15.60:0/2774232976 >> v2:172.21.15.100:3300/0 conn(0x7f74a40d0540 0x7f74a406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.556+0000 7f74a8e00640 1 --2- 172.21.15.60:0/2774232976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a40725d0 0x7f74a40b28f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.556+0000 7f74a8e00640 1 -- 172.21.15.60:0/2774232976 >> 172.21.15.60:0/2774232976 conn(0x7f74a40cfa20 msgr2=0x7f74a40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.556+0000 7f74a8e00640 1 -- 172.21.15.60:0/2774232976 shutdown_connections 2024-09-16T15:45:40.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.556+0000 7f74a8e00640 1 -- 172.21.15.60:0/2774232976 wait complete. 2024-09-16T15:45:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.558+0000 7f74a8e00640 1 Processor -- start 2024-09-16T15:45:40.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.558+0000 7f74a8e00640 1 -- start start 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f74a8e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a40d0540 0x7f74a4155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f74a8e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a415bc30 0x7f74a40c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f74a8e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a4144d30 0x7f74a41451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f74a8e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74a4074660 con 0x7f74a415bc30 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f74a8e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74a415c200 con 0x7f74a4144d30 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f74a8e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74a40c9a20 con 0x7f74a40d0540 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f749aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a415bc30 0x7f74a40c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f749aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a415bc30 0x7f74a40c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33774/0 (socket says 172.21.15.60:33774) 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f749aa00640 1 -- 172.21.15.60:0/779337585 learned_addr learned my addr 172.21.15.60:0/779337585 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f74a3e00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a4144d30 0x7f74a41451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f74a3400640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a40d0540 0x7f74a4155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:40.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.559+0000 7f749aa00640 1 -- 172.21.15.60:0/779337585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a40d0540 msgr2=0x7f74a4155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f749aa00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a40d0540 0x7f74a4155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f749aa00640 1 -- 172.21.15.60:0/779337585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a4144d30 msgr2=0x7f74a41451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f749aa00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a4144d30 0x7f74a41451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f749aa00640 1 -- 172.21.15.60:0/779337585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a4150ff0 con 0x7f74a415bc30 2024-09-16T15:45:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f749aa00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a415bc30 0x7f74a40c94e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f748c009850 tx=0x7f748c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f749be00640 1 -- 172.21.15.60:0/779337585 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c03f070 con 0x7f74a415bc30 2024-09-16T15:45:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a4155c10 con 0x7f74a415bc30 2024-09-16T15:45:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f749be00640 1 -- 172.21.15.60:0/779337585 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f748c003ce0 con 0x7f74a415bc30 2024-09-16T15:45:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f749be00640 1 -- 172.21.15.60:0/779337585 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c035030 con 0x7f74a415bc30 2024-09-16T15:45:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.560+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a41561d0 con 0x7f74a415bc30 2024-09-16T15:45:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.561+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a406f290 con 0x7f74a415bc30 2024-09-16T15:45:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.562+0000 7f749be00640 1 -- 172.21.15.60:0/779337585 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f748c042070 con 0x7f74a415bc30 2024-09-16T15:45:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.564+0000 7f749be00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7480078cd0 0x7f748007b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.564+0000 7f749be00640 1 -- 172.21.15.60:0/779337585 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f748c0bbd80 con 0x7f74a415bc30 2024-09-16T15:45:40.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.564+0000 7f74a3400640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7480078cd0 0x7f748007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:40.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.565+0000 7f74a3400640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7480078cd0 0x7f748007b190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7494001720 tx=0x7f7494009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:40.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.568+0000 7f749be00640 1 -- 172.21.15.60:0/779337585 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f74a406f290 con 0x7f74a415bc30 2024-09-16T15:45:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.788+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f74a406fc40 con 0x7f7480078cd0 2024-09-16T15:45:40.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.789+0000 7f749be00640 1 -- 172.21.15.60:0/779337585 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f74a406fc40 con 0x7f7480078cd0 2024-09-16T15:45:40.790 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:45:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:45:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:45:40.790 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:40.791 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:45:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7480078cd0 msgr2=0x7f748007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7480078cd0 0x7f748007b190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7494001720 tx=0x7f7494009040 comp rx=0 tx=0).stop 2024-09-16T15:45:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a415bc30 msgr2=0x7f74a40c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a415bc30 0x7f74a40c94e0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f748c009850 tx=0x7f748c0035f0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 shutdown_connections 2024-09-16T15:45:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7480078cd0 0x7f748007b190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a4144d30 0x7f74a41451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a415bc30 0x7f74a40c94e0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 --2- 172.21.15.60:0/779337585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a40d0540 0x7f74a4155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 >> 172.21.15.60:0/779337585 conn(0x7f74a40cfa20 msgr2=0x7f74a40ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 shutdown_connections 2024-09-16T15:45:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.792+0000 7f74a8e00640 1 -- 172.21.15.60:0/779337585 wait complete. 2024-09-16T15:45:40.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:45:40.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.970+0000 7f6a51400640 1 -- 172.21.15.60:0/3862190687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c145750 msgr2=0x7f6a4c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.970+0000 7f6a51400640 1 --2- 172.21.15.60:0/3862190687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c145750 0x7f6a4c073440 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009830 tx=0x7f6a3c02a480 comp rx=0 tx=0).stop 2024-09-16T15:45:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.970+0000 7f6a51400640 1 -- 172.21.15.60:0/3862190687 shutdown_connections 2024-09-16T15:45:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.970+0000 7f6a51400640 1 --2- 172.21.15.60:0/3862190687 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a4c0cccd0 0x7f6a4c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.970+0000 7f6a51400640 1 --2- 172.21.15.60:0/3862190687 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c145750 0x7f6a4c073440 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.970+0000 7f6a51400640 1 -- 172.21.15.60:0/3862190687 >> 172.21.15.60:0/3862190687 conn(0x7f6a4c0b1840 msgr2=0x7f6a4c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.970+0000 7f6a51400640 1 -- 172.21.15.60:0/3862190687 shutdown_connections 2024-09-16T15:45:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.970+0000 7f6a51400640 1 -- 172.21.15.60:0/3862190687 wait complete. 2024-09-16T15:45:40.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.972+0000 7f6a51400640 1 Processor -- start 2024-09-16T15:45:40.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.972+0000 7f6a51400640 1 -- start start 2024-09-16T15:45:40.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a51400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a4c06efa0 0x7f6a4c0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:40.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a51400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c0cccd0 0x7f6a4c13bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:40.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a51400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a4c145750 0x7f6a4c0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a51400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0ce480 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a51400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0ce300 con 0x7f6a4c06efa0 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a51400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4c0ce600 con 0x7f6a4c145750 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a41e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c0cccd0 0x7f6a4c13bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a41e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c0cccd0 0x7f6a4c13bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33800/0 (socket says 172.21.15.60:33800) 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a41e00640 1 -- 172.21.15.60:0/457965115 learned_addr learned my addr 172.21.15.60:0/457965115 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a4b200640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a4c145750 0x7f6a4c0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.973+0000 7f6a4a800640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a4c06efa0 0x7f6a4c0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a41e00640 1 -- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a4c145750 msgr2=0x7f6a4c0c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a41e00640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a4c145750 0x7f6a4c0c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a41e00640 1 -- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a4c06efa0 msgr2=0x7f6a4c0b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:40.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a41e00640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a4c06efa0 0x7f6a4c0b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a41e00640 1 -- 172.21.15.60:0/457965115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c0b4070 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a4b200640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a4c145750 0x7f6a4c0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a41e00640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c0cccd0 0x7f6a4c13bc90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400c480 tx=0x7f6a3400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a43400640 1 -- 172.21.15.60:0/457965115 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34016020 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a43400640 1 -- 172.21.15.60:0/457965115 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a34012920 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a43400640 1 -- 172.21.15.60:0/457965115 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34012d70 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4c0b4340 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.974+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4c13da50 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.975+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4c0cdbb0 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.977+0000 7f6a43400640 1 -- 172.21.15.60:0/457965115 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f6a34019020 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.977+0000 7f6a43400640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a28078cd0 0x7f6a2807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:40.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.978+0000 7f6a43400640 1 -- 172.21.15.60:0/457965115 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f6a3409b2b0 con 0x7f6a4c0cccd0 2024-09-16T15:45:40.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.978+0000 7f6a4a800640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a28078cd0 0x7f6a2807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:40.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.979+0000 7f6a4a800640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a28078cd0 0x7f6a2807b190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c034000 tx=0x7f6a3c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:40.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:40.982+0000 7f6a43400640 1 -- 172.21.15.60:0/457965115 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6a34020020 con 0x7f6a4c0cccd0 2024-09-16T15:45:41.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.180+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6a4c143a40 con 0x7f6a4c0cccd0 2024-09-16T15:45:41.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.180+0000 7f6a43400640 1 -- 172.21.15.60:0/457965115 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f6a34063180 con 0x7f6a4c0cccd0 2024-09-16T15:45:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a28078cd0 msgr2=0x7f6a2807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a28078cd0 0x7f6a2807b190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c034000 tx=0x7f6a3c004730 comp rx=0 tx=0).stop 2024-09-16T15:45:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c0cccd0 msgr2=0x7f6a4c13bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c0cccd0 0x7f6a4c13bc90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400c480 tx=0x7f6a3400c940 comp rx=0 tx=0).stop 2024-09-16T15:45:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 shutdown_connections 2024-09-16T15:45:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a28078cd0 0x7f6a2807b190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a4c145750 0x7f6a4c0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a4c0cccd0 0x7f6a4c13bc90 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 --2- 172.21.15.60:0/457965115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a4c06efa0 0x7f6a4c0b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 >> 172.21.15.60:0/457965115 conn(0x7f6a4c0b1840 msgr2=0x7f6a4c0cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 shutdown_connections 2024-09-16T15:45:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:41.183+0000 7f6a51400640 1 -- 172.21.15.60:0/457965115 wait complete. 2024-09-16T15:45:41.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.200 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol51620/437c529e-5f9b-4a31-a702-e3a7acc46905) 2024-09-16T15:45:41.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.264 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol51620/437c529e-5f9b-4a31-a702-e3a7acc46905/.snap/fake) 2024-09-16T15:45:41.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.349 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:45:41.386 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:45:41.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.386 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol51620/437c529e-5f9b-4a31-a702-e3a7acc46905) 2024-09-16T15:45:41.486 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:41.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol51620/.meta 2024-09-16T15:45:41.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.546 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp76900/subvol60041/ff58b255-6725-4c1c-ad10-fe7fbb022376) 2024-09-16T15:45:41.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.646 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp76900/subvol60041/ff58b255-6725-4c1c-ad10-fe7fbb022376/.snap/fake) 2024-09-16T15:45:41.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.732 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:45:41.768 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:45:41.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.769 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp76900/subvol60041/ff58b255-6725-4c1c-ad10-fe7fbb022376) 2024-09-16T15:45:41.869 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:41.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp76900/subvol60041/.meta 2024-09-16T15:45:41.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.926 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol52572/49a7a980-559d-4f5d-b470-4eedbbf4ad17) 2024-09-16T15:45:42.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:42.024 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:45:42.060 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:45:42.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:42.060 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol52572/49a7a980-559d-4f5d-b470-4eedbbf4ad17) 2024-09-16T15:45:42.161 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:42.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol52572/.meta 2024-09-16T15:45:42.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol51620 2024-09-16T15:45:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.422+0000 7f335fe00640 1 -- 172.21.15.60:0/3009344349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3360072560 msgr2=0x7f3360073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.422+0000 7f335fe00640 1 --2- 172.21.15.60:0/3009344349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3360072560 0x7f3360073ef0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3358009830 tx=0x7f335802a520 comp rx=0 tx=0).stop 2024-09-16T15:45:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.422+0000 7f335fe00640 1 -- 172.21.15.60:0/3009344349 shutdown_connections 2024-09-16T15:45:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.422+0000 7f335fe00640 1 --2- 172.21.15.60:0/3009344349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3360072560 0x7f3360073ef0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.422+0000 7f335fe00640 1 --2- 172.21.15.60:0/3009344349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33600cbc00 0x7f336006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.422+0000 7f335fe00640 1 -- 172.21.15.60:0/3009344349 >> 172.21.15.60:0/3009344349 conn(0x7f33600cf9f0 msgr2=0x7f33600cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.422+0000 7f335fe00640 1 -- 172.21.15.60:0/3009344349 shutdown_connections 2024-09-16T15:45:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.422+0000 7f335fe00640 1 -- 172.21.15.60:0/3009344349 wait complete. 2024-09-16T15:45:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.424+0000 7f335fe00640 1 Processor -- start 2024-09-16T15:45:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.424+0000 7f335fe00640 1 -- start start 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f335fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3360072560 0x7f33600c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f335fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33600cbc00 0x7f336014da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f335fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336015bec0 0x7f336014df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f335fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f33600ae990 con 0x7f33600cbc00 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f335fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33600ae810 con 0x7f336015bec0 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f335fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f33600aeb10 con 0x7f3360072560 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f3356000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33600cbc00 0x7f336014da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f3356000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33600cbc00 0x7f336014da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33802/0 (socket says 172.21.15.60:33802) 2024-09-16T15:45:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f3356000640 1 -- 172.21.15.60:0/713527996 learned_addr learned my addr 172.21.15.60:0/713527996 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f335f400640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336015bec0 0x7f336014df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.425+0000 7f335ea00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3360072560 0x7f33600c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3356000640 1 -- 172.21.15.60:0/713527996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3360072560 msgr2=0x7f33600c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3356000640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3360072560 0x7f33600c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3356000640 1 -- 172.21.15.60:0/713527996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336015bec0 msgr2=0x7f336014df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3356000640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336015bec0 0x7f336014df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3356000640 1 -- 172.21.15.60:0/713527996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336015d070 con 0x7f33600cbc00 2024-09-16T15:45:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3356000640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33600cbc00 0x7f336014da40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f334800e9f0 tx=0x7f334800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3357400640 1 -- 172.21.15.60:0/713527996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334800ccc0 con 0x7f33600cbc00 2024-09-16T15:45:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3357400640 1 -- 172.21.15.60:0/713527996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3348010430 con 0x7f33600cbc00 2024-09-16T15:45:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f3357400640 1 -- 172.21.15.60:0/713527996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348010840 con 0x7f33600cbc00 2024-09-16T15:45:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.426+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336015d350 con 0x7f33600cbc00 2024-09-16T15:45:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.427+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336015d910 con 0x7f33600cbc00 2024-09-16T15:45:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.428+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33600af030 con 0x7f33600cbc00 2024-09-16T15:45:42.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.429+0000 7f3357400640 1 -- 172.21.15.60:0/713527996 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f33480109e0 con 0x7f33600cbc00 2024-09-16T15:45:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.430+0000 7f3357400640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f333c078cd0 0x7f333c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.431+0000 7f3357400640 1 -- 172.21.15.60:0/713527996 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f334809bcd0 con 0x7f33600cbc00 2024-09-16T15:45:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.431+0000 7f335ea00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f333c078cd0 0x7f333c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:42.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.431+0000 7f335ea00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f333c078cd0 0x7f333c07b190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3350002200 tx=0x7f33500019f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:42.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.435+0000 7f3357400640 1 -- 172.21.15.60:0/713527996 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3348063550 con 0x7f33600cbc00 2024-09-16T15:45:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.662+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol51620", "target": ["mon-mgr", ""]}) -- 0x7f33600cf4b0 con 0x7f333c078cd0 2024-09-16T15:45:42.670 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:42.670+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:42.671 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:42.670+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:42.671 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:42.670+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:42.671 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:42.670+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:42.671 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:42.670+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.716+0000 7f3357400640 1 -- 172.21.15.60:0/713527996 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f33600cf4b0 con 0x7f333c078cd0 2024-09-16T15:45:42.717 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol51620/437c529e-5f9b-4a31-a702-e3a7acc46905 2024-09-16T15:45:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f333c078cd0 msgr2=0x7f333c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f333c078cd0 0x7f333c07b190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3350002200 tx=0x7f33500019f0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33600cbc00 msgr2=0x7f336014da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33600cbc00 0x7f336014da40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f334800e9f0 tx=0x7f334800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 shutdown_connections 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f333c078cd0 0x7f333c07b190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336015bec0 0x7f336014df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33600cbc00 0x7f336014da40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 --2- 172.21.15.60:0/713527996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3360072560 0x7f33600c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 >> 172.21.15.60:0/713527996 conn(0x7f33600cf9f0 msgr2=0x7f33600af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.718+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 shutdown_connections 2024-09-16T15:45:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.719+0000 7f335fe00640 1 -- 172.21.15.60:0/713527996 wait complete. 2024-09-16T15:45:42.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol60041 subvol_grp76900 2024-09-16T15:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.899+0000 7f663dc00640 1 -- 172.21.15.60:0/959777991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f663806f300 msgr2=0x7f66380b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.899+0000 7f663dc00640 1 --2- 172.21.15.60:0/959777991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f663806f300 0x7f66380b4770 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6630005fd0 tx=0x7f663002a940 comp rx=0 tx=0).stop 2024-09-16T15:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.899+0000 7f663dc00640 1 -- 172.21.15.60:0/959777991 shutdown_connections 2024-09-16T15:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.899+0000 7f663dc00640 1 --2- 172.21.15.60:0/959777991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f663806f300 0x7f66380b4770 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.899+0000 7f663dc00640 1 -- 172.21.15.60:0/959777991 >> 172.21.15.60:0/959777991 conn(0x7f66380b18e0 msgr2=0x7f66380b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.899+0000 7f663dc00640 1 -- 172.21.15.60:0/959777991 shutdown_connections 2024-09-16T15:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.899+0000 7f663dc00640 1 -- 172.21.15.60:0/959777991 wait complete. 2024-09-16T15:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.901+0000 7f663dc00640 1 Processor -- start 2024-09-16T15:45:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.901+0000 7f663dc00640 1 -- start start 2024-09-16T15:45:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.901+0000 7f663dc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f663806ee00 0x7f66381f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f663dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663806f300 0x7f663809c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f663dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f663815dfc0 0x7f663809cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f663dc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f66380ae250 con 0x7f663806f300 2024-09-16T15:45:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f663dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66380ae0d0 con 0x7f663815dfc0 2024-09-16T15:45:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f663dc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f66380ae3d0 con 0x7f663806ee00 2024-09-16T15:45:42.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f662e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663806f300 0x7f663809c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f662e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663806f300 0x7f663809c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33824/0 (socket says 172.21.15.60:33824) 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f662e000640 1 -- 172.21.15.60:0/2002119337 learned_addr learned my addr 172.21.15.60:0/2002119337 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f6637400640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f663815dfc0 0x7f663809cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f6636a00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f663806ee00 0x7f66381f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f662e000640 1 -- 172.21.15.60:0/2002119337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f663806ee00 msgr2=0x7f66381f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f662e000640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f663806ee00 0x7f66381f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f662e000640 1 -- 172.21.15.60:0/2002119337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f663815dfc0 msgr2=0x7f663809cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f662e000640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f663815dfc0 0x7f663809cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:42.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.902+0000 7f662e000640 1 -- 172.21.15.60:0/2002119337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663815d810 con 0x7f663806f300 2024-09-16T15:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.903+0000 7f662e000640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663806f300 0x7f663809c650 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f662000c490 tx=0x7f662000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.903+0000 7f662f400640 1 -- 172.21.15.60:0/2002119337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620016020 con 0x7f663806f300 2024-09-16T15:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.903+0000 7f662f400640 1 -- 172.21.15.60:0/2002119337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6620012a00 con 0x7f663806f300 2024-09-16T15:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.903+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663809ab30 con 0x7f663806f300 2024-09-16T15:45:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.903+0000 7f662f400640 1 -- 172.21.15.60:0/2002119337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620011380 con 0x7f663806f300 2024-09-16T15:45:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.903+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663809b0b0 con 0x7f663806f300 2024-09-16T15:45:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.904+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66380ae880 con 0x7f663806f300 2024-09-16T15:45:42.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.905+0000 7f662f400640 1 -- 172.21.15.60:0/2002119337 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f6620019020 con 0x7f663806f300 2024-09-16T15:45:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.906+0000 7f662f400640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6614078cd0 0x7f661407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.906+0000 7f662f400640 1 -- 172.21.15.60:0/2002119337 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f662009ace0 con 0x7f663806f300 2024-09-16T15:45:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.906+0000 7f6636a00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6614078cd0 0x7f661407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:42.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.907+0000 7f6636a00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6614078cd0 0x7f661407b190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6628007fd0 tx=0x7f6628009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:42.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:42.910+0000 7f662f400640 1 -- 172.21.15.60:0/2002119337 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6620020070 con 0x7f663806f300 2024-09-16T15:45:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.133+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol60041", "group_name": "subvol_grp76900", "target": ["mon-mgr", ""]}) -- 0x7f66380ad920 con 0x7f6614078cd0 2024-09-16T15:45:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.169+0000 7f662f400640 1 -- 172.21.15.60:0/2002119337 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7f66380ad920 con 0x7f6614078cd0 2024-09-16T15:45:43.170 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/subvol_grp76900/subvol60041/ff58b255-6725-4c1c-ad10-fe7fbb022376 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.171+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6614078cd0 msgr2=0x7f661407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.171+0000 7f663dc00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6614078cd0 0x7f661407b190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6628007fd0 tx=0x7f6628009040 comp rx=0 tx=0).stop 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.171+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663806f300 msgr2=0x7f663809c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.171+0000 7f663dc00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663806f300 0x7f663809c650 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f662000c490 tx=0x7f662000c950 comp rx=0 tx=0).stop 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.172+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 shutdown_connections 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.172+0000 7f663dc00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6614078cd0 0x7f661407b190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.172+0000 7f663dc00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f663815dfc0 0x7f663809cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.172+0000 7f663dc00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663806f300 0x7f663809c650 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.172+0000 7f663dc00640 1 --2- 172.21.15.60:0/2002119337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f663806ee00 0x7f66381f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.172+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 >> 172.21.15.60:0/2002119337 conn(0x7f66380b18e0 msgr2=0x7f66380b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.172+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 shutdown_connections 2024-09-16T15:45:43.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.172+0000 7f663dc00640 1 -- 172.21.15.60:0/2002119337 wait complete. 2024-09-16T15:45:43.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol52572 2024-09-16T15:45:43.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.352+0000 7f174f000640 1 -- 172.21.15.60:0/3379365952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17480cbc00 msgr2=0x7f1748073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.352+0000 7f174f000640 1 --2- 172.21.15.60:0/3379365952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17480cbc00 0x7f1748073ef0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f17480c9ff0 tx=0x7f174002aac0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.352+0000 7f174f000640 1 -- 172.21.15.60:0/3379365952 shutdown_connections 2024-09-16T15:45:43.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.352+0000 7f174f000640 1 --2- 172.21.15.60:0/3379365952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17480cbc00 0x7f1748073ef0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.352+0000 7f174f000640 1 --2- 172.21.15.60:0/3379365952 >> v2:172.21.15.100:3300/0 conn(0x7f174815bec0 0x7f174806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.352+0000 7f174f000640 1 -- 172.21.15.60:0/3379365952 >> 172.21.15.60:0/3379365952 conn(0x7f17480cf9f0 msgr2=0x7f17480cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:43.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.352+0000 7f174f000640 1 -- 172.21.15.60:0/3379365952 shutdown_connections 2024-09-16T15:45:43.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.352+0000 7f174f000640 1 -- 172.21.15.60:0/3379365952 wait complete. 2024-09-16T15:45:43.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.354+0000 7f174f000640 1 Processor -- start 2024-09-16T15:45:43.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.354+0000 7f174f000640 1 -- start start 2024-09-16T15:45:43.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1748072560 0x7f17481f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:43.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174f000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17480cbc00 0x7f174814cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:43.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174f000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f174815bec0 0x7f174814d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:43.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174f000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f17480ae990 con 0x7f174815bec0 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f17480ae810 con 0x7f1748072560 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174f000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f17480aeb10 con 0x7f17480cbc00 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f174815bec0 0x7f174814d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f174815bec0 0x7f174814d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33840/0 (socket says 172.21.15.60:33840) 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174e600640 1 -- 172.21.15.60:0/1426415865 learned_addr learned my addr 172.21.15.60:0/1426415865 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174dc00640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1748072560 0x7f17481f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174d200640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17480cbc00 0x7f174814cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174e600640 1 -- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17480cbc00 msgr2=0x7f174814cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174e600640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17480cbc00 0x7f174814cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174e600640 1 -- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1748072560 msgr2=0x7f17481f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174e600640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1748072560 0x7f17481f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174e600640 1 -- 172.21.15.60:0/1426415865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174815a570 con 0x7f174815bec0 2024-09-16T15:45:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.355+0000 7f174dc00640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1748072560 0x7f17481f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:43.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.356+0000 7f174e600640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f174815bec0 0x7f174814d1d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f1740009d60 tx=0x7f1740003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:43.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.356+0000 7f1745600640 1 -- 172.21.15.60:0/1426415865 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740044070 con 0x7f174815bec0 2024-09-16T15:45:43.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.356+0000 7f1745600640 1 -- 172.21.15.60:0/1426415865 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1740003e50 con 0x7f174815bec0 2024-09-16T15:45:43.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.356+0000 7f1745600640 1 -- 172.21.15.60:0/1426415865 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17400044b0 con 0x7f174815bec0 2024-09-16T15:45:43.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.356+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17481f7ff0 con 0x7f174815bec0 2024-09-16T15:45:43.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.356+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17480aad40 con 0x7f174815bec0 2024-09-16T15:45:43.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.357+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17480aefc0 con 0x7f174815bec0 2024-09-16T15:45:43.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.358+0000 7f1745600640 1 -- 172.21.15.60:0/1426415865 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f1740004650 con 0x7f174815bec0 2024-09-16T15:45:43.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.359+0000 7f1745600640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1728078d00 0x7f172807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:43.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.359+0000 7f1745600640 1 -- 172.21.15.60:0/1426415865 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f17400bb8a0 con 0x7f174815bec0 2024-09-16T15:45:43.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.359+0000 7f174dc00640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1728078d00 0x7f172807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:43.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.360+0000 7f174dc00640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1728078d00 0x7f172807b1c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f173c009b80 tx=0x7f173c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:43.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.363+0000 7f1745600640 1 -- 172.21.15.60:0/1426415865 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f17481f7ff0 con 0x7f174815bec0 2024-09-16T15:45:43.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.597+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol52572", "target": ["mon-mgr", ""]}) -- 0x7f17480b0d80 con 0x7f1728078d00 2024-09-16T15:45:43.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.628+0000 7f1745600640 1 -- 172.21.15.60:0/1426415865 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f17480b0d80 con 0x7f1728078d00 2024-09-16T15:45:43.629 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:45:43.629 INFO:teuthology.orchestra.run.smithi060.stdout: "status": { 2024-09-16T15:45:43.629 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "pending", 2024-09-16T15:45:43.629 INFO:teuthology.orchestra.run.smithi060.stdout: "source": { 2024-09-16T15:45:43.629 INFO:teuthology.orchestra.run.smithi060.stdout: "volume": "fake", 2024-09-16T15:45:43.629 INFO:teuthology.orchestra.run.smithi060.stdout: "subvolume": "fake", 2024-09-16T15:45:43.630 INFO:teuthology.orchestra.run.smithi060.stdout: "snapshot": "fake" 2024-09-16T15:45:43.630 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:43.630 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:45:43.630 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1728078d00 msgr2=0x7f172807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1728078d00 0x7f172807b1c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f173c009b80 tx=0x7f173c005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f174815bec0 msgr2=0x7f174814d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f174815bec0 0x7f174814d1d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f1740009d60 tx=0x7f1740003ce0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 shutdown_connections 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1728078d00 0x7f172807b1c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f174815bec0 0x7f174814d1d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17480cbc00 0x7f174814cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 --2- 172.21.15.60:0/1426415865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1748072560 0x7f17481f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.630+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 >> 172.21.15.60:0/1426415865 conn(0x7f17480cf9f0 msgr2=0x7f17480cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:43.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.631+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 shutdown_connections 2024-09-16T15:45:43.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.631+0000 7f174f000640 1 -- 172.21.15.60:0/1426415865 wait complete. 2024-09-16T15:45:43.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol51620 fake 2024-09-16T15:45:43.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.814+0000 7f7f32000640 1 -- 172.21.15.60:0/4277463317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0d2930 msgr2=0x7f7f2c0d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.814+0000 7f7f32000640 1 --2- 172.21.15.60:0/4277463317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0d2930 0x7f7f2c0d0650 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009830 tx=0x7f7f1402a480 comp rx=0 tx=0).stop 2024-09-16T15:45:43.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.814+0000 7f7f32000640 1 -- 172.21.15.60:0/4277463317 shutdown_connections 2024-09-16T15:45:43.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.814+0000 7f7f32000640 1 --2- 172.21.15.60:0/4277463317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c0d2930 0x7f7f2c0d0650 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.814+0000 7f7f32000640 1 --2- 172.21.15.60:0/4277463317 >> v2:172.21.15.100:3300/0 conn(0x7f7f2c15df90 0x7f7f2c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.815+0000 7f7f32000640 1 -- 172.21.15.60:0/4277463317 >> 172.21.15.60:0/4277463317 conn(0x7f7f2c0d3ea0 msgr2=0x7f7f2c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:43.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.815+0000 7f7f32000640 1 -- 172.21.15.60:0/4277463317 shutdown_connections 2024-09-16T15:45:43.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.815+0000 7f7f32000640 1 -- 172.21.15.60:0/4277463317 wait complete. 2024-09-16T15:45:43.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.816+0000 7f7f32000640 1 Processor -- start 2024-09-16T15:45:43.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.816+0000 7f7f32000640 1 -- start start 2024-09-16T15:45:43.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f32000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2c0d2930 0x7f7f2c0a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:43.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f32000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c0d49c0 0x7f7f2c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:43.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f32000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15df90 0x7f7f2c1f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:43.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f32000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0d0d10 con 0x7f7f2c0d2930 2024-09-16T15:45:43.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f32000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0d0b90 con 0x7f7f2c0d49c0 2024-09-16T15:45:43.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f32000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c0d0e90 con 0x7f7f2c15df90 2024-09-16T15:45:43.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f30c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2c0d2930 0x7f7f2c0a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:43.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f30c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2c0d2930 0x7f7f2c0a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33852/0 (socket says 172.21.15.60:33852) 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.817+0000 7f7f30c00640 1 -- 172.21.15.60:0/1138130467 learned_addr learned my addr 172.21.15.60:0/1138130467 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f23e00640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c0d49c0 0x7f7f2c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f31600640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15df90 0x7f7f2c1f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f30c00640 1 -- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15df90 msgr2=0x7f7f2c1f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f30c00640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15df90 0x7f7f2c1f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f30c00640 1 -- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c0d49c0 msgr2=0x7f7f2c15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f30c00640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c0d49c0 0x7f7f2c15a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f30c00640 1 -- 172.21.15.60:0/1138130467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c15af90 con 0x7f7f2c0d2930 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f31600640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15df90 0x7f7f2c1f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:45:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f30c00640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2c0d2930 0x7f7f2c0a1380 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00e790 tx=0x7f7f1c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:43.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f29600640 1 -- 172.21.15.60:0/1138130467 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00eea0 con 0x7f7f2c0d2930 2024-09-16T15:45:43.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f29600640 1 -- 172.21.15.60:0/1138130467 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f1c00cdb0 con 0x7f7f2c0d2930 2024-09-16T15:45:43.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.818+0000 7f7f29600640 1 -- 172.21.15.60:0/1138130467 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c010690 con 0x7f7f2c0d2930 2024-09-16T15:45:43.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.819+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c15b270 con 0x7f7f2c0d2930 2024-09-16T15:45:43.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.819+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c148fa0 con 0x7f7f2c0d2930 2024-09-16T15:45:43.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.820+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f2c0d0650 con 0x7f7f2c0d2930 2024-09-16T15:45:43.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.821+0000 7f7f29600640 1 -- 172.21.15.60:0/1138130467 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f7f1c010830 con 0x7f7f2c0d2930 2024-09-16T15:45:43.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.822+0000 7f7f29600640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7f08078cd0 0x7f7f0807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:43.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.822+0000 7f7f29600640 1 -- 172.21.15.60:0/1138130467 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f7f1c09bc40 con 0x7f7f2c0d2930 2024-09-16T15:45:43.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.823+0000 7f7f23e00640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7f08078cd0 0x7f7f0807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:43.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.823+0000 7f7f23e00640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7f08078cd0 0x7f7f0807b190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f14034000 tx=0x7f7f14004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:43.827+0000 7f7f29600640 1 -- 172.21.15.60:0/1138130467 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7f1c063b10 con 0x7f7f2c0d2930 2024-09-16T15:45:44.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.061+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol51620", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f7f2c0d1450 con 0x7f7f08078cd0 2024-09-16T15:45:44.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.115+0000 7f7f29600640 1 -- 172.21.15.60:0/1138130467 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f2c0d1450 con 0x7f7f08078cd0 2024-09-16T15:45:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7f08078cd0 msgr2=0x7f7f0807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7f08078cd0 0x7f7f0807b190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f14034000 tx=0x7f7f14004730 comp rx=0 tx=0).stop 2024-09-16T15:45:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2c0d2930 msgr2=0x7f7f2c0a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2c0d2930 0x7f7f2c0a1380 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00e790 tx=0x7f7f1c00ec50 comp rx=0 tx=0).stop 2024-09-16T15:45:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 shutdown_connections 2024-09-16T15:45:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7f08078cd0 0x7f7f0807b190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f2c15df90 0x7f7f2c1f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f2c0d49c0 0x7f7f2c15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 --2- 172.21.15.60:0/1138130467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f2c0d2930 0x7f7f2c0a1380 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 >> 172.21.15.60:0/1138130467 conn(0x7f7f2c0d3ea0 msgr2=0x7f7f2c0b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 shutdown_connections 2024-09-16T15:45:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.118+0000 7f7f32000640 1 -- 172.21.15.60:0/1138130467 wait complete. 2024-09-16T15:45:44.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol60041 fake subvol_grp76900 2024-09-16T15:45:44.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.301+0000 7f0872e00640 1 -- 172.21.15.60:0/477520844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0854004800 msgr2=0x7f0854004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.301+0000 7f0872e00640 1 --2- 172.21.15.60:0/477520844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0854004800 0x7f0854004c80 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0858001830 tx=0x7f0854005830 comp rx=0 tx=0).stop 2024-09-16T15:45:44.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.301+0000 7f0872e00640 1 -- 172.21.15.60:0/477520844 shutdown_connections 2024-09-16T15:45:44.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.301+0000 7f0872e00640 1 --2- 172.21.15.60:0/477520844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0854004800 0x7f0854004c80 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.301+0000 7f0872e00640 1 -- 172.21.15.60:0/477520844 >> 172.21.15.60:0/477520844 conn(0x7f086c0d3e60 msgr2=0x7f086c0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:44.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.301+0000 7f0872e00640 1 -- 172.21.15.60:0/477520844 shutdown_connections 2024-09-16T15:45:44.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.301+0000 7f0872e00640 1 -- 172.21.15.60:0/477520844 wait complete. 2024-09-16T15:45:44.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.303+0000 7f0872e00640 1 Processor -- start 2024-09-16T15:45:44.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.303+0000 7f0872e00640 1 -- start start 2024-09-16T15:45:44.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0854004800 0x7f086c09db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:44.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086c15d370 0x7f086c0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:44.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f086c0cd210 0x7f086c0a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:44.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f086c0d0fe0 con 0x7f086c0cd210 2024-09-16T15:45:44.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f086c0d0e60 con 0x7f0854004800 2024-09-16T15:45:44.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f086c0d1160 con 0x7f086c15d370 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f086c0cd210 0x7f086c0a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f086c0cd210 0x7f086c0a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33876/0 (socket says 172.21.15.60:33876) 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0872400640 1 -- 172.21.15.60:0/3022449207 learned_addr learned my addr 172.21.15.60:0/3022449207 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0871a00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0854004800 0x7f086c09db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.304+0000 7f0871000640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086c15d370 0x7f086c0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0872400640 1 -- 172.21.15.60:0/3022449207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086c15d370 msgr2=0x7f086c0cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0872400640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086c15d370 0x7f086c0cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0872400640 1 -- 172.21.15.60:0/3022449207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0854004800 msgr2=0x7f086c09db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0872400640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0854004800 0x7f086c09db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0872400640 1 -- 172.21.15.60:0/3022449207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c09e090 con 0x7f086c0cd210 2024-09-16T15:45:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0872400640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f086c0cd210 0x7f086c0a0f00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0864005fd0 tx=0x7f086400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0869600640 1 -- 172.21.15.60:0/3022449207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864003d80 con 0x7f086c0cd210 2024-09-16T15:45:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0869600640 1 -- 172.21.15.60:0/3022449207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0864004b20 con 0x7f086c0cd210 2024-09-16T15:45:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0869600640 1 -- 172.21.15.60:0/3022449207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864004050 con 0x7f086c0cd210 2024-09-16T15:45:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086c0cb0d0 con 0x7f086c0cd210 2024-09-16T15:45:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.305+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086c0cb690 con 0x7f086c0cd210 2024-09-16T15:45:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.306+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086c0cf2d0 con 0x7f086c0cd210 2024-09-16T15:45:44.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.307+0000 7f0869600640 1 -- 172.21.15.60:0/3022449207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f08640041f0 con 0x7f086c0cd210 2024-09-16T15:45:44.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.308+0000 7f0869600640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f084c078cd0 0x7f084c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:44.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.309+0000 7f0869600640 1 -- 172.21.15.60:0/3022449207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f086409c3a0 con 0x7f086c0cd210 2024-09-16T15:45:44.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.309+0000 7f0871a00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f084c078cd0 0x7f084c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:44.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.309+0000 7f0871a00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f084c078cd0 0x7f084c07b190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0860001cb0 tx=0x7f0860001550 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:44.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.313+0000 7f0869600640 1 -- 172.21.15.60:0/3022449207 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f08640641e0 con 0x7f086c0cd210 2024-09-16T15:45:44.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.546+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol60041", "snap_name": "fake", "group_name": "subvol_grp76900", "target": ["mon-mgr", ""]}) -- 0x7f086c0d0650 con 0x7f084c078cd0 2024-09-16T15:45:44.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.589+0000 7f0869600640 1 -- 172.21.15.60:0/3022449207 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f086c0d0650 con 0x7f084c078cd0 2024-09-16T15:45:44.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f084c078cd0 msgr2=0x7f084c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f084c078cd0 0x7f084c07b190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0860001cb0 tx=0x7f0860001550 comp rx=0 tx=0).stop 2024-09-16T15:45:44.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f086c0cd210 msgr2=0x7f086c0a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f086c0cd210 0x7f086c0a0f00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0864005fd0 tx=0x7f086400f790 comp rx=0 tx=0).stop 2024-09-16T15:45:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 shutdown_connections 2024-09-16T15:45:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f084c078cd0 0x7f084c07b190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f086c0cd210 0x7f086c0a0f00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f086c15d370 0x7f086c0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 --2- 172.21.15.60:0/3022449207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0854004800 0x7f086c09db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 >> 172.21.15.60:0/3022449207 conn(0x7f086c0d3e60 msgr2=0x7f086c160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 shutdown_connections 2024-09-16T15:45:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.592+0000 7f0872e00640 1 -- 172.21.15.60:0/3022449207 wait complete. 2024-09-16T15:45:44.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:44.609 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol51620/.meta) 2024-09-16T15:45:44.666 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:45:44.666 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:45:44.666 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:45:44.666 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/_nogroup/subvol51620/437c529e-5f9b-4a31-a702-e3a7acc46905 2024-09-16T15:45:44.666 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:45:44.666 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:44.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:44.669 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp76900/subvol60041/.meta) 2024-09-16T15:45:44.724 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:45:44.725 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:45:44.725 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:45:44.725 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/subvol_grp76900/subvol60041/ff58b255-6725-4c1c-ad10-fe7fbb022376 2024-09-16T15:45:44.725 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:45:44.725 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:44.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol51620 2024-09-16T15:45:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.892+0000 7f3b5a400640 1 -- 172.21.15.60:0/3011431963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b3c004d40 msgr2=0x7f3b3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.892+0000 7f3b5a400640 1 --2- 172.21.15.60:0/3011431963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b3c004d40 0x7f3b3c0051c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009ed0 tx=0x7f3b3c005830 comp rx=0 tx=0).stop 2024-09-16T15:45:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.892+0000 7f3b59a00640 1 -- 172.21.15.60:0/3011431963 reap_dead start 2024-09-16T15:45:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.892+0000 7f3b5a400640 1 -- 172.21.15.60:0/3011431963 shutdown_connections 2024-09-16T15:45:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.892+0000 7f3b5a400640 1 -- 172.21.15.60:0/3011431963 >> 172.21.15.60:0/3011431963 conn(0x7f3b540d3e60 msgr2=0x7f3b540d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.892+0000 7f3b5a400640 1 -- 172.21.15.60:0/3011431963 shutdown_connections 2024-09-16T15:45:44.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.892+0000 7f3b5a400640 1 -- 172.21.15.60:0/3011431963 wait complete. 2024-09-16T15:45:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.893+0000 7f3b5a400640 1 Processor -- start 2024-09-16T15:45:44.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.894+0000 7f3b5a400640 1 -- start start 2024-09-16T15:45:44.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.894+0000 7f3b5a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3c004d40 0x7f3b541f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:44.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b5a400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b5415b1c0 0x7f3b540a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b5a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541f16c0 0x7f3b5406e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b5a400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b540cecc0 con 0x7f3b541f16c0 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b5a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3b540ceb40 con 0x7f3b3c004d40 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b5a400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3b540a1a30 con 0x7f3b5415b1c0 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541f16c0 0x7f3b5406e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3c004d40 0x7f3b541f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541f16c0 0x7f3b5406e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33896/0 (socket says 172.21.15.60:33896) 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59a00640 1 -- 172.21.15.60:0/723608678 learned_addr learned my addr 172.21.15.60:0/723608678 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b53e00640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b5415b1c0 0x7f3b540a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59a00640 1 -- 172.21.15.60:0/723608678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b5415b1c0 msgr2=0x7f3b540a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59a00640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b5415b1c0 0x7f3b540a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59a00640 1 -- 172.21.15.60:0/723608678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3c004d40 msgr2=0x7f3b541f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:44.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59a00640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3c004d40 0x7f3b541f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:44.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.895+0000 7f3b59a00640 1 -- 172.21.15.60:0/723608678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b540a1480 con 0x7f3b541f16c0 2024-09-16T15:45:44.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.896+0000 7f3b59a00640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541f16c0 0x7f3b5406e3b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c02af20 tx=0x7f3b4c034dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:44.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.896+0000 7f3b50c00640 1 -- 172.21.15.60:0/723608678 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c043070 con 0x7f3b541f16c0 2024-09-16T15:45:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.896+0000 7f3b50c00640 1 -- 172.21.15.60:0/723608678 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b4c003c60 con 0x7f3b541f16c0 2024-09-16T15:45:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.896+0000 7f3b50c00640 1 -- 172.21.15.60:0/723608678 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c038040 con 0x7f3b541f16c0 2024-09-16T15:45:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.896+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b540a1750 con 0x7f3b541f16c0 2024-09-16T15:45:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.896+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b541f7c40 con 0x7f3b541f16c0 2024-09-16T15:45:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.897+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b540c7dd0 con 0x7f3b541f16c0 2024-09-16T15:45:44.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.899+0000 7f3b50c00640 1 -- 172.21.15.60:0/723608678 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f3b4c046070 con 0x7f3b541f16c0 2024-09-16T15:45:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.899+0000 7f3b50c00640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b34078cd0 0x7f3b3407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.900+0000 7f3b50c00640 1 -- 172.21.15.60:0/723608678 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f3b4c0bb3b0 con 0x7f3b541f16c0 2024-09-16T15:45:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.900+0000 7f3b59000640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b34078cd0 0x7f3b3407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.900+0000 7f3b59000640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b34078cd0 0x7f3b3407b190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009b50 tx=0x7f3b48005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:44.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:44.904+0000 7f3b50c00640 1 -- 172.21.15.60:0/723608678 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b4c082b10 con 0x7f3b541f16c0 2024-09-16T15:45:45.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.128+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol51620", "target": ["mon-mgr", ""]}) -- 0x7f3b5406d6c0 con 0x7f3b34078cd0 2024-09-16T15:45:45.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.166+0000 7f3b50c00640 1 -- 172.21.15.60:0/723608678 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b5406d6c0 con 0x7f3b34078cd0 2024-09-16T15:45:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b34078cd0 msgr2=0x7f3b3407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b34078cd0 0x7f3b3407b190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009b50 tx=0x7f3b48005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541f16c0 msgr2=0x7f3b5406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541f16c0 0x7f3b5406e3b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c02af20 tx=0x7f3b4c034dd0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 shutdown_connections 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b34078cd0 0x7f3b3407b190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b541f16c0 0x7f3b5406e3b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b5415b1c0 0x7f3b540a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 --2- 172.21.15.60:0/723608678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b3c004d40 0x7f3b541f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 >> 172.21.15.60:0/723608678 conn(0x7f3b540d3e60 msgr2=0x7f3b5415ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 shutdown_connections 2024-09-16T15:45:45.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.168+0000 7f3b5a400640 1 -- 172.21.15.60:0/723608678 wait complete. 2024-09-16T15:45:45.174 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.173+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.174 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.173+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.174 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.173+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.174 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.173+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.174 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.173+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.188 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol60041 subvol_grp76900 2024-09-16T15:45:45.193 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.191+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.193 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.191+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.193 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.191+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.193 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.191+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.193 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:45.191+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.356+0000 7f8153e00640 1 -- 172.21.15.60:0/4196226197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c06faf0 msgr2=0x7f814c15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.356+0000 7f8153e00640 1 --2- 172.21.15.60:0/4196226197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c06faf0 0x7f814c15e490 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f814c06aa00 tx=0x7f814402aac0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.357+0000 7f8153e00640 1 -- 172.21.15.60:0/4196226197 shutdown_connections 2024-09-16T15:45:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.357+0000 7f8153e00640 1 --2- 172.21.15.60:0/4196226197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c06faf0 0x7f814c15e490 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.357+0000 7f8153e00640 1 --2- 172.21.15.60:0/4196226197 >> v2:172.21.15.100:3300/0 conn(0x7f814c1550c0 0x7f814c0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.357+0000 7f8153e00640 1 -- 172.21.15.60:0/4196226197 >> 172.21.15.60:0/4196226197 conn(0x7f814c0b18e0 msgr2=0x7f814c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:45.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.357+0000 7f8153e00640 1 -- 172.21.15.60:0/4196226197 shutdown_connections 2024-09-16T15:45:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.357+0000 7f8153e00640 1 -- 172.21.15.60:0/4196226197 wait complete. 2024-09-16T15:45:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.358+0000 7f8153e00640 1 Processor -- start 2024-09-16T15:45:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.358+0000 7f8153e00640 1 -- start start 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.359+0000 7f8153e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f814c06faf0 0x7f814c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.359+0000 7f8153e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c0727c0 0x7f814c1468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.359+0000 7f8153e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c1550c0 0x7f814c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.359+0000 7f8153e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f814c15eb50 con 0x7f814c0727c0 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.359+0000 7f8153e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f814c15e9d0 con 0x7f814c1550c0 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.359+0000 7f8153e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f814c15ecd0 con 0x7f814c06faf0 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c0727c0 0x7f814c1468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c0727c0 0x7f814c1468e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33974/0 (socket says 172.21.15.60:33974) 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 -- 172.21.15.60:0/1165184685 learned_addr learned my addr 172.21.15.60:0/1165184685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8151400640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f814c06faf0 0x7f814c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8151e00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c1550c0 0x7f814c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 -- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f814c06faf0 msgr2=0x7f814c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f814c06faf0 0x7f814c161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 -- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c1550c0 msgr2=0x7f814c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c1550c0 0x7f814c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 -- 172.21.15.60:0/1165184685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c146e20 con 0x7f814c0727c0 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8151e00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c1550c0 0x7f814c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.360+0000 7f8150a00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c0727c0 0x7f814c1468e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f813800c500 tx=0x7f813800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.361+0000 7f8149600640 1 -- 172.21.15.60:0/1165184685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138017070 con 0x7f814c0727c0 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.361+0000 7f8149600640 1 -- 172.21.15.60:0/1165184685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81380035f0 con 0x7f814c0727c0 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.361+0000 7f8149600640 1 -- 172.21.15.60:0/1165184685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138003a40 con 0x7f814c0727c0 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.361+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c1f7b50 con 0x7f814c0727c0 2024-09-16T15:45:45.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.361+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1f80d0 con 0x7f814c0727c0 2024-09-16T15:45:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.362+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c069360 con 0x7f814c0727c0 2024-09-16T15:45:45.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.363+0000 7f8149600640 1 -- 172.21.15.60:0/1165184685 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 19) ==== 101695+0+0 (secure 0 0 0) 0x7f813801a070 con 0x7f814c0727c0 2024-09-16T15:45:45.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.364+0000 7f8149600640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f812c078cd0 0x7f812c07b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:45.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.364+0000 7f8149600640 1 -- 172.21.15.60:0/1165184685 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5737+0+0 (secure 0 0 0) 0x7f813809a710 con 0x7f814c0727c0 2024-09-16T15:45:45.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.364+0000 7f8151400640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f812c078cd0 0x7f812c07b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:45.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.365+0000 7f8151400640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f812c078cd0 0x7f812c07b190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8140009ac0 tx=0x7f8140005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:45.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.369+0000 7f8149600640 1 -- 172.21.15.60:0/1165184685 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f814c146e20 con 0x7f814c0727c0 2024-09-16T15:45:45.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.404+0000 7f8149600640 1 -- 172.21.15.60:0/1165184685 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 20) ==== 101746+0+0 (secure 0 0 0) 0x7f813805e1b0 con 0x7f814c0727c0 2024-09-16T15:45:45.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.597+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol60041", "group_name": "subvol_grp76900", "target": ["mon-mgr", ""]}) -- 0x7f814c15f1f0 con 0x7f812c078cd0 2024-09-16T15:45:45.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.640+0000 7f8149600640 1 -- 172.21.15.60:0/1165184685 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f814c15f1f0 con 0x7f812c078cd0 2024-09-16T15:45:45.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f812c078cd0 msgr2=0x7f812c07b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f812c078cd0 0x7f812c07b190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8140009ac0 tx=0x7f8140005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:45.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c0727c0 msgr2=0x7f814c1468e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c0727c0 0x7f814c1468e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f813800c500 tx=0x7f813800c9c0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 shutdown_connections 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f812c078cd0 0x7f812c07b190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c1550c0 0x7f814c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f814c0727c0 0x7f814c1468e0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 --2- 172.21.15.60:0/1165184685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f814c06faf0 0x7f814c161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 >> 172.21.15.60:0/1165184685 conn(0x7f814c0b18e0 msgr2=0x7f814c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 shutdown_connections 2024-09-16T15:45:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.642+0000 7f8153e00640 1 -- 172.21.15.60:0/1165184685 wait complete. 2024-09-16T15:45:45.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol52572 2024-09-16T15:45:45.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.829+0000 7fd7a7e00640 1 -- 172.21.15.60:0/3403356300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a00cccd0 msgr2=0x7fd7a00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.829+0000 7fd7a7e00640 1 --2- 172.21.15.60:0/3403356300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a00cccd0 0x7fd7a00cdbb0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd798005fd0 tx=0x7fd79802a970 comp rx=0 tx=0).stop 2024-09-16T15:45:45.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.829+0000 7fd7a7e00640 1 -- 172.21.15.60:0/3403356300 shutdown_connections 2024-09-16T15:45:45.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.829+0000 7fd7a7e00640 1 --2- 172.21.15.60:0/3403356300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a00cccd0 0x7fd7a00cdbb0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.829+0000 7fd7a7e00640 1 --2- 172.21.15.60:0/3403356300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7a0145750 0x7fd7a0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.829+0000 7fd7a7e00640 1 -- 172.21.15.60:0/3403356300 >> 172.21.15.60:0/3403356300 conn(0x7fd7a00b1840 msgr2=0x7fd7a00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:45.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.829+0000 7fd7a7e00640 1 -- 172.21.15.60:0/3403356300 shutdown_connections 2024-09-16T15:45:45.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.829+0000 7fd7a7e00640 1 -- 172.21.15.60:0/3403356300 wait complete. 2024-09-16T15:45:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.831+0000 7fd7a7e00640 1 Processor -- start 2024-09-16T15:45:45.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.831+0000 7fd7a7e00640 1 -- start start 2024-09-16T15:45:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7a006efa0 0x7fd7a013d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a7e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7a00cccd0 0x7fd7a013bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a0145750 0x7fd7a01f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a7e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a00ce480 con 0x7fd7a0145750 2024-09-16T15:45:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a00ce300 con 0x7fd7a006efa0 2024-09-16T15:45:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a7e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a00ce600 con 0x7fd7a00cccd0 2024-09-16T15:45:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a0145750 0x7fd7a01f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a0145750 0x7fd7a01f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33994/0 (socket says 172.21.15.60:33994) 2024-09-16T15:45:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a5e00640 1 -- 172.21.15.60:0/2779568872 learned_addr learned my addr 172.21.15.60:0/2779568872 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a5400640 1 --2- 172.21.15.60:0/2779568872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7a006efa0 0x7fd7a013d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.832+0000 7fd7a4a00640 1 --2- 172.21.15.60:0/2779568872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7a00cccd0 0x7fd7a013bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd7a5e00640 1 -- 172.21.15.60:0/2779568872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7a00cccd0 msgr2=0x7fd7a013bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd7a5e00640 1 --2- 172.21.15.60:0/2779568872 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7a00cccd0 0x7fd7a013bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd7a5e00640 1 -- 172.21.15.60:0/2779568872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7a006efa0 msgr2=0x7fd7a013d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd7a5e00640 1 --2- 172.21.15.60:0/2779568872 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7a006efa0 0x7fd7a013d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd7a5e00640 1 -- 172.21.15.60:0/2779568872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a0140600 con 0x7fd7a0145750 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd7a5e00640 1 --2- 172.21.15.60:0/2779568872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7a0145750 0x7fd7a01f6aa0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd798009d90 tx=0x7fd798003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd79d600640 1 -- 172.21.15.60:0/2779568872 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798034050 con 0x7fd7a0145750 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd79d600640 1 -- 172.21.15.60:0/2779568872 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd798004450 con 0x7fd7a0145750 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd79d600640 1 -- 172.21.15.60:0/2779568872 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798043440 con 0x7fd7a0145750 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd7a7e00640 1 -- 172.21.15.60:0/2779568872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a0140040 con 0x7fd7a0145750 2024-09-16T15:45:45.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.833+0000 7fd7a7e00640 1 -- 172.21.15.60:0/2779568872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a0141d30 con 0x7fd7a0145750 2024-09-16T15:45:45.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.834+0000 7fd7a7e00640 1 -- 172.21.15.60:0/2779568872 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a00cf830 con 0x7fd7a0145750 2024-09-16T15:45:45.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.836+0000 7fd79d600640 1 -- 172.21.15.60:0/2779568872 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 101746+0+0 (secure 0 0 0) 0x7fd798037020 con 0x7fd7a0145750 2024-09-16T15:45:45.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.836+0000 7fd79d600640 1 --2- 172.21.15.60:0/2779568872 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd780078d60 0x7fd78007b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:45.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.837+0000 7fd79d600640 1 -- 172.21.15.60:0/2779568872 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5825+0+0 (secure 0 0 0) 0x7fd7980bc640 con 0x7fd7a0145750 2024-09-16T15:45:45.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.837+0000 7fd7a5400640 1 --2- 172.21.15.60:0/2779568872 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd780078d60 0x7fd78007b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:45.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.837+0000 7fd7a5400640 1 --2- 172.21.15.60:0/2779568872 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd780078d60 0x7fd78007b220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd794004630 tx=0x7fd79400b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:45.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:45.841+0000 7fd79d600640 1 -- 172.21.15.60:0/2779568872 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd798083d40 con 0x7fd7a0145750 2024-09-16T15:45:46.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.066+0000 7fd7a7e00640 1 -- 172.21.15.60:0/2779568872 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol52572", "target": ["mon-mgr", ""]}) -- 0x7fd7a0141b70 con 0x7fd780078d60 2024-09-16T15:45:46.087 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:45:46.086+0000 7fcb85a00640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol52572' is not ready for operation rm (use --force to override) 2024-09-16T15:45:46.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.086+0000 7fd79d600640 1 -- 172.21.15.60:0/2779568872 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol52572' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7fd794004280 con 0x7fd780078d60 2024-09-16T15:45:46.087 INFO:teuthology.orchestra.run.smithi060.stderr:Error EAGAIN: subvolume 'subvol52572' is not ready for operation rm (use --force to override) 2024-09-16T15:45:46.096 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-16T15:45:46.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:46.097 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol52572/.meta) 2024-09-16T15:45:46.152 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:45:46.152 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:45:46.152 INFO:teuthology.orchestra.run.smithi060.stdout:type = clone 2024-09-16T15:45:46.153 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/_nogroup/subvol52572/49a7a980-559d-4f5d-b470-4eedbbf4ad17 2024-09-16T15:45:46.153 INFO:teuthology.orchestra.run.smithi060.stdout:state = pending 2024-09-16T15:45:46.153 INFO:teuthology.orchestra.run.smithi060.stdout:[source] 2024-09-16T15:45:46.153 INFO:teuthology.orchestra.run.smithi060.stdout:volume = fake 2024-09-16T15:45:46.153 INFO:teuthology.orchestra.run.smithi060.stdout:subvolume = fake 2024-09-16T15:45:46.153 INFO:teuthology.orchestra.run.smithi060.stdout:snapshot = fake 2024-09-16T15:45:46.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol52572 --force 2024-09-16T15:45:46.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 -- 172.21.15.60:0/313738183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340d0410 msgr2=0x7f41340d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:46.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 --2- 172.21.15.60:0/313738183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340d0410 0x7f41340d2950 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f41340748a0 tx=0x7f412802a560 comp rx=0 tx=0).stop 2024-09-16T15:45:46.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 -- 172.21.15.60:0/313738183 shutdown_connections 2024-09-16T15:45:46.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 --2- 172.21.15.60:0/313738183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41340ce4e0 0x7f41340ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 --2- 172.21.15.60:0/313738183 >> v2:172.21.15.100:3300/0 conn(0x7f41340b0ef0 0x7f41340d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 --2- 172.21.15.60:0/313738183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340d0410 0x7f41340d2950 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 -- 172.21.15.60:0/313738183 >> 172.21.15.60:0/313738183 conn(0x7f41340d1e30 msgr2=0x7f41340d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 -- 172.21.15.60:0/313738183 shutdown_connections 2024-09-16T15:45:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.324+0000 7f413c600640 1 -- 172.21.15.60:0/313738183 wait complete. 2024-09-16T15:45:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.325+0000 7f413c600640 1 Processor -- start 2024-09-16T15:45:46.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.326+0000 7f413c600640 1 -- start start 2024-09-16T15:45:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f413c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41340b0ef0 0x7f41341f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f413c600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340ce4e0 0x7f41341f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f413c600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41340d0410 0x7f41341580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f413c600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f41340ceff0 con 0x7f41340ce4e0 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f413c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41340cee70 con 0x7f41340b0ef0 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f413c600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f41340cf170 con 0x7f41340d0410 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340ce4e0 0x7f41341f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340ce4e0 0x7f41341f2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34010/0 (socket says 172.21.15.60:34010) 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139200640 1 -- 172.21.15.60:0/2876315230 learned_addr learned my addr 172.21.15.60:0/2876315230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f413a600640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41340d0410 0x7f41341580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139c00640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41340b0ef0 0x7f41341f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139200640 1 -- 172.21.15.60:0/2876315230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41340d0410 msgr2=0x7f41341580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139200640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41340d0410 0x7f41341580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139200640 1 -- 172.21.15.60:0/2876315230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41340b0ef0 msgr2=0x7f41341f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139200640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41340b0ef0 0x7f41341f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.327+0000 7f4139200640 1 -- 172.21.15.60:0/2876315230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413415e000 con 0x7f41340ce4e0 2024-09-16T15:45:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.328+0000 7f4139200640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340ce4e0 0x7f41341f2060 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f412000d740 tx=0x7f412000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.328+0000 7f4131600640 1 -- 172.21.15.60:0/2876315230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120017070 con 0x7f41340ce4e0 2024-09-16T15:45:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.328+0000 7f4131600640 1 -- 172.21.15.60:0/2876315230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4120009d20 con 0x7f41340ce4e0 2024-09-16T15:45:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.328+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413415e2e0 con 0x7f41340ce4e0 2024-09-16T15:45:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.328+0000 7f4131600640 1 -- 172.21.15.60:0/2876315230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41200106c0 con 0x7f41340ce4e0 2024-09-16T15:45:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.328+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413415e8a0 con 0x7f41340ce4e0 2024-09-16T15:45:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.329+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41340ce930 con 0x7f41340ce4e0 2024-09-16T15:45:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.330+0000 7f4131600640 1 -- 172.21.15.60:0/2876315230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 101746+0+0 (secure 0 0 0) 0x7f412001a070 con 0x7f41340ce4e0 2024-09-16T15:45:46.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.331+0000 7f4131600640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4114078d60 0x7f411407b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:46.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.331+0000 7f4131600640 1 -- 172.21.15.60:0/2876315230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5825+0+0 (secure 0 0 0) 0x7f412009b480 con 0x7f41340ce4e0 2024-09-16T15:45:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.332+0000 7f4139c00640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4114078d60 0x7f411407b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.332+0000 7f4139c00640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4114078d60 0x7f411407b220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4128033000 tx=0x7f4128004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:46.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.336+0000 7f4131600640 1 -- 172.21.15.60:0/2876315230 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f41200632d0 con 0x7f41340ce4e0 2024-09-16T15:45:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.566+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol52572", "force": true, "target": ["mon-mgr", ""]}) -- 0x7f41341f25a0 con 0x7f4114078d60 2024-09-16T15:45:46.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.607+0000 7f4131600640 1 -- 172.21.15.60:0/2876315230 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41341f25a0 con 0x7f4114078d60 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4114078d60 msgr2=0x7f411407b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4114078d60 0x7f411407b220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4128033000 tx=0x7f4128004730 comp rx=0 tx=0).stop 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340ce4e0 msgr2=0x7f41341f2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340ce4e0 0x7f41341f2060 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f412000d740 tx=0x7f412000dc00 comp rx=0 tx=0).stop 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 shutdown_connections 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4114078d60 0x7f411407b220 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41340d0410 0x7f41341580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41340ce4e0 0x7f41341f2060 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 --2- 172.21.15.60:0/2876315230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41340b0ef0 0x7f41341f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 >> 172.21.15.60:0/2876315230 conn(0x7f41340d1e30 msgr2=0x7f41340afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 shutdown_connections 2024-09-16T15:45:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.609+0000 7f413c600640 1 -- 172.21.15.60:0/2876315230 wait complete. 2024-09-16T15:45:46.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-16T15:45:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.789+0000 7fd7c4200640 1 -- 172.21.15.60:0/882502697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 msgr2=0x7fd7bc073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.789+0000 7fd7c4200640 1 --2- 172.21.15.60:0/882502697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 0x7fd7bc073440 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0005fd0 tx=0x7fd7b002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.789+0000 7fd7c4200640 1 -- 172.21.15.60:0/882502697 shutdown_connections 2024-09-16T15:45:46.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.789+0000 7fd7c4200640 1 --2- 172.21.15.60:0/882502697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 0x7fd7bc073440 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.789+0000 7fd7c4200640 1 -- 172.21.15.60:0/882502697 >> 172.21.15.60:0/882502697 conn(0x7fd7bc0b1840 msgr2=0x7fd7bc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.789+0000 7fd7c4200640 1 -- 172.21.15.60:0/882502697 shutdown_connections 2024-09-16T15:45:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.789+0000 7fd7c4200640 1 -- 172.21.15.60:0/882502697 wait complete. 2024-09-16T15:45:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.790+0000 7fd7c4200640 1 Processor -- start 2024-09-16T15:45:46.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.791+0000 7fd7c4200640 1 -- start start 2024-09-16T15:45:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.791+0000 7fd7c4200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc06efa0 0x7fd7bc15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.791+0000 7fd7c4200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7bc0cccd0 0x7fd7bc1f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c4200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 0x7fd7bc1f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c4200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc0ce6a0 con 0x7fd7bc145750 2024-09-16T15:45:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c4200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc0ce520 con 0x7fd7bc06efa0 2024-09-16T15:45:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c4200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc0ce820 con 0x7fd7bc0cccd0 2024-09-16T15:45:46.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c2200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 0x7fd7bc1f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c2200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 0x7fd7bc1f8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34024/0 (socket says 172.21.15.60:34024) 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c2200640 1 -- 172.21.15.60:0/2514786794 learned_addr learned my addr 172.21.15.60:0/2514786794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c1800640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc06efa0 0x7fd7bc15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c0e00640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7bc0cccd0 0x7fd7bc1f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c2200640 1 -- 172.21.15.60:0/2514786794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7bc0cccd0 msgr2=0x7fd7bc1f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c2200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7bc0cccd0 0x7fd7bc1f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c2200640 1 -- 172.21.15.60:0/2514786794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc06efa0 msgr2=0x7fd7bc15f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c2200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc06efa0 0x7fd7bc15f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:46.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.792+0000 7fd7c2200640 1 -- 172.21.15.60:0/2514786794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc1411f0 con 0x7fd7bc145750 2024-09-16T15:45:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.793+0000 7fd7c2200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 0x7fd7bc1f8620 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c7a0 tx=0x7fd7b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.793+0000 7fd7b9600640 1 -- 172.21.15.60:0/2514786794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4003b70 con 0x7fd7bc145750 2024-09-16T15:45:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.793+0000 7fd7b9600640 1 -- 172.21.15.60:0/2514786794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7b4003d10 con 0x7fd7bc145750 2024-09-16T15:45:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.793+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7bc1414d0 con 0x7fd7bc145750 2024-09-16T15:45:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.793+0000 7fd7b9600640 1 -- 172.21.15.60:0/2514786794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400aab0 con 0x7fd7bc145750 2024-09-16T15:45:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.793+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7bc141a90 con 0x7fd7bc145750 2024-09-16T15:45:46.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.794+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7bc0cecd0 con 0x7fd7bc145750 2024-09-16T15:45:46.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.795+0000 7fd7b9600640 1 -- 172.21.15.60:0/2514786794 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 20) ==== 101746+0+0 (secure 0 0 0) 0x7fd7b400ac50 con 0x7fd7bc145750 2024-09-16T15:45:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.796+0000 7fd7b9600640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd79c078d60 0x7fd79c07b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.796+0000 7fd7b9600640 1 -- 172.21.15.60:0/2514786794 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5809+0+0 (secure 0 0 0) 0x7fd7b409b160 con 0x7fd7bc145750 2024-09-16T15:45:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.796+0000 7fd7c1800640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd79c078d60 0x7fd79c07b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:46.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.797+0000 7fd7c1800640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd79c078d60 0x7fd79c07b220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0033000 tx=0x7fd7b0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:46.801+0000 7fd7b9600640 1 -- 172.21.15.60:0/2514786794 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd7b4062830 con 0x7fd7bc145750 2024-09-16T15:45:47.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.022+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fd7bc141c60 con 0x7fd79c078d60 2024-09-16T15:45:47.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.025+0000 7fd7b9600640 1 -- 172.21.15.60:0/2514786794 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fd7bc141c60 con 0x7fd79c078d60 2024-09-16T15:45:47.026 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-09-16T15:45:47.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.027+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd79c078d60 msgr2=0x7fd79c07b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:47.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.027+0000 7fd7c4200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd79c078d60 0x7fd79c07b220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0033000 tx=0x7fd7b0004730 comp rx=0 tx=0).stop 2024-09-16T15:45:47.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.027+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 msgr2=0x7fd7bc1f8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:47.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.027+0000 7fd7c4200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 0x7fd7bc1f8620 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400c7a0 tx=0x7fd7b400cc60 comp rx=0 tx=0).stop 2024-09-16T15:45:47.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.028+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 shutdown_connections 2024-09-16T15:45:47.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.028+0000 7fd7c4200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd79c078d60 0x7fd79c07b220 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.028+0000 7fd7c4200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7bc145750 0x7fd7bc1f8620 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.028+0000 7fd7c4200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7bc0cccd0 0x7fd7bc1f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.028+0000 7fd7c4200640 1 --2- 172.21.15.60:0/2514786794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7bc06efa0 0x7fd7bc15f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.028+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 >> 172.21.15.60:0/2514786794 conn(0x7fd7bc0b1840 msgr2=0x7fd7bc0af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:47.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.028+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 shutdown_connections 2024-09-16T15:45:47.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.028+0000 7fd7c4200640 1 -- 172.21.15.60:0/2514786794 wait complete. 2024-09-16T15:45:47.045 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:45:47.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:45:47.131 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-16T15:45:47.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:45:47.140 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:45:47.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:45:47.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:45:47.308 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:45:47.308 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:45:47.308 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:47.308 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:47.344 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:47.344 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:47.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:45:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.622+0000 7fc617200640 1 -- 172.21.15.60:0/1786327914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6100cdd00 msgr2=0x7fc6100adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.622+0000 7fc617200640 1 --2- 172.21.15.60:0/1786327914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6100cdd00 0x7fc6100adcd0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fc6100cd570 tx=0x7fc60c02aa30 comp rx=0 tx=0).stop 2024-09-16T15:45:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.622+0000 7fc617200640 1 -- 172.21.15.60:0/1786327914 shutdown_connections 2024-09-16T15:45:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.623+0000 7fc617200640 1 --2- 172.21.15.60:0/1786327914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6100cdd00 0x7fc6100adcd0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.623+0000 7fc617200640 1 --2- 172.21.15.60:0/1786327914 >> v2:172.21.15.100:3300/0 conn(0x7fc61006fb90 0x7fc61006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.623+0000 7fc617200640 1 -- 172.21.15.60:0/1786327914 >> 172.21.15.60:0/1786327914 conn(0x7fc6100cfa20 msgr2=0x7fc6100cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.623+0000 7fc617200640 1 -- 172.21.15.60:0/1786327914 shutdown_connections 2024-09-16T15:45:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.623+0000 7fc617200640 1 -- 172.21.15.60:0/1786327914 wait complete. 2024-09-16T15:45:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.624+0000 7fc617200640 1 Processor -- start 2024-09-16T15:45:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.624+0000 7fc617200640 1 -- start start 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.625+0000 7fc617200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc61006fb90 0x7fc6100b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.625+0000 7fc617200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6100ad880 0x7fc61014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.625+0000 7fc617200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6100cdd00 0x7fc6101f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.625+0000 7fc617200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc61015ed70 con 0x7fc61006fb90 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.625+0000 7fc617200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc61015ebf0 con 0x7fc6100ad880 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.625+0000 7fc617200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc61015eef0 con 0x7fc6100cdd00 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.625+0000 7fc615e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc61006fb90 0x7fc6100b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc61006fb90 0x7fc6100b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34042/0 (socket says 172.21.15.60:34042) 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615e00640 1 -- 172.21.15.60:0/1534837394 learned_addr learned my addr 172.21.15.60:0/1534837394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615400640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6100ad880 0x7fc61014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615e00640 1 -- 172.21.15.60:0/1534837394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6100cdd00 msgr2=0x7fc6101f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615e00640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6100cdd00 0x7fc6101f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615e00640 1 -- 172.21.15.60:0/1534837394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6100ad880 msgr2=0x7fc61014f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615e00640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6100ad880 0x7fc61014f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615e00640 1 -- 172.21.15.60:0/1534837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61014fae0 con 0x7fc61006fb90 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615400640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6100ad880 0x7fc61014f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc615e00640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc61006fb90 0x7fc6100b55a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc60400d950 tx=0x7fc60400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc60a000640 1 -- 172.21.15.60:0/1534837394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604017070 con 0x7fc61006fb90 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc60a000640 1 -- 172.21.15.60:0/1534837394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc604009d20 con 0x7fc61006fb90 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61014fdc0 con 0x7fc61006fb90 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.626+0000 7fc60a000640 1 -- 172.21.15.60:0/1534837394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604004280 con 0x7fc61006fb90 2024-09-16T15:45:47.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.627+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc610150380 con 0x7fc61006fb90 2024-09-16T15:45:47.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.628+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61006a180 con 0x7fc61006fb90 2024-09-16T15:45:47.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.629+0000 7fc60a000640 1 -- 172.21.15.60:0/1534837394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fc6040035f0 con 0x7fc61006fb90 2024-09-16T15:45:47.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.630+0000 7fc60a000640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5f4078eb0 0x7fc5f407b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:47.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.630+0000 7fc60a000640 1 -- 172.21.15.60:0/1534837394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5809+0+0 (secure 0 0 0) 0x7fc60409b710 con 0x7fc61006fb90 2024-09-16T15:45:47.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.630+0000 7fc615400640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5f4078eb0 0x7fc5f407b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:47.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.630+0000 7fc615400640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5f4078eb0 0x7fc5f407b370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc61015c0d0 tx=0x7fc600009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:47.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.634+0000 7fc60a000640 1 -- 172.21.15.60:0/1534837394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc61006a180 con 0x7fc61006fb90 2024-09-16T15:45:47.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.845+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc61006ff70 con 0x7fc61006fb90 2024-09-16T15:45:47.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.845+0000 7fc60a000640 1 -- 172.21.15.60:0/1534837394 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) ==== 80+0+0 (secure 0 0 0) 0x7fc604063340 con 0x7fc61006fb90 2024-09-16T15:45:47.846 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:45:47.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5f4078eb0 msgr2=0x7fc5f407b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:47.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5f4078eb0 0x7fc5f407b370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc61015c0d0 tx=0x7fc600009040 comp rx=0 tx=0).stop 2024-09-16T15:45:47.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc61006fb90 msgr2=0x7fc6100b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:47.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc61006fb90 0x7fc6100b55a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc60400d950 tx=0x7fc60400de10 comp rx=0 tx=0).stop 2024-09-16T15:45:47.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 shutdown_connections 2024-09-16T15:45:47.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5f4078eb0 0x7fc5f407b370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6100cdd00 0x7fc6101f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6100ad880 0x7fc61014f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 --2- 172.21.15.60:0/1534837394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc61006fb90 0x7fc6100b55a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:47.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 >> 172.21.15.60:0/1534837394 conn(0x7fc6100cfa20 msgr2=0x7fc610160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:47.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 shutdown_connections 2024-09-16T15:45:47.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:47.848+0000 7fc617200640 1 -- 172.21.15.60:0/1534837394 wait complete. 2024-09-16T15:45:47.864 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:45:47.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:47.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:45:47.957 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:45:47.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:47.958 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:45:48.006 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:45:48.007 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:48.007 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:48.043 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:48.043 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:48.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:45:48.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.282+0000 7fc25e400640 1 -- 172.21.15.60:0/3623325004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 msgr2=0x7fc258073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:48.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.282+0000 7fc25e400640 1 --2- 172.21.15.60:0/3623325004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 0x7fc258073240 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc2580b4890 tx=0x7fc25002aa10 comp rx=0 tx=0).stop 2024-09-16T15:45:48.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.282+0000 7fc25e400640 1 -- 172.21.15.60:0/3623325004 shutdown_connections 2024-09-16T15:45:48.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.282+0000 7fc25e400640 1 --2- 172.21.15.60:0/3623325004 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 0x7fc258073240 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.282+0000 7fc25e400640 1 --2- 172.21.15.60:0/3623325004 >> v2:172.21.15.100:3300/0 conn(0x7fc2580af3d0 0x7fc25806f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.282+0000 7fc25e400640 1 -- 172.21.15.60:0/3623325004 >> 172.21.15.60:0/3623325004 conn(0x7fc2580b1950 msgr2=0x7fc2580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:48.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.282+0000 7fc25e400640 1 -- 172.21.15.60:0/3623325004 shutdown_connections 2024-09-16T15:45:48.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.282+0000 7fc25e400640 1 -- 172.21.15.60:0/3623325004 wait complete. 2024-09-16T15:45:48.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.283+0000 7fc25e400640 1 Processor -- start 2024-09-16T15:45:48.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.283+0000 7fc25e400640 1 -- start start 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.284+0000 7fc25e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 0x7fc2581f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.284+0000 7fc25e400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580af3d0 0x7fc258155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.284+0000 7fc25e400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc258161550 0x7fc2581436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.284+0000 7fc25e400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc258073780 con 0x7fc258072df0 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.284+0000 7fc25e400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc25806fdb0 con 0x7fc2580af3d0 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.284+0000 7fc25e400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc258073900 con 0x7fc258161550 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 0x7fc2581f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc24ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580af3d0 0x7fc258155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 0x7fc2581f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34066/0 (socket says 172.21.15.60:34066) 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 -- 172.21.15.60:0/3861012585 learned_addr learned my addr 172.21.15.60:0/3861012585 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:48.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257e00640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc258161550 0x7fc2581436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:48.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 -- 172.21.15.60:0/3861012585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc258161550 msgr2=0x7fc2581436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:48.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc258161550 0x7fc2581436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 -- 172.21.15.60:0/3861012585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580af3d0 msgr2=0x7fc258155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580af3d0 0x7fc258155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 -- 172.21.15.60:0/3861012585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc258156700 con 0x7fc258072df0 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc24ea00640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580af3d0 0x7fc258155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.285+0000 7fc257400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 0x7fc2581f58e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c3a0 tx=0x7fc24800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.286+0000 7fc24fe00640 1 -- 172.21.15.60:0/3861012585 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248017070 con 0x7fc258072df0 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.286+0000 7fc24fe00640 1 -- 172.21.15.60:0/3861012585 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2480053c0 con 0x7fc258072df0 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.286+0000 7fc24fe00640 1 -- 172.21.15.60:0/3861012585 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248003850 con 0x7fc258072df0 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.286+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2581569e0 con 0x7fc258072df0 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.286+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2580cfb40 con 0x7fc258072df0 2024-09-16T15:45:48.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.287+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc258073df0 con 0x7fc258072df0 2024-09-16T15:45:48.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.288+0000 7fc24fe00640 1 -- 172.21.15.60:0/3861012585 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fc24801a070 con 0x7fc258072df0 2024-09-16T15:45:48.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.289+0000 7fc24fe00640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc234078e80 0x7fc23407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:48.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.290+0000 7fc24fe00640 1 -- 172.21.15.60:0/3861012585 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5809+0+0 (secure 0 0 0) 0x7fc24809a920 con 0x7fc258072df0 2024-09-16T15:45:48.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.290+0000 7fc24ea00640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc234078e80 0x7fc23407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:48.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.290+0000 7fc24ea00640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc234078e80 0x7fc23407b340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc2581f8300 tx=0x7fc24000b390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:48.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.294+0000 7fc24fe00640 1 -- 172.21.15.60:0/3861012585 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc248061e40 con 0x7fc258072df0 2024-09-16T15:45:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.499+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2580aeb10 con 0x7fc258072df0 2024-09-16T15:45:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.499+0000 7fc24fe00640 1 -- 172.21.15.60:0/3861012585 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) ==== 80+0+0 (secure 0 0 0) 0x7fc248066870 con 0x7fc258072df0 2024-09-16T15:45:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:45:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc234078e80 msgr2=0x7fc23407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc234078e80 0x7fc23407b340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc2581f8300 tx=0x7fc24000b390 comp rx=0 tx=0).stop 2024-09-16T15:45:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 msgr2=0x7fc2581f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 0x7fc2581f58e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c3a0 tx=0x7fc24800c860 comp rx=0 tx=0).stop 2024-09-16T15:45:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 shutdown_connections 2024-09-16T15:45:48.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc234078e80 0x7fc23407b340 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc258161550 0x7fc2581436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580af3d0 0x7fc258155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 --2- 172.21.15.60:0/3861012585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc258072df0 0x7fc2581f58e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 >> 172.21.15.60:0/3861012585 conn(0x7fc2580b1950 msgr2=0x7fc2580ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 shutdown_connections 2024-09-16T15:45:48.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.502+0000 7fc25e400640 1 -- 172.21.15.60:0/3861012585 wait complete. 2024-09-16T15:45:48.519 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:45:48.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:48.519 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:45:48.582 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:45:48.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:48.582 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:45:48.619 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:45:48.619 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:48.619 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:48.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:45:48.655 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:45:48.656 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:48.656 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:48.696 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:45:48.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:48.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.857+0000 7fb547600640 1 -- 172.21.15.60:0/1501981050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb540161550 msgr2=0x7fb54006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:48.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.857+0000 7fb547600640 1 --2- 172.21.15.60:0/1501981050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb540161550 0x7fb54006ef50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb530009830 tx=0x7fb53002a480 comp rx=0 tx=0).stop 2024-09-16T15:45:48.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.857+0000 7fb547600640 1 -- 172.21.15.60:0/1501981050 shutdown_connections 2024-09-16T15:45:48.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.857+0000 7fb547600640 1 --2- 172.21.15.60:0/1501981050 >> v2:172.21.15.100:3300/0 conn(0x7fb5400af3d0 0x7fb54006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.857+0000 7fb547600640 1 --2- 172.21.15.60:0/1501981050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb540161550 0x7fb54006ef50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.857+0000 7fb547600640 1 -- 172.21.15.60:0/1501981050 >> 172.21.15.60:0/1501981050 conn(0x7fb5400b1950 msgr2=0x7fb5400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:48.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.857+0000 7fb547600640 1 -- 172.21.15.60:0/1501981050 shutdown_connections 2024-09-16T15:45:48.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.857+0000 7fb547600640 1 -- 172.21.15.60:0/1501981050 wait complete. 2024-09-16T15:45:48.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.859+0000 7fb547600640 1 Processor -- start 2024-09-16T15:45:48.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.859+0000 7fb547600640 1 -- start start 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb547600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb54006f9d0 0x7fb5400cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb547600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5400af3d0 0x7fb5400d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb547600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb540161550 0x7fb540160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb547600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb5401623a0 con 0x7fb540161550 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb547600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb540074010 con 0x7fb5400af3d0 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb547600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb540162520 con 0x7fb54006f9d0 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb545600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb540161550 0x7fb540160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb544c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb54006f9d0 0x7fb5400cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb537e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5400af3d0 0x7fb5400d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:48.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb544c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb54006f9d0 0x7fb5400cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:34884/0 (socket says 172.21.15.60:34884) 2024-09-16T15:45:48.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.860+0000 7fb544c00640 1 -- 172.21.15.60:0/3088205398 learned_addr learned my addr 172.21.15.60:0/3088205398 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:48.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb544c00640 1 -- 172.21.15.60:0/3088205398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5400af3d0 msgr2=0x7fb5400d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:48.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb544c00640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5400af3d0 0x7fb5400d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb544c00640 1 -- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb540161550 msgr2=0x7fb540160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:48.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb544c00640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb540161550 0x7fb540160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:48.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb544c00640 1 -- 172.21.15.60:0/3088205398 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5400cd3f0 con 0x7fb54006f9d0 2024-09-16T15:45:48.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb545600640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb540161550 0x7fb540160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:45:48.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb544c00640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb54006f9d0 0x7fb5400cceb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb530009950 tx=0x7fb530033c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:48.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb53d600640 1 -- 172.21.15.60:0/3088205398 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530005c90 con 0x7fb54006f9d0 2024-09-16T15:45:48.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb53d600640 1 -- 172.21.15.60:0/3088205398 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb530031e50 con 0x7fb54006f9d0 2024-09-16T15:45:48.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5401f8060 con 0x7fb54006f9d0 2024-09-16T15:45:48.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.861+0000 7fb53d600640 1 -- 172.21.15.60:0/3088205398 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530033290 con 0x7fb54006f9d0 2024-09-16T15:45:48.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.862+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5401f8620 con 0x7fb54006f9d0 2024-09-16T15:45:48.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.863+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5400760b0 con 0x7fb54006f9d0 2024-09-16T15:45:48.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.864+0000 7fb53d600640 1 -- 172.21.15.60:0/3088205398 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fb530033430 con 0x7fb54006f9d0 2024-09-16T15:45:48.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.865+0000 7fb53d600640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb51c078e80 0x7fb51c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:48.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.865+0000 7fb53d600640 1 -- 172.21.15.60:0/3088205398 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7fb5300bc980 con 0x7fb54006f9d0 2024-09-16T15:45:48.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.865+0000 7fb537e00640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb51c078e80 0x7fb51c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:48.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.866+0000 7fb537e00640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb51c078e80 0x7fb51c07b340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb54014c7b0 tx=0x7fb528005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:48.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:48.869+0000 7fb53d600640 1 -- 172.21.15.60:0/3088205398 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5400cd3f0 con 0x7fb54006f9d0 2024-09-16T15:45:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.103+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb540162ab0 con 0x7fb54006f9d0 2024-09-16T15:45:49.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.104+0000 7fb53d600640 1 -- 172.21.15.60:0/3088205398 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5225 (secure 0 0 0) 0x7fb53003f030 con 0x7fb54006f9d0 2024-09-16T15:45:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:49.105 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"btime":"2024-09-16T15:45:40:363160+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7225,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/900096462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":900096462},{"type":"v1","addr":"172.21.15.100:6834","nonce":900096462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2427775344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2427775344},{"type":"v1","addr":"172.21.15.100:6835","nonce":2427775344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7584,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1853423008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1853423008},{"type":"v1","addr":"172.21.15.60:6833","nonce":1853423008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:45:29.603235+0000","modified":"2024-09-16T15:45:32.622836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/678119606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":678119606},{"type":"v1","addr":"172.21.15.60:6835","nonce":678119606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7593,"qdb_cluster":[7593]},"id":9}]} 2024-09-16T15:45:49.106 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-09-16T15:45:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb51c078e80 msgr2=0x7fb51c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb51c078e80 0x7fb51c07b340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb54014c7b0 tx=0x7fb528005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb54006f9d0 msgr2=0x7fb5400cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb54006f9d0 0x7fb5400cceb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb530009950 tx=0x7fb530033c20 comp rx=0 tx=0).stop 2024-09-16T15:45:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 shutdown_connections 2024-09-16T15:45:49.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb51c078e80 0x7fb51c07b340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb540161550 0x7fb540160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5400af3d0 0x7fb5400d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 --2- 172.21.15.60:0/3088205398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb54006f9d0 0x7fb5400cceb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 >> 172.21.15.60:0/3088205398 conn(0x7fb5400b1950 msgr2=0x7fb54015f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:49.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 shutdown_connections 2024-09-16T15:45:49.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.107+0000 7fb547600640 1 -- 172.21.15.60:0/3088205398 wait complete. 2024-09-16T15:45:49.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:49.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.286+0000 7f17a1a00640 1 -- 172.21.15.60:0/4056497883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0746a0 msgr2=0x7f179c0732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.286+0000 7f17a1a00640 1 --2- 172.21.15.60:0/4056497883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0746a0 0x7f179c0732b0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f179400acf0 tx=0x7f179402bae0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.286+0000 7f17a1a00640 1 -- 172.21.15.60:0/4056497883 shutdown_connections 2024-09-16T15:45:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.286+0000 7f17a1a00640 1 --2- 172.21.15.60:0/4056497883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0746a0 0x7f179c0732b0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.286+0000 7f17a1a00640 1 -- 172.21.15.60:0/4056497883 >> 172.21.15.60:0/4056497883 conn(0x7f179c0b1950 msgr2=0x7f179c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.286+0000 7f17a1a00640 1 -- 172.21.15.60:0/4056497883 shutdown_connections 2024-09-16T15:45:49.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.286+0000 7f17a1a00640 1 -- 172.21.15.60:0/4056497883 wait complete. 2024-09-16T15:45:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.288+0000 7f17a1a00640 1 Processor -- start 2024-09-16T15:45:49.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.288+0000 7f17a1a00640 1 -- start start 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f17a1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c074150 0x7f179c1f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f17a1a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0746a0 0x7f179c1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f17a1a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0b43f0 0x7f179c159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f17a1a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f179c073970 con 0x7f179c0b43f0 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f17a1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f179c0737f0 con 0x7f179c074150 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f17a1a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f179c073af0 con 0x7f179c0746a0 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f179b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0b43f0 0x7f179c159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f179b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0b43f0 0x7f179c159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56132/0 (socket says 172.21.15.60:56132) 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f179b400640 1 -- 172.21.15.60:0/364958509 learned_addr learned my addr 172.21.15.60:0/364958509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f1792000640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0746a0 0x7f179c1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.289+0000 7f179aa00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c074150 0x7f179c1f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f179b400640 1 -- 172.21.15.60:0/364958509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0746a0 msgr2=0x7f179c1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f179b400640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0746a0 0x7f179c1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f179b400640 1 -- 172.21.15.60:0/364958509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c074150 msgr2=0x7f179c1f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f179b400640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c074150 0x7f179c1f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f179b400640 1 -- 172.21.15.60:0/364958509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c14ac10 con 0x7f179c0b43f0 2024-09-16T15:45:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f179b400640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0b43f0 0x7f179c159830 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f179400acf0 tx=0x7f1794003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f1793400640 1 -- 172.21.15.60:0/364958509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794046070 con 0x7f179c0b43f0 2024-09-16T15:45:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f1793400640 1 -- 172.21.15.60:0/364958509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17940042d0 con 0x7f179c0b43f0 2024-09-16T15:45:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f1793400640 1 -- 172.21.15.60:0/364958509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940046a0 con 0x7f179c0b43f0 2024-09-16T15:45:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.290+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c1e5a60 con 0x7f179c0b43f0 2024-09-16T15:45:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.291+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c1e5780 con 0x7f179c0b43f0 2024-09-16T15:45:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.292+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c0b47d0 con 0x7f179c0b43f0 2024-09-16T15:45:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.293+0000 7f1793400640 1 -- 172.21.15.60:0/364958509 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f1794049070 con 0x7f179c0b43f0 2024-09-16T15:45:49.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.293+0000 7f1793400640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1778078eb0 0x7f177807b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:49.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.294+0000 7f1793400640 1 -- 172.21.15.60:0/364958509 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7f17940bcb60 con 0x7f179c0b43f0 2024-09-16T15:45:49.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.294+0000 7f179aa00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1778078eb0 0x7f177807b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:49.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.294+0000 7f179aa00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1778078eb0 0x7f177807b370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f178c006fd0 tx=0x7f178c009060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:49.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.298+0000 7f1793400640 1 -- 172.21.15.60:0/364958509 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1794033070 con 0x7f179c0b43f0 2024-09-16T15:45:49.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.533+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f179c1e89d0 con 0x7f179c0b43f0 2024-09-16T15:45:49.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.533+0000 7f1793400640 1 -- 172.21.15.60:0/364958509 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5225 (secure 0 0 0) 0x7f1794084140 con 0x7f179c0b43f0 2024-09-16T15:45:49.534 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:49.534 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"btime":"2024-09-16T15:45:40:363160+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7225,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/900096462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":900096462},{"type":"v1","addr":"172.21.15.100:6834","nonce":900096462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2427775344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2427775344},{"type":"v1","addr":"172.21.15.100:6835","nonce":2427775344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7584,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1853423008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1853423008},{"type":"v1","addr":"172.21.15.60:6833","nonce":1853423008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:45:29.603235+0000","modified":"2024-09-16T15:45:32.622836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/678119606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":678119606},{"type":"v1","addr":"172.21.15.60:6835","nonce":678119606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7593,"qdb_cluster":[7593]},"id":9}]} 2024-09-16T15:45:49.535 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-09-16T15:45:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1778078eb0 msgr2=0x7f177807b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1778078eb0 0x7f177807b370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f178c006fd0 tx=0x7f178c009060 comp rx=0 tx=0).stop 2024-09-16T15:45:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0b43f0 msgr2=0x7f179c159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0b43f0 0x7f179c159830 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f179400acf0 tx=0x7f1794003f10 comp rx=0 tx=0).stop 2024-09-16T15:45:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 shutdown_connections 2024-09-16T15:45:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1778078eb0 0x7f177807b370 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f179c0b43f0 0x7f179c159830 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f179c0746a0 0x7f179c1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 --2- 172.21.15.60:0/364958509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f179c074150 0x7f179c1f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 >> 172.21.15.60:0/364958509 conn(0x7f179c0b1950 msgr2=0x7f179c15a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:49.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 shutdown_connections 2024-09-16T15:45:49.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.536+0000 7f17a1a00640 1 -- 172.21.15.60:0/364958509 wait complete. 2024-09-16T15:45:49.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.713+0000 7f6fa8c00640 1 -- 172.21.15.60:0/4220773133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40ccf00 msgr2=0x7f6fa40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.713+0000 7f6fa8c00640 1 --2- 172.21.15.60:0/4220773133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40ccf00 0x7f6fa40b2440 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009830 tx=0x7f6f9002a530 comp rx=0 tx=0).stop 2024-09-16T15:45:49.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.713+0000 7f6fa8c00640 1 -- 172.21.15.60:0/4220773133 shutdown_connections 2024-09-16T15:45:49.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.713+0000 7f6fa8c00640 1 --2- 172.21.15.60:0/4220773133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa40ccf00 0x7f6fa40b2440 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.713+0000 7f6fa8c00640 1 -- 172.21.15.60:0/4220773133 >> 172.21.15.60:0/4220773133 conn(0x7f6fa40b1950 msgr2=0x7f6fa40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:49.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.713+0000 7f6fa8c00640 1 -- 172.21.15.60:0/4220773133 shutdown_connections 2024-09-16T15:45:49.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.713+0000 7f6fa8c00640 1 -- 172.21.15.60:0/4220773133 wait complete. 2024-09-16T15:45:49.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.715+0000 7f6fa8c00640 1 Processor -- start 2024-09-16T15:45:49.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.715+0000 7f6fa8c00640 1 -- start start 2024-09-16T15:45:49.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa8c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fa4073fb0 0x7f6fa414e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:49.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa8c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fa40ccf00 0x7f6fa40b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:49.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa8c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa415f5a0 0x7f6fa40c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa8c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa406f9f0 con 0x7f6fa40ccf00 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa8c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa406f870 con 0x7f6fa415f5a0 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa8c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa4074420 con 0x7f6fa4073fb0 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fa40ccf00 0x7f6fa40b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fa40ccf00 0x7f6fa40b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56148/0 (socket says 172.21.15.60:56148) 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa1400640 1 -- 172.21.15.60:0/3877902069 learned_addr learned my addr 172.21.15.60:0/3877902069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa2800640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa415f5a0 0x7f6fa40c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.716+0000 7f6fa1e00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fa4073fb0 0x7f6fa414e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa1400640 1 -- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fa4073fb0 msgr2=0x7f6fa414e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa1400640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fa4073fb0 0x7f6fa414e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa1400640 1 -- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa415f5a0 msgr2=0x7f6fa40c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa1400640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa415f5a0 0x7f6fa40c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa1400640 1 -- 172.21.15.60:0/3877902069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa40cbc70 con 0x7f6fa40ccf00 2024-09-16T15:45:49.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa2800640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa415f5a0 0x7f6fa40c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:49.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa1400640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fa40ccf00 0x7f6fa40b46a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009780 tx=0x7f6f90032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:49.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6f9a000640 1 -- 172.21.15.60:0/3877902069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90040070 con 0x7f6fa40ccf00 2024-09-16T15:45:49.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6f9a000640 1 -- 172.21.15.60:0/3877902069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f90005310 con 0x7f6fa40ccf00 2024-09-16T15:45:49.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6f9a000640 1 -- 172.21.15.60:0/3877902069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90005c90 con 0x7f6fa40ccf00 2024-09-16T15:45:49.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa40c9170 con 0x7f6fa40ccf00 2024-09-16T15:45:49.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.717+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa40c9730 con 0x7f6fa40ccf00 2024-09-16T15:45:49.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.718+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa40b2440 con 0x7f6fa40ccf00 2024-09-16T15:45:49.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.720+0000 7f6f9a000640 1 -- 172.21.15.60:0/3877902069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f6f90039040 con 0x7f6fa40ccf00 2024-09-16T15:45:49.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.720+0000 7f6f9a000640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f84078e80 0x7f6f8407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:49.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.721+0000 7f6f9a000640 1 -- 172.21.15.60:0/3877902069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7f6f900bc680 con 0x7f6fa40ccf00 2024-09-16T15:45:49.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.721+0000 7f6fa1e00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f84078e80 0x7f6f8407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:49.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.721+0000 7f6fa1e00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f84078e80 0x7f6f8407b340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f94001120 tx=0x7f6f94009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:49.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.725+0000 7f6f9a000640 1 -- 172.21.15.60:0/3877902069 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6f90084450 con 0x7f6fa40ccf00 2024-09-16T15:45:49.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.920+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fa4155ce0 con 0x7f6fa40ccf00 2024-09-16T15:45:49.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.921+0000 7f6f9a000640 1 -- 172.21.15.60:0/3877902069 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16850 (secure 0 0 0) 0x7f6f90083d50 con 0x7f6fa40ccf00 2024-09-16T15:45:49.922 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:49.923 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":125,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:48.576904+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:45:27.412685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:45:28.568791+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.923+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f84078e80 msgr2=0x7f6f8407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.923+0000 7f6fa8c00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f84078e80 0x7f6f8407b340 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f94001120 tx=0x7f6f94009040 comp rx=0 tx=0).stop 2024-09-16T15:45:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.923+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fa40ccf00 msgr2=0x7f6fa40b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.923+0000 7f6fa8c00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fa40ccf00 0x7f6fa40b46a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009780 tx=0x7f6f90032c10 comp rx=0 tx=0).stop 2024-09-16T15:45:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.924+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 shutdown_connections 2024-09-16T15:45:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.924+0000 7f6fa8c00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f84078e80 0x7f6f8407b340 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.924+0000 7f6fa8c00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fa415f5a0 0x7f6fa40c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.924+0000 7f6fa8c00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fa40ccf00 0x7f6fa40b46a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.924+0000 7f6fa8c00640 1 --2- 172.21.15.60:0/3877902069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fa4073fb0 0x7f6fa414e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:49.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.924+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 >> 172.21.15.60:0/3877902069 conn(0x7f6fa40b1950 msgr2=0x7f6fa40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:49.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.924+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 shutdown_connections 2024-09-16T15:45:49.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:49.924+0000 7f6fa8c00640 1 -- 172.21.15.60:0/3877902069 wait complete. 2024-09-16T15:45:49.940 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:45:49.941 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:50.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.100+0000 7ff555800640 1 -- 172.21.15.60:0/1155405580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5500b4050 msgr2=0x7ff5500b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.100+0000 7ff555800640 1 --2- 172.21.15.60:0/1155405580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5500b4050 0x7ff5500b1770 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff5500b6930 tx=0x7ff54802aa30 comp rx=0 tx=0).stop 2024-09-16T15:45:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.100+0000 7ff555800640 1 -- 172.21.15.60:0/1155405580 shutdown_connections 2024-09-16T15:45:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.100+0000 7ff555800640 1 --2- 172.21.15.60:0/1155405580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5500b4050 0x7ff5500b1770 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.100+0000 7ff555800640 1 --2- 172.21.15.60:0/1155405580 >> v2:172.21.15.100:3300/0 conn(0x7ff550161440 0x7ff5500b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.100+0000 7ff555800640 1 -- 172.21.15.60:0/1155405580 >> 172.21.15.60:0/1155405580 conn(0x7ff5500d3df0 msgr2=0x7ff5500d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.100+0000 7ff555800640 1 -- 172.21.15.60:0/1155405580 shutdown_connections 2024-09-16T15:45:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.100+0000 7ff555800640 1 -- 172.21.15.60:0/1155405580 wait complete. 2024-09-16T15:45:50.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.102+0000 7ff555800640 1 Processor -- start 2024-09-16T15:45:50.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.102+0000 7ff555800640 1 -- start start 2024-09-16T15:45:50.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff555800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5500b34b0 0x7ff5501f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff555800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5500b4050 0x7ff5500d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff555800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff550161440 0x7ff5500d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff555800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff5500b2040 con 0x7ff550161440 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff555800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff5500b1ec0 con 0x7ff5500b4050 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff555800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff5500b21c0 con 0x7ff5500b34b0 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff554e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff550161440 0x7ff5500d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff554e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff550161440 0x7ff5500d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56168/0 (socket says 172.21.15.60:56168) 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff554e00640 1 -- 172.21.15.60:0/446159941 learned_addr learned my addr 172.21.15.60:0/446159941 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff547e00640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5500b4050 0x7ff5500d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.103+0000 7ff54fe00640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5500b34b0 0x7ff5501f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff554e00640 1 -- 172.21.15.60:0/446159941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5500b34b0 msgr2=0x7ff5501f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff554e00640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5500b34b0 0x7ff5501f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff554e00640 1 -- 172.21.15.60:0/446159941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5500b4050 msgr2=0x7ff5500d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff554e00640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5500b4050 0x7ff5500d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff554e00640 1 -- 172.21.15.60:0/446159941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5500cfa80 con 0x7ff550161440 2024-09-16T15:45:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff554e00640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff550161440 0x7ff5500d12e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff54802b020 tx=0x7ff548003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:50.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff54cc00640 1 -- 172.21.15.60:0/446159941 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548046070 con 0x7ff550161440 2024-09-16T15:45:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff54cc00640 1 -- 172.21.15.60:0/446159941 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff548004250 con 0x7ff550161440 2024-09-16T15:45:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff54cc00640 1 -- 172.21.15.60:0/446159941 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548033050 con 0x7ff550161440 2024-09-16T15:45:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5500cfd20 con 0x7ff550161440 2024-09-16T15:45:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.104+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5500d02e0 con 0x7ff550161440 2024-09-16T15:45:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.105+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5500b2f60 con 0x7ff550161440 2024-09-16T15:45:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.106+0000 7ff54cc00640 1 -- 172.21.15.60:0/446159941 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7ff548049070 con 0x7ff550161440 2024-09-16T15:45:50.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.107+0000 7ff54cc00640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff52c078e80 0x7ff52c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.108+0000 7ff54cc00640 1 -- 172.21.15.60:0/446159941 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7ff5480bc3b0 con 0x7ff550161440 2024-09-16T15:45:50.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.108+0000 7ff54fe00640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff52c078e80 0x7ff52c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.108+0000 7ff54fe00640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff52c078e80 0x7ff52c07b340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff540009b30 tx=0x7ff540005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:50.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.112+0000 7ff54cc00640 1 -- 172.21.15.60:0/446159941 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff5500b2f60 con 0x7ff550161440 2024-09-16T15:45:50.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.347+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5500cdd40 con 0x7ff550161440 2024-09-16T15:45:50.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.347+0000 7ff54cc00640 1 -- 172.21.15.60:0/446159941 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5225 (secure 0 0 0) 0x7ff548083950 con 0x7ff550161440 2024-09-16T15:45:50.349 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:50.349 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"btime":"2024-09-16T15:45:40:363160+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7225,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/900096462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":900096462},{"type":"v1","addr":"172.21.15.100:6834","nonce":900096462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2427775344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2427775344},{"type":"v1","addr":"172.21.15.100:6835","nonce":2427775344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7584,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1853423008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1853423008},{"type":"v1","addr":"172.21.15.60:6833","nonce":1853423008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:45:29.603235+0000","modified":"2024-09-16T15:45:32.622836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/678119606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":678119606},{"type":"v1","addr":"172.21.15.60:6835","nonce":678119606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7593,"qdb_cluster":[7593]},"id":9}]} 2024-09-16T15:45:50.349 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-09-16T15:45:50.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff52c078e80 msgr2=0x7ff52c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff52c078e80 0x7ff52c07b340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff540009b30 tx=0x7ff540005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:50.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff550161440 msgr2=0x7ff5500d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff550161440 0x7ff5500d12e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff54802b020 tx=0x7ff548003f10 comp rx=0 tx=0).stop 2024-09-16T15:45:50.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 shutdown_connections 2024-09-16T15:45:50.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff52c078e80 0x7ff52c07b340 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff550161440 0x7ff5500d12e0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff5500b4050 0x7ff5500d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 --2- 172.21.15.60:0/446159941 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5500b34b0 0x7ff5501f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 >> 172.21.15.60:0/446159941 conn(0x7ff5500d3df0 msgr2=0x7ff55015bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:50.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 shutdown_connections 2024-09-16T15:45:50.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.350+0000 7ff555800640 1 -- 172.21.15.60:0/446159941 wait complete. 2024-09-16T15:45:50.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.527+0000 7f9454400640 1 -- 172.21.15.60:0/2751997862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 msgr2=0x7f944c06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.527+0000 7f9454400640 1 --2- 172.21.15.60:0/2751997862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 0x7f944c06ef50 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9440005fd0 tx=0x7f944002a540 comp rx=0 tx=0).stop 2024-09-16T15:45:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.527+0000 7f9454400640 1 -- 172.21.15.60:0/2751997862 shutdown_connections 2024-09-16T15:45:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.527+0000 7f9454400640 1 --2- 172.21.15.60:0/2751997862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f944c072df0 0x7f944c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.527+0000 7f9454400640 1 --2- 172.21.15.60:0/2751997862 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 0x7f944c06ef50 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.527+0000 7f9454400640 1 -- 172.21.15.60:0/2751997862 >> 172.21.15.60:0/2751997862 conn(0x7f944c0b1950 msgr2=0x7f944c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.527+0000 7f9454400640 1 -- 172.21.15.60:0/2751997862 shutdown_connections 2024-09-16T15:45:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.527+0000 7f9454400640 1 -- 172.21.15.60:0/2751997862 wait complete. 2024-09-16T15:45:50.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.529+0000 7f9454400640 1 Processor -- start 2024-09-16T15:45:50.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.529+0000 7f9454400640 1 -- start start 2024-09-16T15:45:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9454400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f944c072df0 0x7f944c1f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9454400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c0af3d0 0x7f944c09d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9454400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 0x7f944c1f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9454400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0737e0 con 0x7f944c161550 2024-09-16T15:45:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9454400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f944c06fdb0 con 0x7f944c0af3d0 2024-09-16T15:45:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9454400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f944c073960 con 0x7f944c072df0 2024-09-16T15:45:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9452400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 0x7f944c1f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9452400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 0x7f944c1f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56200/0 (socket says 172.21.15.60:56200) 2024-09-16T15:45:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9452400640 1 -- 172.21.15.60:0/325794772 learned_addr learned my addr 172.21.15.60:0/325794772 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9451a00640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f944c072df0 0x7f944c1f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.530+0000 7f9451000640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c0af3d0 0x7f944c09d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9452400640 1 -- 172.21.15.60:0/325794772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f944c072df0 msgr2=0x7f944c1f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9452400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f944c072df0 0x7f944c1f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9452400640 1 -- 172.21.15.60:0/325794772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c0af3d0 msgr2=0x7f944c09d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9452400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c0af3d0 0x7f944c09d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9452400640 1 -- 172.21.15.60:0/325794772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c1f4730 con 0x7f944c161550 2024-09-16T15:45:50.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9452400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 0x7f944c1f4ee0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f944400eaa0 tx=0x7f944400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9449600640 1 -- 172.21.15.60:0/325794772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944400cdb0 con 0x7f944c161550 2024-09-16T15:45:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9449600640 1 -- 172.21.15.60:0/325794772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9444004c10 con 0x7f944c161550 2024-09-16T15:45:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9449600640 1 -- 172.21.15.60:0/325794772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444005020 con 0x7f944c161550 2024-09-16T15:45:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c1f84f0 con 0x7f944c161550 2024-09-16T15:45:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.531+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c1f8ab0 con 0x7f944c161550 2024-09-16T15:45:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.532+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c073e50 con 0x7f944c161550 2024-09-16T15:45:50.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.534+0000 7f9449600640 1 -- 172.21.15.60:0/325794772 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f94440051c0 con 0x7f944c161550 2024-09-16T15:45:50.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.534+0000 7f9449600640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f942c078e80 0x7f942c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.535+0000 7f9449600640 1 -- 172.21.15.60:0/325794772 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7f944409be90 con 0x7f944c161550 2024-09-16T15:45:50.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.535+0000 7f9451a00640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f942c078e80 0x7f942c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.535+0000 7f9451a00640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f942c078e80 0x7f942c07b340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9440033000 tx=0x7f9440004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:50.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.539+0000 7f9449600640 1 -- 172.21.15.60:0/325794772 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9444016030 con 0x7f944c161550 2024-09-16T15:45:50.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.774+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f944c0b1e60 con 0x7f944c161550 2024-09-16T15:45:50.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.775+0000 7f9449600640 1 -- 172.21.15.60:0/325794772 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5225 (secure 0 0 0) 0x7f9444063470 con 0x7f944c161550 2024-09-16T15:45:50.776 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:50.776 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"btime":"2024-09-16T15:45:40:363160+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7225,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/900096462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":900096462},{"type":"v1","addr":"172.21.15.100:6834","nonce":900096462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2427775344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2427775344},{"type":"v1","addr":"172.21.15.100:6835","nonce":2427775344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7584,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1853423008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1853423008},{"type":"v1","addr":"172.21.15.60:6833","nonce":1853423008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:45:29.603235+0000","modified":"2024-09-16T15:45:32.622836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7593":{"gid":7593,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/678119606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":678119606},{"type":"v1","addr":"172.21.15.60:6835","nonce":678119606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7593,"qdb_cluster":[7593]},"id":9}]} 2024-09-16T15:45:50.777 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.777+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f942c078e80 msgr2=0x7f942c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.777+0000 7f9454400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f942c078e80 0x7f942c07b340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9440033000 tx=0x7f9440004730 comp rx=0 tx=0).stop 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.777+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 msgr2=0x7f944c1f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.777+0000 7f9454400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 0x7f944c1f4ee0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f944400eaa0 tx=0x7f944400ef60 comp rx=0 tx=0).stop 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.778+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 shutdown_connections 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.778+0000 7f9454400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f942c078e80 0x7f942c07b340 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.778+0000 7f9454400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f944c161550 0x7f944c1f4ee0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.778+0000 7f9454400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f944c0af3d0 0x7f944c09d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.778+0000 7f9454400640 1 --2- 172.21.15.60:0/325794772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f944c072df0 0x7f944c1f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.778+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 >> 172.21.15.60:0/325794772 conn(0x7f944c0b1950 msgr2=0x7f944c0b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.778+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 shutdown_connections 2024-09-16T15:45:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.778+0000 7f9454400640 1 -- 172.21.15.60:0/325794772 wait complete. 2024-09-16T15:45:50.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.952+0000 7f7c2d200640 1 -- 172.21.15.60:0/1551279924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c2815f5a0 msgr2=0x7f7c280b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.952+0000 7f7c2d200640 1 --2- 172.21.15.60:0/1551279924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c2815f5a0 0x7f7c280b2440 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7c18005fd0 tx=0x7f7c1802a540 comp rx=0 tx=0).stop 2024-09-16T15:45:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.952+0000 7f7c2d200640 1 -- 172.21.15.60:0/1551279924 shutdown_connections 2024-09-16T15:45:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.952+0000 7f7c2d200640 1 --2- 172.21.15.60:0/1551279924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c28073fb0 0x7f7c28074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.952+0000 7f7c2d200640 1 --2- 172.21.15.60:0/1551279924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c2815f5a0 0x7f7c280b2440 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.952+0000 7f7c2d200640 1 -- 172.21.15.60:0/1551279924 >> 172.21.15.60:0/1551279924 conn(0x7f7c280b1950 msgr2=0x7f7c280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.952+0000 7f7c2d200640 1 -- 172.21.15.60:0/1551279924 shutdown_connections 2024-09-16T15:45:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.952+0000 7f7c2d200640 1 -- 172.21.15.60:0/1551279924 wait complete. 2024-09-16T15:45:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.954+0000 7f7c2d200640 1 Processor -- start 2024-09-16T15:45:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.954+0000 7f7c2d200640 1 -- start start 2024-09-16T15:45:50.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c2d200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c28073fb0 0x7f7c280b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c2d200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c280ccf00 0x7f7c280cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c2d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2815f5a0 0x7f7c281f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c2d200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2806f2e0 con 0x7f7c2815f5a0 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c2d200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2806f160 con 0x7f7c28073fb0 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c2d200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2806f460 con 0x7f7c280ccf00 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c26800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2815f5a0 0x7f7c281f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c26800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2815f5a0 0x7f7c281f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56214/0 (socket says 172.21.15.60:56214) 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c26800640 1 -- 172.21.15.60:0/2743401814 learned_addr learned my addr 172.21.15.60:0/2743401814 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.955+0000 7f7c25e00640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c28073fb0 0x7f7c280b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c1d400640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c280ccf00 0x7f7c280cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c26800640 1 -- 172.21.15.60:0/2743401814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c280ccf00 msgr2=0x7f7c280cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c26800640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c280ccf00 0x7f7c280cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c26800640 1 -- 172.21.15.60:0/2743401814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c28073fb0 msgr2=0x7f7c280b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c26800640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c28073fb0 0x7f7c280b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c26800640 1 -- 172.21.15.60:0/2743401814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c281f56b0 con 0x7f7c2815f5a0 2024-09-16T15:45:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c26800640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2815f5a0 0x7f7c281f49f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000eb10 tx=0x7f7c2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c1ea00640 1 -- 172.21.15.60:0/2743401814 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20004b20 con 0x7f7c2815f5a0 2024-09-16T15:45:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c1ea00640 1 -- 172.21.15.60:0/2743401814 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c20004ce0 con 0x7f7c2815f5a0 2024-09-16T15:45:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c280b2ef0 con 0x7f7c2815f5a0 2024-09-16T15:45:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.956+0000 7f7c1ea00640 1 -- 172.21.15.60:0/2743401814 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c20005280 con 0x7f7c2815f5a0 2024-09-16T15:45:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.957+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c280b34b0 con 0x7f7c2815f5a0 2024-09-16T15:45:50.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.957+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c280b36b0 con 0x7f7c2815f5a0 2024-09-16T15:45:50.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.959+0000 7f7c1ea00640 1 -- 172.21.15.60:0/2743401814 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f7c20005420 con 0x7f7c2815f5a0 2024-09-16T15:45:50.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.960+0000 7f7c1ea00640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c04078e80 0x7f7c0407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:50.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.960+0000 7f7c1ea00640 1 -- 172.21.15.60:0/2743401814 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7f7c2009c190 con 0x7f7c2815f5a0 2024-09-16T15:45:50.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.960+0000 7f7c25e00640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c04078e80 0x7f7c0407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:50.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.961+0000 7f7c25e00640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c04078e80 0x7f7c0407b340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7c18033000 tx=0x7f7c18004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:50.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:50.964+0000 7f7c1ea00640 1 -- 172.21.15.60:0/2743401814 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7c280b36b0 con 0x7f7c2815f5a0 2024-09-16T15:45:51.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.164+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c2806f980 con 0x7f7c2815f5a0 2024-09-16T15:45:51.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.165+0000 7f7c1ea00640 1 -- 172.21.15.60:0/2743401814 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16850 (secure 0 0 0) 0x7f7c20063770 con 0x7f7c2815f5a0 2024-09-16T15:45:51.166 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:51.167 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":125,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:48.576904+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:45:27.412685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:45:28.568791+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:51.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c04078e80 msgr2=0x7f7c0407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c04078e80 0x7f7c0407b340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7c18033000 tx=0x7f7c18004730 comp rx=0 tx=0).stop 2024-09-16T15:45:51.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2815f5a0 msgr2=0x7f7c281f49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2815f5a0 0x7f7c281f49f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7c2000eb10 tx=0x7f7c2000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 shutdown_connections 2024-09-16T15:45:51.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c04078e80 0x7f7c0407b340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c2815f5a0 0x7f7c281f49f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c280ccf00 0x7f7c280cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 --2- 172.21.15.60:0/2743401814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c28073fb0 0x7f7c280b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 >> 172.21.15.60:0/2743401814 conn(0x7f7c280b1950 msgr2=0x7f7c280b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:51.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 shutdown_connections 2024-09-16T15:45:51.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.168+0000 7f7c2d200640 1 -- 172.21.15.60:0/2743401814 wait complete. 2024-09-16T15:45:51.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 -- 172.21.15.60:0/3394038312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 msgr2=0x7f0f80073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 --2- 172.21.15.60:0/3394038312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 0x7f0f80073fb0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f0f80075130 tx=0x7f0f7802a980 comp rx=0 tx=0).stop 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 -- 172.21.15.60:0/3394038312 shutdown_connections 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 --2- 172.21.15.60:0/3394038312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 0x7f0f80073fb0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 --2- 172.21.15.60:0/3394038312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f800ccf00 0x7f0f800acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 --2- 172.21.15.60:0/3394038312 >> v2:172.21.15.100:3300/0 conn(0x7f0f8015f5a0 0x7f0f800b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 -- 172.21.15.60:0/3394038312 >> 172.21.15.60:0/3394038312 conn(0x7f0f800b1950 msgr2=0x7f0f800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 -- 172.21.15.60:0/3394038312 shutdown_connections 2024-09-16T15:45:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.345+0000 7f0f87a00640 1 -- 172.21.15.60:0/3394038312 wait complete. 2024-09-16T15:45:51.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.347+0000 7f0f87a00640 1 Processor -- start 2024-09-16T15:45:51.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.347+0000 7f0f87a00640 1 -- start start 2024-09-16T15:45:51.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f87a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 0x7f0f801f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:51.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f87a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f800ccf00 0x7f0f800cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:51.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f87a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8015f5a0 0x7f0f800cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:51.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f87a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8006f160 con 0x7f0f800ad270 2024-09-16T15:45:51.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f87a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80074780 con 0x7f0f800ccf00 2024-09-16T15:45:51.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f87a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8006f2e0 con 0x7f0f8015f5a0 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f85000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 0x7f0f801f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f85000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 0x7f0f801f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56232/0 (socket says 172.21.15.60:56232) 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f85000640 1 -- 172.21.15.60:0/2780695969 learned_addr learned my addr 172.21.15.60:0/2780695969 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f77e00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f800ccf00 0x7f0f800cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.348+0000 7f0f85a00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8015f5a0 0x7f0f800cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f85000640 1 -- 172.21.15.60:0/2780695969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8015f5a0 msgr2=0x7f0f800cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f85000640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8015f5a0 0x7f0f800cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f85000640 1 -- 172.21.15.60:0/2780695969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f800ccf00 msgr2=0x7f0f800cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f85000640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f800ccf00 0x7f0f800cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f85000640 1 -- 172.21.15.60:0/2780695969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f801f57f0 con 0x7f0f800ad270 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f85000640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 0x7f0f801f52b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000d740 tx=0x7f0f7000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f7d600640 1 -- 172.21.15.60:0/2780695969 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70017070 con 0x7f0f800ad270 2024-09-16T15:45:51.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f7d600640 1 -- 172.21.15.60:0/2780695969 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f70009d20 con 0x7f0f800ad270 2024-09-16T15:45:51.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f800cee60 con 0x7f0f800ad270 2024-09-16T15:45:51.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f7d600640 1 -- 172.21.15.60:0/2780695969 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f700106c0 con 0x7f0f800ad270 2024-09-16T15:45:51.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.349+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f800cf420 con 0x7f0f800ad270 2024-09-16T15:45:51.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.350+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8006f870 con 0x7f0f800ad270 2024-09-16T15:45:51.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.351+0000 7f0f7d600640 1 -- 172.21.15.60:0/2780695969 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f0f7001a070 con 0x7f0f800ad270 2024-09-16T15:45:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.352+0000 7f0f7d600640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f5c078eb0 0x7f0f5c07b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.352+0000 7f0f7d600640 1 -- 172.21.15.60:0/2780695969 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5765+0+0 (secure 0 0 0) 0x7f0f7009b5d0 con 0x7f0f800ad270 2024-09-16T15:45:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.353+0000 7f0f77e00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f5c078eb0 0x7f0f5c07b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:51.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.353+0000 7f0f77e00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f5c078eb0 0x7f0f5c07b370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f800d0a40 tx=0x7f0f6800b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:51.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.356+0000 7f0f7d600640 1 -- 172.21.15.60:0/2780695969 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0f801f57f0 con 0x7f0f800ad270 2024-09-16T15:45:51.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.596+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0f800b0110 con 0x7f0f800ad270 2024-09-16T15:45:51.597 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:45:51.597+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:45:51.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.598+0000 7f0f7d600640 1 -- 172.21.15.60:0/2780695969 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f0f70062bf0 con 0x7f0f800ad270 2024-09-16T15:45:51.600 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:45:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f5c078eb0 msgr2=0x7f0f5c07b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f5c078eb0 0x7f0f5c07b370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f800d0a40 tx=0x7f0f6800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 msgr2=0x7f0f801f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 0x7f0f801f52b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000d740 tx=0x7f0f7000dc00 comp rx=0 tx=0).stop 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 shutdown_connections 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f5c078eb0 0x7f0f5c07b370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f8015f5a0 0x7f0f800cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f800ccf00 0x7f0f800cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 --2- 172.21.15.60:0/2780695969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f800ad270 0x7f0f801f52b0 secure :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000d740 tx=0x7f0f7000dc00 comp rx=0 tx=0).stop 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 >> 172.21.15.60:0/2780695969 conn(0x7f0f800b1950 msgr2=0x7f0f8009cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 shutdown_connections 2024-09-16T15:45:51.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.602+0000 7f0f87a00640 1 -- 172.21.15.60:0/2780695969 wait complete. 2024-09-16T15:45:51.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:45:51.671 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:45:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.783+0000 7f5abd000640 1 -- 172.21.15.60:0/223512383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ab80cbc00 msgr2=0x7f5ab8073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.783+0000 7f5abd000640 1 --2- 172.21.15.60:0/223512383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ab80cbc00 0x7f5ab8073ef0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0005fd0 tx=0x7f5ab002aaa0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.783+0000 7f5abd000640 1 -- 172.21.15.60:0/223512383 shutdown_connections 2024-09-16T15:45:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.783+0000 7f5abd000640 1 --2- 172.21.15.60:0/223512383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ab80cbc00 0x7f5ab8073ef0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.783+0000 7f5abd000640 1 --2- 172.21.15.60:0/223512383 >> v2:172.21.15.100:3300/0 conn(0x7f5ab815bec0 0x7f5ab806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.783+0000 7f5abd000640 1 -- 172.21.15.60:0/223512383 >> 172.21.15.60:0/223512383 conn(0x7f5ab80cf9f0 msgr2=0x7f5ab80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.783+0000 7f5abd000640 1 -- 172.21.15.60:0/223512383 shutdown_connections 2024-09-16T15:45:51.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.783+0000 7f5abd000640 1 -- 172.21.15.60:0/223512383 wait complete. 2024-09-16T15:45:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.785+0000 7f5abd000640 1 Processor -- start 2024-09-16T15:45:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.785+0000 7f5abd000640 1 -- start start 2024-09-16T15:45:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5abd000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ab8072560 0x7f5ab8150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5abd000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ab80cbc00 0x7f5ab80b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5abd000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ab815bec0 0x7f5ab80b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5abd000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab80ae990 con 0x7f5ab80cbc00 2024-09-16T15:45:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5abd000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab80ae810 con 0x7f5ab815bec0 2024-09-16T15:45:51.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5abd000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab80aeb10 con 0x7f5ab8072560 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5ab6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ab80cbc00 0x7f5ab80b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5ab6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ab80cbc00 0x7f5ab80b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56244/0 (socket says 172.21.15.60:56244) 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5ab6a00640 1 -- 172.21.15.60:0/331752966 learned_addr learned my addr 172.21.15.60:0/331752966 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5ab7e00640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ab815bec0 0x7f5ab80b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.786+0000 7f5ab7400640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ab8072560 0x7f5ab8150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5ab6a00640 1 -- 172.21.15.60:0/331752966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ab8072560 msgr2=0x7f5ab8150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5ab6a00640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ab8072560 0x7f5ab8150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5ab6a00640 1 -- 172.21.15.60:0/331752966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ab815bec0 msgr2=0x7f5ab80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5ab6a00640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ab815bec0 0x7f5ab80b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5ab6a00640 1 -- 172.21.15.60:0/331752966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab80c73c0 con 0x7f5ab80cbc00 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5ab6a00640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ab80cbc00 0x7f5ab80b0220 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c450 tx=0x7f5aa400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5aaf400640 1 -- 172.21.15.60:0/331752966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40052d0 con 0x7f5ab80cbc00 2024-09-16T15:45:51.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5aaf400640 1 -- 172.21.15.60:0/331752966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5aa4005470 con 0x7f5ab80cbc00 2024-09-16T15:45:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5aaf400640 1 -- 172.21.15.60:0/331752966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4012870 con 0x7f5ab80cbc00 2024-09-16T15:45:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab80c76a0 con 0x7f5ab80cbc00 2024-09-16T15:45:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.787+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab8157540 con 0x7f5ab80cbc00 2024-09-16T15:45:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.788+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab80aefc0 con 0x7f5ab80cbc00 2024-09-16T15:45:51.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.789+0000 7f5aaf400640 1 -- 172.21.15.60:0/331752966 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f5aa40035f0 con 0x7f5ab80cbc00 2024-09-16T15:45:51.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.790+0000 7f5aaf400640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5a98078e80 0x7f5a9807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.791+0000 7f5aaf400640 1 -- 172.21.15.60:0/331752966 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5823+0+0 (secure 0 0 0) 0x7f5aa409afd0 con 0x7f5ab80cbc00 2024-09-16T15:45:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.791+0000 7f5ab7400640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5a98078e80 0x7f5a9807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:51.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.791+0000 7f5ab7400640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5a98078e80 0x7f5a9807b340 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009b30 tx=0x7f5aa8005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:51.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:51.795+0000 7f5aaf400640 1 -- 172.21.15.60:0/331752966 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5aa4062580 con 0x7f5ab80cbc00 2024-09-16T15:45:52.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.032+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5ab80d00b0 con 0x7f5ab80cbc00 2024-09-16T15:45:52.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.603+0000 7f5aaf400640 1 -- 172.21.15.60:0/331752966 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) ==== 104+0+0 (secure 0 0 0) 0x7f5aa4066fb0 con 0x7f5ab80cbc00 2024-09-16T15:45:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5a98078e80 msgr2=0x7f5a9807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5a98078e80 0x7f5a9807b340 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009b30 tx=0x7f5aa8005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ab80cbc00 msgr2=0x7f5ab80b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ab80cbc00 0x7f5ab80b0220 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c450 tx=0x7f5aa400c910 comp rx=0 tx=0).stop 2024-09-16T15:45:52.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 shutdown_connections 2024-09-16T15:45:52.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5a98078e80 0x7f5a9807b340 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:52.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ab815bec0 0x7f5ab80b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:52.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ab80cbc00 0x7f5ab80b0220 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:52.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 --2- 172.21.15.60:0/331752966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ab8072560 0x7f5ab8150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:52.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 >> 172.21.15.60:0/331752966 conn(0x7f5ab80cf9f0 msgr2=0x7f5ab80b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:52.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 shutdown_connections 2024-09-16T15:45:52.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.607+0000 7f5abd000640 1 -- 172.21.15.60:0/331752966 wait complete. 2024-09-16T15:45:52.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:45:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.796+0000 7f82be800640 1 -- 172.21.15.60:0/3860579164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f829c003c10 msgr2=0x7f829c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.796+0000 7f82be800640 1 --2- 172.21.15.60:0/3860579164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f829c003c10 0x7f829c004010 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f82a8001800 tx=0x7f829c005830 comp rx=0 tx=0).stop 2024-09-16T15:45:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.796+0000 7f82b7e00640 1 -- 172.21.15.60:0/3860579164 reap_dead start 2024-09-16T15:45:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.796+0000 7f82be800640 1 -- 172.21.15.60:0/3860579164 shutdown_connections 2024-09-16T15:45:52.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.796+0000 7f82be800640 1 -- 172.21.15.60:0/3860579164 >> 172.21.15.60:0/3860579164 conn(0x7f82b80b18e0 msgr2=0x7f82b80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:52.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.796+0000 7f82be800640 1 -- 172.21.15.60:0/3860579164 shutdown_connections 2024-09-16T15:45:52.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.796+0000 7f82be800640 1 -- 172.21.15.60:0/3860579164 wait complete. 2024-09-16T15:45:52.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.797+0000 7f82be800640 1 Processor -- start 2024-09-16T15:45:52.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.798+0000 7f82be800640 1 -- start start 2024-09-16T15:45:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.798+0000 7f82be800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f829c003c10 0x7f82b81f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.798+0000 7f82be800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8160820 0x7f82b8153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.798+0000 7f82be800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82b809aa00 0x7f82b809ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.798+0000 7f82be800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f82b806f560 con 0x7f82b8160820 2024-09-16T15:45:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.798+0000 7f82be800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82b806f3e0 con 0x7f82b809aa00 2024-09-16T15:45:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.798+0000 7f82be800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f82b806f6e0 con 0x7f829c003c10 2024-09-16T15:45:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8160820 0x7f82b8153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:52.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8160820 0x7f82b8153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56268/0 (socket says 172.21.15.60:56268) 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 -- 172.21.15.60:0/3783852190 learned_addr learned my addr 172.21.15.60:0/3783852190 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82b7400640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f829c003c10 0x7f82b81f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82b7e00640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82b809aa00 0x7f82b809ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 -- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f829c003c10 msgr2=0x7f82b81f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f829c003c10 0x7f82b81f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 -- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82b809aa00 msgr2=0x7f82b809ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82b809aa00 0x7f82b809ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 -- 172.21.15.60:0/3783852190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b81f8970 con 0x7f82b8160820 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82b7e00640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82b809aa00 0x7f82b809ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.799+0000 7f82aea00640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8160820 0x7f82b8153460 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f82a0011a10 tx=0x7f82a0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.800+0000 7f82afe00640 1 -- 172.21.15.60:0/3783852190 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a000fd30 con 0x7f82b8160820 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.800+0000 7f82afe00640 1 -- 172.21.15.60:0/3783852190 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a0004030 con 0x7f82b8160820 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.800+0000 7f82afe00640 1 -- 172.21.15.60:0/3783852190 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0004480 con 0x7f82b8160820 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.800+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b81f80d0 con 0x7f82b8160820 2024-09-16T15:45:52.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.800+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b81f8690 con 0x7f82b8160820 2024-09-16T15:45:52.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.801+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b80b1ef0 con 0x7f82b8160820 2024-09-16T15:45:52.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.802+0000 7f82afe00640 1 -- 172.21.15.60:0/3783852190 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f82a0005020 con 0x7f82b8160820 2024-09-16T15:45:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.803+0000 7f82afe00640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8294078e80 0x7f829407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.803+0000 7f82afe00640 1 -- 172.21.15.60:0/3783852190 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5823+0+0 (secure 0 0 0) 0x7f82a009c290 con 0x7f82b8160820 2024-09-16T15:45:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.803+0000 7f82b7400640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8294078e80 0x7f829407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:52.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.804+0000 7f82b7400640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8294078e80 0x7f829407b340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f82a8001800 tx=0x7f82a80013b0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:52.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:52.807+0000 7f82afe00640 1 -- 172.21.15.60:0/3783852190 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f82a0063e40 con 0x7f82b8160820 2024-09-16T15:45:53.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.005+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f82b8069a50 con 0x7f82b8160820 2024-09-16T15:45:53.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.605+0000 7f82afe00640 1 -- 172.21.15.60:0/3783852190 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) ==== 181+0+0 (secure 0 0 0) 0x7f82a0063740 con 0x7f82b8160820 2024-09-16T15:45:53.608 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:45:53.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8294078e80 msgr2=0x7f829407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:53.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8294078e80 0x7f829407b340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f82a8001800 tx=0x7f82a80013b0 comp rx=0 tx=0).stop 2024-09-16T15:45:53.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8160820 msgr2=0x7f82b8153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:53.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8160820 0x7f82b8153460 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f82a0011a10 tx=0x7f82a0011ed0 comp rx=0 tx=0).stop 2024-09-16T15:45:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 shutdown_connections 2024-09-16T15:45:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8294078e80 0x7f829407b340 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82b809aa00 0x7f82b809ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f82b8160820 0x7f82b8153460 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 --2- 172.21.15.60:0/3783852190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f829c003c10 0x7f82b81f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 >> 172.21.15.60:0/3783852190 conn(0x7f82b80b18e0 msgr2=0x7f82b8072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.609+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 shutdown_connections 2024-09-16T15:45:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.610+0000 7f82be800640 1 -- 172.21.15.60:0/3783852190 wait complete. 2024-09-16T15:45:53.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:45:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.798+0000 7f0823e00640 1 -- 172.21.15.60:0/2558639504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08080051c0 msgr2=0x7f080800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.798+0000 7f0823e00640 1 --2- 172.21.15.60:0/2558639504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08080051c0 0x7f080800b9c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f08240b3790 tx=0x7f0808005830 comp rx=0 tx=0).stop 2024-09-16T15:45:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.798+0000 7f0823400640 1 -- 172.21.15.60:0/2558639504 reap_dead start 2024-09-16T15:45:53.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.798+0000 7f0823e00640 1 -- 172.21.15.60:0/2558639504 shutdown_connections 2024-09-16T15:45:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.798+0000 7f0823e00640 1 -- 172.21.15.60:0/2558639504 >> 172.21.15.60:0/2558639504 conn(0x7f082409d180 msgr2=0x7f08240d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.798+0000 7f0823e00640 1 -- 172.21.15.60:0/2558639504 shutdown_connections 2024-09-16T15:45:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.798+0000 7f0823e00640 1 -- 172.21.15.60:0/2558639504 wait complete. 2024-09-16T15:45:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.800+0000 7f0823e00640 1 Processor -- start 2024-09-16T15:45:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.800+0000 7f0823e00640 1 -- start start 2024-09-16T15:45:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08080051c0 0x7f082415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08240cf910 0x7f08241f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08240d0770 0x7f08240c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f08240b1cf0 con 0x7f08240d0770 2024-09-16T15:45:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f08240b1b70 con 0x7f08080051c0 2024-09-16T15:45:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f08240b1e70 con 0x7f08240cf910 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08240d0770 0x7f08240c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08240d0770 0x7f08240c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56292/0 (socket says 172.21.15.60:56292) 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823400640 1 -- 172.21.15.60:0/525718579 learned_addr learned my addr 172.21.15.60:0/525718579 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0822a00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08080051c0 0x7f082415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f081a000640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08240cf910 0x7f08241f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823400640 1 -- 172.21.15.60:0/525718579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08240cf910 msgr2=0x7f08241f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823400640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08240cf910 0x7f08241f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823400640 1 -- 172.21.15.60:0/525718579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08080051c0 msgr2=0x7f082415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823400640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08080051c0 0x7f082415e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.801+0000 7f0823400640 1 -- 172.21.15.60:0/525718579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08240b7700 con 0x7f08240d0770 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.802+0000 7f0823400640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08240d0770 0x7f08240c7b60 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f081c02b1e0 tx=0x7f081c02ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:53.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.802+0000 7f081b400640 1 -- 172.21.15.60:0/525718579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c02ce60 con 0x7f08240d0770 2024-09-16T15:45:53.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.802+0000 7f081b400640 1 -- 172.21.15.60:0/525718579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f081c036650 con 0x7f08240d0770 2024-09-16T15:45:53.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.802+0000 7f081b400640 1 -- 172.21.15.60:0/525718579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c0099a0 con 0x7f08240d0770 2024-09-16T15:45:53.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.802+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08240d10a0 con 0x7f08240d0770 2024-09-16T15:45:53.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.802+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08240b7c80 con 0x7f08240d0770 2024-09-16T15:45:53.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.803+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08240d47e0 con 0x7f08240d0770 2024-09-16T15:45:53.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.804+0000 7f081b400640 1 -- 172.21.15.60:0/525718579 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f081c036aa0 con 0x7f08240d0770 2024-09-16T15:45:53.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.805+0000 7f081b400640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0800078eb0 0x7f080007b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:53.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.805+0000 7f081b400640 1 -- 172.21.15.60:0/525718579 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5385+0+0 (secure 0 0 0) 0x7f081c0bb350 con 0x7f08240d0770 2024-09-16T15:45:53.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.806+0000 7f0822a00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0800078eb0 0x7f080007b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:53.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.806+0000 7f0822a00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0800078eb0 0x7f080007b370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0814009cd0 tx=0x7f0814005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:53.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:53.810+0000 7f081b400640 1 -- 172.21.15.60:0/525718579 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f081c082990 con 0x7f08240d0770 2024-09-16T15:45:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.004+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f082408d630 con 0x7f08240d0770 2024-09-16T15:45:54.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.611+0000 7f081b400640 1 -- 172.21.15.60:0/525718579 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) ==== 169+0+0 (secure 0 0 0) 0x7f081c0873c0 con 0x7f08240d0770 2024-09-16T15:45:54.613 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.615+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0800078eb0 msgr2=0x7f080007b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.615+0000 7f0823e00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0800078eb0 0x7f080007b370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0814009cd0 tx=0x7f0814005f90 comp rx=0 tx=0).stop 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.615+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08240d0770 msgr2=0x7f08240c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.615+0000 7f0823e00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08240d0770 0x7f08240c7b60 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f081c02b1e0 tx=0x7f081c02ccf0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.616+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 shutdown_connections 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.616+0000 7f0823e00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0800078eb0 0x7f080007b370 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.616+0000 7f0823e00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08240d0770 0x7f08240c7b60 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.616+0000 7f0823e00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f08240cf910 0x7f08241f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.616+0000 7f0823e00640 1 --2- 172.21.15.60:0/525718579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08080051c0 0x7f082415e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.616+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 >> 172.21.15.60:0/525718579 conn(0x7f082409d180 msgr2=0x7f08240cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:54.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.616+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 shutdown_connections 2024-09-16T15:45:54.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.616+0000 7f0823e00640 1 -- 172.21.15.60:0/525718579 wait complete. 2024-09-16T15:45:54.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:45:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.796+0000 7fb0b1a00640 1 -- 172.21.15.60:0/3402058391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac073c20 msgr2=0x7fb0ac154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.796+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/3402058391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac073c20 0x7fb0ac154720 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4005fd0 tx=0x7fb0a402a8b0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.796+0000 7fb0b1a00640 1 -- 172.21.15.60:0/3402058391 shutdown_connections 2024-09-16T15:45:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.796+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/3402058391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac073c20 0x7fb0ac154720 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.796+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/3402058391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0ac15bb90 0x7fb0ac0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.796+0000 7fb0b1a00640 1 -- 172.21.15.60:0/3402058391 >> 172.21.15.60:0/3402058391 conn(0x7fb0ac0d0b30 msgr2=0x7fb0ac0d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.797+0000 7fb0b1a00640 1 -- 172.21.15.60:0/3402058391 shutdown_connections 2024-09-16T15:45:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.797+0000 7fb0b1a00640 1 -- 172.21.15.60:0/3402058391 wait complete. 2024-09-16T15:45:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.798+0000 7fb0b1a00640 1 Processor -- start 2024-09-16T15:45:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.798+0000 7fb0b1a00640 1 -- start start 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.799+0000 7fb0b1a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac071130 0x7fb0ac1490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.799+0000 7fb0b1a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0ac073c20 0x7fb0ac1f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.799+0000 7fb0b1a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0ac15bb90 0x7fb0ac0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.799+0000 7fb0b1a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac154de0 con 0x7fb0ac071130 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.799+0000 7fb0b1a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac154c60 con 0x7fb0ac073c20 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.799+0000 7fb0b1a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac154f60 con 0x7fb0ac15bb90 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac071130 0x7fb0ac1490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac071130 0x7fb0ac1490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56320/0 (socket says 172.21.15.60:56320) 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 -- 172.21.15.60:0/791635646 learned_addr learned my addr 172.21.15.60:0/791635646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0b1000640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0ac15bb90 0x7fb0ac0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0a3400640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0ac073c20 0x7fb0ac1f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 -- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0ac15bb90 msgr2=0x7fb0ac0c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0ac15bb90 0x7fb0ac0c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 -- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0ac073c20 msgr2=0x7fb0ac1f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0ac073c20 0x7fb0ac1f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 -- 172.21.15.60:0/791635646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac1f4ab0 con 0x7fb0ac071130 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0a3400640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0ac073c20 0x7fb0ac1f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.800+0000 7fb0abe00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac071130 0x7fb0ac1490e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c450 tx=0x7fb09c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.801+0000 7fb0a8c00640 1 -- 172.21.15.60:0/791635646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c016020 con 0x7fb0ac071130 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.801+0000 7fb0a8c00640 1 -- 172.21.15.60:0/791635646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb09c012920 con 0x7fb0ac071130 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.801+0000 7fb0a8c00640 1 -- 172.21.15.60:0/791635646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c012d70 con 0x7fb0ac071130 2024-09-16T15:45:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.801+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac1f4d90 con 0x7fb0ac071130 2024-09-16T15:45:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.801+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac149620 con 0x7fb0ac071130 2024-09-16T15:45:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.802+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac0cb320 con 0x7fb0ac071130 2024-09-16T15:45:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.803+0000 7fb0a8c00640 1 -- 172.21.15.60:0/791635646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fb09c019020 con 0x7fb0ac071130 2024-09-16T15:45:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.804+0000 7fb0a8c00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb088078e80 0x7fb08807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.804+0000 7fb0a8c00640 1 -- 172.21.15.60:0/791635646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4971+0+0 (secure 0 0 0) 0x7fb09c09b1b0 con 0x7fb0ac071130 2024-09-16T15:45:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.804+0000 7fb0a3400640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb088078e80 0x7fb08807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:54.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.805+0000 7fb0a3400640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb088078e80 0x7fb08807b340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb094005fd0 tx=0x7fb09400f360 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:54.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:54.808+0000 7fb0a8c00640 1 -- 172.21.15.60:0/791635646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb0ac0cb320 con 0x7fb0ac071130 2024-09-16T15:45:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.003+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fb0ac06a180 con 0x7fb0ac071130 2024-09-16T15:45:55.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.007+0000 7fb0a8c00640 1 -- 172.21.15.60:0/791635646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7fb0ac06a180 con 0x7fb0ac071130 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.009+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb088078e80 msgr2=0x7fb08807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.009+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb088078e80 0x7fb08807b340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb094005fd0 tx=0x7fb09400f360 comp rx=0 tx=0).stop 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.009+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac071130 msgr2=0x7fb0ac1490e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.009+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac071130 0x7fb0ac1490e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c450 tx=0x7fb09c00c910 comp rx=0 tx=0).stop 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.010+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 shutdown_connections 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.010+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb088078e80 0x7fb08807b340 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.010+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb0ac15bb90 0x7fb0ac0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.010+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0ac073c20 0x7fb0ac1f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.010+0000 7fb0b1a00640 1 --2- 172.21.15.60:0/791635646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0ac071130 0x7fb0ac1490e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.010+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 >> 172.21.15.60:0/791635646 conn(0x7fb0ac0d0b30 msgr2=0x7fb0ac0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.010+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 shutdown_connections 2024-09-16T15:45:55.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.010+0000 7fb0b1a00640 1 -- 172.21.15.60:0/791635646 wait complete. 2024-09-16T15:45:55.027 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-16T15:45:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.186+0000 7f7092400640 1 -- 172.21.15.60:0/327180184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070003a50 msgr2=0x7f7070003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.186+0000 7f7092400640 1 --2- 172.21.15.60:0/327180184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070003a50 0x7f7070003e70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f707c001690 tx=0x7f7070005830 comp rx=0 tx=0).stop 2024-09-16T15:45:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.186+0000 7f7092400640 1 -- 172.21.15.60:0/327180184 shutdown_connections 2024-09-16T15:45:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.186+0000 7f7092400640 1 --2- 172.21.15.60:0/327180184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070003a50 0x7f7070003e70 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.186+0000 7f7092400640 1 -- 172.21.15.60:0/327180184 >> 172.21.15.60:0/327180184 conn(0x7f708c0b1950 msgr2=0x7f708c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.186+0000 7f7092400640 1 -- 172.21.15.60:0/327180184 shutdown_connections 2024-09-16T15:45:55.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.186+0000 7f7092400640 1 -- 172.21.15.60:0/327180184 wait complete. 2024-09-16T15:45:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.188+0000 7f7092400640 1 Processor -- start 2024-09-16T15:45:55.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.188+0000 7f7092400640 1 -- start start 2024-09-16T15:45:55.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f7092400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7070003a50 0x7f708c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f7092400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708c1f4a50 0x7f708c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f7092400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c159650 0x7f708c0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f7092400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f708c0ad0c0 con 0x7f708c1f4a50 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f7092400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f708c0acf40 con 0x7f7070003a50 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f7092400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f708c0ad240 con 0x7f708c159650 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f7082a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708c1f4a50 0x7f708c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f708b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7070003a50 0x7f708c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f708b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7070003a50 0x7f708c15b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37170/0 (socket says 172.21.15.60:37170) 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f7082a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708c1f4a50 0x7f708c1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56336/0 (socket says 172.21.15.60:56336) 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f708b400640 1 -- 172.21.15.60:0/2175552179 learned_addr learned my addr 172.21.15.60:0/2175552179 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.189+0000 7f708be00640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c159650 0x7f708c0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:55.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7082a00640 1 -- 172.21.15.60:0/2175552179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c159650 msgr2=0x7f708c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7082a00640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c159650 0x7f708c0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7082a00640 1 -- 172.21.15.60:0/2175552179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7070003a50 msgr2=0x7f708c15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7082a00640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7070003a50 0x7f708c15b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7082a00640 1 -- 172.21.15.60:0/2175552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c1f51d0 con 0x7f708c1f4a50 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7082a00640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708c1f4a50 0x7f708c1576e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f708c064710 tx=0x7f70740039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7083e00640 1 -- 172.21.15.60:0/2175552179 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074015070 con 0x7f708c1f4a50 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7083e00640 1 -- 172.21.15.60:0/2175552179 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7074011880 con 0x7f708c1f4a50 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7083e00640 1 -- 172.21.15.60:0/2175552179 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074011cd0 con 0x7f708c1f4a50 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c1f54b0 con 0x7f708c1f4a50 2024-09-16T15:45:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.190+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c1f4ed0 con 0x7f708c1f4a50 2024-09-16T15:45:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.192+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c0cdae0 con 0x7f708c1f4a50 2024-09-16T15:45:55.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.193+0000 7f7083e00640 1 -- 172.21.15.60:0/2175552179 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f7074018070 con 0x7f708c1f4a50 2024-09-16T15:45:55.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.194+0000 7f7083e00640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7068078e80 0x7f706807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:55.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.194+0000 7f7083e00640 1 -- 172.21.15.60:0/2175552179 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4971+0+0 (secure 0 0 0) 0x7f707409beb0 con 0x7f708c1f4a50 2024-09-16T15:45:55.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.194+0000 7f708b400640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7068078e80 0x7f706807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:55.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.195+0000 7f708b400640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7068078e80 0x7f706807b340 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c02c040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:55.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.198+0000 7f7083e00640 1 -- 172.21.15.60:0/2175552179 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7074063720 con 0x7f708c1f4a50 2024-09-16T15:45:55.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.476+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f708c0b4ac0 con 0x7f708c1f4a50 2024-09-16T15:45:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.613+0000 7f7083e00640 1 -- 172.21.15.60:0/2175552179 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v406) ==== 148+0+0 (secure 0 0 0) 0x7f7074068150 con 0x7f708c1f4a50 2024-09-16T15:45:55.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7068078e80 msgr2=0x7f706807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7068078e80 0x7f706807b340 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c02c040 comp rx=0 tx=0).stop 2024-09-16T15:45:55.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708c1f4a50 msgr2=0x7f708c1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708c1f4a50 0x7f708c1576e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f708c064710 tx=0x7f70740039c0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 shutdown_connections 2024-09-16T15:45:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7068078e80 0x7f706807b340 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c159650 0x7f708c0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f708c1f4a50 0x7f708c1576e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 --2- 172.21.15.60:0/2175552179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7070003a50 0x7f708c15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 >> 172.21.15.60:0/2175552179 conn(0x7f708c0b1950 msgr2=0x7f708c0afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 shutdown_connections 2024-09-16T15:45:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.616+0000 7f7092400640 1 -- 172.21.15.60:0/2175552179 wait complete. 2024-09-16T15:45:55.633 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-16T15:45:55.634 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:45:55.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:45:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.793+0000 7f37afe00640 1 -- 172.21.15.60:0/1686568692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00d0540 msgr2=0x7f37b006f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.793+0000 7f37afe00640 1 --2- 172.21.15.60:0/1686568692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00d0540 0x7f37b006f320 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005fd0 tx=0x7f37a802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.793+0000 7f37afe00640 1 -- 172.21.15.60:0/1686568692 shutdown_connections 2024-09-16T15:45:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.793+0000 7f37afe00640 1 --2- 172.21.15.60:0/1686568692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00d0540 0x7f37b006f320 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.793+0000 7f37afe00640 1 --2- 172.21.15.60:0/1686568692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37b015bc30 0x7f37b00b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.793+0000 7f37afe00640 1 -- 172.21.15.60:0/1686568692 >> 172.21.15.60:0/1686568692 conn(0x7f37b00cfa20 msgr2=0x7f37b00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.793+0000 7f37afe00640 1 -- 172.21.15.60:0/1686568692 shutdown_connections 2024-09-16T15:45:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.793+0000 7f37afe00640 1 -- 172.21.15.60:0/1686568692 wait complete. 2024-09-16T15:45:55.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.795+0000 7f37afe00640 1 Processor -- start 2024-09-16T15:45:55.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.795+0000 7f37afe00640 1 -- start start 2024-09-16T15:45:55.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37afe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b00d0540 0x7f37b0150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:55.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37afe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37b015bc30 0x7f37b00af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:55.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37afe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00af670 0x7f37b00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37afe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0074660 con 0x7f37b00af670 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37afe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37b015c200 con 0x7f37b00d0540 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37afe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f37b00afb30 con 0x7f37b015bc30 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37af400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00af670 0x7f37b00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37af400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00af670 0x7f37b00c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56346/0 (socket says 172.21.15.60:56346) 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37af400640 1 -- 172.21.15.60:0/1360384737 learned_addr learned my addr 172.21.15.60:0/1360384737 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37a6000640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37b015bc30 0x7f37b00af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37aea00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b00d0540 0x7f37b0150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37af400640 1 -- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37b015bc30 msgr2=0x7f37b00af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37af400640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37b015bc30 0x7f37b00af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37af400640 1 -- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b00d0540 msgr2=0x7f37b0150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37af400640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b00d0540 0x7f37b0150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37af400640 1 -- 172.21.15.60:0/1360384737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b00afe10 con 0x7f37b00af670 2024-09-16T15:45:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.796+0000 7f37aea00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b00d0540 0x7f37b0150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.797+0000 7f37af400640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00af670 0x7f37b00c8cc0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f37a8006010 tx=0x7f37a8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:55.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.797+0000 7f37a7e00640 1 -- 172.21.15.60:0/1360384737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8040070 con 0x7f37b00af670 2024-09-16T15:45:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.797+0000 7f37a7e00640 1 -- 172.21.15.60:0/1360384737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37a8004130 con 0x7f37b00af670 2024-09-16T15:45:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.797+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b00b00f0 con 0x7f37b00af670 2024-09-16T15:45:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.797+0000 7f37a7e00640 1 -- 172.21.15.60:0/1360384737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8004540 con 0x7f37b00af670 2024-09-16T15:45:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.797+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b00b06b0 con 0x7f37b00af670 2024-09-16T15:45:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.798+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b006fc40 con 0x7f37b00af670 2024-09-16T15:45:55.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.800+0000 7f37a7e00640 1 -- 172.21.15.60:0/1360384737 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f37a8039040 con 0x7f37b00af670 2024-09-16T15:45:55.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.800+0000 7f37a7e00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f378c078eb0 0x7f378c07b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.801+0000 7f37a7e00640 1 -- 172.21.15.60:0/1360384737 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4971+0+0 (secure 0 0 0) 0x7f37a80bc7b0 con 0x7f37b00af670 2024-09-16T15:45:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.801+0000 7f37aea00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f378c078eb0 0x7f378c07b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.801+0000 7f37aea00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f378c078eb0 0x7f378c07b370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f37a0005fd0 tx=0x7f37a000f360 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:55.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:55.805+0000 7f37a7e00640 1 -- 172.21.15.60:0/1360384737 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f37a80847e0 con 0x7f37b00af670 2024-09-16T15:45:56.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.007+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37b00b0880 con 0x7f37b00af670 2024-09-16T15:45:56.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.008+0000 7f37a7e00640 1 -- 172.21.15.60:0/1360384737 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+13084 (secure 0 0 0) 0x7f37a80840e0 con 0x7f37b00af670 2024-09-16T15:45:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:56.011 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":128,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:45:54.607572+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/678119606":"2024-09-17T15:45:51.597340+0000","172.21.15.60:6834/678119606":"2024-09-17T15:45:51.597340+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:45:56.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f378c078eb0 msgr2=0x7f378c07b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f378c078eb0 0x7f378c07b370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f37a0005fd0 tx=0x7f37a000f360 comp rx=0 tx=0).stop 2024-09-16T15:45:56.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00af670 msgr2=0x7f37b00c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00af670 0x7f37b00c8cc0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f37a8006010 tx=0x7f37a8003970 comp rx=0 tx=0).stop 2024-09-16T15:45:56.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 shutdown_connections 2024-09-16T15:45:56.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f378c078eb0 0x7f378c07b370 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37b00af670 0x7f37b00c8cc0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37b015bc30 0x7f37b00af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 --2- 172.21.15.60:0/1360384737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37b00d0540 0x7f37b0150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 >> 172.21.15.60:0/1360384737 conn(0x7f37b00cfa20 msgr2=0x7f37b00c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:56.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 shutdown_connections 2024-09-16T15:45:56.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.011+0000 7f37afe00640 1 -- 172.21.15.60:0/1360384737 wait complete. 2024-09-16T15:45:56.030 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:45:56.030 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:45:56.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 -- 172.21.15.60:0/1826086679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 msgr2=0x7ff75c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 --2- 172.21.15.60:0/1826086679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 0x7ff75c073440 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0af750 tx=0x7ff74c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 -- 172.21.15.60:0/1826086679 shutdown_connections 2024-09-16T15:45:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 --2- 172.21.15.60:0/1826086679 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75c0ccb00 0x7ff75c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 --2- 172.21.15.60:0/1826086679 >> v2:172.21.15.100:3300/0 conn(0x7ff75c06efa0 0x7ff75c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 --2- 172.21.15.60:0/1826086679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 0x7ff75c073440 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 -- 172.21.15.60:0/1826086679 >> 172.21.15.60:0/1826086679 conn(0x7ff75c0b1840 msgr2=0x7ff75c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 -- 172.21.15.60:0/1826086679 shutdown_connections 2024-09-16T15:45:56.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.192+0000 7ff762400640 1 -- 172.21.15.60:0/1826086679 wait complete. 2024-09-16T15:45:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.194+0000 7ff762400640 1 Processor -- start 2024-09-16T15:45:56.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.194+0000 7ff762400640 1 -- start start 2024-09-16T15:45:56.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff762400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75c06efa0 0x7ff75c143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff762400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75c0ccb00 0x7ff75c0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff762400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 0x7ff75c13fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff762400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c1602c0 con 0x7ff75c145750 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff762400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c160140 con 0x7ff75c0ccb00 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff762400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff75c160440 con 0x7ff75c06efa0 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff75be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 0x7ff75c13fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff75be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 0x7ff75c13fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56368/0 (socket says 172.21.15.60:56368) 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff75be00640 1 -- 172.21.15.60:0/375889633 learned_addr learned my addr 172.21.15.60:0/375889633 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff75b400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75c06efa0 0x7ff75c143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:56.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.195+0000 7ff752a00640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75c0ccb00 0x7ff75c0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:56.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff75be00640 1 -- 172.21.15.60:0/375889633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75c06efa0 msgr2=0x7ff75c143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff75be00640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75c06efa0 0x7ff75c143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff75be00640 1 -- 172.21.15.60:0/375889633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75c0ccb00 msgr2=0x7ff75c0b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff75be00640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75c0ccb00 0x7ff75c0b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff75be00640 1 -- 172.21.15.60:0/375889633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c144030 con 0x7ff75c145750 2024-09-16T15:45:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff75be00640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 0x7ff75c13fe90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff75400eaa0 tx=0x7ff75400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff753e00640 1 -- 172.21.15.60:0/375889633 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75400cdb0 con 0x7ff75c145750 2024-09-16T15:45:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff753e00640 1 -- 172.21.15.60:0/375889633 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff754004c10 con 0x7ff75c145750 2024-09-16T15:45:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff753e00640 1 -- 172.21.15.60:0/375889633 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754005020 con 0x7ff75c145750 2024-09-16T15:45:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c1f4c70 con 0x7ff75c145750 2024-09-16T15:45:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.196+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c1f5230 con 0x7ff75c145750 2024-09-16T15:45:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.198+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c15f9f0 con 0x7ff75c145750 2024-09-16T15:45:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.199+0000 7ff753e00640 1 -- 172.21.15.60:0/375889633 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7ff7540051c0 con 0x7ff75c145750 2024-09-16T15:45:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.200+0000 7ff753e00640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff738078e80 0x7ff73807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.200+0000 7ff753e00640 1 -- 172.21.15.60:0/375889633 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4971+0+0 (secure 0 0 0) 0x7ff75409bc40 con 0x7ff75c145750 2024-09-16T15:45:56.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.200+0000 7ff75b400640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff738078e80 0x7ff73807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.201+0000 7ff75b400640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff738078e80 0x7ff73807b340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff74c033000 tx=0x7ff74c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:56.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.204+0000 7ff753e00640 1 -- 172.21.15.60:0/375889633 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff754016030 con 0x7ff75c145750 2024-09-16T15:45:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.399+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff75c0b0f70 con 0x7ff75c145750 2024-09-16T15:45:56.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.399+0000 7ff753e00640 1 -- 172.21.15.60:0/375889633 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7ff754063430 con 0x7ff75c145750 2024-09-16T15:45:56.401 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:45:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff738078e80 msgr2=0x7ff73807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff738078e80 0x7ff73807b340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff74c033000 tx=0x7ff74c004730 comp rx=0 tx=0).stop 2024-09-16T15:45:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 msgr2=0x7ff75c13fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 0x7ff75c13fe90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff75400eaa0 tx=0x7ff75400ef60 comp rx=0 tx=0).stop 2024-09-16T15:45:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 shutdown_connections 2024-09-16T15:45:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff738078e80 0x7ff73807b340 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff75c145750 0x7ff75c13fe90 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff75c0ccb00 0x7ff75c0b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 --2- 172.21.15.60:0/375889633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75c06efa0 0x7ff75c143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.401+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 >> 172.21.15.60:0/375889633 conn(0x7ff75c0b1840 msgr2=0x7ff75c0b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.402+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 shutdown_connections 2024-09-16T15:45:56.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.402+0000 7ff762400640 1 -- 172.21.15.60:0/375889633 wait complete. 2024-09-16T15:45:56.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-16T15:45:56.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.578+0000 7f60f7200640 1 -- 172.21.15.60:0/3622996327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 msgr2=0x7f60f806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.578+0000 7f60f7200640 1 --2- 172.21.15.60:0/3622996327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 0x7f60f806f320 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f60f0005fd0 tx=0x7f60f002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.578+0000 7f60f7200640 1 -- 172.21.15.60:0/3622996327 shutdown_connections 2024-09-16T15:45:56.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.578+0000 7f60f7200640 1 --2- 172.21.15.60:0/3622996327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 0x7f60f806f320 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.578+0000 7f60f7200640 1 --2- 172.21.15.60:0/3622996327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60f80725d0 0x7f60f80b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.578+0000 7f60f7200640 1 -- 172.21.15.60:0/3622996327 >> 172.21.15.60:0/3622996327 conn(0x7f60f80cfa20 msgr2=0x7f60f80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:56.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.578+0000 7f60f7200640 1 -- 172.21.15.60:0/3622996327 shutdown_connections 2024-09-16T15:45:56.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.578+0000 7f60f7200640 1 -- 172.21.15.60:0/3622996327 wait complete. 2024-09-16T15:45:56.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.580+0000 7f60f7200640 1 Processor -- start 2024-09-16T15:45:56.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.580+0000 7f60f7200640 1 -- start start 2024-09-16T15:45:56.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 0x7f60f81f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60f815bc30 0x7f60f81f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f7200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60f80b3900 0x7f60f80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f7200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8074660 con 0x7f60f80d0540 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f60f815c200 con 0x7f60f815bc30 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f7200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f60f81f5aa0 con 0x7f60f80b3900 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 0x7f60f81f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 0x7f60f81f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56388/0 (socket says 172.21.15.60:56388) 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f5e00640 1 -- 172.21.15.60:0/1471068628 learned_addr learned my addr 172.21.15.60:0/1471068628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60ed400640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60f815bc30 0x7f60f81f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f6800640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60f80b3900 0x7f60f80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.581+0000 7f60f5e00640 1 -- 172.21.15.60:0/1471068628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60f80b3900 msgr2=0x7f60f80cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60f5e00640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60f80b3900 0x7f60f80cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60f5e00640 1 -- 172.21.15.60:0/1471068628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60f815bc30 msgr2=0x7f60f81f7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:56.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60f5e00640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60f815bc30 0x7f60f81f7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:56.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60f5e00640 1 -- 172.21.15.60:0/1471068628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f81f4be0 con 0x7f60f80d0540 2024-09-16T15:45:56.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60f5e00640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 0x7f60f81f45a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f60e800c3d0 tx=0x7f60e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:56.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60eea00640 1 -- 172.21.15.60:0/1471068628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8011790 con 0x7f60f80d0540 2024-09-16T15:45:56.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60eea00640 1 -- 172.21.15.60:0/1471068628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60e8017070 con 0x7f60f80d0540 2024-09-16T15:45:56.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60eea00640 1 -- 172.21.15.60:0/1471068628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8011cc0 con 0x7f60f80d0540 2024-09-16T15:45:56.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f81f4ec0 con 0x7f60f80d0540 2024-09-16T15:45:56.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.582+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81f5480 con 0x7f60f80d0540 2024-09-16T15:45:56.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.583+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f81f57c0 con 0x7f60f80d0540 2024-09-16T15:45:56.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.584+0000 7f60eea00640 1 -- 172.21.15.60:0/1471068628 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f60e801c460 con 0x7f60f80d0540 2024-09-16T15:45:56.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.585+0000 7f60eea00640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60d4078e80 0x7f60d407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:56.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.585+0000 7f60eea00640 1 -- 172.21.15.60:0/1471068628 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4971+0+0 (secure 0 0 0) 0x7f60e809af10 con 0x7f60f80d0540 2024-09-16T15:45:56.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.586+0000 7f60ed400640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60d4078e80 0x7f60d407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:56.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.586+0000 7f60ed400640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60d4078e80 0x7f60d407b340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f60e0005d80 tx=0x7f60e0005c50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:56.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.589+0000 7f60eea00640 1 -- 172.21.15.60:0/1471068628 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f60f81f57c0 con 0x7f60f80d0540 2024-09-16T15:45:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:56.855+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f60f806fc40 con 0x7f60f80d0540 2024-09-16T15:45:57.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.621+0000 7f60eea00640 1 -- 172.21.15.60:0/1471068628 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v408) ==== 154+0+0 (secure 0 0 0) 0x7f60e8062ea0 con 0x7f60f80d0540 2024-09-16T15:45:57.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60d4078e80 msgr2=0x7f60d407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:57.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60d4078e80 0x7f60d407b340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f60e0005d80 tx=0x7f60e0005c50 comp rx=0 tx=0).stop 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 msgr2=0x7f60f81f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 0x7f60f81f45a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f60e800c3d0 tx=0x7f60e800c890 comp rx=0 tx=0).stop 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 shutdown_connections 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f60d4078e80 0x7f60d407b340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60f80b3900 0x7f60f80cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60f815bc30 0x7f60f81f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 --2- 172.21.15.60:0/1471068628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60f80d0540 0x7f60f81f45a0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 >> 172.21.15.60:0/1471068628 conn(0x7f60f80cfa20 msgr2=0x7f60f80ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 shutdown_connections 2024-09-16T15:45:57.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.624+0000 7f60f7200640 1 -- 172.21.15.60:0/1471068628 wait complete. 2024-09-16T15:45:57.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:45:57.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.809+0000 7fa1cb200640 1 -- 172.21.15.60:0/2879119367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 msgr2=0x7fa1cc0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:57.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.809+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2879119367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 0x7fa1cc0b5370 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0b6930 tx=0x7fa1bc02a920 comp rx=0 tx=0).stop 2024-09-16T15:45:57.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.809+0000 7fa1cb200640 1 -- 172.21.15.60:0/2879119367 shutdown_connections 2024-09-16T15:45:57.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.809+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2879119367 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc0b4050 0x7fa1cc0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:57.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.809+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2879119367 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 0x7fa1cc0b5370 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:57.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.809+0000 7fa1cb200640 1 -- 172.21.15.60:0/2879119367 >> 172.21.15.60:0/2879119367 conn(0x7fa1cc0d3df0 msgr2=0x7fa1cc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.809+0000 7fa1cb200640 1 -- 172.21.15.60:0/2879119367 shutdown_connections 2024-09-16T15:45:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.809+0000 7fa1cb200640 1 -- 172.21.15.60:0/2879119367 wait complete. 2024-09-16T15:45:57.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.810+0000 7fa1cb200640 1 Processor -- start 2024-09-16T15:45:57.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.811+0000 7fa1cb200640 1 -- start start 2024-09-16T15:45:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.811+0000 7fa1cb200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc0b34b0 0x7fa1cc0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.811+0000 7fa1cb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc0b4050 0x7fa1cc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.811+0000 7fa1cb200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 0x7fa1cc153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.811+0000 7fa1cb200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc0b2040 con 0x7fa1cc161440 2024-09-16T15:45:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.811+0000 7fa1cb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc0b1ec0 con 0x7fa1cc0b4050 2024-09-16T15:45:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.811+0000 7fa1cb200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa1cc0b21c0 con 0x7fa1cc0b34b0 2024-09-16T15:45:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 0x7fa1cc153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 0x7fa1cc153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56408/0 (socket says 172.21.15.60:56408) 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 -- 172.21.15.60:0/2729299011 learned_addr learned my addr 172.21.15.60:0/2729299011 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1c9e00640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc0b34b0 0x7fa1cc0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1c1400640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc0b4050 0x7fa1cc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 -- 172.21.15.60:0/2729299011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc0b34b0 msgr2=0x7fa1cc0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc0b34b0 0x7fa1cc0b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 -- 172.21.15.60:0/2729299011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc0b4050 msgr2=0x7fa1cc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc0b4050 0x7fa1cc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 -- 172.21.15.60:0/2729299011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc0cb380 con 0x7fa1cc161440 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.812+0000 7fa1ca800640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 0x7fa1cc153150 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400c970 tx=0x7fa1c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.813+0000 7fa1c2a00640 1 -- 172.21.15.60:0/2729299011 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4007bb0 con 0x7fa1cc161440 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.813+0000 7fa1c2a00640 1 -- 172.21.15.60:0/2729299011 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c400f450 con 0x7fa1cc161440 2024-09-16T15:45:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.813+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc0b72e0 con 0x7fa1cc161440 2024-09-16T15:45:57.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.813+0000 7fa1c2a00640 1 -- 172.21.15.60:0/2729299011 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4007e60 con 0x7fa1cc161440 2024-09-16T15:45:57.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.813+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc0b78a0 con 0x7fa1cc161440 2024-09-16T15:45:57.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.814+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1cc0b1770 con 0x7fa1cc161440 2024-09-16T15:45:57.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.815+0000 7fa1c2a00640 1 -- 172.21.15.60:0/2729299011 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fa1c400f8a0 con 0x7fa1cc161440 2024-09-16T15:45:57.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.816+0000 7fa1c2a00640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa1a8078e80 0x7fa1a807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:57.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.816+0000 7fa1c2a00640 1 -- 172.21.15.60:0/2729299011 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4971+0+0 (secure 0 0 0) 0x7fa1c409bb40 con 0x7fa1cc161440 2024-09-16T15:45:57.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.816+0000 7fa1c9e00640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa1a8078e80 0x7fa1a807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:57.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.817+0000 7fa1c9e00640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa1a8078e80 0x7fa1a807b340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc035000 tx=0x7fa1bc004ab0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:57.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:57.820+0000 7fa1c2a00640 1 -- 172.21.15.60:0/2729299011 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa1cc0cb380 con 0x7fa1cc161440 2024-09-16T15:45:58.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.017+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa1cc0b7ef0 con 0x7fa1cc161440 2024-09-16T15:45:58.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.017+0000 7fa1c2a00640 1 -- 172.21.15.60:0/2729299011 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+1998 (secure 0 0 0) 0x7fa1c4014030 con 0x7fa1cc161440 2024-09-16T15:45:58.019 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:45:58.020 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:45:58.021 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:45:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.019+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa1a8078e80 msgr2=0x7fa1a807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.019+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa1a8078e80 0x7fa1a807b340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc035000 tx=0x7fa1bc004ab0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.019+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 msgr2=0x7fa1cc153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.019+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 0x7fa1cc153150 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400c970 tx=0x7fa1c400ce30 comp rx=0 tx=0).stop 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.020+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 shutdown_connections 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.020+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa1a8078e80 0x7fa1a807b340 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.020+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc161440 0x7fa1cc153150 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.020+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc0b4050 0x7fa1cc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.020+0000 7fa1cb200640 1 --2- 172.21.15.60:0/2729299011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc0b34b0 0x7fa1cc0b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.020+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 >> 172.21.15.60:0/2729299011 conn(0x7fa1cc0d3df0 msgr2=0x7fa1cc0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.020+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 shutdown_connections 2024-09-16T15:45:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.020+0000 7fa1cb200640 1 -- 172.21.15.60:0/2729299011 wait complete. 2024-09-16T15:45:58.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:45:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.199+0000 7f2c24200640 1 -- 172.21.15.60:0/784174349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 msgr2=0x7f2c1c15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.199+0000 7f2c24200640 1 --2- 172.21.15.60:0/784174349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 0x7f2c1c15f9f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0d0cd0 tx=0x7f2c1402a980 comp rx=0 tx=0).stop 2024-09-16T15:45:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.200+0000 7f2c24200640 1 -- 172.21.15.60:0/784174349 shutdown_connections 2024-09-16T15:45:58.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.200+0000 7f2c24200640 1 --2- 172.21.15.60:0/784174349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 0x7f2c1c15f9f0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.200+0000 7f2c24200640 1 --2- 172.21.15.60:0/784174349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c1c06efa0 0x7f2c1c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.200+0000 7f2c24200640 1 --2- 172.21.15.60:0/784174349 >> v2:172.21.15.100:3300/0 conn(0x7f2c1c145750 0x7f2c1c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.200+0000 7f2c24200640 1 -- 172.21.15.60:0/784174349 >> 172.21.15.60:0/784174349 conn(0x7f2c1c0b1840 msgr2=0x7f2c1c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.200+0000 7f2c24200640 1 -- 172.21.15.60:0/784174349 shutdown_connections 2024-09-16T15:45:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.200+0000 7f2c24200640 1 -- 172.21.15.60:0/784174349 wait complete. 2024-09-16T15:45:58.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.201+0000 7f2c24200640 1 Processor -- start 2024-09-16T15:45:58.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.201+0000 7f2c24200640 1 -- start start 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.202+0000 7f2c24200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1c06efa0 0x7f2c1c0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.202+0000 7f2c24200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 0x7f2c1c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.202+0000 7f2c24200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c1c145750 0x7f2c1c1411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.202+0000 7f2c24200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c1600b0 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.202+0000 7f2c24200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c15ff30 con 0x7f2c1c06efa0 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.202+0000 7f2c24200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1c160230 con 0x7f2c1c145750 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.202+0000 7f2c20e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 0x7f2c1c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c20e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 0x7f2c1c0ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56432/0 (socket says 172.21.15.60:56432) 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c20e00640 1 -- 172.21.15.60:0/2378284294 learned_addr learned my addr 172.21.15.60:0/2378284294 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:58.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c20e00640 1 -- 172.21.15.60:0/2378284294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c1c145750 msgr2=0x7f2c1c1411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c20e00640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c1c145750 0x7f2c1c1411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c20e00640 1 -- 172.21.15.60:0/2378284294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1c06efa0 msgr2=0x7f2c1c0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c20e00640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1c06efa0 0x7f2c1c0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c20e00640 1 -- 172.21.15.60:0/2378284294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c0c4d00 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c20e00640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 0x7f2c1c0ae240 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e9f0 tx=0x7f2c0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c19600640 1 -- 172.21.15.60:0/2378284294 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800cc60 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c19600640 1 -- 172.21.15.60:0/2378284294 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c0800ce00 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c19600640 1 -- 172.21.15.60:0/2378284294 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08005280 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.203+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c0c4fe0 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.204+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c0c55a0 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.205+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c0c57a0 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.206+0000 7f2c19600640 1 -- 172.21.15.60:0/2378284294 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f2c08005420 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.207+0000 7f2c19600640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2bfc078e80 0x7f2bfc07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:58.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.207+0000 7f2c19600640 1 -- 172.21.15.60:0/2378284294 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4971+0+0 (secure 0 0 0) 0x7f2c0809bcd0 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.207+0000 7f2c21800640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2bfc078e80 0x7f2bfc07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:58.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.207+0000 7f2c21800640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2bfc078e80 0x7f2bfc07b340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0ae1d0 tx=0x7f2c10005ce0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.211+0000 7f2c19600640 1 -- 172.21.15.60:0/2378284294 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c1c0c57a0 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.407+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2c1c0cc620 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.411+0000 7f2c19600640 1 -- 172.21.15.60:0/2378284294 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f2c08063c20 con 0x7f2c1c0ccb00 2024-09-16T15:45:58.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2bfc078e80 msgr2=0x7f2bfc07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2bfc078e80 0x7f2bfc07b340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0ae1d0 tx=0x7f2c10005ce0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 msgr2=0x7f2c1c0ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 0x7f2c1c0ae240 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e9f0 tx=0x7f2c0800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 shutdown_connections 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2bfc078e80 0x7f2bfc07b340 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c1c145750 0x7f2c1c1411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c1c0ccb00 0x7f2c1c0ae240 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 --2- 172.21.15.60:0/2378284294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1c06efa0 0x7f2c1c0c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 >> 172.21.15.60:0/2378284294 conn(0x7f2c1c0b1840 msgr2=0x7f2c1c0af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 shutdown_connections 2024-09-16T15:45:58.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.414+0000 7f2c24200640 1 -- 172.21.15.60:0/2378284294 wait complete. 2024-09-16T15:45:58.431 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:45:58.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:45:58.521 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:45:58.611 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:45:58.611 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:58.611 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:58.647 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:45:58.648 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:58.648 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:58.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:45:58.686 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:45:58.686 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:58.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:45:58.723 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:45:58.723 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:58.763 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:45:58.763 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:45:58.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 -- 172.21.15.60:0/138211974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae0415f5a0 msgr2=0x7fae040b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 --2- 172.21.15.60:0/138211974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae0415f5a0 0x7fae040b2440 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fae04076720 tx=0x7fadf802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 -- 172.21.15.60:0/138211974 shutdown_connections 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 --2- 172.21.15.60:0/138211974 >> v2:172.21.15.100:3300/0 conn(0x7fae040ad270 0x7fae04073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 --2- 172.21.15.60:0/138211974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ccf00 0x7fae040acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 --2- 172.21.15.60:0/138211974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae0415f5a0 0x7fae040b2440 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 -- 172.21.15.60:0/138211974 >> 172.21.15.60:0/138211974 conn(0x7fae040b1950 msgr2=0x7fae040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 -- 172.21.15.60:0/138211974 shutdown_connections 2024-09-16T15:45:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.923+0000 7fae0bc00640 1 -- 172.21.15.60:0/138211974 wait complete. 2024-09-16T15:45:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.924+0000 7fae0bc00640 1 Processor -- start 2024-09-16T15:45:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.925+0000 7fae0bc00640 1 -- start start 2024-09-16T15:45:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.925+0000 7fae0bc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ad270 0x7fae0414e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.925+0000 7fae0bc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae040ccf00 0x7fae041502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.925+0000 7fae0bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae0415f5a0 0x7fae040b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.925+0000 7fae0bc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fae0406f160 con 0x7fae040ccf00 2024-09-16T15:45:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.925+0000 7fae0bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae04074780 con 0x7fae0415f5a0 2024-09-16T15:45:58.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.925+0000 7fae0bc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fae0406f2e0 con 0x7fae040ad270 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae040ccf00 0x7fae041502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae040ccf00 0x7fae041502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41362/0 (socket says 172.21.15.60:41362) 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 -- 172.21.15.60:0/837252837 learned_addr learned my addr 172.21.15.60:0/837252837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae09200640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ad270 0x7fae0414e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae09c00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae0415f5a0 0x7fae040b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 -- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ad270 msgr2=0x7fae0414e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ad270 0x7fae0414e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 -- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae0415f5a0 msgr2=0x7fae040b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae0415f5a0 0x7fae040b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:58.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 -- 172.21.15.60:0/837252837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae04152c70 con 0x7fae040ccf00 2024-09-16T15:45:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae09c00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae0415f5a0 0x7fae040b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae09200640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ad270 0x7fae0414e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.926+0000 7fae03e00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae040ccf00 0x7fae041502b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fadf000e9f0 tx=0x7fadf000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.927+0000 7fae00c00640 1 -- 172.21.15.60:0/837252837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf000cd80 con 0x7fae040ccf00 2024-09-16T15:45:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.927+0000 7fae00c00640 1 -- 172.21.15.60:0/837252837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadf0005020 con 0x7fae040ccf00 2024-09-16T15:45:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.927+0000 7fae00c00640 1 -- 172.21.15.60:0/837252837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0005450 con 0x7fae040ccf00 2024-09-16T15:45:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.927+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae041f8330 con 0x7fae040ccf00 2024-09-16T15:45:58.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.927+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae041f8bd0 con 0x7fae040ccf00 2024-09-16T15:45:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.928+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0406f870 con 0x7fae040ccf00 2024-09-16T15:45:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.929+0000 7fae00c00640 1 -- 172.21.15.60:0/837252837 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fadf00047a0 con 0x7fae040ccf00 2024-09-16T15:45:58.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.930+0000 7fae00c00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fade4078e80 0x7fade407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:58.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.930+0000 7fae00c00640 1 -- 172.21.15.60:0/837252837 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4971+0+0 (secure 0 0 0) 0x7fadf009bd80 con 0x7fae040ccf00 2024-09-16T15:45:58.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.930+0000 7fae09200640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fade4078e80 0x7fade407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:58.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.931+0000 7fae09200640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fade4078e80 0x7fade407b340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fadf8033000 tx=0x7fadf8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:58.934+0000 7fae00c00640 1 -- 172.21.15.60:0/837252837 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fadf0063d90 con 0x7fae040ccf00 2024-09-16T15:45:59.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.170+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae040afcc0 con 0x7fae040ccf00 2024-09-16T15:45:59.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.170+0000 7fae00c00640 1 -- 172.21.15.60:0/837252837 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+3914 (secure 0 0 0) 0x7fadf0063690 con 0x7fae040ccf00 2024-09-16T15:45:59.173 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:45:59.173 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":84,"btime":"2024-09-16T15:45:52:599910+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7225,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/900096462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":900096462},{"type":"v1","addr":"172.21.15.100:6834","nonce":900096462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7228,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2427775344","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2427775344},{"type":"v1","addr":"172.21.15.100:6835","nonce":2427775344}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7584,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1853423008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1853423008},{"type":"v1","addr":"172.21.15.60:6833","nonce":1853423008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7908,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3840633976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3840633976},{"type":"v1","addr":"172.21.15.60:6835","nonce":3840633976}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[]} 2024-09-16T15:45:59.173 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 84 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fade4078e80 msgr2=0x7fade407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fade4078e80 0x7fade407b340 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fadf8033000 tx=0x7fadf8004730 comp rx=0 tx=0).stop 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae040ccf00 msgr2=0x7fae041502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae040ccf00 0x7fae041502b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fadf000e9f0 tx=0x7fadf000edf0 comp rx=0 tx=0).stop 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 shutdown_connections 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fade4078e80 0x7fade407b340 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae0415f5a0 0x7fae040b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae040ccf00 0x7fae041502b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 --2- 172.21.15.60:0/837252837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae040ad270 0x7fae0414e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:59.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 >> 172.21.15.60:0/837252837 conn(0x7fae040b1950 msgr2=0x7fae040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:59.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 shutdown_connections 2024-09-16T15:45:59.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:45:59.173+0000 7fae0bc00640 1 -- 172.21.15.60:0/837252837 wait complete. 2024-09-16T15:45:59.192 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:45:59.192 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:45:59.192 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:45:59.192 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:45:59.193 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:45:59.193 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:45:59.193 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:45:59.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:45:59.193 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:45:59.194 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:45:59.194 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:45:59.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:45:59.194 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:45:59.194 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:45:59.194 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:45:59.195 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:45:59.212 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:45:59.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:45:59.213 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:45:59.214 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:45:59.214 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:45:59.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:45:59.257 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:45:59.257 DEBUG:teuthology.parallel:result is None 2024-09-16T15:45:59.258 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:45:59.258 DEBUG:teuthology.parallel:result is None 2024-09-16T15:45:59.259 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:45:59.259 DEBUG:teuthology.parallel:result is None 2024-09-16T15:45:59.385 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:45:59.386 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:45:59.386 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:45:59.388 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-09-16T15:45:59.386+0000 7fe8f3600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-16T15:46:01.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:46:01.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:46:02.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-16T15:46:05.212 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:46:05.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:46:05.218 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:46:05.218 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:05.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:46:05.355 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:46:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.384+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3941661361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e015f5a0 msgr2=0x7fe5e00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.384+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3941661361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e015f5a0 0x7fe5e00b2440 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009830 tx=0x7fe5d402a480 comp rx=0 tx=0).stop 2024-09-16T15:46:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.384+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3941661361 shutdown_connections 2024-09-16T15:46:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.384+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3941661361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e00ccf00 0x7fe5e00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.384+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3941661361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e015f5a0 0x7fe5e00b2440 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.384+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3941661361 >> 172.21.15.60:0/3941661361 conn(0x7fe5e00b1950 msgr2=0x7fe5e00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.385+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3941661361 shutdown_connections 2024-09-16T15:46:05.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.385+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3941661361 wait complete. 2024-09-16T15:46:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.386+0000 7fe5e7e00640 1 Processor -- start 2024-09-16T15:46:05.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.386+0000 7fe5e7e00640 1 -- start start 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e7e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e00ad270 0x7fe5e014c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e7e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e00ccf00 0x7fe5e0152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e7e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e015f5a0 0x7fe5e0150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e7e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e006f2e0 con 0x7fe5e00ad270 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e7e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e006f160 con 0x7fe5e015f5a0 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e7e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e006f460 con 0x7fe5e00ccf00 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e00ad270 0x7fe5e014c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e00ad270 0x7fe5e014c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41388/0 (socket says 172.21.15.60:41388) 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.387+0000 7fe5e5400640 1 -- 172.21.15.60:0/3521222271 learned_addr learned my addr 172.21.15.60:0/3521222271 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e5e00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e015f5a0 0x7fe5e0150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e4a00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e00ccf00 0x7fe5e0152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e5400640 1 -- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e00ccf00 msgr2=0x7fe5e0152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:05.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e5400640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e00ccf00 0x7fe5e0152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e5400640 1 -- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e015f5a0 msgr2=0x7fe5e0150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e5400640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e015f5a0 0x7fe5e0150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e5400640 1 -- 172.21.15.60:0/3521222271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e0153560 con 0x7fe5e00ad270 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e4a00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e00ccf00 0x7fe5e0152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e5400640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e00ad270 0x7fe5e014c590 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4000c00 tx=0x7fe5d4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5dd600640 1 -- 172.21.15.60:0/3521222271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4031d60 con 0x7fe5e00ad270 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5dd600640 1 -- 172.21.15.60:0/3521222271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5d4033e10 con 0x7fe5e00ad270 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5dd600640 1 -- 172.21.15.60:0/3521222271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40332b0 con 0x7fe5e00ad270 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.388+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e0153840 con 0x7fe5e00ad270 2024-09-16T15:46:05.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.389+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e0153e00 con 0x7fe5e00ad270 2024-09-16T15:46:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.390+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e006f9f0 con 0x7fe5e00ad270 2024-09-16T15:46:05.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.391+0000 7fe5dd600640 1 -- 172.21.15.60:0/3521222271 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fe5d4033450 con 0x7fe5e00ad270 2024-09-16T15:46:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.392+0000 7fe5dd600640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078e80 0x7fe5c007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.392+0000 7fe5dd600640 1 -- 172.21.15.60:0/3521222271 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4971+0+0 (secure 0 0 0) 0x7fe5d40bc3a0 con 0x7fe5e00ad270 2024-09-16T15:46:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.392+0000 7fe5e4a00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078e80 0x7fe5c007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:05.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.393+0000 7fe5e4a00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078e80 0x7fe5c007b340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00ccd40 tx=0x7fe5cc005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.396+0000 7fe5dd600640 1 -- 172.21.15.60:0/3521222271 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe5d403f030 con 0x7fe5e00ad270 2024-09-16T15:46:05.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:05.603+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe5e009d3f0 con 0x7fe5e00ad270 2024-09-16T15:46:06.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.649+0000 7fe5dd600640 1 -- 172.21.15.60:0/3521222271 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v133) ==== 97+0+0 (secure 0 0 0) 0x7fe5d40843d0 con 0x7fe5e00ad270 2024-09-16T15:46:06.652 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:46:06.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078e80 msgr2=0x7fe5c007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:06.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078e80 0x7fe5c007b340 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00ccd40 tx=0x7fe5cc005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:06.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e00ad270 msgr2=0x7fe5e014c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:06.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e00ad270 0x7fe5e014c590 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4000c00 tx=0x7fe5d4033c20 comp rx=0 tx=0).stop 2024-09-16T15:46:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 shutdown_connections 2024-09-16T15:46:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078e80 0x7fe5c007b340 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e015f5a0 0x7fe5e0150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e00ccf00 0x7fe5e0152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 --2- 172.21.15.60:0/3521222271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e00ad270 0x7fe5e014c590 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 >> 172.21.15.60:0/3521222271 conn(0x7fe5e00b1950 msgr2=0x7fe5e009c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 shutdown_connections 2024-09-16T15:46:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.653+0000 7fe5e7e00640 1 -- 172.21.15.60:0/3521222271 wait complete. 2024-09-16T15:46:06.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.837+0000 7f8e0a200640 1 -- 172.21.15.60:0/3433823311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0415bc30 msgr2=0x7f8e040b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.837+0000 7f8e0a200640 1 --2- 172.21.15.60:0/3433823311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0415bc30 0x7f8e040b28f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8e04073b00 tx=0x7f8df802a560 comp rx=0 tx=0).stop 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.837+0000 7f8e0a200640 1 -- 172.21.15.60:0/3433823311 shutdown_connections 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.837+0000 7f8e0a200640 1 --2- 172.21.15.60:0/3433823311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e040d0540 0x7f8e04074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.837+0000 7f8e0a200640 1 --2- 172.21.15.60:0/3433823311 >> v2:172.21.15.100:3300/0 conn(0x7f8e040725d0 0x7f8e04073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.837+0000 7f8e0a200640 1 --2- 172.21.15.60:0/3433823311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e0415bc30 0x7f8e040b28f0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.837+0000 7f8e0a200640 1 -- 172.21.15.60:0/3433823311 >> 172.21.15.60:0/3433823311 conn(0x7f8e040cfa20 msgr2=0x7f8e040cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.837+0000 7f8e0a200640 1 -- 172.21.15.60:0/3433823311 shutdown_connections 2024-09-16T15:46:06.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.838+0000 7f8e0a200640 1 -- 172.21.15.60:0/3433823311 wait complete. 2024-09-16T15:46:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.839+0000 7f8e0a200640 1 Processor -- start 2024-09-16T15:46:06.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.839+0000 7f8e0a200640 1 -- start start 2024-09-16T15:46:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e0a200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e040d0540 0x7f8e040c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e0a200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0415bc30 0x7f8e041f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e0a200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e041f5010 0x7f8e041f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e0a200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0406f4a0 con 0x7f8e040d0540 2024-09-16T15:46:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e0a200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0406f320 con 0x7f8e041f5010 2024-09-16T15:46:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e0a200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0406f620 con 0x7f8e0415bc30 2024-09-16T15:46:06.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e08e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e040d0540 0x7f8e040c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e08e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e040d0540 0x7f8e040c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41398/0 (socket says 172.21.15.60:41398) 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.840+0000 7f8e08e00640 1 -- 172.21.15.60:0/2784986196 learned_addr learned my addr 172.21.15.60:0/2784986196 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e03e00640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0415bc30 0x7f8e041f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e09800640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e041f5010 0x7f8e041f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e08e00640 1 -- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0415bc30 msgr2=0x7f8e041f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e08e00640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0415bc30 0x7f8e041f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e08e00640 1 -- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e041f5010 msgr2=0x7f8e041f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e08e00640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e041f5010 0x7f8e041f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e08e00640 1 -- 172.21.15.60:0/2784986196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e040c84b0 con 0x7f8e040d0540 2024-09-16T15:46:06.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e03e00640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0415bc30 0x7f8e041f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:06.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e08e00640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e040d0540 0x7f8e040c7b40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8df8034040 tx=0x7f8df80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:06.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e00c00640 1 -- 172.21.15.60:0/2784986196 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df803e030 con 0x7f8e040d0540 2024-09-16T15:46:06.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e00c00640 1 -- 172.21.15.60:0/2784986196 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8df80047a0 con 0x7f8e040d0540 2024-09-16T15:46:06.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.841+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e040c8790 con 0x7f8e040d0540 2024-09-16T15:46:06.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.842+0000 7f8e00c00640 1 -- 172.21.15.60:0/2784986196 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8035040 con 0x7f8e040d0540 2024-09-16T15:46:06.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.842+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e040c8d50 con 0x7f8e040d0540 2024-09-16T15:46:06.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.843+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e040b28f0 con 0x7f8e040d0540 2024-09-16T15:46:06.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.844+0000 7f8e00c00640 1 -- 172.21.15.60:0/2784986196 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f8df8041030 con 0x7f8e040d0540 2024-09-16T15:46:06.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.845+0000 7f8e00c00640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8de4078e80 0x7f8de407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:06.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.845+0000 7f8e00c00640 1 -- 172.21.15.60:0/2784986196 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4885+0+0 (secure 0 0 0) 0x7f8df80bb8d0 con 0x7f8e040d0540 2024-09-16T15:46:06.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.845+0000 7f8e03e00640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8de4078e80 0x7f8de407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:06.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.846+0000 7f8e03e00640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8de4078e80 0x7f8de407b340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8e04072580 tx=0x7f8df0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:06.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:06.849+0000 7f8e00c00640 1 -- 172.21.15.60:0/2784986196 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8df8083950 con 0x7f8e040d0540 2024-09-16T15:46:07.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.115+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8e04069850 con 0x7f8e040d0540 2024-09-16T15:46:07.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.116+0000 7f8e00c00640 1 -- 172.21.15.60:0/2784986196 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f8df8083250 con 0x7f8e040d0540 2024-09-16T15:46:07.118 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:07.118 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:46:07.118 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:46:07.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:07.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:07.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.121 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.122 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:46:07.123 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.124 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:46:07.126 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.128 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:07.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.131 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.131 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:46:07.131 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8de4078e80 msgr2=0x7f8de407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8de4078e80 0x7f8de407b340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8e04072580 tx=0x7f8df0008040 comp rx=0 tx=0).stop 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e040d0540 msgr2=0x7f8e040c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e040d0540 0x7f8e040c7b40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8df8034040 tx=0x7f8df80039e0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 shutdown_connections 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8de4078e80 0x7f8de407b340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e041f5010 0x7f8e041f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e0415bc30 0x7f8e041f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 --2- 172.21.15.60:0/2784986196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e040d0540 0x7f8e040c7b40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 >> 172.21.15.60:0/2784986196 conn(0x7f8e040cfa20 msgr2=0x7f8e040cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 shutdown_connections 2024-09-16T15:46:07.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.119+0000 7f8e0a200640 1 -- 172.21.15.60:0/2784986196 wait complete. 2024-09-16T15:46:07.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:46:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.295+0000 7f999a000640 1 -- 172.21.15.60:0/3338885560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 msgr2=0x7f9994074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.295+0000 7f999a000640 1 --2- 172.21.15.60:0/3338885560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 0x7f9994074460 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f998c005fd0 tx=0x7f998c02a970 comp rx=0 tx=0).stop 2024-09-16T15:46:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.296+0000 7f999a000640 1 -- 172.21.15.60:0/3338885560 shutdown_connections 2024-09-16T15:46:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.296+0000 7f999a000640 1 --2- 172.21.15.60:0/3338885560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 0x7f9994074460 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.296+0000 7f999a000640 1 --2- 172.21.15.60:0/3338885560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f999415f5a0 0x7f99940b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.296+0000 7f999a000640 1 -- 172.21.15.60:0/3338885560 >> 172.21.15.60:0/3338885560 conn(0x7f99940b1950 msgr2=0x7f99940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.296+0000 7f999a000640 1 -- 172.21.15.60:0/3338885560 shutdown_connections 2024-09-16T15:46:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.296+0000 7f999a000640 1 -- 172.21.15.60:0/3338885560 wait complete. 2024-09-16T15:46:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.297+0000 7f999a000640 1 Processor -- start 2024-09-16T15:46:07.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.297+0000 7f999a000640 1 -- start start 2024-09-16T15:46:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.298+0000 7f999a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 0x7f9994148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.298+0000 7f999a000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99940ccf00 0x7f9994145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.298+0000 7f999a000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999415f5a0 0x7f99941462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.298+0000 7f999a000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f999406f2e0 con 0x7f9994073fb0 2024-09-16T15:46:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.298+0000 7f999a000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f999406f160 con 0x7f999415f5a0 2024-09-16T15:46:07.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.298+0000 7f999a000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f999406f460 con 0x7f99940ccf00 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 0x7f9994148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 0x7f9994148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41416/0 (socket says 172.21.15.60:41416) 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 -- 172.21.15.60:0/3060115411 learned_addr learned my addr 172.21.15.60:0/3060115411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f998aa00640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99940ccf00 0x7f9994145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993e00640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999415f5a0 0x7f99941462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 -- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99940ccf00 msgr2=0x7f9994145d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99940ccf00 0x7f9994145d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 -- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999415f5a0 msgr2=0x7f99941462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999415f5a0 0x7f99941462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 -- 172.21.15.60:0/3060115411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9994149260 con 0x7f9994073fb0 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993e00640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999415f5a0 0x7f99941462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:07.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.299+0000 7f9993400640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 0x7f9994148a10 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f998400c970 tx=0x7f998400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:07.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.300+0000 7f998be00640 1 -- 172.21.15.60:0/3060115411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984007bb0 con 0x7f9994073fb0 2024-09-16T15:46:07.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.300+0000 7f998be00640 1 -- 172.21.15.60:0/3060115411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9984007e40 con 0x7f9994073fb0 2024-09-16T15:46:07.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.300+0000 7f998be00640 1 -- 172.21.15.60:0/3060115411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998400f690 con 0x7f9994073fb0 2024-09-16T15:46:07.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.300+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9994149540 con 0x7f9994073fb0 2024-09-16T15:46:07.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.300+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99940ce760 con 0x7f9994073fb0 2024-09-16T15:46:07.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.301+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999406f9f0 con 0x7f9994073fb0 2024-09-16T15:46:07.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.302+0000 7f998be00640 1 -- 172.21.15.60:0/3060115411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f998400f830 con 0x7f9994073fb0 2024-09-16T15:46:07.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.303+0000 7f998be00640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9970078e80 0x7f997007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:07.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.304+0000 7f998be00640 1 -- 172.21.15.60:0/3060115411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4885+0+0 (secure 0 0 0) 0x7f998409b9c0 con 0x7f9994073fb0 2024-09-16T15:46:07.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.304+0000 7f998aa00640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9970078e80 0x7f997007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:07.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.304+0000 7f998aa00640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9970078e80 0x7f997007b340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f99940ae7c0 tx=0x7f997c007e30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:07.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.308+0000 7f998be00640 1 -- 172.21.15.60:0/3060115411 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9984014030 con 0x7f9994073fb0 2024-09-16T15:46:07.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.506+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9994155de0 con 0x7f9994073fb0 2024-09-16T15:46:07.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.506+0000 7f998be00640 1 -- 172.21.15.60:0/3060115411 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) ==== 84+0+65 (secure 0 0 0) 0x7f9984063960 con 0x7f9994073fb0 2024-09-16T15:46:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:07.509 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:46:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:46:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:46:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:07.509 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:46:07.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9970078e80 msgr2=0x7f997007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9970078e80 0x7f997007b340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f99940ae7c0 tx=0x7f997c007e30 comp rx=0 tx=0).stop 2024-09-16T15:46:07.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 msgr2=0x7f9994148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 0x7f9994148a10 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f998400c970 tx=0x7f998400ce30 comp rx=0 tx=0).stop 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 shutdown_connections 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9970078e80 0x7f997007b340 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999415f5a0 0x7f99941462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f99940ccf00 0x7f9994145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 --2- 172.21.15.60:0/3060115411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9994073fb0 0x7f9994148a10 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 >> 172.21.15.60:0/3060115411 conn(0x7f99940b1950 msgr2=0x7f99940b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 shutdown_connections 2024-09-16T15:46:07.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.509+0000 7f999a000640 1 -- 172.21.15.60:0/3060115411 wait complete. 2024-09-16T15:46:07.527 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:46:07.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:46:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.691+0000 7f8f6f400640 1 -- 172.21.15.60:0/3883928958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f68159aa0 msgr2=0x7f8f680741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.691+0000 7f8f6f400640 1 --2- 172.21.15.60:0/3883928958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f68159aa0 0x7f8f680741a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009900 tx=0x7f8f5802a610 comp rx=0 tx=0).stop 2024-09-16T15:46:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.691+0000 7f8f6f400640 1 -- 172.21.15.60:0/3883928958 shutdown_connections 2024-09-16T15:46:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.691+0000 7f8f6f400640 1 --2- 172.21.15.60:0/3883928958 >> v2:172.21.15.100:3300/0 conn(0x7f8f6806f220 0x7f8f6806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.691+0000 7f8f6f400640 1 --2- 172.21.15.60:0/3883928958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f68159aa0 0x7f8f680741a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.691+0000 7f8f6f400640 1 -- 172.21.15.60:0/3883928958 >> 172.21.15.60:0/3883928958 conn(0x7f8f680b18e0 msgr2=0x7f8f680b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.692+0000 7f8f6f400640 1 -- 172.21.15.60:0/3883928958 shutdown_connections 2024-09-16T15:46:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.692+0000 7f8f6f400640 1 -- 172.21.15.60:0/3883928958 wait complete. 2024-09-16T15:46:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.693+0000 7f8f6f400640 1 Processor -- start 2024-09-16T15:46:07.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.693+0000 7f8f6f400640 1 -- start start 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.694+0000 7f8f6f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f6806f220 0x7f8f68152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.694+0000 7f8f6f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f68159aa0 0x7f8f680ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.694+0000 7f8f6f400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f6815cff0 0x7f8f681577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.694+0000 7f8f6f400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680cc520 con 0x7f8f6806f220 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.694+0000 7f8f6f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680cc3a0 con 0x7f8f68159aa0 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.694+0000 7f8f6f400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8f680cc6a0 con 0x7f8f6815cff0 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f6806f220 0x7f8f68152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f6806f220 0x7f8f68152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41432/0 (socket says 172.21.15.60:41432) 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 -- 172.21.15.60:0/1390210262 learned_addr learned my addr 172.21.15.60:0/1390210262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:07.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6d400640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f6815cff0 0x7f8f681577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f5fe00640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f68159aa0 0x7f8f680ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 -- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f6815cff0 msgr2=0x7f8f681577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f6815cff0 0x7f8f681577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 -- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f68159aa0 msgr2=0x7f8f680ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f68159aa0 0x7f8f680ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 -- 172.21.15.60:0/1390210262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f68153080 con 0x7f8f6806f220 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f5fe00640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f68159aa0 0x7f8f680ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.695+0000 7f8f6ca00640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f6806f220 0x7f8f68152810 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8f58006010 tx=0x7f8f580035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.696+0000 7f8f65600640 1 -- 172.21.15.60:0/1390210262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5803f070 con 0x7f8f6806f220 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.696+0000 7f8f65600640 1 -- 172.21.15.60:0/1390210262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f58003ce0 con 0x7f8f6806f220 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.696+0000 7f8f65600640 1 -- 172.21.15.60:0/1390210262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58035030 con 0x7f8f6806f220 2024-09-16T15:46:07.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.696+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f68156520 con 0x7f8f6806f220 2024-09-16T15:46:07.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.696+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f68152d50 con 0x7f8f6806f220 2024-09-16T15:46:07.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.697+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f68069360 con 0x7f8f6806f220 2024-09-16T15:46:07.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.698+0000 7f8f65600640 1 -- 172.21.15.60:0/1390210262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f8f58042070 con 0x7f8f6806f220 2024-09-16T15:46:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.699+0000 7f8f65600640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8f44078eb0 0x7f8f4407b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.699+0000 7f8f65600640 1 -- 172.21.15.60:0/1390210262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 4885+0+0 (secure 0 0 0) 0x7f8f580bbb80 con 0x7f8f6806f220 2024-09-16T15:46:07.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.699+0000 7f8f5fe00640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8f44078eb0 0x7f8f4407b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:07.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.700+0000 7f8f5fe00640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8f44078eb0 0x7f8f4407b370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f680d0be0 tx=0x7f8f50009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:07.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.703+0000 7f8f65600640 1 -- 172.21.15.60:0/1390210262 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8f580835b0 con 0x7f8f6806f220 2024-09-16T15:46:07.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:07.899+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8f68070980 con 0x7f8f6806f220 2024-09-16T15:46:08.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.671+0000 7f8f65600640 1 -- 172.21.15.60:0/1390210262 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v134) ==== 138+0+0 (secure 0 0 0) 0x7f8f58087fe0 con 0x7f8f6806f220 2024-09-16T15:46:08.673 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:46:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.675+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8f44078eb0 msgr2=0x7f8f4407b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.675+0000 7f8f6f400640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8f44078eb0 0x7f8f4407b370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f680d0be0 tx=0x7f8f50009040 comp rx=0 tx=0).stop 2024-09-16T15:46:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.675+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f6806f220 msgr2=0x7f8f68152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.675+0000 7f8f6f400640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f6806f220 0x7f8f68152810 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8f58006010 tx=0x7f8f580035f0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.676+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 shutdown_connections 2024-09-16T15:46:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.676+0000 7f8f6f400640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8f44078eb0 0x7f8f4407b370 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.676+0000 7f8f6f400640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f6815cff0 0x7f8f681577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.676+0000 7f8f6f400640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f68159aa0 0x7f8f680ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.676+0000 7f8f6f400640 1 --2- 172.21.15.60:0/1390210262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f6806f220 0x7f8f68152810 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.676+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 >> 172.21.15.60:0/1390210262 conn(0x7f8f680b18e0 msgr2=0x7f8f6815b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.676+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 shutdown_connections 2024-09-16T15:46:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.676+0000 7f8f6f400640 1 -- 172.21.15.60:0/1390210262 wait complete. 2024-09-16T15:46:08.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:46:08.842 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-16T15:46:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.870+0000 7f2019400640 1 -- 172.21.15.60:0/576981515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20140cda90 msgr2=0x7f20140ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.870+0000 7f2019400640 1 --2- 172.21.15.60:0/576981515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20140cda90 0x7f20140ce320 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2014051f80 tx=0x7f200002a580 comp rx=0 tx=0).stop 2024-09-16T15:46:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.870+0000 7f2019400640 1 -- 172.21.15.60:0/576981515 shutdown_connections 2024-09-16T15:46:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.870+0000 7f2019400640 1 --2- 172.21.15.60:0/576981515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20140cda90 0x7f20140ce320 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.870+0000 7f2019400640 1 --2- 172.21.15.60:0/576981515 >> v2:172.21.15.100:3300/0 conn(0x7f201415aa30 0x7f20140cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.870+0000 7f2019400640 1 -- 172.21.15.60:0/576981515 >> 172.21.15.60:0/576981515 conn(0x7f20140cc490 msgr2=0x7f20140cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.870+0000 7f2019400640 1 -- 172.21.15.60:0/576981515 shutdown_connections 2024-09-16T15:46:08.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.870+0000 7f2019400640 1 -- 172.21.15.60:0/576981515 wait complete. 2024-09-16T15:46:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.872+0000 7f2019400640 1 Processor -- start 2024-09-16T15:46:08.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.872+0000 7f2019400640 1 -- start start 2024-09-16T15:46:08.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2019400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20140cda90 0x7f201415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2019400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20140ce860 0x7f2014142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2019400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201415aa30 0x7f2014142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2019400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f201415ed40 con 0x7f201415aa30 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2019400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f201415ebc0 con 0x7f20140cda90 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2019400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f201415eec0 con 0x7f20140ce860 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2018a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201415aa30 0x7f2014142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2018a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201415aa30 0x7f2014142850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60242/0 (socket says 172.21.15.60:60242) 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2018a00640 1 -- 172.21.15.60:0/18299709 learned_addr learned my addr 172.21.15.60:0/18299709 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2013e00640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20140cda90 0x7f201415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.873+0000 7f2013400640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20140ce860 0x7f2014142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f2018a00640 1 -- 172.21.15.60:0/18299709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20140ce860 msgr2=0x7f2014142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f2018a00640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20140ce860 0x7f2014142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f2018a00640 1 -- 172.21.15.60:0/18299709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20140cda90 msgr2=0x7f201415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f2018a00640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20140cda90 0x7f201415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:08.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f2018a00640 1 -- 172.21.15.60:0/18299709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140b07a0 con 0x7f201415aa30 2024-09-16T15:46:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f2018a00640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201415aa30 0x7f2014142850 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f200c00c7a0 tx=0x7f200c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f200be00640 1 -- 172.21.15.60:0/18299709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c003b70 con 0x7f201415aa30 2024-09-16T15:46:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f200be00640 1 -- 172.21.15.60:0/18299709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f200c003d10 con 0x7f201415aa30 2024-09-16T15:46:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f200be00640 1 -- 172.21.15.60:0/18299709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c00aab0 con 0x7f201415aa30 2024-09-16T15:46:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201415d7d0 con 0x7f201415aa30 2024-09-16T15:46:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.874+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201415dd90 con 0x7f201415aa30 2024-09-16T15:46:08.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.875+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201414e330 con 0x7f201415aa30 2024-09-16T15:46:08.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.877+0000 7f200be00640 1 -- 172.21.15.60:0/18299709 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f200c00ac50 con 0x7f201415aa30 2024-09-16T15:46:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.878+0000 7f200be00640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ff4078e80 0x7f1ff407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.878+0000 7f200be00640 1 -- 172.21.15.60:0/18299709 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5255+0+0 (secure 0 0 0) 0x7f200c09b0b0 con 0x7f201415aa30 2024-09-16T15:46:08.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.878+0000 7f2013e00640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ff4078e80 0x7f1ff407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:08.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.879+0000 7f2013e00640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ff4078e80 0x7f1ff407b340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f200400bb50 tx=0x7f200400b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:08.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:08.883+0000 7f200be00640 1 -- 172.21.15.60:0/18299709 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f200c062890 con 0x7f201415aa30 2024-09-16T15:46:09.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.079+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f20140b0970 con 0x7f201415aa30 2024-09-16T15:46:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.674+0000 7f200be00640 1 -- 172.21.15.60:0/18299709 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v135) ==== 170+0+0 (secure 0 0 0) 0x7f200c0672c0 con 0x7f201415aa30 2024-09-16T15:46:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:46:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.677+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ff4078e80 msgr2=0x7f1ff407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.677+0000 7f2019400640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ff4078e80 0x7f1ff407b340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f200400bb50 tx=0x7f200400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.677+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201415aa30 msgr2=0x7f2014142850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.677+0000 7f2019400640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201415aa30 0x7f2014142850 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f200c00c7a0 tx=0x7f200c00cc60 comp rx=0 tx=0).stop 2024-09-16T15:46:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.678+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 shutdown_connections 2024-09-16T15:46:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.678+0000 7f2019400640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1ff4078e80 0x7f1ff407b340 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.678+0000 7f2019400640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f201415aa30 0x7f2014142850 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.678+0000 7f2019400640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20140ce860 0x7f2014142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.678+0000 7f2019400640 1 --2- 172.21.15.60:0/18299709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20140cda90 0x7f201415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.678+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 >> 172.21.15.60:0/18299709 conn(0x7f20140cc490 msgr2=0x7f20140c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.678+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 shutdown_connections 2024-09-16T15:46:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.678+0000 7f2019400640 1 -- 172.21.15.60:0/18299709 wait complete. 2024-09-16T15:46:09.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:46:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.874+0000 7f6c78400640 1 -- 172.21.15.60:0/3790452272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 msgr2=0x7f6c700cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.874+0000 7f6c78400640 1 --2- 172.21.15.60:0/3790452272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 0x7f6c700cdbb0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6c700d0cd0 tx=0x7f6c6802aa30 comp rx=0 tx=0).stop 2024-09-16T15:46:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.874+0000 7f6c78400640 1 -- 172.21.15.60:0/3790452272 shutdown_connections 2024-09-16T15:46:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.874+0000 7f6c78400640 1 --2- 172.21.15.60:0/3790452272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 0x7f6c700cdbb0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.874+0000 7f6c78400640 1 --2- 172.21.15.60:0/3790452272 >> v2:172.21.15.100:3300/0 conn(0x7f6c70145750 0x7f6c70073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.874+0000 7f6c78400640 1 -- 172.21.15.60:0/3790452272 >> 172.21.15.60:0/3790452272 conn(0x7f6c700b1840 msgr2=0x7f6c700b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.874+0000 7f6c78400640 1 -- 172.21.15.60:0/3790452272 shutdown_connections 2024-09-16T15:46:09.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.874+0000 7f6c78400640 1 -- 172.21.15.60:0/3790452272 wait complete. 2024-09-16T15:46:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.876+0000 7f6c78400640 1 Processor -- start 2024-09-16T15:46:09.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.876+0000 7f6c78400640 1 -- start start 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c78400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c7006efa0 0x7f6c700b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c78400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 0x7f6c700cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c78400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c70145750 0x7f6c700d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c78400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6c700ce480 con 0x7f6c700cccd0 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c78400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c700ce300 con 0x7f6c7006efa0 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c78400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6c700ce600 con 0x7f6c70145750 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c75000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 0x7f6c700cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c75000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 0x7f6c700cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60260/0 (socket says 172.21.15.60:60260) 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.877+0000 7f6c75000640 1 -- 172.21.15.60:0/1965754765 learned_addr learned my addr 172.21.15.60:0/1965754765 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:09.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c76400640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c70145750 0x7f6c700d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c75a00640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c7006efa0 0x7f6c700b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c75000640 1 -- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c70145750 msgr2=0x7f6c700d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c75000640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c70145750 0x7f6c700d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c75000640 1 -- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c7006efa0 msgr2=0x7f6c700b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c75000640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c7006efa0 0x7f6c700b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c75000640 1 -- 172.21.15.60:0/1965754765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c700b3970 con 0x7f6c700cccd0 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c76400640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c70145750 0x7f6c700d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c75a00640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c7006efa0 0x7f6c700b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:09.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c75000640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 0x7f6c700cfdb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00c450 tx=0x7f6c5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c6d600640 1 -- 172.21.15.60:0/1965754765 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0052d0 con 0x7f6c700cccd0 2024-09-16T15:46:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c6d600640 1 -- 172.21.15.60:0/1965754765 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c5c005470 con 0x7f6c700cccd0 2024-09-16T15:46:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c6d600640 1 -- 172.21.15.60:0/1965754765 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c012870 con 0x7f6c700cccd0 2024-09-16T15:46:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.878+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c701f83c0 con 0x7f6c700cccd0 2024-09-16T15:46:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.879+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c701f8980 con 0x7f6c700cccd0 2024-09-16T15:46:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.879+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c700736e0 con 0x7f6c700cccd0 2024-09-16T15:46:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.881+0000 7f6c6d600640 1 -- 172.21.15.60:0/1965754765 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f6c5c0035f0 con 0x7f6c700cccd0 2024-09-16T15:46:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.882+0000 7f6c6d600640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c50078e80 0x7f6c5007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.882+0000 7f6c6d600640 1 -- 172.21.15.60:0/1965754765 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5637+0+0 (secure 0 0 0) 0x7f6c5c09bab0 con 0x7f6c700cccd0 2024-09-16T15:46:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.883+0000 7f6c75a00640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c50078e80 0x7f6c5007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.883+0000 7f6c75a00640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c50078e80 0x7f6c5007b340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005fd0 tx=0x7f6c6400f360 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:09.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:09.886+0000 7f6c6d600640 1 -- 172.21.15.60:0/1965754765 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6c700736e0 con 0x7f6c700cccd0 2024-09-16T15:46:10.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.122+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6c700cdbb0 con 0x7f6c700cccd0 2024-09-16T15:46:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.702+0000 7f6c6d600640 1 -- 172.21.15.60:0/1965754765 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v88) ==== 174+0+0 (secure 0 0 0) 0x7f6c5c0637e0 con 0x7f6c700cccd0 2024-09-16T15:46:10.705 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 20 and data pool 21 2024-09-16T15:46:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.708+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c50078e80 msgr2=0x7f6c5007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.708+0000 7f6c78400640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c50078e80 0x7f6c5007b340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005fd0 tx=0x7f6c6400f360 comp rx=0 tx=0).stop 2024-09-16T15:46:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.708+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 msgr2=0x7f6c700cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:10.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.708+0000 7f6c78400640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 0x7f6c700cfdb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00c450 tx=0x7f6c5c00c910 comp rx=0 tx=0).stop 2024-09-16T15:46:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.709+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 shutdown_connections 2024-09-16T15:46:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.709+0000 7f6c78400640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6c50078e80 0x7f6c5007b340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.709+0000 7f6c78400640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c70145750 0x7f6c700d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.709+0000 7f6c78400640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c700cccd0 0x7f6c700cfdb0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.709+0000 7f6c78400640 1 --2- 172.21.15.60:0/1965754765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c7006efa0 0x7f6c700b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.709+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 >> 172.21.15.60:0/1965754765 conn(0x7f6c700b1840 msgr2=0x7f6c700a9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:10.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.710+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 shutdown_connections 2024-09-16T15:46:10.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.710+0000 7f6c78400640 1 -- 172.21.15.60:0/1965754765 wait complete. 2024-09-16T15:46:10.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 -- 172.21.15.60:0/1779780891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 msgr2=0x7fec8c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 --2- 172.21.15.60:0/1779780891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 0x7fec8c0b2440 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0cfbb0 tx=0x7fec8002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 -- 172.21.15.60:0/1779780891 shutdown_connections 2024-09-16T15:46:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 --2- 172.21.15.60:0/1779780891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec8c0ad270 0x7fec8c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 --2- 172.21.15.60:0/1779780891 >> v2:172.21.15.100:3300/0 conn(0x7fec8c0ccf00 0x7fec8c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 --2- 172.21.15.60:0/1779780891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 0x7fec8c0b2440 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 -- 172.21.15.60:0/1779780891 >> 172.21.15.60:0/1779780891 conn(0x7fec8c0b1950 msgr2=0x7fec8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 -- 172.21.15.60:0/1779780891 shutdown_connections 2024-09-16T15:46:10.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.904+0000 7fec93800640 1 -- 172.21.15.60:0/1779780891 wait complete. 2024-09-16T15:46:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.906+0000 7fec93800640 1 Processor -- start 2024-09-16T15:46:10.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.906+0000 7fec93800640 1 -- start start 2024-09-16T15:46:10.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec93800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec8c0ad270 0x7fec8c1e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec93800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec8c0ccf00 0x7fec8c1f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec93800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 0x7fec8c1f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec93800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c074880 con 0x7fec8c15f5a0 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec93800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c074700 con 0x7fec8c0ccf00 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec93800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec8c06f160 con 0x7fec8c0ad270 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec91800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 0x7fec8c1f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec91800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 0x7fec8c1f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60274/0 (socket says 172.21.15.60:60274) 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec91800640 1 -- 172.21.15.60:0/2773219717 learned_addr learned my addr 172.21.15.60:0/2773219717 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec8be00640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec8c0ccf00 0x7fec8c1f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.907+0000 7fec90e00640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec8c0ad270 0x7fec8c1e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec91800640 1 -- 172.21.15.60:0/2773219717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec8c0ad270 msgr2=0x7fec8c1e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:10.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec91800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec8c0ad270 0x7fec8c1e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec91800640 1 -- 172.21.15.60:0/2773219717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec8c0ccf00 msgr2=0x7fec8c1f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec91800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec8c0ccf00 0x7fec8c1f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:10.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec91800640 1 -- 172.21.15.60:0/2773219717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c0c91b0 con 0x7fec8c15f5a0 2024-09-16T15:46:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec91800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 0x7fec8c1f5420 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e980 tx=0x7fec8400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec88c00640 1 -- 172.21.15.60:0/2773219717 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84004d20 con 0x7fec8c15f5a0 2024-09-16T15:46:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec88c00640 1 -- 172.21.15.60:0/2773219717 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec8400cb20 con 0x7fec8c15f5a0 2024-09-16T15:46:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8c0c9490 con 0x7fec8c15f5a0 2024-09-16T15:46:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec88c00640 1 -- 172.21.15.60:0/2773219717 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84005020 con 0x7fec8c15f5a0 2024-09-16T15:46:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.908+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8c1e5620 con 0x7fec8c15f5a0 2024-09-16T15:46:10.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.909+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec8c0c9f50 con 0x7fec8c15f5a0 2024-09-16T15:46:10.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.910+0000 7fec88c00640 1 -- 172.21.15.60:0/2773219717 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fec840051c0 con 0x7fec8c15f5a0 2024-09-16T15:46:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.911+0000 7fec88c00640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec6c078eb0 0x7fec6c07b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.912+0000 7fec88c00640 1 -- 172.21.15.60:0/2773219717 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5737+0+0 (secure 0 0 0) 0x7fec8409be40 con 0x7fec8c15f5a0 2024-09-16T15:46:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.912+0000 7fec90e00640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec6c078eb0 0x7fec6c07b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:10.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.912+0000 7fec90e00640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec6c078eb0 0x7fec6c07b370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec80033000 tx=0x7fec80004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:10.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:10.916+0000 7fec88c00640 1 -- 172.21.15.60:0/2773219717 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fec8c0c9f50 con 0x7fec8c15f5a0 2024-09-16T15:46:11.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.116+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec8c1e58e0 con 0x7fec8c15f5a0 2024-09-16T15:46:11.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.117+0000 7fec88c00640 1 -- 172.21.15.60:0/2773219717 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+16796 (secure 0 0 0) 0x7fec84016030 con 0x7fec8c15f5a0 2024-09-16T15:46:11.119 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:11.120 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":136,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:10.124923+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:07.910602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:09.091337+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.119+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec6c078eb0 msgr2=0x7fec6c07b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.119+0000 7fec93800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec6c078eb0 0x7fec6c07b370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec80033000 tx=0x7fec80004730 comp rx=0 tx=0).stop 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.119+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 msgr2=0x7fec8c1f5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.119+0000 7fec93800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 0x7fec8c1f5420 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e980 tx=0x7fec8400ee40 comp rx=0 tx=0).stop 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.120+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 shutdown_connections 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.120+0000 7fec93800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec6c078eb0 0x7fec6c07b370 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.120+0000 7fec93800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec8c15f5a0 0x7fec8c1f5420 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.120+0000 7fec93800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec8c0ccf00 0x7fec8c1f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.120+0000 7fec93800640 1 --2- 172.21.15.60:0/2773219717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec8c0ad270 0x7fec8c1e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.120+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 >> 172.21.15.60:0/2773219717 conn(0x7fec8c0b1950 msgr2=0x7fec8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:11.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.120+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 shutdown_connections 2024-09-16T15:46:11.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.120+0000 7fec93800640 1 -- 172.21.15.60:0/2773219717 wait complete. 2024-09-16T15:46:11.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:11.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.297+0000 7f3357400640 1 -- 172.21.15.60:0/2850814146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 msgr2=0x7f3350168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.297+0000 7f3357400640 1 --2- 172.21.15.60:0/2850814146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 0x7f3350168e10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f33500b4890 tx=0x7f334802a890 comp rx=0 tx=0).stop 2024-09-16T15:46:11.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.297+0000 7f3357400640 1 -- 172.21.15.60:0/2850814146 shutdown_connections 2024-09-16T15:46:11.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.298+0000 7f3357400640 1 --2- 172.21.15.60:0/2850814146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 0x7f3350168e10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.298+0000 7f3357400640 1 -- 172.21.15.60:0/2850814146 >> 172.21.15.60:0/2850814146 conn(0x7f33500b1950 msgr2=0x7f33500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:11.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.298+0000 7f3357400640 1 -- 172.21.15.60:0/2850814146 shutdown_connections 2024-09-16T15:46:11.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.298+0000 7f3357400640 1 -- 172.21.15.60:0/2850814146 wait complete. 2024-09-16T15:46:11.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.299+0000 7f3357400640 1 Processor -- start 2024-09-16T15:46:11.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.299+0000 7f3357400640 1 -- start start 2024-09-16T15:46:11.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3357400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 0x7f335014ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:11.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3357400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500af3d0 0x7f335009d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3357400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3350161550 0x7f335015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3357400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f33501694d0 con 0x7f3350073e00 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3357400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3350169350 con 0x7f33500af3d0 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3357400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3350169650 con 0x7f3350161550 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3354a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 0x7f335014ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3354a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 0x7f335014ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60278/0 (socket says 172.21.15.60:60278) 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f3354a00640 1 -- 172.21.15.60:0/610029682 learned_addr learned my addr 172.21.15.60:0/610029682 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f3355400640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3350161550 0x7f335015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.300+0000 7f334fe00640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500af3d0 0x7f335009d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f3354a00640 1 -- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3350161550 msgr2=0x7f335015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f3354a00640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3350161550 0x7f335015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f3354a00640 1 -- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500af3d0 msgr2=0x7f335009d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f3354a00640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500af3d0 0x7f335009d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f3354a00640 1 -- 172.21.15.60:0/610029682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3350156780 con 0x7f3350073e00 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f334fe00640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500af3d0 0x7f335009d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f3354a00640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 0x7f335014ec30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f3344001220 tx=0x7f334400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f334cc00640 1 -- 172.21.15.60:0/610029682 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344011790 con 0x7f3350073e00 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f334cc00640 1 -- 172.21.15.60:0/610029682 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3344017070 con 0x7f3350073e00 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f334cc00640 1 -- 172.21.15.60:0/610029682 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344011cc0 con 0x7f3350073e00 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.301+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335009c080 con 0x7f3350073e00 2024-09-16T15:46:11.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.302+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335009c640 con 0x7f3350073e00 2024-09-16T15:46:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.302+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335006f710 con 0x7f3350073e00 2024-09-16T15:46:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.304+0000 7f334cc00640 1 -- 172.21.15.60:0/610029682 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f334401c460 con 0x7f3350073e00 2024-09-16T15:46:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.305+0000 7f334cc00640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3330078e80 0x7f333007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.305+0000 7f334cc00640 1 -- 172.21.15.60:0/610029682 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5737+0+0 (secure 0 0 0) 0x7f334409b240 con 0x7f3350073e00 2024-09-16T15:46:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.305+0000 7f334fe00640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3330078e80 0x7f333007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.306+0000 7f334fe00640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3330078e80 0x7f333007b340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f33500d0ce0 tx=0x7f333c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.309+0000 7f334cc00640 1 -- 172.21.15.60:0/610029682 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3344062e90 con 0x7f3350073e00 2024-09-16T15:46:11.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.506+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3350069ac0 con 0x7f3350073e00 2024-09-16T15:46:11.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.507+0000 7f334cc00640 1 -- 172.21.15.60:0/610029682 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+16796 (secure 0 0 0) 0x7f3344062790 con 0x7f3350073e00 2024-09-16T15:46:11.510 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:11.510 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":136,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:10.124923+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:07.910602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:09.091337+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:11.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3330078e80 msgr2=0x7f333007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3330078e80 0x7f333007b340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f33500d0ce0 tx=0x7f333c009040 comp rx=0 tx=0).stop 2024-09-16T15:46:11.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 msgr2=0x7f335014ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 0x7f335014ec30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f3344001220 tx=0x7f334400c890 comp rx=0 tx=0).stop 2024-09-16T15:46:11.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 shutdown_connections 2024-09-16T15:46:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3330078e80 0x7f333007b340 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3350161550 0x7f335015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33500af3d0 0x7f335009d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 --2- 172.21.15.60:0/610029682 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3350073e00 0x7f335014ec30 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 >> 172.21.15.60:0/610029682 conn(0x7f33500b1950 msgr2=0x7f3350162200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 shutdown_connections 2024-09-16T15:46:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.510+0000 7f3357400640 1 -- 172.21.15.60:0/610029682 wait complete. 2024-09-16T15:46:11.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:46:11.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.691+0000 7fa747600640 1 -- 172.21.15.60:0/297141368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 msgr2=0x7fa7400cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.691+0000 7fa747600640 1 --2- 172.21.15.60:0/297141368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 0x7fa7400cc5c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa728005fd0 tx=0x7fa72802a540 comp rx=0 tx=0).stop 2024-09-16T15:46:11.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.691+0000 7fa747600640 1 -- 172.21.15.60:0/297141368 shutdown_connections 2024-09-16T15:46:11.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.691+0000 7fa747600640 1 --2- 172.21.15.60:0/297141368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 0x7fa7400cc5c0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.691+0000 7fa747600640 1 --2- 172.21.15.60:0/297141368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa740145750 0x7fa740073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.691+0000 7fa747600640 1 -- 172.21.15.60:0/297141368 >> 172.21.15.60:0/297141368 conn(0x7fa7400b1840 msgr2=0x7fa7400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:11.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.691+0000 7fa747600640 1 -- 172.21.15.60:0/297141368 shutdown_connections 2024-09-16T15:46:11.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.691+0000 7fa747600640 1 -- 172.21.15.60:0/297141368 wait complete. 2024-09-16T15:46:11.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.693+0000 7fa747600640 1 Processor -- start 2024-09-16T15:46:11.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.693+0000 7fa747600640 1 -- start start 2024-09-16T15:46:11.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa747600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 0x7fa74015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:11.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa747600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7400ccb00 0x7fa740142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:11.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa747600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa740145750 0x7fa740142a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:11.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa747600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa7400b26b0 con 0x7fa74006efa0 2024-09-16T15:46:11.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa747600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7400cedd0 con 0x7fa740145750 2024-09-16T15:46:11.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa747600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa740160100 con 0x7fa7400ccb00 2024-09-16T15:46:11.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa744c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 0x7fa74015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:11.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa744c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 0x7fa74015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60296/0 (socket says 172.21.15.60:60296) 2024-09-16T15:46:11.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa744c00640 1 -- 172.21.15.60:0/702412549 learned_addr learned my addr 172.21.15.60:0/702412549 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:11.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa737e00640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7400ccb00 0x7fa740142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:11.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.694+0000 7fa745600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa740145750 0x7fa740142a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:11.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa744c00640 1 -- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7400ccb00 msgr2=0x7fa740142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa744c00640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7400ccb00 0x7fa740142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa744c00640 1 -- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa740145750 msgr2=0x7fa740142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa744c00640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa740145750 0x7fa740142a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa744c00640 1 -- 172.21.15.60:0/702412549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7401f4930 con 0x7fa74006efa0 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa745600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa740145750 0x7fa740142a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa744c00640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 0x7fa74015e6e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa73000d740 tx=0x7fa73000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa73d600640 1 -- 172.21.15.60:0/702412549 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730017070 con 0x7fa74006efa0 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa73d600640 1 -- 172.21.15.60:0/702412549 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa730009cc0 con 0x7fa74006efa0 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7401f4ba0 con 0x7fa74006efa0 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa73d600640 1 -- 172.21.15.60:0/702412549 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7300106c0 con 0x7fa74006efa0 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.695+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7401f5160 con 0x7fa74006efa0 2024-09-16T15:46:11.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.696+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7401605b0 con 0x7fa74006efa0 2024-09-16T15:46:11.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.697+0000 7fa73d600640 1 -- 172.21.15.60:0/702412549 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fa73001a070 con 0x7fa74006efa0 2024-09-16T15:46:11.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.698+0000 7fa73d600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa71c078ee0 0x7fa71c07b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:11.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.699+0000 7fa73d600640 1 -- 172.21.15.60:0/702412549 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5737+0+0 (secure 0 0 0) 0x7fa73009b630 con 0x7fa74006efa0 2024-09-16T15:46:11.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.699+0000 7fa737e00640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa71c078ee0 0x7fa71c07b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:11.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.699+0000 7fa737e00640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa71c078ee0 0x7fa71c07b3a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa728033000 tx=0x7fa728004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:11.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.703+0000 7fa73d600640 1 -- 172.21.15.60:0/702412549 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa730063280 con 0x7fa74006efa0 2024-09-16T15:46:11.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:11.942+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa740069a10 con 0x7fa74006efa0 2024-09-16T15:46:12.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.722+0000 7fa73d600640 1 -- 172.21.15.60:0/702412549 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v91) ==== 118+0+0 (secure 0 0 0) 0x7fa730062b80 con 0x7fa74006efa0 2024-09-16T15:46:12.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa71c078ee0 msgr2=0x7fa71c07b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:12.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa71c078ee0 0x7fa71c07b3a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa728033000 tx=0x7fa728004730 comp rx=0 tx=0).stop 2024-09-16T15:46:12.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 msgr2=0x7fa74015e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:12.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 0x7fa74015e6e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fa73000d740 tx=0x7fa73000dc00 comp rx=0 tx=0).stop 2024-09-16T15:46:12.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 shutdown_connections 2024-09-16T15:46:12.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa71c078ee0 0x7fa71c07b3a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa740145750 0x7fa740142a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa7400ccb00 0x7fa740142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 --2- 172.21.15.60:0/702412549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa74006efa0 0x7fa74015e6e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 >> 172.21.15.60:0/702412549 conn(0x7fa7400b1840 msgr2=0x7fa7400b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 shutdown_connections 2024-09-16T15:46:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.726+0000 7fa747600640 1 -- 172.21.15.60:0/702412549 wait complete. 2024-09-16T15:46:12.746 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:46:12.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:46:12.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.911+0000 7fe4d7600640 1 -- 172.21.15.60:0/4094241261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d0145750 msgr2=0x7fe4d0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.911+0000 7fe4d7600640 1 --2- 172.21.15.60:0/4094241261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d0145750 0x7fe4d0073440 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0005fd0 tx=0x7fe4c002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.911+0000 7fe4d7600640 1 -- 172.21.15.60:0/4094241261 shutdown_connections 2024-09-16T15:46:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.911+0000 7fe4d7600640 1 --2- 172.21.15.60:0/4094241261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4d00cccd0 0x7fe4d00cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.911+0000 7fe4d7600640 1 --2- 172.21.15.60:0/4094241261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d0145750 0x7fe4d0073440 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.911+0000 7fe4d7600640 1 -- 172.21.15.60:0/4094241261 >> 172.21.15.60:0/4094241261 conn(0x7fe4d00b1840 msgr2=0x7fe4d00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.911+0000 7fe4d7600640 1 -- 172.21.15.60:0/4094241261 shutdown_connections 2024-09-16T15:46:12.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.911+0000 7fe4d7600640 1 -- 172.21.15.60:0/4094241261 wait complete. 2024-09-16T15:46:12.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.913+0000 7fe4d7600640 1 Processor -- start 2024-09-16T15:46:12.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.913+0000 7fe4d7600640 1 -- start start 2024-09-16T15:46:12.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d7600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d006efa0 0x7fe4d01416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:12.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d7600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4d00cccd0 0x7fe4d0159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:12.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d7600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d0145750 0x7fe4d013e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:12.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d7600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00ce480 con 0x7fe4d006efa0 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d7600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00ce300 con 0x7fe4d0145750 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d7600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d00ce600 con 0x7fe4d00cccd0 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d006efa0 0x7fe4d01416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d006efa0 0x7fe4d01416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60316/0 (socket says 172.21.15.60:60316) 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d4c00640 1 -- 172.21.15.60:0/3925652296 learned_addr learned my addr 172.21.15.60:0/3925652296 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4c7e00640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4d00cccd0 0x7fe4d0159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.914+0000 7fe4d5600640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d0145750 0x7fe4d013e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4d4c00640 1 -- 172.21.15.60:0/3925652296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4d00cccd0 msgr2=0x7fe4d0159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4d4c00640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4d00cccd0 0x7fe4d0159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4d4c00640 1 -- 172.21.15.60:0/3925652296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d0145750 msgr2=0x7fe4d013e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4d4c00640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d0145750 0x7fe4d013e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4d4c00640 1 -- 172.21.15.60:0/3925652296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d015a180 con 0x7fe4d006efa0 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4d4c00640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d006efa0 0x7fe4d01416e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0034040 tx=0x7fe4c00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:12.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4cd600640 1 -- 172.21.15.60:0/3925652296 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c003e030 con 0x7fe4d006efa0 2024-09-16T15:46:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4cd600640 1 -- 172.21.15.60:0/3925652296 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4c00047a0 con 0x7fe4d006efa0 2024-09-16T15:46:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d00c4ab0 con 0x7fe4d006efa0 2024-09-16T15:46:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4cd600640 1 -- 172.21.15.60:0/3925652296 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0035040 con 0x7fe4d006efa0 2024-09-16T15:46:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.915+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d00c5070 con 0x7fe4d006efa0 2024-09-16T15:46:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.916+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d00cca70 con 0x7fe4d006efa0 2024-09-16T15:46:12.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.917+0000 7fe4cd600640 1 -- 172.21.15.60:0/3925652296 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fe4c0041030 con 0x7fe4d006efa0 2024-09-16T15:46:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.918+0000 7fe4cd600640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe4ac078e80 0x7fe4ac07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:12.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.919+0000 7fe4cd600640 1 -- 172.21.15.60:0/3925652296 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fe4c00bbb20 con 0x7fe4d006efa0 2024-09-16T15:46:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.919+0000 7fe4c7e00640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe4ac078e80 0x7fe4ac07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.919+0000 7fe4c7e00640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe4ac078e80 0x7fe4ac07b340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8004f60 tx=0x7fe4b8007e30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:12.923+0000 7fe4cd600640 1 -- 172.21.15.60:0/3925652296 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe4d00cca70 con 0x7fe4d006efa0 2024-09-16T15:46:13.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.156+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe4d006a370 con 0x7fe4d006efa0 2024-09-16T15:46:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.723+0000 7fe4cd600640 1 -- 172.21.15.60:0/3925652296 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v92) ==== 122+0+0 (secure 0 0 0) 0x7fe4d006a370 con 0x7fe4d006efa0 2024-09-16T15:46:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.726+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe4ac078e80 msgr2=0x7fe4ac07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.726+0000 7fe4d7600640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe4ac078e80 0x7fe4ac07b340 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8004f60 tx=0x7fe4b8007e30 comp rx=0 tx=0).stop 2024-09-16T15:46:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.726+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d006efa0 msgr2=0x7fe4d01416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.726+0000 7fe4d7600640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d006efa0 0x7fe4d01416e0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0034040 tx=0x7fe4c00039e0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.727+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 shutdown_connections 2024-09-16T15:46:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.727+0000 7fe4d7600640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe4ac078e80 0x7fe4ac07b340 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.727+0000 7fe4d7600640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4d0145750 0x7fe4d013e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.727+0000 7fe4d7600640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4d00cccd0 0x7fe4d0159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.727+0000 7fe4d7600640 1 --2- 172.21.15.60:0/3925652296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4d006efa0 0x7fe4d01416e0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.727+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 >> 172.21.15.60:0/3925652296 conn(0x7fe4d00b1840 msgr2=0x7fe4d00aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.727+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 shutdown_connections 2024-09-16T15:46:13.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.727+0000 7fe4d7600640 1 -- 172.21.15.60:0/3925652296 wait complete. 2024-09-16T15:46:13.747 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:46:13.747 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:13.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.910+0000 7fb043600640 1 -- 172.21.15.60:0/4005632929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15e6d0 msgr2=0x7fb03c06cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:13.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.910+0000 7fb043600640 1 --2- 172.21.15.60:0/4005632929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15e6d0 0x7fb03c06cb70 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb03c076530 tx=0x7fb02802a440 comp rx=0 tx=0).stop 2024-09-16T15:46:13.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.910+0000 7fb043600640 1 -- 172.21.15.60:0/4005632929 shutdown_connections 2024-09-16T15:46:13.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.910+0000 7fb043600640 1 --2- 172.21.15.60:0/4005632929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15e6d0 0x7fb03c06cb70 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.910+0000 7fb043600640 1 --2- 172.21.15.60:0/4005632929 >> v2:172.21.15.100:3300/0 conn(0x7fb03c0b3e20 0x7fb03c0688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.910+0000 7fb043600640 1 -- 172.21.15.60:0/4005632929 >> 172.21.15.60:0/4005632929 conn(0x7fb03c0b2230 msgr2=0x7fb03c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:13.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.910+0000 7fb043600640 1 -- 172.21.15.60:0/4005632929 shutdown_connections 2024-09-16T15:46:13.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.910+0000 7fb043600640 1 -- 172.21.15.60:0/4005632929 wait complete. 2024-09-16T15:46:13.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.911+0000 7fb043600640 1 Processor -- start 2024-09-16T15:46:13.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.912+0000 7fb043600640 1 -- start start 2024-09-16T15:46:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.912+0000 7fb043600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b3e20 0x7fb03c1f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.912+0000 7fb043600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15e6d0 0x7fb03c140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.912+0000 7fb043600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15eb60 0x7fb03c1f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb043600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c1629e0 con 0x7fb03c15eb60 2024-09-16T15:46:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb043600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c162860 con 0x7fb03c15e6d0 2024-09-16T15:46:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb043600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb03c162b60 con 0x7fb03c0b3e20 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb041600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15eb60 0x7fb03c1f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb041600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15eb60 0x7fb03c1f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60332/0 (socket says 172.21.15.60:60332) 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb041600640 1 -- 172.21.15.60:0/1241245169 learned_addr learned my addr 172.21.15.60:0/1241245169 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb040c00640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b3e20 0x7fb03c1f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb03be00640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15e6d0 0x7fb03c140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb041600640 1 -- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b3e20 msgr2=0x7fb03c1f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb041600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b3e20 0x7fb03c1f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb041600640 1 -- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15e6d0 msgr2=0x7fb03c140640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb041600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15e6d0 0x7fb03c140640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb041600640 1 -- 172.21.15.60:0/1241245169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03c1f9450 con 0x7fb03c15eb60 2024-09-16T15:46:13.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.913+0000 7fb03be00640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15e6d0 0x7fb03c140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.914+0000 7fb041600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15eb60 0x7fb03c1f5010 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb03400b970 tx=0x7fb03400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.914+0000 7fb038c00640 1 -- 172.21.15.60:0/1241245169 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034019070 con 0x7fb03c15eb60 2024-09-16T15:46:13.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.914+0000 7fb038c00640 1 -- 172.21.15.60:0/1241245169 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb03400cc30 con 0x7fb03c15eb60 2024-09-16T15:46:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.914+0000 7fb038c00640 1 -- 172.21.15.60:0/1241245169 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0340146a0 con 0x7fb03c15eb60 2024-09-16T15:46:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.914+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c1432a0 con 0x7fb03c15eb60 2024-09-16T15:46:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.914+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c143860 con 0x7fb03c15eb60 2024-09-16T15:46:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.915+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c06d330 con 0x7fb03c15eb60 2024-09-16T15:46:13.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.916+0000 7fb038c00640 1 -- 172.21.15.60:0/1241245169 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fb03401c070 con 0x7fb03c15eb60 2024-09-16T15:46:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.917+0000 7fb038c00640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb01c078e80 0x7fb01c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.917+0000 7fb038c00640 1 -- 172.21.15.60:0/1241245169 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fb03409b170 con 0x7fb03c15eb60 2024-09-16T15:46:13.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.918+0000 7fb040c00640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb01c078e80 0x7fb01c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:13.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.918+0000 7fb040c00640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb01c078e80 0x7fb01c07b340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb030005fd0 tx=0x7fb03000b390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:13.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:13.921+0000 7fb038c00640 1 -- 172.21.15.60:0/1241245169 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb03c1f9450 con 0x7fb03c15eb60 2024-09-16T15:46:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.160+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb03c0d0270 con 0x7fb03c15eb60 2024-09-16T15:46:14.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.160+0000 7fb038c00640 1 -- 172.21.15.60:0/1241245169 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7fb034062770 con 0x7fb03c15eb60 2024-09-16T15:46:14.162 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:14.163 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":92,"btime":"2024-09-16T15:46:13:723797+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/766495901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":766495901},{"type":"v1","addr":"172.21.15.100:6835","nonce":766495901}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7585,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/3797165836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3797165836},{"type":"v1","addr":"172.21.15.100:6801","nonce":3797165836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3900896825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3900896825},{"type":"v1","addr":"172.21.15.60:6834","nonce":3900896825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:10.124891+0000","modified":"2024-09-16T15:46:13.158701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/984341325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":984341325},{"type":"v1","addr":"172.21.15.60:6835","nonce":984341325}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7998,"qdb_cluster":[7998]},"id":10}]} 2024-09-16T15:46:14.163 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 92 2024-09-16T15:46:14.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb01c078e80 msgr2=0x7fb01c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb01c078e80 0x7fb01c07b340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb030005fd0 tx=0x7fb03000b390 comp rx=0 tx=0).stop 2024-09-16T15:46:14.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15eb60 msgr2=0x7fb03c1f5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15eb60 0x7fb03c1f5010 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb03400b970 tx=0x7fb03400be30 comp rx=0 tx=0).stop 2024-09-16T15:46:14.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 shutdown_connections 2024-09-16T15:46:14.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb01c078e80 0x7fb01c07b340 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c15eb60 0x7fb03c1f5010 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15e6d0 0x7fb03c140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 --2- 172.21.15.60:0/1241245169 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b3e20 0x7fb03c1f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 >> 172.21.15.60:0/1241245169 conn(0x7fb03c0b2230 msgr2=0x7fb03c15da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:14.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 shutdown_connections 2024-09-16T15:46:14.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.163+0000 7fb043600640 1 -- 172.21.15.60:0/1241245169 wait complete. 2024-09-16T15:46:14.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.338+0000 7f9c0a600640 1 -- 172.21.15.60:0/4148024551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 msgr2=0x7f9c0406f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.338+0000 7f9c0a600640 1 --2- 172.21.15.60:0/4148024551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 0x7f9c0406f660 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9c040b4890 tx=0x7f9bec02a560 comp rx=0 tx=0).stop 2024-09-16T15:46:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.338+0000 7f9c0a600640 1 -- 172.21.15.60:0/4148024551 shutdown_connections 2024-09-16T15:46:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.338+0000 7f9c0a600640 1 --2- 172.21.15.60:0/4148024551 >> v2:172.21.15.100:3300/0 conn(0x7f9c04072df0 0x7f9c04073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.338+0000 7f9c0a600640 1 --2- 172.21.15.60:0/4148024551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 0x7f9c0406f660 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.338+0000 7f9c0a600640 1 -- 172.21.15.60:0/4148024551 >> 172.21.15.60:0/4148024551 conn(0x7f9c040b1950 msgr2=0x7f9c040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.339+0000 7f9c0a600640 1 -- 172.21.15.60:0/4148024551 shutdown_connections 2024-09-16T15:46:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.339+0000 7f9c0a600640 1 -- 172.21.15.60:0/4148024551 wait complete. 2024-09-16T15:46:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.340+0000 7f9c0a600640 1 Processor -- start 2024-09-16T15:46:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.340+0000 7f9c0a600640 1 -- start start 2024-09-16T15:46:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9c0a600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c04072df0 0x7f9c041f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9c0a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 0x7f9c040c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9c0a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c04161550 0x7f9c040c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9c0a600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04073780 con 0x7f9c040af3d0 2024-09-16T15:46:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9c0a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0406fdb0 con 0x7f9c04161550 2024-09-16T15:46:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9c0a600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9c04073900 con 0x7f9c04072df0 2024-09-16T15:46:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9bfaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 0x7f9c040c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:14.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9bfaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 0x7f9c040c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60352/0 (socket says 172.21.15.60:60352) 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.341+0000 7f9bfaa00640 1 -- 172.21.15.60:0/2189583981 learned_addr learned my addr 172.21.15.60:0/2189583981 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9c03e00640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c04161550 0x7f9c040c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9c03400640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c04072df0 0x7f9c041f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfaa00640 1 -- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c04072df0 msgr2=0x7f9c041f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfaa00640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c04072df0 0x7f9c041f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfaa00640 1 -- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c04161550 msgr2=0x7f9c040c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfaa00640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c04161550 0x7f9c040c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfaa00640 1 -- 172.21.15.60:0/2189583981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c041f5a70 con 0x7f9c040af3d0 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9c03e00640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c04161550 0x7f9c040c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfaa00640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 0x7f9c040c8fd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009850 tx=0x7f9bec003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfbe00640 1 -- 172.21.15.60:0/2189583981 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec03f070 con 0x7f9c040af3d0 2024-09-16T15:46:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfbe00640 1 -- 172.21.15.60:0/2189583981 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bec0047a0 con 0x7f9c040af3d0 2024-09-16T15:46:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.342+0000 7f9bfbe00640 1 -- 172.21.15.60:0/2189583981 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec035030 con 0x7f9c040af3d0 2024-09-16T15:46:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.343+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c040cf710 con 0x7f9c040af3d0 2024-09-16T15:46:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.343+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c040cfcd0 con 0x7f9c040af3d0 2024-09-16T15:46:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.344+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0406ff60 con 0x7f9c040af3d0 2024-09-16T15:46:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.345+0000 7f9bfbe00640 1 -- 172.21.15.60:0/2189583981 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f9bec042070 con 0x7f9c040af3d0 2024-09-16T15:46:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.346+0000 7f9bfbe00640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9be0078e80 0x7f9be007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.346+0000 7f9bfbe00640 1 -- 172.21.15.60:0/2189583981 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f9bec0bc240 con 0x7f9c040af3d0 2024-09-16T15:46:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.346+0000 7f9c03400640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9be0078e80 0x7f9be007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.347+0000 7f9c03400640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9be0078e80 0x7f9be007b340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40015e0 tx=0x7f9bf4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:14.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.350+0000 7f9bfbe00640 1 -- 172.21.15.60:0/2189583981 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c0406ff60 con 0x7f9c040af3d0 2024-09-16T15:46:14.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.547+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c04073df0 con 0x7f9c040af3d0 2024-09-16T15:46:14.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.548+0000 7f9bfbe00640 1 -- 172.21.15.60:0/2189583981 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16783 (secure 0 0 0) 0x7f9bec084000 con 0x7f9c040af3d0 2024-09-16T15:46:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:14.551 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":137,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:11.704505+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:07.910602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:09.091337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9be0078e80 msgr2=0x7f9be007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9be0078e80 0x7f9be007b340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40015e0 tx=0x7f9bf4009040 comp rx=0 tx=0).stop 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 msgr2=0x7f9c040c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 0x7f9c040c8fd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009850 tx=0x7f9bec003910 comp rx=0 tx=0).stop 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 shutdown_connections 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9be0078e80 0x7f9be007b340 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c04161550 0x7f9c040c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c040af3d0 0x7f9c040c8fd0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 --2- 172.21.15.60:0/2189583981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c04072df0 0x7f9c041f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 >> 172.21.15.60:0/2189583981 conn(0x7f9c040b1950 msgr2=0x7f9c0415fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:14.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 shutdown_connections 2024-09-16T15:46:14.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.551+0000 7f9c0a600640 1 -- 172.21.15.60:0/2189583981 wait complete. 2024-09-16T15:46:14.570 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:46:14.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:46:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.728+0000 7f7e12c00640 1 -- 172.21.15.60:0/1747659072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 msgr2=0x7f7e0c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.728+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1747659072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 0x7f7e0c06f490 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c06d9e0 tx=0x7f7df802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.729+0000 7f7e12c00640 1 -- 172.21.15.60:0/1747659072 shutdown_connections 2024-09-16T15:46:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.729+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1747659072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 0x7f7e0c06f490 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.729+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1747659072 >> v2:172.21.15.100:3300/0 conn(0x7f7e0c161550 0x7f7e0c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.729+0000 7f7e12c00640 1 -- 172.21.15.60:0/1747659072 >> 172.21.15.60:0/1747659072 conn(0x7f7e0c0b1950 msgr2=0x7f7e0c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:14.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.729+0000 7f7e12c00640 1 -- 172.21.15.60:0/1747659072 shutdown_connections 2024-09-16T15:46:14.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.729+0000 7f7e12c00640 1 -- 172.21.15.60:0/1747659072 wait complete. 2024-09-16T15:46:14.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.730+0000 7f7e12c00640 1 Processor -- start 2024-09-16T15:46:14.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.730+0000 7f7e12c00640 1 -- start start 2024-09-16T15:46:14.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.731+0000 7f7e12c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e0c06f9d0 0x7f7e0c147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:14.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.731+0000 7f7e12c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 0x7f7e0c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:14.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.731+0000 7f7e12c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e0c161550 0x7f7e0c145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.731+0000 7f7e12c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c073f80 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.731+0000 7f7e12c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c073e00 con 0x7f7e0c161550 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.731+0000 7f7e12c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0c074100 con 0x7f7e0c06f9d0 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 0x7f7e0c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 0x7f7e0c144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60372/0 (socket says 172.21.15.60:60372) 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 -- 172.21.15.60:0/1924994716 learned_addr learned my addr 172.21.15.60:0/1924994716 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0be00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e0c06f9d0 0x7f7e0c147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e10c00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e0c161550 0x7f7e0c145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 -- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e0c06f9d0 msgr2=0x7f7e0c147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e0c06f9d0 0x7f7e0c147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 -- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e0c161550 msgr2=0x7f7e0c145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e0c161550 0x7f7e0c145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 -- 172.21.15.60:0/1924994716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c148340 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e10c00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e0c161550 0x7f7e0c145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:14.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.732+0000 7f7e0b400640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 0x7f7e0c144b00 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009780 tx=0x7f7df8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:14.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.733+0000 7f7e03e00640 1 -- 172.21.15.60:0/1924994716 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8040070 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.733+0000 7f7e03e00640 1 -- 172.21.15.60:0/1924994716 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7df8005310 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.733+0000 7f7e03e00640 1 -- 172.21.15.60:0/1924994716 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df80047a0 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.733+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c145040 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.733+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c0ce010 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.734+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e0c1626a0 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.735+0000 7f7e03e00640 1 -- 172.21.15.60:0/1924994716 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f7df8039040 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.736+0000 7f7e03e00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dec078eb0 0x7f7dec07b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:14.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.736+0000 7f7e03e00640 1 -- 172.21.15.60:0/1924994716 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f7df80bc690 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.736+0000 7f7e0be00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dec078eb0 0x7f7dec07b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:14.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.737+0000 7f7e0be00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dec078eb0 0x7f7dec07b370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009ac0 tx=0x7f7dfc005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:14.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.740+0000 7f7e03e00640 1 -- 172.21.15.60:0/1924994716 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7e0c148340 con 0x7f7e0c0af3d0 2024-09-16T15:46:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.932+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e0c074530 con 0x7f7dec078eb0 2024-09-16T15:46:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.937+0000 7f7e03e00640 1 -- 172.21.15.60:0/1924994716 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f7e0c074530 con 0x7f7dec078eb0 2024-09-16T15:46:14.942 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:14.943 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:46:14.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.941+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dec078eb0 msgr2=0x7f7dec07b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.941+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dec078eb0 0x7f7dec07b370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009ac0 tx=0x7f7dfc005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.941+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 msgr2=0x7f7e0c144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.941+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 0x7f7e0c144b00 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009780 tx=0x7f7df8032c10 comp rx=0 tx=0).stop 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.941+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 shutdown_connections 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.942+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7dec078eb0 0x7f7dec07b370 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.942+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e0c161550 0x7f7e0c145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.942+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e0c0af3d0 0x7f7e0c144b00 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.942+0000 7f7e12c00640 1 --2- 172.21.15.60:0/1924994716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e0c06f9d0 0x7f7e0c147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.942+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 >> 172.21.15.60:0/1924994716 conn(0x7f7e0c0b1950 msgr2=0x7f7e0c15ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.942+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 shutdown_connections 2024-09-16T15:46:14.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:14.942+0000 7f7e12c00640 1 -- 172.21.15.60:0/1924994716 wait complete. 2024-09-16T15:46:14.959 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-16T15:46:14.202046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":280656,"kb_used_data":14496,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754694064,"statfs":{"total":773094113280,"available":772806721536,"internally_reserved":0,"allocated":14843904,"data_stored":7185596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.528780"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.709077+0000","last_change":"2024-09-16T15:46:10.713882+0000","last_active":"2024-09-16T15:46:11.709077+0000","last_peered":"2024-09-16T15:46:11.709077+0000","last_clean":"2024-09-16T15:46:11.709077+0000","last_became_active":"2024-09-16T15:46:10.711575+0000","last_became_peered":"2024-09-16T15:46:10.711575+0000","last_unstale":"2024-09-16T15:46:11.709077+0000","last_undegraded":"2024-09-16T15:46:11.709077+0000","last_fullsized":"2024-09-16T15:46:11.709077+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:08:23.227443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.714842+0000","last_change":"2024-09-16T15:46:10.712701+0000","last_active":"2024-09-16T15:46:10.714842+0000","last_peered":"2024-09-16T15:46:10.714842+0000","last_clean":"2024-09-16T15:46:10.714842+0000","last_became_active":"2024-09-16T15:46:10.712364+0000","last_became_peered":"2024-09-16T15:46:10.712364+0000","last_unstale":"2024-09-16T15:46:10.714842+0000","last_undegraded":"2024-09-16T15:46:10.714842+0000","last_fullsized":"2024-09-16T15:46:10.714842+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:03.770333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712651+0000","last_change":"2024-09-16T15:46:10.707416+0000","last_active":"2024-09-16T15:46:11.712651+0000","last_peered":"2024-09-16T15:46:11.712651+0000","last_clean":"2024-09-16T15:46:11.712651+0000","last_became_active":"2024-09-16T15:46:10.704943+0000","last_became_peered":"2024-09-16T15:46:10.704943+0000","last_unstale":"2024-09-16T15:46:11.712651+0000","last_undegraded":"2024-09-16T15:46:11.712651+0000","last_fullsized":"2024-09-16T15:46:11.712651+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:44.390327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.714054+0000","last_change":"2024-09-16T15:46:10.714160+0000","last_active":"2024-09-16T15:46:11.714054+0000","last_peered":"2024-09-16T15:46:11.714054+0000","last_clean":"2024-09-16T15:46:11.714054+0000","last_became_active":"2024-09-16T15:46:10.713816+0000","last_became_peered":"2024-09-16T15:46:10.713816+0000","last_unstale":"2024-09-16T15:46:11.714054+0000","last_undegraded":"2024-09-16T15:46:11.714054+0000","last_fullsized":"2024-09-16T15:46:11.714054+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:19.687321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.718570+0000","last_change":"2024-09-16T15:46:10.716024+0000","last_active":"2024-09-16T15:46:10.718570+0000","last_peered":"2024-09-16T15:46:10.718570+0000","last_clean":"2024-09-16T15:46:10.718570+0000","last_became_active":"2024-09-16T15:46:10.712578+0000","last_became_peered":"2024-09-16T15:46:10.712578+0000","last_unstale":"2024-09-16T15:46:10.718570+0000","last_undegraded":"2024-09-16T15:46:10.718570+0000","last_fullsized":"2024-09-16T15:46:10.718570+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:21.801942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708545+0000","last_change":"2024-09-16T15:46:10.704457+0000","last_active":"2024-09-16T15:46:11.708545+0000","last_peered":"2024-09-16T15:46:11.708545+0000","last_clean":"2024-09-16T15:46:11.708545+0000","last_became_active":"2024-09-16T15:46:10.702645+0000","last_became_peered":"2024-09-16T15:46:10.702645+0000","last_unstale":"2024-09-16T15:46:11.708545+0000","last_undegraded":"2024-09-16T15:46:11.708545+0000","last_fullsized":"2024-09-16T15:46:11.708545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:57.842385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726148+0000","last_change":"2024-09-16T15:46:10.722407+0000","last_active":"2024-09-16T15:46:10.726148+0000","last_peered":"2024-09-16T15:46:10.726148+0000","last_clean":"2024-09-16T15:46:10.726148+0000","last_became_active":"2024-09-16T15:46:10.722116+0000","last_became_peered":"2024-09-16T15:46:10.722116+0000","last_unstale":"2024-09-16T15:46:10.726148+0000","last_undegraded":"2024-09-16T15:46:10.726148+0000","last_fullsized":"2024-09-16T15:46:10.726148+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:31:14.501049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708325+0000","last_change":"2024-09-16T15:46:10.702501+0000","last_active":"2024-09-16T15:46:11.708325+0000","last_peered":"2024-09-16T15:46:11.708325+0000","last_clean":"2024-09-16T15:46:11.708325+0000","last_became_active":"2024-09-16T15:46:10.702187+0000","last_became_peered":"2024-09-16T15:46:10.702187+0000","last_unstale":"2024-09-16T15:46:11.708325+0000","last_undegraded":"2024-09-16T15:46:11.708325+0000","last_fullsized":"2024-09-16T15:46:11.708325+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:52.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.719438+0000","last_change":"2024-09-16T15:46:10.717415+0000","last_active":"2024-09-16T15:46:10.719438+0000","last_peered":"2024-09-16T15:46:10.719438+0000","last_clean":"2024-09-16T15:46:10.719438+0000","last_became_active":"2024-09-16T15:46:10.717097+0000","last_became_peered":"2024-09-16T15:46:10.717097+0000","last_unstale":"2024-09-16T15:46:10.719438+0000","last_undegraded":"2024-09-16T15:46:10.719438+0000","last_fullsized":"2024-09-16T15:46:10.719438+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:19.687321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.720105+0000","last_change":"2024-09-16T15:46:10.717874+0000","last_active":"2024-09-16T15:46:10.720105+0000","last_peered":"2024-09-16T15:46:10.720105+0000","last_clean":"2024-09-16T15:46:10.720105+0000","last_became_active":"2024-09-16T15:46:10.717601+0000","last_became_peered":"2024-09-16T15:46:10.717601+0000","last_unstale":"2024-09-16T15:46:10.720105+0000","last_undegraded":"2024-09-16T15:46:10.720105+0000","last_fullsized":"2024-09-16T15:46:10.720105+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:59:47.021811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708669+0000","last_change":"2024-09-16T15:46:10.703289+0000","last_active":"2024-09-16T15:46:11.708669+0000","last_peered":"2024-09-16T15:46:11.708669+0000","last_clean":"2024-09-16T15:46:11.708669+0000","last_became_active":"2024-09-16T15:46:10.703031+0000","last_became_peered":"2024-09-16T15:46:10.703031+0000","last_unstale":"2024-09-16T15:46:11.708669+0000","last_undegraded":"2024-09-16T15:46:11.708669+0000","last_fullsized":"2024-09-16T15:46:11.708669+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:04.489710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.717962+0000","last_change":"2024-09-16T15:46:10.715278+0000","last_active":"2024-09-16T15:46:10.717962+0000","last_peered":"2024-09-16T15:46:10.717962+0000","last_clean":"2024-09-16T15:46:10.717962+0000","last_became_active":"2024-09-16T15:46:10.714653+0000","last_became_peered":"2024-09-16T15:46:10.714653+0000","last_unstale":"2024-09-16T15:46:10.717962+0000","last_undegraded":"2024-09-16T15:46:10.717962+0000","last_fullsized":"2024-09-16T15:46:10.717962+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:00:10.745271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.717328+0000","last_change":"2024-09-16T15:46:10.712832+0000","last_active":"2024-09-16T15:46:10.717328+0000","last_peered":"2024-09-16T15:46:10.717328+0000","last_clean":"2024-09-16T15:46:10.717328+0000","last_became_active":"2024-09-16T15:46:10.712235+0000","last_became_peered":"2024-09-16T15:46:10.712235+0000","last_unstale":"2024-09-16T15:46:10.717328+0000","last_undegraded":"2024-09-16T15:46:10.717328+0000","last_fullsized":"2024-09-16T15:46:10.717328+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:38:10.672795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713065+0000","last_change":"2024-09-16T15:46:10.722169+0000","last_active":"2024-09-16T15:46:11.713065+0000","last_peered":"2024-09-16T15:46:11.713065+0000","last_clean":"2024-09-16T15:46:11.713065+0000","last_became_active":"2024-09-16T15:46:10.721739+0000","last_became_peered":"2024-09-16T15:46:10.721739+0000","last_unstale":"2024-09-16T15:46:11.713065+0000","last_undegraded":"2024-09-16T15:46:11.713065+0000","last_fullsized":"2024-09-16T15:46:11.713065+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:59.315703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726502+0000","last_change":"2024-09-16T15:46:10.721308+0000","last_active":"2024-09-16T15:46:10.726502+0000","last_peered":"2024-09-16T15:46:10.726502+0000","last_clean":"2024-09-16T15:46:10.726502+0000","last_became_active":"2024-09-16T15:46:10.721008+0000","last_became_peered":"2024-09-16T15:46:10.721008+0000","last_unstale":"2024-09-16T15:46:10.726502+0000","last_undegraded":"2024-09-16T15:46:10.726502+0000","last_fullsized":"2024-09-16T15:46:10.726502+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:22.669833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.718350+0000","last_change":"2024-09-16T15:46:10.708949+0000","last_active":"2024-09-16T15:46:10.718350+0000","last_peered":"2024-09-16T15:46:10.718350+0000","last_clean":"2024-09-16T15:46:10.718350+0000","last_became_active":"2024-09-16T15:46:10.708337+0000","last_became_peered":"2024-09-16T15:46:10.708337+0000","last_unstale":"2024-09-16T15:46:10.718350+0000","last_undegraded":"2024-09-16T15:46:10.718350+0000","last_fullsized":"2024-09-16T15:46:10.718350+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:32.248943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.731747+0000","last_change":"2024-09-16T15:46:10.728760+0000","last_active":"2024-09-16T15:46:10.731747+0000","last_peered":"2024-09-16T15:46:10.731747+0000","last_clean":"2024-09-16T15:46:10.731747+0000","last_became_active":"2024-09-16T15:46:10.728487+0000","last_became_peered":"2024-09-16T15:46:10.728487+0000","last_unstale":"2024-09-16T15:46:10.731747+0000","last_undegraded":"2024-09-16T15:46:10.731747+0000","last_fullsized":"2024-09-16T15:46:10.731747+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:09.914742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712505+0000","last_change":"2024-09-16T15:46:10.704032+0000","last_active":"2024-09-16T15:46:11.712505+0000","last_peered":"2024-09-16T15:46:11.712505+0000","last_clean":"2024-09-16T15:46:11.712505+0000","last_became_active":"2024-09-16T15:46:10.703747+0000","last_became_peered":"2024-09-16T15:46:10.703747+0000","last_unstale":"2024-09-16T15:46:11.712505+0000","last_undegraded":"2024-09-16T15:46:11.712505+0000","last_fullsized":"2024-09-16T15:46:11.712505+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:06.606019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708906+0000","last_change":"2024-09-16T15:46:10.721313+0000","last_active":"2024-09-16T15:46:11.708906+0000","last_peered":"2024-09-16T15:46:11.708906+0000","last_clean":"2024-09-16T15:46:11.708906+0000","last_became_active":"2024-09-16T15:46:10.721025+0000","last_became_peered":"2024-09-16T15:46:10.721025+0000","last_unstale":"2024-09-16T15:46:11.708906+0000","last_undegraded":"2024-09-16T15:46:11.708906+0000","last_fullsized":"2024-09-16T15:46:11.708906+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:35.661793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.725133+0000","last_change":"2024-09-16T15:46:10.720101+0000","last_active":"2024-09-16T15:46:10.725133+0000","last_peered":"2024-09-16T15:46:10.725133+0000","last_clean":"2024-09-16T15:46:10.725133+0000","last_became_active":"2024-09-16T15:46:10.717917+0000","last_became_peered":"2024-09-16T15:46:10.717917+0000","last_unstale":"2024-09-16T15:46:10.725133+0000","last_undegraded":"2024-09-16T15:46:10.725133+0000","last_fullsized":"2024-09-16T15:46:10.725133+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:28.209785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726768+0000","last_change":"2024-09-16T15:46:10.721187+0000","last_active":"2024-09-16T15:46:10.726768+0000","last_peered":"2024-09-16T15:46:10.726768+0000","last_clean":"2024-09-16T15:46:10.726768+0000","last_became_active":"2024-09-16T15:46:10.720874+0000","last_became_peered":"2024-09-16T15:46:10.720874+0000","last_unstale":"2024-09-16T15:46:10.726768+0000","last_undegraded":"2024-09-16T15:46:10.726768+0000","last_fullsized":"2024-09-16T15:46:10.726768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:30.474038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713319+0000","last_change":"2024-09-16T15:46:10.721493+0000","last_active":"2024-09-16T15:46:11.713319+0000","last_peered":"2024-09-16T15:46:11.713319+0000","last_clean":"2024-09-16T15:46:11.713319+0000","last_became_active":"2024-09-16T15:46:10.720425+0000","last_became_peered":"2024-09-16T15:46:10.720425+0000","last_unstale":"2024-09-16T15:46:11.713319+0000","last_undegraded":"2024-09-16T15:46:11.713319+0000","last_fullsized":"2024-09-16T15:46:11.713319+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:03.770333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712557+0000","last_change":"2024-09-16T15:46:10.727062+0000","last_active":"2024-09-16T15:46:11.712557+0000","last_peered":"2024-09-16T15:46:11.712557+0000","last_clean":"2024-09-16T15:46:11.712557+0000","last_became_active":"2024-09-16T15:46:10.726763+0000","last_became_peered":"2024-09-16T15:46:10.726763+0000","last_unstale":"2024-09-16T15:46:11.712557+0000","last_undegraded":"2024-09-16T15:46:11.712557+0000","last_fullsized":"2024-09-16T15:46:11.712557+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:18.852522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.720071+0000","last_change":"2024-09-16T15:46:10.718029+0000","last_active":"2024-09-16T15:46:10.720071+0000","last_peered":"2024-09-16T15:46:10.720071+0000","last_clean":"2024-09-16T15:46:10.720071+0000","last_became_active":"2024-09-16T15:46:10.717750+0000","last_became_peered":"2024-09-16T15:46:10.717750+0000","last_unstale":"2024-09-16T15:46:10.720071+0000","last_undegraded":"2024-09-16T15:46:10.720071+0000","last_fullsized":"2024-09-16T15:46:10.720071+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:20:35.235501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.725260+0000","last_change":"2024-09-16T15:46:10.718427+0000","last_active":"2024-09-16T15:46:10.725260+0000","last_peered":"2024-09-16T15:46:10.725260+0000","last_clean":"2024-09-16T15:46:10.725260+0000","last_became_active":"2024-09-16T15:46:10.718147+0000","last_became_peered":"2024-09-16T15:46:10.718147+0000","last_unstale":"2024-09-16T15:46:10.725260+0000","last_undegraded":"2024-09-16T15:46:10.725260+0000","last_fullsized":"2024-09-16T15:46:10.725260+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:58.163846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708847+0000","last_change":"2024-09-16T15:46:10.727302+0000","last_active":"2024-09-16T15:46:11.708847+0000","last_peered":"2024-09-16T15:46:11.708847+0000","last_clean":"2024-09-16T15:46:11.708847+0000","last_became_active":"2024-09-16T15:46:10.727012+0000","last_became_peered":"2024-09-16T15:46:10.727012+0000","last_unstale":"2024-09-16T15:46:11.708847+0000","last_undegraded":"2024-09-16T15:46:11.708847+0000","last_fullsized":"2024-09-16T15:46:11.708847+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:07.875457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.724408+0000","last_change":"2024-09-16T15:46:10.715286+0000","last_active":"2024-09-16T15:46:10.724408+0000","last_peered":"2024-09-16T15:46:10.724408+0000","last_clean":"2024-09-16T15:46:10.724408+0000","last_became_active":"2024-09-16T15:46:10.713801+0000","last_became_peered":"2024-09-16T15:46:10.713801+0000","last_unstale":"2024-09-16T15:46:10.724408+0000","last_undegraded":"2024-09-16T15:46:10.724408+0000","last_fullsized":"2024-09-16T15:46:10.724408+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:39.025107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.724818+0000","last_change":"2024-09-16T15:46:10.716088+0000","last_active":"2024-09-16T15:46:10.724818+0000","last_peered":"2024-09-16T15:46:10.724818+0000","last_clean":"2024-09-16T15:46:10.724818+0000","last_became_active":"2024-09-16T15:46:10.715794+0000","last_became_peered":"2024-09-16T15:46:10.715794+0000","last_unstale":"2024-09-16T15:46:10.724818+0000","last_undegraded":"2024-09-16T15:46:10.724818+0000","last_fullsized":"2024-09-16T15:46:10.724818+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:02.582741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708376+0000","last_change":"2024-09-16T15:46:10.703838+0000","last_active":"2024-09-16T15:46:11.708376+0000","last_peered":"2024-09-16T15:46:11.708376+0000","last_clean":"2024-09-16T15:46:11.708376+0000","last_became_active":"2024-09-16T15:46:10.702228+0000","last_became_peered":"2024-09-16T15:46:10.702228+0000","last_unstale":"2024-09-16T15:46:11.708376+0000","last_undegraded":"2024-09-16T15:46:11.708376+0000","last_fullsized":"2024-09-16T15:46:11.708376+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:21.750497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.714116+0000","last_change":"2024-09-16T15:46:10.716202+0000","last_active":"2024-09-16T15:46:11.714116+0000","last_peered":"2024-09-16T15:46:11.714116+0000","last_clean":"2024-09-16T15:46:11.714116+0000","last_became_active":"2024-09-16T15:46:10.714205+0000","last_became_peered":"2024-09-16T15:46:10.714205+0000","last_unstale":"2024-09-16T15:46:11.714116+0000","last_undegraded":"2024-09-16T15:46:11.714116+0000","last_fullsized":"2024-09-16T15:46:11.714116+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:22.513670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.719067+0000","last_change":"2024-09-16T15:46:10.717802+0000","last_active":"2024-09-16T15:46:10.719067+0000","last_peered":"2024-09-16T15:46:10.719067+0000","last_clean":"2024-09-16T15:46:10.719067+0000","last_became_active":"2024-09-16T15:46:10.714182+0000","last_became_peered":"2024-09-16T15:46:10.714182+0000","last_unstale":"2024-09-16T15:46:10.719067+0000","last_undegraded":"2024-09-16T15:46:10.719067+0000","last_fullsized":"2024-09-16T15:46:10.719067+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:22.513670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.718378+0000","last_change":"2024-09-16T15:46:10.709178+0000","last_active":"2024-09-16T15:46:10.718378+0000","last_peered":"2024-09-16T15:46:10.718378+0000","last_clean":"2024-09-16T15:46:10.718378+0000","last_became_active":"2024-09-16T15:46:10.708613+0000","last_became_peered":"2024-09-16T15:46:10.708613+0000","last_unstale":"2024-09-16T15:46:10.718378+0000","last_undegraded":"2024-09-16T15:46:10.718378+0000","last_fullsized":"2024-09-16T15:46:10.718378+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:14.210957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.718830+0000","last_change":"2024-09-16T15:46:10.715777+0000","last_active":"2024-09-16T15:46:10.718830+0000","last_peered":"2024-09-16T15:46:10.718830+0000","last_clean":"2024-09-16T15:46:10.718830+0000","last_became_active":"2024-09-16T15:46:10.714528+0000","last_became_peered":"2024-09-16T15:46:10.714528+0000","last_unstale":"2024-09-16T15:46:10.718830+0000","last_undegraded":"2024-09-16T15:46:10.718830+0000","last_fullsized":"2024-09-16T15:46:10.718830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:34:21.839682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"136'4","reported_seq":17,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.733854+0000","last_change":"2024-09-16T15:46:09.701700+0000","last_active":"2024-09-16T15:46:10.733854+0000","last_peered":"2024-09-16T15:46:10.733854+0000","last_clean":"2024-09-16T15:46:10.733854+0000","last_became_active":"2024-09-16T15:46:09.701420+0000","last_became_peered":"2024-09-16T15:46:09.701420+0000","last_unstale":"2024-09-16T15:46:10.733854+0000","last_undegraded":"2024-09-16T15:46:10.733854+0000","last_fullsized":"2024-09-16T15:46:10.733854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:23.313256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.713425+0000","last_change":"2024-09-16T15:46:10.707922+0000","last_active":"2024-09-16T15:46:10.713425+0000","last_peered":"2024-09-16T15:46:10.713425+0000","last_clean":"2024-09-16T15:46:10.713425+0000","last_became_active":"2024-09-16T15:46:10.706230+0000","last_became_peered":"2024-09-16T15:46:10.706230+0000","last_unstale":"2024-09-16T15:46:10.713425+0000","last_undegraded":"2024-09-16T15:46:10.713425+0000","last_fullsized":"2024-09-16T15:46:10.713425+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:18.852522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.733776+0000","last_change":"2024-09-16T15:46:09.688699+0000","last_active":"2024-09-16T15:46:10.733776+0000","last_peered":"2024-09-16T15:46:10.733776+0000","last_clean":"2024-09-16T15:46:10.733776+0000","last_became_active":"2024-09-16T15:46:09.688373+0000","last_became_peered":"2024-09-16T15:46:09.688373+0000","last_unstale":"2024-09-16T15:46:10.733776+0000","last_undegraded":"2024-09-16T15:46:10.733776+0000","last_fullsized":"2024-09-16T15:46:10.733776+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:29.405268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708447+0000","last_change":"2024-09-16T15:46:10.719170+0000","last_active":"2024-09-16T15:46:11.708447+0000","last_peered":"2024-09-16T15:46:11.708447+0000","last_clean":"2024-09-16T15:46:11.708447+0000","last_became_active":"2024-09-16T15:46:10.718809+0000","last_became_peered":"2024-09-16T15:46:10.718809+0000","last_unstale":"2024-09-16T15:46:11.708447+0000","last_undegraded":"2024-09-16T15:46:11.708447+0000","last_fullsized":"2024-09-16T15:46:11.708447+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:58.113588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"136'2","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713757+0000","last_change":"2024-09-16T15:46:09.702133+0000","last_active":"2024-09-16T15:46:11.713757+0000","last_peered":"2024-09-16T15:46:11.713757+0000","last_clean":"2024-09-16T15:46:11.713757+0000","last_became_active":"2024-09-16T15:46:09.701794+0000","last_became_peered":"2024-09-16T15:46:09.701794+0000","last_unstale":"2024-09-16T15:46:11.713757+0000","last_undegraded":"2024-09-16T15:46:11.713757+0000","last_fullsized":"2024-09-16T15:46:11.713757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:59.298193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.719689+0000","last_change":"2024-09-16T15:46:10.716550+0000","last_active":"2024-09-16T15:46:10.719689+0000","last_peered":"2024-09-16T15:46:10.719689+0000","last_clean":"2024-09-16T15:46:10.719689+0000","last_became_active":"2024-09-16T15:46:10.716028+0000","last_became_peered":"2024-09-16T15:46:10.716028+0000","last_unstale":"2024-09-16T15:46:10.719689+0000","last_undegraded":"2024-09-16T15:46:10.719689+0000","last_fullsized":"2024-09-16T15:46:10.719689+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:54.253136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"136'2","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.714015+0000","last_change":"2024-09-16T15:46:09.695904+0000","last_active":"2024-09-16T15:46:11.714015+0000","last_peered":"2024-09-16T15:46:11.714015+0000","last_clean":"2024-09-16T15:46:11.714015+0000","last_became_active":"2024-09-16T15:46:09.695505+0000","last_became_peered":"2024-09-16T15:46:09.695505+0000","last_unstale":"2024-09-16T15:46:11.714015+0000","last_undegraded":"2024-09-16T15:46:11.714015+0000","last_fullsized":"2024-09-16T15:46:11.714015+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:47.891123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.716347+0000","last_change":"2024-09-16T15:46:10.715505+0000","last_active":"2024-09-16T15:46:10.716347+0000","last_peered":"2024-09-16T15:46:10.716347+0000","last_clean":"2024-09-16T15:46:10.716347+0000","last_became_active":"2024-09-16T15:46:10.715238+0000","last_became_peered":"2024-09-16T15:46:10.715238+0000","last_unstale":"2024-09-16T15:46:10.716347+0000","last_undegraded":"2024-09-16T15:46:10.716347+0000","last_fullsized":"2024-09-16T15:46:10.716347+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:54.969339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.734657+0000","last_change":"2024-09-16T15:46:09.699289+0000","last_active":"2024-09-16T15:46:10.734657+0000","last_peered":"2024-09-16T15:46:10.734657+0000","last_clean":"2024-09-16T15:46:10.734657+0000","last_became_active":"2024-09-16T15:46:09.698941+0000","last_became_peered":"2024-09-16T15:46:09.698941+0000","last_unstale":"2024-09-16T15:46:10.734657+0000","last_undegraded":"2024-09-16T15:46:10.734657+0000","last_fullsized":"2024-09-16T15:46:10.734657+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:09.763393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713068+0000","last_change":"2024-09-16T15:46:10.714337+0000","last_active":"2024-09-16T15:46:11.713068+0000","last_peered":"2024-09-16T15:46:11.713068+0000","last_clean":"2024-09-16T15:46:11.713068+0000","last_became_active":"2024-09-16T15:46:10.714060+0000","last_became_peered":"2024-09-16T15:46:10.714060+0000","last_unstale":"2024-09-16T15:46:11.713068+0000","last_undegraded":"2024-09-16T15:46:11.713068+0000","last_fullsized":"2024-09-16T15:46:11.713068+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:56.127578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.732184+0000","last_change":"2024-09-16T15:46:09.701485+0000","last_active":"2024-09-16T15:46:10.732184+0000","last_peered":"2024-09-16T15:46:10.732184+0000","last_clean":"2024-09-16T15:46:10.732184+0000","last_became_active":"2024-09-16T15:46:09.700682+0000","last_became_peered":"2024-09-16T15:46:09.700682+0000","last_unstale":"2024-09-16T15:46:10.732184+0000","last_undegraded":"2024-09-16T15:46:10.732184+0000","last_fullsized":"2024-09-16T15:46:10.732184+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:26.201745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.713796+0000","last_change":"2024-09-16T15:46:10.710349+0000","last_active":"2024-09-16T15:46:10.713796+0000","last_peered":"2024-09-16T15:46:10.713796+0000","last_clean":"2024-09-16T15:46:10.713796+0000","last_became_active":"2024-09-16T15:46:10.710076+0000","last_became_peered":"2024-09-16T15:46:10.710076+0000","last_unstale":"2024-09-16T15:46:10.713796+0000","last_undegraded":"2024-09-16T15:46:10.713796+0000","last_fullsized":"2024-09-16T15:46:10.713796+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:59.315703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.706119+0000","last_change":"2024-09-16T15:46:09.700678+0000","last_active":"2024-09-16T15:46:10.706119+0000","last_peered":"2024-09-16T15:46:10.706119+0000","last_clean":"2024-09-16T15:46:10.706119+0000","last_became_active":"2024-09-16T15:46:09.700422+0000","last_became_peered":"2024-09-16T15:46:09.700422+0000","last_unstale":"2024-09-16T15:46:10.706119+0000","last_undegraded":"2024-09-16T15:46:10.706119+0000","last_fullsized":"2024-09-16T15:46:10.706119+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:19.962361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.711330+0000","last_change":"2024-09-16T15:46:10.706685+0000","last_active":"2024-09-16T15:46:10.711330+0000","last_peered":"2024-09-16T15:46:10.711330+0000","last_clean":"2024-09-16T15:46:10.711330+0000","last_became_active":"2024-09-16T15:46:10.706428+0000","last_became_peered":"2024-09-16T15:46:10.706428+0000","last_unstale":"2024-09-16T15:46:10.711330+0000","last_undegraded":"2024-09-16T15:46:10.711330+0000","last_fullsized":"2024-09-16T15:46:10.711330+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:03.889160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.711894+0000","last_change":"2024-09-16T15:46:09.696972+0000","last_active":"2024-09-16T15:46:11.711894+0000","last_peered":"2024-09-16T15:46:11.711894+0000","last_clean":"2024-09-16T15:46:11.711894+0000","last_became_active":"2024-09-16T15:46:09.695680+0000","last_became_peered":"2024-09-16T15:46:09.695680+0000","last_unstale":"2024-09-16T15:46:11.711894+0000","last_undegraded":"2024-09-16T15:46:11.711894+0000","last_fullsized":"2024-09-16T15:46:11.711894+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:16.309370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.728155+0000","last_change":"2024-09-16T15:46:10.722977+0000","last_active":"2024-09-16T15:46:10.728155+0000","last_peered":"2024-09-16T15:46:10.728155+0000","last_clean":"2024-09-16T15:46:10.728155+0000","last_became_active":"2024-09-16T15:46:10.722226+0000","last_became_peered":"2024-09-16T15:46:10.722226+0000","last_unstale":"2024-09-16T15:46:10.728155+0000","last_undegraded":"2024-09-16T15:46:10.728155+0000","last_fullsized":"2024-09-16T15:46:10.728155+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:40.841651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.700840+0000","last_change":"2024-09-16T15:46:09.694247+0000","last_active":"2024-09-16T15:46:10.700840+0000","last_peered":"2024-09-16T15:46:10.700840+0000","last_clean":"2024-09-16T15:46:10.700840+0000","last_became_active":"2024-09-16T15:46:09.693989+0000","last_became_peered":"2024-09-16T15:46:09.693989+0000","last_unstale":"2024-09-16T15:46:10.700840+0000","last_undegraded":"2024-09-16T15:46:10.700840+0000","last_fullsized":"2024-09-16T15:46:10.700840+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:10.240327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.732197+0000","last_change":"2024-09-16T15:46:10.728747+0000","last_active":"2024-09-16T15:46:10.732197+0000","last_peered":"2024-09-16T15:46:10.732197+0000","last_clean":"2024-09-16T15:46:10.732197+0000","last_became_active":"2024-09-16T15:46:10.728461+0000","last_became_peered":"2024-09-16T15:46:10.728461+0000","last_unstale":"2024-09-16T15:46:10.732197+0000","last_undegraded":"2024-09-16T15:46:10.732197+0000","last_fullsized":"2024-09-16T15:46:10.732197+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:11.026978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.707631+0000","last_change":"2024-09-16T15:46:09.701879+0000","last_active":"2024-09-16T15:46:10.707631+0000","last_peered":"2024-09-16T15:46:10.707631+0000","last_clean":"2024-09-16T15:46:10.707631+0000","last_became_active":"2024-09-16T15:46:09.701105+0000","last_became_peered":"2024-09-16T15:46:09.701105+0000","last_unstale":"2024-09-16T15:46:10.707631+0000","last_undegraded":"2024-09-16T15:46:10.707631+0000","last_fullsized":"2024-09-16T15:46:10.707631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:18.597100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.724380+0000","last_change":"2024-09-16T15:46:10.713686+0000","last_active":"2024-09-16T15:46:10.724380+0000","last_peered":"2024-09-16T15:46:10.724380+0000","last_clean":"2024-09-16T15:46:10.724380+0000","last_became_active":"2024-09-16T15:46:10.713397+0000","last_became_peered":"2024-09-16T15:46:10.713397+0000","last_unstale":"2024-09-16T15:46:10.724380+0000","last_undegraded":"2024-09-16T15:46:10.724380+0000","last_fullsized":"2024-09-16T15:46:10.724380+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:49.518295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"136'6","reported_seq":19,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.736958+0000","last_change":"2024-09-16T15:46:09.703610+0000","last_active":"2024-09-16T15:46:10.736958+0000","last_peered":"2024-09-16T15:46:10.736958+0000","last_clean":"2024-09-16T15:46:10.736958+0000","last_became_active":"2024-09-16T15:46:09.703304+0000","last_became_peered":"2024-09-16T15:46:09.703304+0000","last_unstale":"2024-09-16T15:46:10.736958+0000","last_undegraded":"2024-09-16T15:46:10.736958+0000","last_fullsized":"2024-09-16T15:46:10.736958+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:21.507800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712503+0000","last_change":"2024-09-16T15:46:10.721207+0000","last_active":"2024-09-16T15:46:11.712503+0000","last_peered":"2024-09-16T15:46:11.712503+0000","last_clean":"2024-09-16T15:46:11.712503+0000","last_became_active":"2024-09-16T15:46:10.720911+0000","last_became_peered":"2024-09-16T15:46:10.720911+0000","last_unstale":"2024-09-16T15:46:11.712503+0000","last_undegraded":"2024-09-16T15:46:11.712503+0000","last_fullsized":"2024-09-16T15:46:11.712503+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:17.089130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"136'2","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713496+0000","last_change":"2024-09-16T15:46:09.691911+0000","last_active":"2024-09-16T15:46:11.713496+0000","last_peered":"2024-09-16T15:46:11.713496+0000","last_clean":"2024-09-16T15:46:11.713496+0000","last_became_active":"2024-09-16T15:46:09.691628+0000","last_became_peered":"2024-09-16T15:46:09.691628+0000","last_unstale":"2024-09-16T15:46:11.713496+0000","last_undegraded":"2024-09-16T15:46:11.713496+0000","last_fullsized":"2024-09-16T15:46:11.713496+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:32.128628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.713874+0000","last_change":"2024-09-16T15:46:10.711816+0000","last_active":"2024-09-16T15:46:10.713874+0000","last_peered":"2024-09-16T15:46:10.713874+0000","last_clean":"2024-09-16T15:46:10.713874+0000","last_became_active":"2024-09-16T15:46:10.711544+0000","last_became_peered":"2024-09-16T15:46:10.711544+0000","last_unstale":"2024-09-16T15:46:10.713874+0000","last_undegraded":"2024-09-16T15:46:10.713874+0000","last_fullsized":"2024-09-16T15:46:10.713874+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:02.796248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712522+0000","last_change":"2024-09-16T15:46:09.700489+0000","last_active":"2024-09-16T15:46:11.712522+0000","last_peered":"2024-09-16T15:46:11.712522+0000","last_clean":"2024-09-16T15:46:11.712522+0000","last_became_active":"2024-09-16T15:46:09.700171+0000","last_became_peered":"2024-09-16T15:46:09.700171+0000","last_unstale":"2024-09-16T15:46:11.712522+0000","last_undegraded":"2024-09-16T15:46:11.712522+0000","last_fullsized":"2024-09-16T15:46:11.712522+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:32.697309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.713570+0000","last_change":"2024-09-16T15:46:10.711414+0000","last_active":"2024-09-16T15:46:10.713570+0000","last_peered":"2024-09-16T15:46:10.713570+0000","last_clean":"2024-09-16T15:46:10.713570+0000","last_became_active":"2024-09-16T15:46:10.709551+0000","last_became_peered":"2024-09-16T15:46:10.709551+0000","last_unstale":"2024-09-16T15:46:10.713570+0000","last_undegraded":"2024-09-16T15:46:10.713570+0000","last_fullsized":"2024-09-16T15:46:10.713570+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:38:37.170762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708537+0000","last_change":"2024-09-16T15:46:09.699133+0000","last_active":"2024-09-16T15:46:11.708537+0000","last_peered":"2024-09-16T15:46:11.708537+0000","last_clean":"2024-09-16T15:46:11.708537+0000","last_became_active":"2024-09-16T15:46:09.698839+0000","last_became_peered":"2024-09-16T15:46:09.698839+0000","last_unstale":"2024-09-16T15:46:11.708537+0000","last_undegraded":"2024-09-16T15:46:11.708537+0000","last_fullsized":"2024-09-16T15:46:11.708537+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:54.846393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712262+0000","last_change":"2024-09-16T15:46:10.717064+0000","last_active":"2024-09-16T15:46:11.712262+0000","last_peered":"2024-09-16T15:46:11.712262+0000","last_clean":"2024-09-16T15:46:11.712262+0000","last_became_active":"2024-09-16T15:46:10.716761+0000","last_became_peered":"2024-09-16T15:46:10.716761+0000","last_unstale":"2024-09-16T15:46:11.712262+0000","last_undegraded":"2024-09-16T15:46:11.712262+0000","last_fullsized":"2024-09-16T15:46:11.712262+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:50.794588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"136'8","reported_seq":21,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.735592+0000","last_change":"2024-09-16T15:46:09.701819+0000","last_active":"2024-09-16T15:46:10.735592+0000","last_peered":"2024-09-16T15:46:10.735592+0000","last_clean":"2024-09-16T15:46:10.735592+0000","last_became_active":"2024-09-16T15:46:09.701557+0000","last_became_peered":"2024-09-16T15:46:09.701557+0000","last_unstale":"2024-09-16T15:46:10.735592+0000","last_undegraded":"2024-09-16T15:46:10.735592+0000","last_fullsized":"2024-09-16T15:46:10.735592+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:49.066249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.717157+0000","last_change":"2024-09-16T15:46:10.714133+0000","last_active":"2024-09-16T15:46:10.717157+0000","last_peered":"2024-09-16T15:46:10.717157+0000","last_clean":"2024-09-16T15:46:10.717157+0000","last_became_active":"2024-09-16T15:46:10.713739+0000","last_became_peered":"2024-09-16T15:46:10.713739+0000","last_unstale":"2024-09-16T15:46:10.717157+0000","last_undegraded":"2024-09-16T15:46:10.717157+0000","last_fullsized":"2024-09-16T15:46:10.717157+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:04.300237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.732697+0000","last_change":"2024-09-16T15:46:09.698797+0000","last_active":"2024-09-16T15:46:10.732697+0000","last_peered":"2024-09-16T15:46:10.732697+0000","last_clean":"2024-09-16T15:46:10.732697+0000","last_became_active":"2024-09-16T15:46:09.697903+0000","last_became_peered":"2024-09-16T15:46:09.697903+0000","last_unstale":"2024-09-16T15:46:10.732697+0000","last_undegraded":"2024-09-16T15:46:10.732697+0000","last_fullsized":"2024-09-16T15:46:10.732697+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:43.595929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.725237+0000","last_change":"2024-09-16T15:46:10.723077+0000","last_active":"2024-09-16T15:46:10.725237+0000","last_peered":"2024-09-16T15:46:10.725237+0000","last_clean":"2024-09-16T15:46:10.725237+0000","last_became_active":"2024-09-16T15:46:10.722833+0000","last_became_peered":"2024-09-16T15:46:10.722833+0000","last_unstale":"2024-09-16T15:46:10.725237+0000","last_undegraded":"2024-09-16T15:46:10.725237+0000","last_fullsized":"2024-09-16T15:46:10.725237+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:40:47.747060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"136'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.733636+0000","last_change":"2024-09-16T15:46:09.688516+0000","last_active":"2024-09-16T15:46:10.733636+0000","last_peered":"2024-09-16T15:46:10.733636+0000","last_clean":"2024-09-16T15:46:10.733636+0000","last_became_active":"2024-09-16T15:46:09.687004+0000","last_became_peered":"2024-09-16T15:46:09.687004+0000","last_unstale":"2024-09-16T15:46:10.733636+0000","last_undegraded":"2024-09-16T15:46:10.733636+0000","last_fullsized":"2024-09-16T15:46:10.733636+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:13.774780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713204+0000","last_change":"2024-09-16T15:46:10.711204+0000","last_active":"2024-09-16T15:46:11.713204+0000","last_peered":"2024-09-16T15:46:11.713204+0000","last_clean":"2024-09-16T15:46:11.713204+0000","last_became_active":"2024-09-16T15:46:10.710653+0000","last_became_peered":"2024-09-16T15:46:10.710653+0000","last_unstale":"2024-09-16T15:46:11.713204+0000","last_undegraded":"2024-09-16T15:46:11.713204+0000","last_fullsized":"2024-09-16T15:46:11.713204+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:24.896188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.734817+0000","last_change":"2024-09-16T15:46:09.699732+0000","last_active":"2024-09-16T15:46:10.734817+0000","last_peered":"2024-09-16T15:46:10.734817+0000","last_clean":"2024-09-16T15:46:10.734817+0000","last_became_active":"2024-09-16T15:46:09.698125+0000","last_became_peered":"2024-09-16T15:46:09.698125+0000","last_unstale":"2024-09-16T15:46:10.734817+0000","last_undegraded":"2024-09-16T15:46:10.734817+0000","last_fullsized":"2024-09-16T15:46:10.734817+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:33.769250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708690+0000","last_change":"2024-09-16T15:46:10.717112+0000","last_active":"2024-09-16T15:46:11.708690+0000","last_peered":"2024-09-16T15:46:11.708690+0000","last_clean":"2024-09-16T15:46:11.708690+0000","last_became_active":"2024-09-16T15:46:10.716833+0000","last_became_peered":"2024-09-16T15:46:10.716833+0000","last_unstale":"2024-09-16T15:46:11.708690+0000","last_undegraded":"2024-09-16T15:46:11.708690+0000","last_fullsized":"2024-09-16T15:46:11.708690+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:43.792559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.700375+0000","last_change":"2024-09-16T15:46:09.697393+0000","last_active":"2024-09-16T15:46:10.700375+0000","last_peered":"2024-09-16T15:46:10.700375+0000","last_clean":"2024-09-16T15:46:10.700375+0000","last_became_active":"2024-09-16T15:46:09.697109+0000","last_became_peered":"2024-09-16T15:46:09.697109+0000","last_unstale":"2024-09-16T15:46:10.700375+0000","last_undegraded":"2024-09-16T15:46:10.700375+0000","last_fullsized":"2024-09-16T15:46:10.700375+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:16.005248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713566+0000","last_change":"2024-09-16T15:46:10.716674+0000","last_active":"2024-09-16T15:46:11.713566+0000","last_peered":"2024-09-16T15:46:11.713566+0000","last_clean":"2024-09-16T15:46:11.713566+0000","last_became_active":"2024-09-16T15:46:10.713806+0000","last_became_peered":"2024-09-16T15:46:10.713806+0000","last_unstale":"2024-09-16T15:46:11.713566+0000","last_undegraded":"2024-09-16T15:46:11.713566+0000","last_fullsized":"2024-09-16T15:46:11.713566+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:54.969339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"136'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.731609+0000","last_change":"2024-09-16T15:46:09.690842+0000","last_active":"2024-09-16T15:46:10.731609+0000","last_peered":"2024-09-16T15:46:10.731609+0000","last_clean":"2024-09-16T15:46:10.731609+0000","last_became_active":"2024-09-16T15:46:09.690546+0000","last_became_peered":"2024-09-16T15:46:09.690546+0000","last_unstale":"2024-09-16T15:46:10.731609+0000","last_undegraded":"2024-09-16T15:46:10.731609+0000","last_fullsized":"2024-09-16T15:46:10.731609+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:53.963469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.724797+0000","last_change":"2024-09-16T15:46:10.720141+0000","last_active":"2024-09-16T15:46:10.724797+0000","last_peered":"2024-09-16T15:46:10.724797+0000","last_clean":"2024-09-16T15:46:10.724797+0000","last_became_active":"2024-09-16T15:46:10.719824+0000","last_became_peered":"2024-09-16T15:46:10.719824+0000","last_unstale":"2024-09-16T15:46:10.724797+0000","last_undegraded":"2024-09-16T15:46:10.724797+0000","last_fullsized":"2024-09-16T15:46:10.724797+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:10.807988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.705329+0000","last_change":"2024-09-16T15:46:09.701525+0000","last_active":"2024-09-16T15:46:10.705329+0000","last_peered":"2024-09-16T15:46:10.705329+0000","last_clean":"2024-09-16T15:46:10.705329+0000","last_became_active":"2024-09-16T15:46:09.701238+0000","last_became_peered":"2024-09-16T15:46:09.701238+0000","last_unstale":"2024-09-16T15:46:10.705329+0000","last_undegraded":"2024-09-16T15:46:10.705329+0000","last_fullsized":"2024-09-16T15:46:10.705329+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:34.991632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.711778+0000","last_change":"2024-09-16T15:46:10.727542+0000","last_active":"2024-09-16T15:46:11.711778+0000","last_peered":"2024-09-16T15:46:11.711778+0000","last_clean":"2024-09-16T15:46:11.711778+0000","last_became_active":"2024-09-16T15:46:10.727266+0000","last_became_peered":"2024-09-16T15:46:10.727266+0000","last_unstale":"2024-09-16T15:46:11.711778+0000","last_undegraded":"2024-09-16T15:46:11.711778+0000","last_fullsized":"2024-09-16T15:46:11.711778+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:14.044027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.704970+0000","last_change":"2024-09-16T15:46:09.703778+0000","last_active":"2024-09-16T15:46:10.704970+0000","last_peered":"2024-09-16T15:46:10.704970+0000","last_clean":"2024-09-16T15:46:10.704970+0000","last_became_active":"2024-09-16T15:46:09.703502+0000","last_became_peered":"2024-09-16T15:46:09.703502+0000","last_unstale":"2024-09-16T15:46:10.704970+0000","last_undegraded":"2024-09-16T15:46:10.704970+0000","last_fullsized":"2024-09-16T15:46:10.704970+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:35:39.456944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726124+0000","last_change":"2024-09-16T15:46:10.722381+0000","last_active":"2024-09-16T15:46:10.726124+0000","last_peered":"2024-09-16T15:46:10.726124+0000","last_clean":"2024-09-16T15:46:10.726124+0000","last_became_active":"2024-09-16T15:46:10.721793+0000","last_became_peered":"2024-09-16T15:46:10.721793+0000","last_unstale":"2024-09-16T15:46:10.726124+0000","last_undegraded":"2024-09-16T15:46:10.726124+0000","last_fullsized":"2024-09-16T15:46:10.726124+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:36:50.006598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712041+0000","last_change":"2024-09-16T15:46:09.697476+0000","last_active":"2024-09-16T15:46:11.712041+0000","last_peered":"2024-09-16T15:46:11.712041+0000","last_clean":"2024-09-16T15:46:11.712041+0000","last_became_active":"2024-09-16T15:46:09.696516+0000","last_became_peered":"2024-09-16T15:46:09.696516+0000","last_unstale":"2024-09-16T15:46:11.712041+0000","last_undegraded":"2024-09-16T15:46:11.712041+0000","last_fullsized":"2024-09-16T15:46:11.712041+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:21.330968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.715647+0000","last_change":"2024-09-16T15:46:10.711849+0000","last_active":"2024-09-16T15:46:10.715647+0000","last_peered":"2024-09-16T15:46:10.715647+0000","last_clean":"2024-09-16T15:46:10.715647+0000","last_became_active":"2024-09-16T15:46:10.711549+0000","last_became_peered":"2024-09-16T15:46:10.711549+0000","last_unstale":"2024-09-16T15:46:10.715647+0000","last_undegraded":"2024-09-16T15:46:10.715647+0000","last_fullsized":"2024-09-16T15:46:10.715647+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:28.805644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"136'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.732579+0000","last_change":"2024-09-16T15:46:09.700666+0000","last_active":"2024-09-16T15:46:10.732579+0000","last_peered":"2024-09-16T15:46:10.732579+0000","last_clean":"2024-09-16T15:46:10.732579+0000","last_became_active":"2024-09-16T15:46:09.700320+0000","last_became_peered":"2024-09-16T15:46:09.700320+0000","last_unstale":"2024-09-16T15:46:10.732579+0000","last_undegraded":"2024-09-16T15:46:10.732579+0000","last_fullsized":"2024-09-16T15:46:10.732579+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:17.714993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713389+0000","last_change":"2024-09-16T15:46:10.706580+0000","last_active":"2024-09-16T15:46:11.713389+0000","last_peered":"2024-09-16T15:46:11.713389+0000","last_clean":"2024-09-16T15:46:11.713389+0000","last_became_active":"2024-09-16T15:46:10.706316+0000","last_became_peered":"2024-09-16T15:46:10.706316+0000","last_unstale":"2024-09-16T15:46:11.713389+0000","last_undegraded":"2024-09-16T15:46:11.713389+0000","last_fullsized":"2024-09-16T15:46:11.713389+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:03.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.709188+0000","last_change":"2024-09-16T15:46:09.701102+0000","last_active":"2024-09-16T15:46:11.709188+0000","last_peered":"2024-09-16T15:46:11.709188+0000","last_clean":"2024-09-16T15:46:11.709188+0000","last_became_active":"2024-09-16T15:46:09.700037+0000","last_became_peered":"2024-09-16T15:46:09.700037+0000","last_unstale":"2024-09-16T15:46:11.709188+0000","last_undegraded":"2024-09-16T15:46:11.709188+0000","last_fullsized":"2024-09-16T15:46:11.709188+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:28:51.346549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.720315+0000","last_change":"2024-09-16T15:46:10.719009+0000","last_active":"2024-09-16T15:46:10.720315+0000","last_peered":"2024-09-16T15:46:10.720315+0000","last_clean":"2024-09-16T15:46:10.720315+0000","last_became_active":"2024-09-16T15:46:10.718604+0000","last_became_peered":"2024-09-16T15:46:10.718604+0000","last_unstale":"2024-09-16T15:46:10.720315+0000","last_undegraded":"2024-09-16T15:46:10.720315+0000","last_fullsized":"2024-09-16T15:46:10.720315+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:02.582741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.709010+0000","last_change":"2024-09-16T15:46:09.704066+0000","last_active":"2024-09-16T15:46:11.709010+0000","last_peered":"2024-09-16T15:46:11.709010+0000","last_clean":"2024-09-16T15:46:11.709010+0000","last_became_active":"2024-09-16T15:46:09.703202+0000","last_became_peered":"2024-09-16T15:46:09.703202+0000","last_unstale":"2024-09-16T15:46:11.709010+0000","last_undegraded":"2024-09-16T15:46:11.709010+0000","last_fullsized":"2024-09-16T15:46:11.709010+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:08.725854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.709044+0000","last_change":"2024-09-16T15:46:10.712371+0000","last_active":"2024-09-16T15:46:11.709044+0000","last_peered":"2024-09-16T15:46:11.709044+0000","last_clean":"2024-09-16T15:46:11.709044+0000","last_became_active":"2024-09-16T15:46:10.712080+0000","last_became_peered":"2024-09-16T15:46:10.712080+0000","last_unstale":"2024-09-16T15:46:11.709044+0000","last_undegraded":"2024-09-16T15:46:11.709044+0000","last_fullsized":"2024-09-16T15:46:11.709044+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:26.379452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708427+0000","last_change":"2024-09-16T15:46:09.704500+0000","last_active":"2024-09-16T15:46:11.708427+0000","last_peered":"2024-09-16T15:46:11.708427+0000","last_clean":"2024-09-16T15:46:11.708427+0000","last_became_active":"2024-09-16T15:46:09.703193+0000","last_became_peered":"2024-09-16T15:46:09.703193+0000","last_unstale":"2024-09-16T15:46:11.708427+0000","last_undegraded":"2024-09-16T15:46:11.708427+0000","last_fullsized":"2024-09-16T15:46:11.708427+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:15.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.725211+0000","last_change":"2024-09-16T15:46:10.720162+0000","last_active":"2024-09-16T15:46:10.725211+0000","last_peered":"2024-09-16T15:46:10.725211+0000","last_clean":"2024-09-16T15:46:10.725211+0000","last_became_active":"2024-09-16T15:46:10.719850+0000","last_became_peered":"2024-09-16T15:46:10.719850+0000","last_unstale":"2024-09-16T15:46:10.725211+0000","last_undegraded":"2024-09-16T15:46:10.725211+0000","last_fullsized":"2024-09-16T15:46:10.725211+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:53:37.848035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.705859+0000","last_change":"2024-09-16T15:46:09.695738+0000","last_active":"2024-09-16T15:46:10.705859+0000","last_peered":"2024-09-16T15:46:10.705859+0000","last_clean":"2024-09-16T15:46:10.705859+0000","last_became_active":"2024-09-16T15:46:09.695423+0000","last_became_peered":"2024-09-16T15:46:09.695423+0000","last_unstale":"2024-09-16T15:46:10.705859+0000","last_undegraded":"2024-09-16T15:46:10.705859+0000","last_fullsized":"2024-09-16T15:46:10.705859+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:18.681451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.711646+0000","last_change":"2024-09-16T15:46:10.705692+0000","last_active":"2024-09-16T15:46:10.711646+0000","last_peered":"2024-09-16T15:46:10.711646+0000","last_clean":"2024-09-16T15:46:10.711646+0000","last_became_active":"2024-09-16T15:46:10.704580+0000","last_became_peered":"2024-09-16T15:46:10.704580+0000","last_unstale":"2024-09-16T15:46:10.711646+0000","last_undegraded":"2024-09-16T15:46:10.711646+0000","last_fullsized":"2024-09-16T15:46:10.711646+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:53.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.705546+0000","last_change":"2024-09-16T15:46:09.699570+0000","last_active":"2024-09-16T15:46:10.705546+0000","last_peered":"2024-09-16T15:46:10.705546+0000","last_clean":"2024-09-16T15:46:10.705546+0000","last_became_active":"2024-09-16T15:46:09.699255+0000","last_became_peered":"2024-09-16T15:46:09.699255+0000","last_unstale":"2024-09-16T15:46:10.705546+0000","last_undegraded":"2024-09-16T15:46:10.705546+0000","last_fullsized":"2024-09-16T15:46:10.705546+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:57.702862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726545+0000","last_change":"2024-09-16T15:46:10.721673+0000","last_active":"2024-09-16T15:46:10.726545+0000","last_peered":"2024-09-16T15:46:10.726545+0000","last_clean":"2024-09-16T15:46:10.726545+0000","last_became_active":"2024-09-16T15:46:10.720273+0000","last_became_peered":"2024-09-16T15:46:10.720273+0000","last_unstale":"2024-09-16T15:46:10.726545+0000","last_undegraded":"2024-09-16T15:46:10.726545+0000","last_fullsized":"2024-09-16T15:46:10.726545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:39.124414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712387+0000","last_change":"2024-09-16T15:46:09.696995+0000","last_active":"2024-09-16T15:46:11.712387+0000","last_peered":"2024-09-16T15:46:11.712387+0000","last_clean":"2024-09-16T15:46:11.712387+0000","last_became_active":"2024-09-16T15:46:09.696731+0000","last_became_peered":"2024-09-16T15:46:09.696731+0000","last_unstale":"2024-09-16T15:46:11.712387+0000","last_undegraded":"2024-09-16T15:46:11.712387+0000","last_fullsized":"2024-09-16T15:46:11.712387+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:03.179077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.719056+0000","last_change":"2024-09-16T15:46:10.717633+0000","last_active":"2024-09-16T15:46:10.719056+0000","last_peered":"2024-09-16T15:46:10.719056+0000","last_clean":"2024-09-16T15:46:10.719056+0000","last_became_active":"2024-09-16T15:46:10.714460+0000","last_became_peered":"2024-09-16T15:46:10.714460+0000","last_unstale":"2024-09-16T15:46:10.719056+0000","last_undegraded":"2024-09-16T15:46:10.719056+0000","last_fullsized":"2024-09-16T15:46:10.719056+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:20.670347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.700898+0000","last_change":"2024-09-16T15:46:09.686278+0000","last_active":"2024-09-16T15:46:10.700898+0000","last_peered":"2024-09-16T15:46:10.700898+0000","last_clean":"2024-09-16T15:46:10.700898+0000","last_became_active":"2024-09-16T15:46:09.685999+0000","last_became_peered":"2024-09-16T15:46:09.685999+0000","last_unstale":"2024-09-16T15:46:10.700898+0000","last_undegraded":"2024-09-16T15:46:10.700898+0000","last_fullsized":"2024-09-16T15:46:10.700898+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:13.038157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726470+0000","last_change":"2024-09-16T15:46:10.722461+0000","last_active":"2024-09-16T15:46:10.726470+0000","last_peered":"2024-09-16T15:46:10.726470+0000","last_clean":"2024-09-16T15:46:10.726470+0000","last_became_active":"2024-09-16T15:46:10.720438+0000","last_became_peered":"2024-09-16T15:46:10.720438+0000","last_unstale":"2024-09-16T15:46:10.726470+0000","last_undegraded":"2024-09-16T15:46:10.726470+0000","last_fullsized":"2024-09-16T15:46:10.726470+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:49.144951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"136'3","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712814+0000","last_change":"2024-09-16T15:46:09.684980+0000","last_active":"2024-09-16T15:46:11.712814+0000","last_peered":"2024-09-16T15:46:11.712814+0000","last_clean":"2024-09-16T15:46:11.712814+0000","last_became_active":"2024-09-16T15:46:09.684324+0000","last_became_peered":"2024-09-16T15:46:09.684324+0000","last_unstale":"2024-09-16T15:46:11.712814+0000","last_undegraded":"2024-09-16T15:46:11.712814+0000","last_fullsized":"2024-09-16T15:46:11.712814+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:48.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":347,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.702735+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:46:10.702735+0000","last_peered":"2024-09-16T15:46:10.702735+0000","last_clean":"2024-09-16T15:46:10.702735+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:46:10.702735+0000","last_undegraded":"2024-09-16T15:46:10.702735+0000","last_fullsized":"2024-09-16T15:46:10.702735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509538,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2297856,"data_stored":1339443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509538,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":748569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":748445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1708032,"data_stored":754751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509537,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37008,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334832,"statfs":{"total":96636764160,"available":96598867968,"internally_reserved":0,"allocated":1699840,"data_stored":748513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509536,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37020,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334820,"statfs":{"total":96636764160,"available":96598855680,"internally_reserved":0,"allocated":1712128,"data_stored":750105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37028,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334812,"statfs":{"total":96636764160,"available":96598847488,"internally_reserved":0,"allocated":1712128,"data_stored":755333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37600,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334240,"statfs":{"total":96636764160,"available":96598261760,"internally_reserved":0,"allocated":2306048,"data_stored":1340437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:46:14.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:46:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.157+0000 7fed06400640 1 -- 172.21.15.60:0/645898420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed0015f5a0 msgr2=0x7fed000b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.157+0000 7fed06400640 1 --2- 172.21.15.60:0/645898420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed0015f5a0 0x7fed000b2440 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fed0006dac0 tx=0x7fecf002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.157+0000 7fed06400640 1 -- 172.21.15.60:0/645898420 shutdown_connections 2024-09-16T15:46:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.157+0000 7fed06400640 1 --2- 172.21.15.60:0/645898420 >> v2:172.21.15.100:3300/0 conn(0x7fed00073fb0 0x7fed00074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.157+0000 7fed06400640 1 --2- 172.21.15.60:0/645898420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed0015f5a0 0x7fed000b2440 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.157+0000 7fed06400640 1 -- 172.21.15.60:0/645898420 >> 172.21.15.60:0/645898420 conn(0x7fed000b1950 msgr2=0x7fed000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.157+0000 7fed06400640 1 -- 172.21.15.60:0/645898420 shutdown_connections 2024-09-16T15:46:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.157+0000 7fed06400640 1 -- 172.21.15.60:0/645898420 wait complete. 2024-09-16T15:46:15.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.159+0000 7fed06400640 1 Processor -- start 2024-09-16T15:46:15.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.159+0000 7fed06400640 1 -- start start 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fed06400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed00073fb0 0x7fed0009acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fed06400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed000ccf00 0x7fed00152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fed06400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed0015f5a0 0x7fed00142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fed06400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fed0006f2e0 con 0x7fed000ccf00 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fed06400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed0006f160 con 0x7fed0015f5a0 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fed06400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fed0006f460 con 0x7fed00073fb0 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecfea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed000ccf00 0x7fed00152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecfea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed000ccf00 0x7fed00152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60392/0 (socket says 172.21.15.60:60392) 2024-09-16T15:46:15.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecfea00640 1 -- 172.21.15.60:0/1989630091 learned_addr learned my addr 172.21.15.60:0/1989630091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecff400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed00073fb0 0x7fed0009acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecffe00640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed0015f5a0 0x7fed00142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecfea00640 1 -- 172.21.15.60:0/1989630091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed00073fb0 msgr2=0x7fed0009acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecfea00640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed00073fb0 0x7fed0009acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecfea00640 1 -- 172.21.15.60:0/1989630091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed0015f5a0 msgr2=0x7fed00142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecfea00640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed0015f5a0 0x7fed00142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.160+0000 7fecfea00640 1 -- 172.21.15.60:0/1989630091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0009c6c0 con 0x7fed000ccf00 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.161+0000 7fecfea00640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed000ccf00 0x7fed00152920 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fecec00c410 tx=0x7fecec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.161+0000 7fecf7400640 1 -- 172.21.15.60:0/1989630091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec017070 con 0x7fed000ccf00 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.161+0000 7fecf7400640 1 -- 172.21.15.60:0/1989630091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecec0035f0 con 0x7fed000ccf00 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.161+0000 7fecf7400640 1 -- 172.21.15.60:0/1989630091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec003a80 con 0x7fed000ccf00 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.161+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0009b620 con 0x7fed000ccf00 2024-09-16T15:46:15.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.161+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0009bbe0 con 0x7fed000ccf00 2024-09-16T15:46:15.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.162+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0006fa10 con 0x7fed000ccf00 2024-09-16T15:46:15.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.163+0000 7fecf7400640 1 -- 172.21.15.60:0/1989630091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fecec01a070 con 0x7fed000ccf00 2024-09-16T15:46:15.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.164+0000 7fecf7400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fece0078e80 0x7fece007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:15.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.164+0000 7fecf7400640 1 -- 172.21.15.60:0/1989630091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fecec09a940 con 0x7fed000ccf00 2024-09-16T15:46:15.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.165+0000 7fecff400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fece0078e80 0x7fece007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:15.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.165+0000 7fecff400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fece0078e80 0x7fece007b340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fecf0033000 tx=0x7fecf0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:15.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.169+0000 7fecf7400640 1 -- 172.21.15.60:0/1989630091 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fecec061eb0 con 0x7fed000ccf00 2024-09-16T15:46:15.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.360+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed00155de0 con 0x7fece0078e80 2024-09-16T15:46:15.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.364+0000 7fecf7400640 1 -- 172.21.15.60:0/1989630091 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7fed00155de0 con 0x7fece0078e80 2024-09-16T15:46:15.369 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:15.370 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fece0078e80 msgr2=0x7fece007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fece0078e80 0x7fece007b340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fecf0033000 tx=0x7fecf0004730 comp rx=0 tx=0).stop 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed000ccf00 msgr2=0x7fed00152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed000ccf00 0x7fed00152920 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fecec00c410 tx=0x7fecec00c8d0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 shutdown_connections 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fece0078e80 0x7fece007b340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed0015f5a0 0x7fed00142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed000ccf00 0x7fed00152920 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 --2- 172.21.15.60:0/1989630091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed00073fb0 0x7fed0009acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 >> 172.21.15.60:0/1989630091 conn(0x7fed000b1950 msgr2=0x7fed000ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:15.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 shutdown_connections 2024-09-16T15:46:15.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.368+0000 7fed06400640 1 -- 172.21.15.60:0/1989630091 wait complete. 2024-09-16T15:46:15.385 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-16T15:46:14.202046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":280656,"kb_used_data":14496,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754694064,"statfs":{"total":773094113280,"available":772806721536,"internally_reserved":0,"allocated":14843904,"data_stored":7185596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.528780"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.709077+0000","last_change":"2024-09-16T15:46:10.713882+0000","last_active":"2024-09-16T15:46:11.709077+0000","last_peered":"2024-09-16T15:46:11.709077+0000","last_clean":"2024-09-16T15:46:11.709077+0000","last_became_active":"2024-09-16T15:46:10.711575+0000","last_became_peered":"2024-09-16T15:46:10.711575+0000","last_unstale":"2024-09-16T15:46:11.709077+0000","last_undegraded":"2024-09-16T15:46:11.709077+0000","last_fullsized":"2024-09-16T15:46:11.709077+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:08:23.227443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.714842+0000","last_change":"2024-09-16T15:46:10.712701+0000","last_active":"2024-09-16T15:46:10.714842+0000","last_peered":"2024-09-16T15:46:10.714842+0000","last_clean":"2024-09-16T15:46:10.714842+0000","last_became_active":"2024-09-16T15:46:10.712364+0000","last_became_peered":"2024-09-16T15:46:10.712364+0000","last_unstale":"2024-09-16T15:46:10.714842+0000","last_undegraded":"2024-09-16T15:46:10.714842+0000","last_fullsized":"2024-09-16T15:46:10.714842+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:03.770333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712651+0000","last_change":"2024-09-16T15:46:10.707416+0000","last_active":"2024-09-16T15:46:11.712651+0000","last_peered":"2024-09-16T15:46:11.712651+0000","last_clean":"2024-09-16T15:46:11.712651+0000","last_became_active":"2024-09-16T15:46:10.704943+0000","last_became_peered":"2024-09-16T15:46:10.704943+0000","last_unstale":"2024-09-16T15:46:11.712651+0000","last_undegraded":"2024-09-16T15:46:11.712651+0000","last_fullsized":"2024-09-16T15:46:11.712651+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:53:44.390327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.714054+0000","last_change":"2024-09-16T15:46:10.714160+0000","last_active":"2024-09-16T15:46:11.714054+0000","last_peered":"2024-09-16T15:46:11.714054+0000","last_clean":"2024-09-16T15:46:11.714054+0000","last_became_active":"2024-09-16T15:46:10.713816+0000","last_became_peered":"2024-09-16T15:46:10.713816+0000","last_unstale":"2024-09-16T15:46:11.714054+0000","last_undegraded":"2024-09-16T15:46:11.714054+0000","last_fullsized":"2024-09-16T15:46:11.714054+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:19.687321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.718570+0000","last_change":"2024-09-16T15:46:10.716024+0000","last_active":"2024-09-16T15:46:10.718570+0000","last_peered":"2024-09-16T15:46:10.718570+0000","last_clean":"2024-09-16T15:46:10.718570+0000","last_became_active":"2024-09-16T15:46:10.712578+0000","last_became_peered":"2024-09-16T15:46:10.712578+0000","last_unstale":"2024-09-16T15:46:10.718570+0000","last_undegraded":"2024-09-16T15:46:10.718570+0000","last_fullsized":"2024-09-16T15:46:10.718570+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:21.801942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708545+0000","last_change":"2024-09-16T15:46:10.704457+0000","last_active":"2024-09-16T15:46:11.708545+0000","last_peered":"2024-09-16T15:46:11.708545+0000","last_clean":"2024-09-16T15:46:11.708545+0000","last_became_active":"2024-09-16T15:46:10.702645+0000","last_became_peered":"2024-09-16T15:46:10.702645+0000","last_unstale":"2024-09-16T15:46:11.708545+0000","last_undegraded":"2024-09-16T15:46:11.708545+0000","last_fullsized":"2024-09-16T15:46:11.708545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:27:57.842385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726148+0000","last_change":"2024-09-16T15:46:10.722407+0000","last_active":"2024-09-16T15:46:10.726148+0000","last_peered":"2024-09-16T15:46:10.726148+0000","last_clean":"2024-09-16T15:46:10.726148+0000","last_became_active":"2024-09-16T15:46:10.722116+0000","last_became_peered":"2024-09-16T15:46:10.722116+0000","last_unstale":"2024-09-16T15:46:10.726148+0000","last_undegraded":"2024-09-16T15:46:10.726148+0000","last_fullsized":"2024-09-16T15:46:10.726148+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:31:14.501049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708325+0000","last_change":"2024-09-16T15:46:10.702501+0000","last_active":"2024-09-16T15:46:11.708325+0000","last_peered":"2024-09-16T15:46:11.708325+0000","last_clean":"2024-09-16T15:46:11.708325+0000","last_became_active":"2024-09-16T15:46:10.702187+0000","last_became_peered":"2024-09-16T15:46:10.702187+0000","last_unstale":"2024-09-16T15:46:11.708325+0000","last_undegraded":"2024-09-16T15:46:11.708325+0000","last_fullsized":"2024-09-16T15:46:11.708325+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:11:52.795656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.719438+0000","last_change":"2024-09-16T15:46:10.717415+0000","last_active":"2024-09-16T15:46:10.719438+0000","last_peered":"2024-09-16T15:46:10.719438+0000","last_clean":"2024-09-16T15:46:10.719438+0000","last_became_active":"2024-09-16T15:46:10.717097+0000","last_became_peered":"2024-09-16T15:46:10.717097+0000","last_unstale":"2024-09-16T15:46:10.719438+0000","last_undegraded":"2024-09-16T15:46:10.719438+0000","last_fullsized":"2024-09-16T15:46:10.719438+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:19.687321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.720105+0000","last_change":"2024-09-16T15:46:10.717874+0000","last_active":"2024-09-16T15:46:10.720105+0000","last_peered":"2024-09-16T15:46:10.720105+0000","last_clean":"2024-09-16T15:46:10.720105+0000","last_became_active":"2024-09-16T15:46:10.717601+0000","last_became_peered":"2024-09-16T15:46:10.717601+0000","last_unstale":"2024-09-16T15:46:10.720105+0000","last_undegraded":"2024-09-16T15:46:10.720105+0000","last_fullsized":"2024-09-16T15:46:10.720105+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:59:47.021811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708669+0000","last_change":"2024-09-16T15:46:10.703289+0000","last_active":"2024-09-16T15:46:11.708669+0000","last_peered":"2024-09-16T15:46:11.708669+0000","last_clean":"2024-09-16T15:46:11.708669+0000","last_became_active":"2024-09-16T15:46:10.703031+0000","last_became_peered":"2024-09-16T15:46:10.703031+0000","last_unstale":"2024-09-16T15:46:11.708669+0000","last_undegraded":"2024-09-16T15:46:11.708669+0000","last_fullsized":"2024-09-16T15:46:11.708669+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:04.489710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.717962+0000","last_change":"2024-09-16T15:46:10.715278+0000","last_active":"2024-09-16T15:46:10.717962+0000","last_peered":"2024-09-16T15:46:10.717962+0000","last_clean":"2024-09-16T15:46:10.717962+0000","last_became_active":"2024-09-16T15:46:10.714653+0000","last_became_peered":"2024-09-16T15:46:10.714653+0000","last_unstale":"2024-09-16T15:46:10.717962+0000","last_undegraded":"2024-09-16T15:46:10.717962+0000","last_fullsized":"2024-09-16T15:46:10.717962+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:00:10.745271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.717328+0000","last_change":"2024-09-16T15:46:10.712832+0000","last_active":"2024-09-16T15:46:10.717328+0000","last_peered":"2024-09-16T15:46:10.717328+0000","last_clean":"2024-09-16T15:46:10.717328+0000","last_became_active":"2024-09-16T15:46:10.712235+0000","last_became_peered":"2024-09-16T15:46:10.712235+0000","last_unstale":"2024-09-16T15:46:10.717328+0000","last_undegraded":"2024-09-16T15:46:10.717328+0000","last_fullsized":"2024-09-16T15:46:10.717328+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:38:10.672795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713065+0000","last_change":"2024-09-16T15:46:10.722169+0000","last_active":"2024-09-16T15:46:11.713065+0000","last_peered":"2024-09-16T15:46:11.713065+0000","last_clean":"2024-09-16T15:46:11.713065+0000","last_became_active":"2024-09-16T15:46:10.721739+0000","last_became_peered":"2024-09-16T15:46:10.721739+0000","last_unstale":"2024-09-16T15:46:11.713065+0000","last_undegraded":"2024-09-16T15:46:11.713065+0000","last_fullsized":"2024-09-16T15:46:11.713065+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:59.315703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726502+0000","last_change":"2024-09-16T15:46:10.721308+0000","last_active":"2024-09-16T15:46:10.726502+0000","last_peered":"2024-09-16T15:46:10.726502+0000","last_clean":"2024-09-16T15:46:10.726502+0000","last_became_active":"2024-09-16T15:46:10.721008+0000","last_became_peered":"2024-09-16T15:46:10.721008+0000","last_unstale":"2024-09-16T15:46:10.726502+0000","last_undegraded":"2024-09-16T15:46:10.726502+0000","last_fullsized":"2024-09-16T15:46:10.726502+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:22.669833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.718350+0000","last_change":"2024-09-16T15:46:10.708949+0000","last_active":"2024-09-16T15:46:10.718350+0000","last_peered":"2024-09-16T15:46:10.718350+0000","last_clean":"2024-09-16T15:46:10.718350+0000","last_became_active":"2024-09-16T15:46:10.708337+0000","last_became_peered":"2024-09-16T15:46:10.708337+0000","last_unstale":"2024-09-16T15:46:10.718350+0000","last_undegraded":"2024-09-16T15:46:10.718350+0000","last_fullsized":"2024-09-16T15:46:10.718350+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:32.248943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.731747+0000","last_change":"2024-09-16T15:46:10.728760+0000","last_active":"2024-09-16T15:46:10.731747+0000","last_peered":"2024-09-16T15:46:10.731747+0000","last_clean":"2024-09-16T15:46:10.731747+0000","last_became_active":"2024-09-16T15:46:10.728487+0000","last_became_peered":"2024-09-16T15:46:10.728487+0000","last_unstale":"2024-09-16T15:46:10.731747+0000","last_undegraded":"2024-09-16T15:46:10.731747+0000","last_fullsized":"2024-09-16T15:46:10.731747+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:09.914742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712505+0000","last_change":"2024-09-16T15:46:10.704032+0000","last_active":"2024-09-16T15:46:11.712505+0000","last_peered":"2024-09-16T15:46:11.712505+0000","last_clean":"2024-09-16T15:46:11.712505+0000","last_became_active":"2024-09-16T15:46:10.703747+0000","last_became_peered":"2024-09-16T15:46:10.703747+0000","last_unstale":"2024-09-16T15:46:11.712505+0000","last_undegraded":"2024-09-16T15:46:11.712505+0000","last_fullsized":"2024-09-16T15:46:11.712505+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:06.606019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708906+0000","last_change":"2024-09-16T15:46:10.721313+0000","last_active":"2024-09-16T15:46:11.708906+0000","last_peered":"2024-09-16T15:46:11.708906+0000","last_clean":"2024-09-16T15:46:11.708906+0000","last_became_active":"2024-09-16T15:46:10.721025+0000","last_became_peered":"2024-09-16T15:46:10.721025+0000","last_unstale":"2024-09-16T15:46:11.708906+0000","last_undegraded":"2024-09-16T15:46:11.708906+0000","last_fullsized":"2024-09-16T15:46:11.708906+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:35.661793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.725133+0000","last_change":"2024-09-16T15:46:10.720101+0000","last_active":"2024-09-16T15:46:10.725133+0000","last_peered":"2024-09-16T15:46:10.725133+0000","last_clean":"2024-09-16T15:46:10.725133+0000","last_became_active":"2024-09-16T15:46:10.717917+0000","last_became_peered":"2024-09-16T15:46:10.717917+0000","last_unstale":"2024-09-16T15:46:10.725133+0000","last_undegraded":"2024-09-16T15:46:10.725133+0000","last_fullsized":"2024-09-16T15:46:10.725133+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:28.209785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726768+0000","last_change":"2024-09-16T15:46:10.721187+0000","last_active":"2024-09-16T15:46:10.726768+0000","last_peered":"2024-09-16T15:46:10.726768+0000","last_clean":"2024-09-16T15:46:10.726768+0000","last_became_active":"2024-09-16T15:46:10.720874+0000","last_became_peered":"2024-09-16T15:46:10.720874+0000","last_unstale":"2024-09-16T15:46:10.726768+0000","last_undegraded":"2024-09-16T15:46:10.726768+0000","last_fullsized":"2024-09-16T15:46:10.726768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:32:30.474038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713319+0000","last_change":"2024-09-16T15:46:10.721493+0000","last_active":"2024-09-16T15:46:11.713319+0000","last_peered":"2024-09-16T15:46:11.713319+0000","last_clean":"2024-09-16T15:46:11.713319+0000","last_became_active":"2024-09-16T15:46:10.720425+0000","last_became_peered":"2024-09-16T15:46:10.720425+0000","last_unstale":"2024-09-16T15:46:11.713319+0000","last_undegraded":"2024-09-16T15:46:11.713319+0000","last_fullsized":"2024-09-16T15:46:11.713319+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:03.770333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712557+0000","last_change":"2024-09-16T15:46:10.727062+0000","last_active":"2024-09-16T15:46:11.712557+0000","last_peered":"2024-09-16T15:46:11.712557+0000","last_clean":"2024-09-16T15:46:11.712557+0000","last_became_active":"2024-09-16T15:46:10.726763+0000","last_became_peered":"2024-09-16T15:46:10.726763+0000","last_unstale":"2024-09-16T15:46:11.712557+0000","last_undegraded":"2024-09-16T15:46:11.712557+0000","last_fullsized":"2024-09-16T15:46:11.712557+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:18.852522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.720071+0000","last_change":"2024-09-16T15:46:10.718029+0000","last_active":"2024-09-16T15:46:10.720071+0000","last_peered":"2024-09-16T15:46:10.720071+0000","last_clean":"2024-09-16T15:46:10.720071+0000","last_became_active":"2024-09-16T15:46:10.717750+0000","last_became_peered":"2024-09-16T15:46:10.717750+0000","last_unstale":"2024-09-16T15:46:10.720071+0000","last_undegraded":"2024-09-16T15:46:10.720071+0000","last_fullsized":"2024-09-16T15:46:10.720071+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:20:35.235501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.725260+0000","last_change":"2024-09-16T15:46:10.718427+0000","last_active":"2024-09-16T15:46:10.725260+0000","last_peered":"2024-09-16T15:46:10.725260+0000","last_clean":"2024-09-16T15:46:10.725260+0000","last_became_active":"2024-09-16T15:46:10.718147+0000","last_became_peered":"2024-09-16T15:46:10.718147+0000","last_unstale":"2024-09-16T15:46:10.725260+0000","last_undegraded":"2024-09-16T15:46:10.725260+0000","last_fullsized":"2024-09-16T15:46:10.725260+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:58:58.163846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708847+0000","last_change":"2024-09-16T15:46:10.727302+0000","last_active":"2024-09-16T15:46:11.708847+0000","last_peered":"2024-09-16T15:46:11.708847+0000","last_clean":"2024-09-16T15:46:11.708847+0000","last_became_active":"2024-09-16T15:46:10.727012+0000","last_became_peered":"2024-09-16T15:46:10.727012+0000","last_unstale":"2024-09-16T15:46:11.708847+0000","last_undegraded":"2024-09-16T15:46:11.708847+0000","last_fullsized":"2024-09-16T15:46:11.708847+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:07.875457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.724408+0000","last_change":"2024-09-16T15:46:10.715286+0000","last_active":"2024-09-16T15:46:10.724408+0000","last_peered":"2024-09-16T15:46:10.724408+0000","last_clean":"2024-09-16T15:46:10.724408+0000","last_became_active":"2024-09-16T15:46:10.713801+0000","last_became_peered":"2024-09-16T15:46:10.713801+0000","last_unstale":"2024-09-16T15:46:10.724408+0000","last_undegraded":"2024-09-16T15:46:10.724408+0000","last_fullsized":"2024-09-16T15:46:10.724408+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:39.025107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.724818+0000","last_change":"2024-09-16T15:46:10.716088+0000","last_active":"2024-09-16T15:46:10.724818+0000","last_peered":"2024-09-16T15:46:10.724818+0000","last_clean":"2024-09-16T15:46:10.724818+0000","last_became_active":"2024-09-16T15:46:10.715794+0000","last_became_peered":"2024-09-16T15:46:10.715794+0000","last_unstale":"2024-09-16T15:46:10.724818+0000","last_undegraded":"2024-09-16T15:46:10.724818+0000","last_fullsized":"2024-09-16T15:46:10.724818+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:02.582741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708376+0000","last_change":"2024-09-16T15:46:10.703838+0000","last_active":"2024-09-16T15:46:11.708376+0000","last_peered":"2024-09-16T15:46:11.708376+0000","last_clean":"2024-09-16T15:46:11.708376+0000","last_became_active":"2024-09-16T15:46:10.702228+0000","last_became_peered":"2024-09-16T15:46:10.702228+0000","last_unstale":"2024-09-16T15:46:11.708376+0000","last_undegraded":"2024-09-16T15:46:11.708376+0000","last_fullsized":"2024-09-16T15:46:11.708376+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:21.750497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.714116+0000","last_change":"2024-09-16T15:46:10.716202+0000","last_active":"2024-09-16T15:46:11.714116+0000","last_peered":"2024-09-16T15:46:11.714116+0000","last_clean":"2024-09-16T15:46:11.714116+0000","last_became_active":"2024-09-16T15:46:10.714205+0000","last_became_peered":"2024-09-16T15:46:10.714205+0000","last_unstale":"2024-09-16T15:46:11.714116+0000","last_undegraded":"2024-09-16T15:46:11.714116+0000","last_fullsized":"2024-09-16T15:46:11.714116+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:22.513670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.719067+0000","last_change":"2024-09-16T15:46:10.717802+0000","last_active":"2024-09-16T15:46:10.719067+0000","last_peered":"2024-09-16T15:46:10.719067+0000","last_clean":"2024-09-16T15:46:10.719067+0000","last_became_active":"2024-09-16T15:46:10.714182+0000","last_became_peered":"2024-09-16T15:46:10.714182+0000","last_unstale":"2024-09-16T15:46:10.719067+0000","last_undegraded":"2024-09-16T15:46:10.719067+0000","last_fullsized":"2024-09-16T15:46:10.719067+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:22.513670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.718378+0000","last_change":"2024-09-16T15:46:10.709178+0000","last_active":"2024-09-16T15:46:10.718378+0000","last_peered":"2024-09-16T15:46:10.718378+0000","last_clean":"2024-09-16T15:46:10.718378+0000","last_became_active":"2024-09-16T15:46:10.708613+0000","last_became_peered":"2024-09-16T15:46:10.708613+0000","last_unstale":"2024-09-16T15:46:10.718378+0000","last_undegraded":"2024-09-16T15:46:10.718378+0000","last_fullsized":"2024-09-16T15:46:10.718378+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:14.210957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.718830+0000","last_change":"2024-09-16T15:46:10.715777+0000","last_active":"2024-09-16T15:46:10.718830+0000","last_peered":"2024-09-16T15:46:10.718830+0000","last_clean":"2024-09-16T15:46:10.718830+0000","last_became_active":"2024-09-16T15:46:10.714528+0000","last_became_peered":"2024-09-16T15:46:10.714528+0000","last_unstale":"2024-09-16T15:46:10.718830+0000","last_undegraded":"2024-09-16T15:46:10.718830+0000","last_fullsized":"2024-09-16T15:46:10.718830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:34:21.839682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"136'4","reported_seq":17,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.733854+0000","last_change":"2024-09-16T15:46:09.701700+0000","last_active":"2024-09-16T15:46:10.733854+0000","last_peered":"2024-09-16T15:46:10.733854+0000","last_clean":"2024-09-16T15:46:10.733854+0000","last_became_active":"2024-09-16T15:46:09.701420+0000","last_became_peered":"2024-09-16T15:46:09.701420+0000","last_unstale":"2024-09-16T15:46:10.733854+0000","last_undegraded":"2024-09-16T15:46:10.733854+0000","last_fullsized":"2024-09-16T15:46:10.733854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:23.313256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.713425+0000","last_change":"2024-09-16T15:46:10.707922+0000","last_active":"2024-09-16T15:46:10.713425+0000","last_peered":"2024-09-16T15:46:10.713425+0000","last_clean":"2024-09-16T15:46:10.713425+0000","last_became_active":"2024-09-16T15:46:10.706230+0000","last_became_peered":"2024-09-16T15:46:10.706230+0000","last_unstale":"2024-09-16T15:46:10.713425+0000","last_undegraded":"2024-09-16T15:46:10.713425+0000","last_fullsized":"2024-09-16T15:46:10.713425+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:18.852522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.733776+0000","last_change":"2024-09-16T15:46:09.688699+0000","last_active":"2024-09-16T15:46:10.733776+0000","last_peered":"2024-09-16T15:46:10.733776+0000","last_clean":"2024-09-16T15:46:10.733776+0000","last_became_active":"2024-09-16T15:46:09.688373+0000","last_became_peered":"2024-09-16T15:46:09.688373+0000","last_unstale":"2024-09-16T15:46:10.733776+0000","last_undegraded":"2024-09-16T15:46:10.733776+0000","last_fullsized":"2024-09-16T15:46:10.733776+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:29.405268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708447+0000","last_change":"2024-09-16T15:46:10.719170+0000","last_active":"2024-09-16T15:46:11.708447+0000","last_peered":"2024-09-16T15:46:11.708447+0000","last_clean":"2024-09-16T15:46:11.708447+0000","last_became_active":"2024-09-16T15:46:10.718809+0000","last_became_peered":"2024-09-16T15:46:10.718809+0000","last_unstale":"2024-09-16T15:46:11.708447+0000","last_undegraded":"2024-09-16T15:46:11.708447+0000","last_fullsized":"2024-09-16T15:46:11.708447+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:58:58.113588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"136'2","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713757+0000","last_change":"2024-09-16T15:46:09.702133+0000","last_active":"2024-09-16T15:46:11.713757+0000","last_peered":"2024-09-16T15:46:11.713757+0000","last_clean":"2024-09-16T15:46:11.713757+0000","last_became_active":"2024-09-16T15:46:09.701794+0000","last_became_peered":"2024-09-16T15:46:09.701794+0000","last_unstale":"2024-09-16T15:46:11.713757+0000","last_undegraded":"2024-09-16T15:46:11.713757+0000","last_fullsized":"2024-09-16T15:46:11.713757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:59.298193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.719689+0000","last_change":"2024-09-16T15:46:10.716550+0000","last_active":"2024-09-16T15:46:10.719689+0000","last_peered":"2024-09-16T15:46:10.719689+0000","last_clean":"2024-09-16T15:46:10.719689+0000","last_became_active":"2024-09-16T15:46:10.716028+0000","last_became_peered":"2024-09-16T15:46:10.716028+0000","last_unstale":"2024-09-16T15:46:10.719689+0000","last_undegraded":"2024-09-16T15:46:10.719689+0000","last_fullsized":"2024-09-16T15:46:10.719689+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:54.253136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"136'2","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.714015+0000","last_change":"2024-09-16T15:46:09.695904+0000","last_active":"2024-09-16T15:46:11.714015+0000","last_peered":"2024-09-16T15:46:11.714015+0000","last_clean":"2024-09-16T15:46:11.714015+0000","last_became_active":"2024-09-16T15:46:09.695505+0000","last_became_peered":"2024-09-16T15:46:09.695505+0000","last_unstale":"2024-09-16T15:46:11.714015+0000","last_undegraded":"2024-09-16T15:46:11.714015+0000","last_fullsized":"2024-09-16T15:46:11.714015+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:47.891123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.716347+0000","last_change":"2024-09-16T15:46:10.715505+0000","last_active":"2024-09-16T15:46:10.716347+0000","last_peered":"2024-09-16T15:46:10.716347+0000","last_clean":"2024-09-16T15:46:10.716347+0000","last_became_active":"2024-09-16T15:46:10.715238+0000","last_became_peered":"2024-09-16T15:46:10.715238+0000","last_unstale":"2024-09-16T15:46:10.716347+0000","last_undegraded":"2024-09-16T15:46:10.716347+0000","last_fullsized":"2024-09-16T15:46:10.716347+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:54.969339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.734657+0000","last_change":"2024-09-16T15:46:09.699289+0000","last_active":"2024-09-16T15:46:10.734657+0000","last_peered":"2024-09-16T15:46:10.734657+0000","last_clean":"2024-09-16T15:46:10.734657+0000","last_became_active":"2024-09-16T15:46:09.698941+0000","last_became_peered":"2024-09-16T15:46:09.698941+0000","last_unstale":"2024-09-16T15:46:10.734657+0000","last_undegraded":"2024-09-16T15:46:10.734657+0000","last_fullsized":"2024-09-16T15:46:10.734657+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:09.763393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713068+0000","last_change":"2024-09-16T15:46:10.714337+0000","last_active":"2024-09-16T15:46:11.713068+0000","last_peered":"2024-09-16T15:46:11.713068+0000","last_clean":"2024-09-16T15:46:11.713068+0000","last_became_active":"2024-09-16T15:46:10.714060+0000","last_became_peered":"2024-09-16T15:46:10.714060+0000","last_unstale":"2024-09-16T15:46:11.713068+0000","last_undegraded":"2024-09-16T15:46:11.713068+0000","last_fullsized":"2024-09-16T15:46:11.713068+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:56.127578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.732184+0000","last_change":"2024-09-16T15:46:09.701485+0000","last_active":"2024-09-16T15:46:10.732184+0000","last_peered":"2024-09-16T15:46:10.732184+0000","last_clean":"2024-09-16T15:46:10.732184+0000","last_became_active":"2024-09-16T15:46:09.700682+0000","last_became_peered":"2024-09-16T15:46:09.700682+0000","last_unstale":"2024-09-16T15:46:10.732184+0000","last_undegraded":"2024-09-16T15:46:10.732184+0000","last_fullsized":"2024-09-16T15:46:10.732184+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:26.201745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.713796+0000","last_change":"2024-09-16T15:46:10.710349+0000","last_active":"2024-09-16T15:46:10.713796+0000","last_peered":"2024-09-16T15:46:10.713796+0000","last_clean":"2024-09-16T15:46:10.713796+0000","last_became_active":"2024-09-16T15:46:10.710076+0000","last_became_peered":"2024-09-16T15:46:10.710076+0000","last_unstale":"2024-09-16T15:46:10.713796+0000","last_undegraded":"2024-09-16T15:46:10.713796+0000","last_fullsized":"2024-09-16T15:46:10.713796+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:59.315703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.706119+0000","last_change":"2024-09-16T15:46:09.700678+0000","last_active":"2024-09-16T15:46:10.706119+0000","last_peered":"2024-09-16T15:46:10.706119+0000","last_clean":"2024-09-16T15:46:10.706119+0000","last_became_active":"2024-09-16T15:46:09.700422+0000","last_became_peered":"2024-09-16T15:46:09.700422+0000","last_unstale":"2024-09-16T15:46:10.706119+0000","last_undegraded":"2024-09-16T15:46:10.706119+0000","last_fullsized":"2024-09-16T15:46:10.706119+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:24:19.962361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.711330+0000","last_change":"2024-09-16T15:46:10.706685+0000","last_active":"2024-09-16T15:46:10.711330+0000","last_peered":"2024-09-16T15:46:10.711330+0000","last_clean":"2024-09-16T15:46:10.711330+0000","last_became_active":"2024-09-16T15:46:10.706428+0000","last_became_peered":"2024-09-16T15:46:10.706428+0000","last_unstale":"2024-09-16T15:46:10.711330+0000","last_undegraded":"2024-09-16T15:46:10.711330+0000","last_fullsized":"2024-09-16T15:46:10.711330+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:03.889160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.711894+0000","last_change":"2024-09-16T15:46:09.696972+0000","last_active":"2024-09-16T15:46:11.711894+0000","last_peered":"2024-09-16T15:46:11.711894+0000","last_clean":"2024-09-16T15:46:11.711894+0000","last_became_active":"2024-09-16T15:46:09.695680+0000","last_became_peered":"2024-09-16T15:46:09.695680+0000","last_unstale":"2024-09-16T15:46:11.711894+0000","last_undegraded":"2024-09-16T15:46:11.711894+0000","last_fullsized":"2024-09-16T15:46:11.711894+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:16.309370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.728155+0000","last_change":"2024-09-16T15:46:10.722977+0000","last_active":"2024-09-16T15:46:10.728155+0000","last_peered":"2024-09-16T15:46:10.728155+0000","last_clean":"2024-09-16T15:46:10.728155+0000","last_became_active":"2024-09-16T15:46:10.722226+0000","last_became_peered":"2024-09-16T15:46:10.722226+0000","last_unstale":"2024-09-16T15:46:10.728155+0000","last_undegraded":"2024-09-16T15:46:10.728155+0000","last_fullsized":"2024-09-16T15:46:10.728155+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:40.841651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.700840+0000","last_change":"2024-09-16T15:46:09.694247+0000","last_active":"2024-09-16T15:46:10.700840+0000","last_peered":"2024-09-16T15:46:10.700840+0000","last_clean":"2024-09-16T15:46:10.700840+0000","last_became_active":"2024-09-16T15:46:09.693989+0000","last_became_peered":"2024-09-16T15:46:09.693989+0000","last_unstale":"2024-09-16T15:46:10.700840+0000","last_undegraded":"2024-09-16T15:46:10.700840+0000","last_fullsized":"2024-09-16T15:46:10.700840+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:10.240327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.732197+0000","last_change":"2024-09-16T15:46:10.728747+0000","last_active":"2024-09-16T15:46:10.732197+0000","last_peered":"2024-09-16T15:46:10.732197+0000","last_clean":"2024-09-16T15:46:10.732197+0000","last_became_active":"2024-09-16T15:46:10.728461+0000","last_became_peered":"2024-09-16T15:46:10.728461+0000","last_unstale":"2024-09-16T15:46:10.732197+0000","last_undegraded":"2024-09-16T15:46:10.732197+0000","last_fullsized":"2024-09-16T15:46:10.732197+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:11.026978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.707631+0000","last_change":"2024-09-16T15:46:09.701879+0000","last_active":"2024-09-16T15:46:10.707631+0000","last_peered":"2024-09-16T15:46:10.707631+0000","last_clean":"2024-09-16T15:46:10.707631+0000","last_became_active":"2024-09-16T15:46:09.701105+0000","last_became_peered":"2024-09-16T15:46:09.701105+0000","last_unstale":"2024-09-16T15:46:10.707631+0000","last_undegraded":"2024-09-16T15:46:10.707631+0000","last_fullsized":"2024-09-16T15:46:10.707631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:18.597100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.724380+0000","last_change":"2024-09-16T15:46:10.713686+0000","last_active":"2024-09-16T15:46:10.724380+0000","last_peered":"2024-09-16T15:46:10.724380+0000","last_clean":"2024-09-16T15:46:10.724380+0000","last_became_active":"2024-09-16T15:46:10.713397+0000","last_became_peered":"2024-09-16T15:46:10.713397+0000","last_unstale":"2024-09-16T15:46:10.724380+0000","last_undegraded":"2024-09-16T15:46:10.724380+0000","last_fullsized":"2024-09-16T15:46:10.724380+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:49.518295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"136'6","reported_seq":19,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.736958+0000","last_change":"2024-09-16T15:46:09.703610+0000","last_active":"2024-09-16T15:46:10.736958+0000","last_peered":"2024-09-16T15:46:10.736958+0000","last_clean":"2024-09-16T15:46:10.736958+0000","last_became_active":"2024-09-16T15:46:09.703304+0000","last_became_peered":"2024-09-16T15:46:09.703304+0000","last_unstale":"2024-09-16T15:46:10.736958+0000","last_undegraded":"2024-09-16T15:46:10.736958+0000","last_fullsized":"2024-09-16T15:46:10.736958+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:21.507800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712503+0000","last_change":"2024-09-16T15:46:10.721207+0000","last_active":"2024-09-16T15:46:11.712503+0000","last_peered":"2024-09-16T15:46:11.712503+0000","last_clean":"2024-09-16T15:46:11.712503+0000","last_became_active":"2024-09-16T15:46:10.720911+0000","last_became_peered":"2024-09-16T15:46:10.720911+0000","last_unstale":"2024-09-16T15:46:11.712503+0000","last_undegraded":"2024-09-16T15:46:11.712503+0000","last_fullsized":"2024-09-16T15:46:11.712503+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:17.089130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"136'2","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713496+0000","last_change":"2024-09-16T15:46:09.691911+0000","last_active":"2024-09-16T15:46:11.713496+0000","last_peered":"2024-09-16T15:46:11.713496+0000","last_clean":"2024-09-16T15:46:11.713496+0000","last_became_active":"2024-09-16T15:46:09.691628+0000","last_became_peered":"2024-09-16T15:46:09.691628+0000","last_unstale":"2024-09-16T15:46:11.713496+0000","last_undegraded":"2024-09-16T15:46:11.713496+0000","last_fullsized":"2024-09-16T15:46:11.713496+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:32.128628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.713874+0000","last_change":"2024-09-16T15:46:10.711816+0000","last_active":"2024-09-16T15:46:10.713874+0000","last_peered":"2024-09-16T15:46:10.713874+0000","last_clean":"2024-09-16T15:46:10.713874+0000","last_became_active":"2024-09-16T15:46:10.711544+0000","last_became_peered":"2024-09-16T15:46:10.711544+0000","last_unstale":"2024-09-16T15:46:10.713874+0000","last_undegraded":"2024-09-16T15:46:10.713874+0000","last_fullsized":"2024-09-16T15:46:10.713874+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:02.796248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712522+0000","last_change":"2024-09-16T15:46:09.700489+0000","last_active":"2024-09-16T15:46:11.712522+0000","last_peered":"2024-09-16T15:46:11.712522+0000","last_clean":"2024-09-16T15:46:11.712522+0000","last_became_active":"2024-09-16T15:46:09.700171+0000","last_became_peered":"2024-09-16T15:46:09.700171+0000","last_unstale":"2024-09-16T15:46:11.712522+0000","last_undegraded":"2024-09-16T15:46:11.712522+0000","last_fullsized":"2024-09-16T15:46:11.712522+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:32.697309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.713570+0000","last_change":"2024-09-16T15:46:10.711414+0000","last_active":"2024-09-16T15:46:10.713570+0000","last_peered":"2024-09-16T15:46:10.713570+0000","last_clean":"2024-09-16T15:46:10.713570+0000","last_became_active":"2024-09-16T15:46:10.709551+0000","last_became_peered":"2024-09-16T15:46:10.709551+0000","last_unstale":"2024-09-16T15:46:10.713570+0000","last_undegraded":"2024-09-16T15:46:10.713570+0000","last_fullsized":"2024-09-16T15:46:10.713570+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:38:37.170762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708537+0000","last_change":"2024-09-16T15:46:09.699133+0000","last_active":"2024-09-16T15:46:11.708537+0000","last_peered":"2024-09-16T15:46:11.708537+0000","last_clean":"2024-09-16T15:46:11.708537+0000","last_became_active":"2024-09-16T15:46:09.698839+0000","last_became_peered":"2024-09-16T15:46:09.698839+0000","last_unstale":"2024-09-16T15:46:11.708537+0000","last_undegraded":"2024-09-16T15:46:11.708537+0000","last_fullsized":"2024-09-16T15:46:11.708537+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:54.846393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712262+0000","last_change":"2024-09-16T15:46:10.717064+0000","last_active":"2024-09-16T15:46:11.712262+0000","last_peered":"2024-09-16T15:46:11.712262+0000","last_clean":"2024-09-16T15:46:11.712262+0000","last_became_active":"2024-09-16T15:46:10.716761+0000","last_became_peered":"2024-09-16T15:46:10.716761+0000","last_unstale":"2024-09-16T15:46:11.712262+0000","last_undegraded":"2024-09-16T15:46:11.712262+0000","last_fullsized":"2024-09-16T15:46:11.712262+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:50.794588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"136'8","reported_seq":21,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.735592+0000","last_change":"2024-09-16T15:46:09.701819+0000","last_active":"2024-09-16T15:46:10.735592+0000","last_peered":"2024-09-16T15:46:10.735592+0000","last_clean":"2024-09-16T15:46:10.735592+0000","last_became_active":"2024-09-16T15:46:09.701557+0000","last_became_peered":"2024-09-16T15:46:09.701557+0000","last_unstale":"2024-09-16T15:46:10.735592+0000","last_undegraded":"2024-09-16T15:46:10.735592+0000","last_fullsized":"2024-09-16T15:46:10.735592+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:49.066249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.717157+0000","last_change":"2024-09-16T15:46:10.714133+0000","last_active":"2024-09-16T15:46:10.717157+0000","last_peered":"2024-09-16T15:46:10.717157+0000","last_clean":"2024-09-16T15:46:10.717157+0000","last_became_active":"2024-09-16T15:46:10.713739+0000","last_became_peered":"2024-09-16T15:46:10.713739+0000","last_unstale":"2024-09-16T15:46:10.717157+0000","last_undegraded":"2024-09-16T15:46:10.717157+0000","last_fullsized":"2024-09-16T15:46:10.717157+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:04.300237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.732697+0000","last_change":"2024-09-16T15:46:09.698797+0000","last_active":"2024-09-16T15:46:10.732697+0000","last_peered":"2024-09-16T15:46:10.732697+0000","last_clean":"2024-09-16T15:46:10.732697+0000","last_became_active":"2024-09-16T15:46:09.697903+0000","last_became_peered":"2024-09-16T15:46:09.697903+0000","last_unstale":"2024-09-16T15:46:10.732697+0000","last_undegraded":"2024-09-16T15:46:10.732697+0000","last_fullsized":"2024-09-16T15:46:10.732697+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:43.595929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.725237+0000","last_change":"2024-09-16T15:46:10.723077+0000","last_active":"2024-09-16T15:46:10.725237+0000","last_peered":"2024-09-16T15:46:10.725237+0000","last_clean":"2024-09-16T15:46:10.725237+0000","last_became_active":"2024-09-16T15:46:10.722833+0000","last_became_peered":"2024-09-16T15:46:10.722833+0000","last_unstale":"2024-09-16T15:46:10.725237+0000","last_undegraded":"2024-09-16T15:46:10.725237+0000","last_fullsized":"2024-09-16T15:46:10.725237+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:40:47.747060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"136'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.733636+0000","last_change":"2024-09-16T15:46:09.688516+0000","last_active":"2024-09-16T15:46:10.733636+0000","last_peered":"2024-09-16T15:46:10.733636+0000","last_clean":"2024-09-16T15:46:10.733636+0000","last_became_active":"2024-09-16T15:46:09.687004+0000","last_became_peered":"2024-09-16T15:46:09.687004+0000","last_unstale":"2024-09-16T15:46:10.733636+0000","last_undegraded":"2024-09-16T15:46:10.733636+0000","last_fullsized":"2024-09-16T15:46:10.733636+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:13.774780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713204+0000","last_change":"2024-09-16T15:46:10.711204+0000","last_active":"2024-09-16T15:46:11.713204+0000","last_peered":"2024-09-16T15:46:11.713204+0000","last_clean":"2024-09-16T15:46:11.713204+0000","last_became_active":"2024-09-16T15:46:10.710653+0000","last_became_peered":"2024-09-16T15:46:10.710653+0000","last_unstale":"2024-09-16T15:46:11.713204+0000","last_undegraded":"2024-09-16T15:46:11.713204+0000","last_fullsized":"2024-09-16T15:46:11.713204+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:24.896188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"136'1","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.734817+0000","last_change":"2024-09-16T15:46:09.699732+0000","last_active":"2024-09-16T15:46:10.734817+0000","last_peered":"2024-09-16T15:46:10.734817+0000","last_clean":"2024-09-16T15:46:10.734817+0000","last_became_active":"2024-09-16T15:46:09.698125+0000","last_became_peered":"2024-09-16T15:46:09.698125+0000","last_unstale":"2024-09-16T15:46:10.734817+0000","last_undegraded":"2024-09-16T15:46:10.734817+0000","last_fullsized":"2024-09-16T15:46:10.734817+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:03:33.769250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708690+0000","last_change":"2024-09-16T15:46:10.717112+0000","last_active":"2024-09-16T15:46:11.708690+0000","last_peered":"2024-09-16T15:46:11.708690+0000","last_clean":"2024-09-16T15:46:11.708690+0000","last_became_active":"2024-09-16T15:46:10.716833+0000","last_became_peered":"2024-09-16T15:46:10.716833+0000","last_unstale":"2024-09-16T15:46:11.708690+0000","last_undegraded":"2024-09-16T15:46:11.708690+0000","last_fullsized":"2024-09-16T15:46:11.708690+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:43.792559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.700375+0000","last_change":"2024-09-16T15:46:09.697393+0000","last_active":"2024-09-16T15:46:10.700375+0000","last_peered":"2024-09-16T15:46:10.700375+0000","last_clean":"2024-09-16T15:46:10.700375+0000","last_became_active":"2024-09-16T15:46:09.697109+0000","last_became_peered":"2024-09-16T15:46:09.697109+0000","last_unstale":"2024-09-16T15:46:10.700375+0000","last_undegraded":"2024-09-16T15:46:10.700375+0000","last_fullsized":"2024-09-16T15:46:10.700375+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:16.005248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713566+0000","last_change":"2024-09-16T15:46:10.716674+0000","last_active":"2024-09-16T15:46:11.713566+0000","last_peered":"2024-09-16T15:46:11.713566+0000","last_clean":"2024-09-16T15:46:11.713566+0000","last_became_active":"2024-09-16T15:46:10.713806+0000","last_became_peered":"2024-09-16T15:46:10.713806+0000","last_unstale":"2024-09-16T15:46:11.713566+0000","last_undegraded":"2024-09-16T15:46:11.713566+0000","last_fullsized":"2024-09-16T15:46:11.713566+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:54.969339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"136'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.731609+0000","last_change":"2024-09-16T15:46:09.690842+0000","last_active":"2024-09-16T15:46:10.731609+0000","last_peered":"2024-09-16T15:46:10.731609+0000","last_clean":"2024-09-16T15:46:10.731609+0000","last_became_active":"2024-09-16T15:46:09.690546+0000","last_became_peered":"2024-09-16T15:46:09.690546+0000","last_unstale":"2024-09-16T15:46:10.731609+0000","last_undegraded":"2024-09-16T15:46:10.731609+0000","last_fullsized":"2024-09-16T15:46:10.731609+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:53.963469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.724797+0000","last_change":"2024-09-16T15:46:10.720141+0000","last_active":"2024-09-16T15:46:10.724797+0000","last_peered":"2024-09-16T15:46:10.724797+0000","last_clean":"2024-09-16T15:46:10.724797+0000","last_became_active":"2024-09-16T15:46:10.719824+0000","last_became_peered":"2024-09-16T15:46:10.719824+0000","last_unstale":"2024-09-16T15:46:10.724797+0000","last_undegraded":"2024-09-16T15:46:10.724797+0000","last_fullsized":"2024-09-16T15:46:10.724797+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:10.807988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.705329+0000","last_change":"2024-09-16T15:46:09.701525+0000","last_active":"2024-09-16T15:46:10.705329+0000","last_peered":"2024-09-16T15:46:10.705329+0000","last_clean":"2024-09-16T15:46:10.705329+0000","last_became_active":"2024-09-16T15:46:09.701238+0000","last_became_peered":"2024-09-16T15:46:09.701238+0000","last_unstale":"2024-09-16T15:46:10.705329+0000","last_undegraded":"2024-09-16T15:46:10.705329+0000","last_fullsized":"2024-09-16T15:46:10.705329+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:34.991632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.711778+0000","last_change":"2024-09-16T15:46:10.727542+0000","last_active":"2024-09-16T15:46:11.711778+0000","last_peered":"2024-09-16T15:46:11.711778+0000","last_clean":"2024-09-16T15:46:11.711778+0000","last_became_active":"2024-09-16T15:46:10.727266+0000","last_became_peered":"2024-09-16T15:46:10.727266+0000","last_unstale":"2024-09-16T15:46:11.711778+0000","last_undegraded":"2024-09-16T15:46:11.711778+0000","last_fullsized":"2024-09-16T15:46:11.711778+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:14.044027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.704970+0000","last_change":"2024-09-16T15:46:09.703778+0000","last_active":"2024-09-16T15:46:10.704970+0000","last_peered":"2024-09-16T15:46:10.704970+0000","last_clean":"2024-09-16T15:46:10.704970+0000","last_became_active":"2024-09-16T15:46:09.703502+0000","last_became_peered":"2024-09-16T15:46:09.703502+0000","last_unstale":"2024-09-16T15:46:10.704970+0000","last_undegraded":"2024-09-16T15:46:10.704970+0000","last_fullsized":"2024-09-16T15:46:10.704970+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:35:39.456944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726124+0000","last_change":"2024-09-16T15:46:10.722381+0000","last_active":"2024-09-16T15:46:10.726124+0000","last_peered":"2024-09-16T15:46:10.726124+0000","last_clean":"2024-09-16T15:46:10.726124+0000","last_became_active":"2024-09-16T15:46:10.721793+0000","last_became_peered":"2024-09-16T15:46:10.721793+0000","last_unstale":"2024-09-16T15:46:10.726124+0000","last_undegraded":"2024-09-16T15:46:10.726124+0000","last_fullsized":"2024-09-16T15:46:10.726124+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:36:50.006598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712041+0000","last_change":"2024-09-16T15:46:09.697476+0000","last_active":"2024-09-16T15:46:11.712041+0000","last_peered":"2024-09-16T15:46:11.712041+0000","last_clean":"2024-09-16T15:46:11.712041+0000","last_became_active":"2024-09-16T15:46:09.696516+0000","last_became_peered":"2024-09-16T15:46:09.696516+0000","last_unstale":"2024-09-16T15:46:11.712041+0000","last_undegraded":"2024-09-16T15:46:11.712041+0000","last_fullsized":"2024-09-16T15:46:11.712041+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:23:21.330968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.715647+0000","last_change":"2024-09-16T15:46:10.711849+0000","last_active":"2024-09-16T15:46:10.715647+0000","last_peered":"2024-09-16T15:46:10.715647+0000","last_clean":"2024-09-16T15:46:10.715647+0000","last_became_active":"2024-09-16T15:46:10.711549+0000","last_became_peered":"2024-09-16T15:46:10.711549+0000","last_unstale":"2024-09-16T15:46:10.715647+0000","last_undegraded":"2024-09-16T15:46:10.715647+0000","last_fullsized":"2024-09-16T15:46:10.715647+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:28.805644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"136'2","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.732579+0000","last_change":"2024-09-16T15:46:09.700666+0000","last_active":"2024-09-16T15:46:10.732579+0000","last_peered":"2024-09-16T15:46:10.732579+0000","last_clean":"2024-09-16T15:46:10.732579+0000","last_became_active":"2024-09-16T15:46:09.700320+0000","last_became_peered":"2024-09-16T15:46:09.700320+0000","last_unstale":"2024-09-16T15:46:10.732579+0000","last_undegraded":"2024-09-16T15:46:10.732579+0000","last_fullsized":"2024-09-16T15:46:10.732579+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:17.714993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.713389+0000","last_change":"2024-09-16T15:46:10.706580+0000","last_active":"2024-09-16T15:46:11.713389+0000","last_peered":"2024-09-16T15:46:11.713389+0000","last_clean":"2024-09-16T15:46:11.713389+0000","last_became_active":"2024-09-16T15:46:10.706316+0000","last_became_peered":"2024-09-16T15:46:10.706316+0000","last_unstale":"2024-09-16T15:46:11.713389+0000","last_undegraded":"2024-09-16T15:46:11.713389+0000","last_fullsized":"2024-09-16T15:46:11.713389+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:03.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.709188+0000","last_change":"2024-09-16T15:46:09.701102+0000","last_active":"2024-09-16T15:46:11.709188+0000","last_peered":"2024-09-16T15:46:11.709188+0000","last_clean":"2024-09-16T15:46:11.709188+0000","last_became_active":"2024-09-16T15:46:09.700037+0000","last_became_peered":"2024-09-16T15:46:09.700037+0000","last_unstale":"2024-09-16T15:46:11.709188+0000","last_undegraded":"2024-09-16T15:46:11.709188+0000","last_fullsized":"2024-09-16T15:46:11.709188+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:28:51.346549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.720315+0000","last_change":"2024-09-16T15:46:10.719009+0000","last_active":"2024-09-16T15:46:10.720315+0000","last_peered":"2024-09-16T15:46:10.720315+0000","last_clean":"2024-09-16T15:46:10.720315+0000","last_became_active":"2024-09-16T15:46:10.718604+0000","last_became_peered":"2024-09-16T15:46:10.718604+0000","last_unstale":"2024-09-16T15:46:10.720315+0000","last_undegraded":"2024-09-16T15:46:10.720315+0000","last_fullsized":"2024-09-16T15:46:10.720315+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:02.582741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.709010+0000","last_change":"2024-09-16T15:46:09.704066+0000","last_active":"2024-09-16T15:46:11.709010+0000","last_peered":"2024-09-16T15:46:11.709010+0000","last_clean":"2024-09-16T15:46:11.709010+0000","last_became_active":"2024-09-16T15:46:09.703202+0000","last_became_peered":"2024-09-16T15:46:09.703202+0000","last_unstale":"2024-09-16T15:46:11.709010+0000","last_undegraded":"2024-09-16T15:46:11.709010+0000","last_fullsized":"2024-09-16T15:46:11.709010+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:08.725854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.709044+0000","last_change":"2024-09-16T15:46:10.712371+0000","last_active":"2024-09-16T15:46:11.709044+0000","last_peered":"2024-09-16T15:46:11.709044+0000","last_clean":"2024-09-16T15:46:11.709044+0000","last_became_active":"2024-09-16T15:46:10.712080+0000","last_became_peered":"2024-09-16T15:46:10.712080+0000","last_unstale":"2024-09-16T15:46:11.709044+0000","last_undegraded":"2024-09-16T15:46:11.709044+0000","last_fullsized":"2024-09-16T15:46:11.709044+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:26.379452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.708427+0000","last_change":"2024-09-16T15:46:09.704500+0000","last_active":"2024-09-16T15:46:11.708427+0000","last_peered":"2024-09-16T15:46:11.708427+0000","last_clean":"2024-09-16T15:46:11.708427+0000","last_became_active":"2024-09-16T15:46:09.703193+0000","last_became_peered":"2024-09-16T15:46:09.703193+0000","last_unstale":"2024-09-16T15:46:11.708427+0000","last_undegraded":"2024-09-16T15:46:11.708427+0000","last_fullsized":"2024-09-16T15:46:11.708427+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:15.205017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.725211+0000","last_change":"2024-09-16T15:46:10.720162+0000","last_active":"2024-09-16T15:46:10.725211+0000","last_peered":"2024-09-16T15:46:10.725211+0000","last_clean":"2024-09-16T15:46:10.725211+0000","last_became_active":"2024-09-16T15:46:10.719850+0000","last_became_peered":"2024-09-16T15:46:10.719850+0000","last_unstale":"2024-09-16T15:46:10.725211+0000","last_undegraded":"2024-09-16T15:46:10.725211+0000","last_fullsized":"2024-09-16T15:46:10.725211+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:53:37.848035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.705859+0000","last_change":"2024-09-16T15:46:09.695738+0000","last_active":"2024-09-16T15:46:10.705859+0000","last_peered":"2024-09-16T15:46:10.705859+0000","last_clean":"2024-09-16T15:46:10.705859+0000","last_became_active":"2024-09-16T15:46:09.695423+0000","last_became_peered":"2024-09-16T15:46:09.695423+0000","last_unstale":"2024-09-16T15:46:10.705859+0000","last_undegraded":"2024-09-16T15:46:10.705859+0000","last_fullsized":"2024-09-16T15:46:10.705859+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:18.681451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.711646+0000","last_change":"2024-09-16T15:46:10.705692+0000","last_active":"2024-09-16T15:46:10.711646+0000","last_peered":"2024-09-16T15:46:10.711646+0000","last_clean":"2024-09-16T15:46:10.711646+0000","last_became_active":"2024-09-16T15:46:10.704580+0000","last_became_peered":"2024-09-16T15:46:10.704580+0000","last_unstale":"2024-09-16T15:46:10.711646+0000","last_undegraded":"2024-09-16T15:46:10.711646+0000","last_fullsized":"2024-09-16T15:46:10.711646+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:53.656402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.705546+0000","last_change":"2024-09-16T15:46:09.699570+0000","last_active":"2024-09-16T15:46:10.705546+0000","last_peered":"2024-09-16T15:46:10.705546+0000","last_clean":"2024-09-16T15:46:10.705546+0000","last_became_active":"2024-09-16T15:46:09.699255+0000","last_became_peered":"2024-09-16T15:46:09.699255+0000","last_unstale":"2024-09-16T15:46:10.705546+0000","last_undegraded":"2024-09-16T15:46:10.705546+0000","last_fullsized":"2024-09-16T15:46:10.705546+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:57.702862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726545+0000","last_change":"2024-09-16T15:46:10.721673+0000","last_active":"2024-09-16T15:46:10.726545+0000","last_peered":"2024-09-16T15:46:10.726545+0000","last_clean":"2024-09-16T15:46:10.726545+0000","last_became_active":"2024-09-16T15:46:10.720273+0000","last_became_peered":"2024-09-16T15:46:10.720273+0000","last_unstale":"2024-09-16T15:46:10.726545+0000","last_undegraded":"2024-09-16T15:46:10.726545+0000","last_fullsized":"2024-09-16T15:46:10.726545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:39.124414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712387+0000","last_change":"2024-09-16T15:46:09.696995+0000","last_active":"2024-09-16T15:46:11.712387+0000","last_peered":"2024-09-16T15:46:11.712387+0000","last_clean":"2024-09-16T15:46:11.712387+0000","last_became_active":"2024-09-16T15:46:09.696731+0000","last_became_peered":"2024-09-16T15:46:09.696731+0000","last_unstale":"2024-09-16T15:46:11.712387+0000","last_undegraded":"2024-09-16T15:46:11.712387+0000","last_fullsized":"2024-09-16T15:46:11.712387+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:03.179077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.719056+0000","last_change":"2024-09-16T15:46:10.717633+0000","last_active":"2024-09-16T15:46:10.719056+0000","last_peered":"2024-09-16T15:46:10.719056+0000","last_clean":"2024-09-16T15:46:10.719056+0000","last_became_active":"2024-09-16T15:46:10.714460+0000","last_became_peered":"2024-09-16T15:46:10.714460+0000","last_unstale":"2024-09-16T15:46:10.719056+0000","last_undegraded":"2024-09-16T15:46:10.719056+0000","last_fullsized":"2024-09-16T15:46:10.719056+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:36:20.670347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.700898+0000","last_change":"2024-09-16T15:46:09.686278+0000","last_active":"2024-09-16T15:46:10.700898+0000","last_peered":"2024-09-16T15:46:10.700898+0000","last_clean":"2024-09-16T15:46:10.700898+0000","last_became_active":"2024-09-16T15:46:09.685999+0000","last_became_peered":"2024-09-16T15:46:09.685999+0000","last_unstale":"2024-09-16T15:46:10.700898+0000","last_undegraded":"2024-09-16T15:46:10.700898+0000","last_fullsized":"2024-09-16T15:46:10.700898+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:13.038157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.726470+0000","last_change":"2024-09-16T15:46:10.722461+0000","last_active":"2024-09-16T15:46:10.726470+0000","last_peered":"2024-09-16T15:46:10.726470+0000","last_clean":"2024-09-16T15:46:10.726470+0000","last_became_active":"2024-09-16T15:46:10.720438+0000","last_became_peered":"2024-09-16T15:46:10.720438+0000","last_unstale":"2024-09-16T15:46:10.726470+0000","last_undegraded":"2024-09-16T15:46:10.726470+0000","last_fullsized":"2024-09-16T15:46:10.726470+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:09.672465+0000","last_clean_scrub_stamp":"2024-09-16T15:46:09.672465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:49.144951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"136'3","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-16T15:46:11.712814+0000","last_change":"2024-09-16T15:46:09.684980+0000","last_active":"2024-09-16T15:46:11.712814+0000","last_peered":"2024-09-16T15:46:11.712814+0000","last_clean":"2024-09-16T15:46:11.712814+0000","last_became_active":"2024-09-16T15:46:09.684324+0000","last_became_peered":"2024-09-16T15:46:09.684324+0000","last_unstale":"2024-09-16T15:46:11.712814+0000","last_undegraded":"2024-09-16T15:46:11.712814+0000","last_fullsized":"2024-09-16T15:46:11.712814+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:08.666595+0000","last_clean_scrub_stamp":"2024-09-16T15:46:08.666595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:48.619634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":347,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T15:46:10.702735+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:46:10.702735+0000","last_peered":"2024-09-16T15:46:10.702735+0000","last_clean":"2024-09-16T15:46:10.702735+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:46:10.702735+0000","last_undegraded":"2024-09-16T15:46:10.702735+0000","last_fullsized":"2024-09-16T15:46:10.702735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509538,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2297856,"data_stored":1339443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509538,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":748569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1699840,"data_stored":748445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1708032,"data_stored":754751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509537,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37008,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334832,"statfs":{"total":96636764160,"available":96598867968,"internally_reserved":0,"allocated":1699840,"data_stored":748513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509536,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37020,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334820,"statfs":{"total":96636764160,"available":96598855680,"internally_reserved":0,"allocated":1712128,"data_stored":750105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37028,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334812,"statfs":{"total":96636764160,"available":96598847488,"internally_reserved":0,"allocated":1712128,"data_stored":755333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37600,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334240,"statfs":{"total":96636764160,"available":96598261760,"internally_reserved":0,"allocated":2306048,"data_stored":1340437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:46:15.389 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:46:15.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:46:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.592+0000 7fcfbe200640 1 -- 172.21.15.60:0/2849910834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb80cccd0 msgr2=0x7fcfb80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.592+0000 7fcfbe200640 1 --2- 172.21.15.60:0/2849910834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb80cccd0 0x7fcfb80cdbb0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0005fd0 tx=0x7fcfb002a820 comp rx=0 tx=0).stop 2024-09-16T15:46:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.592+0000 7fcfbe200640 1 -- 172.21.15.60:0/2849910834 shutdown_connections 2024-09-16T15:46:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.592+0000 7fcfbe200640 1 --2- 172.21.15.60:0/2849910834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb80cccd0 0x7fcfb80cdbb0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.592+0000 7fcfbe200640 1 --2- 172.21.15.60:0/2849910834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfb8145750 0x7fcfb8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.592+0000 7fcfbe200640 1 -- 172.21.15.60:0/2849910834 >> 172.21.15.60:0/2849910834 conn(0x7fcfb80b1840 msgr2=0x7fcfb80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.592+0000 7fcfbe200640 1 -- 172.21.15.60:0/2849910834 shutdown_connections 2024-09-16T15:46:15.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.592+0000 7fcfbe200640 1 -- 172.21.15.60:0/2849910834 wait complete. 2024-09-16T15:46:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.593+0000 7fcfbe200640 1 Processor -- start 2024-09-16T15:46:15.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.594+0000 7fcfbe200640 1 -- start start 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.594+0000 7fcfbe200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb806efa0 0x7fcfb813ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfbe200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfb80cccd0 0x7fcfb813e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfbe200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb8145750 0x7fcfb81f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfbe200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb80ce480 con 0x7fcfb806efa0 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfbe200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb80ce300 con 0x7fcfb8145750 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfbe200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb80ce600 con 0x7fcfb80cccd0 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb806efa0 0x7fcfb813ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb806efa0 0x7fcfb813ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60406/0 (socket says 172.21.15.60:60406) 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 -- 172.21.15.60:0/3045247619 learned_addr learned my addr 172.21.15.60:0/3045247619 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 -- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfb80cccd0 msgr2=0x7fcfb813e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb6a00640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfb80cccd0 0x7fcfb813e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfb80cccd0 0x7fcfb813e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7e00640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb8145750 0x7fcfb81f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 -- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb8145750 msgr2=0x7fcfb81f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb8145750 0x7fcfb81f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 -- 172.21.15.60:0/3045247619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb813f510 con 0x7fcfb806efa0 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb6a00640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfb80cccd0 0x7fcfb813e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7e00640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb8145750 0x7fcfb81f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.595+0000 7fcfb7400640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb806efa0 0x7fcfb813ed70 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800d740 tx=0x7fcfa800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.596+0000 7fcfaf400640 1 -- 172.21.15.60:0/3045247619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8017070 con 0x7fcfb806efa0 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.596+0000 7fcfaf400640 1 -- 172.21.15.60:0/3045247619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfa8009d20 con 0x7fcfb806efa0 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.596+0000 7fcfaf400640 1 -- 172.21.15.60:0/3045247619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80106c0 con 0x7fcfb806efa0 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.596+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb813f7f0 con 0x7fcfb806efa0 2024-09-16T15:46:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.596+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb813fdb0 con 0x7fcfb806efa0 2024-09-16T15:46:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.597+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb80cdbb0 con 0x7fcfb806efa0 2024-09-16T15:46:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.598+0000 7fcfaf400640 1 -- 172.21.15.60:0/3045247619 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fcfa801a070 con 0x7fcfb806efa0 2024-09-16T15:46:15.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.599+0000 7fcfaf400640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf98078e80 0x7fcf9807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:15.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.599+0000 7fcfaf400640 1 -- 172.21.15.60:0/3045247619 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fcfa809b680 con 0x7fcfb806efa0 2024-09-16T15:46:15.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.599+0000 7fcfb6a00640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf98078e80 0x7fcf9807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:15.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.600+0000 7fcfb6a00640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf98078e80 0x7fcf9807b340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4007fd0 tx=0x7fcfa4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:15.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.604+0000 7fcfaf400640 1 -- 172.21.15.60:0/3045247619 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcfa8063390 con 0x7fcfb806efa0 2024-09-16T15:46:15.650 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-09-16T15:46:15.648+0000 7fe8ebe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:46:15.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.872+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fcfb806ed80 con 0x7fcfb806efa0 2024-09-16T15:46:15.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.875+0000 7fcfaf400640 1 -- 172.21.15.60:0/3045247619 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7fcfa8062c90 con 0x7fcfb806efa0 2024-09-16T15:46:15.877 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:46:15.877 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:46:15.878 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:46:15.878 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:46:15.878 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:46:15.878 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:46:15.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf98078e80 msgr2=0x7fcf9807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf98078e80 0x7fcf9807b340 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4007fd0 tx=0x7fcfa4009040 comp rx=0 tx=0).stop 2024-09-16T15:46:15.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb806efa0 msgr2=0x7fcfb813ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:15.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb806efa0 0x7fcfb813ed70 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800d740 tx=0x7fcfa800dc00 comp rx=0 tx=0).stop 2024-09-16T15:46:15.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 shutdown_connections 2024-09-16T15:46:15.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcf98078e80 0x7fcf9807b340 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfb8145750 0x7fcfb81f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcfb80cccd0 0x7fcfb813e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 --2- 172.21.15.60:0/3045247619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcfb806efa0 0x7fcfb813ed70 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:15.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 >> 172.21.15.60:0/3045247619 conn(0x7fcfb80b1840 msgr2=0x7fcfb80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:15.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 shutdown_connections 2024-09-16T15:46:15.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:15.878+0000 7fcfbe200640 1 -- 172.21.15.60:0/3045247619 wait complete. 2024-09-16T15:46:15.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.053+0000 7fbba1000640 1 -- 172.21.15.60:0/1350884975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 msgr2=0x7fbb9c074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.053+0000 7fbba1000640 1 --2- 172.21.15.60:0/1350884975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 0x7fbb9c074420 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c075130 tx=0x7fbb9402a980 comp rx=0 tx=0).stop 2024-09-16T15:46:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.053+0000 7fbba1000640 1 -- 172.21.15.60:0/1350884975 shutdown_connections 2024-09-16T15:46:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.053+0000 7fbba1000640 1 --2- 172.21.15.60:0/1350884975 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 0x7fbb9c074420 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.053+0000 7fbba1000640 1 --2- 172.21.15.60:0/1350884975 >> v2:172.21.15.100:3300/0 conn(0x7fbb9c15f5a0 0x7fbb9c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.053+0000 7fbba1000640 1 -- 172.21.15.60:0/1350884975 >> 172.21.15.60:0/1350884975 conn(0x7fbb9c0b1950 msgr2=0x7fbb9c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.053+0000 7fbba1000640 1 -- 172.21.15.60:0/1350884975 shutdown_connections 2024-09-16T15:46:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.053+0000 7fbba1000640 1 -- 172.21.15.60:0/1350884975 wait complete. 2024-09-16T15:46:16.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.055+0000 7fbba1000640 1 Processor -- start 2024-09-16T15:46:16.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.055+0000 7fbba1000640 1 -- start start 2024-09-16T15:46:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbba1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 0x7fbb9c09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbba1000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0ccf00 0x7fbb9c1f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbba1000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15f5a0 0x7fbb9c1f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbba1000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c06f2e0 con 0x7fbb9c073fb0 2024-09-16T15:46:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbba1000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c06f160 con 0x7fbb9c0ccf00 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbba1000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9c06f460 con 0x7fbb9c15f5a0 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbb99e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 0x7fbb9c09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbb99e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 0x7fbb9c09ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60434/0 (socket says 172.21.15.60:60434) 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbb99e00640 1 -- 172.21.15.60:0/3256634128 learned_addr learned my addr 172.21.15.60:0/3256634128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbb91400640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0ccf00 0x7fbb9c1f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.056+0000 7fbb9a800640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15f5a0 0x7fbb9c1f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb99e00640 1 -- 172.21.15.60:0/3256634128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15f5a0 msgr2=0x7fbb9c1f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb99e00640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15f5a0 0x7fbb9c1f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb99e00640 1 -- 172.21.15.60:0/3256634128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0ccf00 msgr2=0x7fbb9c1f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb99e00640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0ccf00 0x7fbb9c1f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb99e00640 1 -- 172.21.15.60:0/3256634128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c15c6b0 con 0x7fbb9c073fb0 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb99e00640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 0x7fbb9c09ba50 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00b410 tx=0x7fbb8c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb92a00640 1 -- 172.21.15.60:0/3256634128 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c015020 con 0x7fbb9c073fb0 2024-09-16T15:46:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb92a00640 1 -- 172.21.15.60:0/3256634128 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb8c011940 con 0x7fbb9c073fb0 2024-09-16T15:46:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbb92a00640 1 -- 172.21.15.60:0/3256634128 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c011d90 con 0x7fbb9c073fb0 2024-09-16T15:46:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c157c30 con 0x7fbb9c073fb0 2024-09-16T15:46:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.057+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c09bf90 con 0x7fbb9c073fb0 2024-09-16T15:46:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.058+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c1541f0 con 0x7fbb9c073fb0 2024-09-16T15:46:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.060+0000 7fbb92a00640 1 -- 172.21.15.60:0/3256634128 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fbb8c018020 con 0x7fbb9c073fb0 2024-09-16T15:46:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.060+0000 7fbb92a00640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb78078e80 0x7fbb7807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.061+0000 7fbb92a00640 1 -- 172.21.15.60:0/3256634128 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fbb8c09b3a0 con 0x7fbb9c073fb0 2024-09-16T15:46:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.061+0000 7fbb91400640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb78078e80 0x7fbb7807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.061+0000 7fbb91400640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb78078e80 0x7fbb7807b340 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb840017f0 tx=0x7fbb84009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:16.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.065+0000 7fbb92a00640 1 -- 172.21.15.60:0/3256634128 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb8c01f020 con 0x7fbb9c073fb0 2024-09-16T15:46:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.296+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb9c156340 con 0x7fbb9c073fb0 2024-09-16T15:46:16.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.296+0000 7fbb92a00640 1 -- 172.21.15.60:0/3256634128 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7fbb8c0629a0 con 0x7fbb9c073fb0 2024-09-16T15:46:16.298 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:16.299 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":92,"btime":"2024-09-16T15:46:13:723797+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/766495901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":766495901},{"type":"v1","addr":"172.21.15.100:6835","nonce":766495901}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7585,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/3797165836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3797165836},{"type":"v1","addr":"172.21.15.100:6801","nonce":3797165836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3900896825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3900896825},{"type":"v1","addr":"172.21.15.60:6834","nonce":3900896825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:10.124891+0000","modified":"2024-09-16T15:46:13.158701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/984341325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":984341325},{"type":"v1","addr":"172.21.15.60:6835","nonce":984341325}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7998,"qdb_cluster":[7998]},"id":10}]} 2024-09-16T15:46:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 92 2024-09-16T15:46:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb78078e80 msgr2=0x7fbb7807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb78078e80 0x7fbb7807b340 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb840017f0 tx=0x7fbb84009040 comp rx=0 tx=0).stop 2024-09-16T15:46:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 msgr2=0x7fbb9c09ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 0x7fbb9c09ba50 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00b410 tx=0x7fbb8c00b8d0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 shutdown_connections 2024-09-16T15:46:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbb78078e80 0x7fbb7807b340 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15f5a0 0x7fbb9c1f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0ccf00 0x7fbb9c1f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 --2- 172.21.15.60:0/3256634128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c073fb0 0x7fbb9c09ba50 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 >> 172.21.15.60:0/3256634128 conn(0x7fbb9c0b1950 msgr2=0x7fbb9c0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 shutdown_connections 2024-09-16T15:46:16.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.299+0000 7fbba1000640 1 -- 172.21.15.60:0/3256634128 wait complete. 2024-09-16T15:46:16.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:46:10.124891+0000', 'modified': '2024-09-16T15:46:13.158701+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7998': {'gid': 7998, 'name': 'a', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/984341325', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 984341325}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 984341325}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7998, 'qdb_cluster': [7998]} 2024-09-16T15:46:16.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:46:16.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-16T15:46:16.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.882+0000 7f6f4c600640 1 -- 172.21.15.60:0/3304139599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 msgr2=0x7f6f4406f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.882+0000 7f6f4c600640 1 --2- 172.21.15.60:0/3304139599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 0x7f6f4406f660 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6f440b4890 tx=0x7f6f3002a560 comp rx=0 tx=0).stop 2024-09-16T15:46:16.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.882+0000 7f6f4c600640 1 -- 172.21.15.60:0/3304139599 shutdown_connections 2024-09-16T15:46:16.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.882+0000 7f6f4c600640 1 --2- 172.21.15.60:0/3304139599 >> v2:172.21.15.100:3300/0 conn(0x7f6f44072df0 0x7f6f44073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.882+0000 7f6f4c600640 1 --2- 172.21.15.60:0/3304139599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 0x7f6f4406f660 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.882+0000 7f6f4c600640 1 -- 172.21.15.60:0/3304139599 >> 172.21.15.60:0/3304139599 conn(0x7f6f440b1950 msgr2=0x7f6f440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.882+0000 7f6f4c600640 1 -- 172.21.15.60:0/3304139599 shutdown_connections 2024-09-16T15:46:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.882+0000 7f6f4c600640 1 -- 172.21.15.60:0/3304139599 wait complete. 2024-09-16T15:46:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.883+0000 7f6f4c600640 1 Processor -- start 2024-09-16T15:46:16.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.884+0000 7f6f4c600640 1 -- start start 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.884+0000 7f6f4c600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f44072df0 0x7f6f44149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.884+0000 7f6f4c600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 0x7f6f440cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.884+0000 7f6f4c600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f44161550 0x7f6f440cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.884+0000 7f6f4c600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44073780 con 0x7f6f440af3d0 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.884+0000 7f6f4c600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4406fdb0 con 0x7f6f44072df0 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.884+0000 7f6f4c600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6f44073900 con 0x7f6f44161550 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 0x7f6f440cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 0x7f6f440cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60452/0 (socket says 172.21.15.60:60452) 2024-09-16T15:46:16.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 -- 172.21.15.60:0/4178322545 learned_addr learned my addr 172.21.15.60:0/4178322545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f4a600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f44161550 0x7f6f440cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 -- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f44161550 msgr2=0x7f6f440cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49c00640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f44072df0 0x7f6f44149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f44161550 0x7f6f440cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 -- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f44072df0 msgr2=0x7f6f44149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f44072df0 0x7f6f44149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 -- 172.21.15.60:0/4178322545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f44149bb0 con 0x7f6f440af3d0 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f4a600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f44161550 0x7f6f440cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49c00640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f44072df0 0x7f6f44149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.885+0000 7f6f49200640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 0x7f6f440cae40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009850 tx=0x7f6f30003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.886+0000 7f6f41600640 1 -- 172.21.15.60:0/4178322545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3003f070 con 0x7f6f440af3d0 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.886+0000 7f6f41600640 1 -- 172.21.15.60:0/4178322545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f300047a0 con 0x7f6f440af3d0 2024-09-16T15:46:16.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.886+0000 7f6f4c600640 1 -- 172.21.15.60:0/4178322545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f440cba00 con 0x7f6f440af3d0 2024-09-16T15:46:16.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.886+0000 7f6f41600640 1 -- 172.21.15.60:0/4178322545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30035030 con 0x7f6f440af3d0 2024-09-16T15:46:16.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.886+0000 7f6f4c600640 1 -- 172.21.15.60:0/4178322545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f440cbfc0 con 0x7f6f440af3d0 2024-09-16T15:46:16.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.888+0000 7f6f41600640 1 -- 172.21.15.60:0/4178322545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f6f30042070 con 0x7f6f440af3d0 2024-09-16T15:46:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.889+0000 7f6f41600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f24078eb0 0x7f6f2407b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.889+0000 7f6f41600640 1 -- 172.21.15.60:0/4178322545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f6f300bc170 con 0x7f6f440af3d0 2024-09-16T15:46:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.889+0000 7f6f49c00640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f24078eb0 0x7f6f2407b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.889+0000 7f6f4f5e0740 1 Processor -- start 2024-09-16T15:46:16.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.890+0000 7f6f4f5e0740 1 -- start start 2024-09-16T15:46:16.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.890+0000 7f6f49c00640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f24078eb0 0x7f6f2407b370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f380016a0 tx=0x7f6f38009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.890+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb165c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.890+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120a20 0x55c2bb164390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.890+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb1648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.890+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c2bb0a3430 con 0x55c2bb1658b0 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.890+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c2bb0935d0 con 0x55c2bb120fa0 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.890+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c2bb082a00 con 0x55c2bb120a20 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb165c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60464/0 (socket says 172.21.15.60:60464) 2024-09-16T15:46:16.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 -- 172.21.15.60:0/2303075838 learned_addr learned my addr 172.21.15.60:0/2303075838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49200640 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120a20 0x55c2bb164390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f4a600640 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb1648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 -- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120a20 msgr2=0x55c2bb164390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120a20 0x55c2bb164390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 -- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 msgr2=0x55c2bb1648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:16.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb1648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 -- 172.21.15.60:0/2303075838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2bb04b520 con 0x55c2bb1658b0 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f4a600640 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb1648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.891+0000 7f6f49c00640 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb165c90 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800da10 tx=0x7f6f38010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2488b8ca8a0439 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f36000640 1 -- 172.21.15.60:0/2303075838 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38024070 con 0x55c2bb1658b0 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f36000640 1 -- 172.21.15.60:0/2303075838 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f38014db0 con 0x55c2bb1658b0 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f36000640 1 -- 172.21.15.60:0/2303075838 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3801f620 con 0x55c2bb1658b0 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/2303075838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 msgr2=0x55c2bb165c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb165c90 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800da10 tx=0x7f6f38010000 comp rx=0 tx=0).stop 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/2303075838 shutdown_connections 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb1648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120a20 0x55c2bb164390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/2303075838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb165c90 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/2303075838 >> 172.21.15.60:0/2303075838 conn(0x55c2bb11bff0 msgr2=0x55c2bb11fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.892+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/2303075838 shutdown_connections 2024-09-16T15:46:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.893+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/2303075838 wait complete. 2024-09-16T15:46:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.893+0000 7f6f4f5e0740 1 -- start start 2024-09-16T15:46:16.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.893+0000 7f6f4f5e0740 1 Processor -- start 2024-09-16T15:46:16.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.894+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb15f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.894+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1658b0 0x55c2bb134a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.894+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1645d0 0x55c2bb161a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.894+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c2bb029330 con 0x55c2bb1645d0 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.894+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c2bb086810 con 0x55c2bb120fa0 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.894+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c2badf3620 con 0x55c2bb1658b0 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.894+0000 7f6f4f5e0740 10 client.0 _ll_register_callbacks cb 0x55c2bb02e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1645d0 0x55c2bb161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1645d0 0x55c2bb161a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60476/0 (socket says 172.21.15.60:60476) 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 -- 172.21.15.60:0/692238991 learned_addr learned my addr 172.21.15.60:0/692238991 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f49c00640 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f49200640 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1658b0 0x55c2bb134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 -- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1658b0 msgr2=0x55c2bb134a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1658b0 0x55c2bb134a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 -- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 msgr2=0x55c2bb15f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb15f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 -- 172.21.15.60:0/692238991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2bade4850 con 0x55c2bb1645d0 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f49c00640 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb15f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f4a600640 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1645d0 0x55c2bb161a50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009f90 tx=0x7f6f3c0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262444e795bf0bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.895+0000 7f6f2aa00640 1 -- 172.21.15.60:0/692238991 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c010a00 con 0x55c2bb1645d0 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/692238991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f38014db0 con 0x55c2bb1645d0 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f2aa00640 1 -- 172.21.15.60:0/692238991 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f6f3c010ba0 con 0x55c2bb1645d0 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f2aa00640 1 -- 172.21.15.60:0/692238991 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c023050 con 0x55c2bb1645d0 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 10 client.8121 fetch_fsmap learned FSMap version 92 2024-09-16T15:46:16.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/692238991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55c2bae5b6b0 con 0x55c2bb1645d0 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f2aa00640 1 -- 172.21.15.60:0/692238991 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f38014db0 con 0x55c2bb1645d0 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f2aa00640 1 -- 172.21.15.60:0/692238991 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f6f3c026020 con 0x55c2bb1645d0 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 10 client.8121 fetch_fsmap finished waiting for FSMap version 92 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 10 client.8121 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 20 client.8121 populate_metadata read hostname 'smithi060' 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] conn(0x55c2bb137860 0x55c2bb149140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 4 client.8121 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/692238991 --> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2bb095520 con 0x55c2bb137860 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.896+0000 7f6f49c00640 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] conn(0x55c2bb137860 0x55c2bb149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.897+0000 7f6f49c00640 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] conn(0x55c2bb137860 0x55c2bb149140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bc92f687b6a8bdc server_cookie=84ef46415e1fe5b0 in_seq=0 out_seq=0 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.897+0000 7f6f2aa00640 10 client.8121 ms_handle_connect on v2:172.21.15.60:6833/984341325 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f2aa00640 1 -- 172.21.15.60:0/692238991 <== mds.0 v2:172.21.15.60:6833/984341325 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2bb095520 con 0x55c2bb137860 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f2aa00640 10 client.8121 handle_command_reply: tid=0 2024-09-16T15:46:16.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 client.8121 shutdown 2024-09-16T15:46:16.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/692238991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1645d0 msgr2=0x55c2bb161a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1645d0 0x55c2bb161a50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009f90 tx=0x7f6f3c0044d0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/692238991 shutdown_connections 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] conn(0x55c2bb137860 0x55c2bb149140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1645d0 0x55c2bb161a50 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1658b0 0x55c2bb134a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/692238991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb15f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f2aa00640 0 client.8121 ms_handle_reset on v2:172.21.15.60:6833/984341325 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/692238991 >> 172.21.15.60:0/692238991 conn(0x55c2bb120a20 msgr2=0x55c2bb11abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/692238991 shutdown_connections 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.898+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/692238991 wait complete. 2024-09-16T15:46:16.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.899+0000 7f6f4f5e0740 20 client.8121 trim_cache size 0 max 16384 2024-09-16T15:46:16.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.918+0000 7f6f4f5e0740 1 Processor -- start 2024-09-16T15:46:16.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.918+0000 7f6f4f5e0740 1 -- start start 2024-09-16T15:46:16.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.918+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120fa0 0x55c2bb1497e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.918+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb1645d0 0x55c2bb176c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.918+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb177180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.918+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44200390 con 0x55c2bb1658b0 2024-09-16T15:46:16.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.918+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f441ff6f0 con 0x55c2bb1645d0 2024-09-16T15:46:16.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.918+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6f441ffd40 con 0x55c2bb120fa0 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb177180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb177180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60484/0 (socket says 172.21.15.60:60484) 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 -- 172.21.15.60:0/16429368 learned_addr learned my addr 172.21.15.60:0/16429368 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f49200640 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb1645d0 0x55c2bb176c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f49c00640 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120fa0 0x55c2bb1497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 -- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120fa0 msgr2=0x55c2bb1497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120fa0 0x55c2bb1497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 -- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb1645d0 msgr2=0x55c2bb176c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb1645d0 0x55c2bb176c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 -- 172.21.15.60:0/16429368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2bb11fb10 con 0x55c2bb1658b0 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f49200640 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb1645d0 0x55c2bb176c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f49c00640 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120fa0 0x55c2bb1497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.919+0000 7f6f4a600640 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb177180 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0109c0 tx=0x7f6f3c0109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d0cc99290eba5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:16.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f35600640 1 -- 172.21.15.60:0/16429368 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c010c10 con 0x55c2bb1658b0 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f35600640 1 -- 172.21.15.60:0/16429368 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f3c01b7b0 con 0x55c2bb1658b0 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f35600640 1 -- 172.21.15.60:0/16429368 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c01bc00 con 0x55c2bb1658b0 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/16429368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 msgr2=0x55c2bb177180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb177180 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0109c0 tx=0x7f6f3c0109f0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/16429368 shutdown_connections 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb177180 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb1645d0 0x55c2bb176c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/16429368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb120fa0 0x55c2bb1497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/16429368 >> 172.21.15.60:0/16429368 conn(0x55c2bb120a20 msgr2=0x55c2bb15dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/16429368 shutdown_connections 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.920+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/16429368 wait complete. 2024-09-16T15:46:16.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.921+0000 7f6f4f5e0740 1 -- start start 2024-09-16T15:46:16.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.921+0000 7f6f4f5e0740 1 Processor -- start 2024-09-16T15:46:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb11a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1645d0 0x55c2bb146220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4f5e0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb1e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55c2bb1376e0 con 0x55c2bb1658b0 2024-09-16T15:46:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c2bb1486c0 con 0x55c2bb120fa0 2024-09-16T15:46:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4f5e0740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x55c2bb137d30 con 0x55c2bb1645d0 2024-09-16T15:46:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4f5e0740 10 client.0 _ll_register_callbacks cb 0x7f6f3c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:46:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb1e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb1e51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60496/0 (socket says 172.21.15.60:60496) 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.922+0000 7f6f4a600640 1 -- 172.21.15.60:0/789133318 learned_addr learned my addr 172.21.15.60:0/789133318 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f49c00640 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb11a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f49200640 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1645d0 0x55c2bb146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f4a600640 1 -- 172.21.15.60:0/789133318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1645d0 msgr2=0x55c2bb146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f4a600640 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1645d0 0x55c2bb146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f4a600640 1 -- 172.21.15.60:0/789133318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 msgr2=0x55c2bb11a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f4a600640 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb11a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f4a600640 1 -- 172.21.15.60:0/789133318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2bb136250 con 0x55c2bb1658b0 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f4a600640 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb1e51b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c000f80 tx=0x7f6f3c01cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685f4d046ff7441b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f2aa00640 1 -- 172.21.15.60:0/789133318 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c01b610 con 0x55c2bb1658b0 2024-09-16T15:46:16.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.923+0000 7f6f2aa00640 1 -- 172.21.15.60:0/789133318 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f6f3c012450 con 0x55c2bb1658b0 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/789133318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f3c01b7b0 con 0x55c2bb1658b0 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f2aa00640 1 -- 172.21.15.60:0/789133318 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c040050 con 0x55c2bb1658b0 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 10 client.8127 fetch_fsmap learned FSMap version 92 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/789133318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55c2bb1e9ab0 con 0x55c2bb1658b0 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f2aa00640 1 -- 172.21.15.60:0/789133318 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f3c01b7b0 con 0x55c2bb1658b0 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f2aa00640 1 -- 172.21.15.60:0/789133318 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f6f3c043020 con 0x55c2bb1658b0 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 10 client.8127 fetch_fsmap finished waiting for FSMap version 92 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 10 client.8127 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 20 client.8127 populate_metadata read hostname 'smithi060' 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] conn(0x55c2bb176f70 0x55c2bb15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 4 client.8127 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/789133318 --> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] -- command(tid 0: {"prefix": "status"}) -- 0x55c2bb090fd0 con 0x55c2bb176f70 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.924+0000 7f6f49c00640 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] conn(0x55c2bb176f70 0x55c2bb15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:16.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f49c00640 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] conn(0x55c2bb176f70 0x55c2bb15c760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2717eb068a658df server_cookie=c1724c57e81e2eb5 in_seq=0 out_seq=0 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f2aa00640 10 client.8127 ms_handle_connect on v2:172.21.15.60:6833/984341325 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f2aa00640 1 -- 172.21.15.60:0/789133318 <== mds.0 v2:172.21.15.60:6833/984341325 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55c2bb090fd0 con 0x55c2bb176f70 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f2aa00640 10 client.8127 handle_command_reply: tid=0 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f4f5e0740 1 client.8127 shutdown 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/789133318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 msgr2=0x55c2bb1e51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb1e51b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c000f80 tx=0x7f6f3c01cf90 comp rx=0 tx=0).stop 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/789133318 shutdown_connections 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.60:6833/984341325,v1:172.21.15.60:6835/984341325] conn(0x55c2bb176f70 0x55c2bb15c760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c2bb1658b0 0x55c2bb1e51b0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c2bb1645d0 0x55c2bb146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f4f5e0740 1 --2- 172.21.15.60:0/789133318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c2bb120fa0 0x55c2bb11a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.925+0000 7f6f2aa00640 0 client.8127 ms_handle_reset on v2:172.21.15.60:6833/984341325 2024-09-16T15:46:16.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/789133318 >> 172.21.15.60:0/789133318 conn(0x55c2bb120a20 msgr2=0x55c2bb11ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/789133318 shutdown_connections 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4f5e0740 1 -- 172.21.15.60:0/789133318 wait complete. 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4f5e0740 20 client.8127 trim_cache size 0 max 16384 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7998, 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:46:16.932 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:46:16.933 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:46:16.933 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2220670409999999, 2024-09-16T15:46:16.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 92, 2024-09-16T15:46:16.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 137, 2024-09-16T15:46:16.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 137, 2024-09-16T15:46:16.933 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 17.535742150000001 2024-09-16T15:46:16.933 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 -- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f24078eb0 msgr2=0x7f6f2407b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f24078eb0 0x7f6f2407b370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f380016a0 tx=0x7f6f38009040 comp rx=0 tx=0).stop 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 -- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 msgr2=0x7f6f440cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 0x7f6f440cae40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009850 tx=0x7f6f30003820 comp rx=0 tx=0).stop 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 -- 172.21.15.60:0/4178322545 shutdown_connections 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6f24078eb0 0x7f6f2407b370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6f44161550 0x7f6f440cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6f440af3d0 0x7f6f440cae40 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 --2- 172.21.15.60:0/4178322545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f44072df0 0x7f6f44149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.926+0000 7f6f4c600640 1 -- 172.21.15.60:0/4178322545 >> 172.21.15.60:0/4178322545 conn(0x7f6f440b1950 msgr2=0x7f6f4415ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.927+0000 7f6f4c600640 1 -- 172.21.15.60:0/4178322545 shutdown_connections 2024-09-16T15:46:16.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:16.927+0000 7f6f4c600640 1 -- 172.21.15.60:0/4178322545 wait complete. 2024-09-16T15:46:16.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:46:17.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.103+0000 7f244c800640 1 -- 172.21.15.60:0/373829776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2444161550 msgr2=0x7f244406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:17.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.103+0000 7f244c800640 1 --2- 172.21.15.60:0/373829776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2444161550 0x7f244406ef50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2438009830 tx=0x7f243802a480 comp rx=0 tx=0).stop 2024-09-16T15:46:17.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.103+0000 7f244c800640 1 -- 172.21.15.60:0/373829776 shutdown_connections 2024-09-16T15:46:17.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.103+0000 7f244c800640 1 --2- 172.21.15.60:0/373829776 >> v2:172.21.15.100:3300/0 conn(0x7f2444072df0 0x7f24440732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.103+0000 7f244c800640 1 --2- 172.21.15.60:0/373829776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2444161550 0x7f244406ef50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.103+0000 7f244c800640 1 -- 172.21.15.60:0/373829776 >> 172.21.15.60:0/373829776 conn(0x7f24440b1950 msgr2=0x7f24440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:17.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.103+0000 7f244c800640 1 -- 172.21.15.60:0/373829776 shutdown_connections 2024-09-16T15:46:17.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.103+0000 7f244c800640 1 -- 172.21.15.60:0/373829776 wait complete. 2024-09-16T15:46:17.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.104+0000 7f244c800640 1 Processor -- start 2024-09-16T15:46:17.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.105+0000 7f244c800640 1 -- start start 2024-09-16T15:46:17.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244c800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2444072df0 0x7f24440cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244c800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24440af3d0 0x7f24440d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244c800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2444161550 0x7f24441f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244c800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f24440737e0 con 0x7f2444161550 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244c800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f244406fdb0 con 0x7f2444072df0 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244c800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2444073960 con 0x7f24440af3d0 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2444161550 0x7f24441f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2444161550 0x7f24441f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60502/0 (socket says 172.21.15.60:60502) 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244a800640 1 -- 172.21.15.60:0/1732901563 learned_addr learned my addr 172.21.15.60:0/1732901563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f2449e00640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2444072df0 0x7f24440cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f2449400640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24440af3d0 0x7f24440d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244a800640 1 -- 172.21.15.60:0/1732901563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24440af3d0 msgr2=0x7f24440d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244a800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24440af3d0 0x7f24440d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244a800640 1 -- 172.21.15.60:0/1732901563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2444072df0 msgr2=0x7f24440cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:17.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244a800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2444072df0 0x7f24440cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.106+0000 7f244a800640 1 -- 172.21.15.60:0/1732901563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244409aa00 con 0x7f2444161550 2024-09-16T15:46:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.107+0000 7f244a800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2444161550 0x7f24441f8120 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f244000eb10 tx=0x7f244000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.107+0000 7f243e000640 1 -- 172.21.15.60:0/1732901563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440004b20 con 0x7f2444161550 2024-09-16T15:46:17.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.107+0000 7f243e000640 1 -- 172.21.15.60:0/1732901563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2440004ce0 con 0x7f2444161550 2024-09-16T15:46:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.107+0000 7f243e000640 1 -- 172.21.15.60:0/1732901563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440005280 con 0x7f2444161550 2024-09-16T15:46:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.107+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244409ace0 con 0x7f2444161550 2024-09-16T15:46:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.107+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244409b580 con 0x7f2444161550 2024-09-16T15:46:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.108+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2444073ec0 con 0x7f2444161550 2024-09-16T15:46:17.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.109+0000 7f243e000640 1 -- 172.21.15.60:0/1732901563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f2440005420 con 0x7f2444161550 2024-09-16T15:46:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.111+0000 7f243e000640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2428078eb0 0x7f242807b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.111+0000 7f243e000640 1 -- 172.21.15.60:0/1732901563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f244009c1f0 con 0x7f2444161550 2024-09-16T15:46:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.111+0000 7f2449e00640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2428078eb0 0x7f242807b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:17.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.112+0000 7f2449e00640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2428078eb0 0x7f242807b370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2438034000 tx=0x7f2438004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:17.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.115+0000 7f243e000640 1 -- 172.21.15.60:0/1732901563 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2440063760 con 0x7f2444161550 2024-09-16T15:46:17.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.348+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f244409aeb0 con 0x7f2444161550 2024-09-16T15:46:17.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.349+0000 7f243e000640 1 -- 172.21.15.60:0/1732901563 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7f2440068190 con 0x7f2444161550 2024-09-16T15:46:17.351 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2428078eb0 msgr2=0x7f242807b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2428078eb0 0x7f242807b370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2438034000 tx=0x7f2438004730 comp rx=0 tx=0).stop 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2444161550 msgr2=0x7f24441f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2444161550 0x7f24441f8120 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f244000eb10 tx=0x7f244000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 shutdown_connections 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2428078eb0 0x7f242807b370 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2444161550 0x7f24441f8120 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24440af3d0 0x7f24440d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 --2- 172.21.15.60:0/1732901563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2444072df0 0x7f24440cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.351+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 >> 172.21.15.60:0/1732901563 conn(0x7f24440b1950 msgr2=0x7f24440740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.352+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 shutdown_connections 2024-09-16T15:46:17.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17.352+0000 7f244c800640 1 -- 172.21.15.60:0/1732901563 wait complete. 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:46:17.369 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:46:17.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:17.369 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:46:17.405 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:46:17.405 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:46:17.405 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:46:17.405 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:17.405 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:46:17.405 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:17.405 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42272sec preferred_lft 42272sec 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:17.406 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:46:17.407 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:17.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:17.408 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:46:17.499 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:46:17.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:17.502 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:46:17.538 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:46:17.538 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:46:17.538 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:46:17.538 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:46:17.538 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:46:17.539 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:46:17.539 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:46:17.539 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:46:17.539 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:46:17.539 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:46:17.539 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:46:17.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:46:17.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:17.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:17.739 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:46:17.774 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:46:17.774 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:46:17.774 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:46:17.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:17.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:46:17.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:46:17.870 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:17.905 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:46:17.905 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:46:17.905 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:17.980 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:46:17.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:17.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:18.108 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:18.108 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:46:18.135 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:18.135 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:46:18.135 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:46:18.135 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:46:18.135 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:46:18.137 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:46:18.137 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:46:18.137 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:46:18.137 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:46:18.137 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:46:18.140 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:46:18.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:18.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:18.195 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:46:18.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:46:18.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:18.292 INFO:teuthology.orchestra.run.smithi060.stdout:client6911 2024-09-16T15:46:18.292 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:46:18.292 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:18.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:46:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.527+0000 7f781fe00640 1 -- 172.21.15.60:0/309445859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7800003a00 msgr2=0x7f7800003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.527+0000 7f781fe00640 1 --2- 172.21.15.60:0/309445859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7800003a00 0x7f7800003e20 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f780c00c310 tx=0x7f7800005830 comp rx=0 tx=0).stop 2024-09-16T15:46:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.527+0000 7f781de00640 1 -- 172.21.15.60:0/309445859 reap_dead start 2024-09-16T15:46:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.527+0000 7f781fe00640 1 -- 172.21.15.60:0/309445859 shutdown_connections 2024-09-16T15:46:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.527+0000 7f781fe00640 1 -- 172.21.15.60:0/309445859 >> 172.21.15.60:0/309445859 conn(0x7f78180b1950 msgr2=0x7f78180b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.527+0000 7f781fe00640 1 -- 172.21.15.60:0/309445859 shutdown_connections 2024-09-16T15:46:18.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.527+0000 7f781fe00640 1 -- 172.21.15.60:0/309445859 wait complete. 2024-09-16T15:46:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.529+0000 7f781fe00640 1 Processor -- start 2024-09-16T15:46:18.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.529+0000 7f781fe00640 1 -- start start 2024-09-16T15:46:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7800003a00 0x7f781815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78180c8fd0 0x7f78180c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78181f8150 0x7f78181f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f781806f9f0 con 0x7f78180c8fd0 2024-09-16T15:46:18.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f781806f870 con 0x7f7800003a00 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f781806fb70 con 0x7f78181f8150 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78180c8fd0 0x7f78180c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78180c8fd0 0x7f78180c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60522/0 (socket says 172.21.15.60:60522) 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781ca00640 1 -- 172.21.15.60:0/935359281 learned_addr learned my addr 172.21.15.60:0/935359281 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781de00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78181f8150 0x7f78181f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.530+0000 7f781d400640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7800003a00 0x7f781815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f781ca00640 1 -- 172.21.15.60:0/935359281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78181f8150 msgr2=0x7f78181f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f781ca00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78181f8150 0x7f78181f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f781ca00640 1 -- 172.21.15.60:0/935359281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7800003a00 msgr2=0x7f781815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f781ca00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7800003a00 0x7f781815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f781ca00640 1 -- 172.21.15.60:0/935359281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78180c9550 con 0x7f78180c8fd0 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f781ca00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78180c8fd0 0x7f78180c9970 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f78040119e0 tx=0x7f7804011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f7815600640 1 -- 172.21.15.60:0/935359281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780400fde0 con 0x7f78180c8fd0 2024-09-16T15:46:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f7815600640 1 -- 172.21.15.60:0/935359281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7804005020 con 0x7f78180c8fd0 2024-09-16T15:46:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f7815600640 1 -- 172.21.15.60:0/935359281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804005450 con 0x7f78180c8fd0 2024-09-16T15:46:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78180cc0d0 con 0x7f78180c8fd0 2024-09-16T15:46:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.531+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78180cc690 con 0x7f78180c8fd0 2024-09-16T15:46:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.532+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78180732a0 con 0x7f78180c8fd0 2024-09-16T15:46:18.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.534+0000 7f7815600640 1 -- 172.21.15.60:0/935359281 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f78040047a0 con 0x7f78180c8fd0 2024-09-16T15:46:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.534+0000 7f7815600640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f77f8078eb0 0x7f77f807b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.535+0000 7f7815600640 1 -- 172.21.15.60:0/935359281 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f780409c280 con 0x7f78180c8fd0 2024-09-16T15:46:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.535+0000 7f781d400640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f77f8078eb0 0x7f77f807b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:18.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.536+0000 7f781d400640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f77f8078eb0 0x7f77f807b370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f780c033000 tx=0x7f780c038040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:18.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.539+0000 7f7815600640 1 -- 172.21.15.60:0/935359281 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f78040637f0 con 0x7f78180c8fd0 2024-09-16T15:46:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.778+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f78180c9720 con 0x7f78180c8fd0 2024-09-16T15:46:18.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.778+0000 7f7815600640 1 -- 172.21.15.60:0/935359281 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7f78180c9720 con 0x7f78180c8fd0 2024-09-16T15:46:18.780 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.780+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f77f8078eb0 msgr2=0x7f77f807b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.780+0000 7f781fe00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f77f8078eb0 0x7f77f807b370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f780c033000 tx=0x7f780c038040 comp rx=0 tx=0).stop 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78180c8fd0 msgr2=0x7f78180c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78180c8fd0 0x7f78180c9970 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f78040119e0 tx=0x7f7804011ea0 comp rx=0 tx=0).stop 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 shutdown_connections 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f77f8078eb0 0x7f77f807b370 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78181f8150 0x7f78181f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78180c8fd0 0x7f78180c9970 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:18.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 --2- 172.21.15.60:0/935359281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7800003a00 0x7f781815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:18.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 >> 172.21.15.60:0/935359281 conn(0x7f78180b1950 msgr2=0x7f781815d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:18.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 shutdown_connections 2024-09-16T15:46:18.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:18.781+0000 7f781fe00640 1 -- 172.21.15.60:0/935359281 wait complete. 2024-09-16T15:46:18.798 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:46:18.799 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:46:18.799 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:46:18.799 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:46:18.799 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:18.799 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:46:18.799 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:46:18.799 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:46:18.799 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:46:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:18.799 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:46:18.839 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:46:18.839 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:46:18.839 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:46:18.839 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:18.839 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:46:18.840 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42270sec preferred_lft 42270sec 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:18.841 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:46:18.842 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:46:18.842 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:18.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:18.843 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:46:18.942 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:46:18.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:18.945 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:46:18.983 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:46:18.983 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:46:18.983 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:46:18.983 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:46:18.983 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:46:18.983 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:46:18.983 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:46:18.983 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:46:18.983 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:46:18.983 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:46:18.984 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:46:19.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:46:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:46:19.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:46:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:19.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:19.179 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:46:19.214 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:46:19.214 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:46:19.214 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:46:19.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:19.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:46:19.312 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:46:19.313 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:19.348 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:46:19.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:46:19.348 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:19.422 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:46:19.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:19.423 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:19.551 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:19.552 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:46:19.579 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:19.579 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:46:19.579 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:46:19.579 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:46:19.579 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:46:19.582 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:46:19.582 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:46:19.582 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:46:19.582 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:46:19.582 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:46:19.585 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:46:19.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:19.585 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:19.640 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:46:19.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:46:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:46:19.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:46:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:19.729 INFO:teuthology.orchestra.run.smithi100.stdout:client8142 2024-09-16T15:46:19.729 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:46:19.729 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:19.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:46:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.939+0000 7fb614c00640 1 -- 172.21.15.60:0/3072975162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb61006e6f0 msgr2=0x7fb61006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.939+0000 7fb614c00640 1 --2- 172.21.15.60:0/3072975162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb61006e6f0 0x7fb61006eb60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb608005fd0 tx=0x7fb60802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:46:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.939+0000 7fb614c00640 1 -- 172.21.15.60:0/3072975162 shutdown_connections 2024-09-16T15:46:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.939+0000 7fb614c00640 1 --2- 172.21.15.60:0/3072975162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb61006e6f0 0x7fb61006eb60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.939+0000 7fb614c00640 1 --2- 172.21.15.60:0/3072975162 >> v2:172.21.15.100:3300/0 conn(0x7fb610158900 0x7fb61015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.939+0000 7fb614c00640 1 -- 172.21.15.60:0/3072975162 >> 172.21.15.60:0/3072975162 conn(0x7fb6100b18e0 msgr2=0x7fb6100b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.940+0000 7fb614c00640 1 -- 172.21.15.60:0/3072975162 shutdown_connections 2024-09-16T15:46:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.940+0000 7fb614c00640 1 -- 172.21.15.60:0/3072975162 wait complete. 2024-09-16T15:46:19.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.941+0000 7fb614c00640 1 Processor -- start 2024-09-16T15:46:19.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.941+0000 7fb614c00640 1 -- start start 2024-09-16T15:46:19.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb614c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb61006e6f0 0x7fb61009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:19.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb614c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb610158900 0x7fb61014bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb614c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb61015b700 0x7fb61014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb614c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb6100ced60 con 0x7fb610158900 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb614c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6100cebe0 con 0x7fb61006e6f0 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb614c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb6100ceee0 con 0x7fb61015b700 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb60d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb610158900 0x7fb61014bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb60de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb61006e6f0 0x7fb61009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.942+0000 7fb60e800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb61015b700 0x7fb61014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb610158900 0x7fb61014bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45972/0 (socket says 172.21.15.60:45972) 2024-09-16T15:46:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60d400640 1 -- 172.21.15.60:0/207557434 learned_addr learned my addr 172.21.15.60:0/207557434 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:19.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60d400640 1 -- 172.21.15.60:0/207557434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb61015b700 msgr2=0x7fb61014e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:19.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60d400640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb61015b700 0x7fb61014e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:19.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60d400640 1 -- 172.21.15.60:0/207557434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb61006e6f0 msgr2=0x7fb61009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:19.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60d400640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb61006e6f0 0x7fb61009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:19.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60d400640 1 -- 172.21.15.60:0/207557434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61009c490 con 0x7fb610158900 2024-09-16T15:46:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60de00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb61006e6f0 0x7fb61009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb60d400640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb610158900 0x7fb61014bcf0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c430 tx=0x7fb5fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.943+0000 7fb606000640 1 -- 172.21.15.60:0/207557434 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc011790 con 0x7fb610158900 2024-09-16T15:46:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.944+0000 7fb606000640 1 -- 172.21.15.60:0/207557434 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5fc017070 con 0x7fb610158900 2024-09-16T15:46:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.944+0000 7fb606000640 1 -- 172.21.15.60:0/207557434 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc011ce0 con 0x7fb610158900 2024-09-16T15:46:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.944+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61014c230 con 0x7fb610158900 2024-09-16T15:46:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.944+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6100cbd80 con 0x7fb610158900 2024-09-16T15:46:19.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.945+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb610069360 con 0x7fb610158900 2024-09-16T15:46:19.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.946+0000 7fb606000640 1 -- 172.21.15.60:0/207557434 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fb5fc01c460 con 0x7fb610158900 2024-09-16T15:46:19.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.947+0000 7fb606000640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5f0078e80 0x7fb5f007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:19.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.947+0000 7fb606000640 1 -- 172.21.15.60:0/207557434 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fb5fc09b240 con 0x7fb610158900 2024-09-16T15:46:19.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.947+0000 7fb60de00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5f0078e80 0x7fb5f007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:19.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.948+0000 7fb60de00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5f0078e80 0x7fb5f007b340 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb6000095f0 tx=0x7fb600005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:19.952+0000 7fb606000640 1 -- 172.21.15.60:0/207557434 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5fc0627f0 con 0x7fb610158900 2024-09-16T15:46:20.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.185+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb6100cf440 con 0x7fb610158900 2024-09-16T15:46:20.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.745+0000 7fb606000640 1 -- 172.21.15.60:0/207557434 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v93) ==== 136+0+0 (secure 0 0 0) 0x7fb5fc067220 con 0x7fb610158900 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5f0078e80 msgr2=0x7fb5f007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5f0078e80 0x7fb5f007b340 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb6000095f0 tx=0x7fb600005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb610158900 msgr2=0x7fb61014bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb610158900 0x7fb61014bcf0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00c430 tx=0x7fb5fc00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 shutdown_connections 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5f0078e80 0x7fb5f007b340 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb61015b700 0x7fb61014e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb610158900 0x7fb61014bcf0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 --2- 172.21.15.60:0/207557434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb61006e6f0 0x7fb61009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.747+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 >> 172.21.15.60:0/207557434 conn(0x7fb6100b18e0 msgr2=0x7fb6100cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.748+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 shutdown_connections 2024-09-16T15:46:20.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.748+0000 7fb614c00640 1 -- 172.21.15.60:0/207557434 wait complete. 2024-09-16T15:46:20.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:46:20.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.929+0000 7ffbd0600640 1 -- 172.21.15.60:0/4194131877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 msgr2=0x7ffbb0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:20.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.929+0000 7ffbd0600640 1 --2- 172.21.15.60:0/4194131877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 0x7ffbb0003ea0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc006f70 tx=0x7ffbb0005830 comp rx=0 tx=0).stop 2024-09-16T15:46:20.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.929+0000 7ffbce600640 1 -- 172.21.15.60:0/4194131877 reap_dead start 2024-09-16T15:46:20.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.929+0000 7ffbd0600640 1 -- 172.21.15.60:0/4194131877 shutdown_connections 2024-09-16T15:46:20.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.929+0000 7ffbd0600640 1 -- 172.21.15.60:0/4194131877 >> 172.21.15.60:0/4194131877 conn(0x7ffbc80b1950 msgr2=0x7ffbc80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:20.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.929+0000 7ffbd0600640 1 -- 172.21.15.60:0/4194131877 shutdown_connections 2024-09-16T15:46:20.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.929+0000 7ffbd0600640 1 -- 172.21.15.60:0/4194131877 wait complete. 2024-09-16T15:46:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.930+0000 7ffbd0600640 1 Processor -- start 2024-09-16T15:46:20.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.931+0000 7ffbd0600640 1 -- start start 2024-09-16T15:46:20.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbd0600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 0x7ffbc8145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:20.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbd0600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbc80cc680 0x7ffbc8152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:20.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbd0600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbc80ccb00 0x7ffbc8152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbd0600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8074840 con 0x7ffbb0003a80 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbd0600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc80746c0 con 0x7ffbc80cc680 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbd0600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc806f160 con 0x7ffbc80ccb00 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcdc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 0x7ffbc8145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcdc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 0x7ffbc8145d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45988/0 (socket says 172.21.15.60:45988) 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcdc00640 1 -- 172.21.15.60:0/4160504596 learned_addr learned my addr 172.21.15.60:0/4160504596 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbce600640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbc80ccb00 0x7ffbc8152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcd200640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbc80cc680 0x7ffbc8152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcdc00640 1 -- 172.21.15.60:0/4160504596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbc80ccb00 msgr2=0x7ffbc8152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcdc00640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbc80ccb00 0x7ffbc8152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcdc00640 1 -- 172.21.15.60:0/4160504596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbc80cc680 msgr2=0x7ffbc8152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcdc00640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbc80cc680 0x7ffbc8152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.932+0000 7ffbcdc00640 1 -- 172.21.15.60:0/4160504596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc8146290 con 0x7ffbb0003a80 2024-09-16T15:46:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.933+0000 7ffbcdc00640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 0x7ffbc8145d50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc031040 tx=0x7ffbbc001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.933+0000 7ffbc5600640 1 -- 172.21.15.60:0/4160504596 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc032030 con 0x7ffbb0003a80 2024-09-16T15:46:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.933+0000 7ffbc5600640 1 -- 172.21.15.60:0/4160504596 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbbc003b10 con 0x7ffbb0003a80 2024-09-16T15:46:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.933+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc81f8440 con 0x7ffbb0003a80 2024-09-16T15:46:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.933+0000 7ffbc5600640 1 -- 172.21.15.60:0/4160504596 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc037450 con 0x7ffbb0003a80 2024-09-16T15:46:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.933+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc81f8a00 con 0x7ffbb0003a80 2024-09-16T15:46:20.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.934+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc806f6c0 con 0x7ffbb0003a80 2024-09-16T15:46:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.935+0000 7ffbc5600640 1 -- 172.21.15.60:0/4160504596 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7ffbbc035020 con 0x7ffbb0003a80 2024-09-16T15:46:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.936+0000 7ffbc5600640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffba8078e80 0x7ffba807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.936+0000 7ffbc5600640 1 -- 172.21.15.60:0/4160504596 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7ffbbc0bbaa0 con 0x7ffbb0003a80 2024-09-16T15:46:20.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.937+0000 7ffbcd200640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffba8078e80 0x7ffba807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:20.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.937+0000 7ffbcd200640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffba8078e80 0x7ffba807b340 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4000c00 tx=0x7ffbb4006ac0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:20.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:20.940+0000 7ffbc5600640 1 -- 172.21.15.60:0/4160504596 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffbbc047030 con 0x7ffbb0003a80 2024-09-16T15:46:21.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.158+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ffbc8153ec0 con 0x7ffba8078e80 2024-09-16T15:46:21.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.159+0000 7ffbc5600640 1 -- 172.21.15.60:0/4160504596 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ffbc8153ec0 con 0x7ffba8078e80 2024-09-16T15:46:21.162 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:46:21.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:21.162 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:46:21.162 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:21.162 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:46:21.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffba8078e80 msgr2=0x7ffba807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffba8078e80 0x7ffba807b340 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4000c00 tx=0x7ffbb4006ac0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 msgr2=0x7ffbc8145d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 0x7ffbc8145d50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc031040 tx=0x7ffbbc001d00 comp rx=0 tx=0).stop 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 shutdown_connections 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffba8078e80 0x7ffba807b340 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffbc80ccb00 0x7ffbc8152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffbc80cc680 0x7ffbc8152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 --2- 172.21.15.60:0/4160504596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffbb0003a80 0x7ffbc8145d50 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 >> 172.21.15.60:0/4160504596 conn(0x7ffbc80b1950 msgr2=0x7ffbc809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 shutdown_connections 2024-09-16T15:46:21.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.162+0000 7ffbd0600640 1 -- 172.21.15.60:0/4160504596 wait complete. 2024-09-16T15:46:21.180 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.342+0000 7f376c000640 1 -- 172.21.15.60:0/2986321117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374c003a00 msgr2=0x7f374c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.342+0000 7f376c000640 1 --2- 172.21.15.60:0/2986321117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374c003a00 0x7f374c003de0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f375800c470 tx=0x7f374c005830 comp rx=0 tx=0).stop 2024-09-16T15:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.342+0000 7f376c000640 1 -- 172.21.15.60:0/2986321117 shutdown_connections 2024-09-16T15:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.342+0000 7f376c000640 1 --2- 172.21.15.60:0/2986321117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f374c003a00 0x7f374c003de0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.342+0000 7f376c000640 1 -- 172.21.15.60:0/2986321117 >> 172.21.15.60:0/2986321117 conn(0x7f37640b1840 msgr2=0x7f37640b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.343+0000 7f376c000640 1 -- 172.21.15.60:0/2986321117 shutdown_connections 2024-09-16T15:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.343+0000 7f376c000640 1 -- 172.21.15.60:0/2986321117 wait complete. 2024-09-16T15:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.344+0000 7f376c000640 1 Processor -- start 2024-09-16T15:46:21.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.344+0000 7f376c000640 1 -- start start 2024-09-16T15:46:21.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376c000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c003a00 0x7f3764156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:21.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376c000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f376415e7f0 0x7f37641f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:21.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376c000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764160a00 0x7f376415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:21.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376c000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f37640cf1f0 con 0x7f3764160a00 2024-09-16T15:46:21.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376c000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37640cf070 con 0x7f374c003a00 2024-09-16T15:46:21.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376c000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f37640cf370 con 0x7f376415e7f0 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764160a00 0x7f376415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376a000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764160a00 0x7f376415c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46012/0 (socket says 172.21.15.60:46012) 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.345+0000 7f376a000640 1 -- 172.21.15.60:0/2695058886 learned_addr learned my addr 172.21.15.60:0/2695058886 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f3769600640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c003a00 0x7f3764156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f3768c00640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f376415e7f0 0x7f37641f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f376a000640 1 -- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f376415e7f0 msgr2=0x7f37641f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f376a000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f376415e7f0 0x7f37641f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f376a000640 1 -- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c003a00 msgr2=0x7f3764156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f376a000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c003a00 0x7f3764156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f376a000640 1 -- 172.21.15.60:0/2695058886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376415ec80 con 0x7f3764160a00 2024-09-16T15:46:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f3768c00640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f376415e7f0 0x7f37641f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f376a000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764160a00 0x7f376415c8b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f37640b4a60 tx=0x7f375c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f3761600640 1 -- 172.21.15.60:0/2695058886 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c00ee70 con 0x7f3764160a00 2024-09-16T15:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f3761600640 1 -- 172.21.15.60:0/2695058886 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f375c015070 con 0x7f3764160a00 2024-09-16T15:46:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f3761600640 1 -- 172.21.15.60:0/2695058886 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c003d00 con 0x7f3764160a00 2024-09-16T15:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.346+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376415ef60 con 0x7f3764160a00 2024-09-16T15:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.347+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641f53d0 con 0x7f3764160a00 2024-09-16T15:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.348+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37640b2ec0 con 0x7f3764160a00 2024-09-16T15:46:21.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.349+0000 7f3761600640 1 -- 172.21.15.60:0/2695058886 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f375c004b20 con 0x7f3764160a00 2024-09-16T15:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.350+0000 7f3761600640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3744078e80 0x7f374407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.350+0000 7f3761600640 1 -- 172.21.15.60:0/2695058886 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f375c09bd00 con 0x7f3764160a00 2024-09-16T15:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.350+0000 7f3769600640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3744078e80 0x7f374407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:21.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.351+0000 7f3769600640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3744078e80 0x7f374407b340 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3758033000 tx=0x7f37640750b0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:21.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.354+0000 7f3761600640 1 -- 172.21.15.60:0/2695058886 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f375c063270 con 0x7f3764160a00 2024-09-16T15:46:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.550+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3764156d60 con 0x7f3764160a00 2024-09-16T15:46:21.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.550+0000 7f3761600640 1 -- 172.21.15.60:0/2695058886 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f375c067ca0 con 0x7f3764160a00 2024-09-16T15:46:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3744078e80 msgr2=0x7f374407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3744078e80 0x7f374407b340 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3758033000 tx=0x7f37640750b0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764160a00 msgr2=0x7f376415c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764160a00 0x7f376415c8b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f37640b4a60 tx=0x7f375c010f40 comp rx=0 tx=0).stop 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 shutdown_connections 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3744078e80 0x7f374407b340 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3764160a00 0x7f376415c8b0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f376415e7f0 0x7f37641f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 --2- 172.21.15.60:0/2695058886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f374c003a00 0x7f3764156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 >> 172.21.15.60:0/2695058886 conn(0x7f37640b1840 msgr2=0x7f37640af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 shutdown_connections 2024-09-16T15:46:21.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.553+0000 7f376c000640 1 -- 172.21.15.60:0/2695058886 wait complete. 2024-09-16T15:46:21.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-16T15:46:21.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.733+0000 7f8170a00640 1 -- 172.21.15.60:0/1621702825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c0af190 msgr2=0x7f816c0b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.733+0000 7f8170a00640 1 --2- 172.21.15.60:0/1621702825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c0af190 0x7f816c0b2b40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8154009830 tx=0x7f815402a480 comp rx=0 tx=0).stop 2024-09-16T15:46:21.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.733+0000 7f8170a00640 1 -- 172.21.15.60:0/1621702825 shutdown_connections 2024-09-16T15:46:21.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.733+0000 7f8170a00640 1 --2- 172.21.15.60:0/1621702825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c0af190 0x7f816c0b2b40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.733+0000 7f8170a00640 1 --2- 172.21.15.60:0/1621702825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c161760 0x7f816c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.733+0000 7f8170a00640 1 -- 172.21.15.60:0/1621702825 >> 172.21.15.60:0/1621702825 conn(0x7f816c0b1840 msgr2=0x7f816c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:21.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.733+0000 7f8170a00640 1 -- 172.21.15.60:0/1621702825 shutdown_connections 2024-09-16T15:46:21.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.733+0000 7f8170a00640 1 -- 172.21.15.60:0/1621702825 wait complete. 2024-09-16T15:46:21.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.735+0000 7f8170a00640 1 Processor -- start 2024-09-16T15:46:21.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.735+0000 7f8170a00640 1 -- start start 2024-09-16T15:46:21.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8170a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c0af190 0x7f816c1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:21.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8170a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c0cec60 0x7f816c1f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:21.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8170a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c161760 0x7f816c0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8170a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f816c162a20 con 0x7f816c0cec60 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8170a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1628a0 con 0x7f816c0af190 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8170a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f816c162ba0 con 0x7f816c161760 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8161400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c0cec60 0x7f816c1f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8161400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c0cec60 0x7f816c1f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46014/0 (socket says 172.21.15.60:46014) 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8161400640 1 -- 172.21.15.60:0/2313934988 learned_addr learned my addr 172.21.15.60:0/2313934988 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8169e00640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c0af190 0x7f816c1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f816a800640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c161760 0x7f816c0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8161400640 1 -- 172.21.15.60:0/2313934988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c161760 msgr2=0x7f816c0cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8161400640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f816c161760 0x7f816c0cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8161400640 1 -- 172.21.15.60:0/2313934988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c0af190 msgr2=0x7f816c1580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:21.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8161400640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c0af190 0x7f816c1580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8161400640 1 -- 172.21.15.60:0/2313934988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c1f4ce0 con 0x7f816c0cec60 2024-09-16T15:46:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.736+0000 7f8169e00640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f816c0af190 0x7f816c1580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.737+0000 7f8161400640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f816c0cec60 0x7f816c1f5480 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8154000c00 tx=0x7f8154032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.737+0000 7f8162a00640 1 -- 172.21.15.60:0/2313934988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154040070 con 0x7f816c0cec60 2024-09-16T15:46:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.737+0000 7f8162a00640 1 -- 172.21.15.60:0/2313934988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81540038e0 con 0x7f816c0cec60 2024-09-16T15:46:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.737+0000 7f8162a00640 1 -- 172.21.15.60:0/2313934988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154003d30 con 0x7f816c0cec60 2024-09-16T15:46:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.737+0000 7f8170a00640 1 -- 172.21.15.60:0/2313934988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c1559f0 con 0x7f816c0cec60 2024-09-16T15:46:21.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.737+0000 7f8170a00640 1 -- 172.21.15.60:0/2313934988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c155710 con 0x7f816c0cec60 2024-09-16T15:46:21.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.738+0000 7f8170a00640 1 -- 172.21.15.60:0/2313934988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c0b2b40 con 0x7f816c0cec60 2024-09-16T15:46:21.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.739+0000 7f8162a00640 1 -- 172.21.15.60:0/2313934988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f8154039040 con 0x7f816c0cec60 2024-09-16T15:46:21.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.740+0000 7f8162a00640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8148078e80 0x7f814807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.740+0000 7f8162a00640 1 -- 172.21.15.60:0/2313934988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f81540bc9c0 con 0x7f816c0cec60 2024-09-16T15:46:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.741+0000 7f8169e00640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8148078e80 0x7f814807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:21.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.741+0000 7f8169e00640 1 --2- 172.21.15.60:0/2313934988 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8148078e80 0x7f814807b340 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f815c0095f0 tx=0x7f815c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:21.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.745+0000 7f8162a00640 1 -- 172.21.15.60:0/2313934988 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8154084130 con 0x7f816c0cec60 2024-09-16T15:46:21.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.966+0000 7f8170a00640 1 -- 172.21.15.60:0/2313934988 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f816c15e0a0 con 0x7f8148078e80 2024-09-16T15:46:21.969 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:21.968+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:21.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:21.968+0000 7f8162a00640 1 -- 172.21.15.60:0/2313934988 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f816c15e0a0 con 0x7f8148078e80 2024-09-16T15:46:21.970 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:21.980 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:21.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-16T15:46:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.141+0000 7ffa55200640 1 -- 172.21.15.60:0/1211130849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa500cccd0 msgr2=0x7ffa50073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.141+0000 7ffa55200640 1 --2- 172.21.15.60:0/1211130849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa500cccd0 0x7ffa50073440 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ffa500cfba0 tx=0x7ffa3802a580 comp rx=0 tx=0).stop 2024-09-16T15:46:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.142+0000 7ffa55200640 1 -- 172.21.15.60:0/1211130849 shutdown_connections 2024-09-16T15:46:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.142+0000 7ffa55200640 1 --2- 172.21.15.60:0/1211130849 >> v2:172.21.15.100:3300/0 conn(0x7ffa5006efa0 0x7ffa500cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.142+0000 7ffa55200640 1 --2- 172.21.15.60:0/1211130849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa500cccd0 0x7ffa50073440 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.142+0000 7ffa55200640 1 -- 172.21.15.60:0/1211130849 >> 172.21.15.60:0/1211130849 conn(0x7ffa500b1840 msgr2=0x7ffa500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.142+0000 7ffa55200640 1 -- 172.21.15.60:0/1211130849 shutdown_connections 2024-09-16T15:46:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.142+0000 7ffa55200640 1 -- 172.21.15.60:0/1211130849 wait complete. 2024-09-16T15:46:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.143+0000 7ffa55200640 1 Processor -- start 2024-09-16T15:46:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.143+0000 7ffa55200640 1 -- start start 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.144+0000 7ffa55200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa5006efa0 0x7ffa501413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.144+0000 7ffa55200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa500cccd0 0x7ffa500aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.144+0000 7ffa55200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa50145750 0x7ffa5015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.144+0000 7ffa55200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffa500ce480 con 0x7ffa5006efa0 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.144+0000 7ffa55200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffa500ce300 con 0x7ffa50145750 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.144+0000 7ffa55200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffa500ce600 con 0x7ffa500cccd0 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa5006efa0 0x7ffa501413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa5006efa0 0x7ffa501413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46034/0 (socket says 172.21.15.60:46034) 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 -- 172.21.15.60:0/1719092069 learned_addr learned my addr 172.21.15.60:0/1719092069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa45400640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa500cccd0 0x7ffa500aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4e800640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa50145750 0x7ffa5015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 -- 172.21.15.60:0/1719092069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa500cccd0 msgr2=0x7ffa500aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa500cccd0 0x7ffa500aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 -- 172.21.15.60:0/1719092069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa50145750 msgr2=0x7ffa5015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa50145750 0x7ffa5015b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 -- 172.21.15.60:0/1719092069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa500c4ab0 con 0x7ffa5006efa0 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4e800640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa50145750 0x7ffa5015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa45400640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa500cccd0 0x7ffa500aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.145+0000 7ffa4de00640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa5006efa0 0x7ffa501413b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000c3a0 tx=0x7ffa4000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.146+0000 7ffa46a00640 1 -- 172.21.15.60:0/1719092069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40017070 con 0x7ffa5006efa0 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.146+0000 7ffa46a00640 1 -- 172.21.15.60:0/1719092069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa400053c0 con 0x7ffa5006efa0 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.146+0000 7ffa46a00640 1 -- 172.21.15.60:0/1719092069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40003850 con 0x7ffa5006efa0 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.146+0000 7ffa55200640 1 -- 172.21.15.60:0/1719092069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa501f8020 con 0x7ffa5006efa0 2024-09-16T15:46:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.146+0000 7ffa55200640 1 -- 172.21.15.60:0/1719092069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501f85e0 con 0x7ffa5006efa0 2024-09-16T15:46:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.147+0000 7ffa55200640 1 -- 172.21.15.60:0/1719092069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa500cdbb0 con 0x7ffa5006efa0 2024-09-16T15:46:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.148+0000 7ffa46a00640 1 -- 172.21.15.60:0/1719092069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7ffa4001a070 con 0x7ffa5006efa0 2024-09-16T15:46:22.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.149+0000 7ffa46a00640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa2c078e80 0x7ffa2c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.149+0000 7ffa46a00640 1 -- 172.21.15.60:0/1719092069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7ffa4009a8f0 con 0x7ffa5006efa0 2024-09-16T15:46:22.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.149+0000 7ffa45400640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa2c078e80 0x7ffa2c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.150+0000 7ffa45400640 1 --2- 172.21.15.60:0/1719092069 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa2c078e80 0x7ffa2c07b340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa38033000 tx=0x7ffa38004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.153+0000 7ffa46a00640 1 -- 172.21.15.60:0/1719092069 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffa40062540 con 0x7ffa5006efa0 2024-09-16T15:46:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.375+0000 7ffa55200640 1 -- 172.21.15.60:0/1719092069 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7ffa500cca60 con 0x7ffa2c078e80 2024-09-16T15:46:22.378 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:22.376+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.377+0000 7ffa46a00640 1 -- 172.21.15.60:0/1719092069 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ffa500cca60 con 0x7ffa2c078e80 2024-09-16T15:46:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:22.389 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:22.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-16T15:46:22.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.551+0000 7fe19b200640 1 -- 172.21.15.60:0/1145252061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c15df50 msgr2=0x7fe19c0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.551+0000 7fe19b200640 1 --2- 172.21.15.60:0/1145252061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c15df50 0x7fe19c0ce930 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0d27b0 tx=0x7fe18c02a5b0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.551+0000 7fe19b200640 1 -- 172.21.15.60:0/1145252061 shutdown_connections 2024-09-16T15:46:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.551+0000 7fe19b200640 1 --2- 172.21.15.60:0/1145252061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19c0cf580 0x7fe19c0c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.551+0000 7fe19b200640 1 --2- 172.21.15.60:0/1145252061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c15df50 0x7fe19c0ce930 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.551+0000 7fe19b200640 1 -- 172.21.15.60:0/1145252061 >> 172.21.15.60:0/1145252061 conn(0x7fe19c0d3e60 msgr2=0x7fe19c0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.551+0000 7fe19b200640 1 -- 172.21.15.60:0/1145252061 shutdown_connections 2024-09-16T15:46:22.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.551+0000 7fe19b200640 1 -- 172.21.15.60:0/1145252061 wait complete. 2024-09-16T15:46:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.553+0000 7fe19b200640 1 Processor -- start 2024-09-16T15:46:22.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.553+0000 7fe19b200640 1 -- start start 2024-09-16T15:46:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe19b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19c0cf580 0x7fe19c0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe19b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c0d4980 0x7fe19c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe19b200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19c15df50 0x7fe19c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe19b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c0c86a0 con 0x7fe19c0d4980 2024-09-16T15:46:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe19b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c0c8520 con 0x7fe19c0cf580 2024-09-16T15:46:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe19b200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c0c8820 con 0x7fe19c15df50 2024-09-16T15:46:22.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe191400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c0d4980 0x7fe19c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe191400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c0d4980 0x7fe19c15b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46052/0 (socket says 172.21.15.60:46052) 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe191400640 1 -- 172.21.15.60:0/1698284884 learned_addr learned my addr 172.21.15.60:0/1698284884 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe199e00640 1 --2- 172.21.15.60:0/1698284884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19c0cf580 0x7fe19c0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe19a800640 1 --2- 172.21.15.60:0/1698284884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19c15df50 0x7fe19c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe191400640 1 -- 172.21.15.60:0/1698284884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19c15df50 msgr2=0x7fe19c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe191400640 1 --2- 172.21.15.60:0/1698284884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe19c15df50 0x7fe19c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe191400640 1 -- 172.21.15.60:0/1698284884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19c0cf580 msgr2=0x7fe19c0cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe191400640 1 --2- 172.21.15.60:0/1698284884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe19c0cf580 0x7fe19c0cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.554+0000 7fe191400640 1 -- 172.21.15.60:0/1698284884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c0cb9e0 con 0x7fe19c0d4980 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.555+0000 7fe191400640 1 --2- 172.21.15.60:0/1698284884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe19c0d4980 0x7fe19c15b0f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe18400c450 tx=0x7fe18400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.555+0000 7fe192a00640 1 -- 172.21.15.60:0/1698284884 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184016020 con 0x7fe19c0d4980 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.555+0000 7fe192a00640 1 -- 172.21.15.60:0/1698284884 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe184005400 con 0x7fe19c0d4980 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.555+0000 7fe19b200640 1 -- 172.21.15.60:0/1698284884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c0cbc80 con 0x7fe19c0d4980 2024-09-16T15:46:22.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.555+0000 7fe192a00640 1 -- 172.21.15.60:0/1698284884 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184007280 con 0x7fe19c0d4980 2024-09-16T15:46:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.555+0000 7fe19b200640 1 -- 172.21.15.60:0/1698284884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c0cc240 con 0x7fe19c0d4980 2024-09-16T15:46:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.556+0000 7fe19b200640 1 -- 172.21.15.60:0/1698284884 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c0c7dd0 con 0x7fe19c0d4980 2024-09-16T15:46:22.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.557+0000 7fe192a00640 1 -- 172.21.15.60:0/1698284884 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fe184007420 con 0x7fe19c0d4980 2024-09-16T15:46:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.558+0000 7fe192a00640 1 --2- 172.21.15.60:0/1698284884 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe178078e80 0x7fe17807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.558+0000 7fe192a00640 1 -- 172.21.15.60:0/1698284884 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fe18409ade0 con 0x7fe19c0d4980 2024-09-16T15:46:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.559+0000 7fe199e00640 1 --2- 172.21.15.60:0/1698284884 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe178078e80 0x7fe17807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.559+0000 7fe199e00640 1 --2- 172.21.15.60:0/1698284884 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe178078e80 0x7fe17807b340 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe18c033000 tx=0x7fe18c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.563+0000 7fe192a00640 1 -- 172.21.15.60:0/1698284884 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe1840a1080 con 0x7fe19c0d4980 2024-09-16T15:46:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.786+0000 7fe19b200640 1 -- 172.21.15.60:0/1698284884 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fe19c0d1160 con 0x7fe178078e80 2024-09-16T15:46:22.789 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:22.787+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:22.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.788+0000 7fe192a00640 1 -- 172.21.15.60:0/1698284884 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe19c0d1160 con 0x7fe178078e80 2024-09-16T15:46:22.790 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:22.800 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:22.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-16T15:46:22.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.961+0000 7f5547a00640 1 -- 172.21.15.60:0/1963086258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55400afcb0 msgr2=0x7f554015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.961+0000 7f5547a00640 1 --2- 172.21.15.60:0/1963086258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55400afcb0 0x7f554015cde0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5528005fd0 tx=0x7f552802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.962+0000 7f5547a00640 1 -- 172.21.15.60:0/1963086258 shutdown_connections 2024-09-16T15:46:22.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.962+0000 7f5547a00640 1 --2- 172.21.15.60:0/1963086258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55400afcb0 0x7f554015cde0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.962+0000 7f5547a00640 1 --2- 172.21.15.60:0/1963086258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5540073740 0x7f55400cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.962+0000 7f5547a00640 1 -- 172.21.15.60:0/1963086258 >> 172.21.15.60:0/1963086258 conn(0x7f55400b2130 msgr2=0x7f55400b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:22.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.962+0000 7f5547a00640 1 -- 172.21.15.60:0/1963086258 shutdown_connections 2024-09-16T15:46:22.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.962+0000 7f5547a00640 1 -- 172.21.15.60:0/1963086258 wait complete. 2024-09-16T15:46:22.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.963+0000 7f5547a00640 1 Processor -- start 2024-09-16T15:46:22.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.963+0000 7f5547a00640 1 -- start start 2024-09-16T15:46:22.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.964+0000 7f5547a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5540073740 0x7f55400b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.964+0000 7f5547a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5540152e30 0x7f55400adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.964+0000 7f5547a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401f46b0 0x7f55401f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.964+0000 7f5547a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f554006d270 con 0x7f5540073740 2024-09-16T15:46:22.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.964+0000 7f5547a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f554006d0f0 con 0x7f5540152e30 2024-09-16T15:46:22.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.964+0000 7f5547a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f554006d3f0 con 0x7f55401f46b0 2024-09-16T15:46:22.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5540073740 0x7f55400b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5540073740 0x7f55400b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46072/0 (socket says 172.21.15.60:46072) 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 -- 172.21.15.60:0/442201582 learned_addr learned my addr 172.21.15.60:0/442201582 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5537e00640 1 --2- 172.21.15.60:0/442201582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5540152e30 0x7f55400adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545a00640 1 --2- 172.21.15.60:0/442201582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401f46b0 0x7f55401f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 -- 172.21.15.60:0/442201582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401f46b0 msgr2=0x7f55401f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 --2- 172.21.15.60:0/442201582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401f46b0 0x7f55401f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 -- 172.21.15.60:0/442201582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5540152e30 msgr2=0x7f55400adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 --2- 172.21.15.60:0/442201582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5540152e30 0x7f55400adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 -- 172.21.15.60:0/442201582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5540156000 con 0x7f5540073740 2024-09-16T15:46:22.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.965+0000 7f5545000640 1 --2- 172.21.15.60:0/442201582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5540073740 0x7f55400b3fd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f553000c3d0 tx=0x7f553000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.966+0000 7f553d600640 1 -- 172.21.15.60:0/442201582 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530017070 con 0x7f5540073740 2024-09-16T15:46:22.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.966+0000 7f553d600640 1 -- 172.21.15.60:0/442201582 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55300053c0 con 0x7f5540073740 2024-09-16T15:46:22.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.966+0000 7f553d600640 1 -- 172.21.15.60:0/442201582 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530003850 con 0x7f5540073740 2024-09-16T15:46:22.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.966+0000 7f5547a00640 1 -- 172.21.15.60:0/442201582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55400ae320 con 0x7f5540073740 2024-09-16T15:46:22.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.966+0000 7f5547a00640 1 -- 172.21.15.60:0/442201582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55400ae590 con 0x7f5540073740 2024-09-16T15:46:22.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.967+0000 7f5547a00640 1 -- 172.21.15.60:0/442201582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554006d910 con 0x7f5540073740 2024-09-16T15:46:22.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.968+0000 7f553d600640 1 -- 172.21.15.60:0/442201582 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f553001a070 con 0x7f5540073740 2024-09-16T15:46:22.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.969+0000 7f553d600640 1 --2- 172.21.15.60:0/442201582 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f551c078e80 0x7f551c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.969+0000 7f553d600640 1 -- 172.21.15.60:0/442201582 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f553009a8c0 con 0x7f5540073740 2024-09-16T15:46:22.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.969+0000 7f5537e00640 1 --2- 172.21.15.60:0/442201582 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f551c078e80 0x7f551c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.970+0000 7f5537e00640 1 --2- 172.21.15.60:0/442201582 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f551c078e80 0x7f551c07b340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5528033000 tx=0x7f5528004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:22.973+0000 7f553d600640 1 -- 172.21.15.60:0/442201582 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5530061e30 con 0x7f5540073740 2024-09-16T15:46:23.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.200+0000 7f5547a00640 1 -- 172.21.15.60:0/442201582 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f5540073e40 con 0x7f551c078e80 2024-09-16T15:46:23.203 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:23.201+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:23.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.201+0000 7f553d600640 1 -- 172.21.15.60:0/442201582 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f5540073e40 con 0x7f551c078e80 2024-09-16T15:46:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:23.213 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:23.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-16T15:46:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.377+0000 7f8749200640 1 -- 172.21.15.60:0/2228096945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874415cff0 msgr2=0x7f874406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.377+0000 7f8749200640 1 --2- 172.21.15.60:0/2228096945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874415cff0 0x7f874406e6f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8744075110 tx=0x7f872c02a560 comp rx=0 tx=0).stop 2024-09-16T15:46:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.377+0000 7f8749200640 1 -- 172.21.15.60:0/2228096945 shutdown_connections 2024-09-16T15:46:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.377+0000 7f8749200640 1 --2- 172.21.15.60:0/2228096945 >> v2:172.21.15.100:3300/0 conn(0x7f874406f220 0x7f874406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.377+0000 7f8749200640 1 --2- 172.21.15.60:0/2228096945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874415cff0 0x7f874406e6f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.377+0000 7f8749200640 1 -- 172.21.15.60:0/2228096945 >> 172.21.15.60:0/2228096945 conn(0x7f87440b18e0 msgr2=0x7f87440b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.377+0000 7f8749200640 1 -- 172.21.15.60:0/2228096945 shutdown_connections 2024-09-16T15:46:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.377+0000 7f8749200640 1 -- 172.21.15.60:0/2228096945 wait complete. 2024-09-16T15:46:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.378+0000 7f8749200640 1 Processor -- start 2024-09-16T15:46:23.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.379+0000 7f8749200640 1 -- start start 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.379+0000 7f8749200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f874406f220 0x7f874409c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.379+0000 7f8749200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8744159aa0 0x7f8744144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.379+0000 7f8749200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874415cff0 0x7f87441f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.379+0000 7f8749200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f87440cc520 con 0x7f874415cff0 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.379+0000 7f8749200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87440cc3a0 con 0x7f8744159aa0 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.379+0000 7f8749200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f87440cc6a0 con 0x7f874406f220 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874415cff0 0x7f87441f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874415cff0 0x7f87441f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46088/0 (socket says 172.21.15.60:46088) 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 -- 172.21.15.60:0/971609671 learned_addr learned my addr 172.21.15.60:0/971609671 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8739400640 1 --2- 172.21.15.60:0/971609671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8744159aa0 0x7f8744144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8741e00640 1 --2- 172.21.15.60:0/971609671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f874406f220 0x7f874409c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 -- 172.21.15.60:0/971609671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f874406f220 msgr2=0x7f874409c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 --2- 172.21.15.60:0/971609671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f874406f220 0x7f874409c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 -- 172.21.15.60:0/971609671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8744159aa0 msgr2=0x7f8744144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 --2- 172.21.15.60:0/971609671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8744159aa0 0x7f8744144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 -- 172.21.15.60:0/971609671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87441f7140 con 0x7f874415cff0 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.380+0000 7f8742800640 1 --2- 172.21.15.60:0/971609671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f874415cff0 0x7f87441f6c00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f873c00ead0 tx=0x7f873c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.381+0000 7f873aa00640 1 -- 172.21.15.60:0/971609671 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c00cde0 con 0x7f874415cff0 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.381+0000 7f873aa00640 1 -- 172.21.15.60:0/971609671 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f873c004c10 con 0x7f874415cff0 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.381+0000 7f873aa00640 1 -- 172.21.15.60:0/971609671 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c005020 con 0x7f874415cff0 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.381+0000 7f8749200640 1 -- 172.21.15.60:0/971609671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874409cf20 con 0x7f874415cff0 2024-09-16T15:46:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.381+0000 7f8749200640 1 -- 172.21.15.60:0/971609671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874409d4e0 con 0x7f874415cff0 2024-09-16T15:46:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.382+0000 7f8749200640 1 -- 172.21.15.60:0/971609671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8744069360 con 0x7f874415cff0 2024-09-16T15:46:23.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.383+0000 7f873aa00640 1 -- 172.21.15.60:0/971609671 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f873c0051c0 con 0x7f874415cff0 2024-09-16T15:46:23.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.384+0000 7f873aa00640 1 --2- 172.21.15.60:0/971609671 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8720078e80 0x7f872007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.385+0000 7f873aa00640 1 -- 172.21.15.60:0/971609671 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f873c09bef0 con 0x7f874415cff0 2024-09-16T15:46:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.385+0000 7f8741e00640 1 --2- 172.21.15.60:0/971609671 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8720078e80 0x7f872007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.385+0000 7f8741e00640 1 --2- 172.21.15.60:0/971609671 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8720078e80 0x7f872007b340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8734004ea0 tx=0x7f8734009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:23.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.389+0000 7f873aa00640 1 -- 172.21.15.60:0/971609671 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f87441f7140 con 0x7f874415cff0 2024-09-16T15:46:23.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.623+0000 7f8749200640 1 -- 172.21.15.60:0/971609671 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7f87440ccb50 con 0x7f8720078e80 2024-09-16T15:46:23.627 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:23.625+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:23.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.625+0000 7f873aa00640 1 -- 172.21.15.60:0/971609671 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f87440ccb50 con 0x7f8720078e80 2024-09-16T15:46:23.627 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:23.637 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:23.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-16T15:46:23.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.805+0000 7fccd9800640 1 -- 172.21.15.60:0/1680533260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd415b700 msgr2=0x7fccd40ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:23.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.805+0000 7fccd9800640 1 --2- 172.21.15.60:0/1680533260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd415b700 0x7fccd40ccca0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fccbc005fd0 tx=0x7fccbc02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.805+0000 7fccd9800640 1 -- 172.21.15.60:0/1680533260 shutdown_connections 2024-09-16T15:46:23.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.805+0000 7fccd9800640 1 --2- 172.21.15.60:0/1680533260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd415b700 0x7fccd40ccca0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.805+0000 7fccd9800640 1 --2- 172.21.15.60:0/1680533260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd4158900 0x7fccd415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.805+0000 7fccd9800640 1 -- 172.21.15.60:0/1680533260 >> 172.21.15.60:0/1680533260 conn(0x7fccd40b18e0 msgr2=0x7fccd40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:23.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.805+0000 7fccd9800640 1 -- 172.21.15.60:0/1680533260 shutdown_connections 2024-09-16T15:46:23.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.805+0000 7fccd9800640 1 -- 172.21.15.60:0/1680533260 wait complete. 2024-09-16T15:46:23.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.807+0000 7fccd9800640 1 Processor -- start 2024-09-16T15:46:23.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.807+0000 7fccd9800640 1 -- start start 2024-09-16T15:46:23.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40cd1e0 0x7fccd40cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:23.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd9800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4158900 0x7fccd40ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd9800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415b700 0x7fccd41f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd9800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fccd40ced60 con 0x7fccd40cd1e0 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd9800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fccd40cebe0 con 0x7fccd4158900 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd9800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fccd40ceee0 con 0x7fccd415b700 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40cd1e0 0x7fccd40cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40cd1e0 0x7fccd40cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46118/0 (socket says 172.21.15.60:46118) 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd2a00640 1 -- 172.21.15.60:0/3674698839 learned_addr learned my addr 172.21.15.60:0/3674698839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd3400640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415b700 0x7fccd41f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccca000640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4158900 0x7fccd40ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd2a00640 1 -- 172.21.15.60:0/3674698839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415b700 msgr2=0x7fccd41f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd2a00640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415b700 0x7fccd41f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd2a00640 1 -- 172.21.15.60:0/3674698839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4158900 msgr2=0x7fccd40ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd2a00640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4158900 0x7fccd40ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccd2a00640 1 -- 172.21.15.60:0/3674698839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd40addf0 con 0x7fccd40cd1e0 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.808+0000 7fccca000640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd4158900 0x7fccd40ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.809+0000 7fccd2a00640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40cd1e0 0x7fccd40cb8f0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fccc400eab0 tx=0x7fccc400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.809+0000 7fcccb400640 1 -- 172.21.15.60:0/3674698839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc400ce40 con 0x7fccd40cd1e0 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.809+0000 7fcccb400640 1 -- 172.21.15.60:0/3674698839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccc4011560 con 0x7fccd40cd1e0 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.809+0000 7fccd9800640 1 -- 172.21.15.60:0/3674698839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd40cbe30 con 0x7fccd40cd1e0 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.809+0000 7fcccb400640 1 -- 172.21.15.60:0/3674698839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4011930 con 0x7fccd40cd1e0 2024-09-16T15:46:23.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.809+0000 7fccd9800640 1 -- 172.21.15.60:0/3674698839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd41f9300 con 0x7fccd40cd1e0 2024-09-16T15:46:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.810+0000 7fccd9800640 1 -- 172.21.15.60:0/3674698839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd4069360 con 0x7fccd40cd1e0 2024-09-16T15:46:23.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.811+0000 7fcccb400640 1 -- 172.21.15.60:0/3674698839 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fccc4011ad0 con 0x7fccd40cd1e0 2024-09-16T15:46:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.812+0000 7fcccb400640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fccb0078e80 0x7fccb007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.812+0000 7fcccb400640 1 -- 172.21.15.60:0/3674698839 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fccc409c310 con 0x7fccd40cd1e0 2024-09-16T15:46:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.813+0000 7fccca000640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fccb0078e80 0x7fccb007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:23.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.813+0000 7fccca000640 1 --2- 172.21.15.60:0/3674698839 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fccb0078e80 0x7fccb007b340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fccbc033000 tx=0x7fccbc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:23.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:23.817+0000 7fcccb400640 1 -- 172.21.15.60:0/3674698839 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fccd40addf0 con 0x7fccd40cd1e0 2024-09-16T15:46:24.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.049+0000 7fccd9800640 1 -- 172.21.15.60:0/3674698839 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7fccd40cf3e0 con 0x7fccb0078e80 2024-09-16T15:46:24.053 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:24.051+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:24.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.051+0000 7fcccb400640 1 -- 172.21.15.60:0/3674698839 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fccd40cf3e0 con 0x7fccb0078e80 2024-09-16T15:46:24.053 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:24.062 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:24.063 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-16T15:46:24.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.224+0000 7fdd68200640 1 -- 172.21.15.60:0/3086931310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd48005210 msgr2=0x7fdd4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:24.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.224+0000 7fdd68200640 1 --2- 172.21.15.60:0/3086931310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd48005210 0x7fdd4800b9c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdd58005fd0 tx=0x7fdd48005830 comp rx=0 tx=0).stop 2024-09-16T15:46:24.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.224+0000 7fdd68200640 1 -- 172.21.15.60:0/3086931310 shutdown_connections 2024-09-16T15:46:24.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.224+0000 7fdd68200640 1 --2- 172.21.15.60:0/3086931310 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd48005210 0x7fdd4800b9c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:24.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.224+0000 7fdd68200640 1 -- 172.21.15.60:0/3086931310 >> 172.21.15.60:0/3086931310 conn(0x7fdd600b1840 msgr2=0x7fdd600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.225+0000 7fdd68200640 1 -- 172.21.15.60:0/3086931310 shutdown_connections 2024-09-16T15:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.225+0000 7fdd68200640 1 -- 172.21.15.60:0/3086931310 wait complete. 2024-09-16T15:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.226+0000 7fdd68200640 1 Processor -- start 2024-09-16T15:46:24.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.226+0000 7fdd68200640 1 -- start start 2024-09-16T15:46:24.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd68200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd48005210 0x7fdd6015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:24.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd68200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd6015dc20 0x7fdd6014e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:24.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd68200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd601f85c0 0x7fdd601f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:24.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd68200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdd600cf1f0 con 0x7fdd48005210 2024-09-16T15:46:24.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd68200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd600cf070 con 0x7fdd601f85c0 2024-09-16T15:46:24.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd68200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdd600cf370 con 0x7fdd6015dc20 2024-09-16T15:46:24.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd65800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd48005210 0x7fdd6015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd65800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd48005210 0x7fdd6015e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46144/0 (socket says 172.21.15.60:46144) 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.227+0000 7fdd65800640 1 -- 172.21.15.60:0/3763263589 learned_addr learned my addr 172.21.15.60:0/3763263589 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd64e00640 1 --2- 172.21.15.60:0/3763263589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd6015dc20 0x7fdd6014e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd66200640 1 --2- 172.21.15.60:0/3763263589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd601f85c0 0x7fdd601f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd65800640 1 -- 172.21.15.60:0/3763263589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd6015dc20 msgr2=0x7fdd6014e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd65800640 1 --2- 172.21.15.60:0/3763263589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd6015dc20 0x7fdd6014e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd65800640 1 -- 172.21.15.60:0/3763263589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd601f85c0 msgr2=0x7fdd601f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd65800640 1 --2- 172.21.15.60:0/3763263589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd601f85c0 0x7fdd601f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd65800640 1 -- 172.21.15.60:0/3763263589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6015d010 con 0x7fdd48005210 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd65800640 1 --2- 172.21.15.60:0/3763263589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd48005210 0x7fdd6015e5e0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdd540066b0 tx=0x7fdd540039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:24.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd5d600640 1 -- 172.21.15.60:0/3763263589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54015070 con 0x7fdd48005210 2024-09-16T15:46:24.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd5d600640 1 -- 172.21.15.60:0/3763263589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd54011900 con 0x7fdd48005210 2024-09-16T15:46:24.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd68200640 1 -- 172.21.15.60:0/3763263589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6015d2f0 con 0x7fdd48005210 2024-09-16T15:46:24.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.228+0000 7fdd5d600640 1 -- 172.21.15.60:0/3763263589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54011d50 con 0x7fdd48005210 2024-09-16T15:46:24.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.229+0000 7fdd68200640 1 -- 172.21.15.60:0/3763263589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd60160f10 con 0x7fdd48005210 2024-09-16T15:46:24.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.230+0000 7fdd68200640 1 -- 172.21.15.60:0/3763263589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd600b2ec0 con 0x7fdd48005210 2024-09-16T15:46:24.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.231+0000 7fdd5d600640 1 -- 172.21.15.60:0/3763263589 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fdd54018070 con 0x7fdd48005210 2024-09-16T15:46:24.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.232+0000 7fdd5d600640 1 --2- 172.21.15.60:0/3763263589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdd40078e80 0x7fdd4007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:24.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.232+0000 7fdd5d600640 1 -- 172.21.15.60:0/3763263589 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fdd5409bd90 con 0x7fdd48005210 2024-09-16T15:46:24.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.232+0000 7fdd64e00640 1 --2- 172.21.15.60:0/3763263589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdd40078e80 0x7fdd4007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:24.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.233+0000 7fdd64e00640 1 --2- 172.21.15.60:0/3763263589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdd40078e80 0x7fdd4007b340 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000c00 tx=0x7fdd4c006b70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:24.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.236+0000 7fdd5d600640 1 -- 172.21.15.60:0/3763263589 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdd600cf5b0 con 0x7fdd48005210 2024-09-16T15:46:24.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.456+0000 7fdd68200640 1 -- 172.21.15.60:0/3763263589 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7fdd6015ec70 con 0x7fdd40078e80 2024-09-16T15:46:24.459 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:24.457+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:24.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.457+0000 7fdd5d600640 1 -- 172.21.15.60:0/3763263589 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fdd6015ec70 con 0x7fdd40078e80 2024-09-16T15:46:24.460 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:24.469 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:24.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-16T15:46:24.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.634+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3419203441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a881620e0 msgr2=0x7f6a880b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:24.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.634+0000 7f6a8ea00640 1 --2- 172.21.15.60:0/3419203441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a881620e0 0x7f6a880b0c30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009860 tx=0x7f6a7802a540 comp rx=0 tx=0).stop 2024-09-16T15:46:24.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.634+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3419203441 shutdown_connections 2024-09-16T15:46:24.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.634+0000 7f6a8ea00640 1 --2- 172.21.15.60:0/3419203441 >> v2:172.21.15.100:3300/0 conn(0x7f6a881624c0 0x7f6a88161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:24.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.634+0000 7f6a8ea00640 1 --2- 172.21.15.60:0/3419203441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a881620e0 0x7f6a880b0c30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:24.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.634+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3419203441 >> 172.21.15.60:0/3419203441 conn(0x7f6a880d1e40 msgr2=0x7f6a880d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:24.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.634+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3419203441 shutdown_connections 2024-09-16T15:46:24.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.634+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3419203441 wait complete. 2024-09-16T15:46:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.636+0000 7f6a8ea00640 1 Processor -- start 2024-09-16T15:46:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.636+0000 7f6a8ea00640 1 -- start start 2024-09-16T15:46:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880d1a00 0x7f6a88071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a881620e0 0x7f6a881f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a881624c0 0x7f6a88159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a8ea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880af260 con 0x7f6a881620e0 2024-09-16T15:46:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a8ea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880af0e0 con 0x7f6a881624c0 2024-09-16T15:46:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a8ea00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6a880af3e0 con 0x7f6a880d1a00 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a7f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a881620e0 0x7f6a881f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a7f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a881620e0 0x7f6a881f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46160/0 (socket says 172.21.15.60:46160) 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a7f400640 1 -- 172.21.15.60:0/3575023394 learned_addr learned my addr 172.21.15.60:0/3575023394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a87e00640 1 --2- 172.21.15.60:0/3575023394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880d1a00 0x7f6a88071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.637+0000 7f6a8ca00640 1 --2- 172.21.15.60:0/3575023394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a881624c0 0x7f6a88159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a7f400640 1 -- 172.21.15.60:0/3575023394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880d1a00 msgr2=0x7f6a88071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a7f400640 1 --2- 172.21.15.60:0/3575023394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880d1a00 0x7f6a88071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a7f400640 1 -- 172.21.15.60:0/3575023394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a881624c0 msgr2=0x7f6a88159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a7f400640 1 --2- 172.21.15.60:0/3575023394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a881624c0 0x7f6a88159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a7f400640 1 -- 172.21.15.60:0/3575023394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8815acd0 con 0x7f6a881620e0 2024-09-16T15:46:24.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a7f400640 1 --2- 172.21.15.60:0/3575023394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a881620e0 0x7f6a881f84f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000c420 tx=0x7f6a7000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:24.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a84c00640 1 -- 172.21.15.60:0/3575023394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011790 con 0x7f6a881620e0 2024-09-16T15:46:24.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a84c00640 1 -- 172.21.15.60:0/3575023394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a70017070 con 0x7f6a881620e0 2024-09-16T15:46:24.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a84c00640 1 -- 172.21.15.60:0/3575023394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011c80 con 0x7f6a881620e0 2024-09-16T15:46:24.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3575023394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8815b010 con 0x7f6a881620e0 2024-09-16T15:46:24.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.638+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3575023394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8815b850 con 0x7f6a881620e0 2024-09-16T15:46:24.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.639+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3575023394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8815ba50 con 0x7f6a881620e0 2024-09-16T15:46:24.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.641+0000 7f6a84c00640 1 -- 172.21.15.60:0/3575023394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f6a7001c460 con 0x7f6a881620e0 2024-09-16T15:46:24.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.642+0000 7f6a84c00640 1 --2- 172.21.15.60:0/3575023394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a64078e80 0x7f6a6407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.642+0000 7f6a84c00640 1 -- 172.21.15.60:0/3575023394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f6a7009b310 con 0x7f6a881620e0 2024-09-16T15:46:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.642+0000 7f6a87e00640 1 --2- 172.21.15.60:0/3575023394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a64078e80 0x7f6a6407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.643+0000 7f6a87e00640 1 --2- 172.21.15.60:0/3575023394 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a64078e80 0x7f6a6407b340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6a78034000 tx=0x7f6a78004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:24.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.646+0000 7f6a84c00640 1 -- 172.21.15.60:0/3575023394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6a8815ba50 con 0x7f6a881620e0 2024-09-16T15:46:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.877+0000 7f6a8ea00640 1 -- 172.21.15.60:0/3575023394 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f6a880b0c30 con 0x7f6a64078e80 2024-09-16T15:46:24.881 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:24.879+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:24.879+0000 7f6a84c00640 1 -- 172.21.15.60:0/3575023394 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f6a880b0c30 con 0x7f6a64078e80 2024-09-16T15:46:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:24.890 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:24.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-16T15:46:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.055+0000 7fd261c00640 1 -- 172.21.15.60:0/3411453939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd25c15b9b0 msgr2=0x7fd25c15ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.055+0000 7fd261c00640 1 --2- 172.21.15.60:0/3411453939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd25c15b9b0 0x7fd25c15ee00 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd24c005fd0 tx=0x7fd24c02a940 comp rx=0 tx=0).stop 2024-09-16T15:46:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.055+0000 7fd261c00640 1 -- 172.21.15.60:0/3411453939 shutdown_connections 2024-09-16T15:46:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.055+0000 7fd261c00640 1 --2- 172.21.15.60:0/3411453939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ce7f0 0x7fd25c0ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.055+0000 7fd261c00640 1 --2- 172.21.15.60:0/3411453939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd25c15b9b0 0x7fd25c15ee00 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.055+0000 7fd261c00640 1 -- 172.21.15.60:0/3411453939 >> 172.21.15.60:0/3411453939 conn(0x7fd25c0cfa70 msgr2=0x7fd25c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.055+0000 7fd261c00640 1 -- 172.21.15.60:0/3411453939 shutdown_connections 2024-09-16T15:46:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.055+0000 7fd261c00640 1 -- 172.21.15.60:0/3411453939 wait complete. 2024-09-16T15:46:25.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.057+0000 7fd261c00640 1 Processor -- start 2024-09-16T15:46:25.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.057+0000 7fd261c00640 1 -- start start 2024-09-16T15:46:25.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd261c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ce7f0 0x7fd25c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd261c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd25c0cf180 0x7fd25c15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd261c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c15b9b0 0x7fd25c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd261c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c0caa40 con 0x7fd25c0cf180 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd261c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c0ca8c0 con 0x7fd25c15b9b0 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd261c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd25c0cabc0 con 0x7fd25c0ce7f0 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd25b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd25c0cf180 0x7fd25c15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd25b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd25c0cf180 0x7fd25c15d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46172/0 (socket says 172.21.15.60:46172) 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd25b400640 1 -- 172.21.15.60:0/949988732 learned_addr learned my addr 172.21.15.60:0/949988732 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd25be00640 1 --2- 172.21.15.60:0/949988732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ce7f0 0x7fd25c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.058+0000 7fd261200640 1 --2- 172.21.15.60:0/949988732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c15b9b0 0x7fd25c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd25b400640 1 -- 172.21.15.60:0/949988732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ce7f0 msgr2=0x7fd25c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd25b400640 1 --2- 172.21.15.60:0/949988732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd25c0ce7f0 0x7fd25c06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd25b400640 1 -- 172.21.15.60:0/949988732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c15b9b0 msgr2=0x7fd25c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd25b400640 1 --2- 172.21.15.60:0/949988732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd25c15b9b0 0x7fd25c151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd25b400640 1 -- 172.21.15.60:0/949988732 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c06fb20 con 0x7fd25c0cf180 2024-09-16T15:46:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd25b400640 1 --2- 172.21.15.60:0/949988732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd25c0cf180 0x7fd25c15d9b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd24800e9f0 tx=0x7fd24800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd253e00640 1 -- 172.21.15.60:0/949988732 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24800cd80 con 0x7fd25c0cf180 2024-09-16T15:46:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd253e00640 1 -- 172.21.15.60:0/949988732 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd248010430 con 0x7fd25c0cf180 2024-09-16T15:46:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd253e00640 1 -- 172.21.15.60:0/949988732 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248010840 con 0x7fd25c0cf180 2024-09-16T15:46:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd261c00640 1 -- 172.21.15.60:0/949988732 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25c06fdc0 con 0x7fd25c0cf180 2024-09-16T15:46:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.059+0000 7fd261c00640 1 -- 172.21.15.60:0/949988732 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25c070380 con 0x7fd25c0cf180 2024-09-16T15:46:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.061+0000 7fd261c00640 1 -- 172.21.15.60:0/949988732 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25c0ca380 con 0x7fd25c0cf180 2024-09-16T15:46:25.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.062+0000 7fd253e00640 1 -- 172.21.15.60:0/949988732 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fd2480109e0 con 0x7fd25c0cf180 2024-09-16T15:46:25.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.062+0000 7fd253e00640 1 --2- 172.21.15.60:0/949988732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd23c078ed0 0x7fd23c07b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.063+0000 7fd253e00640 1 -- 172.21.15.60:0/949988732 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fd24809bf10 con 0x7fd25c0cf180 2024-09-16T15:46:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.063+0000 7fd25be00640 1 --2- 172.21.15.60:0/949988732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd23c078ed0 0x7fd23c07b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.063+0000 7fd25be00640 1 --2- 172.21.15.60:0/949988732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd23c078ed0 0x7fd23c07b390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd24c034000 tx=0x7fd24c004ab0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:25.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.067+0000 7fd253e00640 1 -- 172.21.15.60:0/949988732 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd248063710 con 0x7fd25c0cf180 2024-09-16T15:46:25.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.296+0000 7fd261c00640 1 -- 172.21.15.60:0/949988732 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fd25c1f55d0 con 0x7fd23c078ed0 2024-09-16T15:46:25.300 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:25.298+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:25.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.298+0000 7fd253e00640 1 -- 172.21.15.60:0/949988732 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd25c1f55d0 con 0x7fd23c078ed0 2024-09-16T15:46:25.300 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:25.310 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:25.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-16T15:46:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.474+0000 7f6041400640 1 -- 172.21.15.60:0/1278771233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603c073de0 msgr2=0x7f603c074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.474+0000 7f6041400640 1 --2- 172.21.15.60:0/1278771233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603c073de0 0x7f603c074240 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f6028005fd0 tx=0x7f602802a4f0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.474+0000 7f6041400640 1 -- 172.21.15.60:0/1278771233 shutdown_connections 2024-09-16T15:46:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.474+0000 7f6041400640 1 --2- 172.21.15.60:0/1278771233 >> v2:172.21.15.100:3300/0 conn(0x7f603c06e6f0 0x7f603c06eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.474+0000 7f6041400640 1 --2- 172.21.15.60:0/1278771233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603c073de0 0x7f603c074240 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.474+0000 7f6041400640 1 -- 172.21.15.60:0/1278771233 >> 172.21.15.60:0/1278771233 conn(0x7f603c0b18e0 msgr2=0x7f603c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.474+0000 7f6041400640 1 -- 172.21.15.60:0/1278771233 shutdown_connections 2024-09-16T15:46:25.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.474+0000 7f6041400640 1 -- 172.21.15.60:0/1278771233 wait complete. 2024-09-16T15:46:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.475+0000 7f6041400640 1 Processor -- start 2024-09-16T15:46:25.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.475+0000 7f6041400640 1 -- start start 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.476+0000 7f6041400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f603c06e6f0 0x7f603c158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.476+0000 7f6041400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f603c073de0 0x7f603c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.476+0000 7f6041400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603c15f9a0 0x7f603c1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.476+0000 7f6041400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0ccc20 con 0x7f603c15f9a0 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.476+0000 7f6041400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0cd5e0 con 0x7f603c06e6f0 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.476+0000 7f6041400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f603c06f0e0 con 0x7f603c073de0 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603c15f9a0 0x7f603c1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603c15f9a0 0x7f603c1f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46186/0 (socket says 172.21.15.60:46186) 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603b200640 1 -- 172.21.15.60:0/2032014316 learned_addr learned my addr 172.21.15.60:0/2032014316 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603a800640 1 --2- 172.21.15.60:0/2032014316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f603c06e6f0 0x7f603c158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f6039e00640 1 --2- 172.21.15.60:0/2032014316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f603c073de0 0x7f603c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603b200640 1 -- 172.21.15.60:0/2032014316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f603c073de0 msgr2=0x7f603c15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603b200640 1 --2- 172.21.15.60:0/2032014316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f603c073de0 0x7f603c15c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603b200640 1 -- 172.21.15.60:0/2032014316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f603c06e6f0 msgr2=0x7f603c158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603b200640 1 --2- 172.21.15.60:0/2032014316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f603c06e6f0 0x7f603c158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.477+0000 7f603b200640 1 -- 172.21.15.60:0/2032014316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c15c880 con 0x7f603c15f9a0 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.478+0000 7f603b200640 1 --2- 172.21.15.60:0/2032014316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f603c15f9a0 0x7f603c1f84d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f603400eaa0 tx=0x7f603400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.478+0000 7f6032a00640 1 -- 172.21.15.60:0/2032014316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603400cdb0 con 0x7f603c15f9a0 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.478+0000 7f6032a00640 1 -- 172.21.15.60:0/2032014316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6034004c10 con 0x7f603c15f9a0 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.478+0000 7f6041400640 1 -- 172.21.15.60:0/2032014316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c1f9030 con 0x7f603c15f9a0 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.478+0000 7f6032a00640 1 -- 172.21.15.60:0/2032014316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034005020 con 0x7f603c15f9a0 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.478+0000 7f6041400640 1 -- 172.21.15.60:0/2032014316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c1f6610 con 0x7f603c15f9a0 2024-09-16T15:46:25.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.479+0000 7f6041400640 1 -- 172.21.15.60:0/2032014316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c069360 con 0x7f603c15f9a0 2024-09-16T15:46:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.480+0000 7f6032a00640 1 -- 172.21.15.60:0/2032014316 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f60340051c0 con 0x7f603c15f9a0 2024-09-16T15:46:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.481+0000 7f6032a00640 1 --2- 172.21.15.60:0/2032014316 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f601c078eb0 0x7f601c07b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.481+0000 7f6032a00640 1 -- 172.21.15.60:0/2032014316 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f603409bef0 con 0x7f603c15f9a0 2024-09-16T15:46:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.481+0000 7f603a800640 1 --2- 172.21.15.60:0/2032014316 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f601c078eb0 0x7f601c07b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.482+0000 7f603a800640 1 --2- 172.21.15.60:0/2032014316 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f601c078eb0 0x7f601c07b370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f602c006fd0 tx=0x7f602c009060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.485+0000 7f6032a00640 1 -- 172.21.15.60:0/2032014316 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6034016030 con 0x7f603c15f9a0 2024-09-16T15:46:25.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.715+0000 7f6041400640 1 -- 172.21.15.60:0/2032014316 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f603c0cd7b0 con 0x7f601c078eb0 2024-09-16T15:46:25.718 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:25.716+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.716+0000 7f6032a00640 1 -- 172.21.15.60:0/2032014316 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f603c0cd7b0 con 0x7f601c078eb0 2024-09-16T15:46:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:25.728 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:25.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-16T15:46:25.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.894+0000 7fc5b1e00640 1 -- 172.21.15.60:0/4261449555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ac159aa0 msgr2=0x7fc5ac0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.894+0000 7fc5b1e00640 1 --2- 172.21.15.60:0/4261449555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ac159aa0 0x7fc5ac0741a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009860 tx=0x7fc59c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.894+0000 7fc5b1e00640 1 -- 172.21.15.60:0/4261449555 shutdown_connections 2024-09-16T15:46:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.894+0000 7fc5b1e00640 1 --2- 172.21.15.60:0/4261449555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ac15cff0 0x7fc5ac06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.894+0000 7fc5b1e00640 1 --2- 172.21.15.60:0/4261449555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ac159aa0 0x7fc5ac0741a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.894+0000 7fc5b1e00640 1 -- 172.21.15.60:0/4261449555 >> 172.21.15.60:0/4261449555 conn(0x7fc5ac0b18e0 msgr2=0x7fc5ac0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.894+0000 7fc5b1e00640 1 -- 172.21.15.60:0/4261449555 shutdown_connections 2024-09-16T15:46:25.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.894+0000 7fc5b1e00640 1 -- 172.21.15.60:0/4261449555 wait complete. 2024-09-16T15:46:25.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.896+0000 7fc5b1e00640 1 Processor -- start 2024-09-16T15:46:25.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.896+0000 7fc5b1e00640 1 -- start start 2024-09-16T15:46:25.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5b1e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ac06f220 0x7fc5ac1497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5b1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ac159aa0 0x7fc5ac1f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5b1e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ac15cff0 0x7fc5ac0ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5b1e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac06fa60 con 0x7fc5ac159aa0 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5b1e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac06f8e0 con 0x7fc5ac15cff0 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5b1e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc5ac0cc8a0 con 0x7fc5ac06f220 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5aa000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ac159aa0 0x7fc5ac1f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5aa000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ac159aa0 0x7fc5ac1f57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46188/0 (socket says 172.21.15.60:46188) 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5aa000640 1 -- 172.21.15.60:0/2651457 learned_addr learned my addr 172.21.15.60:0/2651457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5aa000640 1 -- 172.21.15.60:0/2651457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ac06f220 msgr2=0x7fc5ac1497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5aa000640 1 --2- 172.21.15.60:0/2651457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5ac06f220 0x7fc5ac1497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5aa000640 1 -- 172.21.15.60:0/2651457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ac15cff0 msgr2=0x7fc5ac0ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.897+0000 7fc5aa000640 1 --2- 172.21.15.60:0/2651457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5ac15cff0 0x7fc5ac0ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.898+0000 7fc5aa000640 1 -- 172.21.15.60:0/2651457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac1473f0 con 0x7fc5ac159aa0 2024-09-16T15:46:25.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.898+0000 7fc5aa000640 1 --2- 172.21.15.60:0/2651457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5ac159aa0 0x7fc5ac1f57e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc598009fd0 tx=0x7fc59800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.898+0000 7fc5a3400640 1 -- 172.21.15.60:0/2651457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59800ccc0 con 0x7fc5ac159aa0 2024-09-16T15:46:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.898+0000 7fc5a3400640 1 -- 172.21.15.60:0/2651457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc59800ee70 con 0x7fc5ac159aa0 2024-09-16T15:46:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.898+0000 7fc5a3400640 1 -- 172.21.15.60:0/2651457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59800ce60 con 0x7fc5ac159aa0 2024-09-16T15:46:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.898+0000 7fc5b1e00640 1 -- 172.21.15.60:0/2651457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac14a6b0 con 0x7fc5ac159aa0 2024-09-16T15:46:25.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.898+0000 7fc5b1e00640 1 -- 172.21.15.60:0/2651457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac14a0f0 con 0x7fc5ac159aa0 2024-09-16T15:46:25.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.899+0000 7fc5b1e00640 1 -- 172.21.15.60:0/2651457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac06e6f0 con 0x7fc5ac159aa0 2024-09-16T15:46:25.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.900+0000 7fc5a3400640 1 -- 172.21.15.60:0/2651457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fc598004280 con 0x7fc5ac159aa0 2024-09-16T15:46:25.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.901+0000 7fc5a3400640 1 --2- 172.21.15.60:0/2651457 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc58c078e80 0x7fc58c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:25.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.902+0000 7fc5a3400640 1 -- 172.21.15.60:0/2651457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fc59809bf20 con 0x7fc5ac159aa0 2024-09-16T15:46:25.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.902+0000 7fc5aaa00640 1 --2- 172.21.15.60:0/2651457 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc58c078e80 0x7fc58c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:25.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.902+0000 7fc5aaa00640 1 --2- 172.21.15.60:0/2651457 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc58c078e80 0x7fc58c07b340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc59c034000 tx=0x7fc59c0046c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:25.906+0000 7fc5a3400640 1 -- 172.21.15.60:0/2651457 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc598063b70 con 0x7fc5ac159aa0 2024-09-16T15:46:26.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.134+0000 7fc5b1e00640 1 -- 172.21.15.60:0/2651457 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fc5ac070980 con 0x7fc58c078e80 2024-09-16T15:46:26.137 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:26.135+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:26.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.136+0000 7fc5a3400640 1 -- 172.21.15.60:0/2651457 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc5ac070980 con 0x7fc58c078e80 2024-09-16T15:46:26.138 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:26.148 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:26.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-16T15:46:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 -- 172.21.15.60:0/2565017029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4374159aa0 msgr2=0x7f43740741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 --2- 172.21.15.60:0/2565017029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4374159aa0 0x7f43740741a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f4374070850 tx=0x7f436802a560 comp rx=0 tx=0).stop 2024-09-16T15:46:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 -- 172.21.15.60:0/2565017029 shutdown_connections 2024-09-16T15:46:26.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 --2- 172.21.15.60:0/2565017029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437406f220 0x7f437406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:26.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 --2- 172.21.15.60:0/2565017029 >> v2:172.21.15.100:3300/0 conn(0x7f437415cff0 0x7f437406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:26.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 --2- 172.21.15.60:0/2565017029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4374159aa0 0x7f43740741a0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:26.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 -- 172.21.15.60:0/2565017029 >> 172.21.15.60:0/2565017029 conn(0x7f43740b18e0 msgr2=0x7f43740b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:26.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 -- 172.21.15.60:0/2565017029 shutdown_connections 2024-09-16T15:46:26.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.312+0000 7f437ba00640 1 -- 172.21.15.60:0/2565017029 wait complete. 2024-09-16T15:46:26.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.314+0000 7f437ba00640 1 Processor -- start 2024-09-16T15:46:26.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.314+0000 7f437ba00640 1 -- start start 2024-09-16T15:46:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f437ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437406f220 0x7f43740b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f437ba00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4374159aa0 0x7f43740adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:26.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f437ba00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437415cff0 0x7f43740ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f437ba00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f43740cc310 con 0x7f437415cff0 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f437ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43740cc190 con 0x7f437406f220 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f437ba00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f43740cc490 con 0x7f4374159aa0 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437415cff0 0x7f43740ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437406f220 0x7f43740b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437406f220 0x7f43740b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:34846/0 (socket says 172.21.15.60:34846) 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379000640 1 -- 172.21.15.60:0/1277882333 learned_addr learned my addr 172.21.15.60:0/1277882333 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437415cff0 0x7f43740ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46208/0 (socket says 172.21.15.60:46208) 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4373e00640 1 --2- 172.21.15.60:0/1277882333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4374159aa0 0x7f43740adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379a00640 1 -- 172.21.15.60:0/1277882333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4374159aa0 msgr2=0x7f43740adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379a00640 1 --2- 172.21.15.60:0/1277882333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4374159aa0 0x7f43740adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379a00640 1 -- 172.21.15.60:0/1277882333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437406f220 msgr2=0x7f43740b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:26.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379a00640 1 --2- 172.21.15.60:0/1277882333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437406f220 0x7f43740b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:26.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.315+0000 7f4379a00640 1 -- 172.21.15.60:0/1277882333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43740b3530 con 0x7f437415cff0 2024-09-16T15:46:26.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.316+0000 7f4379a00640 1 --2- 172.21.15.60:0/1277882333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437415cff0 0x7f43740ae3f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f436c00eab0 tx=0x7f436c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:26.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.316+0000 7f4370c00640 1 -- 172.21.15.60:0/1277882333 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00ce40 con 0x7f437415cff0 2024-09-16T15:46:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.316+0000 7f4370c00640 1 -- 172.21.15.60:0/1277882333 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f436c004b20 con 0x7f437415cff0 2024-09-16T15:46:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.316+0000 7f4370c00640 1 -- 172.21.15.60:0/1277882333 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c005020 con 0x7f437415cff0 2024-09-16T15:46:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.316+0000 7f437ba00640 1 -- 172.21.15.60:0/1277882333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43740b3810 con 0x7f437415cff0 2024-09-16T15:46:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.316+0000 7f437ba00640 1 -- 172.21.15.60:0/1277882333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43740b3dd0 con 0x7f437415cff0 2024-09-16T15:46:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.317+0000 7f437ba00640 1 -- 172.21.15.60:0/1277882333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43740741a0 con 0x7f437415cff0 2024-09-16T15:46:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.318+0000 7f4370c00640 1 -- 172.21.15.60:0/1277882333 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f436c0051c0 con 0x7f437415cff0 2024-09-16T15:46:26.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.319+0000 7f4370c00640 1 --2- 172.21.15.60:0/1277882333 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4354078e80 0x7f435407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:26.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.319+0000 7f4370c00640 1 -- 172.21.15.60:0/1277882333 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f436c09c000 con 0x7f437415cff0 2024-09-16T15:46:26.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.320+0000 7f4379000640 1 --2- 172.21.15.60:0/1277882333 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4354078e80 0x7f435407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:26.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.320+0000 7f4379000640 1 --2- 172.21.15.60:0/1277882333 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4354078e80 0x7f435407b340 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4368009230 tx=0x7f4368004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.324+0000 7f4370c00640 1 -- 172.21.15.60:0/1277882333 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f436c0635b0 con 0x7f437415cff0 2024-09-16T15:46:26.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.553+0000 7f437ba00640 1 -- 172.21.15.60:0/1277882333 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f43741604e0 con 0x7f4354078e80 2024-09-16T15:46:26.556 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:26.554+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:26.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.554+0000 7f4370c00640 1 -- 172.21.15.60:0/1277882333 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f43741604e0 con 0x7f4354078e80 2024-09-16T15:46:26.556 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:26.566 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:26.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-16T15:46:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.733+0000 7f7091400640 1 -- 172.21.15.60:0/1437106130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070004800 msgr2=0x7f7070004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.733+0000 7f7091400640 1 --2- 172.21.15.60:0/1437106130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070004800 0x7f7070004c80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f70780017c0 tx=0x7f7070005830 comp rx=0 tx=0).stop 2024-09-16T15:46:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.733+0000 7f7091400640 1 -- 172.21.15.60:0/1437106130 shutdown_connections 2024-09-16T15:46:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.733+0000 7f7091400640 1 --2- 172.21.15.60:0/1437106130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070004800 0x7f7070004c80 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.733+0000 7f7091400640 1 -- 172.21.15.60:0/1437106130 >> 172.21.15.60:0/1437106130 conn(0x7f708c0cfa70 msgr2=0x7f708c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.733+0000 7f7091400640 1 -- 172.21.15.60:0/1437106130 shutdown_connections 2024-09-16T15:46:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.733+0000 7f7091400640 1 -- 172.21.15.60:0/1437106130 wait complete. 2024-09-16T15:46:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.734+0000 7f7091400640 1 Processor -- start 2024-09-16T15:46:26.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.735+0000 7f7091400640 1 -- start start 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f7091400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070004800 0x7f708c153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f7091400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708c0b3aa0 0x7f708c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f7091400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c0cc5e0 0x7f708c0cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f7091400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f708c070220 con 0x7f7070004800 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f7091400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f708c0700a0 con 0x7f708c0b3aa0 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f7091400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f708c0703a0 con 0x7f708c0cc5e0 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070004800 0x7f708c153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070004800 0x7f708c153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46222/0 (socket says 172.21.15.60:46222) 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708be00640 1 -- 172.21.15.60:0/4028457421 learned_addr learned my addr 172.21.15.60:0/4028457421 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708b400640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708c0b3aa0 0x7f708c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f7090a00640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c0cc5e0 0x7f708c0cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708be00640 1 -- 172.21.15.60:0/4028457421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c0cc5e0 msgr2=0x7f708c0cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708be00640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f708c0cc5e0 0x7f708c0cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708be00640 1 -- 172.21.15.60:0/4028457421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708c0b3aa0 msgr2=0x7f708c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708be00640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708c0b3aa0 0x7f708c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708be00640 1 -- 172.21.15.60:0/4028457421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c1f8de0 con 0x7f7070004800 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.736+0000 7f708b400640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f708c0b3aa0 0x7f708c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.737+0000 7f708be00640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7070004800 0x7f708c153f50 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f707c011a50 tx=0x7f707c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.737+0000 7f7083e00640 1 -- 172.21.15.60:0/4028457421 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c009720 con 0x7f7070004800 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.737+0000 7f7083e00640 1 -- 172.21.15.60:0/4028457421 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f707c00fc50 con 0x7f7070004800 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.737+0000 7f7083e00640 1 -- 172.21.15.60:0/4028457421 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c005020 con 0x7f7070004800 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.737+0000 7f7091400640 1 -- 172.21.15.60:0/4028457421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c0cd2d0 con 0x7f7070004800 2024-09-16T15:46:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.737+0000 7f7091400640 1 -- 172.21.15.60:0/4028457421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c1f8300 con 0x7f7070004800 2024-09-16T15:46:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.738+0000 7f7091400640 1 -- 172.21.15.60:0/4028457421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c0b4460 con 0x7f7070004800 2024-09-16T15:46:26.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.739+0000 7f7083e00640 1 -- 172.21.15.60:0/4028457421 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f707c0051c0 con 0x7f7070004800 2024-09-16T15:46:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.741+0000 7f7083e00640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f706c078e80 0x7f706c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.741+0000 7f7083e00640 1 -- 172.21.15.60:0/4028457421 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f707c09bf40 con 0x7f7070004800 2024-09-16T15:46:26.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.741+0000 7f708b400640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f706c078e80 0x7f706c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:26.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.742+0000 7f708b400640 1 --2- 172.21.15.60:0/4028457421 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f706c078e80 0x7f706c07b340 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7078032000 tx=0x7f707802c000 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:26.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.745+0000 7f7083e00640 1 -- 172.21.15.60:0/4028457421 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f707c063b00 con 0x7f7070004800 2024-09-16T15:46:26.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.979+0000 7f7091400640 1 -- 172.21.15.60:0/4028457421 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f708c0733f0 con 0x7f706c078e80 2024-09-16T15:46:26.982 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:26.980+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:26.981+0000 7f7083e00640 1 -- 172.21.15.60:0/4028457421 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f708c0733f0 con 0x7f706c078e80 2024-09-16T15:46:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:26.992 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:26.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-16T15:46:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.157+0000 7fb92da00640 1 -- 172.21.15.60:0/3213101018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9280af190 msgr2=0x7fb9280b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.157+0000 7fb92da00640 1 --2- 172.21.15.60:0/3213101018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9280af190 0x7fb9280b2b40 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb914009830 tx=0x7fb91402a520 comp rx=0 tx=0).stop 2024-09-16T15:46:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.157+0000 7fb92da00640 1 -- 172.21.15.60:0/3213101018 shutdown_connections 2024-09-16T15:46:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.157+0000 7fb92da00640 1 --2- 172.21.15.60:0/3213101018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9280cec60 0x7fb9280cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.157+0000 7fb92da00640 1 --2- 172.21.15.60:0/3213101018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9280af190 0x7fb9280b2b40 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.158+0000 7fb92da00640 1 -- 172.21.15.60:0/3213101018 >> 172.21.15.60:0/3213101018 conn(0x7fb9280b1840 msgr2=0x7fb9280b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.158+0000 7fb92da00640 1 -- 172.21.15.60:0/3213101018 shutdown_connections 2024-09-16T15:46:27.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.158+0000 7fb92da00640 1 -- 172.21.15.60:0/3213101018 wait complete. 2024-09-16T15:46:27.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.160+0000 7fb92da00640 1 Processor -- start 2024-09-16T15:46:27.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.160+0000 7fb92da00640 1 -- start start 2024-09-16T15:46:27.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb92da00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9280af190 0x7fb928147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb92da00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9280cec60 0x7fb92814a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb92da00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb928161760 0x7fb9280cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb92da00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb928162310 con 0x7fb928161760 2024-09-16T15:46:27.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb92da00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb928162190 con 0x7fb9280af190 2024-09-16T15:46:27.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb92da00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb928162490 con 0x7fb9280cec60 2024-09-16T15:46:27.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb927400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb928161760 0x7fb9280cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb927400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb928161760 0x7fb9280cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46236/0 (socket says 172.21.15.60:46236) 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb927400640 1 -- 172.21.15.60:0/1897979108 learned_addr learned my addr 172.21.15.60:0/1897979108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb926000640 1 --2- 172.21.15.60:0/1897979108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9280cec60 0x7fb92814a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.161+0000 7fb926a00640 1 --2- 172.21.15.60:0/1897979108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9280af190 0x7fb928147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb927400640 1 -- 172.21.15.60:0/1897979108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9280cec60 msgr2=0x7fb92814a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb927400640 1 --2- 172.21.15.60:0/1897979108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9280cec60 0x7fb92814a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb927400640 1 -- 172.21.15.60:0/1897979108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9280af190 msgr2=0x7fb928147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb927400640 1 --2- 172.21.15.60:0/1897979108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9280af190 0x7fb928147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb927400640 1 -- 172.21.15.60:0/1897979108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928160d30 con 0x7fb928161760 2024-09-16T15:46:27.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb927400640 1 --2- 172.21.15.60:0/1897979108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb928161760 0x7fb9280cc2b0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb92000b970 tx=0x7fb92000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb91ea00640 1 -- 172.21.15.60:0/1897979108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920019070 con 0x7fb928161760 2024-09-16T15:46:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb91ea00640 1 -- 172.21.15.60:0/1897979108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb92000cc50 con 0x7fb928161760 2024-09-16T15:46:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb91ea00640 1 -- 172.21.15.60:0/1897979108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9200146a0 con 0x7fb928161760 2024-09-16T15:46:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.162+0000 7fb92da00640 1 -- 172.21.15.60:0/1897979108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb928159440 con 0x7fb928161760 2024-09-16T15:46:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.163+0000 7fb92da00640 1 -- 172.21.15.60:0/1897979108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb928159a00 con 0x7fb928161760 2024-09-16T15:46:27.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.164+0000 7fb92da00640 1 -- 172.21.15.60:0/1897979108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb928162920 con 0x7fb928161760 2024-09-16T15:46:27.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.165+0000 7fb91ea00640 1 -- 172.21.15.60:0/1897979108 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7fb92001c070 con 0x7fb928161760 2024-09-16T15:46:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.166+0000 7fb91ea00640 1 --2- 172.21.15.60:0/1897979108 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb908078e80 0x7fb90807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.167+0000 7fb91ea00640 1 -- 172.21.15.60:0/1897979108 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fb92009b280 con 0x7fb928161760 2024-09-16T15:46:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.167+0000 7fb926a00640 1 --2- 172.21.15.60:0/1897979108 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb908078e80 0x7fb90807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.167+0000 7fb926a00640 1 --2- 172.21.15.60:0/1897979108 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb908078e80 0x7fb90807b340 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb918001720 tx=0x7fb918007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:27.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.171+0000 7fb91ea00640 1 -- 172.21.15.60:0/1897979108 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb920062830 con 0x7fb928161760 2024-09-16T15:46:27.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.403+0000 7fb92da00640 1 -- 172.21.15.60:0/1897979108 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fb9280adde0 con 0x7fb908078e80 2024-09-16T15:46:27.412 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:27.410+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.412 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:27.410+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.412 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:27.410+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.412 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:27.410+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.412 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:27.410+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.418 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:27.416+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:27.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.417+0000 7fb91ea00640 1 -- 172.21.15.60:0/1897979108 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fb9280adde0 con 0x7fb908078e80 2024-09-16T15:46:27.419 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:27.429 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:27.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-16T15:46:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.592+0000 7f00b5a00640 1 -- 172.21.15.60:0/1336227301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b00cec60 msgr2=0x7f00b00cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.592+0000 7f00b5a00640 1 --2- 172.21.15.60:0/1336227301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b00cec60 0x7f00b00cf090 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f00a8005fd0 tx=0x7f00a802a820 comp rx=0 tx=0).stop 2024-09-16T15:46:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.592+0000 7f00b5a00640 1 -- 172.21.15.60:0/1336227301 shutdown_connections 2024-09-16T15:46:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.592+0000 7f00b5a00640 1 --2- 172.21.15.60:0/1336227301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b00cec60 0x7f00b00cf090 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.592+0000 7f00b5a00640 1 --2- 172.21.15.60:0/1336227301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b0161760 0x7f00b00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.592+0000 7f00b5a00640 1 -- 172.21.15.60:0/1336227301 >> 172.21.15.60:0/1336227301 conn(0x7f00b00b1840 msgr2=0x7f00b00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.592+0000 7f00b5a00640 1 -- 172.21.15.60:0/1336227301 shutdown_connections 2024-09-16T15:46:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.592+0000 7f00b5a00640 1 -- 172.21.15.60:0/1336227301 wait complete. 2024-09-16T15:46:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.594+0000 7f00b5a00640 1 Processor -- start 2024-09-16T15:46:27.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.594+0000 7f00b5a00640 1 -- start start 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00b5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b00af190 0x7f00b01f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00b5a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b00cec60 0x7f00b0159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00b5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0161760 0x7f00b01485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00b5a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0162190 con 0x7f00b00af190 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00b5a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00b0070250 con 0x7f00b0161760 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00b5a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f00b0162310 con 0x7f00b00cec60 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00aea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b00af190 0x7f00b01f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00aea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b00af190 0x7f00b01f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46270/0 (socket says 172.21.15.60:46270) 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00aea00640 1 -- 172.21.15.60:0/536329277 learned_addr learned my addr 172.21.15.60:0/536329277 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00ae000640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b00cec60 0x7f00b0159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.595+0000 7f00af400640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0161760 0x7f00b01485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00aea00640 1 -- 172.21.15.60:0/536329277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b00cec60 msgr2=0x7f00b0159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00aea00640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b00cec60 0x7f00b0159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00aea00640 1 -- 172.21.15.60:0/536329277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0161760 msgr2=0x7f00b01485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00aea00640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0161760 0x7f00b01485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00aea00640 1 -- 172.21.15.60:0/536329277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b0159860 con 0x7f00b00af190 2024-09-16T15:46:27.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00af400640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00b0161760 0x7f00b01485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:27.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00ae000640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00b00cec60 0x7f00b0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:27.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00aea00640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00b00af190 0x7f00b01f4ae0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f00a000e790 tx=0x7f00a000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00a6a00640 1 -- 172.21.15.60:0/536329277 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a000eea0 con 0x7f00b00af190 2024-09-16T15:46:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00a6a00640 1 -- 172.21.15.60:0/536329277 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00a000cdb0 con 0x7f00b00af190 2024-09-16T15:46:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00a6a00640 1 -- 172.21.15.60:0/536329277 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a0010690 con 0x7f00b00af190 2024-09-16T15:46:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00b5a00640 1 -- 172.21.15.60:0/536329277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b0159b40 con 0x7f00b00af190 2024-09-16T15:46:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.596+0000 7f00b5a00640 1 -- 172.21.15.60:0/536329277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b015a100 con 0x7f00b00af190 2024-09-16T15:46:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.598+0000 7f00b5a00640 1 -- 172.21.15.60:0/536329277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b00cf090 con 0x7f00b00af190 2024-09-16T15:46:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.599+0000 7f00a6a00640 1 -- 172.21.15.60:0/536329277 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f00a0010830 con 0x7f00b00af190 2024-09-16T15:46:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.600+0000 7f00a6a00640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0090078e80 0x7f009007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.600+0000 7f00a6a00640 1 -- 172.21.15.60:0/536329277 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f00a009bfb0 con 0x7f00b00af190 2024-09-16T15:46:27.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.600+0000 7f00ae000640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0090078e80 0x7f009007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.601+0000 7f00ae000640 1 --2- 172.21.15.60:0/536329277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0090078e80 0x7f009007b340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f00b01f5c00 tx=0x7f009c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:27.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.605+0000 7f00a6a00640 1 -- 172.21.15.60:0/536329277 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f00a0063bb0 con 0x7f00b00af190 2024-09-16T15:46:27.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.839+0000 7f00b5a00640 1 -- 172.21.15.60:0/536329277 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f00b015e430 con 0x7f0090078e80 2024-09-16T15:46:27.842 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:27.841+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:27.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:27.841+0000 7f00a6a00640 1 -- 172.21.15.60:0/536329277 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f00b015e430 con 0x7f0090078e80 2024-09-16T15:46:27.843 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:27.853 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:27.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-16T15:46:28.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.015+0000 7f17d4a00640 1 -- 172.21.15.60:0/1324359441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d0161460 msgr2=0x7f17d00d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.015+0000 7f17d4a00640 1 --2- 172.21.15.60:0/1324359441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d0161460 0x7f17d00d22f0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009830 tx=0x7f17c002a530 comp rx=0 tx=0).stop 2024-09-16T15:46:28.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.015+0000 7f17d4a00640 1 -- 172.21.15.60:0/1324359441 shutdown_connections 2024-09-16T15:46:28.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.015+0000 7f17d4a00640 1 --2- 172.21.15.60:0/1324359441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d0161460 0x7f17d00d22f0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.015+0000 7f17d4a00640 1 -- 172.21.15.60:0/1324359441 >> 172.21.15.60:0/1324359441 conn(0x7f17d009d0b0 msgr2=0x7f17d00d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:28.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.015+0000 7f17d4a00640 1 -- 172.21.15.60:0/1324359441 shutdown_connections 2024-09-16T15:46:28.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.015+0000 7f17d4a00640 1 -- 172.21.15.60:0/1324359441 wait complete. 2024-09-16T15:46:28.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.016+0000 7f17d4a00640 1 Processor -- start 2024-09-16T15:46:28.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.017+0000 7f17d4a00640 1 -- start start 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.017+0000 7f17d4a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00b43c0 0x7f17d00a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17d4a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d00b5480 0x7f17d015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17d4a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d0161460 0x7f17d01f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17d4a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f17d00d29b0 con 0x7f17d00b43c0 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17d4a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f17d00d2830 con 0x7f17d00b5480 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17d4a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f17d00d2b30 con 0x7f17d0161460 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00b43c0 0x7f17d00a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00b43c0 0x7f17d00a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46294/0 (socket says 172.21.15.60:46294) 2024-09-16T15:46:28.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 -- 172.21.15.60:0/731889253 learned_addr learned my addr 172.21.15.60:0/731889253 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cea00640 1 --2- 172.21.15.60:0/731889253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d00b5480 0x7f17d015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cfe00640 1 --2- 172.21.15.60:0/731889253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d0161460 0x7f17d01f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 -- 172.21.15.60:0/731889253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d0161460 msgr2=0x7f17d01f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 --2- 172.21.15.60:0/731889253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d0161460 0x7f17d01f7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 -- 172.21.15.60:0/731889253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d00b5480 msgr2=0x7f17d015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 --2- 172.21.15.60:0/731889253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d00b5480 0x7f17d015e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 -- 172.21.15.60:0/731889253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d015cef0 con 0x7f17d00b43c0 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.018+0000 7f17cf400640 1 --2- 172.21.15.60:0/731889253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00b43c0 0x7f17d00a1440 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009950 tx=0x7f17c0033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.019+0000 7f17c7400640 1 -- 172.21.15.60:0/731889253 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0031b50 con 0x7f17d00b43c0 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.019+0000 7f17c7400640 1 -- 172.21.15.60:0/731889253 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17c0031de0 con 0x7f17d00b43c0 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.019+0000 7f17c7400640 1 -- 172.21.15.60:0/731889253 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0005280 con 0x7f17d00b43c0 2024-09-16T15:46:28.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.019+0000 7f17d4a00640 1 -- 172.21.15.60:0/731889253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d015d1d0 con 0x7f17d00b43c0 2024-09-16T15:46:28.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.019+0000 7f17d4a00640 1 -- 172.21.15.60:0/731889253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d015d790 con 0x7f17d00b43c0 2024-09-16T15:46:28.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.020+0000 7f17d4a00640 1 -- 172.21.15.60:0/731889253 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d01f57c0 con 0x7f17d00b43c0 2024-09-16T15:46:28.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.021+0000 7f17c7400640 1 -- 172.21.15.60:0/731889253 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f17c0005420 con 0x7f17d00b43c0 2024-09-16T15:46:28.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.022+0000 7f17c7400640 1 --2- 172.21.15.60:0/731889253 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17b0078eb0 0x7f17b007b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.022+0000 7f17c7400640 1 -- 172.21.15.60:0/731889253 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f17c00bc3a0 con 0x7f17d00b43c0 2024-09-16T15:46:28.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.022+0000 7f17cea00640 1 --2- 172.21.15.60:0/731889253 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17b0078eb0 0x7f17b007b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.023+0000 7f17cea00640 1 --2- 172.21.15.60:0/731889253 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17b0078eb0 0x7f17b007b370 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f17d00cf130 tx=0x7f17bc009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:28.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.026+0000 7f17c7400640 1 -- 172.21.15.60:0/731889253 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f17c003f030 con 0x7f17d00b43c0 2024-09-16T15:46:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.247+0000 7f17d4a00640 1 -- 172.21.15.60:0/731889253 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f17d0069d80 con 0x7f17b0078eb0 2024-09-16T15:46:28.250 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:28.248+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.248+0000 7f17c7400640 1 -- 172.21.15.60:0/731889253 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f17bc001c60 con 0x7f17b0078eb0 2024-09-16T15:46:28.251 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:28.260 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:28.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-16T15:46:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.422+0000 7f10dde00640 1 -- 172.21.15.60:0/4283312773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 msgr2=0x7f10d80cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.422+0000 7f10dde00640 1 --2- 172.21.15.60:0/4283312773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 0x7f10d80cc5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10c0005fd0 tx=0x7f10c002a540 comp rx=0 tx=0).stop 2024-09-16T15:46:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.423+0000 7f10dde00640 1 -- 172.21.15.60:0/4283312773 shutdown_connections 2024-09-16T15:46:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.423+0000 7f10dde00640 1 --2- 172.21.15.60:0/4283312773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d806efa0 0x7f10d80cc5c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.423+0000 7f10dde00640 1 --2- 172.21.15.60:0/4283312773 >> v2:172.21.15.100:3300/0 conn(0x7f10d8145750 0x7f10d8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.423+0000 7f10dde00640 1 -- 172.21.15.60:0/4283312773 >> 172.21.15.60:0/4283312773 conn(0x7f10d80b1840 msgr2=0x7f10d80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.423+0000 7f10dde00640 1 -- 172.21.15.60:0/4283312773 shutdown_connections 2024-09-16T15:46:28.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.423+0000 7f10dde00640 1 -- 172.21.15.60:0/4283312773 wait complete. 2024-09-16T15:46:28.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.424+0000 7f10dde00640 1 Processor -- start 2024-09-16T15:46:28.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.424+0000 7f10dde00640 1 -- start start 2024-09-16T15:46:28.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.425+0000 7f10dde00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d806efa0 0x7f10d81f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.425+0000 7f10dde00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d80ccb00 0x7f10d81e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.425+0000 7f10dde00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d8145750 0x7f10d81e9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.425+0000 7f10dde00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80b26b0 con 0x7f10d806efa0 2024-09-16T15:46:28.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.425+0000 7f10dde00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80cedd0 con 0x7f10d80ccb00 2024-09-16T15:46:28.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.425+0000 7f10dde00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f10d8160100 con 0x7f10d8145750 2024-09-16T15:46:28.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d806efa0 0x7f10d81f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d806efa0 0x7f10d81f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46300/0 (socket says 172.21.15.60:46300) 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 -- 172.21.15.60:0/3946006257 learned_addr learned my addr 172.21.15.60:0/3946006257 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10ce000640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d80ccb00 0x7f10d81e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d7400640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d8145750 0x7f10d81e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 -- 172.21.15.60:0/3946006257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d8145750 msgr2=0x7f10d81e9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f10d8145750 0x7f10d81e9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 -- 172.21.15.60:0/3946006257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d80ccb00 msgr2=0x7f10d81e8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d80ccb00 0x7f10d81e8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 -- 172.21.15.60:0/3946006257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d814b700 con 0x7f10d806efa0 2024-09-16T15:46:28.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10ce000640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d80ccb00 0x7f10d81e8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:28.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.426+0000 7f10d6a00640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f10d806efa0 0x7f10d81f4720 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f10c800e7c0 tx=0x7f10c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:28.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.427+0000 7f10cfe00640 1 -- 172.21.15.60:0/3946006257 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800ccc0 con 0x7f10d806efa0 2024-09-16T15:46:28.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.427+0000 7f10cfe00640 1 -- 172.21.15.60:0/3946006257 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10c8010430 con 0x7f10d806efa0 2024-09-16T15:46:28.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.427+0000 7f10dde00640 1 -- 172.21.15.60:0/3946006257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d814b9e0 con 0x7f10d806efa0 2024-09-16T15:46:28.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.427+0000 7f10cfe00640 1 -- 172.21.15.60:0/3946006257 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8010860 con 0x7f10d806efa0 2024-09-16T15:46:28.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.427+0000 7f10dde00640 1 -- 172.21.15.60:0/3946006257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d81e6290 con 0x7f10d806efa0 2024-09-16T15:46:28.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.428+0000 7f10dde00640 1 -- 172.21.15.60:0/3946006257 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d80b23c0 con 0x7f10d806efa0 2024-09-16T15:46:28.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.429+0000 7f10cfe00640 1 -- 172.21.15.60:0/3946006257 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 21) ==== 101848+0+0 (secure 0 0 0) 0x7f10c8010a00 con 0x7f10d806efa0 2024-09-16T15:46:28.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.430+0000 7f10cfe00640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f10b4078e80 0x7f10b407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.430+0000 7f10cfe00640 1 -- 172.21.15.60:0/3946006257 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f10c809bf90 con 0x7f10d806efa0 2024-09-16T15:46:28.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.430+0000 7f10ce000640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f10b4078e80 0x7f10b407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.431+0000 7f10ce000640 1 --2- 172.21.15.60:0/3946006257 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f10b4078e80 0x7f10b407b340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f10c0033000 tx=0x7f10c0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:28.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.434+0000 7f10cfe00640 1 -- 172.21.15.60:0/3946006257 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f10d80b23c0 con 0x7f10d806efa0 2024-09-16T15:46:28.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.654+0000 7f10dde00640 1 -- 172.21.15.60:0/3946006257 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f10d81f4c60 con 0x7f10b4078e80 2024-09-16T15:46:28.657 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:28.656+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.656+0000 7f10cfe00640 1 -- 172.21.15.60:0/3946006257 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f10d81f4c60 con 0x7f10b4078e80 2024-09-16T15:46:28.658 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:28.668 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:28.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-16T15:46:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.835+0000 7f22ab200640 1 -- 172.21.15.60:0/4198395898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22ac0ae5e0 msgr2=0x7f22ac06e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.835+0000 7f22ab200640 1 --2- 172.21.15.60:0/4198395898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22ac0ae5e0 0x7f22ac06e880 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009be0 tx=0x7f22a402a8d0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.835+0000 7f22ab200640 1 -- 172.21.15.60:0/4198395898 shutdown_connections 2024-09-16T15:46:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.835+0000 7f22ab200640 1 --2- 172.21.15.60:0/4198395898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22ac0ae5e0 0x7f22ac06e880 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.835+0000 7f22ab200640 1 -- 172.21.15.60:0/4198395898 >> 172.21.15.60:0/4198395898 conn(0x7f22ac0cf9f0 msgr2=0x7f22ac0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.835+0000 7f22ab200640 1 -- 172.21.15.60:0/4198395898 shutdown_connections 2024-09-16T15:46:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.835+0000 7f22ab200640 1 -- 172.21.15.60:0/4198395898 wait complete. 2024-09-16T15:46:28.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.837+0000 7f22ab200640 1 Processor -- start 2024-09-16T15:46:28.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.837+0000 7f22ab200640 1 -- start start 2024-09-16T15:46:28.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22ab200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22ac0ae5e0 0x7f22ac0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22ab200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ac15e870 0x7f22ac0c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22ab200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22ac15f360 0x7f22ac0ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22ab200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac074440 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22ab200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac0742c0 con 0x7f22ac15e870 2024-09-16T15:46:28.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22ab200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f22ac0745c0 con 0x7f22ac15f360 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22ac0ae5e0 0x7f22ac0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22ac0ae5e0 0x7f22ac0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44746/0 (socket says 172.21.15.60:44746) 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a9e00640 1 -- 172.21.15.60:0/307803111 learned_addr learned my addr 172.21.15.60:0/307803111 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a1400640 1 --2- 172.21.15.60:0/307803111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ac15e870 0x7f22ac0c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22aa800640 1 --2- 172.21.15.60:0/307803111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22ac15f360 0x7f22ac0ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a9e00640 1 -- 172.21.15.60:0/307803111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22ac15f360 msgr2=0x7f22ac0ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a9e00640 1 --2- 172.21.15.60:0/307803111 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22ac15f360 0x7f22ac0ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a9e00640 1 -- 172.21.15.60:0/307803111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ac15e870 msgr2=0x7f22ac0c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a9e00640 1 --2- 172.21.15.60:0/307803111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22ac15e870 0x7f22ac0c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.838+0000 7f22a9e00640 1 -- 172.21.15.60:0/307803111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac0c9bc0 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.839+0000 7f22a9e00640 1 --2- 172.21.15.60:0/307803111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22ac0ae5e0 0x7f22ac0ce0d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c420 tx=0x7f229c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.839+0000 7f22a2a00640 1 -- 172.21.15.60:0/307803111 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c006890 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.839+0000 7f22a2a00640 1 -- 172.21.15.60:0/307803111 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f229c006b20 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.839+0000 7f22a2a00640 1 -- 172.21.15.60:0/307803111 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c007020 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.839+0000 7f22ab200640 1 -- 172.21.15.60:0/307803111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ac15a490 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.839+0000 7f22ab200640 1 -- 172.21.15.60:0/307803111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ac0c93c0 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.840+0000 7f22ab200640 1 -- 172.21.15.60:0/307803111 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ac06a140 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.841+0000 7f22a2a00640 1 -- 172.21.15.60:0/307803111 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f229c019020 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.842+0000 7f22a2a00640 1 --2- 172.21.15.60:0/307803111 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2288078f10 0x7f228807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:28.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.843+0000 7f22a2a00640 1 -- 172.21.15.60:0/307803111 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f229c09abf0 con 0x7f22ac0ae5e0 2024-09-16T15:46:28.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.843+0000 7f22a1400640 1 --2- 172.21.15.60:0/307803111 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2288078f10 0x7f228807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:28.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.843+0000 7f22a1400640 1 --2- 172.21.15.60:0/307803111 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2288078f10 0x7f228807b3d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0cde30 tx=0x7f2294009060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:28.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:28.847+0000 7f22a2a00640 1 -- 172.21.15.60:0/307803111 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f22ac06a140 con 0x7f22ac0ae5e0 2024-09-16T15:46:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.071+0000 7f22ab200640 1 -- 172.21.15.60:0/307803111 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f22ac06e880 con 0x7f2288078f10 2024-09-16T15:46:29.075 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:29.073+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:29.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.073+0000 7f22a2a00640 1 -- 172.21.15.60:0/307803111 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f22ac06e880 con 0x7f2288078f10 2024-09-16T15:46:29.075 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:29.085 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:29.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-16T15:46:29.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.250+0000 7f65c4800640 1 -- 172.21.15.60:0/306807942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65bc15f870 msgr2=0x7f65bc15fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.250+0000 7f65c4800640 1 --2- 172.21.15.60:0/306807942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65bc15f870 0x7f65bc15fc50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009860 tx=0x7f65b002a5a0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.250+0000 7f65c4800640 1 -- 172.21.15.60:0/306807942 shutdown_connections 2024-09-16T15:46:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.250+0000 7f65c4800640 1 --2- 172.21.15.60:0/306807942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65bc15f870 0x7f65bc15fc50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.250+0000 7f65c4800640 1 -- 172.21.15.60:0/306807942 >> 172.21.15.60:0/306807942 conn(0x7f65bc0b18e0 msgr2=0x7f65bc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.250+0000 7f65c4800640 1 -- 172.21.15.60:0/306807942 shutdown_connections 2024-09-16T15:46:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.250+0000 7f65c4800640 1 -- 172.21.15.60:0/306807942 wait complete. 2024-09-16T15:46:29.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.252+0000 7f65c4800640 1 Processor -- start 2024-09-16T15:46:29.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.252+0000 7f65c4800640 1 -- start start 2024-09-16T15:46:29.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.253+0000 7f65c4800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0ccb00 0x7f65bc1467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.253+0000 7f65c4800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65bc1565b0 0x7f65bc146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.253+0000 7f65c4800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65bc15f870 0x7f65bc0cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.253+0000 7f65c4800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc0739a0 con 0x7f65bc0ccb00 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.253+0000 7f65c4800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc0cd310 con 0x7f65bc1565b0 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.253+0000 7f65c4800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f65bc073b20 con 0x7f65bc15f870 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0ccb00 0x7f65bc1467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0ccb00 0x7f65bc1467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44760/0 (socket says 172.21.15.60:44760) 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 -- 172.21.15.60:0/190865036 learned_addr learned my addr 172.21.15.60:0/190865036 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c2800640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65bc15f870 0x7f65bc0cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1400640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65bc1565b0 0x7f65bc146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 -- 172.21.15.60:0/190865036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65bc15f870 msgr2=0x7f65bc0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65bc15f870 0x7f65bc0cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 -- 172.21.15.60:0/190865036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65bc1565b0 msgr2=0x7f65bc146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65bc1565b0 0x7f65bc146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 -- 172.21.15.60:0/190865036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc0ce850 con 0x7f65bc0ccb00 2024-09-16T15:46:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1400640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65bc1565b0 0x7f65bc146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.254+0000 7f65c1e00640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65bc0ccb00 0x7f65bc1467e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f65b002ab90 tx=0x7f65b0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.255+0000 7f65b6000640 1 -- 172.21.15.60:0/190865036 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0005c90 con 0x7f65bc0ccb00 2024-09-16T15:46:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.255+0000 7f65b6000640 1 -- 172.21.15.60:0/190865036 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65b0031e50 con 0x7f65bc0ccb00 2024-09-16T15:46:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.255+0000 7f65b6000640 1 -- 172.21.15.60:0/190865036 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0005260 con 0x7f65bc0ccb00 2024-09-16T15:46:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.255+0000 7f65c4800640 1 -- 172.21.15.60:0/190865036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc0ceb30 con 0x7f65bc0ccb00 2024-09-16T15:46:29.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.255+0000 7f65c4800640 1 -- 172.21.15.60:0/190865036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc0cf0f0 con 0x7f65bc0ccb00 2024-09-16T15:46:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.256+0000 7f65c4800640 1 -- 172.21.15.60:0/190865036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc15fc50 con 0x7f65bc0ccb00 2024-09-16T15:46:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.257+0000 7f65b6000640 1 -- 172.21.15.60:0/190865036 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f65b0005400 con 0x7f65bc0ccb00 2024-09-16T15:46:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.258+0000 7f65b6000640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65a0078f10 0x7f65a007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.258+0000 7f65b6000640 1 -- 172.21.15.60:0/190865036 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f65b00bc430 con 0x7f65bc0ccb00 2024-09-16T15:46:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.258+0000 7f65c1400640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65a0078f10 0x7f65a007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.259+0000 7f65c1400640 1 --2- 172.21.15.60:0/190865036 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65a0078f10 0x7f65a007b3d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f65bc150940 tx=0x7f65ac009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:29.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.263+0000 7f65b6000640 1 -- 172.21.15.60:0/190865036 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f65b003f030 con 0x7f65bc0ccb00 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.487+0000 7f65c4800640 1 -- 172.21.15.60:0/190865036 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7f65bc070980 con 0x7f65a0078f10 2024-09-16T15:46:29.490 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:29.488+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.488+0000 7f65b6000640 1 -- 172.21.15.60:0/190865036 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f65bc070980 con 0x7f65a0078f10 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:29.500 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:29.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-16T15:46:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 -- 172.21.15.60:0/1277919563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5006efa0 msgr2=0x7fdb500cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 --2- 172.21.15.60:0/1277919563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5006efa0 0x7fdb500cc5c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fdb500d0cd0 tx=0x7fdb3802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 -- 172.21.15.60:0/1277919563 shutdown_connections 2024-09-16T15:46:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 --2- 172.21.15.60:0/1277919563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb500ccb00 0x7fdb5015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 --2- 172.21.15.60:0/1277919563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5006efa0 0x7fdb500cc5c0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 --2- 172.21.15.60:0/1277919563 >> v2:172.21.15.100:3300/0 conn(0x7fdb50145750 0x7fdb50073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 -- 172.21.15.60:0/1277919563 >> 172.21.15.60:0/1277919563 conn(0x7fdb500b1840 msgr2=0x7fdb500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:29.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 -- 172.21.15.60:0/1277919563 shutdown_connections 2024-09-16T15:46:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.661+0000 7fdb54c00640 1 -- 172.21.15.60:0/1277919563 wait complete. 2024-09-16T15:46:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.663+0000 7fdb54c00640 1 Processor -- start 2024-09-16T15:46:29.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.663+0000 7fdb54c00640 1 -- start start 2024-09-16T15:46:29.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb54c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5006efa0 0x7fdb50140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb54c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb500ccb00 0x7fdb50154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb54c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb50145750 0x7fdb5013cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb54c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501602c0 con 0x7fdb5006efa0 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb54c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb50160140 con 0x7fdb50145750 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb54c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdb50160440 con 0x7fdb500ccb00 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb4de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5006efa0 0x7fdb50140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb4de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5006efa0 0x7fdb50140000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44776/0 (socket says 172.21.15.60:44776) 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb4de00640 1 -- 172.21.15.60:0/3353245362 learned_addr learned my addr 172.21.15.60:0/3353245362 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb4e800640 1 --2- 172.21.15.60:0/3353245362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb50145750 0x7fdb5013cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.664+0000 7fdb45400640 1 --2- 172.21.15.60:0/3353245362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb500ccb00 0x7fdb50154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb4de00640 1 -- 172.21.15.60:0/3353245362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb500ccb00 msgr2=0x7fdb50154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb4de00640 1 --2- 172.21.15.60:0/3353245362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb500ccb00 0x7fdb50154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb4de00640 1 -- 172.21.15.60:0/3353245362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb50145750 msgr2=0x7fdb5013cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb4de00640 1 --2- 172.21.15.60:0/3353245362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb50145750 0x7fdb5013cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb4de00640 1 -- 172.21.15.60:0/3353245362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb500c4d00 con 0x7fdb5006efa0 2024-09-16T15:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb4de00640 1 --2- 172.21.15.60:0/3353245362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb5006efa0 0x7fdb50140000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000d740 tx=0x7fdb4000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb46a00640 1 -- 172.21.15.60:0/3353245362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40017070 con 0x7fdb5006efa0 2024-09-16T15:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb46a00640 1 -- 172.21.15.60:0/3353245362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb40009d20 con 0x7fdb5006efa0 2024-09-16T15:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb46a00640 1 -- 172.21.15.60:0/3353245362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb400106c0 con 0x7fdb5006efa0 2024-09-16T15:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.665+0000 7fdb54c00640 1 -- 172.21.15.60:0/3353245362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb500c4fe0 con 0x7fdb5006efa0 2024-09-16T15:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.666+0000 7fdb54c00640 1 -- 172.21.15.60:0/3353245362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb500c52c0 con 0x7fdb5006efa0 2024-09-16T15:46:29.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.667+0000 7fdb54c00640 1 -- 172.21.15.60:0/3353245362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5015f9f0 con 0x7fdb5006efa0 2024-09-16T15:46:29.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.669+0000 7fdb46a00640 1 -- 172.21.15.60:0/3353245362 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fdb4001a070 con 0x7fdb5006efa0 2024-09-16T15:46:29.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.670+0000 7fdb46a00640 1 --2- 172.21.15.60:0/3353245362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb2c078f10 0x7fdb2c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.670+0000 7fdb46a00640 1 -- 172.21.15.60:0/3353245362 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fdb4009b5e0 con 0x7fdb5006efa0 2024-09-16T15:46:29.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.670+0000 7fdb45400640 1 --2- 172.21.15.60:0/3353245362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb2c078f10 0x7fdb2c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.670+0000 7fdb45400640 1 --2- 172.21.15.60:0/3353245362 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb2c078f10 0x7fdb2c07b3d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb38033000 tx=0x7fdb38004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:29.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.674+0000 7fdb46a00640 1 -- 172.21.15.60:0/3353245362 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdb40062c40 con 0x7fdb5006efa0 2024-09-16T15:46:29.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.896+0000 7fdb54c00640 1 -- 172.21.15.60:0/3353245362 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7fdb50140700 con 0x7fdb2c078f10 2024-09-16T15:46:29.900 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:29.898+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:29.898+0000 7fdb46a00640 1 -- 172.21.15.60:0/3353245362 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fdb50140700 con 0x7fdb2c078f10 2024-09-16T15:46:29.900 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:29.909 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:29.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-16T15:46:30.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.074+0000 7f842fe00640 1 -- 172.21.15.60:0/2938503322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f843006e590 msgr2=0x7f843006fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.074+0000 7f842fe00640 1 --2- 172.21.15.60:0/2938503322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f843006e590 0x7f843006fe40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f841c009830 tx=0x7f841c02a520 comp rx=0 tx=0).stop 2024-09-16T15:46:30.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.075+0000 7f842fe00640 1 -- 172.21.15.60:0/2938503322 shutdown_connections 2024-09-16T15:46:30.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.075+0000 7f842fe00640 1 --2- 172.21.15.60:0/2938503322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f843006e590 0x7f843006fe40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.075+0000 7f842fe00640 1 --2- 172.21.15.60:0/2938503322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bda0 0x7f8430073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.075+0000 7f842fe00640 1 -- 172.21.15.60:0/2938503322 >> 172.21.15.60:0/2938503322 conn(0x7f84300cfa70 msgr2=0x7f84300cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.075+0000 7f842fe00640 1 -- 172.21.15.60:0/2938503322 shutdown_connections 2024-09-16T15:46:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.075+0000 7f842fe00640 1 -- 172.21.15.60:0/2938503322 wait complete. 2024-09-16T15:46:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.076+0000 7f842fe00640 1 Processor -- start 2024-09-16T15:46:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.076+0000 7f842fe00640 1 -- start start 2024-09-16T15:46:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.077+0000 7f842fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843006e590 0x7f84300ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.077+0000 7f842fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8430070380 0x7f8430151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.077+0000 7f842fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015bda0 0x7f84301f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.077+0000 7f842fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f84300cc350 con 0x7f843015bda0 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.077+0000 7f842fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84300cc1d0 con 0x7f8430070380 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.077+0000 7f842fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f84300740e0 con 0x7f843006e590 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015bda0 0x7f84301f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015bda0 0x7f84301f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44786/0 (socket says 172.21.15.60:44786) 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 -- 172.21.15.60:0/2840719945 learned_addr learned my addr 172.21.15.60:0/2840719945 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842ea00640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843006e590 0x7f84300ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842e000640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8430070380 0x7f8430151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 -- 172.21.15.60:0/2840719945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843006e590 msgr2=0x7f84300ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843006e590 0x7f84300ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 -- 172.21.15.60:0/2840719945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8430070380 msgr2=0x7f8430151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8430070380 0x7f8430151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 -- 172.21.15.60:0/2840719945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84300cd640 con 0x7f843015bda0 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842e000640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8430070380 0x7f8430151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.078+0000 7f842f400640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f843015bda0 0x7f84301f8260 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f842800c7d0 tx=0x7f842800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.079+0000 7f8426a00640 1 -- 172.21.15.60:0/2840719945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428003b70 con 0x7f843015bda0 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.079+0000 7f8426a00640 1 -- 172.21.15.60:0/2840719945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8428003d10 con 0x7f843015bda0 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.079+0000 7f842fe00640 1 -- 172.21.15.60:0/2840719945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84301f8db0 con 0x7f843015bda0 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.079+0000 7f8426a00640 1 -- 172.21.15.60:0/2840719945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800aab0 con 0x7f843015bda0 2024-09-16T15:46:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.079+0000 7f842fe00640 1 -- 172.21.15.60:0/2840719945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84300cd900 con 0x7f843015bda0 2024-09-16T15:46:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.080+0000 7f842fe00640 1 -- 172.21.15.60:0/2840719945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84300af240 con 0x7f843015bda0 2024-09-16T15:46:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.082+0000 7f8426a00640 1 -- 172.21.15.60:0/2840719945 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f842800ac50 con 0x7f843015bda0 2024-09-16T15:46:30.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.082+0000 7f8426a00640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8410078f10 0x7f841007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.082+0000 7f8426a00640 1 -- 172.21.15.60:0/2840719945 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f842809b2a0 con 0x7f843015bda0 2024-09-16T15:46:30.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.082+0000 7f842ea00640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8410078f10 0x7f841007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.083+0000 7f842ea00640 1 --2- 172.21.15.60:0/2840719945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8410078f10 0x7f841007b3d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8420009530 tx=0x7f8420009260 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:30.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.086+0000 7f8426a00640 1 -- 172.21.15.60:0/2840719945 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f84300af240 con 0x7f843015bda0 2024-09-16T15:46:30.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.307+0000 7f842fe00640 1 -- 172.21.15.60:0/2840719945 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f84300aeb40 con 0x7f8410078f10 2024-09-16T15:46:30.310 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:30.308+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.308+0000 7f8426a00640 1 -- 172.21.15.60:0/2840719945 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f84300aeb40 con 0x7f8410078f10 2024-09-16T15:46:30.310 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:30.320 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:30.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-16T15:46:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.484+0000 7f09cbc00640 1 -- 172.21.15.60:0/3754835051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a8003a00 msgr2=0x7f09a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.484+0000 7f09cbc00640 1 --2- 172.21.15.60:0/3754835051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a8003a00 0x7f09a8003e20 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c290 tx=0x7f09a8005830 comp rx=0 tx=0).stop 2024-09-16T15:46:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.484+0000 7f09c9c00640 1 -- 172.21.15.60:0/3754835051 reap_dead start 2024-09-16T15:46:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.485+0000 7f09cbc00640 1 -- 172.21.15.60:0/3754835051 shutdown_connections 2024-09-16T15:46:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.485+0000 7f09cbc00640 1 -- 172.21.15.60:0/3754835051 >> 172.21.15.60:0/3754835051 conn(0x7f09c406cb70 msgr2=0x7f09c406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.485+0000 7f09cbc00640 1 -- 172.21.15.60:0/3754835051 shutdown_connections 2024-09-16T15:46:30.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.485+0000 7f09cbc00640 1 -- 172.21.15.60:0/3754835051 wait complete. 2024-09-16T15:46:30.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.486+0000 7f09cbc00640 1 Processor -- start 2024-09-16T15:46:30.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.487+0000 7f09cbc00640 1 -- start start 2024-09-16T15:46:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.487+0000 7f09cbc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a8003a00 0x7f09c4149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.487+0000 7f09cbc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c41ea950 0x7f09c4157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.487+0000 7f09cbc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09c41e87e0 0x7f09c41e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.487+0000 7f09cbc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f09c406f2a0 con 0x7f09c41ea950 2024-09-16T15:46:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.487+0000 7f09cbc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09c40b3c00 con 0x7f09a8003a00 2024-09-16T15:46:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.487+0000 7f09cbc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f09c406f420 con 0x7f09c41e87e0 2024-09-16T15:46:30.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c41ea950 0x7f09c4157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c41ea950 0x7f09c4157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44806/0 (socket says 172.21.15.60:44806) 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 -- 172.21.15.60:0/2856756411 learned_addr learned my addr 172.21.15.60:0/2856756411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c9200640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a8003a00 0x7f09c4149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c9c00640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09c41e87e0 0x7f09c41e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 -- 172.21.15.60:0/2856756411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09c41e87e0 msgr2=0x7f09c41e8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09c41e87e0 0x7f09c41e8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 -- 172.21.15.60:0/2856756411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a8003a00 msgr2=0x7f09c4149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a8003a00 0x7f09c4149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 -- 172.21.15.60:0/2856756411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c41583d0 con 0x7f09c41ea950 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c9c00640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09c41e87e0 0x7f09c41e8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:30.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.488+0000 7f09c3e00640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09c41ea950 0x7f09c4157c30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f09b0011b10 tx=0x7f09b0009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.489+0000 7f09c0c00640 1 -- 172.21.15.60:0/2856756411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b001e070 con 0x7f09c41ea950 2024-09-16T15:46:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.489+0000 7f09c0c00640 1 -- 172.21.15.60:0/2856756411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09b000fc40 con 0x7f09c41ea950 2024-09-16T15:46:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.489+0000 7f09cbc00640 1 -- 172.21.15.60:0/2856756411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c41586b0 con 0x7f09c41ea950 2024-09-16T15:46:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.489+0000 7f09c0c00640 1 -- 172.21.15.60:0/2856756411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0019410 con 0x7f09c41ea950 2024-09-16T15:46:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.489+0000 7f09cbc00640 1 -- 172.21.15.60:0/2856756411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c4158c70 con 0x7f09c41ea950 2024-09-16T15:46:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.490+0000 7f09cbc00640 1 -- 172.21.15.60:0/2856756411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c4073070 con 0x7f09c41ea950 2024-09-16T15:46:30.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.491+0000 7f09c0c00640 1 -- 172.21.15.60:0/2856756411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f09b0021070 con 0x7f09c41ea950 2024-09-16T15:46:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.494+0000 7f09c0c00640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f09a4078f40 0x7f09a407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.494+0000 7f09c0c00640 1 -- 172.21.15.60:0/2856756411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f09b009bd80 con 0x7f09c41ea950 2024-09-16T15:46:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.494+0000 7f09c9200640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f09a4078f40 0x7f09a407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.495+0000 7f09c9200640 1 --2- 172.21.15.60:0/2856756411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f09a4078f40 0x7f09a407b400 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f09b4033000 tx=0x7f09b4038040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:30.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.498+0000 7f09c0c00640 1 -- 172.21.15.60:0/2856756411 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f09b00632a0 con 0x7f09c41ea950 2024-09-16T15:46:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.721+0000 7f09cbc00640 1 -- 172.21.15.60:0/2856756411 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f09c4158170 con 0x7f09a4078f40 2024-09-16T15:46:30.725 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:30.723+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.723+0000 7f09c0c00640 1 -- 172.21.15.60:0/2856756411 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f09b4001530 con 0x7f09a4078f40 2024-09-16T15:46:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:30.735 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:30.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-16T15:46:30.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.901+0000 7f81dfc00640 1 -- 172.21.15.60:0/2229888589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81d806f220 msgr2=0x7f81d806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.901+0000 7f81dfc00640 1 --2- 172.21.15.60:0/2229888589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81d806f220 0x7f81d806f6d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009ca0 tx=0x7f81d002aa30 comp rx=0 tx=0).stop 2024-09-16T15:46:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.901+0000 7f81dfc00640 1 -- 172.21.15.60:0/2229888589 shutdown_connections 2024-09-16T15:46:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.901+0000 7f81dfc00640 1 --2- 172.21.15.60:0/2229888589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81d806f220 0x7f81d806f6d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.901+0000 7f81dfc00640 1 --2- 172.21.15.60:0/2229888589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d815cff0 0x7f81d806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.901+0000 7f81dfc00640 1 -- 172.21.15.60:0/2229888589 >> 172.21.15.60:0/2229888589 conn(0x7f81d80b18e0 msgr2=0x7f81d80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.901+0000 7f81dfc00640 1 -- 172.21.15.60:0/2229888589 shutdown_connections 2024-09-16T15:46:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.901+0000 7f81dfc00640 1 -- 172.21.15.60:0/2229888589 wait complete. 2024-09-16T15:46:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.903+0000 7f81dfc00640 1 Processor -- start 2024-09-16T15:46:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.903+0000 7f81dfc00640 1 -- start start 2024-09-16T15:46:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dfc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81d806f220 0x7f81d80ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dfc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d8159aa0 0x7f81d80ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dfc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d815cff0 0x7f81d81f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dfc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f81d80cc520 con 0x7f81d806f220 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dfc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f81d80cc3a0 con 0x7f81d8159aa0 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dfc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f81d80cc6a0 con 0x7f81d815cff0 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dd200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81d806f220 0x7f81d80ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dd200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81d806f220 0x7f81d80ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44834/0 (socket says 172.21.15.60:44834) 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.904+0000 7f81dd200640 1 -- 172.21.15.60:0/465447966 learned_addr learned my addr 172.21.15.60:0/465447966 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81dd200640 1 -- 172.21.15.60:0/465447966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d815cff0 msgr2=0x7f81d81f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81ddc00640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d815cff0 0x7f81d81f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81dd200640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d815cff0 0x7f81d81f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81dd200640 1 -- 172.21.15.60:0/465447966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d8159aa0 msgr2=0x7f81d80ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81d7e00640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d8159aa0 0x7f81d80ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81dd200640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d8159aa0 0x7f81d80ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81dd200640 1 -- 172.21.15.60:0/465447966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d809d400 con 0x7f81d806f220 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81ddc00640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81d815cff0 0x7f81d81f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81d7e00640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81d8159aa0 0x7f81d80ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81dd200640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81d806f220 0x7f81d80ad390 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f81c800c430 tx=0x7f81c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81d4c00640 1 -- 172.21.15.60:0/465447966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8011790 con 0x7f81d806f220 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81d4c00640 1 -- 172.21.15.60:0/465447966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81c8017070 con 0x7f81d806f220 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81d4c00640 1 -- 172.21.15.60:0/465447966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8011d00 con 0x7f81d806f220 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81dfc00640 1 -- 172.21.15.60:0/465447966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d809d6e0 con 0x7f81d806f220 2024-09-16T15:46:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.905+0000 7f81dfc00640 1 -- 172.21.15.60:0/465447966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d815a450 con 0x7f81d806f220 2024-09-16T15:46:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.906+0000 7f81dfc00640 1 -- 172.21.15.60:0/465447966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d815a650 con 0x7f81d806f220 2024-09-16T15:46:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.908+0000 7f81d4c00640 1 -- 172.21.15.60:0/465447966 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f81c801c460 con 0x7f81d806f220 2024-09-16T15:46:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.909+0000 7f81d4c00640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f81b8078f10 0x7f81b807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.909+0000 7f81d4c00640 1 -- 172.21.15.60:0/465447966 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f81c809b4a0 con 0x7f81d806f220 2024-09-16T15:46:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.909+0000 7f81d7e00640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f81b8078f10 0x7f81b807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.910+0000 7f81d7e00640 1 --2- 172.21.15.60:0/465447966 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f81b8078f10 0x7f81b807b3d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f81d80cd6c0 tx=0x7f81c4005c90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:30.913+0000 7f81d4c00640 1 -- 172.21.15.60:0/465447966 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f81d815a650 con 0x7f81d806f220 2024-09-16T15:46:31.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.133+0000 7f81dfc00640 1 -- 172.21.15.60:0/465447966 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f81d8070980 con 0x7f81b8078f10 2024-09-16T15:46:31.136 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:31.135+0000 7fcb85a00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-16T15:46:31.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.135+0000 7f81d4c00640 1 -- 172.21.15.60:0/465447966 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f81d8070980 con 0x7f81b8078f10 2024-09-16T15:46:31.136 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-16T15:46:31.146 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:46:31.146 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:46:31.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:46:31.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:46:31.343 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:46:31.343 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:46:31.343 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:46:31.343 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:31.379 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:46:31.379 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:31.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:46:31.583 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:46:31.587 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:46:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.656+0000 7fe5e8200640 1 -- 172.21.15.60:0/3240078228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e006ff60 msgr2=0x7fe5e00b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.656+0000 7fe5e8200640 1 --2- 172.21.15.60:0/3240078228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e006ff60 0x7fe5e00b4610 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8005fd0 tx=0x7fe5d802a940 comp rx=0 tx=0).stop 2024-09-16T15:46:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.656+0000 7fe5e8200640 1 -- 172.21.15.60:0/3240078228 shutdown_connections 2024-09-16T15:46:31.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.656+0000 7fe5e8200640 1 --2- 172.21.15.60:0/3240078228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e006ff60 0x7fe5e00b4610 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.656+0000 7fe5e8200640 1 -- 172.21.15.60:0/3240078228 >> 172.21.15.60:0/3240078228 conn(0x7fe5e00b1950 msgr2=0x7fe5e00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:31.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.656+0000 7fe5e8200640 1 -- 172.21.15.60:0/3240078228 shutdown_connections 2024-09-16T15:46:31.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.656+0000 7fe5e8200640 1 -- 172.21.15.60:0/3240078228 wait complete. 2024-09-16T15:46:31.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.658+0000 7fe5e8200640 1 Processor -- start 2024-09-16T15:46:31.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.658+0000 7fe5e8200640 1 -- start start 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e8200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e006ff60 0x7fe5e0159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e8200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e0070920 0x7fe5e01644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e8200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e01438b0 0x7fe5e0140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e8200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e00b4ee0 con 0x7fe5e01438b0 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e8200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e00b4d60 con 0x7fe5e0070920 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e8200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e00ccf30 con 0x7fe5e006ff60 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e6200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e01438b0 0x7fe5e0140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e6200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e01438b0 0x7fe5e0140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44844/0 (socket says 172.21.15.60:44844) 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e6200640 1 -- 172.21.15.60:0/2396194630 learned_addr learned my addr 172.21.15.60:0/2396194630 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e4e00640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e0070920 0x7fe5e01644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e5800640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e006ff60 0x7fe5e0159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:31.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e6200640 1 -- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e006ff60 msgr2=0x7fe5e0159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e6200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e006ff60 0x7fe5e0159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e6200640 1 -- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e0070920 msgr2=0x7fe5e01644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e6200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e0070920 0x7fe5e01644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e6200640 1 -- 172.21.15.60:0/2396194630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e0164a20 con 0x7fe5e01438b0 2024-09-16T15:46:31.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.659+0000 7fe5e5800640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e006ff60 0x7fe5e0159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:31.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.660+0000 7fe5e6200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e01438b0 0x7fe5e0140b70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8006010 tx=0x7fe5d8031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:31.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.660+0000 7fe5dd600640 1 -- 172.21.15.60:0/2396194630 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8004240 con 0x7fe5e01438b0 2024-09-16T15:46:31.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.660+0000 7fe5dd600640 1 -- 172.21.15.60:0/2396194630 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5d80043e0 con 0x7fe5e01438b0 2024-09-16T15:46:31.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.660+0000 7fe5dd600640 1 -- 172.21.15.60:0/2396194630 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8004830 con 0x7fe5e01438b0 2024-09-16T15:46:31.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.660+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e01402f0 con 0x7fe5e01438b0 2024-09-16T15:46:31.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.660+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e0163ce0 con 0x7fe5e01438b0 2024-09-16T15:46:31.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.661+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e006f850 con 0x7fe5e01438b0 2024-09-16T15:46:31.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.663+0000 7fe5dd600640 1 -- 172.21.15.60:0/2396194630 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fe5d8039440 con 0x7fe5e01438b0 2024-09-16T15:46:31.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.663+0000 7fe5dd600640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078f40 0x7fe5c007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:31.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.664+0000 7fe5dd600640 1 -- 172.21.15.60:0/2396194630 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fe5d80bcf60 con 0x7fe5e01438b0 2024-09-16T15:46:31.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.664+0000 7fe5e5800640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078f40 0x7fe5c007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:31.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.664+0000 7fe5e5800640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078f40 0x7fe5c007b400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40046b0 tx=0x7fe5d4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:31.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.668+0000 7fe5dd600640 1 -- 172.21.15.60:0/2396194630 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe5d8084c10 con 0x7fe5e01438b0 2024-09-16T15:46:31.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.875+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe5e015e140 con 0x7fe5e01438b0 2024-09-16T15:46:31.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.876+0000 7fe5dd600640 1 -- 172.21.15.60:0/2396194630 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v137) ==== 80+0+0 (secure 0 0 0) 0x7fe5d8084510 con 0x7fe5e01438b0 2024-09-16T15:46:31.877 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:46:31.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078f40 msgr2=0x7fe5c007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078f40 0x7fe5c007b400 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40046b0 tx=0x7fe5d4009040 comp rx=0 tx=0).stop 2024-09-16T15:46:31.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e01438b0 msgr2=0x7fe5e0140b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e01438b0 0x7fe5e0140b70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8006010 tx=0x7fe5d8031e20 comp rx=0 tx=0).stop 2024-09-16T15:46:31.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 shutdown_connections 2024-09-16T15:46:31.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe5c0078f40 0x7fe5c007b400 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5e01438b0 0x7fe5e0140b70 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5e0070920 0x7fe5e01644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 --2- 172.21.15.60:0/2396194630 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5e006ff60 0x7fe5e0159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 >> 172.21.15.60:0/2396194630 conn(0x7fe5e00b1950 msgr2=0x7fe5e00ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.878+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 shutdown_connections 2024-09-16T15:46:31.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:31.879+0000 7fe5e8200640 1 -- 172.21.15.60:0/2396194630 wait complete. 2024-09-16T15:46:31.895 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:46:31.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:31.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:31.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:46:31.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:31.958 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:46:31.995 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:46:31.995 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:46:31.995 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:32.035 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:46:32.035 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:32.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:46:32.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.273+0000 7efdbba00640 1 -- 172.21.15.60:0/3209486779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4161550 msgr2=0x7efdb406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.273+0000 7efdbba00640 1 --2- 172.21.15.60:0/3209486779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4161550 0x7efdb406ef50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7efda8005fd0 tx=0x7efda802a540 comp rx=0 tx=0).stop 2024-09-16T15:46:32.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.273+0000 7efdbba00640 1 -- 172.21.15.60:0/3209486779 shutdown_connections 2024-09-16T15:46:32.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.273+0000 7efdbba00640 1 --2- 172.21.15.60:0/3209486779 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4161550 0x7efdb406ef50 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.273+0000 7efdbba00640 1 -- 172.21.15.60:0/3209486779 >> 172.21.15.60:0/3209486779 conn(0x7efdb40b1950 msgr2=0x7efdb40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:32.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.273+0000 7efdbba00640 1 -- 172.21.15.60:0/3209486779 shutdown_connections 2024-09-16T15:46:32.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.273+0000 7efdbba00640 1 -- 172.21.15.60:0/3209486779 wait complete. 2024-09-16T15:46:32.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.275+0000 7efdbba00640 1 Processor -- start 2024-09-16T15:46:32.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.275+0000 7efdbba00640 1 -- start start 2024-09-16T15:46:32.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.275+0000 7efdbba00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4072df0 0x7efdb4146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdbba00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efdb40af3d0 0x7efdb41f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdbba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdb4161550 0x7efdb40d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdbba00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40739b0 con 0x7efdb4072df0 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdbba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efdb406fdb0 con 0x7efdb4161550 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdbba00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7efdb4073b30 con 0x7efdb40af3d0 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4072df0 0x7efdb4146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4072df0 0x7efdb4146110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44858/0 (socket says 172.21.15.60:44858) 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9000640 1 -- 172.21.15.60:0/1416760543 learned_addr learned my addr 172.21.15.60:0/1416760543 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9000640 1 -- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efdb40af3d0 msgr2=0x7efdb41f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9a00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdb4161550 0x7efdb40d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb3e00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efdb40af3d0 0x7efdb41f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9000640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efdb40af3d0 0x7efdb41f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9000640 1 -- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdb4161550 msgr2=0x7efdb40d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9000640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdb4161550 0x7efdb40d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9000640 1 -- 172.21.15.60:0/1416760543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb41490e0 con 0x7efdb4072df0 2024-09-16T15:46:32.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb9a00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdb4161550 0x7efdb40d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:32.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.276+0000 7efdb3e00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efdb40af3d0 0x7efdb41f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:32.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.277+0000 7efdb9000640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4072df0 0x7efdb4146110 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7efda8034040 tx=0x7efda8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.277+0000 7efdb0c00640 1 -- 172.21.15.60:0/1416760543 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda803e030 con 0x7efdb4072df0 2024-09-16T15:46:32.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.277+0000 7efdb0c00640 1 -- 172.21.15.60:0/1416760543 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efda80047a0 con 0x7efdb4072df0 2024-09-16T15:46:32.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.277+0000 7efdb0c00640 1 -- 172.21.15.60:0/1416760543 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8035040 con 0x7efdb4072df0 2024-09-16T15:46:32.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.277+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb41f57c0 con 0x7efdb4072df0 2024-09-16T15:46:32.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.277+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb4148b20 con 0x7efdb4072df0 2024-09-16T15:46:32.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.278+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb40af8c0 con 0x7efdb4072df0 2024-09-16T15:46:32.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.279+0000 7efdb0c00640 1 -- 172.21.15.60:0/1416760543 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7efda8041030 con 0x7efdb4072df0 2024-09-16T15:46:32.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.280+0000 7efdb0c00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd94078f10 0x7efd9407b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.280+0000 7efdb0c00640 1 -- 172.21.15.60:0/1416760543 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7efda80bbee0 con 0x7efdb4072df0 2024-09-16T15:46:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.281+0000 7efdb3e00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd94078f10 0x7efd9407b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.281+0000 7efdb3e00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd94078f10 0x7efd9407b3d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efda0007fd0 tx=0x7efda0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.284+0000 7efdb0c00640 1 -- 172.21.15.60:0/1416760543 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efdb40af8c0 con 0x7efdb4072df0 2024-09-16T15:46:32.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.493+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efdb4074040 con 0x7efdb4072df0 2024-09-16T15:46:32.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.494+0000 7efdb0c00640 1 -- 172.21.15.60:0/1416760543 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v137) ==== 80+0+0 (secure 0 0 0) 0x7efdb4074040 con 0x7efdb4072df0 2024-09-16T15:46:32.495 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:46:32.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd94078f10 msgr2=0x7efd9407b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd94078f10 0x7efd9407b3d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efda0007fd0 tx=0x7efda0009040 comp rx=0 tx=0).stop 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4072df0 msgr2=0x7efdb4146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4072df0 0x7efdb4146110 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7efda8034040 tx=0x7efda8003970 comp rx=0 tx=0).stop 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 shutdown_connections 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd94078f10 0x7efd9407b3d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efdb4161550 0x7efdb40d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efdb40af3d0 0x7efdb41f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 --2- 172.21.15.60:0/1416760543 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efdb4072df0 0x7efdb4146110 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 >> 172.21.15.60:0/1416760543 conn(0x7efdb40b1950 msgr2=0x7efdb415ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 shutdown_connections 2024-09-16T15:46:32.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.496+0000 7efdbba00640 1 -- 172.21.15.60:0/1416760543 wait complete. 2024-09-16T15:46:32.513 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:46:32.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:32.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:32.583 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:46:32.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:32.583 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:46:32.620 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:46:32.620 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:46:32.620 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:32.656 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:46:32.656 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:46:32.656 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:46:32.656 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:32.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:46:32.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:32.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.857+0000 7f20dec00640 1 -- 172.21.15.60:0/2335636466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20d815bc30 msgr2=0x7f20d80b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.857+0000 7f20dec00640 1 --2- 172.21.15.60:0/2335636466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20d815bc30 0x7f20d80b28f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009860 tx=0x7f20cc02a5a0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.857+0000 7f20dec00640 1 -- 172.21.15.60:0/2335636466 shutdown_connections 2024-09-16T15:46:32.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.857+0000 7f20dec00640 1 --2- 172.21.15.60:0/2335636466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20d815bc30 0x7f20d80b28f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.857+0000 7f20dec00640 1 -- 172.21.15.60:0/2335636466 >> 172.21.15.60:0/2335636466 conn(0x7f20d80cfa20 msgr2=0x7f20d80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:32.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.857+0000 7f20dec00640 1 -- 172.21.15.60:0/2335636466 shutdown_connections 2024-09-16T15:46:32.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.857+0000 7f20dec00640 1 -- 172.21.15.60:0/2335636466 wait complete. 2024-09-16T15:46:32.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.858+0000 7f20dec00640 1 Processor -- start 2024-09-16T15:46:32.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.859+0000 7f20dec00640 1 -- start start 2024-09-16T15:46:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.859+0000 7f20dec00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20d80d0540 0x7f20d815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20dec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d815bc30 0x7f20d81f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20dec00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d81f3d90 0x7f20d8157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20dec00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8074660 con 0x7f20d81f3d90 2024-09-16T15:46:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20dec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f20d806fa30 con 0x7f20d815bc30 2024-09-16T15:46:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20dec00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f20d815c7f0 con 0x7f20d80d0540 2024-09-16T15:46:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20de200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d81f3d90 0x7f20d8157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20de200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d81f3d90 0x7f20d8157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44872/0 (socket says 172.21.15.60:44872) 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20de200640 1 -- 172.21.15.60:0/1860462903 learned_addr learned my addr 172.21.15.60:0/1860462903 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20dd800640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20d80d0540 0x7f20d815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20dce00640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d815bc30 0x7f20d81f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20de200640 1 -- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20d80d0540 msgr2=0x7f20d815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20de200640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20d80d0540 0x7f20d815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20de200640 1 -- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d815bc30 msgr2=0x7f20d81f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20de200640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d815bc30 0x7f20d81f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20de200640 1 -- 172.21.15.60:0/1860462903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d81f8a80 con 0x7f20d81f3d90 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.860+0000 7f20dce00640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d815bc30 0x7f20d81f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.861+0000 7f20de200640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d81f3d90 0x7f20d8157b40 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f20d000c780 tx=0x7f20d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.861+0000 7f20d5600640 1 -- 172.21.15.60:0/1860462903 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0003b70 con 0x7f20d81f3d90 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.861+0000 7f20d5600640 1 -- 172.21.15.60:0/1860462903 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20d0003d10 con 0x7f20d81f3d90 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.861+0000 7f20d5600640 1 -- 172.21.15.60:0/1860462903 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d000aab0 con 0x7f20d81f3d90 2024-09-16T15:46:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.861+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d815e2e0 con 0x7f20d81f3d90 2024-09-16T15:46:32.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.861+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d815eb20 con 0x7f20d81f3d90 2024-09-16T15:46:32.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.862+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d80b28f0 con 0x7f20d81f3d90 2024-09-16T15:46:32.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.863+0000 7f20d5600640 1 -- 172.21.15.60:0/1860462903 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f20d000ac50 con 0x7f20d81f3d90 2024-09-16T15:46:32.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.864+0000 7f20d5600640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20b8078f10 0x7f20b807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.864+0000 7f20d5600640 1 -- 172.21.15.60:0/1860462903 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f20d009b420 con 0x7f20d81f3d90 2024-09-16T15:46:32.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.864+0000 7f20dd800640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20b8078f10 0x7f20b807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.865+0000 7f20dd800640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20b8078f10 0x7f20b807b3d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f20cc034000 tx=0x7f20cc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:32.868+0000 7f20d5600640 1 -- 172.21.15.60:0/1860462903 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f20d815e2e0 con 0x7f20d81f3d90 2024-09-16T15:46:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.102+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20d80cae20 con 0x7f20d81f3d90 2024-09-16T15:46:33.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.103+0000 7f20d5600640 1 -- 172.21.15.60:0/1860462903 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5227 (secure 0 0 0) 0x7f20d81f8a80 con 0x7f20d81f3d90 2024-09-16T15:46:33.104 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:33.104 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"btime":"2024-09-16T15:46:20:740803+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/766495901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":766495901},{"type":"v1","addr":"172.21.15.100:6835","nonce":766495901}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7585,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/3797165836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3797165836},{"type":"v1","addr":"172.21.15.100:6801","nonce":3797165836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3900896825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3900896825},{"type":"v1","addr":"172.21.15.60:6834","nonce":3900896825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:10.124891+0000","modified":"2024-09-16T15:46:13.158701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/984341325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":984341325},{"type":"v1","addr":"172.21.15.60:6835","nonce":984341325}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7998,"qdb_cluster":[7998]},"id":10}]} 2024-09-16T15:46:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 93 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20b8078f10 msgr2=0x7f20b807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20b8078f10 0x7f20b807b3d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f20cc034000 tx=0x7f20cc004730 comp rx=0 tx=0).stop 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d81f3d90 msgr2=0x7f20d8157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d81f3d90 0x7f20d8157b40 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f20d000c780 tx=0x7f20d000cc40 comp rx=0 tx=0).stop 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 shutdown_connections 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f20b8078f10 0x7f20b807b3d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20d81f3d90 0x7f20d8157b40 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d815bc30 0x7f20d81f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 --2- 172.21.15.60:0/1860462903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20d80d0540 0x7f20d815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.105+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 >> 172.21.15.60:0/1860462903 conn(0x7f20d80cfa20 msgr2=0x7f20d80b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.106+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 shutdown_connections 2024-09-16T15:46:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.106+0000 7f20dec00640 1 -- 172.21.15.60:0/1860462903 wait complete. 2024-09-16T15:46:33.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.283+0000 7fc08f000640 1 -- 172.21.15.60:0/1028444141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 msgr2=0x7fc0880b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.283+0000 7fc08f000640 1 --2- 172.21.15.60:0/1028444141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 0x7fc0880b28f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009830 tx=0x7fc07c02a510 comp rx=0 tx=0).stop 2024-09-16T15:46:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.283+0000 7fc08f000640 1 -- 172.21.15.60:0/1028444141 shutdown_connections 2024-09-16T15:46:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.283+0000 7fc08f000640 1 --2- 172.21.15.60:0/1028444141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0880d0540 0x7fc08806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.283+0000 7fc08f000640 1 --2- 172.21.15.60:0/1028444141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 0x7fc0880b28f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.283+0000 7fc08f000640 1 -- 172.21.15.60:0/1028444141 >> 172.21.15.60:0/1028444141 conn(0x7fc0880cfa20 msgr2=0x7fc0880cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.283+0000 7fc08f000640 1 -- 172.21.15.60:0/1028444141 shutdown_connections 2024-09-16T15:46:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.283+0000 7fc08f000640 1 -- 172.21.15.60:0/1028444141 wait complete. 2024-09-16T15:46:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.285+0000 7fc08f000640 1 Processor -- start 2024-09-16T15:46:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.285+0000 7fc08f000640 1 -- start start 2024-09-16T15:46:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08f000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0880d0540 0x7fc0880b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08f000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 0x7fc08815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08f000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc088145d20 0x7fc08815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08f000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc088074660 con 0x7fc088145d20 2024-09-16T15:46:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08f000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc08806f860 con 0x7fc08815bc30 2024-09-16T15:46:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08f000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc0881f5430 con 0x7fc0880d0540 2024-09-16T15:46:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc088145d20 0x7fc08815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc088145d20 0x7fc08815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44876/0 (socket says 172.21.15.60:44876) 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.286+0000 7fc08e600640 1 -- 172.21.15.60:0/2642826376 learned_addr learned my addr 172.21.15.60:0/2642826376 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08dc00640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0880d0540 0x7fc0880b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08d200640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 0x7fc08815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08e600640 1 -- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0880d0540 msgr2=0x7fc0880b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08e600640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0880d0540 0x7fc0880b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08e600640 1 -- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 msgr2=0x7fc08815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08e600640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 0x7fc08815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08e600640 1 -- 172.21.15.60:0/2642826376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0880c7ce0 con 0x7fc088145d20 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08d200640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 0x7fc08815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08dc00640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0880d0540 0x7fc0880b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc08e600640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc088145d20 0x7fc08815f0a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc08000eb10 tx=0x7fc08000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:33.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.287+0000 7fc085600640 1 -- 172.21.15.60:0/2642826376 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080004b20 con 0x7fc088145d20 2024-09-16T15:46:33.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.288+0000 7fc085600640 1 -- 172.21.15.60:0/2642826376 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc080004ce0 con 0x7fc088145d20 2024-09-16T15:46:33.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.288+0000 7fc085600640 1 -- 172.21.15.60:0/2642826376 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080005280 con 0x7fc088145d20 2024-09-16T15:46:33.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.288+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0880c7fc0 con 0x7fc088145d20 2024-09-16T15:46:33.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.288+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0880b12f0 con 0x7fc088145d20 2024-09-16T15:46:33.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.289+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08806fdf0 con 0x7fc088145d20 2024-09-16T15:46:33.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.290+0000 7fc085600640 1 -- 172.21.15.60:0/2642826376 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fc080005420 con 0x7fc088145d20 2024-09-16T15:46:33.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.291+0000 7fc085600640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc068078f10 0x7fc06807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.292+0000 7fc085600640 1 -- 172.21.15.60:0/2642826376 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7fc08009c380 con 0x7fc088145d20 2024-09-16T15:46:33.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.292+0000 7fc08dc00640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc068078f10 0x7fc06807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.292+0000 7fc08dc00640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc068078f10 0x7fc06807b3d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc07c034000 tx=0x7fc07c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:33.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.296+0000 7fc085600640 1 -- 172.21.15.60:0/2642826376 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc080063f70 con 0x7fc088145d20 2024-09-16T15:46:33.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.536+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc08806eab0 con 0x7fc088145d20 2024-09-16T15:46:33.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.537+0000 7fc085600640 1 -- 172.21.15.60:0/2642826376 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5227 (secure 0 0 0) 0x7fc080063870 con 0x7fc088145d20 2024-09-16T15:46:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:33.538 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"btime":"2024-09-16T15:46:20:740803+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/766495901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":766495901},{"type":"v1","addr":"172.21.15.100:6835","nonce":766495901}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7585,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/3797165836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3797165836},{"type":"v1","addr":"172.21.15.100:6801","nonce":3797165836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3900896825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3900896825},{"type":"v1","addr":"172.21.15.60:6834","nonce":3900896825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:10.124891+0000","modified":"2024-09-16T15:46:13.158701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/984341325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":984341325},{"type":"v1","addr":"172.21.15.60:6835","nonce":984341325}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7998,"qdb_cluster":[7998]},"id":10}]} 2024-09-16T15:46:33.539 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 93 2024-09-16T15:46:33.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc068078f10 msgr2=0x7fc06807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc068078f10 0x7fc06807b3d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc07c034000 tx=0x7fc07c004730 comp rx=0 tx=0).stop 2024-09-16T15:46:33.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc088145d20 msgr2=0x7fc08815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc088145d20 0x7fc08815f0a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc08000eb10 tx=0x7fc08000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 shutdown_connections 2024-09-16T15:46:33.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc068078f10 0x7fc06807b3d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc088145d20 0x7fc08815f0a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08815bc30 0x7fc08815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 --2- 172.21.15.60:0/2642826376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc0880d0540 0x7fc0880b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 >> 172.21.15.60:0/2642826376 conn(0x7fc0880cfa20 msgr2=0x7fc088157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 shutdown_connections 2024-09-16T15:46:33.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.540+0000 7fc08f000640 1 -- 172.21.15.60:0/2642826376 wait complete. 2024-09-16T15:46:33.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:33.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.718+0000 7f12ea400640 1 -- 172.21.15.60:0/898716996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e40af3d0 msgr2=0x7f12e406f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.718+0000 7f12ea400640 1 --2- 172.21.15.60:0/898716996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e40af3d0 0x7f12e406f490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009830 tx=0x7f12cc02a480 comp rx=0 tx=0).stop 2024-09-16T15:46:33.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.718+0000 7f12ea400640 1 -- 172.21.15.60:0/898716996 shutdown_connections 2024-09-16T15:46:33.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.718+0000 7f12ea400640 1 --2- 172.21.15.60:0/898716996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e40af3d0 0x7f12e406f490 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.718+0000 7f12ea400640 1 --2- 172.21.15.60:0/898716996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12e4161550 0x7f12e406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.718+0000 7f12ea400640 1 -- 172.21.15.60:0/898716996 >> 172.21.15.60:0/898716996 conn(0x7f12e40b1950 msgr2=0x7f12e40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.719+0000 7f12ea400640 1 -- 172.21.15.60:0/898716996 shutdown_connections 2024-09-16T15:46:33.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.719+0000 7f12ea400640 1 -- 172.21.15.60:0/898716996 wait complete. 2024-09-16T15:46:33.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.720+0000 7f12ea400640 1 Processor -- start 2024-09-16T15:46:33.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.720+0000 7f12ea400640 1 -- start start 2024-09-16T15:46:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.721+0000 7f12ea400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e406f9d0 0x7f12e4148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.721+0000 7f12ea400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12e40af3d0 0x7f12e40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.721+0000 7f12ea400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12e4161550 0x7f12e414a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.721+0000 7f12ea400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41623a0 con 0x7f12e40af3d0 2024-09-16T15:46:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.721+0000 7f12ea400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4074010 con 0x7f12e406f9d0 2024-09-16T15:46:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.721+0000 7f12ea400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f12e4162520 con 0x7f12e4161550 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.721+0000 7f12daa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12e40af3d0 0x7f12e40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12e3400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e406f9d0 0x7f12e4148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12daa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12e40af3d0 0x7f12e40ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44878/0 (socket says 172.21.15.60:44878) 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12daa00640 1 -- 172.21.15.60:0/1096177336 learned_addr learned my addr 172.21.15.60:0/1096177336 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12e3e00640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12e4161550 0x7f12e414a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12daa00640 1 -- 172.21.15.60:0/1096177336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12e4161550 msgr2=0x7f12e414a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12daa00640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12e4161550 0x7f12e414a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12daa00640 1 -- 172.21.15.60:0/1096177336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e406f9d0 msgr2=0x7f12e4148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12daa00640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e406f9d0 0x7f12e4148900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12daa00640 1 -- 172.21.15.60:0/1096177336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e414ae70 con 0x7f12e40af3d0 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12e3400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e406f9d0 0x7f12e4148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.722+0000 7f12daa00640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12e40af3d0 0x7f12e40ae130 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f12cc000c00 tx=0x7f12cc032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.723+0000 7f12dbe00640 1 -- 172.21.15.60:0/1096177336 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc040070 con 0x7f12e40af3d0 2024-09-16T15:46:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.723+0000 7f12dbe00640 1 -- 172.21.15.60:0/1096177336 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12cc0038e0 con 0x7f12e40af3d0 2024-09-16T15:46:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.723+0000 7f12dbe00640 1 -- 172.21.15.60:0/1096177336 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc005c90 con 0x7f12e40af3d0 2024-09-16T15:46:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.723+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e409c7e0 con 0x7f12e40af3d0 2024-09-16T15:46:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.723+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e414b3e0 con 0x7f12e40af3d0 2024-09-16T15:46:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.724+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e40760b0 con 0x7f12e40af3d0 2024-09-16T15:46:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.725+0000 7f12dbe00640 1 -- 172.21.15.60:0/1096177336 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f12cc039040 con 0x7f12e40af3d0 2024-09-16T15:46:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.726+0000 7f12dbe00640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f12c0078f10 0x7f12c007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.726+0000 7f12dbe00640 1 -- 172.21.15.60:0/1096177336 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f12cc0bcce0 con 0x7f12e40af3d0 2024-09-16T15:46:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.726+0000 7f12e3400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f12c0078f10 0x7f12c007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.727+0000 7f12e3400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f12c0078f10 0x7f12c007b3d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009f20 tx=0x7f12d4005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.730+0000 7f12dbe00640 1 -- 172.21.15.60:0/1096177336 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f12cc083f60 con 0x7f12e40af3d0 2024-09-16T15:46:33.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.932+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12e4162ab0 con 0x7f12e40af3d0 2024-09-16T15:46:33.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.933+0000 7f12dbe00640 1 -- 172.21.15.60:0/1096177336 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16783 (secure 0 0 0) 0x7f12cc088990 con 0x7f12e40af3d0 2024-09-16T15:46:33.935 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:33.935 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":137,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:11.704505+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:07.910602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:09.091337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f12c0078f10 msgr2=0x7f12c007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f12c0078f10 0x7f12c007b3d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009f20 tx=0x7f12d4005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12e40af3d0 msgr2=0x7f12e40ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12e40af3d0 0x7f12e40ae130 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f12cc000c00 tx=0x7f12cc032c10 comp rx=0 tx=0).stop 2024-09-16T15:46:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 shutdown_connections 2024-09-16T15:46:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f12c0078f10 0x7f12c007b3d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12e4161550 0x7f12e414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12e40af3d0 0x7f12e40ae130 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 --2- 172.21.15.60:0/1096177336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12e406f9d0 0x7f12e4148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 >> 172.21.15.60:0/1096177336 conn(0x7f12e40b1950 msgr2=0x7f12e406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 shutdown_connections 2024-09-16T15:46:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:33.936+0000 7f12ea400640 1 -- 172.21.15.60:0/1096177336 wait complete. 2024-09-16T15:46:33.953 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:46:33.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.113+0000 7f3761e00640 1 -- 172.21.15.60:0/1361580041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c15f5a0 msgr2=0x7f375c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.113+0000 7f3761e00640 1 --2- 172.21.15.60:0/1361580041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c15f5a0 0x7f375c0b2440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f374c005fd0 tx=0x7f374c02a540 comp rx=0 tx=0).stop 2024-09-16T15:46:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.113+0000 7f3761e00640 1 -- 172.21.15.60:0/1361580041 shutdown_connections 2024-09-16T15:46:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.113+0000 7f3761e00640 1 --2- 172.21.15.60:0/1361580041 >> v2:172.21.15.100:3300/0 conn(0x7f375c073fb0 0x7f375c074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.113+0000 7f3761e00640 1 --2- 172.21.15.60:0/1361580041 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c15f5a0 0x7f375c0b2440 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.113+0000 7f3761e00640 1 -- 172.21.15.60:0/1361580041 >> 172.21.15.60:0/1361580041 conn(0x7f375c0b1950 msgr2=0x7f375c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.113+0000 7f3761e00640 1 -- 172.21.15.60:0/1361580041 shutdown_connections 2024-09-16T15:46:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.113+0000 7f3761e00640 1 -- 172.21.15.60:0/1361580041 wait complete. 2024-09-16T15:46:34.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.115+0000 7f3761e00640 1 Processor -- start 2024-09-16T15:46:34.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.115+0000 7f3761e00640 1 -- start start 2024-09-16T15:46:34.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f3761e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c073fb0 0x7f375c0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f3761e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0ccf00 0x7f375c1f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f3761e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c15f5a0 0x7f375c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f3761e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f375c06f2e0 con 0x7f375c15f5a0 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f3761e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f375c06f160 con 0x7f375c073fb0 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f3761e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f375c06f460 con 0x7f375c0ccf00 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c15f5a0 0x7f375c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c15f5a0 0x7f375c15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44886/0 (socket says 172.21.15.60:44886) 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375b400640 1 -- 172.21.15.60:0/2138157215 learned_addr learned my addr 172.21.15.60:0/2138157215 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375aa00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c073fb0 0x7f375c0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f3752000640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0ccf00 0x7f375c1f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375b400640 1 -- 172.21.15.60:0/2138157215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0ccf00 msgr2=0x7f375c1f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375b400640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0ccf00 0x7f375c1f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375b400640 1 -- 172.21.15.60:0/2138157215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c073fb0 msgr2=0x7f375c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375b400640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c073fb0 0x7f375c0cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.116+0000 7f375b400640 1 -- 172.21.15.60:0/2138157215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c1f7020 con 0x7f375c15f5a0 2024-09-16T15:46:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.117+0000 7f375b400640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c15f5a0 0x7f375c15c380 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3754009cf0 tx=0x7f375400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.117+0000 7f3753e00640 1 -- 172.21.15.60:0/2138157215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754003b70 con 0x7f375c15f5a0 2024-09-16T15:46:34.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.117+0000 7f3753e00640 1 -- 172.21.15.60:0/2138157215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3754003e00 con 0x7f375c15f5a0 2024-09-16T15:46:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.117+0000 7f3753e00640 1 -- 172.21.15.60:0/2138157215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754004280 con 0x7f375c15f5a0 2024-09-16T15:46:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.117+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c159d10 con 0x7f375c15f5a0 2024-09-16T15:46:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.117+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c1570e0 con 0x7f375c15f5a0 2024-09-16T15:46:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.118+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375c06f9f0 con 0x7f375c15f5a0 2024-09-16T15:46:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.120+0000 7f3753e00640 1 -- 172.21.15.60:0/2138157215 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f3754004420 con 0x7f375c15f5a0 2024-09-16T15:46:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.120+0000 7f3753e00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3738078f10 0x7f373807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.121+0000 7f3753e00640 1 -- 172.21.15.60:0/2138157215 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f375409c4e0 con 0x7f375c15f5a0 2024-09-16T15:46:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.121+0000 7f375aa00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3738078f10 0x7f373807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.121+0000 7f375aa00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3738078f10 0x7f373807b3d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f374c033000 tx=0x7f374c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.125+0000 7f3753e00640 1 -- 172.21.15.60:0/2138157215 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3754063970 con 0x7f375c15f5a0 2024-09-16T15:46:34.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.358+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f375c0b08d0 con 0x7f375c15f5a0 2024-09-16T15:46:34.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.359+0000 7f3753e00640 1 -- 172.21.15.60:0/2138157215 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5227 (secure 0 0 0) 0x7f37540683a0 con 0x7f375c15f5a0 2024-09-16T15:46:34.360 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:34.360 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"btime":"2024-09-16T15:46:20:740803+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/766495901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":766495901},{"type":"v1","addr":"172.21.15.100:6835","nonce":766495901}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7585,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/3797165836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3797165836},{"type":"v1","addr":"172.21.15.100:6801","nonce":3797165836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3900896825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3900896825},{"type":"v1","addr":"172.21.15.60:6834","nonce":3900896825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:10.124891+0000","modified":"2024-09-16T15:46:13.158701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/984341325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":984341325},{"type":"v1","addr":"172.21.15.60:6835","nonce":984341325}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7998,"qdb_cluster":[7998]},"id":10}]} 2024-09-16T15:46:34.360 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 93 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3738078f10 msgr2=0x7f373807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3738078f10 0x7f373807b3d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f374c033000 tx=0x7f374c004730 comp rx=0 tx=0).stop 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c15f5a0 msgr2=0x7f375c15c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c15f5a0 0x7f375c15c380 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3754009cf0 tx=0x7f375400c6a0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 shutdown_connections 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3738078f10 0x7f373807b3d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f375c15f5a0 0x7f375c15c380 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f375c0ccf00 0x7f375c1f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 --2- 172.21.15.60:0/2138157215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f375c073fb0 0x7f375c0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 >> 172.21.15.60:0/2138157215 conn(0x7f375c0b1950 msgr2=0x7f375c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:34.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 shutdown_connections 2024-09-16T15:46:34.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.361+0000 7f3761e00640 1 -- 172.21.15.60:0/2138157215 wait complete. 2024-09-16T15:46:34.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:34.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.540+0000 7f094ba00640 1 -- 172.21.15.60:0/2827270058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094415f5a0 msgr2=0x7f09440b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.540+0000 7f094ba00640 1 --2- 172.21.15.60:0/2827270058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094415f5a0 0x7f09440b2440 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0944076720 tx=0x7f093402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.541+0000 7f094ba00640 1 -- 172.21.15.60:0/2827270058 shutdown_connections 2024-09-16T15:46:34.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.541+0000 7f094ba00640 1 --2- 172.21.15.60:0/2827270058 >> v2:172.21.15.100:3300/0 conn(0x7f09440ad270 0x7f0944073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.541+0000 7f094ba00640 1 --2- 172.21.15.60:0/2827270058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09440ccf00 0x7f09440acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.541+0000 7f094ba00640 1 --2- 172.21.15.60:0/2827270058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f094415f5a0 0x7f09440b2440 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.541+0000 7f094ba00640 1 -- 172.21.15.60:0/2827270058 >> 172.21.15.60:0/2827270058 conn(0x7f09440b1950 msgr2=0x7f09440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:34.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.541+0000 7f094ba00640 1 -- 172.21.15.60:0/2827270058 shutdown_connections 2024-09-16T15:46:34.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.541+0000 7f094ba00640 1 -- 172.21.15.60:0/2827270058 wait complete. 2024-09-16T15:46:34.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.542+0000 7f094ba00640 1 Processor -- start 2024-09-16T15:46:34.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.542+0000 7f094ba00640 1 -- start start 2024-09-16T15:46:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.543+0000 7f094ba00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09440ad270 0x7f0944150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.543+0000 7f094ba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09440ccf00 0x7f094414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.543+0000 7f094ba00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f094415f5a0 0x7f09441f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.543+0000 7f094ba00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f094406f160 con 0x7f09440ad270 2024-09-16T15:46:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.543+0000 7f094ba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0944074780 con 0x7f09440ccf00 2024-09-16T15:46:34.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.543+0000 7f094ba00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f094406f2e0 con 0x7f094415f5a0 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09440ad270 0x7f0944150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09440ad270 0x7f0944150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44908/0 (socket says 172.21.15.60:44908) 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 -- 172.21.15.60:0/2281954152 learned_addr learned my addr 172.21.15.60:0/2281954152 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f093be00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09440ccf00 0x7f094414c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949a00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f094415f5a0 0x7f09441f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 -- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f094415f5a0 msgr2=0x7f09441f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f094415f5a0 0x7f09441f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 -- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09440ccf00 msgr2=0x7f094414c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09440ccf00 0x7f094414c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 -- 172.21.15.60:0/2281954152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09440b3970 con 0x7f09440ad270 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949a00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f094415f5a0 0x7f09441f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:34.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.544+0000 7f0949000640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09440ad270 0x7f0944150bd0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f09340099c0 tx=0x7f09340039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.545+0000 7f0941600640 1 -- 172.21.15.60:0/2281954152 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093403e030 con 0x7f09440ad270 2024-09-16T15:46:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.545+0000 7f0941600640 1 -- 172.21.15.60:0/2281954152 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09340047a0 con 0x7f09440ad270 2024-09-16T15:46:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.545+0000 7f0941600640 1 -- 172.21.15.60:0/2281954152 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934035040 con 0x7f09440ad270 2024-09-16T15:46:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.545+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09440b3c50 con 0x7f09440ad270 2024-09-16T15:46:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.545+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094414cbe0 con 0x7f09440ad270 2024-09-16T15:46:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.546+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094406f870 con 0x7f09440ad270 2024-09-16T15:46:34.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.547+0000 7f0941600640 1 -- 172.21.15.60:0/2281954152 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f0934041030 con 0x7f09440ad270 2024-09-16T15:46:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.548+0000 7f0941600640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0920078f10 0x7f092007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.548+0000 7f0941600640 1 -- 172.21.15.60:0/2281954152 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f09340bbc30 con 0x7f09440ad270 2024-09-16T15:46:34.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.548+0000 7f093be00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0920078f10 0x7f092007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.549+0000 7f093be00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0920078f10 0x7f092007b3d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f09441f80e0 tx=0x7f092c00b390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:34.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.552+0000 7f0941600640 1 -- 172.21.15.60:0/2281954152 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0934083150 con 0x7f09440ad270 2024-09-16T15:46:34.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.790+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09440b00d0 con 0x7f09440ad270 2024-09-16T15:46:34.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.790+0000 7f0941600640 1 -- 172.21.15.60:0/2281954152 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5227 (secure 0 0 0) 0x7f0934087b80 con 0x7f09440ad270 2024-09-16T15:46:34.792 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:34.792 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"btime":"2024-09-16T15:46:20:740803+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/766495901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":766495901},{"type":"v1","addr":"172.21.15.100:6835","nonce":766495901}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7585,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/3797165836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3797165836},{"type":"v1","addr":"172.21.15.100:6801","nonce":3797165836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3900896825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3900896825},{"type":"v1","addr":"172.21.15.60:6834","nonce":3900896825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:10.124891+0000","modified":"2024-09-16T15:46:13.158701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/984341325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":984341325},{"type":"v1","addr":"172.21.15.60:6835","nonce":984341325}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7998,"qdb_cluster":[7998]},"id":10}]} 2024-09-16T15:46:34.792 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 93 2024-09-16T15:46:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0920078f10 msgr2=0x7f092007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0920078f10 0x7f092007b3d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f09441f80e0 tx=0x7f092c00b390 comp rx=0 tx=0).stop 2024-09-16T15:46:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09440ad270 msgr2=0x7f0944150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09440ad270 0x7f0944150bd0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f09340099c0 tx=0x7f09340039e0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 shutdown_connections 2024-09-16T15:46:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0920078f10 0x7f092007b3d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f094415f5a0 0x7f09441f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09440ccf00 0x7f094414c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 --2- 172.21.15.60:0/2281954152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09440ad270 0x7f0944150bd0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 >> 172.21.15.60:0/2281954152 conn(0x7f09440b1950 msgr2=0x7f09440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 shutdown_connections 2024-09-16T15:46:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.793+0000 7f094ba00640 1 -- 172.21.15.60:0/2281954152 wait complete. 2024-09-16T15:46:34.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:34.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.971+0000 7f13ea000640 1 -- 172.21.15.60:0/3641020464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13c8003a30 msgr2=0x7f13c8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.971+0000 7f13ea000640 1 --2- 172.21.15.60:0/3641020464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13c8003a30 0x7f13c8003e70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f13d400c4a0 tx=0x7f13c8005830 comp rx=0 tx=0).stop 2024-09-16T15:46:34.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.971+0000 7f13e9600640 1 -- 172.21.15.60:0/3641020464 reap_dead start 2024-09-16T15:46:34.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.972+0000 7f13ea000640 1 -- 172.21.15.60:0/3641020464 shutdown_connections 2024-09-16T15:46:34.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.972+0000 7f13ea000640 1 -- 172.21.15.60:0/3641020464 >> 172.21.15.60:0/3641020464 conn(0x7f13e40d3df0 msgr2=0x7f13e40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:34.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.972+0000 7f13ea000640 1 -- 172.21.15.60:0/3641020464 shutdown_connections 2024-09-16T15:46:34.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.972+0000 7f13ea000640 1 -- 172.21.15.60:0/3641020464 wait complete. 2024-09-16T15:46:34.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.973+0000 7f13ea000640 1 Processor -- start 2024-09-16T15:46:34.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.973+0000 7f13ea000640 1 -- start start 2024-09-16T15:46:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.974+0000 7f13ea000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c8003a30 0x7f13e40b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.974+0000 7f13ea000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40ca1e0 0x7f13e41574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.974+0000 7f13ea000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e40c7f50 0x7f13e40c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.974+0000 7f13ea000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4164d00 con 0x7f13e40c7f50 2024-09-16T15:46:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.974+0000 7f13ea000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4164b80 con 0x7f13c8003a30 2024-09-16T15:46:34.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.974+0000 7f13ea000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f13e4164e80 con 0x7f13e40ca1e0 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e40c7f50 0x7f13e40c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e40c7f50 0x7f13e40c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44920/0 (socket says 172.21.15.60:44920) 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 -- 172.21.15.60:0/3286062574 learned_addr learned my addr 172.21.15.60:0/3286062574 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13dbe00640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40ca1e0 0x7f13e41574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 -- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40ca1e0 msgr2=0x7f13e41574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40ca1e0 0x7f13e41574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e8c00640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c8003a30 0x7f13e40b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 -- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c8003a30 msgr2=0x7f13e40b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c8003a30 0x7f13e40b8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 -- 172.21.15.60:0/3286062574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e40b8ff0 con 0x7f13e40c7f50 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13dbe00640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40ca1e0 0x7f13e41574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:34.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e9600640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e40c7f50 0x7f13e40c8400 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f13e40c78c0 tx=0x7f13dc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.975+0000 7f13e1600640 1 -- 172.21.15.60:0/3286062574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0107d0 con 0x7f13e40c7f50 2024-09-16T15:46:34.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.976+0000 7f13e1600640 1 -- 172.21.15.60:0/3286062574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13dc010970 con 0x7f13e40c7f50 2024-09-16T15:46:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.976+0000 7f13e1600640 1 -- 172.21.15.60:0/3286062574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc010dc0 con 0x7f13e40c7f50 2024-09-16T15:46:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.976+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e40b92d0 con 0x7f13e40c7f50 2024-09-16T15:46:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.976+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e40cfcf0 con 0x7f13e40c7f50 2024-09-16T15:46:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.977+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e40b4e00 con 0x7f13e40c7f50 2024-09-16T15:46:34.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.978+0000 7f13e1600640 1 -- 172.21.15.60:0/3286062574 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f13dc01a400 con 0x7f13e40c7f50 2024-09-16T15:46:34.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.979+0000 7f13e1600640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13c0078f10 0x7f13c007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.979+0000 7f13e1600640 1 -- 172.21.15.60:0/3286062574 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f13dc09bb30 con 0x7f13e40c7f50 2024-09-16T15:46:34.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.979+0000 7f13e8c00640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13c0078f10 0x7f13c007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.980+0000 7f13e8c00640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13c0078f10 0x7f13c007b3d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f13d4033000 tx=0x7f13e40d2bb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:34.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:34.983+0000 7f13e1600640 1 -- 172.21.15.60:0/3286062574 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f13e40b5510 con 0x7f13e40c7f50 2024-09-16T15:46:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.179+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13e415d4a0 con 0x7f13e40c7f50 2024-09-16T15:46:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.181+0000 7f13e1600640 1 -- 172.21.15.60:0/3286062574 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16783 (secure 0 0 0) 0x7f13dc063830 con 0x7f13e40c7f50 2024-09-16T15:46:35.182 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:35.182 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":137,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:11.704505+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:07.910602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:09.091337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:35.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13c0078f10 msgr2=0x7f13c007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13c0078f10 0x7f13c007b3d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f13d4033000 tx=0x7f13e40d2bb0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e40c7f50 msgr2=0x7f13e40c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e40c7f50 0x7f13e40c8400 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f13e40c78c0 tx=0x7f13dc0019b0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 shutdown_connections 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13c0078f10 0x7f13c007b3d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13e40c7f50 0x7f13e40c8400 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13e40ca1e0 0x7f13e41574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 --2- 172.21.15.60:0/3286062574 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c8003a30 0x7f13e40b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 >> 172.21.15.60:0/3286062574 conn(0x7f13e40d3df0 msgr2=0x7f13e406e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 shutdown_connections 2024-09-16T15:46:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.183+0000 7f13ea000640 1 -- 172.21.15.60:0/3286062574 wait complete. 2024-09-16T15:46:35.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:46:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.362+0000 7f7c72a00640 1 -- 172.21.15.60:0/375763968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c0cdd00 msgr2=0x7f7c6c0adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.362+0000 7f7c72a00640 1 --2- 172.21.15.60:0/375763968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c0cdd00 0x7f7c6c0adcd0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0cd570 tx=0x7f7c6402a560 comp rx=0 tx=0).stop 2024-09-16T15:46:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.362+0000 7f7c72a00640 1 -- 172.21.15.60:0/375763968 shutdown_connections 2024-09-16T15:46:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.362+0000 7f7c72a00640 1 --2- 172.21.15.60:0/375763968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c0cdd00 0x7f7c6c0adcd0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.362+0000 7f7c72a00640 1 --2- 172.21.15.60:0/375763968 >> v2:172.21.15.100:3300/0 conn(0x7f7c6c06fb90 0x7f7c6c06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.362+0000 7f7c72a00640 1 -- 172.21.15.60:0/375763968 >> 172.21.15.60:0/375763968 conn(0x7f7c6c0cfa20 msgr2=0x7f7c6c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.362+0000 7f7c72a00640 1 -- 172.21.15.60:0/375763968 shutdown_connections 2024-09-16T15:46:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.362+0000 7f7c72a00640 1 -- 172.21.15.60:0/375763968 wait complete. 2024-09-16T15:46:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.363+0000 7f7c72a00640 1 Processor -- start 2024-09-16T15:46:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.363+0000 7f7c72a00640 1 -- start start 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.364+0000 7f7c72a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c06fb90 0x7f7c6c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.364+0000 7f7c72a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c6c0ad880 0x7f7c6c1f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.364+0000 7f7c72a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c0cdd00 0x7f7c6c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.364+0000 7f7c72a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c15ed70 con 0x7f7c6c06fb90 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.364+0000 7f7c72a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c15ebf0 con 0x7f7c6c0cdd00 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.364+0000 7f7c72a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6c15eef0 con 0x7f7c6c0ad880 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c06fb90 0x7f7c6c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c06fb90 0x7f7c6c0b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44932/0 (socket says 172.21.15.60:44932) 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 -- 172.21.15.60:0/3385061774 learned_addr learned my addr 172.21.15.60:0/3385061774 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c72000640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c0cdd00 0x7f7c6c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c70c00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c6c0ad880 0x7f7c6c1f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 -- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c6c0ad880 msgr2=0x7f7c6c1f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c6c0ad880 0x7f7c6c1f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 -- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c0cdd00 msgr2=0x7f7c6c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c0cdd00 0x7f7c6c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 -- 172.21.15.60:0/3385061774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c1f5340 con 0x7f7c6c06fb90 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c72000640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c0cdd00 0x7f7c6c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.365+0000 7f7c71600640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c06fb90 0x7f7c6c0b50a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000ea80 tx=0x7f7c6000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.366+0000 7f7c69600640 1 -- 172.21.15.60:0/3385061774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000ce40 con 0x7f7c6c06fb90 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.366+0000 7f7c69600640 1 -- 172.21.15.60:0/3385061774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c60004c10 con 0x7f7c6c06fb90 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.366+0000 7f7c69600640 1 -- 172.21.15.60:0/3385061774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60005020 con 0x7f7c6c06fb90 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.366+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c156500 con 0x7f7c6c06fb90 2024-09-16T15:46:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.366+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c155eb0 con 0x7f7c6c06fb90 2024-09-16T15:46:35.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.367+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6c06ff70 con 0x7f7c6c06fb90 2024-09-16T15:46:35.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.368+0000 7f7c69600640 1 -- 172.21.15.60:0/3385061774 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f7c600051e0 con 0x7f7c6c06fb90 2024-09-16T15:46:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.369+0000 7f7c69600640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c4c078f10 0x7f7c4c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.369+0000 7f7c69600640 1 -- 172.21.15.60:0/3385061774 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5737+0+0 (secure 0 0 0) 0x7f7c6009c0e0 con 0x7f7c6c06fb90 2024-09-16T15:46:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.369+0000 7f7c70c00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c4c078f10 0x7f7c4c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.370+0000 7f7c70c00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c4c078f10 0x7f7c4c07b3d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0c9a20 tx=0x7f7c58009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:35.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.373+0000 7f7c69600640 1 -- 172.21.15.60:0/3385061774 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7c6c1f5340 con 0x7f7c6c06fb90 2024-09-16T15:46:35.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.608+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7c6c0af700 con 0x7f7c6c06fb90 2024-09-16T15:46:35.610 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:46:35.609+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:46:35.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.611+0000 7f7c69600640 1 -- 172.21.15.60:0/3385061774 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) ==== 161+0+0 (secure 0 0 0) 0x7f7c60016030 con 0x7f7c6c06fb90 2024-09-16T15:46:35.612 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c4c078f10 msgr2=0x7f7c4c07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c4c078f10 0x7f7c4c07b3d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0c9a20 tx=0x7f7c58009040 comp rx=0 tx=0).stop 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c06fb90 msgr2=0x7f7c6c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c06fb90 0x7f7c6c0b50a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000ea80 tx=0x7f7c6000ef40 comp rx=0 tx=0).stop 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 shutdown_connections 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c4c078f10 0x7f7c4c07b3d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c6c0cdd00 0x7f7c6c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c6c0ad880 0x7f7c6c1f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 --2- 172.21.15.60:0/3385061774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6c06fb90 0x7f7c6c0b50a0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 >> 172.21.15.60:0/3385061774 conn(0x7f7c6c0cfa20 msgr2=0x7f7c6c15daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:35.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 shutdown_connections 2024-09-16T15:46:35.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.615+0000 7f7c72a00640 1 -- 172.21.15.60:0/3385061774 wait complete. 2024-09-16T15:46:35.619 INFO:tasks.ceph.mds.a.smithi060.stderr: -4387> 2024-09-16T15:45:59.386+0000 7fe8f3600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-16T15:46:35.622 INFO:tasks.ceph.mds.a.smithi060.stderr: -2951> 2024-09-16T15:46:15.648+0000 7fe8ebe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:46:35.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:46:35.672 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:46:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.800+0000 7f640fe00640 1 -- 172.21.15.60:0/1851451395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6410161460 msgr2=0x7f64100d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.800+0000 7f640fe00640 1 --2- 172.21.15.60:0/1851451395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6410161460 0x7f64100d22f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6400009830 tx=0x7f640002a480 comp rx=0 tx=0).stop 2024-09-16T15:46:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.800+0000 7f640fe00640 1 -- 172.21.15.60:0/1851451395 shutdown_connections 2024-09-16T15:46:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.800+0000 7f640fe00640 1 --2- 172.21.15.60:0/1851451395 >> v2:172.21.15.100:3300/0 conn(0x7f64100b5480 0x7f64100d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.800+0000 7f640fe00640 1 --2- 172.21.15.60:0/1851451395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6410161460 0x7f64100d22f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.800+0000 7f640fe00640 1 -- 172.21.15.60:0/1851451395 >> 172.21.15.60:0/1851451395 conn(0x7f641009d0b0 msgr2=0x7f64100d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:35.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.800+0000 7f640fe00640 1 -- 172.21.15.60:0/1851451395 shutdown_connections 2024-09-16T15:46:35.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.800+0000 7f640fe00640 1 -- 172.21.15.60:0/1851451395 wait complete. 2024-09-16T15:46:35.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.802+0000 7f640fe00640 1 Processor -- start 2024-09-16T15:46:35.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.802+0000 7f640fe00640 1 -- start start 2024-09-16T15:46:35.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64100b4f40 0x7f64101f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:35.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64100b5480 0x7f64100cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:35.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6410161460 0x7f64100cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:35.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f64100cf0b0 con 0x7f64100b5480 2024-09-16T15:46:35.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f64100cef30 con 0x7f6410161460 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f64100cf230 con 0x7f64100b4f40 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64100b5480 0x7f64100cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64100b5480 0x7f64100cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44964/0 (socket says 172.21.15.60:44964) 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640e000640 1 -- 172.21.15.60:0/3298019901 learned_addr learned my addr 172.21.15.60:0/3298019901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640ea00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64100b4f40 0x7f64101f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640f400640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6410161460 0x7f64100cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640e000640 1 -- 172.21.15.60:0/3298019901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64100b4f40 msgr2=0x7f64101f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640e000640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64100b4f40 0x7f64101f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640e000640 1 -- 172.21.15.60:0/3298019901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6410161460 msgr2=0x7f64100cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640e000640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6410161460 0x7f64100cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.803+0000 7f640e000640 1 -- 172.21.15.60:0/3298019901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64101f4b20 con 0x7f64100b5480 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.804+0000 7f640e000640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64100b5480 0x7f64100cd420 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00e970 tx=0x7f63fc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:35.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.804+0000 7f6406a00640 1 -- 172.21.15.60:0/3298019901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc00cd80 con 0x7f64100b5480 2024-09-16T15:46:35.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.804+0000 7f6406a00640 1 -- 172.21.15.60:0/3298019901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63fc010430 con 0x7f64100b5480 2024-09-16T15:46:35.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.804+0000 7f6406a00640 1 -- 172.21.15.60:0/3298019901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc010880 con 0x7f64100b5480 2024-09-16T15:46:35.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.804+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64101f4de0 con 0x7f64100b5480 2024-09-16T15:46:35.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.804+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64101f53a0 con 0x7f64100b5480 2024-09-16T15:46:35.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.805+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64100cf750 con 0x7f64100b5480 2024-09-16T15:46:35.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.806+0000 7f6406a00640 1 -- 172.21.15.60:0/3298019901 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f63fc01f020 con 0x7f64100b5480 2024-09-16T15:46:35.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.807+0000 7f6406a00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f63f0078f10 0x7f63f007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:35.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.808+0000 7f6406a00640 1 -- 172.21.15.60:0/3298019901 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5823+0+0 (secure 0 0 0) 0x7f63fc09bdf0 con 0x7f64100b5480 2024-09-16T15:46:35.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.808+0000 7f640ea00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f63f0078f10 0x7f63f007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:35.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.808+0000 7f640ea00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f63f0078f10 0x7f63f007b3d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6400034000 tx=0x7f6400004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:35.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:35.812+0000 7f6406a00640 1 -- 172.21.15.60:0/3298019901 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f63fc063230 con 0x7f64100b5480 2024-09-16T15:46:36.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.047+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f64100ccf00 con 0x7f64100b5480 2024-09-16T15:46:36.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.613+0000 7f6406a00640 1 -- 172.21.15.60:0/3298019901 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v95) ==== 104+0+0 (secure 0 0 0) 0x7f63fc067c60 con 0x7f64100b5480 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.616+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f63f0078f10 msgr2=0x7f63f007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.616+0000 7f640fe00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f63f0078f10 0x7f63f007b3d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6400034000 tx=0x7f6400004730 comp rx=0 tx=0).stop 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.616+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64100b5480 msgr2=0x7f64100cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.616+0000 7f640fe00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64100b5480 0x7f64100cd420 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00e970 tx=0x7f63fc00ee30 comp rx=0 tx=0).stop 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.617+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 shutdown_connections 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.617+0000 7f640fe00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f63f0078f10 0x7f63f007b3d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.617+0000 7f640fe00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6410161460 0x7f64100cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.617+0000 7f640fe00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64100b5480 0x7f64100cd420 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.617+0000 7f640fe00640 1 --2- 172.21.15.60:0/3298019901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64100b4f40 0x7f64101f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.617+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 >> 172.21.15.60:0/3298019901 conn(0x7f641009d0b0 msgr2=0x7f6410160700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:36.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.617+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 shutdown_connections 2024-09-16T15:46:36.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.617+0000 7f640fe00640 1 -- 172.21.15.60:0/3298019901 wait complete. 2024-09-16T15:46:36.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:46:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.846+0000 7fc625800640 1 -- 172.21.15.60:0/2748721163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 msgr2=0x7fc62006f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.846+0000 7fc625800640 1 --2- 172.21.15.60:0/2748721163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 0x7fc62006f6b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fc618005fd0 tx=0x7fc61802a940 comp rx=0 tx=0).stop 2024-09-16T15:46:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.846+0000 7fc625800640 1 -- 172.21.15.60:0/2748721163 shutdown_connections 2024-09-16T15:46:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.846+0000 7fc625800640 1 --2- 172.21.15.60:0/2748721163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 0x7fc62006f6b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.846+0000 7fc625800640 1 -- 172.21.15.60:0/2748721163 >> 172.21.15.60:0/2748721163 conn(0x7fc6200b18e0 msgr2=0x7fc6200b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.846+0000 7fc625800640 1 -- 172.21.15.60:0/2748721163 shutdown_connections 2024-09-16T15:46:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.846+0000 7fc625800640 1 -- 172.21.15.60:0/2748721163 wait complete. 2024-09-16T15:46:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.847+0000 7fc625800640 1 Processor -- start 2024-09-16T15:46:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.847+0000 7fc625800640 1 -- start start 2024-09-16T15:46:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.848+0000 7fc625800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 0x7fc620154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.848+0000 7fc625800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc620159aa0 0x7fc62015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.848+0000 7fc625800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62015cff0 0x7fc620149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.848+0000 7fc625800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc6200cc740 con 0x7fc62006f220 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.848+0000 7fc625800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc6200cc5c0 con 0x7fc620159aa0 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.848+0000 7fc625800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc6200cc8c0 con 0x7fc62015cff0 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 0x7fc620154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 0x7fc620154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44994/0 (socket says 172.21.15.60:44994) 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 -- 172.21.15.60:0/1559299644 learned_addr learned my addr 172.21.15.60:0/1559299644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc616000640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc620159aa0 0x7fc62015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61f400640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62015cff0 0x7fc620149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 -- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62015cff0 msgr2=0x7fc620149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62015cff0 0x7fc620149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 -- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc620159aa0 msgr2=0x7fc62015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc620159aa0 0x7fc62015e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 -- 172.21.15.60:0/1559299644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201f3550 con 0x7fc62006f220 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc616000640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc620159aa0 0x7fc62015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.849+0000 7fc61ea00640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 0x7fc620154ee0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c450 tx=0x7fc61000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.850+0000 7fc617400640 1 -- 172.21.15.60:0/1559299644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100052d0 con 0x7fc62006f220 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.850+0000 7fc617400640 1 -- 172.21.15.60:0/1559299644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc610005470 con 0x7fc62006f220 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.850+0000 7fc617400640 1 -- 172.21.15.60:0/1559299644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610012870 con 0x7fc62006f220 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.850+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201f3830 con 0x7fc62006f220 2024-09-16T15:46:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.850+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201546e0 con 0x7fc62006f220 2024-09-16T15:46:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.851+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620069360 con 0x7fc62006f220 2024-09-16T15:46:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.852+0000 7fc617400640 1 -- 172.21.15.60:0/1559299644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fc6100035f0 con 0x7fc62006f220 2024-09-16T15:46:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.853+0000 7fc617400640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5fc078f10 0x7fc5fc07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.854+0000 7fc617400640 1 -- 172.21.15.60:0/1559299644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5823+0+0 (secure 0 0 0) 0x7fc61009b120 con 0x7fc62006f220 2024-09-16T15:46:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.854+0000 7fc616000640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5fc078f10 0x7fc5fc07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:36.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.854+0000 7fc616000640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5fc078f10 0x7fc5fc07b3d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6200cde60 tx=0x7fc608009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:36.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:36.858+0000 7fc617400640 1 -- 172.21.15.60:0/1559299644 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc610062d40 con 0x7fc62006f220 2024-09-16T15:46:37.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.052+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc620070980 con 0x7fc62006f220 2024-09-16T15:46:37.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.802+0000 7fc617400640 1 -- 172.21.15.60:0/1559299644 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) ==== 181+0+0 (secure 0 0 0) 0x7fc610062640 con 0x7fc62006f220 2024-09-16T15:46:37.804 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:46:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.806+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5fc078f10 msgr2=0x7fc5fc07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.806+0000 7fc625800640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5fc078f10 0x7fc5fc07b3d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6200cde60 tx=0x7fc608009040 comp rx=0 tx=0).stop 2024-09-16T15:46:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.806+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 msgr2=0x7fc620154ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.806+0000 7fc625800640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 0x7fc620154ee0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c450 tx=0x7fc61000c910 comp rx=0 tx=0).stop 2024-09-16T15:46:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.807+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 shutdown_connections 2024-09-16T15:46:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.807+0000 7fc625800640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc5fc078f10 0x7fc5fc07b3d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.807+0000 7fc625800640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62015cff0 0x7fc620149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.807+0000 7fc625800640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc620159aa0 0x7fc62015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.807+0000 7fc625800640 1 --2- 172.21.15.60:0/1559299644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc62006f220 0x7fc620154ee0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.807+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 >> 172.21.15.60:0/1559299644 conn(0x7fc6200b18e0 msgr2=0x7fc62015a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:37.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.807+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 shutdown_connections 2024-09-16T15:46:37.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.807+0000 7fc625800640 1 -- 172.21.15.60:0/1559299644 wait complete. 2024-09-16T15:46:37.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:46:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.994+0000 7f649be00640 1 -- 172.21.15.60:0/1169695334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 msgr2=0x7f649c0ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.994+0000 7f649be00640 1 --2- 172.21.15.60:0/1169695334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 0x7f649c0ca0c0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f649c0cd500 tx=0x7f648402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.995+0000 7f649be00640 1 -- 172.21.15.60:0/1169695334 shutdown_connections 2024-09-16T15:46:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.995+0000 7f649be00640 1 --2- 172.21.15.60:0/1169695334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 0x7f649c0ca0c0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.995+0000 7f649be00640 1 --2- 172.21.15.60:0/1169695334 >> v2:172.21.15.100:3300/0 conn(0x7f649c15a230 0x7f649c06f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.995+0000 7f649be00640 1 -- 172.21.15.60:0/1169695334 >> 172.21.15.60:0/1169695334 conn(0x7f649c0cfa70 msgr2=0x7f649c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.995+0000 7f649be00640 1 -- 172.21.15.60:0/1169695334 shutdown_connections 2024-09-16T15:46:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.995+0000 7f649be00640 1 -- 172.21.15.60:0/1169695334 wait complete. 2024-09-16T15:46:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.996+0000 7f649be00640 1 Processor -- start 2024-09-16T15:46:37.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.996+0000 7f649be00640 1 -- start start 2024-09-16T15:46:37.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 0x7f649c0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:37.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f649c0ca600 0x7f649c15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f649c15a230 0x7f649c154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f649c073dc0 con 0x7f649c074540 2024-09-16T15:46:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f649c073c40 con 0x7f649c0ca600 2024-09-16T15:46:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649be00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f649c073f40 con 0x7f649c15a230 2024-09-16T15:46:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 0x7f649c0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 0x7f649c0c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45016/0 (socket says 172.21.15.60:45016) 2024-09-16T15:46:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.997+0000 7f649aa00640 1 -- 172.21.15.60:0/819076463 learned_addr learned my addr 172.21.15.60:0/819076463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f649aa00640 1 -- 172.21.15.60:0/819076463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f649c15a230 msgr2=0x7f649c154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f649aa00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f649c15a230 0x7f649c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f649aa00640 1 -- 172.21.15.60:0/819076463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f649c0ca600 msgr2=0x7f649c15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f649aa00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f649c0ca600 0x7f649c15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f649aa00640 1 -- 172.21.15.60:0/819076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c0c7e60 con 0x7f649c074540 2024-09-16T15:46:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f649aa00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 0x7f649c0c9810 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f648c00db90 tx=0x7f648c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f6493400640 1 -- 172.21.15.60:0/819076463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c01b070 con 0x7f649c074540 2024-09-16T15:46:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f6493400640 1 -- 172.21.15.60:0/819076463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f648c00ae50 con 0x7f649c074540 2024-09-16T15:46:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c0c8140 con 0x7f649c074540 2024-09-16T15:46:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.998+0000 7f6493400640 1 -- 172.21.15.60:0/819076463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c00e9e0 con 0x7f649c074540 2024-09-16T15:46:38.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:37.999+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c0c8700 con 0x7f649c074540 2024-09-16T15:46:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.000+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649c0ca0c0 con 0x7f649c074540 2024-09-16T15:46:38.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.001+0000 7f6493400640 1 -- 172.21.15.60:0/819076463 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f648c01e070 con 0x7f649c074540 2024-09-16T15:46:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.002+0000 7f6493400640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6478078f10 0x7f647807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.002+0000 7f6493400640 1 -- 172.21.15.60:0/819076463 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5385+0+0 (secure 0 0 0) 0x7f648c09b900 con 0x7f649c074540 2024-09-16T15:46:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.002+0000 7f6492000640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6478078f10 0x7f647807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.003+0000 7f6492000640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6478078f10 0x7f647807b3d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6484033000 tx=0x7f64840046c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:38.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.012+0000 7f6493400640 1 -- 172.21.15.60:0/819076463 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f648c063080 con 0x7f649c074540 2024-09-16T15:46:38.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.213+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f649c0b02d0 con 0x7f649c074540 2024-09-16T15:46:38.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.823+0000 7f6493400640 1 -- 172.21.15.60:0/819076463 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) ==== 169+0+0 (secure 0 0 0) 0x7f648c067ab0 con 0x7f649c074540 2024-09-16T15:46:38.825 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:46:38.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.827+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6478078f10 msgr2=0x7f647807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.827+0000 7f649be00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6478078f10 0x7f647807b3d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6484033000 tx=0x7f64840046c0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.827+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 msgr2=0x7f649c0c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.827+0000 7f649be00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 0x7f649c0c9810 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f648c00db90 tx=0x7f648c00a620 comp rx=0 tx=0).stop 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.828+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 shutdown_connections 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.828+0000 7f649be00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6478078f10 0x7f647807b3d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.828+0000 7f649be00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f649c15a230 0x7f649c154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.828+0000 7f649be00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f649c0ca600 0x7f649c15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.828+0000 7f649be00640 1 --2- 172.21.15.60:0/819076463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f649c074540 0x7f649c0c9810 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.828+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 >> 172.21.15.60:0/819076463 conn(0x7f649c0cfa70 msgr2=0x7f649c0c6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.828+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 shutdown_connections 2024-09-16T15:46:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:38.828+0000 7f649be00640 1 -- 172.21.15.60:0/819076463 wait complete. 2024-09-16T15:46:38.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:46:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.012+0000 7f2361400640 1 -- 172.21.15.60:0/527476605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ccd00 msgr2=0x7f235c06f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.012+0000 7f2361400640 1 --2- 172.21.15.60:0/527476605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ccd00 0x7f235c06f850 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f235c06f3d0 tx=0x7f234802b830 comp rx=0 tx=0).stop 2024-09-16T15:46:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.012+0000 7f2361400640 1 -- 172.21.15.60:0/527476605 shutdown_connections 2024-09-16T15:46:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.012+0000 7f2361400640 1 --2- 172.21.15.60:0/527476605 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c0ccd00 0x7f235c06f850 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.012+0000 7f2361400640 1 -- 172.21.15.60:0/527476605 >> 172.21.15.60:0/527476605 conn(0x7f235c0b1950 msgr2=0x7f235c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.012+0000 7f2361400640 1 -- 172.21.15.60:0/527476605 shutdown_connections 2024-09-16T15:46:39.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.012+0000 7f2361400640 1 -- 172.21.15.60:0/527476605 wait complete. 2024-09-16T15:46:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.014+0000 7f2361400640 1 Processor -- start 2024-09-16T15:46:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.014+0000 7f2361400640 1 -- start start 2024-09-16T15:46:39.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f2361400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c06fd90 0x7f235c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f2361400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c0ccd00 0x7f235c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f2361400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c1438b0 0x7f235c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f2361400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f235c0b4610 con 0x7f235c1438b0 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f2361400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f235c070b30 con 0x7f235c0ccd00 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f2361400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f235c0b4790 con 0x7f235c06fd90 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f235b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c1438b0 0x7f235c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f235b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c1438b0 0x7f235c0caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57470/0 (socket says 172.21.15.60:57470) 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f235b200640 1 -- 172.21.15.60:0/241877042 learned_addr learned my addr 172.21.15.60:0/241877042 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f235a800640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c06fd90 0x7f235c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.015+0000 7f2359e00640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c0ccd00 0x7f235c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f235b200640 1 -- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c06fd90 msgr2=0x7f235c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f235b200640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c06fd90 0x7f235c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f235b200640 1 -- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c0ccd00 msgr2=0x7f235c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f235b200640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c0ccd00 0x7f235c0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f235b200640 1 -- 172.21.15.60:0/241877042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c0ca430 con 0x7f235c1438b0 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f2359e00640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c0ccd00 0x7f235c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f235b200640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c1438b0 0x7f235c0caf70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f235400c780 tx=0x7f235400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f2352a00640 1 -- 172.21.15.60:0/241877042 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354004b60 con 0x7f235c1438b0 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f2352a00640 1 -- 172.21.15.60:0/241877042 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2354004df0 con 0x7f235c1438b0 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c0ca710 con 0x7f235c1438b0 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f2352a00640 1 -- 172.21.15.60:0/241877042 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235400aa70 con 0x7f235c1438b0 2024-09-16T15:46:39.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.016+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c15b250 con 0x7f235c1438b0 2024-09-16T15:46:39.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.017+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235c06f850 con 0x7f235c1438b0 2024-09-16T15:46:39.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.018+0000 7f2352a00640 1 -- 172.21.15.60:0/241877042 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f235400ac10 con 0x7f235c1438b0 2024-09-16T15:46:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.019+0000 7f2352a00640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f233c078f10 0x7f233c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.020+0000 7f2352a00640 1 -- 172.21.15.60:0/241877042 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7f235409b3b0 con 0x7f235c1438b0 2024-09-16T15:46:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.020+0000 7f235a800640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f233c078f10 0x7f233c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.020+0000 7f235a800640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f233c078f10 0x7f233c07b3d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f234c001720 tx=0x7f234c007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.024+0000 7f2352a00640 1 -- 172.21.15.60:0/241877042 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2354020070 con 0x7f235c1438b0 2024-09-16T15:46:39.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.228+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f235c0aea90 con 0x7f235c1438b0 2024-09-16T15:46:39.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.232+0000 7f2352a00640 1 -- 172.21.15.60:0/241877042 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7f2354063420 con 0x7f235c1438b0 2024-09-16T15:46:39.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f233c078f10 msgr2=0x7f233c07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f233c078f10 0x7f233c07b3d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f234c001720 tx=0x7f234c007e90 comp rx=0 tx=0).stop 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c1438b0 msgr2=0x7f235c0caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c1438b0 0x7f235c0caf70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f235400c780 tx=0x7f235400cc40 comp rx=0 tx=0).stop 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 shutdown_connections 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f233c078f10 0x7f233c07b3d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f235c1438b0 0x7f235c0caf70 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f235c0ccd00 0x7f235c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 --2- 172.21.15.60:0/241877042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f235c06fd90 0x7f235c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 >> 172.21.15.60:0/241877042 conn(0x7f235c0b1950 msgr2=0x7f235c0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 shutdown_connections 2024-09-16T15:46:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.234+0000 7f2361400640 1 -- 172.21.15.60:0/241877042 wait complete. 2024-09-16T15:46:39.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-16T15:46:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.413+0000 7fa6e2e00640 1 -- 172.21.15.60:0/1821581720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc0cf300 msgr2=0x7fa6dc0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.413+0000 7fa6e2e00640 1 --2- 172.21.15.60:0/1821581720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc0cf300 0x7fa6dc0729a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009890 tx=0x7fa6c802a650 comp rx=0 tx=0).stop 2024-09-16T15:46:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.413+0000 7fa6e2e00640 1 -- 172.21.15.60:0/1821581720 shutdown_connections 2024-09-16T15:46:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.413+0000 7fa6e2e00640 1 --2- 172.21.15.60:0/1821581720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc0cf300 0x7fa6dc0729a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.413+0000 7fa6e2e00640 1 -- 172.21.15.60:0/1821581720 >> 172.21.15.60:0/1821581720 conn(0x7fa6dc0b1950 msgr2=0x7fa6dc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.413+0000 7fa6e2e00640 1 -- 172.21.15.60:0/1821581720 shutdown_connections 2024-09-16T15:46:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.413+0000 7fa6e2e00640 1 -- 172.21.15.60:0/1821581720 wait complete. 2024-09-16T15:46:39.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.415+0000 7fa6e2e00640 1 Processor -- start 2024-09-16T15:46:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.415+0000 7fa6e2e00640 1 -- start start 2024-09-16T15:46:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6e2e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6dc0ccfd0 0x7fa6dc09d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6e2e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6dc0cf300 0x7fa6dc15e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6e2e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc1445d0 0x7fa6dc157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6e2e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0735f0 con 0x7fa6dc0cf300 2024-09-16T15:46:39.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6e2e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0b4f30 con 0x7fa6dc0ccfd0 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6e2e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc073770 con 0x7fa6dc1445d0 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6db400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6dc0cf300 0x7fa6dc15e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6db400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6dc0cf300 0x7fa6dc15e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57474/0 (socket says 172.21.15.60:57474) 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6db400640 1 -- 172.21.15.60:0/2482374172 learned_addr learned my addr 172.21.15.60:0/2482374172 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6dbe00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6dc0ccfd0 0x7fa6dc09d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6e0e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc1445d0 0x7fa6dc157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6db400640 1 -- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc1445d0 msgr2=0x7fa6dc157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6db400640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc1445d0 0x7fa6dc157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6db400640 1 -- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6dc0ccfd0 msgr2=0x7fa6dc09d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6db400640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6dc0ccfd0 0x7fa6dc09d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6db400640 1 -- 172.21.15.60:0/2482374172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc15afe0 con 0x7fa6dc0cf300 2024-09-16T15:46:39.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.416+0000 7fa6e0e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc1445d0 0x7fa6dc157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.417+0000 7fa6db400640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6dc0cf300 0x7fa6dc15e150 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80099b0 tx=0x7fa6c8032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.417+0000 7fa6d3e00640 1 -- 172.21.15.60:0/2482374172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8040070 con 0x7fa6dc0cf300 2024-09-16T15:46:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.417+0000 7fa6d3e00640 1 -- 172.21.15.60:0/2482374172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6c80038e0 con 0x7fa6dc0cf300 2024-09-16T15:46:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.417+0000 7fa6d3e00640 1 -- 172.21.15.60:0/2482374172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8003d30 con 0x7fa6dc0cf300 2024-09-16T15:46:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.417+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc09c5e0 con 0x7fa6dc0cf300 2024-09-16T15:46:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.417+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc09cba0 con 0x7fa6dc0cf300 2024-09-16T15:46:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.418+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc073c20 con 0x7fa6dc0cf300 2024-09-16T15:46:39.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.419+0000 7fa6d3e00640 1 -- 172.21.15.60:0/2482374172 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fa6c8039040 con 0x7fa6dc0cf300 2024-09-16T15:46:39.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.420+0000 7fa6d3e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa6bc078f10 0x7fa6bc07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.420+0000 7fa6d3e00640 1 -- 172.21.15.60:0/2482374172 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7fa6c80bc720 con 0x7fa6dc0cf300 2024-09-16T15:46:39.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.421+0000 7fa6dbe00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa6bc078f10 0x7fa6bc07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.421+0000 7fa6dbe00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa6bc078f10 0x7fa6bc07b3d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc001120 tx=0x7fa6cc009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.424+0000 7fa6d3e00640 1 -- 172.21.15.60:0/2482374172 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa6c8084590 con 0x7fa6dc0cf300 2024-09-16T15:46:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.693+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7fa6dc15f880 con 0x7fa6dc0cf300 2024-09-16T15:46:39.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.825+0000 7fa6d3e00640 1 -- 172.21.15.60:0/2482374172 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v447) ==== 151+0+0 (secure 0 0 0) 0x7fa6c8083e90 con 0x7fa6dc0cf300 2024-09-16T15:46:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa6bc078f10 msgr2=0x7fa6bc07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa6bc078f10 0x7fa6bc07b3d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc001120 tx=0x7fa6cc009040 comp rx=0 tx=0).stop 2024-09-16T15:46:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6dc0cf300 msgr2=0x7fa6dc15e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6dc0cf300 0x7fa6dc15e150 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80099b0 tx=0x7fa6c8032b90 comp rx=0 tx=0).stop 2024-09-16T15:46:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 shutdown_connections 2024-09-16T15:46:39.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa6bc078f10 0x7fa6bc07b3d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6dc1445d0 0x7fa6dc157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6dc0cf300 0x7fa6dc15e150 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 --2- 172.21.15.60:0/2482374172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6dc0ccfd0 0x7fa6dc09d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 >> 172.21.15.60:0/2482374172 conn(0x7fa6dc0b1950 msgr2=0x7fa6dc0af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 shutdown_connections 2024-09-16T15:46:39.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:39.828+0000 7fa6e2e00640 1 -- 172.21.15.60:0/2482374172 wait complete. 2024-09-16T15:46:39.844 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-16T15:46:39.845 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-16T15:46:39.845 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:46:39.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.006+0000 7f5ae1800640 1 -- 172.21.15.60:0/2900580749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc15f5a0 msgr2=0x7f5adc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.006+0000 7f5ae1800640 1 --2- 172.21.15.60:0/2900580749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc15f5a0 0x7f5adc0b2440 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f5adc06dac0 tx=0x7f5acc02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.006+0000 7f5ae1800640 1 -- 172.21.15.60:0/2900580749 shutdown_connections 2024-09-16T15:46:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.006+0000 7f5ae1800640 1 --2- 172.21.15.60:0/2900580749 >> v2:172.21.15.100:3300/0 conn(0x7f5adc073fb0 0x7f5adc074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.006+0000 7f5ae1800640 1 --2- 172.21.15.60:0/2900580749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc15f5a0 0x7f5adc0b2440 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.006+0000 7f5ae1800640 1 -- 172.21.15.60:0/2900580749 >> 172.21.15.60:0/2900580749 conn(0x7f5adc0b1950 msgr2=0x7f5adc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.006+0000 7f5ae1800640 1 -- 172.21.15.60:0/2900580749 shutdown_connections 2024-09-16T15:46:40.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.006+0000 7f5ae1800640 1 -- 172.21.15.60:0/2900580749 wait complete. 2024-09-16T15:46:40.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.007+0000 7f5ae1800640 1 Processor -- start 2024-09-16T15:46:40.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.008+0000 7f5ae1800640 1 -- start start 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.008+0000 7f5ae1800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5adc073fb0 0x7f5adc14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.008+0000 7f5ae1800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc0ccf00 0x7f5adc0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.008+0000 7f5ae1800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5adc15f5a0 0x7f5adc0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.008+0000 7f5ae1800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc06f2e0 con 0x7f5adc0ccf00 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.008+0000 7f5ae1800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc06f160 con 0x7f5adc15f5a0 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.008+0000 7f5ae1800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc06f460 con 0x7f5adc073fb0 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc0ccf00 0x7f5adc0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc0ccf00 0x7f5adc0c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57480/0 (socket says 172.21.15.60:57480) 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 -- 172.21.15.60:0/143400308 learned_addr learned my addr 172.21.15.60:0/143400308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5adaa00640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5adc073fb0 0x7f5adc14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5adb400640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5adc15f5a0 0x7f5adc0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 -- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5adc073fb0 msgr2=0x7f5adc14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5adc073fb0 0x7f5adc14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 -- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5adc15f5a0 msgr2=0x7f5adc0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5adc15f5a0 0x7f5adc0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 -- 172.21.15.60:0/143400308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc0ca230 con 0x7f5adc0ccf00 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5adb400640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5adc15f5a0 0x7f5adc0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5adaa00640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5adc073fb0 0x7f5adc14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.009+0000 7f5ada000640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc0ccf00 0x7f5adc0c8fd0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800c430 tx=0x7f5ac800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.010+0000 7f5ad2a00640 1 -- 172.21.15.60:0/143400308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8011790 con 0x7f5adc0ccf00 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.010+0000 7f5ad2a00640 1 -- 172.21.15.60:0/143400308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ac8017070 con 0x7f5adc0ccf00 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.010+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc0c9900 con 0x7f5adc0ccf00 2024-09-16T15:46:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.010+0000 7f5ad2a00640 1 -- 172.21.15.60:0/143400308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8011d00 con 0x7f5adc0ccf00 2024-09-16T15:46:40.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.010+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc0ca4d0 con 0x7f5adc0ccf00 2024-09-16T15:46:40.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.011+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5adc0c9f80 con 0x7f5adc0ccf00 2024-09-16T15:46:40.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.012+0000 7f5ad2a00640 1 -- 172.21.15.60:0/143400308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f5ac801c460 con 0x7f5adc0ccf00 2024-09-16T15:46:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.013+0000 7f5ad2a00640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5abc078f10 0x7f5abc07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.013+0000 7f5ad2a00640 1 -- 172.21.15.60:0/143400308 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7f5ac809b0f0 con 0x7f5adc0ccf00 2024-09-16T15:46:40.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.013+0000 7f5adaa00640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5abc078f10 0x7f5abc07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.014+0000 7f5adaa00640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5abc078f10 0x7f5abc07b3d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5acc033000 tx=0x7f5acc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.017+0000 7f5ad2a00640 1 -- 172.21.15.60:0/143400308 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5ac80630f0 con 0x7f5adc0ccf00 2024-09-16T15:46:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.216+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5adc06f9f0 con 0x7f5adc0ccf00 2024-09-16T15:46:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.216+0000 7f5ad2a00640 1 -- 172.21.15.60:0/143400308 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+13084 (secure 0 0 0) 0x7f5ac80629f0 con 0x7f5adc0ccf00 2024-09-16T15:46:40.217 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:40.218 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":140,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:38.814266+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/984341325":"2024-09-17T15:46:35.609958+0000","172.21.15.60:6833/984341325":"2024-09-17T15:46:35.609958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:40.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5abc078f10 msgr2=0x7f5abc07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5abc078f10 0x7f5abc07b3d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5acc033000 tx=0x7f5acc004730 comp rx=0 tx=0).stop 2024-09-16T15:46:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc0ccf00 msgr2=0x7f5adc0c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc0ccf00 0x7f5adc0c8fd0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800c430 tx=0x7f5ac800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 shutdown_connections 2024-09-16T15:46:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5abc078f10 0x7f5abc07b3d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5adc15f5a0 0x7f5adc0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5adc0ccf00 0x7f5adc0c8fd0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 --2- 172.21.15.60:0/143400308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5adc073fb0 0x7f5adc14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 >> 172.21.15.60:0/143400308 conn(0x7f5adc0b1950 msgr2=0x7f5adc0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.219+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 shutdown_connections 2024-09-16T15:46:40.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.220+0000 7f5ae1800640 1 -- 172.21.15.60:0/143400308 wait complete. 2024-09-16T15:46:40.238 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:46:40.238 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:46:40.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.401+0000 7f3c9b400640 1 -- 172.21.15.60:0/3826616372 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c940cccd0 msgr2=0x7f3c94073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.401+0000 7f3c9b400640 1 --2- 172.21.15.60:0/3826616372 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c940cccd0 0x7f3c94073440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009860 tx=0x7f3c7c02a510 comp rx=0 tx=0).stop 2024-09-16T15:46:40.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.402+0000 7f3c9b400640 1 -- 172.21.15.60:0/3826616372 shutdown_connections 2024-09-16T15:46:40.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.402+0000 7f3c9b400640 1 --2- 172.21.15.60:0/3826616372 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c940cccd0 0x7f3c94073440 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.402+0000 7f3c9b400640 1 -- 172.21.15.60:0/3826616372 >> 172.21.15.60:0/3826616372 conn(0x7f3c940b1840 msgr2=0x7f3c940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.402+0000 7f3c9b400640 1 -- 172.21.15.60:0/3826616372 shutdown_connections 2024-09-16T15:46:40.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.402+0000 7f3c9b400640 1 -- 172.21.15.60:0/3826616372 wait complete. 2024-09-16T15:46:40.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.403+0000 7f3c9b400640 1 Processor -- start 2024-09-16T15:46:40.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.403+0000 7f3c9b400640 1 -- start start 2024-09-16T15:46:40.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.404+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c9406efa0 0x7f3c94138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.404+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c940cccd0 0x7f3c9415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.404+0000 7f3c9b400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c94145750 0x7f3c940ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.404+0000 7f3c9b400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3c940ce6d0 con 0x7f3c940cccd0 2024-09-16T15:46:40.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.404+0000 7f3c9b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3c940ce550 con 0x7f3c9406efa0 2024-09-16T15:46:40.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.404+0000 7f3c9b400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3c940ce850 con 0x7f3c94145750 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c8be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c940cccd0 0x7f3c9415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c8be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c940cccd0 0x7f3c9415e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57492/0 (socket says 172.21.15.60:57492) 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c8be00640 1 -- 172.21.15.60:0/3483952610 learned_addr learned my addr 172.21.15.60:0/3483952610 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c99400640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c94145750 0x7f3c940ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c98a00640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c9406efa0 0x7f3c94138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c8be00640 1 -- 172.21.15.60:0/3483952610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c94145750 msgr2=0x7f3c940ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c8be00640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c94145750 0x7f3c940ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c8be00640 1 -- 172.21.15.60:0/3483952610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c9406efa0 msgr2=0x7f3c94138470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c8be00640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c9406efa0 0x7f3c94138470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.405+0000 7f3c8be00640 1 -- 172.21.15.60:0/3483952610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c941393d0 con 0x7f3c940cccd0 2024-09-16T15:46:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.406+0000 7f3c8be00640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c940cccd0 0x7f3c9415e9e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009980 tx=0x7f3c7c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.406+0000 7f3c91600640 1 -- 172.21.15.60:0/3483952610 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c040070 con 0x7f3c940cccd0 2024-09-16T15:46:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.406+0000 7f3c91600640 1 -- 172.21.15.60:0/3483952610 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c7c005310 con 0x7f3c940cccd0 2024-09-16T15:46:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.406+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c94138b30 con 0x7f3c940cccd0 2024-09-16T15:46:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.406+0000 7f3c91600640 1 -- 172.21.15.60:0/3483952610 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c005c90 con 0x7f3c940cccd0 2024-09-16T15:46:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.406+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c941390f0 con 0x7f3c940cccd0 2024-09-16T15:46:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.407+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c940ced00 con 0x7f3c940cccd0 2024-09-16T15:46:40.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.408+0000 7f3c91600640 1 -- 172.21.15.60:0/3483952610 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f3c7c039040 con 0x7f3c940cccd0 2024-09-16T15:46:40.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.410+0000 7f3c91600640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3c70078f10 0x7f3c7007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.410+0000 7f3c91600640 1 -- 172.21.15.60:0/3483952610 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7f3c7c0bc3a0 con 0x7f3c940cccd0 2024-09-16T15:46:40.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.410+0000 7f3c98a00640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3c70078f10 0x7f3c7007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.410+0000 7f3c98a00640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3c70078f10 0x7f3c7007b3d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3c84001120 tx=0x7f3c84009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.414+0000 7f3c91600640 1 -- 172.21.15.60:0/3483952610 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3c7c083cc0 con 0x7f3c940cccd0 2024-09-16T15:46:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.610+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c94144af0 con 0x7f3c940cccd0 2024-09-16T15:46:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.610+0000 7f3c91600640 1 -- 172.21.15.60:0/3483952610 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) ==== 93+0+10 (secure 0 0 0) 0x7f3c7c0886f0 con 0x7f3c940cccd0 2024-09-16T15:46:40.611 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:46:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3c70078f10 msgr2=0x7f3c7007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3c70078f10 0x7f3c7007b3d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3c84001120 tx=0x7f3c84009040 comp rx=0 tx=0).stop 2024-09-16T15:46:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c940cccd0 msgr2=0x7f3c9415e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c940cccd0 0x7f3c9415e9e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009980 tx=0x7f3c7c032c10 comp rx=0 tx=0).stop 2024-09-16T15:46:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 shutdown_connections 2024-09-16T15:46:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3c70078f10 0x7f3c7007b3d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c94145750 0x7f3c940ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c940cccd0 0x7f3c9415e9e0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 --2- 172.21.15.60:0/3483952610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c9406efa0 0x7f3c94138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 >> 172.21.15.60:0/3483952610 conn(0x7f3c940b1840 msgr2=0x7f3c940a9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 shutdown_connections 2024-09-16T15:46:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.613+0000 7f3c9b400640 1 -- 172.21.15.60:0/3483952610 wait complete. 2024-09-16T15:46:40.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-16T15:46:40.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.790+0000 7f3f85600640 1 -- 172.21.15.60:0/3669853587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 msgr2=0x7f3f80074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.790+0000 7f3f85600640 1 --2- 172.21.15.60:0/3669853587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 0x7f3f80074420 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3f800cfbb0 tx=0x7f3f7802a820 comp rx=0 tx=0).stop 2024-09-16T15:46:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.790+0000 7f3f85600640 1 -- 172.21.15.60:0/3669853587 shutdown_connections 2024-09-16T15:46:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.790+0000 7f3f85600640 1 --2- 172.21.15.60:0/3669853587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 0x7f3f80074420 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.790+0000 7f3f85600640 1 --2- 172.21.15.60:0/3669853587 >> v2:172.21.15.100:3300/0 conn(0x7f3f800ccf00 0x7f3f800b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.790+0000 7f3f85600640 1 -- 172.21.15.60:0/3669853587 >> 172.21.15.60:0/3669853587 conn(0x7f3f800b1950 msgr2=0x7f3f800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.790+0000 7f3f85600640 1 -- 172.21.15.60:0/3669853587 shutdown_connections 2024-09-16T15:46:40.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.790+0000 7f3f85600640 1 -- 172.21.15.60:0/3669853587 wait complete. 2024-09-16T15:46:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.792+0000 7f3f85600640 1 Processor -- start 2024-09-16T15:46:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.792+0000 7f3f85600640 1 -- start start 2024-09-16T15:46:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f85600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 0x7f3f800d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f85600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ccf00 0x7f3f80150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f85600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015f5a0 0x7f3f80150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f85600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8006f2e0 con 0x7f3f80073fb0 2024-09-16T15:46:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f85600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8006f160 con 0x7f3f8015f5a0 2024-09-16T15:46:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f85600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8006f460 con 0x7f3f800ccf00 2024-09-16T15:46:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f7ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 0x7f3f800d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f7ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 0x7f3f800d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57510/0 (socket says 172.21.15.60:57510) 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.793+0000 7f3f7ea00640 1 -- 172.21.15.60:0/2954060923 learned_addr learned my addr 172.21.15.60:0/2954060923 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f7f400640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015f5a0 0x7f3f80150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f76000640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ccf00 0x7f3f80150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f7ea00640 1 -- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ccf00 msgr2=0x7f3f80150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f7ea00640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ccf00 0x7f3f80150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f7ea00640 1 -- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015f5a0 msgr2=0x7f3f80150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f7ea00640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015f5a0 0x7f3f80150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f7ea00640 1 -- 172.21.15.60:0/2954060923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f80152a90 con 0x7f3f80073fb0 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f76000640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ccf00 0x7f3f80150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f7ea00640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 0x7f3f800d0040 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c3d0 tx=0x7f3f7000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f77400640 1 -- 172.21.15.60:0/2954060923 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70011790 con 0x7f3f80073fb0 2024-09-16T15:46:40.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f77400640 1 -- 172.21.15.60:0/2954060923 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f70017070 con 0x7f3f80073fb0 2024-09-16T15:46:40.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.794+0000 7f3f77400640 1 -- 172.21.15.60:0/2954060923 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70011cc0 con 0x7f3f80073fb0 2024-09-16T15:46:40.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.795+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f80152d10 con 0x7f3f80073fb0 2024-09-16T15:46:40.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.795+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f801532d0 con 0x7f3f80073fb0 2024-09-16T15:46:40.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.796+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8006f9f0 con 0x7f3f80073fb0 2024-09-16T15:46:40.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.797+0000 7f3f77400640 1 -- 172.21.15.60:0/2954060923 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f3f7001c460 con 0x7f3f80073fb0 2024-09-16T15:46:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.798+0000 7f3f77400640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f5c078f10 0x7f3f5c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.798+0000 7f3f77400640 1 -- 172.21.15.60:0/2954060923 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7f3f7009aea0 con 0x7f3f80073fb0 2024-09-16T15:46:40.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.798+0000 7f3f76000640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f5c078f10 0x7f3f5c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.799+0000 7f3f76000640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f5c078f10 0x7f3f5c07b3d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3f8014ce20 tx=0x7f3f6800b2c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:40.802+0000 7f3f77400640 1 -- 172.21.15.60:0/2954060923 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3f80152a90 con 0x7f3f80073fb0 2024-09-16T15:46:41.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.076+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f3f800affb0 con 0x7f3f80073fb0 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.835+0000 7f3f77400640 1 -- 172.21.15.60:0/2954060923 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v449) ==== 152+0+0 (secure 0 0 0) 0x7f3f70062ea0 con 0x7f3f80073fb0 2024-09-16T15:46:41.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f5c078f10 msgr2=0x7f3f5c07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:41.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f5c078f10 0x7f3f5c07b3d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3f8014ce20 tx=0x7f3f6800b2c0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 msgr2=0x7f3f800d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:41.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 0x7f3f800d0040 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c3d0 tx=0x7f3f7000c890 comp rx=0 tx=0).stop 2024-09-16T15:46:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 shutdown_connections 2024-09-16T15:46:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3f5c078f10 0x7f3f5c07b3d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8015f5a0 0x7f3f80150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f800ccf00 0x7f3f80150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 --2- 172.21.15.60:0/2954060923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f80073fb0 0x7f3f800d0040 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 >> 172.21.15.60:0/2954060923 conn(0x7f3f800b1950 msgr2=0x7f3f8015dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 shutdown_connections 2024-09-16T15:46:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:41.838+0000 7f3f85600640 1 -- 172.21.15.60:0/2954060923 wait complete. 2024-09-16T15:46:41.859 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:46:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.024+0000 7ff950400640 1 -- 172.21.15.60:0/1635870957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff94815f5a0 msgr2=0x7ff9480b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.024+0000 7ff950400640 1 --2- 172.21.15.60:0/1635870957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff94815f5a0 0x7ff9480b2440 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009830 tx=0x7ff93c02a480 comp rx=0 tx=0).stop 2024-09-16T15:46:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.024+0000 7ff950400640 1 -- 172.21.15.60:0/1635870957 shutdown_connections 2024-09-16T15:46:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.024+0000 7ff950400640 1 --2- 172.21.15.60:0/1635870957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff948073fb0 0x7ff948074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.024+0000 7ff950400640 1 --2- 172.21.15.60:0/1635870957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff94815f5a0 0x7ff9480b2440 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.024+0000 7ff950400640 1 -- 172.21.15.60:0/1635870957 >> 172.21.15.60:0/1635870957 conn(0x7ff9480b1950 msgr2=0x7ff9480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.024+0000 7ff950400640 1 -- 172.21.15.60:0/1635870957 shutdown_connections 2024-09-16T15:46:42.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.024+0000 7ff950400640 1 -- 172.21.15.60:0/1635870957 wait complete. 2024-09-16T15:46:42.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.026+0000 7ff950400640 1 Processor -- start 2024-09-16T15:46:42.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.026+0000 7ff950400640 1 -- start start 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff950400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff948073fb0 0x7ff9481518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff950400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9480ccf00 0x7ff9481502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff950400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff94815f5a0 0x7ff9481f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff950400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff94806f2e0 con 0x7ff94815f5a0 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff950400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff94806f160 con 0x7ff948073fb0 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff950400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff94806f460 con 0x7ff9480ccf00 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff94e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff94815f5a0 0x7ff9481f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff94e400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff94815f5a0 0x7ff9481f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57524/0 (socket says 172.21.15.60:57524) 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff94e400640 1 -- 172.21.15.60:0/1670478198 learned_addr learned my addr 172.21.15.60:0/1670478198 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff94da00640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff948073fb0 0x7ff9481518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.027+0000 7ff94d000640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9480ccf00 0x7ff9481502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff94e400640 1 -- 172.21.15.60:0/1670478198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9480ccf00 msgr2=0x7ff9481502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff94e400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9480ccf00 0x7ff9481502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff94e400640 1 -- 172.21.15.60:0/1670478198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff948073fb0 msgr2=0x7ff9481518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff94e400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff948073fb0 0x7ff9481518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff94e400640 1 -- 172.21.15.60:0/1670478198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9481507e0 con 0x7ff94815f5a0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff94e400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff94815f5a0 0x7ff9481f3a40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ff94000efd0 tx=0x7ff94000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff945600640 1 -- 172.21.15.60:0/1670478198 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940003b70 con 0x7ff94815f5a0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff945600640 1 -- 172.21.15.60:0/1670478198 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff940003e00 con 0x7ff94815f5a0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff945600640 1 -- 172.21.15.60:0/1670478198 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940004280 con 0x7ff94815f5a0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9481f3f80 con 0x7ff94815f5a0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.028+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94814d970 con 0x7ff94815f5a0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.029+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94806f980 con 0x7ff94815f5a0 2024-09-16T15:46:42.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.031+0000 7ff945600640 1 -- 172.21.15.60:0/1670478198 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7ff940004420 con 0x7ff94815f5a0 2024-09-16T15:46:42.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.031+0000 7ff945600640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff928078f10 0x7ff92807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.032+0000 7ff945600640 1 -- 172.21.15.60:0/1670478198 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7ff94009c1e0 con 0x7ff94815f5a0 2024-09-16T15:46:42.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.032+0000 7ff94da00640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff928078f10 0x7ff92807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.032+0000 7ff94da00640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff928078f10 0x7ff92807b3d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff93c034000 tx=0x7ff93c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:42.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.036+0000 7ff945600640 1 -- 172.21.15.60:0/1670478198 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff940063a00 con 0x7ff94815f5a0 2024-09-16T15:46:42.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.239+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff948154a40 con 0x7ff94815f5a0 2024-09-16T15:46:42.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.239+0000 7ff945600640 1 -- 172.21.15.60:0/1670478198 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+1998 (secure 0 0 0) 0x7ff940068430 con 0x7ff94815f5a0 2024-09-16T15:46:42.240 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:46:42.240 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:46:42.240 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:46:42.241 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:46:42.242 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff928078f10 msgr2=0x7ff92807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff928078f10 0x7ff92807b3d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff93c034000 tx=0x7ff93c004730 comp rx=0 tx=0).stop 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff94815f5a0 msgr2=0x7ff9481f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff94815f5a0 0x7ff9481f3a40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ff94000efd0 tx=0x7ff94000c6a0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 shutdown_connections 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff928078f10 0x7ff92807b3d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff94815f5a0 0x7ff9481f3a40 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9480ccf00 0x7ff9481502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 --2- 172.21.15.60:0/1670478198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff948073fb0 0x7ff9481518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 >> 172.21.15.60:0/1670478198 conn(0x7ff9480b1950 msgr2=0x7ff9480b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.241+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 shutdown_connections 2024-09-16T15:46:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.242+0000 7ff950400640 1 -- 172.21.15.60:0/1670478198 wait complete. 2024-09-16T15:46:42.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:46:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.421+0000 7fc1c8a00640 1 -- 172.21.15.60:0/2590523022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 msgr2=0x7fc1c4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.421+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/2590523022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 0x7fc1c4073440 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009830 tx=0x7fc1b402a480 comp rx=0 tx=0).stop 2024-09-16T15:46:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.421+0000 7fc1c8a00640 1 -- 172.21.15.60:0/2590523022 shutdown_connections 2024-09-16T15:46:42.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.421+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/2590523022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1c406efa0 0x7fc1c40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.421+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/2590523022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 0x7fc1c4073440 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.421+0000 7fc1c8a00640 1 -- 172.21.15.60:0/2590523022 >> 172.21.15.60:0/2590523022 conn(0x7fc1c40b1840 msgr2=0x7fc1c40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.421+0000 7fc1c8a00640 1 -- 172.21.15.60:0/2590523022 shutdown_connections 2024-09-16T15:46:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.421+0000 7fc1c8a00640 1 -- 172.21.15.60:0/2590523022 wait complete. 2024-09-16T15:46:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.422+0000 7fc1c8a00640 1 Processor -- start 2024-09-16T15:46:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.423+0000 7fc1c8a00640 1 -- start start 2024-09-16T15:46:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.423+0000 7fc1c8a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1c406efa0 0x7fc1c40c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.423+0000 7fc1c8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c40ccb00 0x7fc1c40c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.423+0000 7fc1c8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 0x7fc1c40c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.423+0000 7fc1c8a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40b26b0 con 0x7fc1c4145750 2024-09-16T15:46:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.423+0000 7fc1c8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40cedd0 con 0x7fc1c40ccb00 2024-09-16T15:46:42.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.423+0000 7fc1c8a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c4160100 con 0x7fc1c406efa0 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 0x7fc1c40c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 0x7fc1c40c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57548/0 (socket says 172.21.15.60:57548) 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 -- 172.21.15.60:0/912992341 learned_addr learned my addr 172.21.15.60:0/912992341 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c1400640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c40ccb00 0x7fc1c40c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c1e00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1c406efa0 0x7fc1c40c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 -- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1c406efa0 msgr2=0x7fc1c40c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1c406efa0 0x7fc1c40c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 -- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c40ccb00 msgr2=0x7fc1c40c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c40ccb00 0x7fc1c40c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 -- 172.21.15.60:0/912992341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c41325d0 con 0x7fc1c4145750 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c1e00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1c406efa0 0x7fc1c40c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c1400640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c40ccb00 0x7fc1c40c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.424+0000 7fc1c2800640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 0x7fc1c40c7d20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00cbf0 tx=0x7fc1bc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.425+0000 7fc1ba000640 1 -- 172.21.15.60:0/912992341 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc016020 con 0x7fc1c4145750 2024-09-16T15:46:42.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.425+0000 7fc1ba000640 1 -- 172.21.15.60:0/912992341 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1bc00ad50 con 0x7fc1c4145750 2024-09-16T15:46:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.425+0000 7fc1ba000640 1 -- 172.21.15.60:0/912992341 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc012680 con 0x7fc1c4145750 2024-09-16T15:46:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.425+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c41328b0 con 0x7fc1c4145750 2024-09-16T15:46:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.425+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c4132e70 con 0x7fc1c4145750 2024-09-16T15:46:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.426+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c415f2a0 con 0x7fc1c4145750 2024-09-16T15:46:42.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.427+0000 7fc1ba000640 1 -- 172.21.15.60:0/912992341 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fc1bc019020 con 0x7fc1c4145750 2024-09-16T15:46:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.428+0000 7fc1ba000640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc1a4078f40 0x7fc1a407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.428+0000 7fc1ba000640 1 -- 172.21.15.60:0/912992341 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7fc1bc09b7d0 con 0x7fc1c4145750 2024-09-16T15:46:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.428+0000 7fc1c1e00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc1a4078f40 0x7fc1a407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.429+0000 7fc1c1e00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc1a4078f40 0x7fc1a407b400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4034000 tx=0x7fc1b4004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.432+0000 7fc1ba000640 1 -- 172.21.15.60:0/912992341 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc1c415f2a0 con 0x7fc1c4145750 2024-09-16T15:46:42.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.626+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc1c41605b0 con 0x7fc1c4145750 2024-09-16T15:46:42.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.630+0000 7fc1ba000640 1 -- 172.21.15.60:0/912992341 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fc1bc020030 con 0x7fc1c4145750 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.632+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc1a4078f40 msgr2=0x7fc1a407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.632+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc1a4078f40 0x7fc1a407b400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4034000 tx=0x7fc1b4004730 comp rx=0 tx=0).stop 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 msgr2=0x7fc1c40c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 0x7fc1c40c7d20 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00cbf0 tx=0x7fc1bc00a430 comp rx=0 tx=0).stop 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 shutdown_connections 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc1a4078f40 0x7fc1a407b400 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1c4145750 0x7fc1c40c7d20 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c40ccb00 0x7fc1c40c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 --2- 172.21.15.60:0/912992341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1c406efa0 0x7fc1c40c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 >> 172.21.15.60:0/912992341 conn(0x7fc1c40b1840 msgr2=0x7fc1c40b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:42.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 shutdown_connections 2024-09-16T15:46:42.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:42.633+0000 7fc1c8a00640 1 -- 172.21.15.60:0/912992341 wait complete. 2024-09-16T15:46:42.650 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:46:42.651 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:46:42.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:46:42.835 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:46:42.836 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:46:42.836 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:42.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:46:42.872 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:46:42.872 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:42.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:46:42.910 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:46:42.910 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:42.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:46:42.947 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:46:42.947 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:42.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:46:42.986 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:46:42.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.146+0000 7f869d400640 1 -- 172.21.15.60:0/975390641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 msgr2=0x7f86980b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.146+0000 7f869d400640 1 --2- 172.21.15.60:0/975390641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 0x7f86980b2440 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8688009830 tx=0x7f868802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.147+0000 7f869d400640 1 -- 172.21.15.60:0/975390641 shutdown_connections 2024-09-16T15:46:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.147+0000 7f869d400640 1 --2- 172.21.15.60:0/975390641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 0x7f86980b2440 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.147+0000 7f869d400640 1 -- 172.21.15.60:0/975390641 >> 172.21.15.60:0/975390641 conn(0x7f86980b1950 msgr2=0x7f86980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.147+0000 7f869d400640 1 -- 172.21.15.60:0/975390641 shutdown_connections 2024-09-16T15:46:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.147+0000 7f869d400640 1 -- 172.21.15.60:0/975390641 wait complete. 2024-09-16T15:46:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.148+0000 7f869d400640 1 Processor -- start 2024-09-16T15:46:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.148+0000 7f869d400640 1 -- start start 2024-09-16T15:46:43.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.149+0000 7f869d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8698073fb0 0x7f86980cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.149+0000 7f869d400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86980ccf00 0x7f86981f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.149+0000 7f869d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 0x7f8698148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.149+0000 7f869d400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f869806f9f0 con 0x7f8698073fb0 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.149+0000 7f869d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f869806f870 con 0x7f869815f5a0 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.149+0000 7f869d400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8698074420 con 0x7f86980ccf00 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8698073fb0 0x7f86980cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8698073fb0 0x7f86980cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57562/0 (socket says 172.21.15.60:57562) 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 -- 172.21.15.60:0/482636552 learned_addr learned my addr 172.21.15.60:0/482636552 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8695e00640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86980ccf00 0x7f86981f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8697200640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 0x7f8698148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 -- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86980ccf00 msgr2=0x7f86981f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86980ccf00 0x7f86981f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 -- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 msgr2=0x7f8698148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 0x7f8698148510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 -- 172.21.15.60:0/482636552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869814b300 con 0x7f8698073fb0 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8697200640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 0x7f8698148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.150+0000 7f8696800640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8698073fb0 0x7f86980cb410 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f8688009950 tx=0x7f8688030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.151+0000 7f868ea00640 1 -- 172.21.15.60:0/482636552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868803f070 con 0x7f8698073fb0 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.151+0000 7f868ea00640 1 -- 172.21.15.60:0/482636552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8688030d40 con 0x7f8698073fb0 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.151+0000 7f868ea00640 1 -- 172.21.15.60:0/482636552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688004280 con 0x7f8698073fb0 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.151+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869814b5e0 con 0x7f8698073fb0 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.151+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869814bba0 con 0x7f8698073fb0 2024-09-16T15:46:43.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.152+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86980b2440 con 0x7f8698073fb0 2024-09-16T15:46:43.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.153+0000 7f868ea00640 1 -- 172.21.15.60:0/482636552 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f86880035f0 con 0x7f8698073fb0 2024-09-16T15:46:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.154+0000 7f868ea00640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8678078f10 0x7f867807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.154+0000 7f868ea00640 1 -- 172.21.15.60:0/482636552 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7f86880bbc10 con 0x7f8698073fb0 2024-09-16T15:46:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.154+0000 7f8695e00640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8678078f10 0x7f867807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.155+0000 7f8695e00640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8678078f10 0x7f867807b3d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8698155eb0 tx=0x7f8684009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:43.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.158+0000 7f868ea00640 1 -- 172.21.15.60:0/482636552 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f86880833a0 con 0x7f8698073fb0 2024-09-16T15:46:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.391+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86980b2140 con 0x7f8698073fb0 2024-09-16T15:46:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.392+0000 7f868ea00640 1 -- 172.21.15.60:0/482636552 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+3914 (secure 0 0 0) 0x7f8688087dd0 con 0x7f8698073fb0 2024-09-16T15:46:43.393 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:43.393 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":95,"btime":"2024-09-16T15:46:36:612332+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6833,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/766495901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":766495901},{"type":"v1","addr":"172.21.15.100:6835","nonce":766495901}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7585,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/3797165836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3797165836},{"type":"v1","addr":"172.21.15.100:6801","nonce":3797165836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3900896825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3900896825},{"type":"v1","addr":"172.21.15.60:6834","nonce":3900896825}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8367,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1697418176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1697418176},{"type":"v1","addr":"172.21.15.60:6835","nonce":1697418176}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[]} 2024-09-16T15:46:43.393 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 95 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8678078f10 msgr2=0x7f867807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8678078f10 0x7f867807b3d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8698155eb0 tx=0x7f8684009040 comp rx=0 tx=0).stop 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8698073fb0 msgr2=0x7f86980cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8698073fb0 0x7f86980cb410 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f8688009950 tx=0x7f8688030650 comp rx=0 tx=0).stop 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 shutdown_connections 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8678078f10 0x7f867807b3d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f869815f5a0 0x7f8698148510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86980ccf00 0x7f86981f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 --2- 172.21.15.60:0/482636552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8698073fb0 0x7f86980cb410 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 >> 172.21.15.60:0/482636552 conn(0x7f86980b1950 msgr2=0x7f869809d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.394+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 shutdown_connections 2024-09-16T15:46:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.395+0000 7f869d400640 1 -- 172.21.15.60:0/482636552 wait complete. 2024-09-16T15:46:43.411 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:46:43.411 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:46:43.412 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:46:43.412 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:46:43.412 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:46:43.412 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:46:43.412 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:46:43.412 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:46:43.412 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:46:43.412 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:46:43.413 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:46:43.413 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:46:43.413 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:46:43.413 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:46:43.413 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:46:43.413 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:46:43.426 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:46:43.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:46:43.427 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:46:43.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:46:43.430 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:46:43.430 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:46:43.431 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:46:43.431 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:46:43.471 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:46:43.471 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:43.473 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:46:43.473 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:43.474 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:46:43.475 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:43.477 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:46:43.477 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:43.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:46:43.605 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:46:43.605 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:46:43.605 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:46:43.606 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:46:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.646+0000 7ff846400640 1 -- 172.21.15.60:0/3045111500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff840073fb0 msgr2=0x7ff840074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.646+0000 7ff846400640 1 --2- 172.21.15.60:0/3045111500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff840073fb0 0x7ff840074420 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff840075130 tx=0x7ff83802aa30 comp rx=0 tx=0).stop 2024-09-16T15:46:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.646+0000 7ff846400640 1 -- 172.21.15.60:0/3045111500 shutdown_connections 2024-09-16T15:46:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.646+0000 7ff846400640 1 --2- 172.21.15.60:0/3045111500 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff840073fb0 0x7ff840074420 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.646+0000 7ff846400640 1 --2- 172.21.15.60:0/3045111500 >> v2:172.21.15.100:3300/0 conn(0x7ff84015f5a0 0x7ff8400b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.646+0000 7ff846400640 1 -- 172.21.15.60:0/3045111500 >> 172.21.15.60:0/3045111500 conn(0x7ff8400b1950 msgr2=0x7ff8400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.646+0000 7ff846400640 1 -- 172.21.15.60:0/3045111500 shutdown_connections 2024-09-16T15:46:43.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.647+0000 7ff846400640 1 -- 172.21.15.60:0/3045111500 wait complete. 2024-09-16T15:46:43.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.648+0000 7ff846400640 1 Processor -- start 2024-09-16T15:46:43.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.648+0000 7ff846400640 1 -- start start 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.649+0000 7ff846400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff840073fb0 0x7ff8401534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.649+0000 7ff846400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8400ccf00 0x7ff840156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.649+0000 7ff846400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff84015f5a0 0x7ff840157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.649+0000 7ff846400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff84006f2e0 con 0x7ff84015f5a0 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.649+0000 7ff846400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff84006f160 con 0x7ff840073fb0 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.649+0000 7ff846400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff84006f460 con 0x7ff8400ccf00 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.649+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff84015f5a0 0x7ff840157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff84015f5a0 0x7ff840157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57588/0 (socket says 172.21.15.60:57588) 2024-09-16T15:46:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83fe00640 1 -- 172.21.15.60:0/2120888721 learned_addr learned my addr 172.21.15.60:0/2120888721 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83f400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff840073fb0 0x7ff8401534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff836a00640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8400ccf00 0x7ff840156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83fe00640 1 -- 172.21.15.60:0/2120888721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8400ccf00 msgr2=0x7ff840156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83fe00640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8400ccf00 0x7ff840156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83fe00640 1 -- 172.21.15.60:0/2120888721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff840073fb0 msgr2=0x7ff8401534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83fe00640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff840073fb0 0x7ff8401534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83fe00640 1 -- 172.21.15.60:0/2120888721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff840153ff0 con 0x7ff84015f5a0 2024-09-16T15:46:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.650+0000 7ff83fe00640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff84015f5a0 0x7ff840157500 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff838009d60 tx=0x7ff838003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.651+0000 7ff837e00640 1 -- 172.21.15.60:0/2120888721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838047070 con 0x7ff84015f5a0 2024-09-16T15:46:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.651+0000 7ff837e00640 1 -- 172.21.15.60:0/2120888721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff838004250 con 0x7ff84015f5a0 2024-09-16T15:46:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.651+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff840153d10 con 0x7ff84015f5a0 2024-09-16T15:46:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.651+0000 7ff837e00640 1 -- 172.21.15.60:0/2120888721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838033050 con 0x7ff84015f5a0 2024-09-16T15:46:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.651+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff840158c40 con 0x7ff84015f5a0 2024-09-16T15:46:43.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.652+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84006f9f0 con 0x7ff84015f5a0 2024-09-16T15:46:43.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.653+0000 7ff837e00640 1 -- 172.21.15.60:0/2120888721 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7ff83804a070 con 0x7ff84015f5a0 2024-09-16T15:46:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.654+0000 7ff837e00640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff81c078f10 0x7ff81c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.654+0000 7ff837e00640 1 -- 172.21.15.60:0/2120888721 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4971+0+0 (secure 0 0 0) 0x7ff8380bc5a0 con 0x7ff84015f5a0 2024-09-16T15:46:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.654+0000 7ff83f400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff81c078f10 0x7ff81c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.655+0000 7ff83f400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff81c078f10 0x7ff81c07b3d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff830009b30 tx=0x7ff830005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:43.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.658+0000 7ff837e00640 1 -- 172.21.15.60:0/2120888721 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff838083dc0 con 0x7ff84015f5a0 2024-09-16T15:46:43.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:43.867+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff8400b26e0 con 0x7ff84015f5a0 2024-09-16T15:46:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.863+0000 7ff837e00640 1 -- 172.21.15.60:0/2120888721 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v145) ==== 97+0+0 (secure 0 0 0) 0x7ff8380887f0 con 0x7ff84015f5a0 2024-09-16T15:46:44.864 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:46:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff81c078f10 msgr2=0x7ff81c07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff81c078f10 0x7ff81c07b3d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff830009b30 tx=0x7ff830005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff84015f5a0 msgr2=0x7ff840157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff84015f5a0 0x7ff840157500 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff838009d60 tx=0x7ff838003f10 comp rx=0 tx=0).stop 2024-09-16T15:46:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 shutdown_connections 2024-09-16T15:46:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff81c078f10 0x7ff81c07b3d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff84015f5a0 0x7ff840157500 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8400ccf00 0x7ff840156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 --2- 172.21.15.60:0/2120888721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff840073fb0 0x7ff8401534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 >> 172.21.15.60:0/2120888721 conn(0x7ff8400b1950 msgr2=0x7ff8400ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:44.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 shutdown_connections 2024-09-16T15:46:44.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:44.867+0000 7ff846400640 1 -- 172.21.15.60:0/2120888721 wait complete. 2024-09-16T15:46:44.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:46:45.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.051+0000 7f9e23c00640 1 -- 172.21.15.60:0/3673881781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0d03a0 msgr2=0x7f9e1c0d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.051+0000 7f9e23c00640 1 --2- 172.21.15.60:0/3673881781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0d03a0 0x7f9e1c0d07f0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f9e04005fd0 tx=0x7f9e0402a670 comp rx=0 tx=0).stop 2024-09-16T15:46:45.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.051+0000 7f9e23c00640 1 -- 172.21.15.60:0/3673881781 shutdown_connections 2024-09-16T15:46:45.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.051+0000 7f9e23c00640 1 --2- 172.21.15.60:0/3673881781 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0d03a0 0x7f9e1c0d07f0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.051+0000 7f9e23c00640 1 -- 172.21.15.60:0/3673881781 >> 172.21.15.60:0/3673881781 conn(0x7f9e1c0d1dc0 msgr2=0x7f9e1c0d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.051+0000 7f9e23c00640 1 -- 172.21.15.60:0/3673881781 shutdown_connections 2024-09-16T15:46:45.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.051+0000 7f9e23c00640 1 -- 172.21.15.60:0/3673881781 wait complete. 2024-09-16T15:46:45.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.053+0000 7f9e23c00640 1 Processor -- start 2024-09-16T15:46:45.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.053+0000 7f9e23c00640 1 -- start start 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e23c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0b1b70 0x7f9e1c0c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e23c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c0d03a0 0x7f9e1c1f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e23c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e1c15f450 0x7f9e1c0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e23c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c0b26c0 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e23c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c0cd1b0 con 0x7f9e1c0d03a0 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e23c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c0b2840 con 0x7f9e1c15f450 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e21200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0b1b70 0x7f9e1c0c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e21200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0b1b70 0x7f9e1c0c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57596/0 (socket says 172.21.15.60:57596) 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e21200640 1 -- 172.21.15.60:0/953414831 learned_addr learned my addr 172.21.15.60:0/953414831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.054+0000 7f9e13e00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c0d03a0 0x7f9e1c1f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e21c00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e1c15f450 0x7f9e1c0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e21200640 1 -- 172.21.15.60:0/953414831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e1c15f450 msgr2=0x7f9e1c0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e21200640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e1c15f450 0x7f9e1c0c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e21200640 1 -- 172.21.15.60:0/953414831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c0d03a0 msgr2=0x7f9e1c1f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e21200640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c0d03a0 0x7f9e1c1f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e21200640 1 -- 172.21.15.60:0/953414831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c0c5990 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e21200640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0b1b70 0x7f9e1c0c6750 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00c370 tx=0x7f9e0c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e19600640 1 -- 172.21.15.60:0/953414831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c017070 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e19600640 1 -- 172.21.15.60:0/953414831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e0c0035f0 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.055+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c0c5c70 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.056+0000 7f9e19600640 1 -- 172.21.15.60:0/953414831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c003a40 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.056+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c0c6230 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.057+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c0b3520 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.058+0000 7f9e19600640 1 -- 172.21.15.60:0/953414831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f9e0c01a070 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.059+0000 7f9e19600640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9df8078f10 0x7f9df807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.059+0000 7f9e19600640 1 -- 172.21.15.60:0/953414831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4885+0+0 (secure 0 0 0) 0x7f9e0c09a360 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.059+0000 7f9e13e00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9df8078f10 0x7f9df807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.060+0000 7f9e13e00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9df8078f10 0x7f9df807b3d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9e04033000 tx=0x7f9e04004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.063+0000 7f9e19600640 1 -- 172.21.15.60:0/953414831 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9e0c061d10 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.332+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9e1c0b3b40 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.333+0000 7f9e19600640 1 -- 172.21.15.60:0/953414831 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f9e0c066740 con 0x7f9e1c0b1b70 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:45.334 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.337 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:45.339 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:46:45.340 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.341 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:46:45.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.345 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:46:45.346 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:46:45.348 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:46:45.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:46:45.349 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:46:45.349 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:46:45.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.349 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:46:45.349 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:46:45.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9df8078f10 msgr2=0x7f9df807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9df8078f10 0x7f9df807b3d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9e04033000 tx=0x7f9e04004730 comp rx=0 tx=0).stop 2024-09-16T15:46:45.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0b1b70 msgr2=0x7f9e1c0c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0b1b70 0x7f9e1c0c6750 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00c370 tx=0x7f9e0c00c830 comp rx=0 tx=0).stop 2024-09-16T15:46:45.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 shutdown_connections 2024-09-16T15:46:45.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9df8078f10 0x7f9df807b3d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e1c15f450 0x7f9e1c0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e1c0d03a0 0x7f9e1c1f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 --2- 172.21.15.60:0/953414831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e1c0b1b70 0x7f9e1c0c6750 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.335+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 >> 172.21.15.60:0/953414831 conn(0x7f9e1c0d1dc0 msgr2=0x7f9e1c0aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.336+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 shutdown_connections 2024-09-16T15:46:45.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.336+0000 7f9e23c00640 1 -- 172.21.15.60:0/953414831 wait complete. 2024-09-16T15:46:45.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:46:45.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.513+0000 7f6fe7200640 1 -- 172.21.15.60:0/3553664024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe80b2780 msgr2=0x7f6fe8168c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.513+0000 7f6fe7200640 1 --2- 172.21.15.60:0/3553664024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe80b2780 0x7f6fe8168c30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80d4ea0 tx=0x7f6fe002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.513+0000 7f6fe7200640 1 -- 172.21.15.60:0/3553664024 shutdown_connections 2024-09-16T15:46:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.513+0000 7f6fe7200640 1 --2- 172.21.15.60:0/3553664024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe80b2780 0x7f6fe8168c30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.513+0000 7f6fe7200640 1 -- 172.21.15.60:0/3553664024 >> 172.21.15.60:0/3553664024 conn(0x7f6fe80d3df0 msgr2=0x7f6fe80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.513+0000 7f6fe7200640 1 -- 172.21.15.60:0/3553664024 shutdown_connections 2024-09-16T15:46:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.513+0000 7f6fe7200640 1 -- 172.21.15.60:0/3553664024 wait complete. 2024-09-16T15:46:45.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.515+0000 7f6fe7200640 1 Processor -- start 2024-09-16T15:46:45.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.515+0000 7f6fe7200640 1 -- start start 2024-09-16T15:46:45.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe80b2780 0x7f6fe815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe7200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fe80b5370 0x7f6fe80b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8161440 0x7f6fe80b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe7200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe81692f0 con 0x7f6fe80b2780 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe8169170 con 0x7f6fe8161440 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe7200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe8169470 con 0x7f6fe80b5370 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe80b2780 0x7f6fe815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe80b2780 0x7f6fe815b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57604/0 (socket says 172.21.15.60:57604) 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe5e00640 1 -- 172.21.15.60:0/1203884326 learned_addr learned my addr 172.21.15.60:0/1203884326 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe6800640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8161440 0x7f6fe80b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fdd400640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fe80b5370 0x7f6fe80b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe5e00640 1 -- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fe80b5370 msgr2=0x7f6fe80b8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.516+0000 7f6fe5e00640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fe80b5370 0x7f6fe80b8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fe5e00640 1 -- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8161440 msgr2=0x7f6fe80b8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fe5e00640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8161440 0x7f6fe80b8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fe5e00640 1 -- 172.21.15.60:0/1203884326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe80d11d0 con 0x7f6fe80b2780 2024-09-16T15:46:45.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fdd400640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fe80b5370 0x7f6fe80b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fe6800640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8161440 0x7f6fe80b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fe5e00640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe80b2780 0x7f6fe815b730 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8001f80 tx=0x7f6fd800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fdea00640 1 -- 172.21.15.60:0/1203884326 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8012420 con 0x7f6fe80b2780 2024-09-16T15:46:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fdea00640 1 -- 172.21.15.60:0/1203884326 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fd8017070 con 0x7f6fe80b2780 2024-09-16T15:46:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe813b8c0 con 0x7f6fe80b2780 2024-09-16T15:46:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fdea00640 1 -- 172.21.15.60:0/1203884326 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8012990 con 0x7f6fe80b2780 2024-09-16T15:46:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.517+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe813c160 con 0x7f6fe80b2780 2024-09-16T15:46:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.518+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe80b34b0 con 0x7f6fe80b2780 2024-09-16T15:46:45.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.519+0000 7f6fdea00640 1 -- 172.21.15.60:0/1203884326 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f6fd8012b30 con 0x7f6fe80b2780 2024-09-16T15:46:45.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.520+0000 7f6fdea00640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6fc4078f10 0x7f6fc407b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.520+0000 7f6fdea00640 1 -- 172.21.15.60:0/1203884326 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4885+0+0 (secure 0 0 0) 0x7f6fd809b790 con 0x7f6fe80b2780 2024-09-16T15:46:45.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.521+0000 7f6fdd400640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6fc4078f10 0x7f6fc407b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.521+0000 7f6fdd400640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6fc4078f10 0x7f6fc407b3d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0007fd0 tx=0x7f6fd0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.525+0000 7f6fdea00640 1 -- 172.21.15.60:0/1203884326 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6fd80637e0 con 0x7f6fe80b2780 2024-09-16T15:46:45.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.723+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6fe8168f50 con 0x7f6fe80b2780 2024-09-16T15:46:45.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.723+0000 7f6fdea00640 1 -- 172.21.15.60:0/1203884326 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v145) ==== 84+0+65 (secure 0 0 0) 0x7f6fd80630e0 con 0x7f6fe80b2780 2024-09-16T15:46:45.725 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:45.725 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:46:45.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:45.725 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:46:45.725 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:46:45.725 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:45.725 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:46:45.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6fc4078f10 msgr2=0x7f6fc407b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6fc4078f10 0x7f6fc407b3d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0007fd0 tx=0x7f6fd0009040 comp rx=0 tx=0).stop 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe80b2780 msgr2=0x7f6fe815b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe80b2780 0x7f6fe815b730 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8001f80 tx=0x7f6fd800a430 comp rx=0 tx=0).stop 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 shutdown_connections 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6fc4078f10 0x7f6fc407b3d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fe8161440 0x7f6fe80b8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fe80b5370 0x7f6fe80b8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 --2- 172.21.15.60:0/1203884326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe80b2780 0x7f6fe815b730 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 >> 172.21.15.60:0/1203884326 conn(0x7f6fe80d3df0 msgr2=0x7f6fe815e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 shutdown_connections 2024-09-16T15:46:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.726+0000 7f6fe7200640 1 -- 172.21.15.60:0/1203884326 wait complete. 2024-09-16T15:46:45.743 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:46:45.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:46:45.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.906+0000 7f5fbf200640 1 -- 172.21.15.60:0/1173670046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc0070550 msgr2=0x7f5fc00cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.906+0000 7f5fbf200640 1 --2- 172.21.15.60:0/1173670046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc0070550 0x7f5fc00cb1c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8005fd0 tx=0x7f5fb802aa00 comp rx=0 tx=0).stop 2024-09-16T15:46:45.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.906+0000 7f5fbf200640 1 -- 172.21.15.60:0/1173670046 shutdown_connections 2024-09-16T15:46:45.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.906+0000 7f5fbf200640 1 --2- 172.21.15.60:0/1173670046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc0070550 0x7f5fc00cb1c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.906+0000 7f5fbf200640 1 --2- 172.21.15.60:0/1173670046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc015bda0 0x7f5fc0073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.906+0000 7f5fbf200640 1 -- 172.21.15.60:0/1173670046 >> 172.21.15.60:0/1173670046 conn(0x7f5fc00cfa70 msgr2=0x7f5fc00cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.906+0000 7f5fbf200640 1 -- 172.21.15.60:0/1173670046 shutdown_connections 2024-09-16T15:46:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.906+0000 7f5fbf200640 1 -- 172.21.15.60:0/1173670046 wait complete. 2024-09-16T15:46:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.908+0000 7f5fbf200640 1 Processor -- start 2024-09-16T15:46:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.908+0000 7f5fbf200640 1 -- start start 2024-09-16T15:46:45.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbf200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc006e590 0x7f5fc0156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbf200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0070550 0x7f5fc01550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbf200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc015bda0 0x7f5fc01f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbf200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc00cba90 con 0x7f5fc006e590 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbf200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc00cb910 con 0x7f5fc0070550 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbf200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc00cbc10 con 0x7f5fc015bda0 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbde00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc006e590 0x7f5fc0156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbde00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc006e590 0x7f5fc0156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57616/0 (socket says 172.21.15.60:57616) 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbde00640 1 -- 172.21.15.60:0/3607804661 learned_addr learned my addr 172.21.15.60:0/3607804661 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbde00640 1 -- 172.21.15.60:0/3607804661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc015bda0 msgr2=0x7f5fc01f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbde00640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc015bda0 0x7f5fc01f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbde00640 1 -- 172.21.15.60:0/3607804661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0070550 msgr2=0x7f5fc01550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbde00640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0070550 0x7f5fc01550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.909+0000 7f5fbde00640 1 -- 172.21.15.60:0/3607804661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc0156fc0 con 0x7f5fc006e590 2024-09-16T15:46:45.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.910+0000 7f5fbde00640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc006e590 0x7f5fc0156a80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000d6d0 tx=0x7f5fb000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.910+0000 7f5fb6000640 1 -- 172.21.15.60:0/3607804661 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0017070 con 0x7f5fc006e590 2024-09-16T15:46:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.910+0000 7f5fb6000640 1 -- 172.21.15.60:0/3607804661 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fb0005330 con 0x7f5fc006e590 2024-09-16T15:46:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.910+0000 7f5fb6000640 1 -- 172.21.15.60:0/3607804661 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0009820 con 0x7f5fc006e590 2024-09-16T15:46:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.910+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc01f2fc0 con 0x7f5fc006e590 2024-09-16T15:46:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.910+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc0156280 con 0x7f5fc006e590 2024-09-16T15:46:45.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.911+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc01f34e0 con 0x7f5fc006e590 2024-09-16T15:46:45.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.912+0000 7f5fb6000640 1 -- 172.21.15.60:0/3607804661 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f5fb0009bd0 con 0x7f5fc006e590 2024-09-16T15:46:45.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.913+0000 7f5fb6000640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fa0078f10 0x7f5fa007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.913+0000 7f5fb6000640 1 -- 172.21.15.60:0/3607804661 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4885+0+0 (secure 0 0 0) 0x7f5fb009b610 con 0x7f5fc006e590 2024-09-16T15:46:45.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.913+0000 7f5fbd400640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fa0078f10 0x7f5fa007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.914+0000 7f5fbd400640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fa0078f10 0x7f5fa007b3d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00c9b40 tx=0x7f5fac0079b0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:45.918+0000 7f5fb6000640 1 -- 172.21.15.60:0/3607804661 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5fb00634d0 con 0x7f5fc006e590 2024-09-16T15:46:46.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.116+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5fc015a630 con 0x7f5fc006e590 2024-09-16T15:46:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.860+0000 7f5fb6000640 1 -- 172.21.15.60:0/3607804661 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v146) ==== 138+0+0 (secure 0 0 0) 0x7f5fb0062dd0 con 0x7f5fc006e590 2024-09-16T15:46:46.862 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:46:46.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.864+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fa0078f10 msgr2=0x7f5fa007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:46.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.864+0000 7f5fbf200640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fa0078f10 0x7f5fa007b3d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00c9b40 tx=0x7f5fac0079b0 comp rx=0 tx=0).stop 2024-09-16T15:46:46.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.864+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc006e590 msgr2=0x7f5fc0156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:46.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.864+0000 7f5fbf200640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc006e590 0x7f5fc0156a80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000d6d0 tx=0x7f5fb000db90 comp rx=0 tx=0).stop 2024-09-16T15:46:46.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.865+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 shutdown_connections 2024-09-16T15:46:46.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.865+0000 7f5fbf200640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fa0078f10 0x7f5fa007b3d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.865+0000 7f5fbf200640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fc015bda0 0x7f5fc01f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.865+0000 7f5fbf200640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc0070550 0x7f5fc01550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.865+0000 7f5fbf200640 1 --2- 172.21.15.60:0/3607804661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc006e590 0x7f5fc0156a80 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.865+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 >> 172.21.15.60:0/3607804661 conn(0x7f5fc00cfa70 msgr2=0x7f5fc00b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.866+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 shutdown_connections 2024-09-16T15:46:46.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:46.866+0000 7f5fbf200640 1 -- 172.21.15.60:0/3607804661 wait complete. 2024-09-16T15:46:46.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:46:47.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.060+0000 7f4381000640 1 -- 172.21.15.60:0/468210028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 msgr2=0x7f43600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.060+0000 7f4381000640 1 --2- 172.21.15.60:0/468210028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 0x7f43600051c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f437400c640 tx=0x7f4360005830 comp rx=0 tx=0).stop 2024-09-16T15:46:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.060+0000 7f437a800640 1 -- 172.21.15.60:0/468210028 reap_dead start 2024-09-16T15:46:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.060+0000 7f4381000640 1 -- 172.21.15.60:0/468210028 shutdown_connections 2024-09-16T15:46:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.060+0000 7f4381000640 1 -- 172.21.15.60:0/468210028 >> 172.21.15.60:0/468210028 conn(0x7f437c0b1840 msgr2=0x7f437c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.060+0000 7f4381000640 1 -- 172.21.15.60:0/468210028 shutdown_connections 2024-09-16T15:46:47.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.060+0000 7f4381000640 1 -- 172.21.15.60:0/468210028 wait complete. 2024-09-16T15:46:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.061+0000 7f4381000640 1 Processor -- start 2024-09-16T15:46:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.062+0000 7f4381000640 1 -- start start 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4381000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 0x7f437c1413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4381000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437c13ba70 0x7f437c142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4381000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c1430d0 0x7f437c0ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4381000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0ced40 con 0x7f4360004d40 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4381000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0cebc0 con 0x7f437c13ba70 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4381000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f437c0ceec0 con 0x7f437c1430d0 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 0x7f437c1413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 0x7f437c1413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57636/0 (socket says 172.21.15.60:57636) 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379e00640 1 -- 172.21.15.60:0/3854882542 learned_addr learned my addr 172.21.15.60:0/3854882542 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f437a800640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c1430d0 0x7f437c0ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379400640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437c13ba70 0x7f437c142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379e00640 1 -- 172.21.15.60:0/3854882542 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c1430d0 msgr2=0x7f437c0ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379e00640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c1430d0 0x7f437c0ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379e00640 1 -- 172.21.15.60:0/3854882542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437c13ba70 msgr2=0x7f437c142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379e00640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437c13ba70 0x7f437c142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.063+0000 7f4379e00640 1 -- 172.21.15.60:0/3854882542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c141900 con 0x7f4360004d40 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.064+0000 7f4379e00640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 0x7f437c1413c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f436c011ae0 tx=0x7f436c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.064+0000 7f4372000640 1 -- 172.21.15.60:0/3854882542 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00fcb0 con 0x7f4360004d40 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.064+0000 7f4372000640 1 -- 172.21.15.60:0/3854882542 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f436c00fe50 con 0x7f4360004d40 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.064+0000 7f4372000640 1 -- 172.21.15.60:0/3854882542 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c004280 con 0x7f4360004d40 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.064+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c140320 con 0x7f4360004d40 2024-09-16T15:46:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.064+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c1408e0 con 0x7f4360004d40 2024-09-16T15:46:47.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.065+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c073440 con 0x7f4360004d40 2024-09-16T15:46:47.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.066+0000 7f4372000640 1 -- 172.21.15.60:0/3854882542 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f436c0035f0 con 0x7f4360004d40 2024-09-16T15:46:47.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.067+0000 7f4372000640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f435c078f10 0x7f435c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.067+0000 7f4372000640 1 -- 172.21.15.60:0/3854882542 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5255+0+0 (secure 0 0 0) 0x7f436c09c060 con 0x7f4360004d40 2024-09-16T15:46:47.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.068+0000 7f4379400640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f435c078f10 0x7f435c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.068+0000 7f4379400640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f435c078f10 0x7f435c07b3d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f437c141be0 tx=0x7f4368005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:47.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.072+0000 7f4372000640 1 -- 172.21.15.60:0/3854882542 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f436c063d20 con 0x7f4360004d40 2024-09-16T15:46:47.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.269+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f437c0cf5e0 con 0x7f4360004d40 2024-09-16T15:46:47.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.869+0000 7f4372000640 1 -- 172.21.15.60:0/3854882542 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v147) ==== 170+0+0 (secure 0 0 0) 0x7f436c063620 con 0x7f4360004d40 2024-09-16T15:46:47.870 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f435c078f10 msgr2=0x7f435c07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f435c078f10 0x7f435c07b3d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f437c141be0 tx=0x7f4368005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 msgr2=0x7f437c1413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 0x7f437c1413c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f436c011ae0 tx=0x7f436c009f10 comp rx=0 tx=0).stop 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 shutdown_connections 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f435c078f10 0x7f435c07b3d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c1430d0 0x7f437c0ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f437c13ba70 0x7f437c142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 --2- 172.21.15.60:0/3854882542 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4360004d40 0x7f437c1413c0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.873+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 >> 172.21.15.60:0/3854882542 conn(0x7f437c0b1840 msgr2=0x7f437c143500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.874+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 shutdown_connections 2024-09-16T15:46:47.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:47.875+0000 7f4381000640 1 -- 172.21.15.60:0/3854882542 wait complete. 2024-09-16T15:46:47.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:46:48.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.073+0000 7f0157200640 1 -- 172.21.15.60:0/3257678082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01580cf830 msgr2=0x7f01580b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.073+0000 7f0157200640 1 --2- 172.21.15.60:0/3257678082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01580cf830 0x7f01580b33b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0144009890 tx=0x7f014402a530 comp rx=0 tx=0).stop 2024-09-16T15:46:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.073+0000 7f0157200640 1 -- 172.21.15.60:0/3257678082 shutdown_connections 2024-09-16T15:46:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.073+0000 7f0157200640 1 --2- 172.21.15.60:0/3257678082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01580cf830 0x7f01580b33b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.073+0000 7f0157200640 1 -- 172.21.15.60:0/3257678082 >> 172.21.15.60:0/3257678082 conn(0x7f015809d0b0 msgr2=0x7f01580d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.073+0000 7f0157200640 1 -- 172.21.15.60:0/3257678082 shutdown_connections 2024-09-16T15:46:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.073+0000 7f0157200640 1 -- 172.21.15.60:0/3257678082 wait complete. 2024-09-16T15:46:48.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.075+0000 7f0157200640 1 Processor -- start 2024-09-16T15:46:48.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.075+0000 7f0157200640 1 -- start start 2024-09-16T15:46:48.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0157200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580cf830 0x7f015815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0157200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01580d2a00 0x7f015815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0157200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158161460 0x7f015815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0157200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f01580b4000 con 0x7f01580cf830 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0157200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f01580b5150 con 0x7f0158161460 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0157200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f01580b4180 con 0x7f01580d2a00 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0155e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580cf830 0x7f015815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0155e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580cf830 0x7f015815a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57650/0 (socket says 172.21.15.60:57650) 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0155e00640 1 -- 172.21.15.60:0/2716297222 learned_addr learned my addr 172.21.15.60:0/2716297222 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0156800640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158161460 0x7f015815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.076+0000 7f0155400640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01580d2a00 0x7f015815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0155e00640 1 -- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01580d2a00 msgr2=0x7f015815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0155e00640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01580d2a00 0x7f015815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0155e00640 1 -- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158161460 msgr2=0x7f015815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0155e00640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158161460 0x7f015815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0155e00640 1 -- 172.21.15.60:0/2716297222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015815a8a0 con 0x7f01580cf830 2024-09-16T15:46:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0156800640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158161460 0x7f015815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0155e00640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580cf830 0x7f015815a360 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f014800c430 tx=0x7f014800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f014e000640 1 -- 172.21.15.60:0/2716297222 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148017070 con 0x7f01580cf830 2024-09-16T15:46:48.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f014e000640 1 -- 172.21.15.60:0/2716297222 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01480053e0 con 0x7f01580cf830 2024-09-16T15:46:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f014e000640 1 -- 172.21.15.60:0/2716297222 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148003850 con 0x7f01580cf830 2024-09-16T15:46:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01581f59a0 con 0x7f01580cf830 2024-09-16T15:46:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.077+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015815b730 con 0x7f01580cf830 2024-09-16T15:46:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.078+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01580d3780 con 0x7f01580cf830 2024-09-16T15:46:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.080+0000 7f014e000640 1 -- 172.21.15.60:0/2716297222 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f014801a070 con 0x7f01580cf830 2024-09-16T15:46:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.081+0000 7f014e000640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0138078f40 0x7f013807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.081+0000 7f014e000640 1 -- 172.21.15.60:0/2716297222 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5637+0+0 (secure 0 0 0) 0x7f014809ada0 con 0x7f01580cf830 2024-09-16T15:46:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.081+0000 7f0155400640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0138078f40 0x7f013807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.082+0000 7f0155400640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0138078f40 0x7f013807b400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0144034000 tx=0x7f0144004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:48.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.085+0000 7f014e000640 1 -- 172.21.15.60:0/2716297222 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f01580d3780 con 0x7f01580cf830 2024-09-16T15:46:48.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.319+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f01580d22f0 con 0x7f01580cf830 2024-09-16T15:46:48.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.882+0000 7f014e000640 1 -- 172.21.15.60:0/2716297222 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v98) ==== 174+0+0 (secure 0 0 0) 0x7f0148021020 con 0x7f01580cf830 2024-09-16T15:46:48.888 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 22 and data pool 23 2024-09-16T15:46:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.889+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0138078f40 msgr2=0x7f013807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.889+0000 7f0157200640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0138078f40 0x7f013807b400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0144034000 tx=0x7f0144004730 comp rx=0 tx=0).stop 2024-09-16T15:46:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.889+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580cf830 msgr2=0x7f015815a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.889+0000 7f0157200640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580cf830 0x7f015815a360 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f014800c430 tx=0x7f014800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.890+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 shutdown_connections 2024-09-16T15:46:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.890+0000 7f0157200640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0138078f40 0x7f013807b400 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.890+0000 7f0157200640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0158161460 0x7f015815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.890+0000 7f0157200640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01580d2a00 0x7f015815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.890+0000 7f0157200640 1 --2- 172.21.15.60:0/2716297222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01580cf830 0x7f015815a360 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.890+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 >> 172.21.15.60:0/2716297222 conn(0x7f015809d0b0 msgr2=0x7f01580cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:48.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.892+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 shutdown_connections 2024-09-16T15:46:48.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:48.892+0000 7f0157200640 1 -- 172.21.15.60:0/2716297222 wait complete. 2024-09-16T15:46:48.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.083+0000 7f8d0cc00640 1 -- 172.21.15.60:0/4200353550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 msgr2=0x7f8d080b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.083+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/4200353550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 0x7f8d080b1770 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8d080ce3d0 tx=0x7f8d0002aa30 comp rx=0 tx=0).stop 2024-09-16T15:46:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.083+0000 7f8d0cc00640 1 -- 172.21.15.60:0/4200353550 shutdown_connections 2024-09-16T15:46:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.083+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/4200353550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 0x7f8d080b1770 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.083+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/4200353550 >> v2:172.21.15.100:3300/0 conn(0x7f8d080b5370 0x7f8d080b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.083+0000 7f8d0cc00640 1 -- 172.21.15.60:0/4200353550 >> 172.21.15.60:0/4200353550 conn(0x7f8d080d3df0 msgr2=0x7f8d080d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.083+0000 7f8d0cc00640 1 -- 172.21.15.60:0/4200353550 shutdown_connections 2024-09-16T15:46:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.083+0000 7f8d0cc00640 1 -- 172.21.15.60:0/4200353550 wait complete. 2024-09-16T15:46:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.084+0000 7f8d0cc00640 1 Processor -- start 2024-09-16T15:46:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.084+0000 7f8d0cc00640 1 -- start start 2024-09-16T15:46:49.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.085+0000 7f8d0cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 0x7f8d080cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.085+0000 7f8d0cc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d080b5370 0x7f8d081575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.085+0000 7f8d0cc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d08161440 0x7f8d080b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.085+0000 7f8d0cc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8d080b2040 con 0x7f8d080b4100 2024-09-16T15:46:49.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.085+0000 7f8d0cc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d080b1ec0 con 0x7f8d080b5370 2024-09-16T15:46:49.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.085+0000 7f8d0cc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8d080b21c0 con 0x7f8d08161440 2024-09-16T15:46:49.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 0x7f8d080cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 0x7f8d080cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56496/0 (socket says 172.21.15.60:56496) 2024-09-16T15:46:49.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 -- 172.21.15.60:0/3143104319 learned_addr learned my addr 172.21.15.60:0/3143104319 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:49.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 -- 172.21.15.60:0/3143104319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d08161440 msgr2=0x7f8d080b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d08161440 0x7f8d080b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 -- 172.21.15.60:0/3143104319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d080b5370 msgr2=0x7f8d081575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d080b5370 0x7f8d081575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 -- 172.21.15.60:0/3143104319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d081f8480 con 0x7f8d080b4100 2024-09-16T15:46:49.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.086+0000 7f8d07400640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 0x7f8d080cd6f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c3a0 tx=0x7f8cf800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.087+0000 7f8cff400640 1 -- 172.21.15.60:0/3143104319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8017070 con 0x7f8d080b4100 2024-09-16T15:46:49.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.087+0000 7f8cff400640 1 -- 172.21.15.60:0/3143104319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cf80053c0 con 0x7f8d080b4100 2024-09-16T15:46:49.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.087+0000 7f8cff400640 1 -- 172.21.15.60:0/3143104319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8003850 con 0x7f8d080b4100 2024-09-16T15:46:49.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.087+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d081f8760 con 0x7f8d080b4100 2024-09-16T15:46:49.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.087+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d081f8d20 con 0x7f8d080b4100 2024-09-16T15:46:49.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.088+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d080b1770 con 0x7f8d080b4100 2024-09-16T15:46:49.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.089+0000 7f8cff400640 1 -- 172.21.15.60:0/3143104319 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f8cf801a070 con 0x7f8d080b4100 2024-09-16T15:46:49.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.090+0000 7f8cff400640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8ce8078f10 0x7f8ce807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.090+0000 7f8cff400640 1 -- 172.21.15.60:0/3143104319 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5737+0+0 (secure 0 0 0) 0x7f8cf809aa90 con 0x7f8d080b4100 2024-09-16T15:46:49.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.090+0000 7f8d06a00640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8ce8078f10 0x7f8ce807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.091+0000 7f8d06a00640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8ce8078f10 0x7f8ce807b3d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40058c0 tx=0x7f8cf4005850 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.095+0000 7f8cff400640 1 -- 172.21.15.60:0/3143104319 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8cf8062570 con 0x7f8d080b4100 2024-09-16T15:46:49.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.290+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d08158840 con 0x7f8d080b4100 2024-09-16T15:46:49.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.291+0000 7f8cff400640 1 -- 172.21.15.60:0/3143104319 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+16788 (secure 0 0 0) 0x7f8cf8061e70 con 0x7f8d080b4100 2024-09-16T15:46:49.293 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:49.293 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":148,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:48.320877+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:46.126734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:47.279485+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:49.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8ce8078f10 msgr2=0x7f8ce807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8ce8078f10 0x7f8ce807b3d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40058c0 tx=0x7f8cf4005850 comp rx=0 tx=0).stop 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 msgr2=0x7f8d080cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 0x7f8d080cd6f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c3a0 tx=0x7f8cf800c860 comp rx=0 tx=0).stop 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 shutdown_connections 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8ce8078f10 0x7f8ce807b3d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d08161440 0x7f8d080b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d080b5370 0x7f8d081575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 --2- 172.21.15.60:0/3143104319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d080b4100 0x7f8d080cd6f0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 >> 172.21.15.60:0/3143104319 conn(0x7f8d080d3df0 msgr2=0x7f8d080d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 shutdown_connections 2024-09-16T15:46:49.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.294+0000 7f8d0cc00640 1 -- 172.21.15.60:0/3143104319 wait complete. 2024-09-16T15:46:49.311 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.470+0000 7f5b0d400640 1 -- 172.21.15.60:0/1668943090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 msgr2=0x7f5b08074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.470+0000 7f5b0d400640 1 --2- 172.21.15.60:0/1668943090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 0x7f5b08074420 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5b080cfbb0 tx=0x7f5b0002a820 comp rx=0 tx=0).stop 2024-09-16T15:46:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.470+0000 7f5b0d400640 1 -- 172.21.15.60:0/1668943090 shutdown_connections 2024-09-16T15:46:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.471+0000 7f5b0d400640 1 --2- 172.21.15.60:0/1668943090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 0x7f5b08074420 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.471+0000 7f5b0d400640 1 --2- 172.21.15.60:0/1668943090 >> v2:172.21.15.100:3300/0 conn(0x7f5b080ccf00 0x7f5b080b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.471+0000 7f5b0d400640 1 -- 172.21.15.60:0/1668943090 >> 172.21.15.60:0/1668943090 conn(0x7f5b080b1950 msgr2=0x7f5b080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.471+0000 7f5b0d400640 1 -- 172.21.15.60:0/1668943090 shutdown_connections 2024-09-16T15:46:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.471+0000 7f5b0d400640 1 -- 172.21.15.60:0/1668943090 wait complete. 2024-09-16T15:46:49.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.472+0000 7f5b0d400640 1 Processor -- start 2024-09-16T15:46:49.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.472+0000 7f5b0d400640 1 -- start start 2024-09-16T15:46:49.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.473+0000 7f5b0d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 0x7f5b081f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.473+0000 7f5b0d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b080ccf00 0x7f5b08158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.473+0000 7f5b0d400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b0815f5a0 0x7f5b0815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.473+0000 7f5b0d400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0806f2e0 con 0x7f5b08073fb0 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.473+0000 7f5b0d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0806f160 con 0x7f5b080ccf00 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.473+0000 7f5b0d400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0806f460 con 0x7f5b0815f5a0 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 0x7f5b081f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 0x7f5b081f6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56520/0 (socket says 172.21.15.60:56520) 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 -- 172.21.15.60:0/4247311148 learned_addr learned my addr 172.21.15.60:0/4247311148 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 -- 172.21.15.60:0/4247311148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b0815f5a0 msgr2=0x7f5b0815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b0815f5a0 0x7f5b0815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 -- 172.21.15.60:0/4247311148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b080ccf00 msgr2=0x7f5b08158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b080ccf00 0x7f5b08158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 -- 172.21.15.60:0/4247311148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b08159240 con 0x7f5b08073fb0 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5b06800640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 0x7f5b081f6620 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5af800c430 tx=0x7f5af800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.474+0000 7f5aff400640 1 -- 172.21.15.60:0/4247311148 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8011790 con 0x7f5b08073fb0 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.475+0000 7f5aff400640 1 -- 172.21.15.60:0/4247311148 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5af8017070 con 0x7f5b08073fb0 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.475+0000 7f5aff400640 1 -- 172.21.15.60:0/4247311148 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8011ce0 con 0x7f5b08073fb0 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.475+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0815c920 con 0x7f5b08073fb0 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.475+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b081f28f0 con 0x7f5b08073fb0 2024-09-16T15:46:49.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.476+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b08155ce0 con 0x7f5b08073fb0 2024-09-16T15:46:49.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.477+0000 7f5aff400640 1 -- 172.21.15.60:0/4247311148 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f5af801c460 con 0x7f5b08073fb0 2024-09-16T15:46:49.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.478+0000 7f5aff400640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ae4078f10 0x7f5ae407b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.478+0000 7f5aff400640 1 -- 172.21.15.60:0/4247311148 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5737+0+0 (secure 0 0 0) 0x7f5af809b300 con 0x7f5b08073fb0 2024-09-16T15:46:49.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.478+0000 7f5afde00640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ae4078f10 0x7f5ae407b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.479+0000 7f5afde00640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ae4078f10 0x7f5ae407b3d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005fd0 tx=0x7f5af0005f40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.482+0000 7f5aff400640 1 -- 172.21.15.60:0/4247311148 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5b08159240 con 0x7f5b08073fb0 2024-09-16T15:46:49.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.681+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b0806f9f0 con 0x7f5b08073fb0 2024-09-16T15:46:49.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.682+0000 7f5aff400640 1 -- 172.21.15.60:0/4247311148 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+16788 (secure 0 0 0) 0x7f5af8063000 con 0x7f5b08073fb0 2024-09-16T15:46:49.684 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:49.684 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":148,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:48.320877+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:46.126734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:47.279485+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:49.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ae4078f10 msgr2=0x7f5ae407b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ae4078f10 0x7f5ae407b3d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005fd0 tx=0x7f5af0005f40 comp rx=0 tx=0).stop 2024-09-16T15:46:49.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 msgr2=0x7f5b081f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 0x7f5b081f6620 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f5af800c430 tx=0x7f5af800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 shutdown_connections 2024-09-16T15:46:49.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ae4078f10 0x7f5ae407b3d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5b0815f5a0 0x7f5b0815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5b080ccf00 0x7f5b08158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 --2- 172.21.15.60:0/4247311148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5b08073fb0 0x7f5b081f6620 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 >> 172.21.15.60:0/4247311148 conn(0x7f5b080b1950 msgr2=0x7f5b0815cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 shutdown_connections 2024-09-16T15:46:49.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.685+0000 7f5b0d400640 1 -- 172.21.15.60:0/4247311148 wait complete. 2024-09-16T15:46:49.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:46:49.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.866+0000 7ff068400640 1 -- 172.21.15.60:0/44705204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff060145750 msgr2=0x7ff060073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.866+0000 7ff068400640 1 --2- 172.21.15.60:0/44705204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff060145750 0x7ff060073440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff054005fd0 tx=0x7ff05402a540 comp rx=0 tx=0).stop 2024-09-16T15:46:49.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.866+0000 7ff068400640 1 -- 172.21.15.60:0/44705204 shutdown_connections 2024-09-16T15:46:49.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.866+0000 7ff068400640 1 --2- 172.21.15.60:0/44705204 >> v2:172.21.15.100:3300/0 conn(0x7ff06006efa0 0x7ff0600cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.866+0000 7ff068400640 1 --2- 172.21.15.60:0/44705204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff060145750 0x7ff060073440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.866+0000 7ff068400640 1 -- 172.21.15.60:0/44705204 >> 172.21.15.60:0/44705204 conn(0x7ff0600b1840 msgr2=0x7ff0600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.867+0000 7ff068400640 1 -- 172.21.15.60:0/44705204 shutdown_connections 2024-09-16T15:46:49.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.867+0000 7ff068400640 1 -- 172.21.15.60:0/44705204 wait complete. 2024-09-16T15:46:49.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.868+0000 7ff068400640 1 Processor -- start 2024-09-16T15:46:49.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.868+0000 7ff068400640 1 -- start start 2024-09-16T15:46:49.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.869+0000 7ff068400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff06006efa0 0x7ff0600b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.869+0000 7ff068400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0600ccb00 0x7ff060143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.869+0000 7ff068400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff060145750 0x7ff060143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.869+0000 7ff068400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff0600b26b0 con 0x7ff060145750 2024-09-16T15:46:49.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.869+0000 7ff068400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff0600cedd0 con 0x7ff0600ccb00 2024-09-16T15:46:49.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.869+0000 7ff068400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff060160100 con 0x7ff06006efa0 2024-09-16T15:46:49.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.869+0000 7ff066400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff060145750 0x7ff060143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff065a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff06006efa0 0x7ff0600b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff066400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff060145750 0x7ff060143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56526/0 (socket says 172.21.15.60:56526) 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff066400640 1 -- 172.21.15.60:0/679913451 learned_addr learned my addr 172.21.15.60:0/679913451 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff065000640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0600ccb00 0x7ff060143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff066400640 1 -- 172.21.15.60:0/679913451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff06006efa0 msgr2=0x7ff0600b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff066400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff06006efa0 0x7ff0600b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff066400640 1 -- 172.21.15.60:0/679913451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0600ccb00 msgr2=0x7ff060143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff066400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0600ccb00 0x7ff060143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff066400640 1 -- 172.21.15.60:0/679913451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0600b3900 con 0x7ff060145750 2024-09-16T15:46:49.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.870+0000 7ff066400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff060145750 0x7ff060143760 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff05800c770 tx=0x7ff05800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.871+0000 7ff05d600640 1 -- 172.21.15.60:0/679913451 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058017070 con 0x7ff060145750 2024-09-16T15:46:49.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.871+0000 7ff05d600640 1 -- 172.21.15.60:0/679913451 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff058005070 con 0x7ff060145750 2024-09-16T15:46:49.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.871+0000 7ff05d600640 1 -- 172.21.15.60:0/679913451 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580054c0 con 0x7ff060145750 2024-09-16T15:46:49.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.871+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0601f7cd0 con 0x7ff060145750 2024-09-16T15:46:49.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.871+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0600b1340 con 0x7ff060145750 2024-09-16T15:46:49.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.872+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0601605b0 con 0x7ff060145750 2024-09-16T15:46:49.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.873+0000 7ff05d600640 1 -- 172.21.15.60:0/679913451 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7ff05801a070 con 0x7ff060145750 2024-09-16T15:46:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.874+0000 7ff05d600640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff040078f10 0x7ff04007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.874+0000 7ff05d600640 1 -- 172.21.15.60:0/679913451 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5737+0+0 (secure 0 0 0) 0x7ff05809acd0 con 0x7ff060145750 2024-09-16T15:46:49.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.874+0000 7ff065a00640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff040078f10 0x7ff04007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.875+0000 7ff065a00640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff040078f10 0x7ff04007b3d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff054033000 tx=0x7ff054004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:49.878+0000 7ff05d600640 1 -- 172.21.15.60:0/679913451 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff0580621a0 con 0x7ff060145750 2024-09-16T15:46:50.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.114+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff06015f3b0 con 0x7ff060145750 2024-09-16T15:46:50.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.894+0000 7ff05d600640 1 -- 172.21.15.60:0/679913451 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v101) ==== 118+0+0 (secure 0 0 0) 0x7ff058066bd0 con 0x7ff060145750 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff040078f10 msgr2=0x7ff04007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff040078f10 0x7ff04007b3d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff054033000 tx=0x7ff054004730 comp rx=0 tx=0).stop 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff060145750 msgr2=0x7ff060143760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff060145750 0x7ff060143760 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff05800c770 tx=0x7ff05800cc30 comp rx=0 tx=0).stop 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 shutdown_connections 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff040078f10 0x7ff04007b3d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff060145750 0x7ff060143760 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0600ccb00 0x7ff060143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 --2- 172.21.15.60:0/679913451 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff06006efa0 0x7ff0600b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.897+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 >> 172.21.15.60:0/679913451 conn(0x7ff0600b1840 msgr2=0x7ff0601449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:50.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.898+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 shutdown_connections 2024-09-16T15:46:50.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:50.898+0000 7ff068400640 1 -- 172.21.15.60:0/679913451 wait complete. 2024-09-16T15:46:50.917 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:46:50.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:46:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.085+0000 7fe12d600640 1 -- 172.21.15.60:0/3897921133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1280cf830 msgr2=0x7fe1280b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.085+0000 7fe12d600640 1 --2- 172.21.15.60:0/3897921133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1280cf830 0x7fe1280b33b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe114005fd0 tx=0x7fe11402a560 comp rx=0 tx=0).stop 2024-09-16T15:46:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.086+0000 7fe12d600640 1 -- 172.21.15.60:0/3897921133 shutdown_connections 2024-09-16T15:46:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.086+0000 7fe12d600640 1 --2- 172.21.15.60:0/3897921133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1280d2a00 0x7fe1280b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.086+0000 7fe12d600640 1 --2- 172.21.15.60:0/3897921133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1280cf830 0x7fe1280b33b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.086+0000 7fe12d600640 1 -- 172.21.15.60:0/3897921133 >> 172.21.15.60:0/3897921133 conn(0x7fe12809d0b0 msgr2=0x7fe1280d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:51.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.086+0000 7fe12d600640 1 -- 172.21.15.60:0/3897921133 shutdown_connections 2024-09-16T15:46:51.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.086+0000 7fe12d600640 1 -- 172.21.15.60:0/3897921133 wait complete. 2024-09-16T15:46:51.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.087+0000 7fe12d600640 1 Processor -- start 2024-09-16T15:46:51.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.087+0000 7fe12d600640 1 -- start start 2024-09-16T15:46:51.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.088+0000 7fe12d600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1280cf830 0x7fe1281f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.088+0000 7fe12d600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1280d2a00 0x7fe1281572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.088+0000 7fe12d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128161460 0x7fe1280cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.088+0000 7fe12d600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280b50c0 con 0x7fe128161460 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.088+0000 7fe12d600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe1280b4f40 con 0x7fe1280cf830 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.088+0000 7fe12d600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe1280b3e30 con 0x7fe1280d2a00 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.088+0000 7fe12cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128161460 0x7fe1280cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe12cc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128161460 0x7fe1280cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56538/0 (socket says 172.21.15.60:56538) 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe12cc00640 1 -- 172.21.15.60:0/3224680371 learned_addr learned my addr 172.21.15.60:0/3224680371 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe127400640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1280d2a00 0x7fe1281572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:51.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe127e00640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1280cf830 0x7fe1281f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe12cc00640 1 -- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1280d2a00 msgr2=0x7fe1281572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe12cc00640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1280d2a00 0x7fe1281572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe12cc00640 1 -- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1280cf830 msgr2=0x7fe1281f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe12cc00640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1280cf830 0x7fe1281f5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe12cc00640 1 -- 172.21.15.60:0/3224680371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe128159360 con 0x7fe128161460 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe127e00640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1280cf830 0x7fe1281f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.089+0000 7fe12cc00640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128161460 0x7fe1280cc6c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe12000b9e0 tx=0x7fe12000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.090+0000 7fe11fe00640 1 -- 172.21.15.60:0/3224680371 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120019070 con 0x7fe128161460 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.090+0000 7fe11fe00640 1 -- 172.21.15.60:0/3224680371 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe12000cc30 con 0x7fe128161460 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.090+0000 7fe11fe00640 1 -- 172.21.15.60:0/3224680371 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200146a0 con 0x7fe128161460 2024-09-16T15:46:51.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.090+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe128159640 con 0x7fe128161460 2024-09-16T15:46:51.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.090+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe128159c00 con 0x7fe128161460 2024-09-16T15:46:51.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.091+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1280d3780 con 0x7fe128161460 2024-09-16T15:46:51.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.092+0000 7fe11fe00640 1 -- 172.21.15.60:0/3224680371 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fe12001c070 con 0x7fe128161460 2024-09-16T15:46:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.093+0000 7fe11fe00640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe108078f10 0x7fe10807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.093+0000 7fe11fe00640 1 -- 172.21.15.60:0/3224680371 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7fe12009b230 con 0x7fe128161460 2024-09-16T15:46:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.093+0000 7fe127e00640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe108078f10 0x7fe10807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:51.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.094+0000 7fe127e00640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe108078f10 0x7fe10807b3d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe128074720 tx=0x7fe118009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:51.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.097+0000 7fe11fe00640 1 -- 172.21.15.60:0/3224680371 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe1280d3780 con 0x7fe128161460 2024-09-16T15:46:51.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.341+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe1280b42e0 con 0x7fe128161460 2024-09-16T15:46:51.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.896+0000 7fe11fe00640 1 -- 172.21.15.60:0/3224680371 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v102) ==== 122+0+0 (secure 0 0 0) 0x7fe12000ce20 con 0x7fe128161460 2024-09-16T15:46:51.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe108078f10 msgr2=0x7fe10807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe108078f10 0x7fe10807b3d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe128074720 tx=0x7fe118009040 comp rx=0 tx=0).stop 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128161460 msgr2=0x7fe1280cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128161460 0x7fe1280cc6c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe12000b9e0 tx=0x7fe12000bea0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 shutdown_connections 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe108078f10 0x7fe10807b3d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe128161460 0x7fe1280cc6c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1280d2a00 0x7fe1281572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 --2- 172.21.15.60:0/3224680371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1280cf830 0x7fe1281f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 >> 172.21.15.60:0/3224680371 conn(0x7fe12809d0b0 msgr2=0x7fe1280b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 shutdown_connections 2024-09-16T15:46:51.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:51.899+0000 7fe12d600640 1 -- 172.21.15.60:0/3224680371 wait complete. 2024-09-16T15:46:51.921 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:46:51.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:52.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.084+0000 7f4aa2a00640 1 -- 172.21.15.60:0/431320853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b1680 msgr2=0x7f4a9c0b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.084+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/431320853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b1680 0x7f4a9c0b1b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4a94005fd0 tx=0x7f4a9402a8a0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.084+0000 7f4aa2a00640 1 -- 172.21.15.60:0/431320853 shutdown_connections 2024-09-16T15:46:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.084+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/431320853 >> v2:172.21.15.100:3300/0 conn(0x7f4a9c0d0670 0x7f4a9c0d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.084+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/431320853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c0b1680 0x7f4a9c0b1b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.084+0000 7f4aa2a00640 1 -- 172.21.15.60:0/431320853 >> 172.21.15.60:0/431320853 conn(0x7f4a9c0d1c40 msgr2=0x7f4a9c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.084+0000 7f4aa2a00640 1 -- 172.21.15.60:0/431320853 shutdown_connections 2024-09-16T15:46:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.084+0000 7f4aa2a00640 1 -- 172.21.15.60:0/431320853 wait complete. 2024-09-16T15:46:52.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.086+0000 7f4aa2a00640 1 Processor -- start 2024-09-16T15:46:52.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.086+0000 7f4aa2a00640 1 -- start start 2024-09-16T15:46:52.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b1680 0x7f4a9c152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0d0670 0x7f4a9c0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c15bd30 0x7f4a9c1f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa2a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c0b21f0 con 0x7f4a9c0b1680 2024-09-16T15:46:52.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa2a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c0b2070 con 0x7f4a9c0d0670 2024-09-16T15:46:52.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa2a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c0b2370 con 0x7f4a9c15bd30 2024-09-16T15:46:52.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b1680 0x7f4a9c152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa1600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b1680 0x7f4a9c152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56550/0 (socket says 172.21.15.60:56550) 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa1600640 1 -- 172.21.15.60:0/1121044569 learned_addr learned my addr 172.21.15.60:0/1121044569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa2000640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c15bd30 0x7f4a9c1f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.087+0000 7f4aa0c00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0d0670 0x7f4a9c0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4aa1600640 1 -- 172.21.15.60:0/1121044569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c15bd30 msgr2=0x7f4a9c1f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4aa1600640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c15bd30 0x7f4a9c1f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4aa1600640 1 -- 172.21.15.60:0/1121044569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0d0670 msgr2=0x7f4a9c0c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4aa1600640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0d0670 0x7f4a9c0c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4aa1600640 1 -- 172.21.15.60:0/1121044569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c0cb5a0 con 0x7f4a9c0b1680 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4aa1600640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b1680 0x7f4a9c152050 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000c3a0 tx=0x7f4a9000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4a99600640 1 -- 172.21.15.60:0/1121044569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90011790 con 0x7f4a9c0b1680 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c0cb880 con 0x7f4a9c0b1680 2024-09-16T15:46:52.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4a99600640 1 -- 172.21.15.60:0/1121044569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a90017070 con 0x7f4a9c0b1680 2024-09-16T15:46:52.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4a99600640 1 -- 172.21.15.60:0/1121044569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90011cc0 con 0x7f4a9c0b1680 2024-09-16T15:46:52.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.088+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c0cbe40 con 0x7f4a9c0b1680 2024-09-16T15:46:52.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.089+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c0b2890 con 0x7f4a9c0b1680 2024-09-16T15:46:52.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.090+0000 7f4a99600640 1 -- 172.21.15.60:0/1121044569 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f4a9001c460 con 0x7f4a9c0b1680 2024-09-16T15:46:52.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.091+0000 7f4a99600640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4a7c078f10 0x7f4a7c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.091+0000 7f4a99600640 1 -- 172.21.15.60:0/1121044569 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f4a9009b360 con 0x7f4a9c0b1680 2024-09-16T15:46:52.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.092+0000 7f4aa0c00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4a7c078f10 0x7f4a7c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.092+0000 7f4aa0c00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4a7c078f10 0x7f4a7c07b3d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a880095f0 tx=0x7f4a88005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:52.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.095+0000 7f4a99600640 1 -- 172.21.15.60:0/1121044569 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4a90062fe0 con 0x7f4a9c0b1680 2024-09-16T15:46:52.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.331+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a9c0b0310 con 0x7f4a9c0b1680 2024-09-16T15:46:52.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.332+0000 7f4a99600640 1 -- 172.21.15.60:0/1121044569 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5229 (secure 0 0 0) 0x7f4a900628e0 con 0x7f4a9c0b1680 2024-09-16T15:46:52.333 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:52.333 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":102,"btime":"2024-09-16T15:46:51:894437+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7133,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/822830116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":822830116},{"type":"v1","addr":"172.21.15.100:6834","nonce":822830116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/467713690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":467713690},{"type":"v1","addr":"172.21.15.100:6835","nonce":467713690}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":7981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2740321673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2740321673},{"type":"v1","addr":"172.21.15.60:6834","nonce":2740321673}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:48.320845+0000","modified":"2024-09-16T15:46:51.342321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2397719730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2397719730},{"type":"v1","addr":"172.21.15.60:6835","nonce":2397719730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8472,"qdb_cluster":[8472]},"id":11}]} 2024-09-16T15:46:52.333 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 102 2024-09-16T15:46:52.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4a7c078f10 msgr2=0x7f4a7c07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4a7c078f10 0x7f4a7c07b3d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a880095f0 tx=0x7f4a88005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b1680 msgr2=0x7f4a9c152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b1680 0x7f4a9c152050 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000c3a0 tx=0x7f4a9000c860 comp rx=0 tx=0).stop 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 shutdown_connections 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4a7c078f10 0x7f4a7c07b3d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9c15bd30 0x7f4a9c1f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a9c0d0670 0x7f4a9c0c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 --2- 172.21.15.60:0/1121044569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a9c0b1680 0x7f4a9c152050 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.334+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 >> 172.21.15.60:0/1121044569 conn(0x7f4a9c0d1c40 msgr2=0x7f4a9c0ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.335+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 shutdown_connections 2024-09-16T15:46:52.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.335+0000 7f4aa2a00640 1 -- 172.21.15.60:0/1121044569 wait complete. 2024-09-16T15:46:52.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.511+0000 7ff7d5000640 1 -- 172.21.15.60:0/2248204186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d0073e00 msgr2=0x7ff7d0168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.511+0000 7ff7d5000640 1 --2- 172.21.15.60:0/2248204186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d0073e00 0x7ff7d0168e10 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00b4890 tx=0x7ff7c802aaa0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.511+0000 7ff7d5000640 1 -- 172.21.15.60:0/2248204186 shutdown_connections 2024-09-16T15:46:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.511+0000 7ff7d5000640 1 --2- 172.21.15.60:0/2248204186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d0073e00 0x7ff7d0168e10 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.511+0000 7ff7d5000640 1 -- 172.21.15.60:0/2248204186 >> 172.21.15.60:0/2248204186 conn(0x7ff7d00b1950 msgr2=0x7ff7d00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.511+0000 7ff7d5000640 1 -- 172.21.15.60:0/2248204186 shutdown_connections 2024-09-16T15:46:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.511+0000 7ff7d5000640 1 -- 172.21.15.60:0/2248204186 wait complete. 2024-09-16T15:46:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.512+0000 7ff7d5000640 1 Processor -- start 2024-09-16T15:46:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.513+0000 7ff7d5000640 1 -- start start 2024-09-16T15:46:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7d5000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0073e00 0x7ff7d0157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7d5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d00af3d0 0x7ff7d01500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7d5000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7d0161550 0x7ff7d0147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7d5000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d01694d0 con 0x7ff7d00af3d0 2024-09-16T15:46:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7d5000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d0169350 con 0x7ff7d0073e00 2024-09-16T15:46:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7d5000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d0169650 con 0x7ff7d0161550 2024-09-16T15:46:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d00af3d0 0x7ff7d01500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cd400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d00af3d0 0x7ff7d01500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56560/0 (socket says 172.21.15.60:56560) 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cd400640 1 -- 172.21.15.60:0/2034054071 learned_addr learned my addr 172.21.15.60:0/2034054071 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cde00640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0073e00 0x7ff7d0157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cd400640 1 -- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7d0161550 msgr2=0x7ff7d0147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7ce800640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7d0161550 0x7ff7d0147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cd400640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7d0161550 0x7ff7d0147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cd400640 1 -- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0073e00 msgr2=0x7ff7d0157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cd400640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0073e00 0x7ff7d0157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cd400640 1 -- 172.21.15.60:0/2034054071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d014c920 con 0x7ff7d00af3d0 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7cde00640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0073e00 0x7ff7d0157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.514+0000 7ff7ce800640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7d0161550 0x7ff7d0147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.515+0000 7ff7cd400640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d00af3d0 0x7ff7d01500d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00c500 tx=0x7ff7bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.515+0000 7ff7c6000640 1 -- 172.21.15.60:0/2034054071 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0052d0 con 0x7ff7d00af3d0 2024-09-16T15:46:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.515+0000 7ff7c6000640 1 -- 172.21.15.60:0/2034054071 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7bc005470 con 0x7ff7d00af3d0 2024-09-16T15:46:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.515+0000 7ff7c6000640 1 -- 172.21.15.60:0/2034054071 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc012870 con 0x7ff7d00af3d0 2024-09-16T15:46:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.515+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d0147dd0 con 0x7ff7d00af3d0 2024-09-16T15:46:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.515+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d014f8d0 con 0x7ff7d00af3d0 2024-09-16T15:46:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.516+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d006ef50 con 0x7ff7d00af3d0 2024-09-16T15:46:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.517+0000 7ff7c6000640 1 -- 172.21.15.60:0/2034054071 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7ff7bc0035f0 con 0x7ff7d00af3d0 2024-09-16T15:46:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.518+0000 7ff7c6000640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7b0078f10 0x7ff7b007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.518+0000 7ff7c6000640 1 -- 172.21.15.60:0/2034054071 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7ff7bc09b190 con 0x7ff7d00af3d0 2024-09-16T15:46:52.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.518+0000 7ff7cde00640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7b0078f10 0x7ff7b007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.519+0000 7ff7cde00640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7b0078f10 0x7ff7b007b3d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00016a0 tx=0x7ff7c0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:52.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.523+0000 7ff7c6000640 1 -- 172.21.15.60:0/2034054071 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff7bc062e10 con 0x7ff7d00af3d0 2024-09-16T15:46:52.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.719+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7d00b3df0 con 0x7ff7d00af3d0 2024-09-16T15:46:52.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.720+0000 7ff7c6000640 1 -- 172.21.15.60:0/2034054071 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16775 (secure 0 0 0) 0x7ff7bc062710 con 0x7ff7d00af3d0 2024-09-16T15:46:52.722 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:52.722 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":149,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:49.882617+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:46.126734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:47.279485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:52.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7b0078f10 msgr2=0x7ff7b007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7b0078f10 0x7ff7b007b3d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00016a0 tx=0x7ff7c0009040 comp rx=0 tx=0).stop 2024-09-16T15:46:52.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d00af3d0 msgr2=0x7ff7d01500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d00af3d0 0x7ff7d01500d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00c500 tx=0x7ff7bc00c9c0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 shutdown_connections 2024-09-16T15:46:52.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7b0078f10 0x7ff7b007b3d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7d0161550 0x7ff7d0147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7d00af3d0 0x7ff7d01500d0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 --2- 172.21.15.60:0/2034054071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0073e00 0x7ff7d0157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 >> 172.21.15.60:0/2034054071 conn(0x7ff7d00b1950 msgr2=0x7ff7d00ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:52.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 shutdown_connections 2024-09-16T15:46:52.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.723+0000 7ff7d5000640 1 -- 172.21.15.60:0/2034054071 wait complete. 2024-09-16T15:46:52.740 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:46:52.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:46:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.899+0000 7f8023c00640 1 -- 172.21.15.60:0/3113566363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 msgr2=0x7f801c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.899+0000 7f8023c00640 1 --2- 172.21.15.60:0/3113566363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 0x7f801c0b2440 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f8004005fd0 tx=0x7f800402a5f0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.900+0000 7f8023c00640 1 -- 172.21.15.60:0/3113566363 shutdown_connections 2024-09-16T15:46:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.900+0000 7f8023c00640 1 --2- 172.21.15.60:0/3113566363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 0x7f801c0b2440 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.900+0000 7f8023c00640 1 -- 172.21.15.60:0/3113566363 >> 172.21.15.60:0/3113566363 conn(0x7f801c0b1950 msgr2=0x7f801c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.900+0000 7f8023c00640 1 -- 172.21.15.60:0/3113566363 shutdown_connections 2024-09-16T15:46:52.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.900+0000 7f8023c00640 1 -- 172.21.15.60:0/3113566363 wait complete. 2024-09-16T15:46:52.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.901+0000 7f8023c00640 1 Processor -- start 2024-09-16T15:46:52.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.901+0000 7f8023c00640 1 -- start start 2024-09-16T15:46:52.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.902+0000 7f8023c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801c073fb0 0x7f801c09d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.902+0000 7f8023c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 0x7f801c143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.902+0000 7f8023c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801c15f5a0 0x7f801c1f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.902+0000 7f8023c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f801c06f9f0 con 0x7f801c0ccf00 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.902+0000 7f8023c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f801c06f870 con 0x7f801c073fb0 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.902+0000 7f8023c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f801c074420 con 0x7f801c15f5a0 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 0x7f801c143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 0x7f801c143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56572/0 (socket says 172.21.15.60:56572) 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 -- 172.21.15.60:0/765794502 learned_addr learned my addr 172.21.15.60:0/765794502 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8021200640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801c073fb0 0x7f801c09d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8021c00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801c15f5a0 0x7f801c1f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 -- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801c15f5a0 msgr2=0x7f801c1f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801c15f5a0 0x7f801c1f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 -- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801c073fb0 msgr2=0x7f801c09d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801c073fb0 0x7f801c09d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 -- 172.21.15.60:0/765794502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c150c10 con 0x7f801c0ccf00 2024-09-16T15:46:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8021200640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801c073fb0 0x7f801c09d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8021c00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801c15f5a0 0x7f801c1f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.903+0000 7f8013e00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 0x7f801c143fc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8004009850 tx=0x7f80040039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:52.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.904+0000 7f8019600640 1 -- 172.21.15.60:0/765794502 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800403f070 con 0x7f801c0ccf00 2024-09-16T15:46:52.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.904+0000 7f8019600640 1 -- 172.21.15.60:0/765794502 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8004004890 con 0x7f801c0ccf00 2024-09-16T15:46:52.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.904+0000 7f8019600640 1 -- 172.21.15.60:0/765794502 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004035030 con 0x7f801c0ccf00 2024-09-16T15:46:52.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.904+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c0d00f0 con 0x7f801c0ccf00 2024-09-16T15:46:52.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.904+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c0d06b0 con 0x7f801c0ccf00 2024-09-16T15:46:52.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.906+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c0b2440 con 0x7f801c0ccf00 2024-09-16T15:46:52.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.907+0000 7f8019600640 1 -- 172.21.15.60:0/765794502 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f8004042070 con 0x7f801c0ccf00 2024-09-16T15:46:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.908+0000 7f8019600640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ff8078f10 0x7f7ff807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.908+0000 7f8019600640 1 -- 172.21.15.60:0/765794502 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f80040bc5f0 con 0x7f801c0ccf00 2024-09-16T15:46:52.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.909+0000 7f8021200640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ff8078f10 0x7f7ff807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.909+0000 7f8021200640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ff8078f10 0x7f7ff807b3d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f800c007fd0 tx=0x7f800c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:52.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:52.915+0000 7f8019600640 1 -- 172.21.15.60:0/765794502 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f80040841a0 con 0x7f801c0ccf00 2024-09-16T15:46:53.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.108+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f801c06fbe0 con 0x7f7ff8078f10 2024-09-16T15:46:53.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.113+0000 7f8019600640 1 -- 172.21.15.60:0/765794502 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f800c001cd0 con 0x7f7ff8078f10 2024-09-16T15:46:53.117 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:53.118 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:46:53.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ff8078f10 msgr2=0x7f7ff807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ff8078f10 0x7f7ff807b3d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f800c007fd0 tx=0x7f800c009040 comp rx=0 tx=0).stop 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 msgr2=0x7f801c143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 0x7f801c143fc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8004009850 tx=0x7f80040039e0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 shutdown_connections 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7ff8078f10 0x7f7ff807b3d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f801c15f5a0 0x7f801c1f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f801c0ccf00 0x7f801c143fc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 --2- 172.21.15.60:0/765794502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f801c073fb0 0x7f801c09d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 >> 172.21.15.60:0/765794502 conn(0x7f801c0b1950 msgr2=0x7f801c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 shutdown_connections 2024-09-16T15:46:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.117+0000 7f8023c00640 1 -- 172.21.15.60:0/765794502 wait complete. 2024-09-16T15:46:53.134 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-16T15:46:52.209323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281944,"kb_used_data":15784,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692776,"statfs":{"total":773094113280,"available":772805402624,"internally_reserved":0,"allocated":16162816,"data_stored":7774372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.347478"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926222+0000","last_change":"2024-09-16T15:46:48.900967+0000","last_active":"2024-09-16T15:46:50.926222+0000","last_peered":"2024-09-16T15:46:50.926222+0000","last_clean":"2024-09-16T15:46:50.926222+0000","last_became_active":"2024-09-16T15:46:48.900406+0000","last_became_peered":"2024-09-16T15:46:48.900406+0000","last_unstale":"2024-09-16T15:46:50.926222+0000","last_undegraded":"2024-09-16T15:46:50.926222+0000","last_fullsized":"2024-09-16T15:46:50.926222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:16.276589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892923+0000","last_change":"2024-09-16T15:46:48.890412+0000","last_active":"2024-09-16T15:46:49.892923+0000","last_peered":"2024-09-16T15:46:49.892923+0000","last_clean":"2024-09-16T15:46:49.892923+0000","last_became_active":"2024-09-16T15:46:48.890150+0000","last_became_peered":"2024-09-16T15:46:48.890150+0000","last_unstale":"2024-09-16T15:46:49.892923+0000","last_undegraded":"2024-09-16T15:46:49.892923+0000","last_fullsized":"2024-09-16T15:46:49.892923+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:25.426392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897782+0000","last_change":"2024-09-16T15:46:48.893129+0000","last_active":"2024-09-16T15:46:49.897782+0000","last_peered":"2024-09-16T15:46:49.897782+0000","last_clean":"2024-09-16T15:46:49.897782+0000","last_became_active":"2024-09-16T15:46:48.892855+0000","last_became_peered":"2024-09-16T15:46:48.892855+0000","last_unstale":"2024-09-16T15:46:49.897782+0000","last_undegraded":"2024-09-16T15:46:49.897782+0000","last_fullsized":"2024-09-16T15:46:49.897782+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:47:00.702665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926141+0000","last_change":"2024-09-16T15:46:48.892204+0000","last_active":"2024-09-16T15:46:50.926141+0000","last_peered":"2024-09-16T15:46:50.926141+0000","last_clean":"2024-09-16T15:46:50.926141+0000","last_became_active":"2024-09-16T15:46:48.891055+0000","last_became_peered":"2024-09-16T15:46:48.891055+0000","last_unstale":"2024-09-16T15:46:50.926141+0000","last_undegraded":"2024-09-16T15:46:50.926141+0000","last_fullsized":"2024-09-16T15:46:50.926141+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:38.346672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893431+0000","last_change":"2024-09-16T15:46:48.896205+0000","last_active":"2024-09-16T15:46:49.893431+0000","last_peered":"2024-09-16T15:46:49.893431+0000","last_clean":"2024-09-16T15:46:49.893431+0000","last_became_active":"2024-09-16T15:46:48.895832+0000","last_became_peered":"2024-09-16T15:46:48.895832+0000","last_unstale":"2024-09-16T15:46:49.893431+0000","last_undegraded":"2024-09-16T15:46:49.893431+0000","last_fullsized":"2024-09-16T15:46:49.893431+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:00.253371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890778+0000","last_change":"2024-09-16T15:46:48.910436+0000","last_active":"2024-09-16T15:46:49.890778+0000","last_peered":"2024-09-16T15:46:49.890778+0000","last_clean":"2024-09-16T15:46:49.890778+0000","last_became_active":"2024-09-16T15:46:48.910165+0000","last_became_peered":"2024-09-16T15:46:48.910165+0000","last_unstale":"2024-09-16T15:46:49.890778+0000","last_undegraded":"2024-09-16T15:46:49.890778+0000","last_fullsized":"2024-09-16T15:46:49.890778+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:27.085988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926995+0000","last_change":"2024-09-16T15:46:48.912355+0000","last_active":"2024-09-16T15:46:50.926995+0000","last_peered":"2024-09-16T15:46:50.926995+0000","last_clean":"2024-09-16T15:46:50.926995+0000","last_became_active":"2024-09-16T15:46:48.910750+0000","last_became_peered":"2024-09-16T15:46:48.910750+0000","last_unstale":"2024-09-16T15:46:50.926995+0000","last_undegraded":"2024-09-16T15:46:50.926995+0000","last_fullsized":"2024-09-16T15:46:50.926995+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:51.818119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892214+0000","last_change":"2024-09-16T15:46:48.894641+0000","last_active":"2024-09-16T15:46:49.892214+0000","last_peered":"2024-09-16T15:46:49.892214+0000","last_clean":"2024-09-16T15:46:49.892214+0000","last_became_active":"2024-09-16T15:46:48.892781+0000","last_became_peered":"2024-09-16T15:46:48.892781+0000","last_unstale":"2024-09-16T15:46:49.892214+0000","last_undegraded":"2024-09-16T15:46:49.892214+0000","last_fullsized":"2024-09-16T15:46:49.892214+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:11.301209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893191+0000","last_change":"2024-09-16T15:46:48.891546+0000","last_active":"2024-09-16T15:46:49.893191+0000","last_peered":"2024-09-16T15:46:49.893191+0000","last_clean":"2024-09-16T15:46:49.893191+0000","last_became_active":"2024-09-16T15:46:48.891280+0000","last_became_peered":"2024-09-16T15:46:48.891280+0000","last_unstale":"2024-09-16T15:46:49.893191+0000","last_undegraded":"2024-09-16T15:46:49.893191+0000","last_fullsized":"2024-09-16T15:46:49.893191+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:07.510561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893177+0000","last_change":"2024-09-16T15:46:48.892079+0000","last_active":"2024-09-16T15:46:49.893177+0000","last_peered":"2024-09-16T15:46:49.893177+0000","last_clean":"2024-09-16T15:46:49.893177+0000","last_became_active":"2024-09-16T15:46:48.891759+0000","last_became_peered":"2024-09-16T15:46:48.891759+0000","last_unstale":"2024-09-16T15:46:49.893177+0000","last_undegraded":"2024-09-16T15:46:49.893177+0000","last_fullsized":"2024-09-16T15:46:49.893177+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:09.763706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893564+0000","last_change":"2024-09-16T15:46:48.892197+0000","last_active":"2024-09-16T15:46:49.893564+0000","last_peered":"2024-09-16T15:46:49.893564+0000","last_clean":"2024-09-16T15:46:49.893564+0000","last_became_active":"2024-09-16T15:46:48.891886+0000","last_became_peered":"2024-09-16T15:46:48.891886+0000","last_unstale":"2024-09-16T15:46:49.893564+0000","last_undegraded":"2024-09-16T15:46:49.893564+0000","last_fullsized":"2024-09-16T15:46:49.893564+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:30:16.160509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892936+0000","last_change":"2024-09-16T15:46:48.895518+0000","last_active":"2024-09-16T15:46:49.892936+0000","last_peered":"2024-09-16T15:46:49.892936+0000","last_clean":"2024-09-16T15:46:49.892936+0000","last_became_active":"2024-09-16T15:46:48.893174+0000","last_became_peered":"2024-09-16T15:46:48.893174+0000","last_unstale":"2024-09-16T15:46:49.892936+0000","last_undegraded":"2024-09-16T15:46:49.892936+0000","last_fullsized":"2024-09-16T15:46:49.892936+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:31.518240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897554+0000","last_change":"2024-09-16T15:46:48.892191+0000","last_active":"2024-09-16T15:46:49.897554+0000","last_peered":"2024-09-16T15:46:49.897554+0000","last_clean":"2024-09-16T15:46:49.897554+0000","last_became_active":"2024-09-16T15:46:48.890579+0000","last_became_peered":"2024-09-16T15:46:48.890579+0000","last_unstale":"2024-09-16T15:46:49.897554+0000","last_undegraded":"2024-09-16T15:46:49.897554+0000","last_fullsized":"2024-09-16T15:46:49.897554+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:41.959328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897407+0000","last_change":"2024-09-16T15:46:48.891945+0000","last_active":"2024-09-16T15:46:49.897407+0000","last_peered":"2024-09-16T15:46:49.897407+0000","last_clean":"2024-09-16T15:46:49.897407+0000","last_became_active":"2024-09-16T15:46:48.891603+0000","last_became_peered":"2024-09-16T15:46:48.891603+0000","last_unstale":"2024-09-16T15:46:49.897407+0000","last_undegraded":"2024-09-16T15:46:49.897407+0000","last_fullsized":"2024-09-16T15:46:49.897407+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:57.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925812+0000","last_change":"2024-09-16T15:46:48.901267+0000","last_active":"2024-09-16T15:46:50.925812+0000","last_peered":"2024-09-16T15:46:50.925812+0000","last_clean":"2024-09-16T15:46:50.925812+0000","last_became_active":"2024-09-16T15:46:48.899570+0000","last_became_peered":"2024-09-16T15:46:48.899570+0000","last_unstale":"2024-09-16T15:46:50.925812+0000","last_undegraded":"2024-09-16T15:46:50.925812+0000","last_fullsized":"2024-09-16T15:46:50.925812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:33.082360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925979+0000","last_change":"2024-09-16T15:46:48.902551+0000","last_active":"2024-09-16T15:46:50.925979+0000","last_peered":"2024-09-16T15:46:50.925979+0000","last_clean":"2024-09-16T15:46:50.925979+0000","last_became_active":"2024-09-16T15:46:48.902284+0000","last_became_peered":"2024-09-16T15:46:48.902284+0000","last_unstale":"2024-09-16T15:46:50.925979+0000","last_undegraded":"2024-09-16T15:46:50.925979+0000","last_fullsized":"2024-09-16T15:46:50.925979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.418395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890972+0000","last_change":"2024-09-16T15:46:48.900865+0000","last_active":"2024-09-16T15:46:49.890972+0000","last_peered":"2024-09-16T15:46:49.890972+0000","last_clean":"2024-09-16T15:46:49.890972+0000","last_became_active":"2024-09-16T15:46:48.900520+0000","last_became_peered":"2024-09-16T15:46:48.900520+0000","last_unstale":"2024-09-16T15:46:49.890972+0000","last_undegraded":"2024-09-16T15:46:49.890972+0000","last_fullsized":"2024-09-16T15:46:49.890972+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:34.377608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927607+0000","last_change":"2024-09-16T15:46:48.895657+0000","last_active":"2024-09-16T15:46:50.927607+0000","last_peered":"2024-09-16T15:46:50.927607+0000","last_clean":"2024-09-16T15:46:50.927607+0000","last_became_active":"2024-09-16T15:46:48.895328+0000","last_became_peered":"2024-09-16T15:46:48.895328+0000","last_unstale":"2024-09-16T15:46:50.927607+0000","last_undegraded":"2024-09-16T15:46:50.927607+0000","last_fullsized":"2024-09-16T15:46:50.927607+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:56:41.551383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.891108+0000","last_change":"2024-09-16T15:46:48.900547+0000","last_active":"2024-09-16T15:46:49.891108+0000","last_peered":"2024-09-16T15:46:49.891108+0000","last_clean":"2024-09-16T15:46:49.891108+0000","last_became_active":"2024-09-16T15:46:48.900138+0000","last_became_peered":"2024-09-16T15:46:48.900138+0000","last_unstale":"2024-09-16T15:46:49.891108+0000","last_undegraded":"2024-09-16T15:46:49.891108+0000","last_fullsized":"2024-09-16T15:46:49.891108+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:39.066478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897488+0000","last_change":"2024-09-16T15:46:48.891116+0000","last_active":"2024-09-16T15:46:49.897488+0000","last_peered":"2024-09-16T15:46:49.897488+0000","last_clean":"2024-09-16T15:46:49.897488+0000","last_became_active":"2024-09-16T15:46:48.890807+0000","last_became_peered":"2024-09-16T15:46:48.890807+0000","last_unstale":"2024-09-16T15:46:49.897488+0000","last_undegraded":"2024-09-16T15:46:49.897488+0000","last_fullsized":"2024-09-16T15:46:49.897488+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:40.985244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890619+0000","last_change":"2024-09-16T15:46:48.891714+0000","last_active":"2024-09-16T15:46:49.890619+0000","last_peered":"2024-09-16T15:46:49.890619+0000","last_clean":"2024-09-16T15:46:49.890619+0000","last_became_active":"2024-09-16T15:46:48.890078+0000","last_became_peered":"2024-09-16T15:46:48.890078+0000","last_unstale":"2024-09-16T15:46:49.890619+0000","last_undegraded":"2024-09-16T15:46:49.890619+0000","last_fullsized":"2024-09-16T15:46:49.890619+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:39.713606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893718+0000","last_change":"2024-09-16T15:46:48.912110+0000","last_active":"2024-09-16T15:46:49.893718+0000","last_peered":"2024-09-16T15:46:49.893718+0000","last_clean":"2024-09-16T15:46:49.893718+0000","last_became_active":"2024-09-16T15:46:48.911800+0000","last_became_peered":"2024-09-16T15:46:48.911800+0000","last_unstale":"2024-09-16T15:46:49.893718+0000","last_undegraded":"2024-09-16T15:46:49.893718+0000","last_fullsized":"2024-09-16T15:46:49.893718+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:59.301801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890550+0000","last_change":"2024-09-16T15:46:48.893641+0000","last_active":"2024-09-16T15:46:49.890550+0000","last_peered":"2024-09-16T15:46:49.890550+0000","last_clean":"2024-09-16T15:46:49.890550+0000","last_became_active":"2024-09-16T15:46:48.893300+0000","last_became_peered":"2024-09-16T15:46:48.893300+0000","last_unstale":"2024-09-16T15:46:49.890550+0000","last_undegraded":"2024-09-16T15:46:49.890550+0000","last_fullsized":"2024-09-16T15:46:49.890550+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:30.000652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892430+0000","last_change":"2024-09-16T15:46:48.898630+0000","last_active":"2024-09-16T15:46:49.892430+0000","last_peered":"2024-09-16T15:46:49.892430+0000","last_clean":"2024-09-16T15:46:49.892430+0000","last_became_active":"2024-09-16T15:46:48.898307+0000","last_became_peered":"2024-09-16T15:46:48.898307+0000","last_unstale":"2024-09-16T15:46:49.892430+0000","last_undegraded":"2024-09-16T15:46:49.892430+0000","last_fullsized":"2024-09-16T15:46:49.892430+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:50.364555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.898127+0000","last_change":"2024-09-16T15:46:48.898293+0000","last_active":"2024-09-16T15:46:49.898127+0000","last_peered":"2024-09-16T15:46:49.898127+0000","last_clean":"2024-09-16T15:46:49.898127+0000","last_became_active":"2024-09-16T15:46:48.898035+0000","last_became_peered":"2024-09-16T15:46:48.898035+0000","last_unstale":"2024-09-16T15:46:49.898127+0000","last_undegraded":"2024-09-16T15:46:49.898127+0000","last_fullsized":"2024-09-16T15:46:49.898127+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:37:28.195593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927340+0000","last_change":"2024-09-16T15:46:48.904716+0000","last_active":"2024-09-16T15:46:50.927340+0000","last_peered":"2024-09-16T15:46:50.927340+0000","last_clean":"2024-09-16T15:46:50.927340+0000","last_became_active":"2024-09-16T15:46:48.904392+0000","last_became_peered":"2024-09-16T15:46:48.904392+0000","last_unstale":"2024-09-16T15:46:50.927340+0000","last_undegraded":"2024-09-16T15:46:50.927340+0000","last_fullsized":"2024-09-16T15:46:50.927340+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:08.303485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897845+0000","last_change":"2024-09-16T15:46:48.895242+0000","last_active":"2024-09-16T15:46:49.897845+0000","last_peered":"2024-09-16T15:46:49.897845+0000","last_clean":"2024-09-16T15:46:49.897845+0000","last_became_active":"2024-09-16T15:46:48.894898+0000","last_became_peered":"2024-09-16T15:46:48.894898+0000","last_unstale":"2024-09-16T15:46:49.897845+0000","last_undegraded":"2024-09-16T15:46:49.897845+0000","last_fullsized":"2024-09-16T15:46:49.897845+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:40.771736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926220+0000","last_change":"2024-09-16T15:46:48.898174+0000","last_active":"2024-09-16T15:46:50.926220+0000","last_peered":"2024-09-16T15:46:50.926220+0000","last_clean":"2024-09-16T15:46:50.926220+0000","last_became_active":"2024-09-16T15:46:48.897911+0000","last_became_peered":"2024-09-16T15:46:48.897911+0000","last_unstale":"2024-09-16T15:46:50.926220+0000","last_undegraded":"2024-09-16T15:46:50.926220+0000","last_fullsized":"2024-09-16T15:46:50.926220+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:16.429461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897590+0000","last_change":"2024-09-16T15:46:48.882660+0000","last_active":"2024-09-16T15:46:49.897590+0000","last_peered":"2024-09-16T15:46:49.897590+0000","last_clean":"2024-09-16T15:46:49.897590+0000","last_became_active":"2024-09-16T15:46:48.882342+0000","last_became_peered":"2024-09-16T15:46:48.882342+0000","last_unstale":"2024-09-16T15:46:49.897590+0000","last_undegraded":"2024-09-16T15:46:49.897590+0000","last_fullsized":"2024-09-16T15:46:49.897590+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:33.158335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.891930+0000","last_change":"2024-09-16T15:46:48.896621+0000","last_active":"2024-09-16T15:46:49.891930+0000","last_peered":"2024-09-16T15:46:49.891930+0000","last_clean":"2024-09-16T15:46:49.891930+0000","last_became_active":"2024-09-16T15:46:48.894218+0000","last_became_peered":"2024-09-16T15:46:48.894218+0000","last_unstale":"2024-09-16T15:46:49.891930+0000","last_undegraded":"2024-09-16T15:46:49.891930+0000","last_fullsized":"2024-09-16T15:46:49.891930+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:59.138850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892084+0000","last_change":"2024-09-16T15:46:48.911817+0000","last_active":"2024-09-16T15:46:49.892084+0000","last_peered":"2024-09-16T15:46:49.892084+0000","last_clean":"2024-09-16T15:46:49.892084+0000","last_became_active":"2024-09-16T15:46:48.910955+0000","last_became_peered":"2024-09-16T15:46:48.910955+0000","last_unstale":"2024-09-16T15:46:49.892084+0000","last_undegraded":"2024-09-16T15:46:49.892084+0000","last_fullsized":"2024-09-16T15:46:49.892084+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:00.288599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925925+0000","last_change":"2024-09-16T15:46:48.897561+0000","last_active":"2024-09-16T15:46:50.925925+0000","last_peered":"2024-09-16T15:46:50.925925+0000","last_clean":"2024-09-16T15:46:50.925925+0000","last_became_active":"2024-09-16T15:46:48.895591+0000","last_became_peered":"2024-09-16T15:46:48.895591+0000","last_unstale":"2024-09-16T15:46:50.925925+0000","last_undegraded":"2024-09-16T15:46:50.925925+0000","last_fullsized":"2024-09-16T15:46:50.925925+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:35.606013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927344+0000","last_change":"2024-09-16T15:46:48.903227+0000","last_active":"2024-09-16T15:46:50.927344+0000","last_peered":"2024-09-16T15:46:50.927344+0000","last_clean":"2024-09-16T15:46:50.927344+0000","last_became_active":"2024-09-16T15:46:48.902969+0000","last_became_peered":"2024-09-16T15:46:48.902969+0000","last_unstale":"2024-09-16T15:46:50.927344+0000","last_undegraded":"2024-09-16T15:46:50.927344+0000","last_fullsized":"2024-09-16T15:46:50.927344+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:56:41.551383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"148'4","reported_seq":19,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926101+0000","last_change":"2024-09-16T15:46:47.900622+0000","last_active":"2024-09-16T15:46:50.926101+0000","last_peered":"2024-09-16T15:46:50.926101+0000","last_clean":"2024-09-16T15:46:50.926101+0000","last_became_active":"2024-09-16T15:46:47.900325+0000","last_became_peered":"2024-09-16T15:46:47.900325+0000","last_unstale":"2024-09-16T15:46:50.926101+0000","last_undegraded":"2024-09-16T15:46:50.926101+0000","last_fullsized":"2024-09-16T15:46:50.926101+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:25.748554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893008+0000","last_change":"2024-09-16T15:46:48.903410+0000","last_active":"2024-09-16T15:46:49.893008+0000","last_peered":"2024-09-16T15:46:49.893008+0000","last_clean":"2024-09-16T15:46:49.893008+0000","last_became_active":"2024-09-16T15:46:48.903092+0000","last_became_peered":"2024-09-16T15:46:48.903092+0000","last_unstale":"2024-09-16T15:46:49.893008+0000","last_undegraded":"2024-09-16T15:46:49.893008+0000","last_fullsized":"2024-09-16T15:46:49.893008+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:50.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892291+0000","last_change":"2024-09-16T15:46:47.890007+0000","last_active":"2024-09-16T15:46:49.892291+0000","last_peered":"2024-09-16T15:46:49.892291+0000","last_clean":"2024-09-16T15:46:49.892291+0000","last_became_active":"2024-09-16T15:46:47.889684+0000","last_became_peered":"2024-09-16T15:46:47.889684+0000","last_unstale":"2024-09-16T15:46:49.892291+0000","last_undegraded":"2024-09-16T15:46:49.892291+0000","last_fullsized":"2024-09-16T15:46:49.892291+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:27.373954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890697+0000","last_change":"2024-09-16T15:46:48.892286+0000","last_active":"2024-09-16T15:46:49.890697+0000","last_peered":"2024-09-16T15:46:49.890697+0000","last_clean":"2024-09-16T15:46:49.890697+0000","last_became_active":"2024-09-16T15:46:48.890993+0000","last_became_peered":"2024-09-16T15:46:48.890993+0000","last_unstale":"2024-09-16T15:46:49.890697+0000","last_undegraded":"2024-09-16T15:46:49.890697+0000","last_fullsized":"2024-09-16T15:46:49.890697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:42.006424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926156+0000","last_change":"2024-09-16T15:46:47.895466+0000","last_active":"2024-09-16T15:46:50.926156+0000","last_peered":"2024-09-16T15:46:50.926156+0000","last_clean":"2024-09-16T15:46:50.926156+0000","last_became_active":"2024-09-16T15:46:47.895174+0000","last_became_peered":"2024-09-16T15:46:47.895174+0000","last_unstale":"2024-09-16T15:46:50.926156+0000","last_undegraded":"2024-09-16T15:46:50.926156+0000","last_fullsized":"2024-09-16T15:46:50.926156+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:51.379730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.898020+0000","last_change":"2024-09-16T15:46:48.900075+0000","last_active":"2024-09-16T15:46:49.898020+0000","last_peered":"2024-09-16T15:46:49.898020+0000","last_clean":"2024-09-16T15:46:49.898020+0000","last_became_active":"2024-09-16T15:46:48.897114+0000","last_became_peered":"2024-09-16T15:46:48.897114+0000","last_unstale":"2024-09-16T15:46:49.898020+0000","last_undegraded":"2024-09-16T15:46:49.898020+0000","last_fullsized":"2024-09-16T15:46:49.898020+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:48.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926302+0000","last_change":"2024-09-16T15:46:47.896773+0000","last_active":"2024-09-16T15:46:50.926302+0000","last_peered":"2024-09-16T15:46:50.926302+0000","last_clean":"2024-09-16T15:46:50.926302+0000","last_became_active":"2024-09-16T15:46:47.894534+0000","last_became_peered":"2024-09-16T15:46:47.894534+0000","last_unstale":"2024-09-16T15:46:50.926302+0000","last_undegraded":"2024-09-16T15:46:50.926302+0000","last_fullsized":"2024-09-16T15:46:50.926302+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:30.340375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.928039+0000","last_change":"2024-09-16T15:46:48.907726+0000","last_active":"2024-09-16T15:46:50.928039+0000","last_peered":"2024-09-16T15:46:50.928039+0000","last_clean":"2024-09-16T15:46:50.928039+0000","last_became_active":"2024-09-16T15:46:48.901962+0000","last_became_peered":"2024-09-16T15:46:48.901962+0000","last_unstale":"2024-09-16T15:46:50.928039+0000","last_undegraded":"2024-09-16T15:46:50.928039+0000","last_fullsized":"2024-09-16T15:46:50.928039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:39.933120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897622+0000","last_change":"2024-09-16T15:46:47.887430+0000","last_active":"2024-09-16T15:46:49.897622+0000","last_peered":"2024-09-16T15:46:49.897622+0000","last_clean":"2024-09-16T15:46:49.897622+0000","last_became_active":"2024-09-16T15:46:47.887160+0000","last_became_peered":"2024-09-16T15:46:47.887160+0000","last_unstale":"2024-09-16T15:46:49.897622+0000","last_undegraded":"2024-09-16T15:46:49.897622+0000","last_fullsized":"2024-09-16T15:46:49.897622+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:28.992390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926559+0000","last_change":"2024-09-16T15:46:48.905413+0000","last_active":"2024-09-16T15:46:50.926559+0000","last_peered":"2024-09-16T15:46:50.926559+0000","last_clean":"2024-09-16T15:46:50.926559+0000","last_became_active":"2024-09-16T15:46:48.905113+0000","last_became_peered":"2024-09-16T15:46:48.905113+0000","last_unstale":"2024-09-16T15:46:50.926559+0000","last_undegraded":"2024-09-16T15:46:50.926559+0000","last_fullsized":"2024-09-16T15:46:50.926559+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:24.926201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925427+0000","last_change":"2024-09-16T15:46:47.891563+0000","last_active":"2024-09-16T15:46:50.925427+0000","last_peered":"2024-09-16T15:46:50.925427+0000","last_clean":"2024-09-16T15:46:50.925427+0000","last_became_active":"2024-09-16T15:46:47.890927+0000","last_became_peered":"2024-09-16T15:46:47.890927+0000","last_unstale":"2024-09-16T15:46:50.925427+0000","last_undegraded":"2024-09-16T15:46:50.925427+0000","last_fullsized":"2024-09-16T15:46:50.925427+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:25.694143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925517+0000","last_change":"2024-09-16T15:46:48.892896+0000","last_active":"2024-09-16T15:46:50.925517+0000","last_peered":"2024-09-16T15:46:50.925517+0000","last_clean":"2024-09-16T15:46:50.925517+0000","last_became_active":"2024-09-16T15:46:48.892531+0000","last_became_peered":"2024-09-16T15:46:48.892531+0000","last_unstale":"2024-09-16T15:46:50.925517+0000","last_undegraded":"2024-09-16T15:46:50.925517+0000","last_fullsized":"2024-09-16T15:46:50.925517+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:24.205491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.891937+0000","last_change":"2024-09-16T15:46:47.896064+0000","last_active":"2024-09-16T15:46:49.891937+0000","last_peered":"2024-09-16T15:46:49.891937+0000","last_clean":"2024-09-16T15:46:49.891937+0000","last_became_active":"2024-09-16T15:46:47.895101+0000","last_became_peered":"2024-09-16T15:46:47.895101+0000","last_unstale":"2024-09-16T15:46:49.891937+0000","last_undegraded":"2024-09-16T15:46:49.891937+0000","last_fullsized":"2024-09-16T15:46:49.891937+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:47.334242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890763+0000","last_change":"2024-09-16T15:46:48.890952+0000","last_active":"2024-09-16T15:46:49.890763+0000","last_peered":"2024-09-16T15:46:49.890763+0000","last_clean":"2024-09-16T15:46:49.890763+0000","last_became_active":"2024-09-16T15:46:48.890599+0000","last_became_peered":"2024-09-16T15:46:48.890599+0000","last_unstale":"2024-09-16T15:46:49.890763+0000","last_undegraded":"2024-09-16T15:46:49.890763+0000","last_fullsized":"2024-09-16T15:46:49.890763+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:55.504236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926240+0000","last_change":"2024-09-16T15:46:47.883227+0000","last_active":"2024-09-16T15:46:50.926240+0000","last_peered":"2024-09-16T15:46:50.926240+0000","last_clean":"2024-09-16T15:46:50.926240+0000","last_became_active":"2024-09-16T15:46:47.882952+0000","last_became_peered":"2024-09-16T15:46:47.882952+0000","last_unstale":"2024-09-16T15:46:50.926240+0000","last_undegraded":"2024-09-16T15:46:50.926240+0000","last_fullsized":"2024-09-16T15:46:50.926240+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:02.804132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.928188+0000","last_change":"2024-09-16T15:46:48.897117+0000","last_active":"2024-09-16T15:46:50.928188+0000","last_peered":"2024-09-16T15:46:50.928188+0000","last_clean":"2024-09-16T15:46:50.928188+0000","last_became_active":"2024-09-16T15:46:48.896715+0000","last_became_peered":"2024-09-16T15:46:48.896715+0000","last_unstale":"2024-09-16T15:46:50.928188+0000","last_undegraded":"2024-09-16T15:46:50.928188+0000","last_fullsized":"2024-09-16T15:46:50.928188+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:33:52.360259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890962+0000","last_change":"2024-09-16T15:46:47.891622+0000","last_active":"2024-09-16T15:46:49.890962+0000","last_peered":"2024-09-16T15:46:49.890962+0000","last_clean":"2024-09-16T15:46:49.890962+0000","last_became_active":"2024-09-16T15:46:47.889681+0000","last_became_peered":"2024-09-16T15:46:47.889681+0000","last_unstale":"2024-09-16T15:46:49.890962+0000","last_undegraded":"2024-09-16T15:46:49.890962+0000","last_fullsized":"2024-09-16T15:46:49.890962+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:48.411182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927579+0000","last_change":"2024-09-16T15:46:48.889512+0000","last_active":"2024-09-16T15:46:50.927579+0000","last_peered":"2024-09-16T15:46:50.927579+0000","last_clean":"2024-09-16T15:46:50.927579+0000","last_became_active":"2024-09-16T15:46:48.889246+0000","last_became_peered":"2024-09-16T15:46:48.889246+0000","last_unstale":"2024-09-16T15:46:50.927579+0000","last_undegraded":"2024-09-16T15:46:50.927579+0000","last_fullsized":"2024-09-16T15:46:50.927579+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:24.343279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"148'3","reported_seq":18,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.891093+0000","last_change":"2024-09-16T15:46:47.895328+0000","last_active":"2024-09-16T15:46:49.891093+0000","last_peered":"2024-09-16T15:46:49.891093+0000","last_clean":"2024-09-16T15:46:49.891093+0000","last_became_active":"2024-09-16T15:46:47.895042+0000","last_became_peered":"2024-09-16T15:46:47.895042+0000","last_unstale":"2024-09-16T15:46:49.891093+0000","last_undegraded":"2024-09-16T15:46:49.891093+0000","last_fullsized":"2024-09-16T15:46:49.891093+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:32.462718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893407+0000","last_change":"2024-09-16T15:46:48.900432+0000","last_active":"2024-09-16T15:46:49.893407+0000","last_peered":"2024-09-16T15:46:49.893407+0000","last_clean":"2024-09-16T15:46:49.893407+0000","last_became_active":"2024-09-16T15:46:48.900128+0000","last_became_peered":"2024-09-16T15:46:48.900128+0000","last_unstale":"2024-09-16T15:46:49.893407+0000","last_undegraded":"2024-09-16T15:46:49.893407+0000","last_fullsized":"2024-09-16T15:46:49.893407+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:08.899758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"148'6","reported_seq":21,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892855+0000","last_change":"2024-09-16T15:46:47.893045+0000","last_active":"2024-09-16T15:46:49.892855+0000","last_peered":"2024-09-16T15:46:49.892855+0000","last_clean":"2024-09-16T15:46:49.892855+0000","last_became_active":"2024-09-16T15:46:47.892764+0000","last_became_peered":"2024-09-16T15:46:47.892764+0000","last_unstale":"2024-09-16T15:46:49.892855+0000","last_undegraded":"2024-09-16T15:46:49.892855+0000","last_fullsized":"2024-09-16T15:46:49.892855+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:21.759260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926876+0000","last_change":"2024-09-16T15:46:48.885309+0000","last_active":"2024-09-16T15:46:50.926876+0000","last_peered":"2024-09-16T15:46:50.926876+0000","last_clean":"2024-09-16T15:46:50.926876+0000","last_became_active":"2024-09-16T15:46:48.885017+0000","last_became_peered":"2024-09-16T15:46:48.885017+0000","last_unstale":"2024-09-16T15:46:50.926876+0000","last_undegraded":"2024-09-16T15:46:50.926876+0000","last_fullsized":"2024-09-16T15:46:50.926876+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:12.267438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925720+0000","last_change":"2024-09-16T15:46:47.895591+0000","last_active":"2024-09-16T15:46:50.925720+0000","last_peered":"2024-09-16T15:46:50.925720+0000","last_clean":"2024-09-16T15:46:50.925720+0000","last_became_active":"2024-09-16T15:46:47.895309+0000","last_became_peered":"2024-09-16T15:46:47.895309+0000","last_unstale":"2024-09-16T15:46:50.925720+0000","last_undegraded":"2024-09-16T15:46:50.925720+0000","last_fullsized":"2024-09-16T15:46:50.925720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:18.519639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926988+0000","last_change":"2024-09-16T15:46:48.904929+0000","last_active":"2024-09-16T15:46:50.926988+0000","last_peered":"2024-09-16T15:46:50.926988+0000","last_clean":"2024-09-16T15:46:50.926988+0000","last_became_active":"2024-09-16T15:46:48.904665+0000","last_became_peered":"2024-09-16T15:46:48.904665+0000","last_unstale":"2024-09-16T15:46:50.926988+0000","last_undegraded":"2024-09-16T15:46:50.926988+0000","last_fullsized":"2024-09-16T15:46:50.926988+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:56.929174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927145+0000","last_change":"2024-09-16T15:46:47.885564+0000","last_active":"2024-09-16T15:46:50.927145+0000","last_peered":"2024-09-16T15:46:50.927145+0000","last_clean":"2024-09-16T15:46:50.927145+0000","last_became_active":"2024-09-16T15:46:47.885190+0000","last_became_peered":"2024-09-16T15:46:47.885190+0000","last_unstale":"2024-09-16T15:46:50.927145+0000","last_undegraded":"2024-09-16T15:46:50.927145+0000","last_fullsized":"2024-09-16T15:46:50.927145+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:30.807658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925863+0000","last_change":"2024-09-16T15:46:48.883784+0000","last_active":"2024-09-16T15:46:50.925863+0000","last_peered":"2024-09-16T15:46:50.925863+0000","last_clean":"2024-09-16T15:46:50.925863+0000","last_became_active":"2024-09-16T15:46:48.883499+0000","last_became_peered":"2024-09-16T15:46:48.883499+0000","last_unstale":"2024-09-16T15:46:50.925863+0000","last_undegraded":"2024-09-16T15:46:50.925863+0000","last_fullsized":"2024-09-16T15:46:50.925863+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:35.959607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925430+0000","last_change":"2024-09-16T15:46:47.891083+0000","last_active":"2024-09-16T15:46:50.925430+0000","last_peered":"2024-09-16T15:46:50.925430+0000","last_clean":"2024-09-16T15:46:50.925430+0000","last_became_active":"2024-09-16T15:46:47.890770+0000","last_became_peered":"2024-09-16T15:46:47.890770+0000","last_unstale":"2024-09-16T15:46:50.925430+0000","last_undegraded":"2024-09-16T15:46:50.925430+0000","last_fullsized":"2024-09-16T15:46:50.925430+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:20.995661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897279+0000","last_change":"2024-09-16T15:46:48.895825+0000","last_active":"2024-09-16T15:46:49.897279+0000","last_peered":"2024-09-16T15:46:49.897279+0000","last_clean":"2024-09-16T15:46:49.897279+0000","last_became_active":"2024-09-16T15:46:48.895548+0000","last_became_peered":"2024-09-16T15:46:48.895548+0000","last_unstale":"2024-09-16T15:46:49.897279+0000","last_undegraded":"2024-09-16T15:46:49.897279+0000","last_fullsized":"2024-09-16T15:46:49.897279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:52.233022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"148'8","reported_seq":23,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892184+0000","last_change":"2024-09-16T15:46:47.897314+0000","last_active":"2024-09-16T15:46:49.892184+0000","last_peered":"2024-09-16T15:46:49.892184+0000","last_clean":"2024-09-16T15:46:49.892184+0000","last_became_active":"2024-09-16T15:46:47.896498+0000","last_became_peered":"2024-09-16T15:46:47.896498+0000","last_unstale":"2024-09-16T15:46:49.892184+0000","last_undegraded":"2024-09-16T15:46:49.892184+0000","last_fullsized":"2024-09-16T15:46:49.892184+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:39.812717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927101+0000","last_change":"2024-09-16T15:46:48.901465+0000","last_active":"2024-09-16T15:46:50.927101+0000","last_peered":"2024-09-16T15:46:50.927101+0000","last_clean":"2024-09-16T15:46:50.927101+0000","last_became_active":"2024-09-16T15:46:48.901133+0000","last_became_peered":"2024-09-16T15:46:48.901133+0000","last_unstale":"2024-09-16T15:46:50.927101+0000","last_undegraded":"2024-09-16T15:46:50.927101+0000","last_fullsized":"2024-09-16T15:46:50.927101+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:33.674633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.898009+0000","last_change":"2024-09-16T15:46:47.891866+0000","last_active":"2024-09-16T15:46:49.898009+0000","last_peered":"2024-09-16T15:46:49.898009+0000","last_clean":"2024-09-16T15:46:49.898009+0000","last_became_active":"2024-09-16T15:46:47.891581+0000","last_became_peered":"2024-09-16T15:46:47.891581+0000","last_unstale":"2024-09-16T15:46:49.898009+0000","last_undegraded":"2024-09-16T15:46:49.898009+0000","last_fullsized":"2024-09-16T15:46:49.898009+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:30.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.928112+0000","last_change":"2024-09-16T15:46:48.897355+0000","last_active":"2024-09-16T15:46:50.928112+0000","last_peered":"2024-09-16T15:46:50.928112+0000","last_clean":"2024-09-16T15:46:50.928112+0000","last_became_active":"2024-09-16T15:46:48.897053+0000","last_became_peered":"2024-09-16T15:46:48.897053+0000","last_unstale":"2024-09-16T15:46:50.928112+0000","last_undegraded":"2024-09-16T15:46:50.928112+0000","last_fullsized":"2024-09-16T15:46:50.928112+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:26.756816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897720+0000","last_change":"2024-09-16T15:46:47.897237+0000","last_active":"2024-09-16T15:46:49.897720+0000","last_peered":"2024-09-16T15:46:49.897720+0000","last_clean":"2024-09-16T15:46:49.897720+0000","last_became_active":"2024-09-16T15:46:47.895746+0000","last_became_peered":"2024-09-16T15:46:47.895746+0000","last_unstale":"2024-09-16T15:46:49.897720+0000","last_undegraded":"2024-09-16T15:46:49.897720+0000","last_fullsized":"2024-09-16T15:46:49.897720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:41.460497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926361+0000","last_change":"2024-09-16T15:46:48.906158+0000","last_active":"2024-09-16T15:46:50.926361+0000","last_peered":"2024-09-16T15:46:50.926361+0000","last_clean":"2024-09-16T15:46:50.926361+0000","last_became_active":"2024-09-16T15:46:48.905888+0000","last_became_peered":"2024-09-16T15:46:48.905888+0000","last_unstale":"2024-09-16T15:46:50.926361+0000","last_undegraded":"2024-09-16T15:46:50.926361+0000","last_fullsized":"2024-09-16T15:46:50.926361+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:25.044209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925818+0000","last_change":"2024-09-16T15:46:47.898515+0000","last_active":"2024-09-16T15:46:50.925818+0000","last_peered":"2024-09-16T15:46:50.925818+0000","last_clean":"2024-09-16T15:46:50.925818+0000","last_became_active":"2024-09-16T15:46:47.898272+0000","last_became_peered":"2024-09-16T15:46:47.898272+0000","last_unstale":"2024-09-16T15:46:50.925818+0000","last_undegraded":"2024-09-16T15:46:50.925818+0000","last_fullsized":"2024-09-16T15:46:50.925818+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:47.299272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926967+0000","last_change":"2024-09-16T15:46:48.915720+0000","last_active":"2024-09-16T15:46:50.926967+0000","last_peered":"2024-09-16T15:46:50.926967+0000","last_clean":"2024-09-16T15:46:50.926967+0000","last_became_active":"2024-09-16T15:46:48.915372+0000","last_became_peered":"2024-09-16T15:46:48.915372+0000","last_unstale":"2024-09-16T15:46:50.926967+0000","last_undegraded":"2024-09-16T15:46:50.926967+0000","last_fullsized":"2024-09-16T15:46:50.926967+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:07.863660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926040+0000","last_change":"2024-09-16T15:46:47.899653+0000","last_active":"2024-09-16T15:46:50.926040+0000","last_peered":"2024-09-16T15:46:50.926040+0000","last_clean":"2024-09-16T15:46:50.926040+0000","last_became_active":"2024-09-16T15:46:47.898882+0000","last_became_peered":"2024-09-16T15:46:47.898882+0000","last_unstale":"2024-09-16T15:46:50.926040+0000","last_undegraded":"2024-09-16T15:46:50.926040+0000","last_fullsized":"2024-09-16T15:46:50.926040+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:50.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897453+0000","last_change":"2024-09-16T15:46:48.889425+0000","last_active":"2024-09-16T15:46:49.897453+0000","last_peered":"2024-09-16T15:46:49.897453+0000","last_clean":"2024-09-16T15:46:49.897453+0000","last_became_active":"2024-09-16T15:46:48.888802+0000","last_became_peered":"2024-09-16T15:46:48.888802+0000","last_unstale":"2024-09-16T15:46:49.897453+0000","last_undegraded":"2024-09-16T15:46:49.897453+0000","last_fullsized":"2024-09-16T15:46:49.897453+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:37.504699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925839+0000","last_change":"2024-09-16T15:46:47.895959+0000","last_active":"2024-09-16T15:46:50.925839+0000","last_peered":"2024-09-16T15:46:50.925839+0000","last_clean":"2024-09-16T15:46:50.925839+0000","last_became_active":"2024-09-16T15:46:47.894748+0000","last_became_peered":"2024-09-16T15:46:47.894748+0000","last_unstale":"2024-09-16T15:46:50.925839+0000","last_undegraded":"2024-09-16T15:46:50.925839+0000","last_fullsized":"2024-09-16T15:46:50.925839+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:01.766307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927198+0000","last_change":"2024-09-16T15:46:48.909353+0000","last_active":"2024-09-16T15:46:50.927198+0000","last_peered":"2024-09-16T15:46:50.927198+0000","last_clean":"2024-09-16T15:46:50.927198+0000","last_became_active":"2024-09-16T15:46:48.909018+0000","last_became_peered":"2024-09-16T15:46:48.909018+0000","last_unstale":"2024-09-16T15:46:50.927198+0000","last_undegraded":"2024-09-16T15:46:50.927198+0000","last_fullsized":"2024-09-16T15:46:50.927198+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:40.601143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925749+0000","last_change":"2024-09-16T15:46:47.898112+0000","last_active":"2024-09-16T15:46:50.925749+0000","last_peered":"2024-09-16T15:46:50.925749+0000","last_clean":"2024-09-16T15:46:50.925749+0000","last_became_active":"2024-09-16T15:46:47.897822+0000","last_became_peered":"2024-09-16T15:46:47.897822+0000","last_unstale":"2024-09-16T15:46:50.925749+0000","last_undegraded":"2024-09-16T15:46:50.925749+0000","last_fullsized":"2024-09-16T15:46:50.925749+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:23.335017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925863+0000","last_change":"2024-09-16T15:46:48.897203+0000","last_active":"2024-09-16T15:46:50.925863+0000","last_peered":"2024-09-16T15:46:50.925863+0000","last_clean":"2024-09-16T15:46:50.925863+0000","last_became_active":"2024-09-16T15:46:48.895256+0000","last_became_peered":"2024-09-16T15:46:48.895256+0000","last_unstale":"2024-09-16T15:46:50.925863+0000","last_undegraded":"2024-09-16T15:46:50.925863+0000","last_fullsized":"2024-09-16T15:46:50.925863+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:56.233753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925490+0000","last_change":"2024-09-16T15:46:47.892309+0000","last_active":"2024-09-16T15:46:50.925490+0000","last_peered":"2024-09-16T15:46:50.925490+0000","last_clean":"2024-09-16T15:46:50.925490+0000","last_became_active":"2024-09-16T15:46:47.891709+0000","last_became_peered":"2024-09-16T15:46:47.891709+0000","last_unstale":"2024-09-16T15:46:50.925490+0000","last_undegraded":"2024-09-16T15:46:50.925490+0000","last_fullsized":"2024-09-16T15:46:50.925490+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:16.238494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890970+0000","last_change":"2024-09-16T15:46:48.887168+0000","last_active":"2024-09-16T15:46:49.890970+0000","last_peered":"2024-09-16T15:46:49.890970+0000","last_clean":"2024-09-16T15:46:49.890970+0000","last_became_active":"2024-09-16T15:46:48.886927+0000","last_became_peered":"2024-09-16T15:46:48.886927+0000","last_unstale":"2024-09-16T15:46:49.890970+0000","last_undegraded":"2024-09-16T15:46:49.890970+0000","last_fullsized":"2024-09-16T15:46:49.890970+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:47.777129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925876+0000","last_change":"2024-09-16T15:46:47.899580+0000","last_active":"2024-09-16T15:46:50.925876+0000","last_peered":"2024-09-16T15:46:50.925876+0000","last_clean":"2024-09-16T15:46:50.925876+0000","last_became_active":"2024-09-16T15:46:47.898928+0000","last_became_peered":"2024-09-16T15:46:47.898928+0000","last_unstale":"2024-09-16T15:46:50.925876+0000","last_undegraded":"2024-09-16T15:46:50.925876+0000","last_fullsized":"2024-09-16T15:46:50.925876+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:56:40.543121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927934+0000","last_change":"2024-09-16T15:46:48.886053+0000","last_active":"2024-09-16T15:46:50.927934+0000","last_peered":"2024-09-16T15:46:50.927934+0000","last_clean":"2024-09-16T15:46:50.927934+0000","last_became_active":"2024-09-16T15:46:48.885785+0000","last_became_peered":"2024-09-16T15:46:48.885785+0000","last_unstale":"2024-09-16T15:46:50.927934+0000","last_undegraded":"2024-09-16T15:46:50.927934+0000","last_fullsized":"2024-09-16T15:46:50.927934+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:51.033590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.928148+0000","last_change":"2024-09-16T15:46:47.892421+0000","last_active":"2024-09-16T15:46:50.928148+0000","last_peered":"2024-09-16T15:46:50.928148+0000","last_clean":"2024-09-16T15:46:50.928148+0000","last_became_active":"2024-09-16T15:46:47.892107+0000","last_became_peered":"2024-09-16T15:46:47.892107+0000","last_unstale":"2024-09-16T15:46:50.928148+0000","last_undegraded":"2024-09-16T15:46:50.928148+0000","last_fullsized":"2024-09-16T15:46:50.928148+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:26.857268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927552+0000","last_change":"2024-09-16T15:46:48.912341+0000","last_active":"2024-09-16T15:46:50.927552+0000","last_peered":"2024-09-16T15:46:50.927552+0000","last_clean":"2024-09-16T15:46:50.927552+0000","last_became_active":"2024-09-16T15:46:48.910886+0000","last_became_peered":"2024-09-16T15:46:48.910886+0000","last_unstale":"2024-09-16T15:46:50.927552+0000","last_undegraded":"2024-09-16T15:46:50.927552+0000","last_fullsized":"2024-09-16T15:46:50.927552+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:15.479426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"149'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897822+0000","last_change":"2024-09-16T15:46:47.888773+0000","last_active":"2024-09-16T15:46:49.897822+0000","last_peered":"2024-09-16T15:46:49.897822+0000","last_clean":"2024-09-16T15:46:49.897822+0000","last_became_active":"2024-09-16T15:46:47.888159+0000","last_became_peered":"2024-09-16T15:46:47.888159+0000","last_unstale":"2024-09-16T15:46:49.897822+0000","last_undegraded":"2024-09-16T15:46:49.897822+0000","last_fullsized":"2024-09-16T15:46:49.897822+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:40.998162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890764+0000","last_change":"2024-09-16T15:46:48.897741+0000","last_active":"2024-09-16T15:46:49.890764+0000","last_peered":"2024-09-16T15:46:49.890764+0000","last_clean":"2024-09-16T15:46:49.890764+0000","last_became_active":"2024-09-16T15:46:48.896800+0000","last_became_peered":"2024-09-16T15:46:48.896800+0000","last_unstale":"2024-09-16T15:46:49.890764+0000","last_undegraded":"2024-09-16T15:46:49.890764+0000","last_fullsized":"2024-09-16T15:46:49.890764+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:25.528026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893437+0000","last_change":"2024-09-16T15:46:47.896061+0000","last_active":"2024-09-16T15:46:49.893437+0000","last_peered":"2024-09-16T15:46:49.893437+0000","last_clean":"2024-09-16T15:46:49.893437+0000","last_became_active":"2024-09-16T15:46:47.895517+0000","last_became_peered":"2024-09-16T15:46:47.895517+0000","last_unstale":"2024-09-16T15:46:49.893437+0000","last_undegraded":"2024-09-16T15:46:49.893437+0000","last_fullsized":"2024-09-16T15:46:49.893437+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:01.127898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927552+0000","last_change":"2024-09-16T15:46:48.892833+0000","last_active":"2024-09-16T15:46:50.927552+0000","last_peered":"2024-09-16T15:46:50.927552+0000","last_clean":"2024-09-16T15:46:50.927552+0000","last_became_active":"2024-09-16T15:46:48.892521+0000","last_became_peered":"2024-09-16T15:46:48.892521+0000","last_unstale":"2024-09-16T15:46:50.927552+0000","last_undegraded":"2024-09-16T15:46:50.927552+0000","last_fullsized":"2024-09-16T15:46:50.927552+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:12.880086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927395+0000","last_change":"2024-09-16T15:46:47.887865+0000","last_active":"2024-09-16T15:46:50.927395+0000","last_peered":"2024-09-16T15:46:50.927395+0000","last_clean":"2024-09-16T15:46:50.927395+0000","last_became_active":"2024-09-16T15:46:47.887538+0000","last_became_peered":"2024-09-16T15:46:47.887538+0000","last_unstale":"2024-09-16T15:46:50.927395+0000","last_undegraded":"2024-09-16T15:46:50.927395+0000","last_fullsized":"2024-09-16T15:46:50.927395+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:02.804132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927082+0000","last_change":"2024-09-16T15:46:48.898080+0000","last_active":"2024-09-16T15:46:50.927082+0000","last_peered":"2024-09-16T15:46:50.927082+0000","last_clean":"2024-09-16T15:46:50.927082+0000","last_became_active":"2024-09-16T15:46:48.896476+0000","last_became_peered":"2024-09-16T15:46:48.896476+0000","last_unstale":"2024-09-16T15:46:50.927082+0000","last_undegraded":"2024-09-16T15:46:50.927082+0000","last_fullsized":"2024-09-16T15:46:50.927082+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:51.818119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893243+0000","last_change":"2024-09-16T15:46:47.891693+0000","last_active":"2024-09-16T15:46:49.893243+0000","last_peered":"2024-09-16T15:46:49.893243+0000","last_clean":"2024-09-16T15:46:49.893243+0000","last_became_active":"2024-09-16T15:46:47.891395+0000","last_became_peered":"2024-09-16T15:46:47.891395+0000","last_unstale":"2024-09-16T15:46:49.893243+0000","last_undegraded":"2024-09-16T15:46:49.893243+0000","last_fullsized":"2024-09-16T15:46:49.893243+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:57.110332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893177+0000","last_change":"2024-09-16T15:46:48.912231+0000","last_active":"2024-09-16T15:46:49.893177+0000","last_peered":"2024-09-16T15:46:49.893177+0000","last_clean":"2024-09-16T15:46:49.893177+0000","last_became_active":"2024-09-16T15:46:48.911189+0000","last_became_peered":"2024-09-16T15:46:48.911189+0000","last_unstale":"2024-09-16T15:46:49.893177+0000","last_undegraded":"2024-09-16T15:46:49.893177+0000","last_fullsized":"2024-09-16T15:46:49.893177+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:04.918984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925564+0000","last_change":"2024-09-16T15:46:47.891906+0000","last_active":"2024-09-16T15:46:50.925564+0000","last_peered":"2024-09-16T15:46:50.925564+0000","last_clean":"2024-09-16T15:46:50.925564+0000","last_became_active":"2024-09-16T15:46:47.891304+0000","last_became_peered":"2024-09-16T15:46:47.891304+0000","last_unstale":"2024-09-16T15:46:50.925564+0000","last_undegraded":"2024-09-16T15:46:50.925564+0000","last_fullsized":"2024-09-16T15:46:50.925564+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:53:39.577659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926182+0000","last_change":"2024-09-16T15:46:48.905195+0000","last_active":"2024-09-16T15:46:50.926182+0000","last_peered":"2024-09-16T15:46:50.926182+0000","last_clean":"2024-09-16T15:46:50.926182+0000","last_became_active":"2024-09-16T15:46:48.901594+0000","last_became_peered":"2024-09-16T15:46:48.901594+0000","last_unstale":"2024-09-16T15:46:50.926182+0000","last_undegraded":"2024-09-16T15:46:50.926182+0000","last_fullsized":"2024-09-16T15:46:50.926182+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:17.100042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893622+0000","last_change":"2024-09-16T15:46:47.897438+0000","last_active":"2024-09-16T15:46:49.893622+0000","last_peered":"2024-09-16T15:46:49.893622+0000","last_clean":"2024-09-16T15:46:49.893622+0000","last_became_active":"2024-09-16T15:46:47.896065+0000","last_became_peered":"2024-09-16T15:46:47.896065+0000","last_unstale":"2024-09-16T15:46:49.893622+0000","last_undegraded":"2024-09-16T15:46:49.893622+0000","last_fullsized":"2024-09-16T15:46:49.893622+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:57.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897613+0000","last_change":"2024-09-16T15:46:48.894679+0000","last_active":"2024-09-16T15:46:49.897613+0000","last_peered":"2024-09-16T15:46:49.897613+0000","last_clean":"2024-09-16T15:46:49.897613+0000","last_became_active":"2024-09-16T15:46:48.894389+0000","last_became_peered":"2024-09-16T15:46:48.894389+0000","last_unstale":"2024-09-16T15:46:49.897613+0000","last_undegraded":"2024-09-16T15:46:49.897613+0000","last_fullsized":"2024-09-16T15:46:49.897613+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:57.876317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892317+0000","last_change":"2024-09-16T15:46:47.885950+0000","last_active":"2024-09-16T15:46:49.892317+0000","last_peered":"2024-09-16T15:46:49.892317+0000","last_clean":"2024-09-16T15:46:49.892317+0000","last_became_active":"2024-09-16T15:46:47.885466+0000","last_became_peered":"2024-09-16T15:46:47.885466+0000","last_unstale":"2024-09-16T15:46:49.892317+0000","last_undegraded":"2024-09-16T15:46:49.892317+0000","last_fullsized":"2024-09-16T15:46:49.892317+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:48.207712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927910+0000","last_change":"2024-09-16T15:46:48.898845+0000","last_active":"2024-09-16T15:46:50.927910+0000","last_peered":"2024-09-16T15:46:50.927910+0000","last_clean":"2024-09-16T15:46:50.927910+0000","last_became_active":"2024-09-16T15:46:48.898575+0000","last_became_peered":"2024-09-16T15:46:48.898575+0000","last_unstale":"2024-09-16T15:46:50.927910+0000","last_undegraded":"2024-09-16T15:46:50.927910+0000","last_fullsized":"2024-09-16T15:46:50.927910+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:48.307534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893078+0000","last_change":"2024-09-16T15:46:47.894952+0000","last_active":"2024-09-16T15:46:49.893078+0000","last_peered":"2024-09-16T15:46:49.893078+0000","last_clean":"2024-09-16T15:46:49.893078+0000","last_became_active":"2024-09-16T15:46:47.894645+0000","last_became_peered":"2024-09-16T15:46:47.894645+0000","last_unstale":"2024-09-16T15:46:49.893078+0000","last_undegraded":"2024-09-16T15:46:49.893078+0000","last_fullsized":"2024-09-16T15:46:49.893078+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:08.176340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":373,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925935+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:46:50.925935+0000","last_peered":"2024-09-16T15:46:50.925935+0000","last_clean":"2024-09-16T15:46:50.925935+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:46:50.925935+0000","last_undegraded":"2024-09-16T15:46:50.925935+0000","last_fullsized":"2024-09-16T15:46:50.925935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33616,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338224,"statfs":{"total":96636764160,"available":96602341376,"internally_reserved":0,"allocated":2482176,"data_stored":1416324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1867776,"data_stored":823624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1871872,"data_stored":824206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509544,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1880064,"data_stored":825284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":823636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1867776,"data_stored":823692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1867776,"data_stored":823636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37748,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334092,"statfs":{"total":96636764160,"available":96598110208,"internally_reserved":0,"allocated":2457600,"data_stored":1413970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:46:53.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:46:53.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.334+0000 7ff0aa600640 1 -- 172.21.15.60:0/4060310645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 msgr2=0x7ff088003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.334+0000 7ff0aa600640 1 --2- 172.21.15.60:0/4060310645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 0x7ff088003ec0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff094009b50 tx=0x7ff088005830 comp rx=0 tx=0).stop 2024-09-16T15:46:53.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.334+0000 7ff0aa600640 1 -- 172.21.15.60:0/4060310645 shutdown_connections 2024-09-16T15:46:53.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.334+0000 7ff0aa600640 1 --2- 172.21.15.60:0/4060310645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 0x7ff088003ec0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.334+0000 7ff0aa600640 1 -- 172.21.15.60:0/4060310645 >> 172.21.15.60:0/4060310645 conn(0x7ff0a406d230 msgr2=0x7ff0a40688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:53.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.334+0000 7ff0aa600640 1 -- 172.21.15.60:0/4060310645 shutdown_connections 2024-09-16T15:46:53.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.334+0000 7ff0aa600640 1 -- 172.21.15.60:0/4060310645 wait complete. 2024-09-16T15:46:53.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.335+0000 7ff0aa600640 1 Processor -- start 2024-09-16T15:46:53.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.336+0000 7ff0aa600640 1 -- start start 2024-09-16T15:46:53.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.336+0000 7ff0aa600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 0x7ff0a40d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0aa600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a414b1e0 0x7ff0a414b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0aa600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a414a5b0 0x7ff0a414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0aa600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4071d70 con 0x7ff088003aa0 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0aa600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4071bf0 con 0x7ff0a414a5b0 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0aa600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a4071ef0 con 0x7ff0a414b1e0 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 0x7ff0a40d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 0x7ff0a40d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56592/0 (socket says 172.21.15.60:56592) 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3400640 1 -- 172.21.15.60:0/2227934892 learned_addr learned my addr 172.21.15.60:0/2227934892 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3e00640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a414a5b0 0x7ff0a414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a2a00640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a414b1e0 0x7ff0a414b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3400640 1 -- 172.21.15.60:0/2227934892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a414b1e0 msgr2=0x7ff0a414b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3400640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a414b1e0 0x7ff0a414b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3400640 1 -- 172.21.15.60:0/2227934892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a414a5b0 msgr2=0x7ff0a414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3400640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a414a5b0 0x7ff0a414aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.337+0000 7ff0a3400640 1 -- 172.21.15.60:0/2227934892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a40c7c80 con 0x7ff088003aa0 2024-09-16T15:46:53.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.338+0000 7ff0a3400640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 0x7ff0a40d0fe0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff094009a70 tx=0x7ff0940336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:53.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.338+0000 7ff09b400640 1 -- 172.21.15.60:0/2227934892 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094038040 con 0x7ff088003aa0 2024-09-16T15:46:53.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.338+0000 7ff09b400640 1 -- 172.21.15.60:0/2227934892 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0940339d0 con 0x7ff088003aa0 2024-09-16T15:46:53.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.338+0000 7ff09b400640 1 -- 172.21.15.60:0/2227934892 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094033e20 con 0x7ff088003aa0 2024-09-16T15:46:53.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.338+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a40c7f60 con 0x7ff088003aa0 2024-09-16T15:46:53.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.338+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a40c8520 con 0x7ff088003aa0 2024-09-16T15:46:53.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.339+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a40ced50 con 0x7ff088003aa0 2024-09-16T15:46:53.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.340+0000 7ff09b400640 1 -- 172.21.15.60:0/2227934892 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7ff094043020 con 0x7ff088003aa0 2024-09-16T15:46:53.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.341+0000 7ff09b400640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff084078f10 0x7ff08407b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.341+0000 7ff09b400640 1 -- 172.21.15.60:0/2227934892 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7ff0940bb550 con 0x7ff088003aa0 2024-09-16T15:46:53.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.341+0000 7ff0a2a00640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff084078f10 0x7ff08407b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.342+0000 7ff0a2a00640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff084078f10 0x7ff08407b3d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff0a414f340 tx=0x7ff090006b70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:53.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.345+0000 7ff09b400640 1 -- 172.21.15.60:0/2227934892 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff09404a020 con 0x7ff088003aa0 2024-09-16T15:46:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.536+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0a415a800 con 0x7ff084078f10 2024-09-16T15:46:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.541+0000 7ff09b400640 1 -- 172.21.15.60:0/2227934892 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7ff0a415a800 con 0x7ff084078f10 2024-09-16T15:46:53.545 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:53.546 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:46:53.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff084078f10 msgr2=0x7ff08407b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff084078f10 0x7ff08407b3d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff0a414f340 tx=0x7ff090006b70 comp rx=0 tx=0).stop 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 msgr2=0x7ff0a40d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 0x7ff0a40d0fe0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff094009a70 tx=0x7ff0940336f0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 shutdown_connections 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff084078f10 0x7ff08407b3d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a414a5b0 0x7ff0a414aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a414b1e0 0x7ff0a414b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 --2- 172.21.15.60:0/2227934892 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff088003aa0 0x7ff0a40d0fe0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 >> 172.21.15.60:0/2227934892 conn(0x7ff0a406d230 msgr2=0x7ff0a4157860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 shutdown_connections 2024-09-16T15:46:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.545+0000 7ff0aa600640 1 -- 172.21.15.60:0/2227934892 wait complete. 2024-09-16T15:46:53.562 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-16T15:46:52.209323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281944,"kb_used_data":15784,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692776,"statfs":{"total":773094113280,"available":772805402624,"internally_reserved":0,"allocated":16162816,"data_stored":7774372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.347478"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926222+0000","last_change":"2024-09-16T15:46:48.900967+0000","last_active":"2024-09-16T15:46:50.926222+0000","last_peered":"2024-09-16T15:46:50.926222+0000","last_clean":"2024-09-16T15:46:50.926222+0000","last_became_active":"2024-09-16T15:46:48.900406+0000","last_became_peered":"2024-09-16T15:46:48.900406+0000","last_unstale":"2024-09-16T15:46:50.926222+0000","last_undegraded":"2024-09-16T15:46:50.926222+0000","last_fullsized":"2024-09-16T15:46:50.926222+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:16.276589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892923+0000","last_change":"2024-09-16T15:46:48.890412+0000","last_active":"2024-09-16T15:46:49.892923+0000","last_peered":"2024-09-16T15:46:49.892923+0000","last_clean":"2024-09-16T15:46:49.892923+0000","last_became_active":"2024-09-16T15:46:48.890150+0000","last_became_peered":"2024-09-16T15:46:48.890150+0000","last_unstale":"2024-09-16T15:46:49.892923+0000","last_undegraded":"2024-09-16T15:46:49.892923+0000","last_fullsized":"2024-09-16T15:46:49.892923+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:53:25.426392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897782+0000","last_change":"2024-09-16T15:46:48.893129+0000","last_active":"2024-09-16T15:46:49.897782+0000","last_peered":"2024-09-16T15:46:49.897782+0000","last_clean":"2024-09-16T15:46:49.897782+0000","last_became_active":"2024-09-16T15:46:48.892855+0000","last_became_peered":"2024-09-16T15:46:48.892855+0000","last_unstale":"2024-09-16T15:46:49.897782+0000","last_undegraded":"2024-09-16T15:46:49.897782+0000","last_fullsized":"2024-09-16T15:46:49.897782+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:47:00.702665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926141+0000","last_change":"2024-09-16T15:46:48.892204+0000","last_active":"2024-09-16T15:46:50.926141+0000","last_peered":"2024-09-16T15:46:50.926141+0000","last_clean":"2024-09-16T15:46:50.926141+0000","last_became_active":"2024-09-16T15:46:48.891055+0000","last_became_peered":"2024-09-16T15:46:48.891055+0000","last_unstale":"2024-09-16T15:46:50.926141+0000","last_undegraded":"2024-09-16T15:46:50.926141+0000","last_fullsized":"2024-09-16T15:46:50.926141+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:05:38.346672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893431+0000","last_change":"2024-09-16T15:46:48.896205+0000","last_active":"2024-09-16T15:46:49.893431+0000","last_peered":"2024-09-16T15:46:49.893431+0000","last_clean":"2024-09-16T15:46:49.893431+0000","last_became_active":"2024-09-16T15:46:48.895832+0000","last_became_peered":"2024-09-16T15:46:48.895832+0000","last_unstale":"2024-09-16T15:46:49.893431+0000","last_undegraded":"2024-09-16T15:46:49.893431+0000","last_fullsized":"2024-09-16T15:46:49.893431+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:00.253371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890778+0000","last_change":"2024-09-16T15:46:48.910436+0000","last_active":"2024-09-16T15:46:49.890778+0000","last_peered":"2024-09-16T15:46:49.890778+0000","last_clean":"2024-09-16T15:46:49.890778+0000","last_became_active":"2024-09-16T15:46:48.910165+0000","last_became_peered":"2024-09-16T15:46:48.910165+0000","last_unstale":"2024-09-16T15:46:49.890778+0000","last_undegraded":"2024-09-16T15:46:49.890778+0000","last_fullsized":"2024-09-16T15:46:49.890778+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:27.085988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926995+0000","last_change":"2024-09-16T15:46:48.912355+0000","last_active":"2024-09-16T15:46:50.926995+0000","last_peered":"2024-09-16T15:46:50.926995+0000","last_clean":"2024-09-16T15:46:50.926995+0000","last_became_active":"2024-09-16T15:46:48.910750+0000","last_became_peered":"2024-09-16T15:46:48.910750+0000","last_unstale":"2024-09-16T15:46:50.926995+0000","last_undegraded":"2024-09-16T15:46:50.926995+0000","last_fullsized":"2024-09-16T15:46:50.926995+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:51.818119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892214+0000","last_change":"2024-09-16T15:46:48.894641+0000","last_active":"2024-09-16T15:46:49.892214+0000","last_peered":"2024-09-16T15:46:49.892214+0000","last_clean":"2024-09-16T15:46:49.892214+0000","last_became_active":"2024-09-16T15:46:48.892781+0000","last_became_peered":"2024-09-16T15:46:48.892781+0000","last_unstale":"2024-09-16T15:46:49.892214+0000","last_undegraded":"2024-09-16T15:46:49.892214+0000","last_fullsized":"2024-09-16T15:46:49.892214+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:11.301209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893191+0000","last_change":"2024-09-16T15:46:48.891546+0000","last_active":"2024-09-16T15:46:49.893191+0000","last_peered":"2024-09-16T15:46:49.893191+0000","last_clean":"2024-09-16T15:46:49.893191+0000","last_became_active":"2024-09-16T15:46:48.891280+0000","last_became_peered":"2024-09-16T15:46:48.891280+0000","last_unstale":"2024-09-16T15:46:49.893191+0000","last_undegraded":"2024-09-16T15:46:49.893191+0000","last_fullsized":"2024-09-16T15:46:49.893191+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:07.510561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893177+0000","last_change":"2024-09-16T15:46:48.892079+0000","last_active":"2024-09-16T15:46:49.893177+0000","last_peered":"2024-09-16T15:46:49.893177+0000","last_clean":"2024-09-16T15:46:49.893177+0000","last_became_active":"2024-09-16T15:46:48.891759+0000","last_became_peered":"2024-09-16T15:46:48.891759+0000","last_unstale":"2024-09-16T15:46:49.893177+0000","last_undegraded":"2024-09-16T15:46:49.893177+0000","last_fullsized":"2024-09-16T15:46:49.893177+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:34:09.763706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893564+0000","last_change":"2024-09-16T15:46:48.892197+0000","last_active":"2024-09-16T15:46:49.893564+0000","last_peered":"2024-09-16T15:46:49.893564+0000","last_clean":"2024-09-16T15:46:49.893564+0000","last_became_active":"2024-09-16T15:46:48.891886+0000","last_became_peered":"2024-09-16T15:46:48.891886+0000","last_unstale":"2024-09-16T15:46:49.893564+0000","last_undegraded":"2024-09-16T15:46:49.893564+0000","last_fullsized":"2024-09-16T15:46:49.893564+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:30:16.160509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892936+0000","last_change":"2024-09-16T15:46:48.895518+0000","last_active":"2024-09-16T15:46:49.892936+0000","last_peered":"2024-09-16T15:46:49.892936+0000","last_clean":"2024-09-16T15:46:49.892936+0000","last_became_active":"2024-09-16T15:46:48.893174+0000","last_became_peered":"2024-09-16T15:46:48.893174+0000","last_unstale":"2024-09-16T15:46:49.892936+0000","last_undegraded":"2024-09-16T15:46:49.892936+0000","last_fullsized":"2024-09-16T15:46:49.892936+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:31.518240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897554+0000","last_change":"2024-09-16T15:46:48.892191+0000","last_active":"2024-09-16T15:46:49.897554+0000","last_peered":"2024-09-16T15:46:49.897554+0000","last_clean":"2024-09-16T15:46:49.897554+0000","last_became_active":"2024-09-16T15:46:48.890579+0000","last_became_peered":"2024-09-16T15:46:48.890579+0000","last_unstale":"2024-09-16T15:46:49.897554+0000","last_undegraded":"2024-09-16T15:46:49.897554+0000","last_fullsized":"2024-09-16T15:46:49.897554+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:41.959328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897407+0000","last_change":"2024-09-16T15:46:48.891945+0000","last_active":"2024-09-16T15:46:49.897407+0000","last_peered":"2024-09-16T15:46:49.897407+0000","last_clean":"2024-09-16T15:46:49.897407+0000","last_became_active":"2024-09-16T15:46:48.891603+0000","last_became_peered":"2024-09-16T15:46:48.891603+0000","last_unstale":"2024-09-16T15:46:49.897407+0000","last_undegraded":"2024-09-16T15:46:49.897407+0000","last_fullsized":"2024-09-16T15:46:49.897407+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:57.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925812+0000","last_change":"2024-09-16T15:46:48.901267+0000","last_active":"2024-09-16T15:46:50.925812+0000","last_peered":"2024-09-16T15:46:50.925812+0000","last_clean":"2024-09-16T15:46:50.925812+0000","last_became_active":"2024-09-16T15:46:48.899570+0000","last_became_peered":"2024-09-16T15:46:48.899570+0000","last_unstale":"2024-09-16T15:46:50.925812+0000","last_undegraded":"2024-09-16T15:46:50.925812+0000","last_fullsized":"2024-09-16T15:46:50.925812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:33.082360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925979+0000","last_change":"2024-09-16T15:46:48.902551+0000","last_active":"2024-09-16T15:46:50.925979+0000","last_peered":"2024-09-16T15:46:50.925979+0000","last_clean":"2024-09-16T15:46:50.925979+0000","last_became_active":"2024-09-16T15:46:48.902284+0000","last_became_peered":"2024-09-16T15:46:48.902284+0000","last_unstale":"2024-09-16T15:46:50.925979+0000","last_undegraded":"2024-09-16T15:46:50.925979+0000","last_fullsized":"2024-09-16T15:46:50.925979+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.418395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890972+0000","last_change":"2024-09-16T15:46:48.900865+0000","last_active":"2024-09-16T15:46:49.890972+0000","last_peered":"2024-09-16T15:46:49.890972+0000","last_clean":"2024-09-16T15:46:49.890972+0000","last_became_active":"2024-09-16T15:46:48.900520+0000","last_became_peered":"2024-09-16T15:46:48.900520+0000","last_unstale":"2024-09-16T15:46:49.890972+0000","last_undegraded":"2024-09-16T15:46:49.890972+0000","last_fullsized":"2024-09-16T15:46:49.890972+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:34.377608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927607+0000","last_change":"2024-09-16T15:46:48.895657+0000","last_active":"2024-09-16T15:46:50.927607+0000","last_peered":"2024-09-16T15:46:50.927607+0000","last_clean":"2024-09-16T15:46:50.927607+0000","last_became_active":"2024-09-16T15:46:48.895328+0000","last_became_peered":"2024-09-16T15:46:48.895328+0000","last_unstale":"2024-09-16T15:46:50.927607+0000","last_undegraded":"2024-09-16T15:46:50.927607+0000","last_fullsized":"2024-09-16T15:46:50.927607+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:56:41.551383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.891108+0000","last_change":"2024-09-16T15:46:48.900547+0000","last_active":"2024-09-16T15:46:49.891108+0000","last_peered":"2024-09-16T15:46:49.891108+0000","last_clean":"2024-09-16T15:46:49.891108+0000","last_became_active":"2024-09-16T15:46:48.900138+0000","last_became_peered":"2024-09-16T15:46:48.900138+0000","last_unstale":"2024-09-16T15:46:49.891108+0000","last_undegraded":"2024-09-16T15:46:49.891108+0000","last_fullsized":"2024-09-16T15:46:49.891108+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:39.066478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897488+0000","last_change":"2024-09-16T15:46:48.891116+0000","last_active":"2024-09-16T15:46:49.897488+0000","last_peered":"2024-09-16T15:46:49.897488+0000","last_clean":"2024-09-16T15:46:49.897488+0000","last_became_active":"2024-09-16T15:46:48.890807+0000","last_became_peered":"2024-09-16T15:46:48.890807+0000","last_unstale":"2024-09-16T15:46:49.897488+0000","last_undegraded":"2024-09-16T15:46:49.897488+0000","last_fullsized":"2024-09-16T15:46:49.897488+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:40.985244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890619+0000","last_change":"2024-09-16T15:46:48.891714+0000","last_active":"2024-09-16T15:46:49.890619+0000","last_peered":"2024-09-16T15:46:49.890619+0000","last_clean":"2024-09-16T15:46:49.890619+0000","last_became_active":"2024-09-16T15:46:48.890078+0000","last_became_peered":"2024-09-16T15:46:48.890078+0000","last_unstale":"2024-09-16T15:46:49.890619+0000","last_undegraded":"2024-09-16T15:46:49.890619+0000","last_fullsized":"2024-09-16T15:46:49.890619+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:39.713606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893718+0000","last_change":"2024-09-16T15:46:48.912110+0000","last_active":"2024-09-16T15:46:49.893718+0000","last_peered":"2024-09-16T15:46:49.893718+0000","last_clean":"2024-09-16T15:46:49.893718+0000","last_became_active":"2024-09-16T15:46:48.911800+0000","last_became_peered":"2024-09-16T15:46:48.911800+0000","last_unstale":"2024-09-16T15:46:49.893718+0000","last_undegraded":"2024-09-16T15:46:49.893718+0000","last_fullsized":"2024-09-16T15:46:49.893718+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:59.301801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890550+0000","last_change":"2024-09-16T15:46:48.893641+0000","last_active":"2024-09-16T15:46:49.890550+0000","last_peered":"2024-09-16T15:46:49.890550+0000","last_clean":"2024-09-16T15:46:49.890550+0000","last_became_active":"2024-09-16T15:46:48.893300+0000","last_became_peered":"2024-09-16T15:46:48.893300+0000","last_unstale":"2024-09-16T15:46:49.890550+0000","last_undegraded":"2024-09-16T15:46:49.890550+0000","last_fullsized":"2024-09-16T15:46:49.890550+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:30.000652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892430+0000","last_change":"2024-09-16T15:46:48.898630+0000","last_active":"2024-09-16T15:46:49.892430+0000","last_peered":"2024-09-16T15:46:49.892430+0000","last_clean":"2024-09-16T15:46:49.892430+0000","last_became_active":"2024-09-16T15:46:48.898307+0000","last_became_peered":"2024-09-16T15:46:48.898307+0000","last_unstale":"2024-09-16T15:46:49.892430+0000","last_undegraded":"2024-09-16T15:46:49.892430+0000","last_fullsized":"2024-09-16T15:46:49.892430+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:50.364555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.898127+0000","last_change":"2024-09-16T15:46:48.898293+0000","last_active":"2024-09-16T15:46:49.898127+0000","last_peered":"2024-09-16T15:46:49.898127+0000","last_clean":"2024-09-16T15:46:49.898127+0000","last_became_active":"2024-09-16T15:46:48.898035+0000","last_became_peered":"2024-09-16T15:46:48.898035+0000","last_unstale":"2024-09-16T15:46:49.898127+0000","last_undegraded":"2024-09-16T15:46:49.898127+0000","last_fullsized":"2024-09-16T15:46:49.898127+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:37:28.195593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927340+0000","last_change":"2024-09-16T15:46:48.904716+0000","last_active":"2024-09-16T15:46:50.927340+0000","last_peered":"2024-09-16T15:46:50.927340+0000","last_clean":"2024-09-16T15:46:50.927340+0000","last_became_active":"2024-09-16T15:46:48.904392+0000","last_became_peered":"2024-09-16T15:46:48.904392+0000","last_unstale":"2024-09-16T15:46:50.927340+0000","last_undegraded":"2024-09-16T15:46:50.927340+0000","last_fullsized":"2024-09-16T15:46:50.927340+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:08.303485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897845+0000","last_change":"2024-09-16T15:46:48.895242+0000","last_active":"2024-09-16T15:46:49.897845+0000","last_peered":"2024-09-16T15:46:49.897845+0000","last_clean":"2024-09-16T15:46:49.897845+0000","last_became_active":"2024-09-16T15:46:48.894898+0000","last_became_peered":"2024-09-16T15:46:48.894898+0000","last_unstale":"2024-09-16T15:46:49.897845+0000","last_undegraded":"2024-09-16T15:46:49.897845+0000","last_fullsized":"2024-09-16T15:46:49.897845+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:40.771736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926220+0000","last_change":"2024-09-16T15:46:48.898174+0000","last_active":"2024-09-16T15:46:50.926220+0000","last_peered":"2024-09-16T15:46:50.926220+0000","last_clean":"2024-09-16T15:46:50.926220+0000","last_became_active":"2024-09-16T15:46:48.897911+0000","last_became_peered":"2024-09-16T15:46:48.897911+0000","last_unstale":"2024-09-16T15:46:50.926220+0000","last_undegraded":"2024-09-16T15:46:50.926220+0000","last_fullsized":"2024-09-16T15:46:50.926220+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:45:16.429461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897590+0000","last_change":"2024-09-16T15:46:48.882660+0000","last_active":"2024-09-16T15:46:49.897590+0000","last_peered":"2024-09-16T15:46:49.897590+0000","last_clean":"2024-09-16T15:46:49.897590+0000","last_became_active":"2024-09-16T15:46:48.882342+0000","last_became_peered":"2024-09-16T15:46:48.882342+0000","last_unstale":"2024-09-16T15:46:49.897590+0000","last_undegraded":"2024-09-16T15:46:49.897590+0000","last_fullsized":"2024-09-16T15:46:49.897590+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:05:33.158335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.891930+0000","last_change":"2024-09-16T15:46:48.896621+0000","last_active":"2024-09-16T15:46:49.891930+0000","last_peered":"2024-09-16T15:46:49.891930+0000","last_clean":"2024-09-16T15:46:49.891930+0000","last_became_active":"2024-09-16T15:46:48.894218+0000","last_became_peered":"2024-09-16T15:46:48.894218+0000","last_unstale":"2024-09-16T15:46:49.891930+0000","last_undegraded":"2024-09-16T15:46:49.891930+0000","last_fullsized":"2024-09-16T15:46:49.891930+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:59.138850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892084+0000","last_change":"2024-09-16T15:46:48.911817+0000","last_active":"2024-09-16T15:46:49.892084+0000","last_peered":"2024-09-16T15:46:49.892084+0000","last_clean":"2024-09-16T15:46:49.892084+0000","last_became_active":"2024-09-16T15:46:48.910955+0000","last_became_peered":"2024-09-16T15:46:48.910955+0000","last_unstale":"2024-09-16T15:46:49.892084+0000","last_undegraded":"2024-09-16T15:46:49.892084+0000","last_fullsized":"2024-09-16T15:46:49.892084+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:00.288599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925925+0000","last_change":"2024-09-16T15:46:48.897561+0000","last_active":"2024-09-16T15:46:50.925925+0000","last_peered":"2024-09-16T15:46:50.925925+0000","last_clean":"2024-09-16T15:46:50.925925+0000","last_became_active":"2024-09-16T15:46:48.895591+0000","last_became_peered":"2024-09-16T15:46:48.895591+0000","last_unstale":"2024-09-16T15:46:50.925925+0000","last_undegraded":"2024-09-16T15:46:50.925925+0000","last_fullsized":"2024-09-16T15:46:50.925925+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:35.606013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927344+0000","last_change":"2024-09-16T15:46:48.903227+0000","last_active":"2024-09-16T15:46:50.927344+0000","last_peered":"2024-09-16T15:46:50.927344+0000","last_clean":"2024-09-16T15:46:50.927344+0000","last_became_active":"2024-09-16T15:46:48.902969+0000","last_became_peered":"2024-09-16T15:46:48.902969+0000","last_unstale":"2024-09-16T15:46:50.927344+0000","last_undegraded":"2024-09-16T15:46:50.927344+0000","last_fullsized":"2024-09-16T15:46:50.927344+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:56:41.551383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"148'4","reported_seq":19,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926101+0000","last_change":"2024-09-16T15:46:47.900622+0000","last_active":"2024-09-16T15:46:50.926101+0000","last_peered":"2024-09-16T15:46:50.926101+0000","last_clean":"2024-09-16T15:46:50.926101+0000","last_became_active":"2024-09-16T15:46:47.900325+0000","last_became_peered":"2024-09-16T15:46:47.900325+0000","last_unstale":"2024-09-16T15:46:50.926101+0000","last_undegraded":"2024-09-16T15:46:50.926101+0000","last_fullsized":"2024-09-16T15:46:50.926101+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:25.748554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893008+0000","last_change":"2024-09-16T15:46:48.903410+0000","last_active":"2024-09-16T15:46:49.893008+0000","last_peered":"2024-09-16T15:46:49.893008+0000","last_clean":"2024-09-16T15:46:49.893008+0000","last_became_active":"2024-09-16T15:46:48.903092+0000","last_became_peered":"2024-09-16T15:46:48.903092+0000","last_unstale":"2024-09-16T15:46:49.893008+0000","last_undegraded":"2024-09-16T15:46:49.893008+0000","last_fullsized":"2024-09-16T15:46:49.893008+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:50.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892291+0000","last_change":"2024-09-16T15:46:47.890007+0000","last_active":"2024-09-16T15:46:49.892291+0000","last_peered":"2024-09-16T15:46:49.892291+0000","last_clean":"2024-09-16T15:46:49.892291+0000","last_became_active":"2024-09-16T15:46:47.889684+0000","last_became_peered":"2024-09-16T15:46:47.889684+0000","last_unstale":"2024-09-16T15:46:49.892291+0000","last_undegraded":"2024-09-16T15:46:49.892291+0000","last_fullsized":"2024-09-16T15:46:49.892291+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:27.373954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890697+0000","last_change":"2024-09-16T15:46:48.892286+0000","last_active":"2024-09-16T15:46:49.890697+0000","last_peered":"2024-09-16T15:46:49.890697+0000","last_clean":"2024-09-16T15:46:49.890697+0000","last_became_active":"2024-09-16T15:46:48.890993+0000","last_became_peered":"2024-09-16T15:46:48.890993+0000","last_unstale":"2024-09-16T15:46:49.890697+0000","last_undegraded":"2024-09-16T15:46:49.890697+0000","last_fullsized":"2024-09-16T15:46:49.890697+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:42.006424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926156+0000","last_change":"2024-09-16T15:46:47.895466+0000","last_active":"2024-09-16T15:46:50.926156+0000","last_peered":"2024-09-16T15:46:50.926156+0000","last_clean":"2024-09-16T15:46:50.926156+0000","last_became_active":"2024-09-16T15:46:47.895174+0000","last_became_peered":"2024-09-16T15:46:47.895174+0000","last_unstale":"2024-09-16T15:46:50.926156+0000","last_undegraded":"2024-09-16T15:46:50.926156+0000","last_fullsized":"2024-09-16T15:46:50.926156+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:51.379730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.898020+0000","last_change":"2024-09-16T15:46:48.900075+0000","last_active":"2024-09-16T15:46:49.898020+0000","last_peered":"2024-09-16T15:46:49.898020+0000","last_clean":"2024-09-16T15:46:49.898020+0000","last_became_active":"2024-09-16T15:46:48.897114+0000","last_became_peered":"2024-09-16T15:46:48.897114+0000","last_unstale":"2024-09-16T15:46:49.898020+0000","last_undegraded":"2024-09-16T15:46:49.898020+0000","last_fullsized":"2024-09-16T15:46:49.898020+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:31:48.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926302+0000","last_change":"2024-09-16T15:46:47.896773+0000","last_active":"2024-09-16T15:46:50.926302+0000","last_peered":"2024-09-16T15:46:50.926302+0000","last_clean":"2024-09-16T15:46:50.926302+0000","last_became_active":"2024-09-16T15:46:47.894534+0000","last_became_peered":"2024-09-16T15:46:47.894534+0000","last_unstale":"2024-09-16T15:46:50.926302+0000","last_undegraded":"2024-09-16T15:46:50.926302+0000","last_fullsized":"2024-09-16T15:46:50.926302+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:30.340375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.928039+0000","last_change":"2024-09-16T15:46:48.907726+0000","last_active":"2024-09-16T15:46:50.928039+0000","last_peered":"2024-09-16T15:46:50.928039+0000","last_clean":"2024-09-16T15:46:50.928039+0000","last_became_active":"2024-09-16T15:46:48.901962+0000","last_became_peered":"2024-09-16T15:46:48.901962+0000","last_unstale":"2024-09-16T15:46:50.928039+0000","last_undegraded":"2024-09-16T15:46:50.928039+0000","last_fullsized":"2024-09-16T15:46:50.928039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:39.933120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897622+0000","last_change":"2024-09-16T15:46:47.887430+0000","last_active":"2024-09-16T15:46:49.897622+0000","last_peered":"2024-09-16T15:46:49.897622+0000","last_clean":"2024-09-16T15:46:49.897622+0000","last_became_active":"2024-09-16T15:46:47.887160+0000","last_became_peered":"2024-09-16T15:46:47.887160+0000","last_unstale":"2024-09-16T15:46:49.897622+0000","last_undegraded":"2024-09-16T15:46:49.897622+0000","last_fullsized":"2024-09-16T15:46:49.897622+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:28.992390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926559+0000","last_change":"2024-09-16T15:46:48.905413+0000","last_active":"2024-09-16T15:46:50.926559+0000","last_peered":"2024-09-16T15:46:50.926559+0000","last_clean":"2024-09-16T15:46:50.926559+0000","last_became_active":"2024-09-16T15:46:48.905113+0000","last_became_peered":"2024-09-16T15:46:48.905113+0000","last_unstale":"2024-09-16T15:46:50.926559+0000","last_undegraded":"2024-09-16T15:46:50.926559+0000","last_fullsized":"2024-09-16T15:46:50.926559+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:24.926201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925427+0000","last_change":"2024-09-16T15:46:47.891563+0000","last_active":"2024-09-16T15:46:50.925427+0000","last_peered":"2024-09-16T15:46:50.925427+0000","last_clean":"2024-09-16T15:46:50.925427+0000","last_became_active":"2024-09-16T15:46:47.890927+0000","last_became_peered":"2024-09-16T15:46:47.890927+0000","last_unstale":"2024-09-16T15:46:50.925427+0000","last_undegraded":"2024-09-16T15:46:50.925427+0000","last_fullsized":"2024-09-16T15:46:50.925427+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:25.694143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925517+0000","last_change":"2024-09-16T15:46:48.892896+0000","last_active":"2024-09-16T15:46:50.925517+0000","last_peered":"2024-09-16T15:46:50.925517+0000","last_clean":"2024-09-16T15:46:50.925517+0000","last_became_active":"2024-09-16T15:46:48.892531+0000","last_became_peered":"2024-09-16T15:46:48.892531+0000","last_unstale":"2024-09-16T15:46:50.925517+0000","last_undegraded":"2024-09-16T15:46:50.925517+0000","last_fullsized":"2024-09-16T15:46:50.925517+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:24.205491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.891937+0000","last_change":"2024-09-16T15:46:47.896064+0000","last_active":"2024-09-16T15:46:49.891937+0000","last_peered":"2024-09-16T15:46:49.891937+0000","last_clean":"2024-09-16T15:46:49.891937+0000","last_became_active":"2024-09-16T15:46:47.895101+0000","last_became_peered":"2024-09-16T15:46:47.895101+0000","last_unstale":"2024-09-16T15:46:49.891937+0000","last_undegraded":"2024-09-16T15:46:49.891937+0000","last_fullsized":"2024-09-16T15:46:49.891937+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:47.334242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890763+0000","last_change":"2024-09-16T15:46:48.890952+0000","last_active":"2024-09-16T15:46:49.890763+0000","last_peered":"2024-09-16T15:46:49.890763+0000","last_clean":"2024-09-16T15:46:49.890763+0000","last_became_active":"2024-09-16T15:46:48.890599+0000","last_became_peered":"2024-09-16T15:46:48.890599+0000","last_unstale":"2024-09-16T15:46:49.890763+0000","last_undegraded":"2024-09-16T15:46:49.890763+0000","last_fullsized":"2024-09-16T15:46:49.890763+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:55.504236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926240+0000","last_change":"2024-09-16T15:46:47.883227+0000","last_active":"2024-09-16T15:46:50.926240+0000","last_peered":"2024-09-16T15:46:50.926240+0000","last_clean":"2024-09-16T15:46:50.926240+0000","last_became_active":"2024-09-16T15:46:47.882952+0000","last_became_peered":"2024-09-16T15:46:47.882952+0000","last_unstale":"2024-09-16T15:46:50.926240+0000","last_undegraded":"2024-09-16T15:46:50.926240+0000","last_fullsized":"2024-09-16T15:46:50.926240+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:02.804132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.928188+0000","last_change":"2024-09-16T15:46:48.897117+0000","last_active":"2024-09-16T15:46:50.928188+0000","last_peered":"2024-09-16T15:46:50.928188+0000","last_clean":"2024-09-16T15:46:50.928188+0000","last_became_active":"2024-09-16T15:46:48.896715+0000","last_became_peered":"2024-09-16T15:46:48.896715+0000","last_unstale":"2024-09-16T15:46:50.928188+0000","last_undegraded":"2024-09-16T15:46:50.928188+0000","last_fullsized":"2024-09-16T15:46:50.928188+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:33:52.360259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890962+0000","last_change":"2024-09-16T15:46:47.891622+0000","last_active":"2024-09-16T15:46:49.890962+0000","last_peered":"2024-09-16T15:46:49.890962+0000","last_clean":"2024-09-16T15:46:49.890962+0000","last_became_active":"2024-09-16T15:46:47.889681+0000","last_became_peered":"2024-09-16T15:46:47.889681+0000","last_unstale":"2024-09-16T15:46:49.890962+0000","last_undegraded":"2024-09-16T15:46:49.890962+0000","last_fullsized":"2024-09-16T15:46:49.890962+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:48.411182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927579+0000","last_change":"2024-09-16T15:46:48.889512+0000","last_active":"2024-09-16T15:46:50.927579+0000","last_peered":"2024-09-16T15:46:50.927579+0000","last_clean":"2024-09-16T15:46:50.927579+0000","last_became_active":"2024-09-16T15:46:48.889246+0000","last_became_peered":"2024-09-16T15:46:48.889246+0000","last_unstale":"2024-09-16T15:46:50.927579+0000","last_undegraded":"2024-09-16T15:46:50.927579+0000","last_fullsized":"2024-09-16T15:46:50.927579+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:24.343279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"148'3","reported_seq":18,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.891093+0000","last_change":"2024-09-16T15:46:47.895328+0000","last_active":"2024-09-16T15:46:49.891093+0000","last_peered":"2024-09-16T15:46:49.891093+0000","last_clean":"2024-09-16T15:46:49.891093+0000","last_became_active":"2024-09-16T15:46:47.895042+0000","last_became_peered":"2024-09-16T15:46:47.895042+0000","last_unstale":"2024-09-16T15:46:49.891093+0000","last_undegraded":"2024-09-16T15:46:49.891093+0000","last_fullsized":"2024-09-16T15:46:49.891093+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:32.462718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893407+0000","last_change":"2024-09-16T15:46:48.900432+0000","last_active":"2024-09-16T15:46:49.893407+0000","last_peered":"2024-09-16T15:46:49.893407+0000","last_clean":"2024-09-16T15:46:49.893407+0000","last_became_active":"2024-09-16T15:46:48.900128+0000","last_became_peered":"2024-09-16T15:46:48.900128+0000","last_unstale":"2024-09-16T15:46:49.893407+0000","last_undegraded":"2024-09-16T15:46:49.893407+0000","last_fullsized":"2024-09-16T15:46:49.893407+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:08.899758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"148'6","reported_seq":21,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892855+0000","last_change":"2024-09-16T15:46:47.893045+0000","last_active":"2024-09-16T15:46:49.892855+0000","last_peered":"2024-09-16T15:46:49.892855+0000","last_clean":"2024-09-16T15:46:49.892855+0000","last_became_active":"2024-09-16T15:46:47.892764+0000","last_became_peered":"2024-09-16T15:46:47.892764+0000","last_unstale":"2024-09-16T15:46:49.892855+0000","last_undegraded":"2024-09-16T15:46:49.892855+0000","last_fullsized":"2024-09-16T15:46:49.892855+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:21.759260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926876+0000","last_change":"2024-09-16T15:46:48.885309+0000","last_active":"2024-09-16T15:46:50.926876+0000","last_peered":"2024-09-16T15:46:50.926876+0000","last_clean":"2024-09-16T15:46:50.926876+0000","last_became_active":"2024-09-16T15:46:48.885017+0000","last_became_peered":"2024-09-16T15:46:48.885017+0000","last_unstale":"2024-09-16T15:46:50.926876+0000","last_undegraded":"2024-09-16T15:46:50.926876+0000","last_fullsized":"2024-09-16T15:46:50.926876+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:12.267438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925720+0000","last_change":"2024-09-16T15:46:47.895591+0000","last_active":"2024-09-16T15:46:50.925720+0000","last_peered":"2024-09-16T15:46:50.925720+0000","last_clean":"2024-09-16T15:46:50.925720+0000","last_became_active":"2024-09-16T15:46:47.895309+0000","last_became_peered":"2024-09-16T15:46:47.895309+0000","last_unstale":"2024-09-16T15:46:50.925720+0000","last_undegraded":"2024-09-16T15:46:50.925720+0000","last_fullsized":"2024-09-16T15:46:50.925720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:18.519639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926988+0000","last_change":"2024-09-16T15:46:48.904929+0000","last_active":"2024-09-16T15:46:50.926988+0000","last_peered":"2024-09-16T15:46:50.926988+0000","last_clean":"2024-09-16T15:46:50.926988+0000","last_became_active":"2024-09-16T15:46:48.904665+0000","last_became_peered":"2024-09-16T15:46:48.904665+0000","last_unstale":"2024-09-16T15:46:50.926988+0000","last_undegraded":"2024-09-16T15:46:50.926988+0000","last_fullsized":"2024-09-16T15:46:50.926988+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:56.929174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927145+0000","last_change":"2024-09-16T15:46:47.885564+0000","last_active":"2024-09-16T15:46:50.927145+0000","last_peered":"2024-09-16T15:46:50.927145+0000","last_clean":"2024-09-16T15:46:50.927145+0000","last_became_active":"2024-09-16T15:46:47.885190+0000","last_became_peered":"2024-09-16T15:46:47.885190+0000","last_unstale":"2024-09-16T15:46:50.927145+0000","last_undegraded":"2024-09-16T15:46:50.927145+0000","last_fullsized":"2024-09-16T15:46:50.927145+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:30.807658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925863+0000","last_change":"2024-09-16T15:46:48.883784+0000","last_active":"2024-09-16T15:46:50.925863+0000","last_peered":"2024-09-16T15:46:50.925863+0000","last_clean":"2024-09-16T15:46:50.925863+0000","last_became_active":"2024-09-16T15:46:48.883499+0000","last_became_peered":"2024-09-16T15:46:48.883499+0000","last_unstale":"2024-09-16T15:46:50.925863+0000","last_undegraded":"2024-09-16T15:46:50.925863+0000","last_fullsized":"2024-09-16T15:46:50.925863+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:35.959607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925430+0000","last_change":"2024-09-16T15:46:47.891083+0000","last_active":"2024-09-16T15:46:50.925430+0000","last_peered":"2024-09-16T15:46:50.925430+0000","last_clean":"2024-09-16T15:46:50.925430+0000","last_became_active":"2024-09-16T15:46:47.890770+0000","last_became_peered":"2024-09-16T15:46:47.890770+0000","last_unstale":"2024-09-16T15:46:50.925430+0000","last_undegraded":"2024-09-16T15:46:50.925430+0000","last_fullsized":"2024-09-16T15:46:50.925430+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:20.995661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897279+0000","last_change":"2024-09-16T15:46:48.895825+0000","last_active":"2024-09-16T15:46:49.897279+0000","last_peered":"2024-09-16T15:46:49.897279+0000","last_clean":"2024-09-16T15:46:49.897279+0000","last_became_active":"2024-09-16T15:46:48.895548+0000","last_became_peered":"2024-09-16T15:46:48.895548+0000","last_unstale":"2024-09-16T15:46:49.897279+0000","last_undegraded":"2024-09-16T15:46:49.897279+0000","last_fullsized":"2024-09-16T15:46:49.897279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:52.233022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"148'8","reported_seq":23,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892184+0000","last_change":"2024-09-16T15:46:47.897314+0000","last_active":"2024-09-16T15:46:49.892184+0000","last_peered":"2024-09-16T15:46:49.892184+0000","last_clean":"2024-09-16T15:46:49.892184+0000","last_became_active":"2024-09-16T15:46:47.896498+0000","last_became_peered":"2024-09-16T15:46:47.896498+0000","last_unstale":"2024-09-16T15:46:49.892184+0000","last_undegraded":"2024-09-16T15:46:49.892184+0000","last_fullsized":"2024-09-16T15:46:49.892184+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:39.812717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927101+0000","last_change":"2024-09-16T15:46:48.901465+0000","last_active":"2024-09-16T15:46:50.927101+0000","last_peered":"2024-09-16T15:46:50.927101+0000","last_clean":"2024-09-16T15:46:50.927101+0000","last_became_active":"2024-09-16T15:46:48.901133+0000","last_became_peered":"2024-09-16T15:46:48.901133+0000","last_unstale":"2024-09-16T15:46:50.927101+0000","last_undegraded":"2024-09-16T15:46:50.927101+0000","last_fullsized":"2024-09-16T15:46:50.927101+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:33.674633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.898009+0000","last_change":"2024-09-16T15:46:47.891866+0000","last_active":"2024-09-16T15:46:49.898009+0000","last_peered":"2024-09-16T15:46:49.898009+0000","last_clean":"2024-09-16T15:46:49.898009+0000","last_became_active":"2024-09-16T15:46:47.891581+0000","last_became_peered":"2024-09-16T15:46:47.891581+0000","last_unstale":"2024-09-16T15:46:49.898009+0000","last_undegraded":"2024-09-16T15:46:49.898009+0000","last_fullsized":"2024-09-16T15:46:49.898009+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:30.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.928112+0000","last_change":"2024-09-16T15:46:48.897355+0000","last_active":"2024-09-16T15:46:50.928112+0000","last_peered":"2024-09-16T15:46:50.928112+0000","last_clean":"2024-09-16T15:46:50.928112+0000","last_became_active":"2024-09-16T15:46:48.897053+0000","last_became_peered":"2024-09-16T15:46:48.897053+0000","last_unstale":"2024-09-16T15:46:50.928112+0000","last_undegraded":"2024-09-16T15:46:50.928112+0000","last_fullsized":"2024-09-16T15:46:50.928112+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:26.756816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897720+0000","last_change":"2024-09-16T15:46:47.897237+0000","last_active":"2024-09-16T15:46:49.897720+0000","last_peered":"2024-09-16T15:46:49.897720+0000","last_clean":"2024-09-16T15:46:49.897720+0000","last_became_active":"2024-09-16T15:46:47.895746+0000","last_became_peered":"2024-09-16T15:46:47.895746+0000","last_unstale":"2024-09-16T15:46:49.897720+0000","last_undegraded":"2024-09-16T15:46:49.897720+0000","last_fullsized":"2024-09-16T15:46:49.897720+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:41.460497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926361+0000","last_change":"2024-09-16T15:46:48.906158+0000","last_active":"2024-09-16T15:46:50.926361+0000","last_peered":"2024-09-16T15:46:50.926361+0000","last_clean":"2024-09-16T15:46:50.926361+0000","last_became_active":"2024-09-16T15:46:48.905888+0000","last_became_peered":"2024-09-16T15:46:48.905888+0000","last_unstale":"2024-09-16T15:46:50.926361+0000","last_undegraded":"2024-09-16T15:46:50.926361+0000","last_fullsized":"2024-09-16T15:46:50.926361+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:25.044209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925818+0000","last_change":"2024-09-16T15:46:47.898515+0000","last_active":"2024-09-16T15:46:50.925818+0000","last_peered":"2024-09-16T15:46:50.925818+0000","last_clean":"2024-09-16T15:46:50.925818+0000","last_became_active":"2024-09-16T15:46:47.898272+0000","last_became_peered":"2024-09-16T15:46:47.898272+0000","last_unstale":"2024-09-16T15:46:50.925818+0000","last_undegraded":"2024-09-16T15:46:50.925818+0000","last_fullsized":"2024-09-16T15:46:50.925818+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:47.299272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926967+0000","last_change":"2024-09-16T15:46:48.915720+0000","last_active":"2024-09-16T15:46:50.926967+0000","last_peered":"2024-09-16T15:46:50.926967+0000","last_clean":"2024-09-16T15:46:50.926967+0000","last_became_active":"2024-09-16T15:46:48.915372+0000","last_became_peered":"2024-09-16T15:46:48.915372+0000","last_unstale":"2024-09-16T15:46:50.926967+0000","last_undegraded":"2024-09-16T15:46:50.926967+0000","last_fullsized":"2024-09-16T15:46:50.926967+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:07.863660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926040+0000","last_change":"2024-09-16T15:46:47.899653+0000","last_active":"2024-09-16T15:46:50.926040+0000","last_peered":"2024-09-16T15:46:50.926040+0000","last_clean":"2024-09-16T15:46:50.926040+0000","last_became_active":"2024-09-16T15:46:47.898882+0000","last_became_peered":"2024-09-16T15:46:47.898882+0000","last_unstale":"2024-09-16T15:46:50.926040+0000","last_undegraded":"2024-09-16T15:46:50.926040+0000","last_fullsized":"2024-09-16T15:46:50.926040+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:50.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897453+0000","last_change":"2024-09-16T15:46:48.889425+0000","last_active":"2024-09-16T15:46:49.897453+0000","last_peered":"2024-09-16T15:46:49.897453+0000","last_clean":"2024-09-16T15:46:49.897453+0000","last_became_active":"2024-09-16T15:46:48.888802+0000","last_became_peered":"2024-09-16T15:46:48.888802+0000","last_unstale":"2024-09-16T15:46:49.897453+0000","last_undegraded":"2024-09-16T15:46:49.897453+0000","last_fullsized":"2024-09-16T15:46:49.897453+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:37.504699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925839+0000","last_change":"2024-09-16T15:46:47.895959+0000","last_active":"2024-09-16T15:46:50.925839+0000","last_peered":"2024-09-16T15:46:50.925839+0000","last_clean":"2024-09-16T15:46:50.925839+0000","last_became_active":"2024-09-16T15:46:47.894748+0000","last_became_peered":"2024-09-16T15:46:47.894748+0000","last_unstale":"2024-09-16T15:46:50.925839+0000","last_undegraded":"2024-09-16T15:46:50.925839+0000","last_fullsized":"2024-09-16T15:46:50.925839+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:01.766307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927198+0000","last_change":"2024-09-16T15:46:48.909353+0000","last_active":"2024-09-16T15:46:50.927198+0000","last_peered":"2024-09-16T15:46:50.927198+0000","last_clean":"2024-09-16T15:46:50.927198+0000","last_became_active":"2024-09-16T15:46:48.909018+0000","last_became_peered":"2024-09-16T15:46:48.909018+0000","last_unstale":"2024-09-16T15:46:50.927198+0000","last_undegraded":"2024-09-16T15:46:50.927198+0000","last_fullsized":"2024-09-16T15:46:50.927198+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:40.601143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925749+0000","last_change":"2024-09-16T15:46:47.898112+0000","last_active":"2024-09-16T15:46:50.925749+0000","last_peered":"2024-09-16T15:46:50.925749+0000","last_clean":"2024-09-16T15:46:50.925749+0000","last_became_active":"2024-09-16T15:46:47.897822+0000","last_became_peered":"2024-09-16T15:46:47.897822+0000","last_unstale":"2024-09-16T15:46:50.925749+0000","last_undegraded":"2024-09-16T15:46:50.925749+0000","last_fullsized":"2024-09-16T15:46:50.925749+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:02:23.335017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925863+0000","last_change":"2024-09-16T15:46:48.897203+0000","last_active":"2024-09-16T15:46:50.925863+0000","last_peered":"2024-09-16T15:46:50.925863+0000","last_clean":"2024-09-16T15:46:50.925863+0000","last_became_active":"2024-09-16T15:46:48.895256+0000","last_became_peered":"2024-09-16T15:46:48.895256+0000","last_unstale":"2024-09-16T15:46:50.925863+0000","last_undegraded":"2024-09-16T15:46:50.925863+0000","last_fullsized":"2024-09-16T15:46:50.925863+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:56.233753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925490+0000","last_change":"2024-09-16T15:46:47.892309+0000","last_active":"2024-09-16T15:46:50.925490+0000","last_peered":"2024-09-16T15:46:50.925490+0000","last_clean":"2024-09-16T15:46:50.925490+0000","last_became_active":"2024-09-16T15:46:47.891709+0000","last_became_peered":"2024-09-16T15:46:47.891709+0000","last_unstale":"2024-09-16T15:46:50.925490+0000","last_undegraded":"2024-09-16T15:46:50.925490+0000","last_fullsized":"2024-09-16T15:46:50.925490+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:16.238494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890970+0000","last_change":"2024-09-16T15:46:48.887168+0000","last_active":"2024-09-16T15:46:49.890970+0000","last_peered":"2024-09-16T15:46:49.890970+0000","last_clean":"2024-09-16T15:46:49.890970+0000","last_became_active":"2024-09-16T15:46:48.886927+0000","last_became_peered":"2024-09-16T15:46:48.886927+0000","last_unstale":"2024-09-16T15:46:49.890970+0000","last_undegraded":"2024-09-16T15:46:49.890970+0000","last_fullsized":"2024-09-16T15:46:49.890970+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:47.777129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925876+0000","last_change":"2024-09-16T15:46:47.899580+0000","last_active":"2024-09-16T15:46:50.925876+0000","last_peered":"2024-09-16T15:46:50.925876+0000","last_clean":"2024-09-16T15:46:50.925876+0000","last_became_active":"2024-09-16T15:46:47.898928+0000","last_became_peered":"2024-09-16T15:46:47.898928+0000","last_unstale":"2024-09-16T15:46:50.925876+0000","last_undegraded":"2024-09-16T15:46:50.925876+0000","last_fullsized":"2024-09-16T15:46:50.925876+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:56:40.543121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927934+0000","last_change":"2024-09-16T15:46:48.886053+0000","last_active":"2024-09-16T15:46:50.927934+0000","last_peered":"2024-09-16T15:46:50.927934+0000","last_clean":"2024-09-16T15:46:50.927934+0000","last_became_active":"2024-09-16T15:46:48.885785+0000","last_became_peered":"2024-09-16T15:46:48.885785+0000","last_unstale":"2024-09-16T15:46:50.927934+0000","last_undegraded":"2024-09-16T15:46:50.927934+0000","last_fullsized":"2024-09-16T15:46:50.927934+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:51.033590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.928148+0000","last_change":"2024-09-16T15:46:47.892421+0000","last_active":"2024-09-16T15:46:50.928148+0000","last_peered":"2024-09-16T15:46:50.928148+0000","last_clean":"2024-09-16T15:46:50.928148+0000","last_became_active":"2024-09-16T15:46:47.892107+0000","last_became_peered":"2024-09-16T15:46:47.892107+0000","last_unstale":"2024-09-16T15:46:50.928148+0000","last_undegraded":"2024-09-16T15:46:50.928148+0000","last_fullsized":"2024-09-16T15:46:50.928148+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:26.857268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927552+0000","last_change":"2024-09-16T15:46:48.912341+0000","last_active":"2024-09-16T15:46:50.927552+0000","last_peered":"2024-09-16T15:46:50.927552+0000","last_clean":"2024-09-16T15:46:50.927552+0000","last_became_active":"2024-09-16T15:46:48.910886+0000","last_became_peered":"2024-09-16T15:46:48.910886+0000","last_unstale":"2024-09-16T15:46:50.927552+0000","last_undegraded":"2024-09-16T15:46:50.927552+0000","last_fullsized":"2024-09-16T15:46:50.927552+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:15.479426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"149'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897822+0000","last_change":"2024-09-16T15:46:47.888773+0000","last_active":"2024-09-16T15:46:49.897822+0000","last_peered":"2024-09-16T15:46:49.897822+0000","last_clean":"2024-09-16T15:46:49.897822+0000","last_became_active":"2024-09-16T15:46:47.888159+0000","last_became_peered":"2024-09-16T15:46:47.888159+0000","last_unstale":"2024-09-16T15:46:49.897822+0000","last_undegraded":"2024-09-16T15:46:49.897822+0000","last_fullsized":"2024-09-16T15:46:49.897822+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:40.998162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.890764+0000","last_change":"2024-09-16T15:46:48.897741+0000","last_active":"2024-09-16T15:46:49.890764+0000","last_peered":"2024-09-16T15:46:49.890764+0000","last_clean":"2024-09-16T15:46:49.890764+0000","last_became_active":"2024-09-16T15:46:48.896800+0000","last_became_peered":"2024-09-16T15:46:48.896800+0000","last_unstale":"2024-09-16T15:46:49.890764+0000","last_undegraded":"2024-09-16T15:46:49.890764+0000","last_fullsized":"2024-09-16T15:46:49.890764+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:25.528026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893437+0000","last_change":"2024-09-16T15:46:47.896061+0000","last_active":"2024-09-16T15:46:49.893437+0000","last_peered":"2024-09-16T15:46:49.893437+0000","last_clean":"2024-09-16T15:46:49.893437+0000","last_became_active":"2024-09-16T15:46:47.895517+0000","last_became_peered":"2024-09-16T15:46:47.895517+0000","last_unstale":"2024-09-16T15:46:49.893437+0000","last_undegraded":"2024-09-16T15:46:49.893437+0000","last_fullsized":"2024-09-16T15:46:49.893437+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:01.127898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927552+0000","last_change":"2024-09-16T15:46:48.892833+0000","last_active":"2024-09-16T15:46:50.927552+0000","last_peered":"2024-09-16T15:46:50.927552+0000","last_clean":"2024-09-16T15:46:50.927552+0000","last_became_active":"2024-09-16T15:46:48.892521+0000","last_became_peered":"2024-09-16T15:46:48.892521+0000","last_unstale":"2024-09-16T15:46:50.927552+0000","last_undegraded":"2024-09-16T15:46:50.927552+0000","last_fullsized":"2024-09-16T15:46:50.927552+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:12.880086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927395+0000","last_change":"2024-09-16T15:46:47.887865+0000","last_active":"2024-09-16T15:46:50.927395+0000","last_peered":"2024-09-16T15:46:50.927395+0000","last_clean":"2024-09-16T15:46:50.927395+0000","last_became_active":"2024-09-16T15:46:47.887538+0000","last_became_peered":"2024-09-16T15:46:47.887538+0000","last_unstale":"2024-09-16T15:46:50.927395+0000","last_undegraded":"2024-09-16T15:46:50.927395+0000","last_fullsized":"2024-09-16T15:46:50.927395+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:02.804132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927082+0000","last_change":"2024-09-16T15:46:48.898080+0000","last_active":"2024-09-16T15:46:50.927082+0000","last_peered":"2024-09-16T15:46:50.927082+0000","last_clean":"2024-09-16T15:46:50.927082+0000","last_became_active":"2024-09-16T15:46:48.896476+0000","last_became_peered":"2024-09-16T15:46:48.896476+0000","last_unstale":"2024-09-16T15:46:50.927082+0000","last_undegraded":"2024-09-16T15:46:50.927082+0000","last_fullsized":"2024-09-16T15:46:50.927082+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:51.818119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893243+0000","last_change":"2024-09-16T15:46:47.891693+0000","last_active":"2024-09-16T15:46:49.893243+0000","last_peered":"2024-09-16T15:46:49.893243+0000","last_clean":"2024-09-16T15:46:49.893243+0000","last_became_active":"2024-09-16T15:46:47.891395+0000","last_became_peered":"2024-09-16T15:46:47.891395+0000","last_unstale":"2024-09-16T15:46:49.893243+0000","last_undegraded":"2024-09-16T15:46:49.893243+0000","last_fullsized":"2024-09-16T15:46:49.893243+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:57.110332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893177+0000","last_change":"2024-09-16T15:46:48.912231+0000","last_active":"2024-09-16T15:46:49.893177+0000","last_peered":"2024-09-16T15:46:49.893177+0000","last_clean":"2024-09-16T15:46:49.893177+0000","last_became_active":"2024-09-16T15:46:48.911189+0000","last_became_peered":"2024-09-16T15:46:48.911189+0000","last_unstale":"2024-09-16T15:46:49.893177+0000","last_undegraded":"2024-09-16T15:46:49.893177+0000","last_fullsized":"2024-09-16T15:46:49.893177+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:04.918984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925564+0000","last_change":"2024-09-16T15:46:47.891906+0000","last_active":"2024-09-16T15:46:50.925564+0000","last_peered":"2024-09-16T15:46:50.925564+0000","last_clean":"2024-09-16T15:46:50.925564+0000","last_became_active":"2024-09-16T15:46:47.891304+0000","last_became_peered":"2024-09-16T15:46:47.891304+0000","last_unstale":"2024-09-16T15:46:50.925564+0000","last_undegraded":"2024-09-16T15:46:50.925564+0000","last_fullsized":"2024-09-16T15:46:50.925564+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:53:39.577659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.926182+0000","last_change":"2024-09-16T15:46:48.905195+0000","last_active":"2024-09-16T15:46:50.926182+0000","last_peered":"2024-09-16T15:46:50.926182+0000","last_clean":"2024-09-16T15:46:50.926182+0000","last_became_active":"2024-09-16T15:46:48.901594+0000","last_became_peered":"2024-09-16T15:46:48.901594+0000","last_unstale":"2024-09-16T15:46:50.926182+0000","last_undegraded":"2024-09-16T15:46:50.926182+0000","last_fullsized":"2024-09-16T15:46:50.926182+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:17.100042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893622+0000","last_change":"2024-09-16T15:46:47.897438+0000","last_active":"2024-09-16T15:46:49.893622+0000","last_peered":"2024-09-16T15:46:49.893622+0000","last_clean":"2024-09-16T15:46:49.893622+0000","last_became_active":"2024-09-16T15:46:47.896065+0000","last_became_peered":"2024-09-16T15:46:47.896065+0000","last_unstale":"2024-09-16T15:46:49.893622+0000","last_undegraded":"2024-09-16T15:46:49.893622+0000","last_fullsized":"2024-09-16T15:46:49.893622+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:57.512422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.897613+0000","last_change":"2024-09-16T15:46:48.894679+0000","last_active":"2024-09-16T15:46:49.897613+0000","last_peered":"2024-09-16T15:46:49.897613+0000","last_clean":"2024-09-16T15:46:49.897613+0000","last_became_active":"2024-09-16T15:46:48.894389+0000","last_became_peered":"2024-09-16T15:46:48.894389+0000","last_unstale":"2024-09-16T15:46:49.897613+0000","last_undegraded":"2024-09-16T15:46:49.897613+0000","last_fullsized":"2024-09-16T15:46:49.897613+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:57.876317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.892317+0000","last_change":"2024-09-16T15:46:47.885950+0000","last_active":"2024-09-16T15:46:49.892317+0000","last_peered":"2024-09-16T15:46:49.892317+0000","last_clean":"2024-09-16T15:46:49.892317+0000","last_became_active":"2024-09-16T15:46:47.885466+0000","last_became_peered":"2024-09-16T15:46:47.885466+0000","last_unstale":"2024-09-16T15:46:49.892317+0000","last_undegraded":"2024-09-16T15:46:49.892317+0000","last_fullsized":"2024-09-16T15:46:49.892317+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:48.207712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.927910+0000","last_change":"2024-09-16T15:46:48.898845+0000","last_active":"2024-09-16T15:46:50.927910+0000","last_peered":"2024-09-16T15:46:50.927910+0000","last_clean":"2024-09-16T15:46:50.927910+0000","last_became_active":"2024-09-16T15:46:48.898575+0000","last_became_peered":"2024-09-16T15:46:48.898575+0000","last_unstale":"2024-09-16T15:46:50.927910+0000","last_undegraded":"2024-09-16T15:46:50.927910+0000","last_fullsized":"2024-09-16T15:46:50.927910+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:47.861461+0000","last_clean_scrub_stamp":"2024-09-16T15:46:47.861461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:48.307534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.893078+0000","last_change":"2024-09-16T15:46:47.894952+0000","last_active":"2024-09-16T15:46:49.893078+0000","last_peered":"2024-09-16T15:46:49.893078+0000","last_clean":"2024-09-16T15:46:49.893078+0000","last_became_active":"2024-09-16T15:46:47.894645+0000","last_became_peered":"2024-09-16T15:46:47.894645+0000","last_unstale":"2024-09-16T15:46:49.893078+0000","last_undegraded":"2024-09-16T15:46:49.893078+0000","last_fullsized":"2024-09-16T15:46:49.893078+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:46.853199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:46.853199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:08.176340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":373,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-16T15:46:50.925935+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:46:50.925935+0000","last_peered":"2024-09-16T15:46:50.925935+0000","last_clean":"2024-09-16T15:46:50.925935+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:46:50.925935+0000","last_undegraded":"2024-09-16T15:46:50.925935+0000","last_fullsized":"2024-09-16T15:46:50.925935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33616,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338224,"statfs":{"total":96636764160,"available":96602341376,"internally_reserved":0,"allocated":2482176,"data_stored":1416324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1867776,"data_stored":823624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1871872,"data_stored":824206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509544,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1880064,"data_stored":825284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1867776,"data_stored":823636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1867776,"data_stored":823692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1867776,"data_stored":823636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37748,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334092,"statfs":{"total":96636764160,"available":96598110208,"internally_reserved":0,"allocated":2457600,"data_stored":1413970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:46:53.566 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:46:53.566 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:46:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.770+0000 7ffaa3200640 1 -- 172.21.15.60:0/3971598288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa40ce320 msgr2=0x7ffaa40ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.770+0000 7ffaa3200640 1 --2- 172.21.15.60:0/3971598288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa40ce320 0x7ffaa40ce770 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009830 tx=0x7ffa8c02a480 comp rx=0 tx=0).stop 2024-09-16T15:46:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.770+0000 7ffaa3200640 1 -- 172.21.15.60:0/3971598288 shutdown_connections 2024-09-16T15:46:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.770+0000 7ffaa3200640 1 --2- 172.21.15.60:0/3971598288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa40ce320 0x7ffaa40ce770 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.770+0000 7ffaa3200640 1 --2- 172.21.15.60:0/3971598288 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa40c48d0 0x7ffaa40c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.770+0000 7ffaa3200640 1 -- 172.21.15.60:0/3971598288 >> 172.21.15.60:0/3971598288 conn(0x7ffaa40cc490 msgr2=0x7ffaa40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.770+0000 7ffaa3200640 1 -- 172.21.15.60:0/3971598288 shutdown_connections 2024-09-16T15:46:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.771+0000 7ffaa3200640 1 -- 172.21.15.60:0/3971598288 wait complete. 2024-09-16T15:46:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.772+0000 7ffaa3200640 1 Processor -- start 2024-09-16T15:46:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.772+0000 7ffaa3200640 1 -- start start 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.773+0000 7ffaa3200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40c48d0 0x7ffaa41f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.773+0000 7ffaa3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa40ce320 0x7ffaa40b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.773+0000 7ffaa3200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa415aa30 0x7ffaa40b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.773+0000 7ffaa3200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa40c5420 con 0x7ffaa40c48d0 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.773+0000 7ffaa3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa40c52a0 con 0x7ffaa40ce320 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.773+0000 7ffaa3200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa40c55a0 con 0x7ffaa415aa30 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40c48d0 0x7ffaa41f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40c48d0 0x7ffaa41f2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56600/0 (socket says 172.21.15.60:56600) 2024-09-16T15:46:53.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 -- 172.21.15.60:0/1550104866 learned_addr learned my addr 172.21.15.60:0/1550104866 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:53.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 -- 172.21.15.60:0/1550104866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa415aa30 msgr2=0x7ffaa40b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa415aa30 0x7ffaa40b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 -- 172.21.15.60:0/1550104866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa40ce320 msgr2=0x7ffaa40b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa40ce320 0x7ffaa40b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 -- 172.21.15.60:0/1550104866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa40b2d80 con 0x7ffaa40c48d0 2024-09-16T15:46:53.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffaa1e00640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40c48d0 0x7ffaa41f2e20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400c430 tx=0x7ffa9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:53.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffa9aa00640 1 -- 172.21.15.60:0/1550104866 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94017070 con 0x7ffaa40c48d0 2024-09-16T15:46:53.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffa9aa00640 1 -- 172.21.15.60:0/1550104866 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa940053e0 con 0x7ffaa40c48d0 2024-09-16T15:46:53.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.774+0000 7ffa9aa00640 1 -- 172.21.15.60:0/1550104866 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94003850 con 0x7ffaa40c48d0 2024-09-16T15:46:53.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.775+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa40b3060 con 0x7ffaa40c48d0 2024-09-16T15:46:53.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.775+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa40b3620 con 0x7ffaa40c48d0 2024-09-16T15:46:53.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.776+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa40c4d60 con 0x7ffaa40c48d0 2024-09-16T15:46:53.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.777+0000 7ffa9aa00640 1 -- 172.21.15.60:0/1550104866 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7ffa9401a070 con 0x7ffaa40c48d0 2024-09-16T15:46:53.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.778+0000 7ffa9aa00640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa80078f40 0x7ffa8007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.778+0000 7ffa9aa00640 1 -- 172.21.15.60:0/1550104866 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7ffa9409a9f0 con 0x7ffaa40c48d0 2024-09-16T15:46:53.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.778+0000 7ffa99400640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa80078f40 0x7ffa8007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.779+0000 7ffa99400640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa80078f40 0x7ffa8007b400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c034000 tx=0x7ffa8c0046c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:53.782+0000 7ffa9aa00640 1 -- 172.21.15.60:0/1550104866 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffa94062560 con 0x7ffaa40c48d0 2024-09-16T15:46:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.054+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ffaa40c6810 con 0x7ffaa40c48d0 2024-09-16T15:46:54.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.057+0000 7ffa9aa00640 1 -- 172.21.15.60:0/1550104866 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7ffa94061e60 con 0x7ffaa40c48d0 2024-09-16T15:46:54.058 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:46:54.058 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:46:54.058 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:46:54.059 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:46:54.059 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:46:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:46:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.059+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa80078f40 msgr2=0x7ffa8007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.059+0000 7ffaa3200640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa80078f40 0x7ffa8007b400 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c034000 tx=0x7ffa8c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.059+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40c48d0 msgr2=0x7ffaa41f2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.059+0000 7ffaa3200640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40c48d0 0x7ffaa41f2e20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400c430 tx=0x7ffa9400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.060+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 shutdown_connections 2024-09-16T15:46:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.060+0000 7ffaa3200640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffa80078f40 0x7ffa8007b400 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.060+0000 7ffaa3200640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaa415aa30 0x7ffaa40b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.060+0000 7ffaa3200640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffaa40ce320 0x7ffaa40b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.060+0000 7ffaa3200640 1 --2- 172.21.15.60:0/1550104866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaa40c48d0 0x7ffaa41f2e20 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.060+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 >> 172.21.15.60:0/1550104866 conn(0x7ffaa40cc490 msgr2=0x7ffaa4152030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.060+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 shutdown_connections 2024-09-16T15:46:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.060+0000 7ffaa3200640 1 -- 172.21.15.60:0/1550104866 wait complete. 2024-09-16T15:46:54.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:54.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.236+0000 7f6703800640 1 -- 172.21.15.60:0/4251047857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66e0003a50 msgr2=0x7f66e0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.236+0000 7f6703800640 1 --2- 172.21.15.60:0/4251047857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66e0003a50 0x7f66e0003e70 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f66ec006fb0 tx=0x7f66e0005830 comp rx=0 tx=0).stop 2024-09-16T15:46:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.236+0000 7f6701800640 1 -- 172.21.15.60:0/4251047857 reap_dead start 2024-09-16T15:46:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.236+0000 7f6703800640 1 -- 172.21.15.60:0/4251047857 shutdown_connections 2024-09-16T15:46:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.236+0000 7f6703800640 1 -- 172.21.15.60:0/4251047857 >> 172.21.15.60:0/4251047857 conn(0x7f66fc0b1950 msgr2=0x7f66fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.236+0000 7f6703800640 1 -- 172.21.15.60:0/4251047857 shutdown_connections 2024-09-16T15:46:54.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.236+0000 7f6703800640 1 -- 172.21.15.60:0/4251047857 wait complete. 2024-09-16T15:46:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.238+0000 7f6703800640 1 Processor -- start 2024-09-16T15:46:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.238+0000 7f6703800640 1 -- start start 2024-09-16T15:46:54.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f6703800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66e0003a50 0x7f66fc1f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:54.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f6703800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc0b2d50 0x7f66fc15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:54.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f6703800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc158b80 0x7f66fc159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f6703800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc0745c0 con 0x7f66fc0b2d50 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f6703800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc074440 con 0x7f66fc158b80 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f6703800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f66fc074740 con 0x7f66e0003a50 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc0b2d50 0x7f66fc15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f66f3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc0b2d50 0x7f66fc15c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56616/0 (socket says 172.21.15.60:56616) 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f66f3e00640 1 -- 172.21.15.60:0/3505915769 learned_addr learned my addr 172.21.15.60:0/3505915769 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f6700e00640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66e0003a50 0x7f66fc1f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.239+0000 7f6701800640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc158b80 0x7f66fc159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f3e00640 1 -- 172.21.15.60:0/3505915769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66e0003a50 msgr2=0x7f66fc1f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f3e00640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66e0003a50 0x7f66fc1f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f3e00640 1 -- 172.21.15.60:0/3505915769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc158b80 msgr2=0x7f66fc159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f3e00640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc158b80 0x7f66fc159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f3e00640 1 -- 172.21.15.60:0/3505915769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc159540 con 0x7f66fc0b2d50 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f3e00640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc0b2d50 0x7f66fc15c6a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e970 tx=0x7f66e400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f9600640 1 -- 172.21.15.60:0/3505915769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e400ccc0 con 0x7f66fc0b2d50 2024-09-16T15:46:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f9600640 1 -- 172.21.15.60:0/3505915769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66e400ce60 con 0x7f66fc0b2d50 2024-09-16T15:46:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f66f9600640 1 -- 172.21.15.60:0/3505915769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4004270 con 0x7f66fc0b2d50 2024-09-16T15:46:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc0b31b0 con 0x7f66fc0b2d50 2024-09-16T15:46:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.240+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc0b3760 con 0x7f66fc0b2d50 2024-09-16T15:46:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.241+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66fc0b2440 con 0x7f66fc0b2d50 2024-09-16T15:46:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.243+0000 7f66f9600640 1 -- 172.21.15.60:0/3505915769 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f66e40035f0 con 0x7f66fc0b2d50 2024-09-16T15:46:54.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.244+0000 7f66f9600640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66d8078f40 0x7f66d807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:54.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.244+0000 7f66f9600640 1 -- 172.21.15.60:0/3505915769 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f66e409be50 con 0x7f66fc0b2d50 2024-09-16T15:46:54.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.244+0000 7f6700e00640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66d8078f40 0x7f66d807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:54.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.245+0000 7f6700e00640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66d8078f40 0x7f66d807b400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f66ec033000 tx=0x7f66ec038040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:54.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.248+0000 7f66f9600640 1 -- 172.21.15.60:0/3505915769 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66e4063b50 con 0x7f66fc0b2d50 2024-09-16T15:46:54.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.486+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66fc0afbd0 con 0x7f66fc0b2d50 2024-09-16T15:46:54.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.487+0000 7f66f9600640 1 -- 172.21.15.60:0/3505915769 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5229 (secure 0 0 0) 0x7f66e4063450 con 0x7f66fc0b2d50 2024-09-16T15:46:54.488 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:46:54.488 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":102,"btime":"2024-09-16T15:46:51:894437+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7133,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/822830116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":822830116},{"type":"v1","addr":"172.21.15.100:6834","nonce":822830116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/467713690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":467713690},{"type":"v1","addr":"172.21.15.100:6835","nonce":467713690}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":7981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2740321673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2740321673},{"type":"v1","addr":"172.21.15.60:6834","nonce":2740321673}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:48.320845+0000","modified":"2024-09-16T15:46:51.342321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2397719730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2397719730},{"type":"v1","addr":"172.21.15.60:6835","nonce":2397719730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8472,"qdb_cluster":[8472]},"id":11}]} 2024-09-16T15:46:54.488 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 102 2024-09-16T15:46:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66d8078f40 msgr2=0x7f66d807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66d8078f40 0x7f66d807b400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f66ec033000 tx=0x7f66ec038040 comp rx=0 tx=0).stop 2024-09-16T15:46:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc0b2d50 msgr2=0x7f66fc15c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc0b2d50 0x7f66fc15c6a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e970 tx=0x7f66e400ed70 comp rx=0 tx=0).stop 2024-09-16T15:46:54.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 shutdown_connections 2024-09-16T15:46:54.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66d8078f40 0x7f66d807b400 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66fc158b80 0x7f66fc159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66fc0b2d50 0x7f66fc15c6a0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 --2- 172.21.15.60:0/3505915769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66e0003a50 0x7f66fc1f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 >> 172.21.15.60:0/3505915769 conn(0x7f66fc0b1950 msgr2=0x7f66fc06f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:54.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 shutdown_connections 2024-09-16T15:46:54.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:54.490+0000 7f6703800640 1 -- 172.21.15.60:0/3505915769 wait complete. 2024-09-16T15:46:54.860 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-09-16T15:46:54.859+0000 7fe6d7e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:46:54.910 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:46:48.320845+0000', 'modified': '2024-09-16T15:46:51.342321+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8472': {'gid': 8472, 'name': 'c', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/2397719730', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 2397719730}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2397719730}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8472, 'qdb_cluster': [8472]} 2024-09-16T15:46:54.910 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:46:54.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:46:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.073+0000 7f1a35c00640 1 -- 172.21.15.60:0/1284404899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a3006fb90 msgr2=0x7f1a3006ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.073+0000 7f1a35c00640 1 --2- 172.21.15.60:0/1284404899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a3006fb90 0x7f1a3006ff70 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1a301588f0 tx=0x7f1a2002a560 comp rx=0 tx=0).stop 2024-09-16T15:46:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.074+0000 7f1a35c00640 1 -- 172.21.15.60:0/1284404899 shutdown_connections 2024-09-16T15:46:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.074+0000 7f1a35c00640 1 --2- 172.21.15.60:0/1284404899 >> v2:172.21.15.100:3300/0 conn(0x7f1a300cd5f0 0x7f1a300cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.074+0000 7f1a35c00640 1 --2- 172.21.15.60:0/1284404899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a300ad880 0x7f1a300adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.074+0000 7f1a35c00640 1 --2- 172.21.15.60:0/1284404899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a3006fb90 0x7f1a3006ff70 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.074+0000 7f1a35c00640 1 -- 172.21.15.60:0/1284404899 >> 172.21.15.60:0/1284404899 conn(0x7f1a300cfa20 msgr2=0x7f1a300cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.074+0000 7f1a35c00640 1 -- 172.21.15.60:0/1284404899 shutdown_connections 2024-09-16T15:46:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.074+0000 7f1a35c00640 1 -- 172.21.15.60:0/1284404899 wait complete. 2024-09-16T15:46:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.075+0000 7f1a35c00640 1 Processor -- start 2024-09-16T15:46:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.075+0000 7f1a35c00640 1 -- start start 2024-09-16T15:46:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.076+0000 7f1a35c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3006fb90 0x7f1a301f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.076+0000 7f1a35c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a300ad880 0x7f1a300b5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.076+0000 7f1a35c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a300cd5f0 0x7f1a300cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.076+0000 7f1a35c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3015eac0 con 0x7f1a300cd5f0 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.076+0000 7f1a35c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3015e940 con 0x7f1a3006fb90 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.076+0000 7f1a35c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3015ec40 con 0x7f1a300ad880 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a300cd5f0 0x7f1a300cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a300cd5f0 0x7f1a300cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56636/0 (socket says 172.21.15.60:56636) 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 -- 172.21.15.60:0/926024705 learned_addr learned my addr 172.21.15.60:0/926024705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3006fb90 0x7f1a301f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a27400640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a300ad880 0x7f1a300b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 -- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a300ad880 msgr2=0x7f1a300b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a300ad880 0x7f1a300b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 -- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3006fb90 msgr2=0x7f1a301f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3006fb90 0x7f1a301f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 -- 172.21.15.60:0/926024705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a301f51e0 con 0x7f1a300cd5f0 2024-09-16T15:46:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a27400640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a300ad880 0x7f1a300b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35200640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a300cd5f0 0x7f1a300cbef0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e960 tx=0x7f1a2800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a2cc00640 1 -- 172.21.15.60:0/926024705 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28004d20 con 0x7f1a300cd5f0 2024-09-16T15:46:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a2cc00640 1 -- 172.21.15.60:0/926024705 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a2800cb20 con 0x7f1a300cd5f0 2024-09-16T15:46:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a2cc00640 1 -- 172.21.15.60:0/926024705 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28005020 con 0x7f1a300cd5f0 2024-09-16T15:46:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.077+0000 7f1a35c00640 1 -- 172.21.15.60:0/926024705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301f57c0 con 0x7f1a300cd5f0 2024-09-16T15:46:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.078+0000 7f1a35c00640 1 -- 172.21.15.60:0/926024705 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3006eea0 con 0x7f1a300cd5f0 2024-09-16T15:46:55.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.080+0000 7f1a2cc00640 1 -- 172.21.15.60:0/926024705 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f1a280051c0 con 0x7f1a300cd5f0 2024-09-16T15:46:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.081+0000 7f1a2cc00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1a0c078f10 0x7f1a0c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.081+0000 7f1a2cc00640 1 -- 172.21.15.60:0/926024705 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f1a2809bfa0 con 0x7f1a300cd5f0 2024-09-16T15:46:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.081+0000 7f1a3a1f7740 1 Processor -- start 2024-09-16T15:46:55.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.081+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1a0c078f10 0x7f1a0c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.081+0000 7f1a3a1f7740 1 -- start start 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.082+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1a0c078f10 0x7f1a0c07b3d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1a20033000 tx=0x7f1a20004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.082+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed0daa0 0x56221ed0de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.082+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc8c10 0x56221ed0c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.082+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ed0cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.082+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56221ec4b430 con 0x56221ecc9190 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.082+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56221ec3b5d0 con 0x56221ed0daa0 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.082+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x56221ec2aa00 con 0x56221ecc8c10 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ed0cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ed0cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56648/0 (socket says 172.21.15.60:56648) 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 -- 172.21.15.60:0/4062649504 learned_addr learned my addr 172.21.15.60:0/4062649504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a27400640 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc8c10 0x56221ed0c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed0daa0 0x56221ed0de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 -- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc8c10 msgr2=0x56221ed0c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc8c10 0x56221ed0c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 -- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed0daa0 msgr2=0x56221ed0de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed0daa0 0x56221ed0de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 -- 172.21.15.60:0/4062649504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221ebf3520 con 0x56221ecc9190 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed0daa0 0x56221ed0de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.083+0000 7f1a35200640 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ed0cac0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1a28063110 tx=0x7f1a28062e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d9c0d912a18650 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a24c00640 1 -- 172.21.15.60:0/4062649504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28089070 con 0x56221ecc9190 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a24c00640 1 -- 172.21.15.60:0/4062649504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a28063410 con 0x56221ecc9190 2024-09-16T15:46:55.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a24c00640 1 -- 172.21.15.60:0/4062649504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2801f040 con 0x56221ecc9190 2024-09-16T15:46:55.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/4062649504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 msgr2=0x56221ed0cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ed0cac0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1a28063110 tx=0x7f1a28062e20 comp rx=0 tx=0).stop 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/4062649504 shutdown_connections 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ed0cac0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc8c10 0x56221ed0c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/4062649504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed0daa0 0x56221ed0de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/4062649504 >> 172.21.15.60:0/4062649504 conn(0x56221ecc41e0 msgr2=0x56221ecc7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/4062649504 shutdown_connections 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.084+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/4062649504 wait complete. 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.085+0000 7f1a3a1f7740 1 -- start start 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.085+0000 7f1a3a1f7740 1 Processor -- start 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.086+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 0x56221ed1f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.086+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221ecf2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.086+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ed20150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.086+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56221ebd1330 con 0x56221ed0daa0 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.086+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56221ec2e810 con 0x56221ed05510 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.086+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x56221e99b620 con 0x56221ecc9190 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.086+0000 7f1a3a1f7740 10 client.0 _ll_register_callbacks cb 0x56221ebd6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221ecf2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221ecf2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56656/0 (socket says 172.21.15.60:56656) 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 -- 172.21.15.60:0/2938075612 learned_addr learned my addr 172.21.15.60:0/2938075612 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 0x56221ed1f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a35200640 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ed20150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 -- 172.21.15.60:0/2938075612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 msgr2=0x56221ed1f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 0x56221ed1f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 -- 172.21.15.60:0/2938075612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 msgr2=0x56221ed20150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ed20150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 -- 172.21.15.60:0/2938075612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221e98c850 con 0x56221ed0daa0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a27400640 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221ecf2010 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009de0 tx=0x7f1a180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a682a066f9fe34e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.087+0000 7f1a1d600640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a1d600640 1 -- 172.21.15.60:0/2938075612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18012860 con 0x56221ed0daa0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a1d600640 1 -- 172.21.15.60:0/2938075612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f1a18012a00 con 0x56221ed0daa0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/2938075612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a28063410 con 0x56221ed0daa0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a1d600640 1 -- 172.21.15.60:0/2938075612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f1a18024050 con 0x56221ed0daa0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 10 client.8589 fetch_fsmap learned FSMap version 102 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/2938075612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x56221ea036b0 con 0x56221ed0daa0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a1d600640 1 -- 172.21.15.60:0/2938075612 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a28063410 con 0x56221ed0daa0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a1d600640 1 -- 172.21.15.60:0/2938075612 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 102) ==== 3502+0+0 (secure 0 0 0) 0x7f1a18027020 con 0x56221ed0daa0 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 10 client.8589 fetch_fsmap finished waiting for FSMap version 102 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 10 client.8589 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 20 client.8589 populate_metadata read hostname 'smithi060' 2024-09-16T15:46:55.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] conn(0x56221ed1e5f0 0x56221ed1e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 4 client.8589 mds_command: new command op to 8472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:46:55.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.088+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/2938075612 --> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56221ec3d520 con 0x56221ed1e5f0 2024-09-16T15:46:55.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.089+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] conn(0x56221ed1e5f0 0x56221ed1e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.089+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] conn(0x56221ed1e5f0 0x56221ed1e9d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73386562aa307467 server_cookie=b4661089dffa2c20 in_seq=0 out_seq=0 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.089+0000 7f1a1d600640 10 client.8589 ms_handle_connect on v2:172.21.15.60:6833/2397719730 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a1d600640 1 -- 172.21.15.60:0/2938075612 <== mds.0 v2:172.21.15.60:6833/2397719730 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56221ec3d520 con 0x56221ed1e5f0 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a1d600640 10 client.8589 handle_command_reply: tid=0 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 client.8589 shutdown 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/2938075612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 msgr2=0x56221ecf2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221ecf2010 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009de0 tx=0x7f1a180117b0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/2938075612 shutdown_connections 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] conn(0x56221ed1e5f0 0x56221ed1e9d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ed20150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221ecf2010 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/2938075612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 0x56221ed1f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a1d600640 0 client.8589 ms_handle_reset on v2:172.21.15.60:6833/2397719730 2024-09-16T15:46:55.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/2938075612 >> 172.21.15.60:0/2938075612 conn(0x56221ecc8c10 msgr2=0x56221ecc2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.090+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/2938075612 shutdown_connections 2024-09-16T15:46:55.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.091+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/2938075612 wait complete. 2024-09-16T15:46:55.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.091+0000 7f1a3a1f7740 20 client.8589 trim_cache size 0 max 16384 2024-09-16T15:46:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.110+0000 7f1a3a1f7740 1 Processor -- start 2024-09-16T15:46:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.110+0000 7f1a3a1f7740 1 -- start start 2024-09-16T15:46:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.110+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ecc2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.110+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ecc7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ed0daa0 0x56221ed0c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1a302010c0 con 0x56221ecc9190 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30200100 con 0x56221ed05510 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1a301f6f10 con 0x56221ed0daa0 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ecc2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ecc2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56668/0 (socket says 172.21.15.60:56668) 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 -- 172.21.15.60:0/1868336988 learned_addr learned my addr 172.21.15.60:0/1868336988 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 -- 172.21.15.60:0/1868336988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ed0daa0 msgr2=0x56221ed0c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/1868336988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ed0daa0 0x56221ed0c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 -- 172.21.15.60:0/1868336988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 msgr2=0x56221ecc7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/1868336988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ecc7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 -- 172.21.15.60:0/1868336988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221ecc25c0 con 0x56221ecc9190 2024-09-16T15:46:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.111+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/1868336988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ecc2d20 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f1a200096c0 tx=0x7f1a2003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fffbaa87c7186dd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a1fe00640 1 -- 172.21.15.60:0/1868336988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20035050 con 0x56221ecc9190 2024-09-16T15:46:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a1fe00640 1 -- 172.21.15.60:0/1868336988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a200523f0 con 0x56221ecc9190 2024-09-16T15:46:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a1fe00640 1 -- 172.21.15.60:0/1868336988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a200527a0 con 0x56221ecc9190 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/1868336988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 msgr2=0x56221ecc2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/1868336988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ecc2d20 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f1a200096c0 tx=0x7f1a2003c000 comp rx=0 tx=0).stop 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/1868336988 shutdown_connections 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/1868336988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ed0daa0 0x56221ed0c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/1868336988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ecc7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/1868336988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ecc9190 0x56221ecc2d20 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/1868336988 >> 172.21.15.60:0/1868336988 conn(0x56221ecc8c10 msgr2=0x56221ecf1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/1868336988 shutdown_connections 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.112+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/1868336988 wait complete. 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.113+0000 7f1a3a1f7740 1 -- start start 2024-09-16T15:46:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.113+0000 7f1a3a1f7740 1 Processor -- start 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 0x56221ed8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ececcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a3a1f7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221eced200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x56221ecf25f0 con 0x56221ed0daa0 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a301f7560 con 0x56221ed05510 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a3a1f7740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x56221ed1d470 con 0x56221ecc9190 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a3a1f7740 10 client.0 _ll_register_callbacks cb 0x7f1a18009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a35200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221eced200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a2fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 0x56221ed8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a35200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221eced200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56674/0 (socket says 172.21.15.60:56674) 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.114+0000 7f1a35200640 1 -- 172.21.15.60:0/543326243 learned_addr learned my addr 172.21.15.60:0/543326243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a27400640 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ececcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a35200640 1 -- 172.21.15.60:0/543326243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 msgr2=0x56221ed8b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a35200640 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 0x56221ed8b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a35200640 1 -- 172.21.15.60:0/543326243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 msgr2=0x56221ececcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a35200640 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ececcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a35200640 1 -- 172.21.15.60:0/543326243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221ecc6cc0 con 0x56221ed0daa0 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a35200640 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221eced200 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f1a2805f990 tx=0x7f1a2806a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c13fcc3782d214 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a1d600640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a1d600640 1 -- 172.21.15.60:0/543326243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2804e020 con 0x56221ed0daa0 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a1d600640 1 -- 172.21.15.60:0/543326243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f1a2805ef70 con 0x56221ed0daa0 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.115+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/543326243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a200523f0 con 0x56221ed0daa0 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a1d600640 1 -- 172.21.15.60:0/543326243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f1a28082050 con 0x56221ed0daa0 2024-09-16T15:46:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a3a1f7740 10 client.8595 fetch_fsmap learned FSMap version 102 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/543326243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x56221ed2f600 con 0x56221ed0daa0 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a1d600640 1 -- 172.21.15.60:0/543326243 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a200523f0 con 0x56221ed0daa0 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a1d600640 1 -- 172.21.15.60:0/543326243 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 102) ==== 3502+0+0 (secure 0 0 0) 0x7f1a28085020 con 0x56221ed0daa0 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a3a1f7740 10 client.8595 fetch_fsmap finished waiting for FSMap version 102 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a3a1f7740 10 client.8595 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a3a1f7740 20 client.8595 populate_metadata read hostname 'smithi060' 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] conn(0x56221ed8eb50 0x56221ed0c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a3a1f7740 4 client.8595 mds_command: new command op to 8472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/543326243 --> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] -- command(tid 0: {"prefix": "status"}) -- 0x56221ec38fd0 con 0x56221ed8eb50 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.116+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] conn(0x56221ed8eb50 0x56221ed0c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a1d600640 10 client.8595 ms_handle_connect on v2:172.21.15.60:6833/2397719730 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a2fe00640 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] conn(0x56221ed8eb50 0x56221ed0c370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151eaad7cc4a41c1 server_cookie=1b74da4f3b7e274c in_seq=0 out_seq=0 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a1d600640 1 -- 172.21.15.60:0/543326243 <== mds.0 v2:172.21.15.60:6833/2397719730 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x56221ec38fd0 con 0x56221ed8eb50 2024-09-16T15:46:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a1d600640 10 client.8595 handle_command_reply: tid=0 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 client.8595 shutdown 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/543326243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 msgr2=0x56221eced200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221eced200 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f1a2805f990 tx=0x7f1a2806a330 comp rx=0 tx=0).stop 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/543326243 shutdown_connections 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.60:6833/2397719730,v1:172.21.15.60:6835/2397719730] conn(0x56221ed8eb50 0x56221ed0c370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56221ed0daa0 0x56221eced200 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56221ed05510 0x56221ececcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 --2- 172.21.15.60:0/543326243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56221ecc9190 0x56221ed8b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a1d600640 0 client.8595 ms_handle_reset on v2:172.21.15.60:6833/2397719730 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/543326243 >> 172.21.15.60:0/543326243 conn(0x56221ecc8c10 msgr2=0x56221ecc7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/543326243 shutdown_connections 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.117+0000 7f1a3a1f7740 1 -- 172.21.15.60:0/543326243 wait complete. 2024-09-16T15:46:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a3a1f7740 20 client.8595 trim_cache size 0 max 16384 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8472, 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2334685580000002, 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 102, 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 149, 2024-09-16T15:46:55.123 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 149, 2024-09-16T15:46:55.124 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.508776086999999 2024-09-16T15:46:55.124 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:46:55.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 -- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1a0c078f10 msgr2=0x7f1a0c07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1a0c078f10 0x7f1a0c07b3d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1a20033000 tx=0x7f1a20004730 comp rx=0 tx=0).stop 2024-09-16T15:46:55.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 -- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a300cd5f0 msgr2=0x7f1a300cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a300cd5f0 0x7f1a300cbef0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e960 tx=0x7f1a2800ee20 comp rx=0 tx=0).stop 2024-09-16T15:46:55.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 -- 172.21.15.60:0/926024705 shutdown_connections 2024-09-16T15:46:55.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1a0c078f10 0x7f1a0c07b3d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a300cd5f0 0x7f1a300cbef0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a300ad880 0x7f1a300b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 --2- 172.21.15.60:0/926024705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a3006fb90 0x7f1a301f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 -- 172.21.15.60:0/926024705 >> 172.21.15.60:0/926024705 conn(0x7f1a300cfa20 msgr2=0x7f1a30074150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 -- 172.21.15.60:0/926024705 shutdown_connections 2024-09-16T15:46:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.118+0000 7f1a35c00640 1 -- 172.21.15.60:0/926024705 wait complete. 2024-09-16T15:46:55.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:46:55.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.293+0000 7faf67e00640 1 -- 172.21.15.60:0/3417276612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf60073fb0 msgr2=0x7faf60074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.293+0000 7faf67e00640 1 --2- 172.21.15.60:0/3417276612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf60073fb0 0x7faf60074420 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7faf600cfbb0 tx=0x7faf5802aa30 comp rx=0 tx=0).stop 2024-09-16T15:46:55.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.293+0000 7faf67e00640 1 -- 172.21.15.60:0/3417276612 shutdown_connections 2024-09-16T15:46:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.293+0000 7faf67e00640 1 --2- 172.21.15.60:0/3417276612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf60073fb0 0x7faf60074420 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.293+0000 7faf67e00640 1 --2- 172.21.15.60:0/3417276612 >> v2:172.21.15.100:3300/0 conn(0x7faf600ccf00 0x7faf600b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.293+0000 7faf67e00640 1 -- 172.21.15.60:0/3417276612 >> 172.21.15.60:0/3417276612 conn(0x7faf600b1950 msgr2=0x7faf600b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.293+0000 7faf67e00640 1 -- 172.21.15.60:0/3417276612 shutdown_connections 2024-09-16T15:46:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.293+0000 7faf67e00640 1 -- 172.21.15.60:0/3417276612 wait complete. 2024-09-16T15:46:55.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.295+0000 7faf67e00640 1 Processor -- start 2024-09-16T15:46:55.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.295+0000 7faf67e00640 1 -- start start 2024-09-16T15:46:55.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf67e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf60073fb0 0x7faf601465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf67e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf600ccf00 0x7faf6014e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf67e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf6015f5a0 0x7faf6014ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf67e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faf6006f2e0 con 0x7faf600ccf00 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf67e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faf6006f160 con 0x7faf60073fb0 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf67e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7faf6006f460 con 0x7faf6015f5a0 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf64a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf600ccf00 0x7faf6014e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf64a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf600ccf00 0x7faf6014e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56678/0 (socket says 172.21.15.60:56678) 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf64a00640 1 -- 172.21.15.60:0/1994155273 learned_addr learned my addr 172.21.15.60:0/1994155273 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf65400640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf60073fb0 0x7faf601465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.296+0000 7faf65e00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf6015f5a0 0x7faf6014ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf64a00640 1 -- 172.21.15.60:0/1994155273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf6015f5a0 msgr2=0x7faf6014ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf64a00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf6015f5a0 0x7faf6014ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf64a00640 1 -- 172.21.15.60:0/1994155273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf60073fb0 msgr2=0x7faf601465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf64a00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf60073fb0 0x7faf601465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf64a00640 1 -- 172.21.15.60:0/1994155273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf600ce920 con 0x7faf600ccf00 2024-09-16T15:46:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf64a00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf600ccf00 0x7faf6014e530 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00d770 tx=0x7faf4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf5d600640 1 -- 172.21.15.60:0/1994155273 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c017070 con 0x7faf600ccf00 2024-09-16T15:46:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf5d600640 1 -- 172.21.15.60:0/1994155273 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf4c009d20 con 0x7faf600ccf00 2024-09-16T15:46:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf5d600640 1 -- 172.21.15.60:0/1994155273 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c005280 con 0x7faf600ccf00 2024-09-16T15:46:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf600cebc0 con 0x7faf600ccf00 2024-09-16T15:46:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.297+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf600cf180 con 0x7faf600ccf00 2024-09-16T15:46:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.298+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6006f9f0 con 0x7faf600ccf00 2024-09-16T15:46:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.300+0000 7faf5d600640 1 -- 172.21.15.60:0/1994155273 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7faf4c005420 con 0x7faf600ccf00 2024-09-16T15:46:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.301+0000 7faf5d600640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faf40078f10 0x7faf4007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.301+0000 7faf5d600640 1 -- 172.21.15.60:0/1994155273 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7faf4c09b820 con 0x7faf600ccf00 2024-09-16T15:46:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.301+0000 7faf65400640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faf40078f10 0x7faf4007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.301+0000 7faf65400640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faf40078f10 0x7faf4007b3d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf54001720 tx=0x7faf54009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.305+0000 7faf5d600640 1 -- 172.21.15.60:0/1994155273 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faf4c062cb0 con 0x7faf600ccf00 2024-09-16T15:46:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.538+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faf6014d2a0 con 0x7faf600ccf00 2024-09-16T15:46:55.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.538+0000 7faf5d600640 1 -- 172.21.15.60:0/1994155273 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7faf4c0676e0 con 0x7faf600ccf00 2024-09-16T15:46:55.539 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.540+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faf40078f10 msgr2=0x7faf4007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.540+0000 7faf67e00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faf40078f10 0x7faf4007b3d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf54001720 tx=0x7faf54009040 comp rx=0 tx=0).stop 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.540+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf600ccf00 msgr2=0x7faf6014e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.540+0000 7faf67e00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf600ccf00 0x7faf6014e530 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00d770 tx=0x7faf4c00dc30 comp rx=0 tx=0).stop 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.541+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 shutdown_connections 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.541+0000 7faf67e00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faf40078f10 0x7faf4007b3d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.541+0000 7faf67e00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf6015f5a0 0x7faf6014ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.541+0000 7faf67e00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf600ccf00 0x7faf6014e530 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.541+0000 7faf67e00640 1 --2- 172.21.15.60:0/1994155273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf60073fb0 0x7faf601465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.541+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 >> 172.21.15.60:0/1994155273 conn(0x7faf600b1950 msgr2=0x7faf6015cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.541+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 shutdown_connections 2024-09-16T15:46:55.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55.541+0000 7faf67e00640 1 -- 172.21.15.60:0/1994155273 wait complete. 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:46:55.558 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:46:55.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:55.558 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:46:55.594 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:46:55.594 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:46:55.594 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:46:55.594 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:55.594 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:46:55.594 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42234sec preferred_lft 42234sec 2024-09-16T15:46:55.595 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:46:55.596 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:46:55.597 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:55.597 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:55.597 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:46:55.597 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:46:55.597 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:55.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:55.597 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:46:55.689 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:46:55.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:55.692 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:46:55.728 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:46:55.728 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:46:55.728 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:46:55.728 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:46:55.728 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:46:55.729 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:46:55.729 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:46:55.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:46:55.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:46:55.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:46:55.729 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:46:55.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:46:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:55.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:55.928 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:46:55.964 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:46:55.998 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:46:55.998 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:46:55.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:55.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:46:56.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:46:56.059 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:56.094 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:46:56.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:46:56.095 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:56.170 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:46:56.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:56.170 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:56.298 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:56.298 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:46:56.325 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:56.325 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:46:56.325 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:46:56.325 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:46:56.325 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:46:56.328 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:46:56.328 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:46:56.328 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:46:56.328 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:46:56.328 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:46:56.331 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:46:56.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:56.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:56.387 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:46:56.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:46:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:56.485 INFO:teuthology.orchestra.run.smithi060.stdout:client8604 2024-09-16T15:46:56.485 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:46:56.485 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:56.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:46:56.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.721+0000 7fc724e00640 1 -- 172.21.15.60:0/2208080144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7200ccf00 msgr2=0x7fc7200b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.721+0000 7fc724e00640 1 --2- 172.21.15.60:0/2208080144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7200ccf00 0x7fc7200b2440 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc72006dac0 tx=0x7fc70802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.721+0000 7fc724e00640 1 -- 172.21.15.60:0/2208080144 shutdown_connections 2024-09-16T15:46:56.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.721+0000 7fc724e00640 1 --2- 172.21.15.60:0/2208080144 >> v2:172.21.15.100:3300/0 conn(0x7fc720073fb0 0x7fc720074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.721+0000 7fc724e00640 1 --2- 172.21.15.60:0/2208080144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7200ccf00 0x7fc7200b2440 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.721+0000 7fc724e00640 1 -- 172.21.15.60:0/2208080144 >> 172.21.15.60:0/2208080144 conn(0x7fc7200b1950 msgr2=0x7fc7200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:56.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.721+0000 7fc724e00640 1 -- 172.21.15.60:0/2208080144 shutdown_connections 2024-09-16T15:46:56.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.721+0000 7fc724e00640 1 -- 172.21.15.60:0/2208080144 wait complete. 2024-09-16T15:46:56.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.723+0000 7fc724e00640 1 Processor -- start 2024-09-16T15:46:56.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.723+0000 7fc724e00640 1 -- start start 2024-09-16T15:46:56.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc724e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc720073fb0 0x7fc720156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc724e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7200ccf00 0x7fc72014c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc724e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72015f5a0 0x7fc7200cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc724e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc72006f2e0 con 0x7fc72015f5a0 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc724e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc72006f160 con 0x7fc720073fb0 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc724e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc72006f460 con 0x7fc7200ccf00 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72015f5a0 0x7fc7200cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72015f5a0 0x7fc7200cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56684/0 (socket says 172.21.15.60:56684) 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71e800640 1 -- 172.21.15.60:0/2658606939 learned_addr learned my addr 172.21.15.60:0/2658606939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:56.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc715400640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7200ccf00 0x7fc72014c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71de00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc720073fb0 0x7fc720156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71e800640 1 -- 172.21.15.60:0/2658606939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7200ccf00 msgr2=0x7fc72014c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71e800640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7200ccf00 0x7fc72014c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71e800640 1 -- 172.21.15.60:0/2658606939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc720073fb0 msgr2=0x7fc720156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71e800640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc720073fb0 0x7fc720156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.724+0000 7fc71e800640 1 -- 172.21.15.60:0/2658606939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7201f6a60 con 0x7fc72015f5a0 2024-09-16T15:46:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.725+0000 7fc71e800640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72015f5a0 0x7fc7200cfcb0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc71800eaa0 tx=0x7fc71800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:56.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.725+0000 7fc716a00640 1 -- 172.21.15.60:0/2658606939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71800cdb0 con 0x7fc72015f5a0 2024-09-16T15:46:56.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.725+0000 7fc716a00640 1 -- 172.21.15.60:0/2658606939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc718004c10 con 0x7fc72015f5a0 2024-09-16T15:46:56.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.725+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7201f6d40 con 0x7fc72015f5a0 2024-09-16T15:46:56.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.725+0000 7fc716a00640 1 -- 172.21.15.60:0/2658606939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718005020 con 0x7fc72015f5a0 2024-09-16T15:46:56.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.725+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc720156410 con 0x7fc72015f5a0 2024-09-16T15:46:56.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.726+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72006f9f0 con 0x7fc72015f5a0 2024-09-16T15:46:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.727+0000 7fc716a00640 1 -- 172.21.15.60:0/2658606939 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fc7180051c0 con 0x7fc72015f5a0 2024-09-16T15:46:56.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.728+0000 7fc716a00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc6fc078f10 0x7fc6fc07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.729+0000 7fc716a00640 1 -- 172.21.15.60:0/2658606939 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7fc71809c060 con 0x7fc72015f5a0 2024-09-16T15:46:56.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.729+0000 7fc71de00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc6fc078f10 0x7fc6fc07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.729+0000 7fc71de00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc6fc078f10 0x7fc6fc07b3d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc710000f60 tx=0x7fc710009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:56.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.733+0000 7fc716a00640 1 -- 172.21.15.60:0/2658606939 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc718016030 con 0x7fc72015f5a0 2024-09-16T15:46:56.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.970+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc720155ce0 con 0x7fc72015f5a0 2024-09-16T15:46:56.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.971+0000 7fc716a00640 1 -- 172.21.15.60:0/2658606939 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7fc7180634f0 con 0x7fc72015f5a0 2024-09-16T15:46:56.972 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:46:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.973+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc6fc078f10 msgr2=0x7fc6fc07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.973+0000 7fc724e00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc6fc078f10 0x7fc6fc07b3d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc710000f60 tx=0x7fc710009040 comp rx=0 tx=0).stop 2024-09-16T15:46:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.973+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72015f5a0 msgr2=0x7fc7200cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.973+0000 7fc724e00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72015f5a0 0x7fc7200cfcb0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc71800eaa0 tx=0x7fc71800ef60 comp rx=0 tx=0).stop 2024-09-16T15:46:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.974+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 shutdown_connections 2024-09-16T15:46:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.974+0000 7fc724e00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc6fc078f10 0x7fc6fc07b3d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.974+0000 7fc724e00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc72015f5a0 0x7fc7200cfcb0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.974+0000 7fc724e00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7200ccf00 0x7fc72014c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.974+0000 7fc724e00640 1 --2- 172.21.15.60:0/2658606939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc720073fb0 0x7fc720156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.974+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 >> 172.21.15.60:0/2658606939 conn(0x7fc7200b1950 msgr2=0x7fc7200b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:56.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.974+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 shutdown_connections 2024-09-16T15:46:56.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:56.974+0000 7fc724e00640 1 -- 172.21.15.60:0/2658606939 wait complete. 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:46:56.991 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:46:56.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:56.992 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:57.033 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42232sec preferred_lft 42232sec 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:57.034 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:57.035 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:46:57.035 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:46:57.035 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:57.035 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:57.035 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:46:57.035 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:46:57.035 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:57.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:57.035 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:46:57.131 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:46:57.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:57.135 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:46:57.172 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:46:57.172 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:46:57.172 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:46:57.172 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:46:57.172 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:46:57.173 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:46:57.173 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:46:57.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:46:57.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:46:57.173 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:46:57.173 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:46:57.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:46:57.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:57.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:57.369 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:46:57.405 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:46:57.405 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:46:57.405 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:46:57.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:57.405 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:46:57.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:46:57.501 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:57.536 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:46:57.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:46:57.536 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:57.611 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:46:57.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:57.611 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:57.740 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:57.740 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:46:57.768 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:46:57.768 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:46:57.768 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:46:57.768 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:46:57.768 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:46:57.770 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:46:57.770 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:46:57.771 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:46:57.771 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:46:57.771 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:46:57.773 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:46:57.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:57.774 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:46:57.828 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:46:57.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:46:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:46:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:57.916 INFO:teuthology.orchestra.run.smithi100.stdout:client7214 2024-09-16T15:46:57.916 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:46:57.916 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:57.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.125+0000 7f85c5400640 1 -- 172.21.15.60:0/1968634241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 msgr2=0x7f85c00cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.125+0000 7f85c5400640 1 --2- 172.21.15.60:0/1968634241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 0x7f85c00cdad0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f85c0072750 tx=0x7f85b002a560 comp rx=0 tx=0).stop 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.125+0000 7f85c5400640 1 -- 172.21.15.60:0/1968634241 shutdown_connections 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.125+0000 7f85c5400640 1 --2- 172.21.15.60:0/1968634241 >> v2:172.21.15.100:3300/0 conn(0x7f85c0074290 0x7f85c0074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.125+0000 7f85c5400640 1 --2- 172.21.15.60:0/1968634241 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c006e910 0x7f85c0073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.125+0000 7f85c5400640 1 --2- 172.21.15.60:0/1968634241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 0x7f85c00cdad0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.125+0000 7f85c5400640 1 -- 172.21.15.60:0/1968634241 >> 172.21.15.60:0/1968634241 conn(0x7f85c00b18e0 msgr2=0x7f85c00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.126+0000 7f85c5400640 1 -- 172.21.15.60:0/1968634241 shutdown_connections 2024-09-16T15:46:58.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.126+0000 7f85c5400640 1 -- 172.21.15.60:0/1968634241 wait complete. 2024-09-16T15:46:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.127+0000 7f85c5400640 1 Processor -- start 2024-09-16T15:46:58.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.127+0000 7f85c5400640 1 -- start start 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.128+0000 7f85c5400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c006e910 0x7f85c015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.128+0000 7f85c5400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c0074290 0x7f85c015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.128+0000 7f85c5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 0x7f85c0153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.128+0000 7f85c5400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f85c015fd00 con 0x7f85c015bd60 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.128+0000 7f85c5400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85c015fb80 con 0x7f85c0074290 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.128+0000 7f85c5400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f85c015fe80 con 0x7f85c006e910 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85bf200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 0x7f85c0153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c0074290 0x7f85c015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85be800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c006e910 0x7f85c015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c0074290 0x7f85c015abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:46806/0 (socket says 172.21.15.60:46806) 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 -- 172.21.15.60:0/808538732 learned_addr learned my addr 172.21.15.60:0/808538732 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 -- 172.21.15.60:0/808538732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c006e910 msgr2=0x7f85c015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c006e910 0x7f85c015ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 -- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 msgr2=0x7f85c0153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 0x7f85c0153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 -- 172.21.15.60:0/808538732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c01f5260 con 0x7f85c0074290 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85bf200640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 0x7f85c0153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.129+0000 7f85b5e00640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c0074290 0x7f85c015abf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f85a800e9f0 tx=0x7f85a800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.130+0000 7f85b7400640 1 -- 172.21.15.60:0/808538732 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a800ccc0 con 0x7f85c0074290 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.130+0000 7f85b7400640 1 -- 172.21.15.60:0/808538732 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85a800ce60 con 0x7f85c0074290 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.130+0000 7f85b7400640 1 -- 172.21.15.60:0/808538732 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8005260 con 0x7f85c0074290 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.130+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c01f5540 con 0x7f85c0074290 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.130+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c01f5b00 con 0x7f85c0074290 2024-09-16T15:46:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.131+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c0069360 con 0x7f85c0074290 2024-09-16T15:46:58.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.132+0000 7f85b7400640 1 -- 172.21.15.60:0/808538732 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f85a8005400 con 0x7f85c0074290 2024-09-16T15:46:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.133+0000 7f85b7400640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f859c078f10 0x7f859c07b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.133+0000 7f85b7400640 1 -- 172.21.15.60:0/808538732 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f85a809be90 con 0x7f85c0074290 2024-09-16T15:46:58.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.133+0000 7f85be800640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f859c078f10 0x7f859c07b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.134+0000 7f85be800640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f859c078f10 0x7f859c07b3d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85b0033000 tx=0x7f85b0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:58.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.137+0000 7f85b7400640 1 -- 172.21.15.60:0/808538732 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f85a8063540 con 0x7f85c0074290 2024-09-16T15:46:58.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.374+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f85c01603a0 con 0x7f85c0074290 2024-09-16T15:46:58.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.607+0000 7f85b7400640 1 -- 172.21.15.60:0/808538732 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v103) ==== 136+0+0 (secure 0 0 0) 0x7f85a8067f70 con 0x7f85c0074290 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f859c078f10 msgr2=0x7f859c07b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f859c078f10 0x7f859c07b3d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85b0033000 tx=0x7f85b0004730 comp rx=0 tx=0).stop 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c0074290 msgr2=0x7f85c015abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c0074290 0x7f85c015abf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f85a800e9f0 tx=0x7f85a800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 shutdown_connections 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f859c078f10 0x7f859c07b3d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c015bd60 0x7f85c0153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c0074290 0x7f85c015abf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 --2- 172.21.15.60:0/808538732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c006e910 0x7f85c015ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 >> 172.21.15.60:0/808538732 conn(0x7f85c00b18e0 msgr2=0x7f85c00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 shutdown_connections 2024-09-16T15:46:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.610+0000 7f85c5400640 1 -- 172.21.15.60:0/808538732 wait complete. 2024-09-16T15:46:58.628 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:46:58.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.790+0000 7fb05a800640 1 -- 172.21.15.60:0/3209839553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 msgr2=0x7fb038004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.790+0000 7fb05a800640 1 --2- 172.21.15.60:0/3209839553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 0x7fb038004c80 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb03c006f80 tx=0x7fb038005830 comp rx=0 tx=0).stop 2024-09-16T15:46:58.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.790+0000 7fb05a800640 1 -- 172.21.15.60:0/3209839553 shutdown_connections 2024-09-16T15:46:58.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.790+0000 7fb05a800640 1 --2- 172.21.15.60:0/3209839553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 0x7fb038004c80 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.790+0000 7fb05a800640 1 -- 172.21.15.60:0/3209839553 >> 172.21.15.60:0/3209839553 conn(0x7fb0540b1950 msgr2=0x7fb0540b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:58.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.790+0000 7fb05a800640 1 -- 172.21.15.60:0/3209839553 shutdown_connections 2024-09-16T15:46:58.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.790+0000 7fb05a800640 1 -- 172.21.15.60:0/3209839553 wait complete. 2024-09-16T15:46:58.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.791+0000 7fb05a800640 1 Processor -- start 2024-09-16T15:46:58.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.792+0000 7fb05a800640 1 -- start start 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb05a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 0x7fb0541f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb05a800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb05409d010 0x7fb05409d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb05a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0540cf9e0 0x7fb0540cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb05a800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb05406f0d0 con 0x7fb038004800 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb05a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb05406ef50 con 0x7fb0540cf9e0 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb05a800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb05406f250 con 0x7fb05409d010 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 0x7fb0541f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 0x7fb0541f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47100/0 (socket says 172.21.15.60:47100) 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 -- 172.21.15.60:0/750951799 learned_addr learned my addr 172.21.15.60:0/750951799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:58.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb04aa00640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb05409d010 0x7fb05409d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053e00640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0540cf9e0 0x7fb0540cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 -- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb05409d010 msgr2=0x7fb05409d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb05409d010 0x7fb05409d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 -- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0540cf9e0 msgr2=0x7fb0540cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0540cf9e0 0x7fb0540cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 -- 172.21.15.60:0/750951799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0540cdc10 con 0x7fb038004800 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053e00640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0540cf9e0 0x7fb0540cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.793+0000 7fb053400640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 0x7fb0541f5460 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fb044003e60 tx=0x7fb044001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.794+0000 7fb04be00640 1 -- 172.21.15.60:0/750951799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044015070 con 0x7fb038004800 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.794+0000 7fb04be00640 1 -- 172.21.15.60:0/750951799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb044011840 con 0x7fb038004800 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.794+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0540cdeb0 con 0x7fb038004800 2024-09-16T15:46:58.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.794+0000 7fb04be00640 1 -- 172.21.15.60:0/750951799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044011c90 con 0x7fb038004800 2024-09-16T15:46:58.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.794+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0540ce470 con 0x7fb038004800 2024-09-16T15:46:58.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.795+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05406f7b0 con 0x7fb038004800 2024-09-16T15:46:58.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.796+0000 7fb04be00640 1 -- 172.21.15.60:0/750951799 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7fb044018070 con 0x7fb038004800 2024-09-16T15:46:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.797+0000 7fb04be00640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb030078f10 0x7fb03007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.797+0000 7fb04be00640 1 -- 172.21.15.60:0/750951799 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7fb04409c360 con 0x7fb038004800 2024-09-16T15:46:58.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.797+0000 7fb04aa00640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb030078f10 0x7fb03007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.798+0000 7fb04aa00640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb030078f10 0x7fb03007b3d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb03c033000 tx=0x7fb03c001930 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:58.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:58.801+0000 7fb04be00640 1 -- 172.21.15.60:0/750951799 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb044063ed0 con 0x7fb038004800 2024-09-16T15:46:59.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.019+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb0540cf7c0 con 0x7fb030078f10 2024-09-16T15:46:59.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.020+0000 7fb04be00640 1 -- 172.21.15.60:0/750951799 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb0540cf7c0 con 0x7fb030078f10 2024-09-16T15:46:59.021 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:46:59.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:46:59.022 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:46:59.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:46:59.022 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:46:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.022+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb030078f10 msgr2=0x7fb03007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.022+0000 7fb05a800640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb030078f10 0x7fb03007b3d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb03c033000 tx=0x7fb03c001930 comp rx=0 tx=0).stop 2024-09-16T15:46:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.022+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 msgr2=0x7fb0541f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.022+0000 7fb05a800640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 0x7fb0541f5460 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fb044003e60 tx=0x7fb044001730 comp rx=0 tx=0).stop 2024-09-16T15:46:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.023+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 shutdown_connections 2024-09-16T15:46:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.023+0000 7fb05a800640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb030078f10 0x7fb03007b3d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.023+0000 7fb05a800640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0540cf9e0 0x7fb0540cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.023+0000 7fb05a800640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb05409d010 0x7fb05409d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.023+0000 7fb05a800640 1 --2- 172.21.15.60:0/750951799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb038004800 0x7fb0541f5460 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.023+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 >> 172.21.15.60:0/750951799 conn(0x7fb0540b1950 msgr2=0x7fb05415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.023+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 shutdown_connections 2024-09-16T15:46:59.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.023+0000 7fb05a800640 1 -- 172.21.15.60:0/750951799 wait complete. 2024-09-16T15:46:59.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:46:59.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.203+0000 7f8029600640 1 -- 172.21.15.60:0/620128658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80240cec60 msgr2=0x7f80240cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.203+0000 7f8029600640 1 --2- 172.21.15.60:0/620128658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80240cec60 0x7f80240cf070 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f801c005fd0 tx=0x7f801c02a940 comp rx=0 tx=0).stop 2024-09-16T15:46:59.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.203+0000 7f8029600640 1 -- 172.21.15.60:0/620128658 shutdown_connections 2024-09-16T15:46:59.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.203+0000 7f8029600640 1 --2- 172.21.15.60:0/620128658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80240cec60 0x7f80240cf070 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.203+0000 7f8029600640 1 -- 172.21.15.60:0/620128658 >> 172.21.15.60:0/620128658 conn(0x7f80240b1840 msgr2=0x7f80240b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.203+0000 7f8029600640 1 -- 172.21.15.60:0/620128658 shutdown_connections 2024-09-16T15:46:59.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.203+0000 7f8029600640 1 -- 172.21.15.60:0/620128658 wait complete. 2024-09-16T15:46:59.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.205+0000 7f8029600640 1 Processor -- start 2024-09-16T15:46:59.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.205+0000 7f8029600640 1 -- start start 2024-09-16T15:46:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8029600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240af190 0x7f80241f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8029600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80240cec60 0x7f80241f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8029600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8024161760 0x7f80240cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8029600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f80241623a0 con 0x7f8024161760 2024-09-16T15:46:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8029600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8024070530 con 0x7f80240af190 2024-09-16T15:46:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8029600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8024162520 con 0x7f80240cec60 2024-09-16T15:46:59.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8023400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8024161760 0x7f80240cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8023400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8024161760 0x7f80240cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47108/0 (socket says 172.21.15.60:47108) 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8023400640 1 -- 172.21.15.60:0/1925931283 learned_addr learned my addr 172.21.15.60:0/1925931283 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8022a00640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240af190 0x7f80241f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f801a000640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80240cec60 0x7f80241f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8023400640 1 -- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80240cec60 msgr2=0x7f80241f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8023400640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80240cec60 0x7f80241f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8023400640 1 -- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240af190 msgr2=0x7f80241f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8023400640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240af190 0x7f80241f83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f8023400640 1 -- 172.21.15.60:0/1925931283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80241f5680 con 0x7f8024161760 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.206+0000 7f801a000640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80240cec60 0x7f80241f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.207+0000 7f8023400640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8024161760 0x7f80240cc1f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f801c006010 tx=0x7f801c031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.207+0000 7f801b400640 1 -- 172.21.15.60:0/1925931283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c030430 con 0x7f8024161760 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.207+0000 7f801b400640 1 -- 172.21.15.60:0/1925931283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f801c004240 con 0x7f8024161760 2024-09-16T15:46:59.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.207+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80241f5960 con 0x7f8024161760 2024-09-16T15:46:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.207+0000 7f801b400640 1 -- 172.21.15.60:0/1925931283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c004650 con 0x7f8024161760 2024-09-16T15:46:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.207+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80241f8f20 con 0x7f8024161760 2024-09-16T15:46:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.208+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802415ed50 con 0x7f8024161760 2024-09-16T15:46:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.209+0000 7f801b400640 1 -- 172.21.15.60:0/1925931283 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f801c039440 con 0x7f8024161760 2024-09-16T15:46:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.210+0000 7f801b400640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8000078f10 0x7f800007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.210+0000 7f801b400640 1 -- 172.21.15.60:0/1925931283 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f801c0bce10 con 0x7f8024161760 2024-09-16T15:46:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.210+0000 7f8022a00640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8000078f10 0x7f800007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.211+0000 7f8022a00640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8000078f10 0x7f800007b3d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8014007fd0 tx=0x7f8014007b00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.214+0000 7f801b400640 1 -- 172.21.15.60:0/1925931283 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f801c084a80 con 0x7f8024161760 2024-09-16T15:46:59.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.418+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f80241629d0 con 0x7f8024161760 2024-09-16T15:46:59.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.418+0000 7f801b400640 1 -- 172.21.15.60:0/1925931283 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f801c084380 con 0x7f8024161760 2024-09-16T15:46:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8000078f10 msgr2=0x7f800007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8000078f10 0x7f800007b3d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8014007fd0 tx=0x7f8014007b00 comp rx=0 tx=0).stop 2024-09-16T15:46:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8024161760 msgr2=0x7f80240cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8024161760 0x7f80240cc1f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f801c006010 tx=0x7f801c031e20 comp rx=0 tx=0).stop 2024-09-16T15:46:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 shutdown_connections 2024-09-16T15:46:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8000078f10 0x7f800007b3d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8024161760 0x7f80240cc1f0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80240cec60 0x7f80241f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 --2- 172.21.15.60:0/1925931283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80240af190 0x7f80241f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 >> 172.21.15.60:0/1925931283 conn(0x7f80240b1840 msgr2=0x7f80240731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 shutdown_connections 2024-09-16T15:46:59.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.421+0000 7f8029600640 1 -- 172.21.15.60:0/1925931283 wait complete. 2024-09-16T15:46:59.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:59.438 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol41180) 2024-09-16T15:46:59.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:59.502 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp41840/subvol2771) 2024-09-16T15:46:59.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41180 2024-09-16T15:46:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.725+0000 7f3af6a00640 1 -- 172.21.15.60:0/739095512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00b5480 msgr2=0x7f3af00d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.725+0000 7f3af6a00640 1 --2- 172.21.15.60:0/739095512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00b5480 0x7f3af00d2830 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f3af006cdc0 tx=0x7f3adc02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.725+0000 7f3af6a00640 1 -- 172.21.15.60:0/739095512 shutdown_connections 2024-09-16T15:46:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.725+0000 7f3af6a00640 1 --2- 172.21.15.60:0/739095512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af00b5480 0x7f3af00d2830 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.725+0000 7f3af6a00640 1 --2- 172.21.15.60:0/739095512 >> v2:172.21.15.100:3300/0 conn(0x7f3af0161460 0x7f3af00d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.725+0000 7f3af6a00640 1 -- 172.21.15.60:0/739095512 >> 172.21.15.60:0/739095512 conn(0x7f3af009d0b0 msgr2=0x7f3af00d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.725+0000 7f3af6a00640 1 -- 172.21.15.60:0/739095512 shutdown_connections 2024-09-16T15:46:59.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.725+0000 7f3af6a00640 1 -- 172.21.15.60:0/739095512 wait complete. 2024-09-16T15:46:59.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.726+0000 7f3af6a00640 1 Processor -- start 2024-09-16T15:46:59.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.727+0000 7f3af6a00640 1 -- start start 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.727+0000 7f3af6a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af00b4f40 0x7f3af00a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.727+0000 7f3af6a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af00b5480 0x7f3af00cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.727+0000 7f3af6a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af0161460 0x7f3af00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00cef30 con 0x7f3af0161460 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00b4610 con 0x7f3af00b4f40 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3af00cf0b0 con 0x7f3af00b5480 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af0161460 0x7f3af00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af0161460 0x7f3af00ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47122/0 (socket says 172.21.15.60:47122) 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 -- 172.21.15.60:0/404641307 learned_addr learned my addr 172.21.15.60:0/404641307 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:46:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af5600640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af00b4f40 0x7f3af00a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af4c00640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af00b5480 0x7f3af00cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 -- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af00b5480 msgr2=0x7f3af00cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af00b5480 0x7f3af00cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 -- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af00b4f40 msgr2=0x7f3af00a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af00b4f40 0x7f3af00a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 -- 172.21.15.60:0/404641307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af0150d30 con 0x7f3af0161460 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af4c00640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af00b5480 0x7f3af00cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af5600640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af00b4f40 0x7f3af00a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.728+0000 7f3af6000640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af0161460 0x7f3af00ccee0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c780 tx=0x7f3ae800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.729+0000 7f3aed600640 1 -- 172.21.15.60:0/404641307 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8004b60 con 0x7f3af0161460 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.729+0000 7f3aed600640 1 -- 172.21.15.60:0/404641307 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ae8004df0 con 0x7f3af0161460 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.729+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af01f57c0 con 0x7f3af0161460 2024-09-16T15:46:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.729+0000 7f3aed600640 1 -- 172.21.15.60:0/404641307 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800aa90 con 0x7f3af0161460 2024-09-16T15:46:59.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.729+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af01512d0 con 0x7f3af0161460 2024-09-16T15:46:59.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.730+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af00ce820 con 0x7f3af0161460 2024-09-16T15:46:59.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.731+0000 7f3aed600640 1 -- 172.21.15.60:0/404641307 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f3ae800ac30 con 0x7f3af0161460 2024-09-16T15:46:59.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.732+0000 7f3aed600640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ad0078f10 0x7f3ad007b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.732+0000 7f3aed600640 1 -- 172.21.15.60:0/404641307 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f3ae809b6a0 con 0x7f3af0161460 2024-09-16T15:46:59.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.732+0000 7f3af5600640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ad0078f10 0x7f3ad007b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.733+0000 7f3af5600640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ad0078f10 0x7f3ad007b3d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3af009fa20 tx=0x7f3ae4005d10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.736+0000 7f3aed600640 1 -- 172.21.15.60:0/404641307 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3ae8020070 con 0x7f3af0161460 2024-09-16T15:46:59.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.963+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41180", "target": ["mon-mgr", ""]}) -- 0x7f3af0069d80 con 0x7f3ad0078f10 2024-09-16T15:46:59.970 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:59.969+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:59.970 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:59.969+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:59.970 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:59.969+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:59.970 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:59.969+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:59.971 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:46:59.969+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:46:59.998+0000 7f3aed600640 1 -- 172.21.15.60:0/404641307 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f3ae400da80 con 0x7f3ad0078f10 2024-09-16T15:47:00.000 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol41180 2024-09-16T15:47:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ad0078f10 msgr2=0x7f3ad007b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ad0078f10 0x7f3ad007b3d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3af009fa20 tx=0x7f3ae4005d10 comp rx=0 tx=0).stop 2024-09-16T15:47:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af0161460 msgr2=0x7f3af00ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af0161460 0x7f3af00ccee0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c780 tx=0x7f3ae800cc40 comp rx=0 tx=0).stop 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 shutdown_connections 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3ad0078f10 0x7f3ad007b3d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3af0161460 0x7f3af00ccee0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3af00b5480 0x7f3af00cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 --2- 172.21.15.60:0/404641307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3af00b4f40 0x7f3af00a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 >> 172.21.15.60:0/404641307 conn(0x7f3af009d0b0 msgr2=0x7f3af0160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 shutdown_connections 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.001+0000 7f3af6a00640 1 -- 172.21.15.60:0/404641307 wait complete. 2024-09-16T15:47:00.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol2771 subvol_grp41840 2024-09-16T15:47:00.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.181+0000 7f24a0000640 1 -- 172.21.15.60:0/3948360518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2498159aa0 msgr2=0x7f24980741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.181+0000 7f24a0000640 1 --2- 172.21.15.60:0/3948360518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2498159aa0 0x7f24980741a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f248c009830 tx=0x7f248c02a5c0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.181+0000 7f24a0000640 1 -- 172.21.15.60:0/3948360518 shutdown_connections 2024-09-16T15:47:00.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.181+0000 7f24a0000640 1 --2- 172.21.15.60:0/3948360518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2498159aa0 0x7f24980741a0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.181+0000 7f24a0000640 1 -- 172.21.15.60:0/3948360518 >> 172.21.15.60:0/3948360518 conn(0x7f24980b18e0 msgr2=0x7f24980b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:00.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.181+0000 7f24a0000640 1 -- 172.21.15.60:0/3948360518 shutdown_connections 2024-09-16T15:47:00.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.181+0000 7f24a0000640 1 -- 172.21.15.60:0/3948360518 wait complete. 2024-09-16T15:47:00.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.183+0000 7f24a0000640 1 Processor -- start 2024-09-16T15:47:00.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.183+0000 7f24a0000640 1 -- start start 2024-09-16T15:47:00.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f24a0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f249806f220 0x7f24980ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f24a0000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2498159aa0 0x7f2498154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f24a0000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f249815cff0 0x7f24981f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f24a0000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f249806f6b0 con 0x7f249815cff0 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f24a0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24980ccc50 con 0x7f249806f220 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f24a0000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f249806f830 con 0x7f2498159aa0 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f249e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f249815cff0 0x7f24981f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f249e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f249815cff0 0x7f24981f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47156/0 (socket says 172.21.15.60:47156) 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f249e000640 1 -- 172.21.15.60:0/4045515032 learned_addr learned my addr 172.21.15.60:0/4045515032 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f249d600640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f249806f220 0x7f24980ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.184+0000 7f249cc00640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2498159aa0 0x7f2498154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f249e000640 1 -- 172.21.15.60:0/4045515032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2498159aa0 msgr2=0x7f2498154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f249e000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2498159aa0 0x7f2498154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f249e000640 1 -- 172.21.15.60:0/4045515032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f249806f220 msgr2=0x7f24980ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f249e000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f249806f220 0x7f24980ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f249e000640 1 -- 172.21.15.60:0/4045515032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24980cea20 con 0x7f249815cff0 2024-09-16T15:47:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f249e000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f249815cff0 0x7f24981f3170 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f249000c7a0 tx=0x7f249000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f2495600640 1 -- 172.21.15.60:0/4045515032 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490003b70 con 0x7f249815cff0 2024-09-16T15:47:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f2495600640 1 -- 172.21.15.60:0/4045515032 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2490003d10 con 0x7f249815cff0 2024-09-16T15:47:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f2495600640 1 -- 172.21.15.60:0/4045515032 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249000aab0 con 0x7f249815cff0 2024-09-16T15:47:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.185+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24981f2990 con 0x7f249815cff0 2024-09-16T15:47:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.186+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249809c2c0 con 0x7f249815cff0 2024-09-16T15:47:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.186+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2498069360 con 0x7f249815cff0 2024-09-16T15:47:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.188+0000 7f2495600640 1 -- 172.21.15.60:0/4045515032 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 22) ==== 101899+0+0 (secure 0 0 0) 0x7f249000ac50 con 0x7f249815cff0 2024-09-16T15:47:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.189+0000 7f2495600640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2478078f10 0x7f247807b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.189+0000 7f2495600640 1 -- 172.21.15.60:0/4045515032 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f249009b270 con 0x7f249815cff0 2024-09-16T15:47:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.189+0000 7f249d600640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2478078f10 0x7f247807b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.190+0000 7f249d600640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2478078f10 0x7f247807b3d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f248c034000 tx=0x7f248c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:00.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.193+0000 7f2495600640 1 -- 172.21.15.60:0/4045515032 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2490062920 con 0x7f249815cff0 2024-09-16T15:47:00.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.414+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol2771", "group_name": "subvol_grp41840", "target": ["mon-mgr", ""]}) -- 0x7f24980ae530 con 0x7f2478078f10 2024-09-16T15:47:00.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.431+0000 7f2495600640 1 -- 172.21.15.60:0/4045515032 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f24980ae530 con 0x7f2478078f10 2024-09-16T15:47:00.432 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/subvol_grp41840/subvol2771 2024-09-16T15:47:00.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2478078f10 msgr2=0x7f247807b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2478078f10 0x7f247807b3d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f248c034000 tx=0x7f248c004730 comp rx=0 tx=0).stop 2024-09-16T15:47:00.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f249815cff0 msgr2=0x7f24981f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f249815cff0 0x7f24981f3170 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f249000c7a0 tx=0x7f249000cc60 comp rx=0 tx=0).stop 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 shutdown_connections 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2478078f10 0x7f247807b3d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f249815cff0 0x7f24981f3170 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2498159aa0 0x7f2498154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 --2- 172.21.15.60:0/4045515032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f249806f220 0x7f24980ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 >> 172.21.15.60:0/4045515032 conn(0x7f24980b18e0 msgr2=0x7f249815ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 shutdown_connections 2024-09-16T15:47:00.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.434+0000 7f24a0000640 1 -- 172.21.15.60:0/4045515032 wait complete. 2024-09-16T15:47:00.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41180 2024-09-16T15:47:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.621+0000 7f1347e00640 1 -- 172.21.15.60:0/2758559567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 msgr2=0x7f134806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.621+0000 7f1347e00640 1 --2- 172.21.15.60:0/2758559567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 0x7f134806fce0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f1348072b20 tx=0x7f133802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.621+0000 7f1347e00640 1 -- 172.21.15.60:0/2758559567 shutdown_connections 2024-09-16T15:47:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.621+0000 7f1347e00640 1 --2- 172.21.15.60:0/2758559567 >> v2:172.21.15.100:3300/0 conn(0x7f1348072560 0x7f1348073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.621+0000 7f1347e00640 1 --2- 172.21.15.60:0/2758559567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 0x7f134806fce0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.621+0000 7f1347e00640 1 -- 172.21.15.60:0/2758559567 >> 172.21.15.60:0/2758559567 conn(0x7f13480cf9f0 msgr2=0x7f13480cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.621+0000 7f1347e00640 1 -- 172.21.15.60:0/2758559567 shutdown_connections 2024-09-16T15:47:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.621+0000 7f1347e00640 1 -- 172.21.15.60:0/2758559567 wait complete. 2024-09-16T15:47:00.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.623+0000 7f1347e00640 1 Processor -- start 2024-09-16T15:47:00.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.623+0000 7f1347e00640 1 -- start start 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1348072560 0x7f13480ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13480cb4f0 0x7f1348159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 0x7f13480ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f13480cbdf0 con 0x7f134815bec0 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13480cbc70 con 0x7f13480cb4f0 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f13480aebd0 con 0x7f1348072560 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 0x7f13480ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 0x7f13480ab5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47168/0 (socket says 172.21.15.60:47168) 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1347400640 1 -- 172.21.15.60:0/226705136 learned_addr learned my addr 172.21.15.60:0/226705136 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1346a00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1348072560 0x7f13480ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.624+0000 7f1346000640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13480cb4f0 0x7f1348159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:00.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f1347400640 1 -- 172.21.15.60:0/226705136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1348072560 msgr2=0x7f13480ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f1347400640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1348072560 0x7f13480ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f1347400640 1 -- 172.21.15.60:0/226705136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13480cb4f0 msgr2=0x7f1348159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f1347400640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13480cb4f0 0x7f1348159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f1347400640 1 -- 172.21.15.60:0/226705136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13481f2cc0 con 0x7f134815bec0 2024-09-16T15:47:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f1347400640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 0x7f13480ab5b0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f134000c7d0 tx=0x7f134000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:00.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f133ea00640 1 -- 172.21.15.60:0/226705136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340003b70 con 0x7f134815bec0 2024-09-16T15:47:00.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f133ea00640 1 -- 172.21.15.60:0/226705136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1340003d10 con 0x7f134815bec0 2024-09-16T15:47:00.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f133ea00640 1 -- 172.21.15.60:0/226705136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134000aab0 con 0x7f134815bec0 2024-09-16T15:47:00.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13481f2fa0 con 0x7f134815bec0 2024-09-16T15:47:00.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.625+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13480a9470 con 0x7f134815bec0 2024-09-16T15:47:00.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.626+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1348070ec0 con 0x7f134815bec0 2024-09-16T15:47:00.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.628+0000 7f133ea00640 1 -- 172.21.15.60:0/226705136 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 101950+0+0 (secure 0 0 0) 0x7f134000ac50 con 0x7f134815bec0 2024-09-16T15:47:00.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.629+0000 7f133ea00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1328078fa0 0x7f132807b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:00.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.629+0000 7f133ea00640 1 -- 172.21.15.60:0/226705136 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f134009b820 con 0x7f134815bec0 2024-09-16T15:47:00.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.629+0000 7f1346a00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1328078fa0 0x7f132807b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:00.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.630+0000 7f1346a00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1328078fa0 0x7f132807b460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1338033000 tx=0x7f1338004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:00.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.633+0000 7f133ea00640 1 -- 172.21.15.60:0/226705136 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1348070ec0 con 0x7f134815bec0 2024-09-16T15:47:00.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.859+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41180", "target": ["mon-mgr", ""]}) -- 0x7f13480af050 con 0x7f1328078fa0 2024-09-16T15:47:00.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.864+0000 7f133ea00640 1 -- 172.21.15.60:0/226705136 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f13480af050 con 0x7f1328078fa0 2024-09-16T15:47:00.865 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol41180 2024-09-16T15:47:00.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1328078fa0 msgr2=0x7f132807b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1328078fa0 0x7f132807b460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1338033000 tx=0x7f1338004730 comp rx=0 tx=0).stop 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 msgr2=0x7f13480ab5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 0x7f13480ab5b0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f134000c7d0 tx=0x7f134000cc90 comp rx=0 tx=0).stop 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 shutdown_connections 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1328078fa0 0x7f132807b460 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f134815bec0 0x7f13480ab5b0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13480cb4f0 0x7f1348159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 --2- 172.21.15.60:0/226705136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1348072560 0x7f13480ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 >> 172.21.15.60:0/226705136 conn(0x7f13480cf9f0 msgr2=0x7f13480b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 shutdown_connections 2024-09-16T15:47:00.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:00.866+0000 7f1347e00640 1 -- 172.21.15.60:0/226705136 wait complete. 2024-09-16T15:47:00.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:00.884 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/0890bb875eb89ba3dcd47b5bca977445.meta) 2024-09-16T15:47:00.946 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:47:00.947 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:47:00.947 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:47:00.947 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/_nogroup/subvol41180 2024-09-16T15:47:00.947 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:47:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:00.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol2771 subvol_grp41840 2024-09-16T15:47:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.113+0000 7f2ef7800640 1 -- 172.21.15.60:0/1811807100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef006ee00 msgr2=0x7f2ef006f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.113+0000 7f2ef7800640 1 --2- 172.21.15.60:0/1811807100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef006ee00 0x7f2ef006f300 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009830 tx=0x7f2ed802a510 comp rx=0 tx=0).stop 2024-09-16T15:47:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.113+0000 7f2ef7800640 1 -- 172.21.15.60:0/1811807100 shutdown_connections 2024-09-16T15:47:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.113+0000 7f2ef7800640 1 --2- 172.21.15.60:0/1811807100 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ef00b4770 0x7f2ef00ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.113+0000 7f2ef7800640 1 --2- 172.21.15.60:0/1811807100 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef006ee00 0x7f2ef006f300 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.113+0000 7f2ef7800640 1 -- 172.21.15.60:0/1811807100 >> 172.21.15.60:0/1811807100 conn(0x7f2ef00b18e0 msgr2=0x7f2ef00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.114+0000 7f2ef7800640 1 -- 172.21.15.60:0/1811807100 shutdown_connections 2024-09-16T15:47:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.114+0000 7f2ef7800640 1 -- 172.21.15.60:0/1811807100 wait complete. 2024-09-16T15:47:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.115+0000 7f2ef7800640 1 Processor -- start 2024-09-16T15:47:01.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.115+0000 7f2ef7800640 1 -- start start 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.116+0000 7f2ef7800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ef006ee00 0x7f2ef014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.116+0000 7f2ef7800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef00b4770 0x7f2ef01f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.116+0000 7f2ef7800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef015dfc0 0x7f2ef00740d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.116+0000 7f2ef7800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef00adfa0 con 0x7f2ef015dfc0 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.116+0000 7f2ef7800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef00ade20 con 0x7f2ef00b4770 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.116+0000 7f2ef7800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef00ae120 con 0x7f2ef006ee00 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef015dfc0 0x7f2ef00740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef015dfc0 0x7f2ef00740d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47176/0 (socket says 172.21.15.60:47176) 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 -- 172.21.15.60:0/4283576120 learned_addr learned my addr 172.21.15.60:0/4283576120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef4e00640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ef006ee00 0x7f2ef014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ee7e00640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef00b4770 0x7f2ef01f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 -- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ef006ee00 msgr2=0x7f2ef014cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ef006ee00 0x7f2ef014cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 -- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef00b4770 msgr2=0x7f2ef01f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef00b4770 0x7f2ef01f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 -- 172.21.15.60:0/4283576120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef01f5ac0 con 0x7f2ef015dfc0 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef4e00640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ef006ee00 0x7f2ef014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ee7e00640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef00b4770 0x7f2ef01f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.117+0000 7f2ef5800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef015dfc0 0x7f2ef00740d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800da70 tx=0x7f2ee800df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.118+0000 7f2eed600640 1 -- 172.21.15.60:0/4283576120 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8017070 con 0x7f2ef015dfc0 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.118+0000 7f2eed600640 1 -- 172.21.15.60:0/4283576120 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ee8003970 con 0x7f2ef015dfc0 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.118+0000 7f2eed600640 1 -- 172.21.15.60:0/4283576120 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8003dc0 con 0x7f2ef015dfc0 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.118+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef01f84d0 con 0x7f2ef015dfc0 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.118+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01f8a90 con 0x7f2ef015dfc0 2024-09-16T15:47:01.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.119+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef0069360 con 0x7f2ef015dfc0 2024-09-16T15:47:01.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.120+0000 7f2eed600640 1 -- 172.21.15.60:0/4283576120 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 101950+0+0 (secure 0 0 0) 0x7f2ee8003f60 con 0x7f2ef015dfc0 2024-09-16T15:47:01.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.121+0000 7f2eed600640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ecc078fd0 0x7f2ecc07b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.121+0000 7f2eed600640 1 -- 172.21.15.60:0/4283576120 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f2ee809c040 con 0x7f2ef015dfc0 2024-09-16T15:47:01.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.121+0000 7f2ef4e00640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ecc078fd0 0x7f2ecc07b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.122+0000 7f2ef4e00640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ecc078fd0 0x7f2ecc07b490 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0000f60 tx=0x7f2ee0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.126+0000 7f2eed600640 1 -- 172.21.15.60:0/4283576120 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ef01f5ac0 con 0x7f2ef015dfc0 2024-09-16T15:47:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.351+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol2771", "group_name": "subvol_grp41840", "target": ["mon-mgr", ""]}) -- 0x7f2ef0073940 con 0x7f2ecc078fd0 2024-09-16T15:47:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.355+0000 7f2eed600640 1 -- 172.21.15.60:0/4283576120 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f2ef0073940 con 0x7f2ecc078fd0 2024-09-16T15:47:01.356 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/subvol_grp41840/subvol2771 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.357+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ecc078fd0 msgr2=0x7f2ecc07b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.357+0000 7f2ef7800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ecc078fd0 0x7f2ecc07b490 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0000f60 tx=0x7f2ee0009040 comp rx=0 tx=0).stop 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.357+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef015dfc0 msgr2=0x7f2ef00740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.357+0000 7f2ef7800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef015dfc0 0x7f2ef00740d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800da70 tx=0x7f2ee800df30 comp rx=0 tx=0).stop 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.358+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 shutdown_connections 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.358+0000 7f2ef7800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2ecc078fd0 0x7f2ecc07b490 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.358+0000 7f2ef7800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ef015dfc0 0x7f2ef00740d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.358+0000 7f2ef7800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ef00b4770 0x7f2ef01f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.358+0000 7f2ef7800640 1 --2- 172.21.15.60:0/4283576120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ef006ee00 0x7f2ef014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.358+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 >> 172.21.15.60:0/4283576120 conn(0x7f2ef00b18e0 msgr2=0x7f2ef00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.358+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 shutdown_connections 2024-09-16T15:47:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.358+0000 7f2ef7800640 1 -- 172.21.15.60:0/4283576120 wait complete. 2024-09-16T15:47:01.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:01.375 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/77133116f8aed46b261e83736dae094c.meta) 2024-09-16T15:47:01.434 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:47:01.434 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:47:01.435 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:47:01.435 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/subvol_grp41840/subvol2771 2024-09-16T15:47:01.435 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:47:01.435 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:01.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol41180 2024-09-16T15:47:01.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.599+0000 7efd6b200640 1 -- 172.21.15.60:0/1261596230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c0cf830 msgr2=0x7efd6c0b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.599+0000 7efd6b200640 1 --2- 172.21.15.60:0/1261596230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c0cf830 0x7efd6c0b33b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7efd58005fd0 tx=0x7efd5802a540 comp rx=0 tx=0).stop 2024-09-16T15:47:01.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.599+0000 7efd6b200640 1 -- 172.21.15.60:0/1261596230 shutdown_connections 2024-09-16T15:47:01.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.599+0000 7efd6b200640 1 --2- 172.21.15.60:0/1261596230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c0cf830 0x7efd6c0b33b0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.599+0000 7efd6b200640 1 --2- 172.21.15.60:0/1261596230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd6c0d2a00 0x7efd6c0b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.599+0000 7efd6b200640 1 -- 172.21.15.60:0/1261596230 >> 172.21.15.60:0/1261596230 conn(0x7efd6c09d0b0 msgr2=0x7efd6c0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.599+0000 7efd6b200640 1 -- 172.21.15.60:0/1261596230 shutdown_connections 2024-09-16T15:47:01.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.599+0000 7efd6b200640 1 -- 172.21.15.60:0/1261596230 wait complete. 2024-09-16T15:47:01.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.600+0000 7efd6b200640 1 Processor -- start 2024-09-16T15:47:01.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.600+0000 7efd6b200640 1 -- start start 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.601+0000 7efd6b200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd6c0cf830 0x7efd6c1f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.601+0000 7efd6b200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0d2a00 0x7efd6c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.601+0000 7efd6b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c161460 0x7efd6c0c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.601+0000 7efd6b200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c0b50c0 con 0x7efd6c161460 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.601+0000 7efd6b200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c0b4f40 con 0x7efd6c0d2a00 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.601+0000 7efd6b200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7efd6c0b3e30 con 0x7efd6c0cf830 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c161460 0x7efd6c0c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c161460 0x7efd6c0c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47180/0 (socket says 172.21.15.60:47180) 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 -- 172.21.15.60:0/841389493 learned_addr learned my addr 172.21.15.60:0/841389493 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd69400640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0d2a00 0x7efd6c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 -- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd6c0cf830 msgr2=0x7efd6c1f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd69e00640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd6c0cf830 0x7efd6c1f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd6c0cf830 0x7efd6c1f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 -- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0d2a00 msgr2=0x7efd6c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0d2a00 0x7efd6c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 -- 172.21.15.60:0/841389493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c1f8c30 con 0x7efd6c161460 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd69400640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0d2a00 0x7efd6c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd69e00640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd6c0cf830 0x7efd6c1f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.602+0000 7efd6a800640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c161460 0x7efd6c0c9930 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7efd6400da10 tx=0x7efd6400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.603+0000 7efd62000640 1 -- 172.21.15.60:0/841389493 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64016070 con 0x7efd6c161460 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.603+0000 7efd62000640 1 -- 172.21.15.60:0/841389493 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd64003970 con 0x7efd6c161460 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.603+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c0b98a0 con 0x7efd6c161460 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.603+0000 7efd62000640 1 -- 172.21.15.60:0/841389493 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64003dc0 con 0x7efd6c161460 2024-09-16T15:47:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.603+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c1f4e90 con 0x7efd6c161460 2024-09-16T15:47:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.604+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6c0b42e0 con 0x7efd6c161460 2024-09-16T15:47:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.605+0000 7efd62000640 1 -- 172.21.15.60:0/841389493 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 101950+0+0 (secure 0 0 0) 0x7efd64003f60 con 0x7efd6c161460 2024-09-16T15:47:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.606+0000 7efd62000640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd4c078fa0 0x7efd4c07b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.606+0000 7efd62000640 1 -- 172.21.15.60:0/841389493 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7efd6409c280 con 0x7efd6c161460 2024-09-16T15:47:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.606+0000 7efd69e00640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd4c078fa0 0x7efd4c07b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.607+0000 7efd69e00640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd4c078fa0 0x7efd4c07b460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0016a0 tx=0x7efd5c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.610+0000 7efd62000640 1 -- 172.21.15.60:0/841389493 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efd6c1f8c30 con 0x7efd6c161460 2024-09-16T15:47:01.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:47:01.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:47:01.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.836+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol41180", "target": ["mon-mgr", ""]}) -- 0x7efd6c06e630 con 0x7efd4c078fa0 2024-09-16T15:47:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.865+0000 7efd62000640 1 -- 172.21.15.60:0/841389493 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd6c06e630 con 0x7efd4c078fa0 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.867+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd4c078fa0 msgr2=0x7efd4c07b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.867+0000 7efd6b200640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd4c078fa0 0x7efd4c07b460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0016a0 tx=0x7efd5c009040 comp rx=0 tx=0).stop 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.867+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c161460 msgr2=0x7efd6c0c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.867+0000 7efd6b200640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c161460 0x7efd6c0c9930 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7efd6400da10 tx=0x7efd6400ded0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.868+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 shutdown_connections 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.868+0000 7efd6b200640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7efd4c078fa0 0x7efd4c07b460 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.868+0000 7efd6b200640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd6c161460 0x7efd6c0c9930 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.868+0000 7efd6b200640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd6c0d2a00 0x7efd6c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.868+0000 7efd6b200640 1 --2- 172.21.15.60:0/841389493 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd6c0cf830 0x7efd6c1f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.868+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 >> 172.21.15.60:0/841389493 conn(0x7efd6c09d0b0 msgr2=0x7efd6c06e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.868+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 shutdown_connections 2024-09-16T15:47:01.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:01.868+0000 7efd6b200640 1 -- 172.21.15.60:0/841389493 wait complete. 2024-09-16T15:47:01.872 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.871+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.872 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.871+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.872 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.871+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.873 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.871+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.873 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.871+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol2771 subvol_grp41840 2024-09-16T15:47:01.892 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.890+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.892 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.890+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.893 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.890+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.893 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.890+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:01.893 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:01.890+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:02.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 -- 172.21.15.60:0/146413107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 msgr2=0x7fbce006fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 --2- 172.21.15.60:0/146413107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 0x7fbce006fe40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fbce0070ae0 tx=0x7fbcc802a560 comp rx=0 tx=0).stop 2024-09-16T15:47:02.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 -- 172.21.15.60:0/146413107 shutdown_connections 2024-09-16T15:47:02.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 --2- 172.21.15.60:0/146413107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbce0070380 0x7fbce00739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 --2- 172.21.15.60:0/146413107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 0x7fbce006fe40 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 --2- 172.21.15.60:0/146413107 >> v2:172.21.15.100:3300/0 conn(0x7fbce015bda0 0x7fbce0073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 -- 172.21.15.60:0/146413107 >> 172.21.15.60:0/146413107 conn(0x7fbce00cfa70 msgr2=0x7fbce00cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:02.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 -- 172.21.15.60:0/146413107 shutdown_connections 2024-09-16T15:47:02.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.052+0000 7fbce4c00640 1 -- 172.21.15.60:0/146413107 wait complete. 2024-09-16T15:47:02.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.054+0000 7fbce4c00640 1 Processor -- start 2024-09-16T15:47:02.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.054+0000 7fbce4c00640 1 -- start start 2024-09-16T15:47:02.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbce4c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 0x7fbce00c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbce4c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbce0070380 0x7fbce00c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbce4c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbce015bda0 0x7fbce00c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbce4c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0073f10 con 0x7fbce006e590 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbce4c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbce00708d0 con 0x7fbce015bda0 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbce4c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbce0074090 con 0x7fbce0070380 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbcdf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 0x7fbce00c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbcdf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 0x7fbce00c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47236/0 (socket says 172.21.15.60:47236) 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbcdf400640 1 -- 172.21.15.60:0/809816589 learned_addr learned my addr 172.21.15.60:0/809816589 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbcdfe00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbce015bda0 0x7fbce00c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.055+0000 7fbcd6a00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbce0070380 0x7fbce00c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcdf400640 1 -- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbce0070380 msgr2=0x7fbce00c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcdf400640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbce0070380 0x7fbce00c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcdf400640 1 -- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbce015bda0 msgr2=0x7fbce00c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcdf400640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbce015bda0 0x7fbce00c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcdf400640 1 -- 172.21.15.60:0/809816589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce00c8d00 con 0x7fbce006e590 2024-09-16T15:47:02.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcd6a00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbce0070380 0x7fbce00c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:02.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcdf400640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 0x7fbce00c9500 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000e9f0 tx=0x7fbcd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcd7e00640 1 -- 172.21.15.60:0/809816589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000ccc0 con 0x7fbce006e590 2024-09-16T15:47:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcd7e00640 1 -- 172.21.15.60:0/809816589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcd000ce60 con 0x7fbce006e590 2024-09-16T15:47:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce015dac0 con 0x7fbce006e590 2024-09-16T15:47:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbcd7e00640 1 -- 172.21.15.60:0/809816589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0004280 con 0x7fbce006e590 2024-09-16T15:47:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.056+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01f4d30 con 0x7fbce006e590 2024-09-16T15:47:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.057+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce00745c0 con 0x7fbce006e590 2024-09-16T15:47:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.059+0000 7fbcd7e00640 1 -- 172.21.15.60:0/809816589 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 23) ==== 101950+0+0 (secure 0 0 0) 0x7fbcd00035f0 con 0x7fbce006e590 2024-09-16T15:47:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.059+0000 7fbcd7e00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbcbc078fa0 0x7fbcbc07b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.060+0000 7fbcd7e00640 1 -- 172.21.15.60:0/809816589 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7fbcd009c0e0 con 0x7fbce006e590 2024-09-16T15:47:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.060+0000 7fbcd6a00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbcbc078fa0 0x7fbcbc07b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.060+0000 7fbcd6a00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbcbc078fa0 0x7fbcbc07b460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8033000 tx=0x7fbcc8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:02.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.064+0000 7fbcd7e00640 1 -- 172.21.15.60:0/809816589 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbce015dac0 con 0x7fbce006e590 2024-09-16T15:47:02.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.285+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol2771", "group_name": "subvol_grp41840", "target": ["mon-mgr", ""]}) -- 0x7fbce00cc070 con 0x7fbcbc078fa0 2024-09-16T15:47:02.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:02.626+0000 7fbcd7e00640 1 -- 172.21.15.60:0/809816589 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fbcd005f6f0 con 0x7fbce006e590 2024-09-16T15:47:08.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.629+0000 7fbcd7e00640 1 -- 172.21.15.60:0/809816589 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbce00cc070 con 0x7fbcbc078fa0 2024-09-16T15:47:08.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbcbc078fa0 msgr2=0x7fbcbc07b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:08.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbcbc078fa0 0x7fbcbc07b460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8033000 tx=0x7fbcc8004730 comp rx=0 tx=0).stop 2024-09-16T15:47:08.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 msgr2=0x7fbce00c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:08.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 0x7fbce00c9500 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000e9f0 tx=0x7fbcd000edf0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 shutdown_connections 2024-09-16T15:47:08.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbcbc078fa0 0x7fbcbc07b460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbce015bda0 0x7fbce00c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbce0070380 0x7fbce00c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 --2- 172.21.15.60:0/809816589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbce006e590 0x7fbce00c9500 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 >> 172.21.15.60:0/809816589 conn(0x7fbce00cfa70 msgr2=0x7fbce01596a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:08.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 shutdown_connections 2024-09-16T15:47:08.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.633+0000 7fbce4c00640 1 -- 172.21.15.60:0/809816589 wait complete. 2024-09-16T15:47:08.655 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:47:08.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:47:08.745 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-16T15:47:08.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:47:08.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp41840 2024-09-16T15:47:08.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 -- 172.21.15.60:0/788262478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c145750 msgr2=0x7f660c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:08.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 --2- 172.21.15.60:0/788262478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c145750 0x7f660c073440 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f660c0af750 tx=0x7f65fc02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 -- 172.21.15.60:0/788262478 shutdown_connections 2024-09-16T15:47:08.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 --2- 172.21.15.60:0/788262478 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c0ccb00 0x7f660c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 --2- 172.21.15.60:0/788262478 >> v2:172.21.15.100:3300/0 conn(0x7f660c06efa0 0x7f660c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 --2- 172.21.15.60:0/788262478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c145750 0x7f660c073440 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 -- 172.21.15.60:0/788262478 >> 172.21.15.60:0/788262478 conn(0x7f660c0b1840 msgr2=0x7f660c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:08.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 -- 172.21.15.60:0/788262478 shutdown_connections 2024-09-16T15:47:08.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.918+0000 7f6611c00640 1 -- 172.21.15.60:0/788262478 wait complete. 2024-09-16T15:47:08.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.919+0000 7f6611c00640 1 Processor -- start 2024-09-16T15:47:08.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.919+0000 7f6611c00640 1 -- start start 2024-09-16T15:47:08.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.920+0000 7f6611c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c06efa0 0x7f660c143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:08.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.920+0000 7f6611c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c0ccb00 0x7f660c13acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:08.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.920+0000 7f6611c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c145750 0x7f660c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:08.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.920+0000 7f6611c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1602c0 con 0x7f660c0ccb00 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.920+0000 7f6611c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f660c160140 con 0x7f660c06efa0 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.920+0000 7f6611c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f660c160440 con 0x7f660c145750 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c0ccb00 0x7f660c13acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c0ccb00 0x7f660c13acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51976/0 (socket says 172.21.15.60:51976) 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 -- 172.21.15.60:0/2474405072 learned_addr learned my addr 172.21.15.60:0/2474405072 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f660b400640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c145750 0x7f660c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 -- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c145750 msgr2=0x7f660c0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f660aa00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c06efa0 0x7f660c143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c145750 0x7f660c0adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 -- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c06efa0 msgr2=0x7f660c143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c06efa0 0x7f660c143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 -- 172.21.15.60:0/2474405072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c153740 con 0x7f660c0ccb00 2024-09-16T15:47:08.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f660aa00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c06efa0 0x7f660c143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:08.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f660b400640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c145750 0x7f660c0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:08.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.921+0000 7f6603e00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c0ccb00 0x7f660c13acc0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f65f400d770 tx=0x7f65f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:08.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.922+0000 7f6602a00640 1 -- 172.21.15.60:0/2474405072 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4017070 con 0x7f660c0ccb00 2024-09-16T15:47:08.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.922+0000 7f6602a00640 1 -- 172.21.15.60:0/2474405072 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65f4009d20 con 0x7f660c0ccb00 2024-09-16T15:47:08.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.922+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c153a20 con 0x7f660c0ccb00 2024-09-16T15:47:08.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.922+0000 7f6602a00640 1 -- 172.21.15.60:0/2474405072 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f40106c0 con 0x7f660c0ccb00 2024-09-16T15:47:08.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.922+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c153fe0 con 0x7f660c0ccb00 2024-09-16T15:47:08.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.923+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660c0af2f0 con 0x7f660c0ccb00 2024-09-16T15:47:08.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.924+0000 7f6602a00640 1 -- 172.21.15.60:0/2474405072 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f65f401a070 con 0x7f660c0ccb00 2024-09-16T15:47:08.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.925+0000 7f6602a00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65e80790f0 0x7f65e807b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:08.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.925+0000 7f6602a00640 1 -- 172.21.15.60:0/2474405072 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f65f409bd70 con 0x7f660c0ccb00 2024-09-16T15:47:08.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.925+0000 7f660aa00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65e80790f0 0x7f65e807b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:08.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.926+0000 7f660aa00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65e80790f0 0x7f65e807b5b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f65fc033000 tx=0x7f65fc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:08.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:08.930+0000 7f6602a00640 1 -- 172.21.15.60:0/2474405072 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f660c0af2f0 con 0x7f660c0ccb00 2024-09-16T15:47:09.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.149+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp41840", "target": ["mon-mgr", ""]}) -- 0x7f660c15f9f0 con 0x7f65e80790f0 2024-09-16T15:47:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.153+0000 7f6602a00640 1 -- 172.21.15.60:0/2474405072 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f660c15f9f0 con 0x7f65e80790f0 2024-09-16T15:47:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65e80790f0 msgr2=0x7f65e807b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65e80790f0 0x7f65e807b5b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f65fc033000 tx=0x7f65fc004730 comp rx=0 tx=0).stop 2024-09-16T15:47:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c0ccb00 msgr2=0x7f660c13acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c0ccb00 0x7f660c13acc0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f65f400d770 tx=0x7f65f400dc30 comp rx=0 tx=0).stop 2024-09-16T15:47:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 shutdown_connections 2024-09-16T15:47:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f65e80790f0 0x7f65e807b5b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f660c145750 0x7f660c0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f660c0ccb00 0x7f660c13acc0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 --2- 172.21.15.60:0/2474405072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f660c06efa0 0x7f660c143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 >> 172.21.15.60:0/2474405072 conn(0x7f660c0b1840 msgr2=0x7f660c0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 shutdown_connections 2024-09-16T15:47:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.155+0000 7f6611c00640 1 -- 172.21.15.60:0/2474405072 wait complete. 2024-09-16T15:47:09.172 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:47:09.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:47:09.262 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:47:09.365 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:47:09.365 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:47:09.365 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:09.365 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:09.400 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:09.401 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:09.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:47:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.679+0000 7fdba2000640 1 -- 172.21.15.60:0/3669333853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 msgr2=0x7fdb9c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.679+0000 7fdba2000640 1 --2- 172.21.15.60:0/3669333853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 0x7fdb9c0b28f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005fd0 tx=0x7fdb9002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.679+0000 7fdba2000640 1 -- 172.21.15.60:0/3669333853 shutdown_connections 2024-09-16T15:47:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.679+0000 7fdba2000640 1 --2- 172.21.15.60:0/3669333853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c0725d0 0x7fdb9c073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.679+0000 7fdba2000640 1 --2- 172.21.15.60:0/3669333853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 0x7fdb9c0b28f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.679+0000 7fdba2000640 1 -- 172.21.15.60:0/3669333853 >> 172.21.15.60:0/3669333853 conn(0x7fdb9c0cfa20 msgr2=0x7fdb9c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.679+0000 7fdba2000640 1 -- 172.21.15.60:0/3669333853 shutdown_connections 2024-09-16T15:47:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.679+0000 7fdba2000640 1 -- 172.21.15.60:0/3669333853 wait complete. 2024-09-16T15:47:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.680+0000 7fdba2000640 1 Processor -- start 2024-09-16T15:47:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.681+0000 7fdba2000640 1 -- start start 2024-09-16T15:47:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.681+0000 7fdba2000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb9c0d0540 0x7fdb9c1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.681+0000 7fdba2000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 0x7fdb9c159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.681+0000 7fdba2000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c15a220 0x7fdb9c159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.681+0000 7fdba2000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c074610 con 0x7fdb9c15bc30 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.681+0000 7fdba2000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c074490 con 0x7fdb9c0d0540 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.681+0000 7fdba2000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c074790 con 0x7fdb9c15a220 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdb9be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 0x7fdb9c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdba0c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb9c0d0540 0x7fdb9c1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdb9be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 0x7fdb9c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51982/0 (socket says 172.21.15.60:51982) 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdb9be00640 1 -- 172.21.15.60:0/2741026606 learned_addr learned my addr 172.21.15.60:0/2741026606 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdba1600640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c15a220 0x7fdb9c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdb9be00640 1 -- 172.21.15.60:0/2741026606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c15a220 msgr2=0x7fdb9c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdb9be00640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c15a220 0x7fdb9c159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdb9be00640 1 -- 172.21.15.60:0/2741026606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb9c0d0540 msgr2=0x7fdb9c1526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdb9be00640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb9c0d0540 0x7fdb9c1526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.682+0000 7fdb9be00640 1 -- 172.21.15.60:0/2741026606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c1e8df0 con 0x7fdb9c15bc30 2024-09-16T15:47:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.683+0000 7fdb9be00640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 0x7fdb9c159100 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800d7a0 tx=0x7fdb8800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.683+0000 7fdb98c00640 1 -- 172.21.15.60:0/2741026606 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88017070 con 0x7fdb9c15bc30 2024-09-16T15:47:09.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.683+0000 7fdb98c00640 1 -- 172.21.15.60:0/2741026606 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb88009d20 con 0x7fdb9c15bc30 2024-09-16T15:47:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.683+0000 7fdb98c00640 1 -- 172.21.15.60:0/2741026606 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88005280 con 0x7fdb9c15bc30 2024-09-16T15:47:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.683+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c1e90d0 con 0x7fdb9c15bc30 2024-09-16T15:47:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.683+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c153270 con 0x7fdb9c15bc30 2024-09-16T15:47:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.684+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c06fc60 con 0x7fdb9c15bc30 2024-09-16T15:47:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.685+0000 7fdb98c00640 1 -- 172.21.15.60:0/2741026606 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fdb88005420 con 0x7fdb9c15bc30 2024-09-16T15:47:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.686+0000 7fdb98c00640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb7c0790c0 0x7fdb7c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.686+0000 7fdb98c00640 1 -- 172.21.15.60:0/2741026606 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7fdb8809bd60 con 0x7fdb9c15bc30 2024-09-16T15:47:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.686+0000 7fdba0c00640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb7c0790c0 0x7fdb7c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.687+0000 7fdba0c00640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb7c0790c0 0x7fdb7c07b580 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdb90033000 tx=0x7fdb90004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:09.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.690+0000 7fdb98c00640 1 -- 172.21.15.60:0/2741026606 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdb88062d30 con 0x7fdb9c15bc30 2024-09-16T15:47:09.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.898+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdb9c0cff80 con 0x7fdb9c15bc30 2024-09-16T15:47:09.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.899+0000 7fdb98c00640 1 -- 172.21.15.60:0/2741026606 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) ==== 80+0+0 (secure 0 0 0) 0x7fdb88067760 con 0x7fdb9c15bc30 2024-09-16T15:47:09.900 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:47:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.901+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb7c0790c0 msgr2=0x7fdb7c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.901+0000 7fdba2000640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb7c0790c0 0x7fdb7c07b580 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdb90033000 tx=0x7fdb90004730 comp rx=0 tx=0).stop 2024-09-16T15:47:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.901+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 msgr2=0x7fdb9c159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.901+0000 7fdba2000640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 0x7fdb9c159100 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800d7a0 tx=0x7fdb8800dc60 comp rx=0 tx=0).stop 2024-09-16T15:47:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.902+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 shutdown_connections 2024-09-16T15:47:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.902+0000 7fdba2000640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdb7c0790c0 0x7fdb7c07b580 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.902+0000 7fdba2000640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb9c15a220 0x7fdb9c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.902+0000 7fdba2000640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb9c15bc30 0x7fdb9c159100 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.902+0000 7fdba2000640 1 --2- 172.21.15.60:0/2741026606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb9c0d0540 0x7fdb9c1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.902+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 >> 172.21.15.60:0/2741026606 conn(0x7fdb9c0cfa20 msgr2=0x7fdb9c15a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.902+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 shutdown_connections 2024-09-16T15:47:09.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:09.902+0000 7fdba2000640 1 -- 172.21.15.60:0/2741026606 wait complete. 2024-09-16T15:47:09.919 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:47:09.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:09.919 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:09.983 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:47:09.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:09.983 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:47:10.018 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:47:10.019 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:10.019 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:10.058 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:10.058 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:10.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:47:10.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.297+0000 7fd19d400640 1 -- 172.21.15.60:0/3720205721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 msgr2=0x7fd1980b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:10.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.297+0000 7fd19d400640 1 --2- 172.21.15.60:0/3720205721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 0x7fd1980b2440 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd188005fd0 tx=0x7fd18802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.297+0000 7fd19d400640 1 -- 172.21.15.60:0/3720205721 shutdown_connections 2024-09-16T15:47:10.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.297+0000 7fd19d400640 1 --2- 172.21.15.60:0/3720205721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980ccf00 0x7fd1980acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.297+0000 7fd19d400640 1 --2- 172.21.15.60:0/3720205721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 0x7fd1980b2440 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.297+0000 7fd19d400640 1 -- 172.21.15.60:0/3720205721 >> 172.21.15.60:0/3720205721 conn(0x7fd1980b1950 msgr2=0x7fd1980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:10.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.297+0000 7fd19d400640 1 -- 172.21.15.60:0/3720205721 shutdown_connections 2024-09-16T15:47:10.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.297+0000 7fd19d400640 1 -- 172.21.15.60:0/3720205721 wait complete. 2024-09-16T15:47:10.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.299+0000 7fd19d400640 1 Processor -- start 2024-09-16T15:47:10.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.299+0000 7fd19d400640 1 -- start start 2024-09-16T15:47:10.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd19d400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980ad270 0x7fd1981f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:10.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd19d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980ccf00 0x7fd1980cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:10.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd19d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 0x7fd1980d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd19d400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd19806f2e0 con 0x7fd19815f5a0 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd19d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd19806f160 con 0x7fd1980ccf00 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd19d400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd19806f460 con 0x7fd1980ad270 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd197200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 0x7fd1980d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd196800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980ad270 0x7fd1981f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd197200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 0x7fd1980d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51988/0 (socket says 172.21.15.60:51988) 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd197200640 1 -- 172.21.15.60:0/1915464511 learned_addr learned my addr 172.21.15.60:0/1915464511 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.300+0000 7fd195e00640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980ccf00 0x7fd1980cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd197200640 1 -- 172.21.15.60:0/1915464511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980ad270 msgr2=0x7fd1981f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd197200640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980ad270 0x7fd1981f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd197200640 1 -- 172.21.15.60:0/1915464511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980ccf00 msgr2=0x7fd1980cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd197200640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980ccf00 0x7fd1980cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd197200640 1 -- 172.21.15.60:0/1915464511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981e53f0 con 0x7fd19815f5a0 2024-09-16T15:47:10.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd197200640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 0x7fd1980d01f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c780 tx=0x7fd19000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd18ea00640 1 -- 172.21.15.60:0/1915464511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190017070 con 0x7fd19815f5a0 2024-09-16T15:47:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd18ea00640 1 -- 172.21.15.60:0/1915464511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd190003c60 con 0x7fd19815f5a0 2024-09-16T15:47:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1981f80b0 con 0x7fd19815f5a0 2024-09-16T15:47:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.301+0000 7fd18ea00640 1 -- 172.21.15.60:0/1915464511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900052d0 con 0x7fd19815f5a0 2024-09-16T15:47:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.302+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981f4690 con 0x7fd19815f5a0 2024-09-16T15:47:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.303+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19806f9f0 con 0x7fd19815f5a0 2024-09-16T15:47:10.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.304+0000 7fd18ea00640 1 -- 172.21.15.60:0/1915464511 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fd19001a070 con 0x7fd19815f5a0 2024-09-16T15:47:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.305+0000 7fd18ea00640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1780790f0 0x7fd17807b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.305+0000 7fd18ea00640 1 -- 172.21.15.60:0/1915464511 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7fd19009b140 con 0x7fd19815f5a0 2024-09-16T15:47:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.305+0000 7fd196800640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1780790f0 0x7fd17807b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:10.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.305+0000 7fd196800640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1780790f0 0x7fd17807b5b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd188033000 tx=0x7fd188004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:10.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.309+0000 7fd18ea00640 1 -- 172.21.15.60:0/1915464511 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd1900621a0 con 0x7fd19815f5a0 2024-09-16T15:47:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.522+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1981f7e60 con 0x7fd19815f5a0 2024-09-16T15:47:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.522+0000 7fd18ea00640 1 -- 172.21.15.60:0/1915464511 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) ==== 80+0+0 (secure 0 0 0) 0x7fd190066bd0 con 0x7fd19815f5a0 2024-09-16T15:47:10.523 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.524+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1780790f0 msgr2=0x7fd17807b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.524+0000 7fd19d400640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1780790f0 0x7fd17807b5b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd188033000 tx=0x7fd188004730 comp rx=0 tx=0).stop 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.524+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 msgr2=0x7fd1980d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.524+0000 7fd19d400640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 0x7fd1980d01f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c780 tx=0x7fd19000cb80 comp rx=0 tx=0).stop 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.525+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 shutdown_connections 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.525+0000 7fd19d400640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd1780790f0 0x7fd17807b5b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.525+0000 7fd19d400640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd19815f5a0 0x7fd1980d01f0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.525+0000 7fd19d400640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980ccf00 0x7fd1980cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.525+0000 7fd19d400640 1 --2- 172.21.15.60:0/1915464511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980ad270 0x7fd1981f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.525+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 >> 172.21.15.60:0/1915464511 conn(0x7fd1980b1950 msgr2=0x7fd1980af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:10.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.525+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 shutdown_connections 2024-09-16T15:47:10.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.525+0000 7fd19d400640 1 -- 172.21.15.60:0/1915464511 wait complete. 2024-09-16T15:47:10.541 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:47:10.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:10.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:47:10.610 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:47:10.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:10.610 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:47:10.649 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:47:10.649 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:10.649 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:10.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:47:10.684 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:47:10.685 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:10.685 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:10.725 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:47:10.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:10.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.884+0000 7f1329200640 1 -- 172.21.15.60:0/1646490241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1324073fb0 msgr2=0x7f1324074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:10.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.884+0000 7f1329200640 1 --2- 172.21.15.60:0/1646490241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1324073fb0 0x7f1324074420 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f13240cfbb0 tx=0x7f131c02aa30 comp rx=0 tx=0).stop 2024-09-16T15:47:10.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.884+0000 7f1329200640 1 -- 172.21.15.60:0/1646490241 shutdown_connections 2024-09-16T15:47:10.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.884+0000 7f1329200640 1 --2- 172.21.15.60:0/1646490241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1324073fb0 0x7f1324074420 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.884+0000 7f1329200640 1 --2- 172.21.15.60:0/1646490241 >> v2:172.21.15.100:3300/0 conn(0x7f13240ccf00 0x7f13240b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.884+0000 7f1329200640 1 -- 172.21.15.60:0/1646490241 >> 172.21.15.60:0/1646490241 conn(0x7f13240b1950 msgr2=0x7f13240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:10.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.884+0000 7f1329200640 1 -- 172.21.15.60:0/1646490241 shutdown_connections 2024-09-16T15:47:10.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.884+0000 7f1329200640 1 -- 172.21.15.60:0/1646490241 wait complete. 2024-09-16T15:47:10.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.886+0000 7f1329200640 1 Processor -- start 2024-09-16T15:47:10.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.886+0000 7f1329200640 1 -- start start 2024-09-16T15:47:10.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1329200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1324073fb0 0x7f13241f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1329200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13240ccf00 0x7f13240cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1329200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f132415f5a0 0x7f13240b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1329200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f132406f2e0 con 0x7f132415f5a0 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1329200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f132406f160 con 0x7f13240ccf00 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1329200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f132406f460 con 0x7f1324073fb0 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1322800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f132415f5a0 0x7f13240b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1322800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f132415f5a0 0x7f13240b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52010/0 (socket says 172.21.15.60:52010) 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1322800640 1 -- 172.21.15.60:0/1664228838 learned_addr learned my addr 172.21.15.60:0/1664228838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f131be00640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13240ccf00 0x7f13240cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1321e00640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1324073fb0 0x7f13241f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1322800640 1 -- 172.21.15.60:0/1664228838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1324073fb0 msgr2=0x7f13241f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:10.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1322800640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1324073fb0 0x7f13241f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1322800640 1 -- 172.21.15.60:0/1664228838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13240ccf00 msgr2=0x7f13240cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:10.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1322800640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13240ccf00 0x7f13240cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:10.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.887+0000 7f1322800640 1 -- 172.21.15.60:0/1664228838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13241f5a20 con 0x7f132415f5a0 2024-09-16T15:47:10.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.888+0000 7f1322800640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f132415f5a0 0x7f13240b3a00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f131c02b020 tx=0x7f131c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:10.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.888+0000 7f131a000640 1 -- 172.21.15.60:0/1664228838 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c046070 con 0x7f132415f5a0 2024-09-16T15:47:10.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.888+0000 7f131a000640 1 -- 172.21.15.60:0/1664228838 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f131c004250 con 0x7f132415f5a0 2024-09-16T15:47:10.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.888+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13240cfc30 con 0x7f132415f5a0 2024-09-16T15:47:10.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.888+0000 7f131a000640 1 -- 172.21.15.60:0/1664228838 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c033050 con 0x7f132415f5a0 2024-09-16T15:47:10.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.888+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13240d01f0 con 0x7f132415f5a0 2024-09-16T15:47:10.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.889+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132406f9f0 con 0x7f132415f5a0 2024-09-16T15:47:10.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.890+0000 7f131a000640 1 -- 172.21.15.60:0/1664228838 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f131c049070 con 0x7f132415f5a0 2024-09-16T15:47:10.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.892+0000 7f131a000640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13000790c0 0x7f130007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:10.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.892+0000 7f131a000640 1 -- 172.21.15.60:0/1664228838 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f131c0bc690 con 0x7f132415f5a0 2024-09-16T15:47:10.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.892+0000 7f1321e00640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13000790c0 0x7f130007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:10.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.893+0000 7f1321e00640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13000790c0 0x7f130007b580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f13240b2fe0 tx=0x7f1314009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:10.897+0000 7f131a000640 1 -- 172.21.15.60:0/1664228838 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f131c083b40 con 0x7f132415f5a0 2024-09-16T15:47:11.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.130+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13240aff20 con 0x7f132415f5a0 2024-09-16T15:47:11.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.131+0000 7f131a000640 1 -- 172.21.15.60:0/1664228838 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5229 (secure 0 0 0) 0x7f131c088570 con 0x7f132415f5a0 2024-09-16T15:47:11.132 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:11.132 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":103,"btime":"2024-09-16T15:46:58:600780+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7133,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/822830116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":822830116},{"type":"v1","addr":"172.21.15.100:6834","nonce":822830116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/467713690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":467713690},{"type":"v1","addr":"172.21.15.100:6835","nonce":467713690}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":7981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2740321673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2740321673},{"type":"v1","addr":"172.21.15.60:6834","nonce":2740321673}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:48.320845+0000","modified":"2024-09-16T15:46:51.342321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2397719730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2397719730},{"type":"v1","addr":"172.21.15.60:6835","nonce":2397719730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8472,"qdb_cluster":[8472]},"id":11}]} 2024-09-16T15:47:11.133 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 103 2024-09-16T15:47:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.133+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13000790c0 msgr2=0x7f130007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.133+0000 7f1329200640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13000790c0 0x7f130007b580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f13240b2fe0 tx=0x7f1314009040 comp rx=0 tx=0).stop 2024-09-16T15:47:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.133+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f132415f5a0 msgr2=0x7f13240b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.133+0000 7f1329200640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f132415f5a0 0x7f13240b3a00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f131c02b020 tx=0x7f131c003f10 comp rx=0 tx=0).stop 2024-09-16T15:47:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.134+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 shutdown_connections 2024-09-16T15:47:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.134+0000 7f1329200640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f13000790c0 0x7f130007b580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.134+0000 7f1329200640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f132415f5a0 0x7f13240b3a00 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.134+0000 7f1329200640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13240ccf00 0x7f13240cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.134+0000 7f1329200640 1 --2- 172.21.15.60:0/1664228838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1324073fb0 0x7f13241f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.134+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 >> 172.21.15.60:0/1664228838 conn(0x7f13240b1950 msgr2=0x7f132409cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.134+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 shutdown_connections 2024-09-16T15:47:11.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.134+0000 7f1329200640 1 -- 172.21.15.60:0/1664228838 wait complete. 2024-09-16T15:47:11.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.310+0000 7f4943400640 1 -- 172.21.15.60:0/1613227301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c0ca4d0 msgr2=0x7f493c15e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.310+0000 7f4943400640 1 --2- 172.21.15.60:0/1613227301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c0ca4d0 0x7f493c15e390 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4928005fd0 tx=0x7f492802a540 comp rx=0 tx=0).stop 2024-09-16T15:47:11.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.310+0000 7f4943400640 1 -- 172.21.15.60:0/1613227301 shutdown_connections 2024-09-16T15:47:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.310+0000 7f4943400640 1 --2- 172.21.15.60:0/1613227301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c0ca4d0 0x7f493c15e390 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.310+0000 7f4943400640 1 --2- 172.21.15.60:0/1613227301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f493c06cb70 0x7f493c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.310+0000 7f4943400640 1 -- 172.21.15.60:0/1613227301 >> 172.21.15.60:0/1613227301 conn(0x7f493c0b2b90 msgr2=0x7f493c0b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.310+0000 7f4943400640 1 -- 172.21.15.60:0/1613227301 shutdown_connections 2024-09-16T15:47:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.310+0000 7f4943400640 1 -- 172.21.15.60:0/1613227301 wait complete. 2024-09-16T15:47:11.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.312+0000 7f4943400640 1 Processor -- start 2024-09-16T15:47:11.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.312+0000 7f4943400640 1 -- start start 2024-09-16T15:47:11.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4943400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c06cb70 0x7f493c0b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:11.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4943400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f493c0709e0 0x7f493c153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:11.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4943400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f493c0ca4d0 0x7f493c1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:11.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4943400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0cfd50 con 0x7f493c06cb70 2024-09-16T15:47:11.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4943400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f493c0cfbd0 con 0x7f493c0709e0 2024-09-16T15:47:11.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4943400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f493c0cfed0 con 0x7f493c0ca4d0 2024-09-16T15:47:11.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4940a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c06cb70 0x7f493c0b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4940a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c06cb70 0x7f493c0b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52034/0 (socket says 172.21.15.60:52034) 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4940a00640 1 -- 172.21.15.60:0/551241008 learned_addr learned my addr 172.21.15.60:0/551241008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4941400640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f493c0ca4d0 0x7f493c1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f493be00640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f493c0709e0 0x7f493c153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4940a00640 1 -- 172.21.15.60:0/551241008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f493c0ca4d0 msgr2=0x7f493c1f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4940a00640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f493c0ca4d0 0x7f493c1f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4940a00640 1 -- 172.21.15.60:0/551241008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f493c0709e0 msgr2=0x7f493c153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4940a00640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f493c0709e0 0x7f493c153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.313+0000 7f4940a00640 1 -- 172.21.15.60:0/551241008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1e9c40 con 0x7f493c06cb70 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.314+0000 7f4940a00640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c06cb70 0x7f493c0b6080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f493000c450 tx=0x7f493000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.314+0000 7f4938c00640 1 -- 172.21.15.60:0/551241008 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300052d0 con 0x7f493c06cb70 2024-09-16T15:47:11.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.314+0000 7f4938c00640 1 -- 172.21.15.60:0/551241008 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4930005470 con 0x7f493c06cb70 2024-09-16T15:47:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.314+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c0b67b0 con 0x7f493c06cb70 2024-09-16T15:47:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.314+0000 7f4938c00640 1 -- 172.21.15.60:0/551241008 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930012870 con 0x7f493c06cb70 2024-09-16T15:47:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.314+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c0b6d70 con 0x7f493c06cb70 2024-09-16T15:47:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.315+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493c0b3440 con 0x7f493c06cb70 2024-09-16T15:47:11.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.316+0000 7f4938c00640 1 -- 172.21.15.60:0/551241008 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f49300035f0 con 0x7f493c06cb70 2024-09-16T15:47:11.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.317+0000 7f4938c00640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f491c0790c0 0x7f491c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:11.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.317+0000 7f4938c00640 1 -- 172.21.15.60:0/551241008 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f493009b860 con 0x7f493c06cb70 2024-09-16T15:47:11.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.317+0000 7f493be00640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f491c0790c0 0x7f491c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:11.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.318+0000 7f493be00640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f491c0790c0 0x7f491c07b580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4928033000 tx=0x7f4928004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:11.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.321+0000 7f4938c00640 1 -- 172.21.15.60:0/551241008 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f493c0b3440 con 0x7f493c06cb70 2024-09-16T15:47:11.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.558+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f493c0d0380 con 0x7f493c06cb70 2024-09-16T15:47:11.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.559+0000 7f4938c00640 1 -- 172.21.15.60:0/551241008 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5229 (secure 0 0 0) 0x7f4930062e00 con 0x7f493c06cb70 2024-09-16T15:47:11.560 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:11.560 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":103,"btime":"2024-09-16T15:46:58:600780+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7133,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/822830116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":822830116},{"type":"v1","addr":"172.21.15.100:6834","nonce":822830116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/467713690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":467713690},{"type":"v1","addr":"172.21.15.100:6835","nonce":467713690}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":7981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2740321673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2740321673},{"type":"v1","addr":"172.21.15.60:6834","nonce":2740321673}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:48.320845+0000","modified":"2024-09-16T15:46:51.342321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2397719730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2397719730},{"type":"v1","addr":"172.21.15.60:6835","nonce":2397719730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8472,"qdb_cluster":[8472]},"id":11}]} 2024-09-16T15:47:11.560 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 103 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.561+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f491c0790c0 msgr2=0x7f491c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.561+0000 7f4943400640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f491c0790c0 0x7f491c07b580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4928033000 tx=0x7f4928004730 comp rx=0 tx=0).stop 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.561+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c06cb70 msgr2=0x7f493c0b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.561+0000 7f4943400640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c06cb70 0x7f493c0b6080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f493000c450 tx=0x7f493000c910 comp rx=0 tx=0).stop 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.562+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 shutdown_connections 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.562+0000 7f4943400640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f491c0790c0 0x7f491c07b580 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.562+0000 7f4943400640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f493c0ca4d0 0x7f493c1f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.562+0000 7f4943400640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f493c0709e0 0x7f493c153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.562+0000 7f4943400640 1 --2- 172.21.15.60:0/551241008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f493c06cb70 0x7f493c0b6080 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.562+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 >> 172.21.15.60:0/551241008 conn(0x7f493c0b2b90 msgr2=0x7f493c0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.562+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 shutdown_connections 2024-09-16T15:47:11.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.562+0000 7f4943400640 1 -- 172.21.15.60:0/551241008 wait complete. 2024-09-16T15:47:11.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:11.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.742+0000 7f4423e00640 1 -- 172.21.15.60:0/1458154634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4424074ca0 msgr2=0x7f44240cc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.742+0000 7f4423e00640 1 --2- 172.21.15.60:0/1458154634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4424074ca0 0x7f44240cc860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f440c009860 tx=0x7f440c02a5a0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.742+0000 7f4423e00640 1 -- 172.21.15.60:0/1458154634 shutdown_connections 2024-09-16T15:47:11.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.742+0000 7f4423e00640 1 --2- 172.21.15.60:0/1458154634 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4424074ca0 0x7f44240cc860 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.742+0000 7f4423e00640 1 -- 172.21.15.60:0/1458154634 >> 172.21.15.60:0/1458154634 conn(0x7f44240b1e90 msgr2=0x7f44240b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:11.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.742+0000 7f4423e00640 1 -- 172.21.15.60:0/1458154634 shutdown_connections 2024-09-16T15:47:11.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.742+0000 7f4423e00640 1 -- 172.21.15.60:0/1458154634 wait complete. 2024-09-16T15:47:11.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.743+0000 7f4423e00640 1 Processor -- start 2024-09-16T15:47:11.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.744+0000 7f4423e00640 1 -- start start 2024-09-16T15:47:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.744+0000 7f4423e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4424074ca0 0x7f44240c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.744+0000 7f4423e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240afde0 0x7f44240b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.744+0000 7f4423e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44240ce100 0x7f4424159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:11.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.744+0000 7f4423e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f44240b1000 con 0x7f44240ce100 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.744+0000 7f4423e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f44240b0e80 con 0x7f44240afde0 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.744+0000 7f4423e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f44241662b0 con 0x7f4424074ca0 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44240ce100 0x7f4424159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44240ce100 0x7f4424159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52036/0 (socket says 172.21.15.60:52036) 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 -- 172.21.15.60:0/2600348370 learned_addr learned my addr 172.21.15.60:0/2600348370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f441a000640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240afde0 0x7f44240b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4422a00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4424074ca0 0x7f44240c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 -- 172.21.15.60:0/2600348370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4424074ca0 msgr2=0x7f44240c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4424074ca0 0x7f44240c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 -- 172.21.15.60:0/2600348370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240afde0 msgr2=0x7f44240b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240afde0 0x7f44240b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 -- 172.21.15.60:0/2600348370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4424099170 con 0x7f44240ce100 2024-09-16T15:47:11.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.745+0000 7f4423400640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44240ce100 0x7f4424159cb0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f441c00cba0 tx=0x7f441c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:11.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.746+0000 7f441b400640 1 -- 172.21.15.60:0/2600348370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c016020 con 0x7f44240ce100 2024-09-16T15:47:11.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.746+0000 7f441b400640 1 -- 172.21.15.60:0/2600348370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f441c00ae50 con 0x7f44240ce100 2024-09-16T15:47:11.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.746+0000 7f441b400640 1 -- 172.21.15.60:0/2600348370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c012680 con 0x7f44240ce100 2024-09-16T15:47:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.746+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4424099450 con 0x7f44240ce100 2024-09-16T15:47:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.746+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4424099d00 con 0x7f44240ce100 2024-09-16T15:47:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.747+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44240cd9e0 con 0x7f44240ce100 2024-09-16T15:47:11.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.748+0000 7f441b400640 1 -- 172.21.15.60:0/2600348370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f441c019020 con 0x7f44240ce100 2024-09-16T15:47:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.750+0000 7f441b400640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44000790c0 0x7f440007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.750+0000 7f441b400640 1 -- 172.21.15.60:0/2600348370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f441c09c2a0 con 0x7f44240ce100 2024-09-16T15:47:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.750+0000 7f4422a00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44000790c0 0x7f440007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:11.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.750+0000 7f4422a00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44000790c0 0x7f440007b580 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f44240c65e0 tx=0x7f4414009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:11.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.754+0000 7f441b400640 1 -- 172.21.15.60:0/2600348370 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f441c020020 con 0x7f44240ce100 2024-09-16T15:47:11.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.952+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4424099970 con 0x7f44240ce100 2024-09-16T15:47:11.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.953+0000 7f441b400640 1 -- 172.21.15.60:0/2600348370 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16775 (secure 0 0 0) 0x7f441c063950 con 0x7f44240ce100 2024-09-16T15:47:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:11.955 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":149,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:49.882617+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:46.126734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:47.279485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:11.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44000790c0 msgr2=0x7f440007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44000790c0 0x7f440007b580 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f44240c65e0 tx=0x7f4414009040 comp rx=0 tx=0).stop 2024-09-16T15:47:11.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44240ce100 msgr2=0x7f4424159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:11.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44240ce100 0x7f4424159cb0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f441c00cba0 tx=0x7f441c00a430 comp rx=0 tx=0).stop 2024-09-16T15:47:11.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 shutdown_connections 2024-09-16T15:47:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f44000790c0 0x7f440007b580 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44240ce100 0x7f4424159cb0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44240afde0 0x7f44240b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 --2- 172.21.15.60:0/2600348370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4424074ca0 0x7f44240c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 >> 172.21.15.60:0/2600348370 conn(0x7f44240b1e90 msgr2=0x7f44240b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 shutdown_connections 2024-09-16T15:47:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:11.956+0000 7f4423e00640 1 -- 172.21.15.60:0/2600348370 wait complete. 2024-09-16T15:47:11.973 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:47:11.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:12.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.133+0000 7f2d77200640 1 -- 172.21.15.60:0/3067840202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d780cd5f0 msgr2=0x7f2d780cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.133+0000 7f2d77200640 1 --2- 172.21.15.60:0/3067840202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d780cd5f0 0x7f2d780cda10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f2d70005fd0 tx=0x7f2d7002aa90 comp rx=0 tx=0).stop 2024-09-16T15:47:12.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.133+0000 7f2d77200640 1 -- 172.21.15.60:0/3067840202 shutdown_connections 2024-09-16T15:47:12.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.133+0000 7f2d77200640 1 --2- 172.21.15.60:0/3067840202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d780cd5f0 0x7f2d780cda10 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.133+0000 7f2d77200640 1 --2- 172.21.15.60:0/3067840202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d780ad880 0x7f2d780adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.133+0000 7f2d77200640 1 -- 172.21.15.60:0/3067840202 >> 172.21.15.60:0/3067840202 conn(0x7f2d780cfa20 msgr2=0x7f2d780cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:12.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.133+0000 7f2d77200640 1 -- 172.21.15.60:0/3067840202 shutdown_connections 2024-09-16T15:47:12.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.133+0000 7f2d77200640 1 -- 172.21.15.60:0/3067840202 wait complete. 2024-09-16T15:47:12.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.135+0000 7f2d77200640 1 Processor -- start 2024-09-16T15:47:12.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.135+0000 7f2d77200640 1 -- start start 2024-09-16T15:47:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d77200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d7806fb90 0x7f2d780c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d77200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d780ad880 0x7f2d780c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d77200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d780cd5f0 0x7f2d781f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d77200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7815eda0 con 0x7f2d7806fb90 2024-09-16T15:47:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d77200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7815ec20 con 0x7f2d780cd5f0 2024-09-16T15:47:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d77200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7815ef20 con 0x7f2d780ad880 2024-09-16T15:47:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d7806fb90 0x7f2d780c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d7806fb90 0x7f2d780c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52058/0 (socket says 172.21.15.60:52058) 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75e00640 1 -- 172.21.15.60:0/3528808423 learned_addr learned my addr 172.21.15.60:0/3528808423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75400640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d780ad880 0x7f2d780c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d76800640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d780cd5f0 0x7f2d781f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75e00640 1 -- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d780ad880 msgr2=0x7f2d780c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75e00640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d780ad880 0x7f2d780c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75e00640 1 -- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d780cd5f0 msgr2=0x7f2d781f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75e00640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d780cd5f0 0x7f2d781f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75e00640 1 -- 172.21.15.60:0/3528808423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d780c7220 con 0x7f2d7806fb90 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.136+0000 7f2d75400640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d780ad880 0x7f2d780c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.137+0000 7f2d75e00640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d7806fb90 0x7f2d780c6af0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800c480 tx=0x7f2d6800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.137+0000 7f2d6e000640 1 -- 172.21.15.60:0/3528808423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d680052d0 con 0x7f2d7806fb90 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.137+0000 7f2d6e000640 1 -- 172.21.15.60:0/3528808423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d68005470 con 0x7f2d7806fb90 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.137+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d780c7500 con 0x7f2d7806fb90 2024-09-16T15:47:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.137+0000 7f2d6e000640 1 -- 172.21.15.60:0/3528808423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68012870 con 0x7f2d7806fb90 2024-09-16T15:47:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.137+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781f5b00 con 0x7f2d7806fb90 2024-09-16T15:47:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.138+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d780c7740 con 0x7f2d7806fb90 2024-09-16T15:47:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.139+0000 7f2d6e000640 1 -- 172.21.15.60:0/3528808423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f2d680035f0 con 0x7f2d7806fb90 2024-09-16T15:47:12.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.140+0000 7f2d6e000640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2d580790c0 0x7f2d5807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.140+0000 7f2d6e000640 1 -- 172.21.15.60:0/3528808423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f2d6809b6b0 con 0x7f2d7806fb90 2024-09-16T15:47:12.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.140+0000 7f2d75400640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2d580790c0 0x7f2d5807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.141+0000 7f2d75400640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2d580790c0 0x7f2d5807b580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2d7806fa10 tx=0x7f2d64005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.144+0000 7f2d6e000640 1 -- 172.21.15.60:0/3528808423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2d780c7740 con 0x7f2d7806fb90 2024-09-16T15:47:12.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.376+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d780adcd0 con 0x7f2d7806fb90 2024-09-16T15:47:12.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.377+0000 7f2d6e000640 1 -- 172.21.15.60:0/3528808423 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5229 (secure 0 0 0) 0x7f2d68062e60 con 0x7f2d7806fb90 2024-09-16T15:47:12.378 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:12.378 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":103,"btime":"2024-09-16T15:46:58:600780+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7133,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/822830116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":822830116},{"type":"v1","addr":"172.21.15.100:6834","nonce":822830116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/467713690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":467713690},{"type":"v1","addr":"172.21.15.100:6835","nonce":467713690}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":7981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2740321673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2740321673},{"type":"v1","addr":"172.21.15.60:6834","nonce":2740321673}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:48.320845+0000","modified":"2024-09-16T15:46:51.342321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2397719730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2397719730},{"type":"v1","addr":"172.21.15.60:6835","nonce":2397719730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8472,"qdb_cluster":[8472]},"id":11}]} 2024-09-16T15:47:12.378 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 103 2024-09-16T15:47:12.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2d580790c0 msgr2=0x7f2d5807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2d580790c0 0x7f2d5807b580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2d7806fa10 tx=0x7f2d64005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:12.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d7806fb90 msgr2=0x7f2d780c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d7806fb90 0x7f2d780c6af0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800c480 tx=0x7f2d6800c940 comp rx=0 tx=0).stop 2024-09-16T15:47:12.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 shutdown_connections 2024-09-16T15:47:12.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2d580790c0 0x7f2d5807b580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d780cd5f0 0x7f2d781f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d780ad880 0x7f2d780c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 --2- 172.21.15.60:0/3528808423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d7806fb90 0x7f2d780c6af0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 >> 172.21.15.60:0/3528808423 conn(0x7f2d780cfa20 msgr2=0x7f2d7815fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:12.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 shutdown_connections 2024-09-16T15:47:12.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.379+0000 7f2d77200640 1 -- 172.21.15.60:0/3528808423 wait complete. 2024-09-16T15:47:12.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:12.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.556+0000 7f5f3be00640 1 -- 172.21.15.60:0/1904425938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 msgr2=0x7f5f3c0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.556+0000 7f5f3be00640 1 --2- 172.21.15.60:0/1904425938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 0x7f5f3c0b5370 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0b6960 tx=0x7f5f2c02a580 comp rx=0 tx=0).stop 2024-09-16T15:47:12.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.556+0000 7f5f3be00640 1 -- 172.21.15.60:0/1904425938 shutdown_connections 2024-09-16T15:47:12.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.556+0000 7f5f3be00640 1 --2- 172.21.15.60:0/1904425938 >> v2:172.21.15.100:3300/0 conn(0x7f5f3c0b4050 0x7f5f3c0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.556+0000 7f5f3be00640 1 --2- 172.21.15.60:0/1904425938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 0x7f5f3c0b5370 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.556+0000 7f5f3be00640 1 -- 172.21.15.60:0/1904425938 >> 172.21.15.60:0/1904425938 conn(0x7f5f3c0d3df0 msgr2=0x7f5f3c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:12.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.556+0000 7f5f3be00640 1 -- 172.21.15.60:0/1904425938 shutdown_connections 2024-09-16T15:47:12.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.556+0000 7f5f3be00640 1 -- 172.21.15.60:0/1904425938 wait complete. 2024-09-16T15:47:12.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.557+0000 7f5f3be00640 1 Processor -- start 2024-09-16T15:47:12.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.557+0000 7f5f3be00640 1 -- start start 2024-09-16T15:47:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.558+0000 7f5f3be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c0b34b0 0x7f5f3c0b9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.558+0000 7f5f3be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c0b4050 0x7f5f3c1f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.558+0000 7f5f3be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 0x7f5f3c0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.558+0000 7f5f3be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c0b2040 con 0x7f5f3c161440 2024-09-16T15:47:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.558+0000 7f5f3be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c0b1ec0 con 0x7f5f3c0b34b0 2024-09-16T15:47:12.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.558+0000 7f5f3be00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3c0b21c0 con 0x7f5f3c0b4050 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 0x7f5f3c0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3aa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c0b34b0 0x7f5f3c0b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 0x7f5f3c0cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52076/0 (socket says 172.21.15.60:52076) 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 -- 172.21.15.60:0/2030309304 learned_addr learned my addr 172.21.15.60:0/2030309304 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f33e00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c0b4050 0x7f5f3c1f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 -- 172.21.15.60:0/2030309304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c0b4050 msgr2=0x7f5f3c1f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c0b4050 0x7f5f3c1f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 -- 172.21.15.60:0/2030309304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c0b34b0 msgr2=0x7f5f3c0b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c0b34b0 0x7f5f3c0b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 -- 172.21.15.60:0/2030309304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c1f8290 con 0x7f5f3c161440 2024-09-16T15:47:12.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.559+0000 7f5f3b400640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 0x7f5f3c0cd5b0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400eaa0 tx=0x7f5f3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:12.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.560+0000 7f5f33400640 1 -- 172.21.15.60:0/2030309304 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400cdb0 con 0x7f5f3c161440 2024-09-16T15:47:12.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.560+0000 7f5f33400640 1 -- 172.21.15.60:0/2030309304 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f34004c10 con 0x7f5f3c161440 2024-09-16T15:47:12.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.560+0000 7f5f33400640 1 -- 172.21.15.60:0/2030309304 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34005020 con 0x7f5f3c161440 2024-09-16T15:47:12.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.560+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3c1f8570 con 0x7f5f3c161440 2024-09-16T15:47:12.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.560+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3c1f44e0 con 0x7f5f3c161440 2024-09-16T15:47:12.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.561+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3c0b1770 con 0x7f5f3c161440 2024-09-16T15:47:12.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.562+0000 7f5f33400640 1 -- 172.21.15.60:0/2030309304 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f5f340051c0 con 0x7f5f3c161440 2024-09-16T15:47:12.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.563+0000 7f5f33400640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5f180790c0 0x7f5f1807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.563+0000 7f5f33400640 1 -- 172.21.15.60:0/2030309304 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f5f3409c580 con 0x7f5f3c161440 2024-09-16T15:47:12.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.563+0000 7f5f3aa00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5f180790c0 0x7f5f1807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.564+0000 7f5f3aa00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5f180790c0 0x7f5f1807b580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c033000 tx=0x7f5f2c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.567+0000 7f5f33400640 1 -- 172.21.15.60:0/2030309304 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5f34016030 con 0x7f5f3c161440 2024-09-16T15:47:12.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.806+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f3c0760b0 con 0x7f5f3c161440 2024-09-16T15:47:12.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.807+0000 7f5f33400640 1 -- 172.21.15.60:0/2030309304 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5229 (secure 0 0 0) 0x7f5f34063550 con 0x7f5f3c161440 2024-09-16T15:47:12.808 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:12.808 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":103,"btime":"2024-09-16T15:46:58:600780+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7133,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/822830116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":822830116},{"type":"v1","addr":"172.21.15.100:6834","nonce":822830116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/467713690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":467713690},{"type":"v1","addr":"172.21.15.100:6835","nonce":467713690}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":7981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2740321673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2740321673},{"type":"v1","addr":"172.21.15.60:6834","nonce":2740321673}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:48.320845+0000","modified":"2024-09-16T15:46:51.342321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8472":{"gid":8472,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2397719730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2397719730},{"type":"v1","addr":"172.21.15.60:6835","nonce":2397719730}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8472,"qdb_cluster":[8472]},"id":11}]} 2024-09-16T15:47:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 103 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5f180790c0 msgr2=0x7f5f1807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5f180790c0 0x7f5f1807b580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c033000 tx=0x7f5f2c004730 comp rx=0 tx=0).stop 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 msgr2=0x7f5f3c0cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 0x7f5f3c0cd5b0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400eaa0 tx=0x7f5f3400ef60 comp rx=0 tx=0).stop 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 shutdown_connections 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5f180790c0 0x7f5f1807b580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f3c161440 0x7f5f3c0cd5b0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f3c0b4050 0x7f5f3c1f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 --2- 172.21.15.60:0/2030309304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f3c0b34b0 0x7f5f3c0b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 >> 172.21.15.60:0/2030309304 conn(0x7f5f3c0d3df0 msgr2=0x7f5f3c15fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.809+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 shutdown_connections 2024-09-16T15:47:12.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.810+0000 7f5f3be00640 1 -- 172.21.15.60:0/2030309304 wait complete. 2024-09-16T15:47:12.826 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.990+0000 7f1815800640 1 -- 172.21.15.60:0/4209272666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18100709e0 msgr2=0x7f1810073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.990+0000 7f1815800640 1 --2- 172.21.15.60:0/4209272666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18100709e0 0x7f1810073af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1800009830 tx=0x7f180002a510 comp rx=0 tx=0).stop 2024-09-16T15:47:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.991+0000 7f1815800640 1 -- 172.21.15.60:0/4209272666 shutdown_connections 2024-09-16T15:47:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.991+0000 7f1815800640 1 --2- 172.21.15.60:0/4209272666 >> v2:172.21.15.100:3300/0 conn(0x7f18100ca4d0 0x7f181015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.991+0000 7f1815800640 1 --2- 172.21.15.60:0/4209272666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18100709e0 0x7f1810073af0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.991+0000 7f1815800640 1 -- 172.21.15.60:0/4209272666 >> 172.21.15.60:0/4209272666 conn(0x7f18100b2b90 msgr2=0x7f18100b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.991+0000 7f1815800640 1 -- 172.21.15.60:0/4209272666 shutdown_connections 2024-09-16T15:47:12.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.991+0000 7f1815800640 1 -- 172.21.15.60:0/4209272666 wait complete. 2024-09-16T15:47:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.992+0000 7f1815800640 1 Processor -- start 2024-09-16T15:47:12.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.992+0000 7f1815800640 1 -- start start 2024-09-16T15:47:12.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.993+0000 7f1815800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f181006cb70 0x7f18101f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.993+0000 7f1815800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100709e0 0x7f1810152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.993+0000 7f1815800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18100ca4d0 0x7f181015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.993+0000 7f1815800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f18100d0250 con 0x7f18100ca4d0 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.993+0000 7f1815800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18100d00d0 con 0x7f18100709e0 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.993+0000 7f1815800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f18100d03d0 con 0x7f181006cb70 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.993+0000 7f180f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18100ca4d0 0x7f181015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18100ca4d0 0x7f181015a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52082/0 (socket says 172.21.15.60:52082) 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180f400640 1 -- 172.21.15.60:0/1573805991 learned_addr learned my addr 172.21.15.60:0/1573805991 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f1807e00640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100709e0 0x7f1810152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180ea00640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f181006cb70 0x7f18101f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180f400640 1 -- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f181006cb70 msgr2=0x7f18101f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180f400640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f181006cb70 0x7f18101f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180f400640 1 -- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100709e0 msgr2=0x7f1810152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180f400640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100709e0 0x7f1810152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:12.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180f400640 1 -- 172.21.15.60:0/1573805991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18100b40e0 con 0x7f18100ca4d0 2024-09-16T15:47:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f1807e00640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100709e0 0x7f1810152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f180f400640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18100ca4d0 0x7f181015a3e0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f180800cbc0 tx=0x7f180800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f1806a00640 1 -- 172.21.15.60:0/1573805991 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808016020 con 0x7f18100ca4d0 2024-09-16T15:47:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f1806a00640 1 -- 172.21.15.60:0/1573805991 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f180800ae30 con 0x7f18100ca4d0 2024-09-16T15:47:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f1806a00640 1 -- 172.21.15.60:0/1573805991 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808012660 con 0x7f18100ca4d0 2024-09-16T15:47:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.994+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18100b43c0 con 0x7f18100ca4d0 2024-09-16T15:47:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.995+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18100b4980 con 0x7f18100ca4d0 2024-09-16T15:47:12.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.996+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181006d2f0 con 0x7f18100ca4d0 2024-09-16T15:47:12.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.997+0000 7f1806a00640 1 -- 172.21.15.60:0/1573805991 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f1808019020 con 0x7f18100ca4d0 2024-09-16T15:47:12.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.998+0000 7f1806a00640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17ec0790f0 0x7f17ec07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:12.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.998+0000 7f1806a00640 1 -- 172.21.15.60:0/1573805991 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7f1808099840 con 0x7f18100ca4d0 2024-09-16T15:47:12.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.998+0000 7f180ea00640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17ec0790f0 0x7f17ec07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:12.999+0000 7f180ea00640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17ec0790f0 0x7f17ec07b5b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1800034000 tx=0x7f1800004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:13.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.002+0000 7f1806a00640 1 -- 172.21.15.60:0/1573805991 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1808020020 con 0x7f18100ca4d0 2024-09-16T15:47:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.198+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18100b4b50 con 0x7f18100ca4d0 2024-09-16T15:47:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.199+0000 7f1806a00640 1 -- 172.21.15.60:0/1573805991 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16775 (secure 0 0 0) 0x7f1808063960 con 0x7f18100ca4d0 2024-09-16T15:47:13.200 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:13.200 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":149,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:46:49.882617+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:46.126734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:47.279485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:13.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17ec0790f0 msgr2=0x7f17ec07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17ec0790f0 0x7f17ec07b5b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1800034000 tx=0x7f1800004730 comp rx=0 tx=0).stop 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18100ca4d0 msgr2=0x7f181015a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18100ca4d0 0x7f181015a3e0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f180800cbc0 tx=0x7f180800a430 comp rx=0 tx=0).stop 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 shutdown_connections 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f17ec0790f0 0x7f17ec07b5b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18100ca4d0 0x7f181015a3e0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18100709e0 0x7f1810152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 --2- 172.21.15.60:0/1573805991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f181006cb70 0x7f18101f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 >> 172.21.15.60:0/1573805991 conn(0x7f18100b2b90 msgr2=0x7f18101574c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 shutdown_connections 2024-09-16T15:47:13.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.201+0000 7f1815800640 1 -- 172.21.15.60:0/1573805991 wait complete. 2024-09-16T15:47:13.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:47:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.380+0000 7fbfb7200640 1 -- 172.21.15.60:0/2451050158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 msgr2=0x7fbfb00cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.380+0000 7fbfb7200640 1 --2- 172.21.15.60:0/2451050158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 0x7fbfb00cccb0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005fd0 tx=0x7fbfa002a530 comp rx=0 tx=0).stop 2024-09-16T15:47:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.380+0000 7fbfb7200640 1 -- 172.21.15.60:0/2451050158 shutdown_connections 2024-09-16T15:47:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.380+0000 7fbfb7200640 1 --2- 172.21.15.60:0/2451050158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 0x7fbfb00cccb0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.380+0000 7fbfb7200640 1 --2- 172.21.15.60:0/2451050158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb00cc8d0 0x7fbfb015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.380+0000 7fbfb7200640 1 -- 172.21.15.60:0/2451050158 >> 172.21.15.60:0/2451050158 conn(0x7fbfb00d0b30 msgr2=0x7fbfb00d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.380+0000 7fbfb7200640 1 -- 172.21.15.60:0/2451050158 shutdown_connections 2024-09-16T15:47:13.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.380+0000 7fbfb7200640 1 -- 172.21.15.60:0/2451050158 wait complete. 2024-09-16T15:47:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.381+0000 7fbfb7200640 1 Processor -- start 2024-09-16T15:47:13.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.381+0000 7fbfb7200640 1 -- start start 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.382+0000 7fbfb7200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb0073ee0 0x7fbfb015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.382+0000 7fbfb7200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb00cc8d0 0x7fbfb00b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.382+0000 7fbfb7200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 0x7fbfb00c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.382+0000 7fbfb7200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0074390 con 0x7fbfb00d1620 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.382+0000 7fbfb7200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01565e0 con 0x7fbfb00cc8d0 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.382+0000 7fbfb7200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb0074510 con 0x7fbfb0073ee0 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 0x7fbfb00c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 0x7fbfb00c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52112/0 (socket says 172.21.15.60:52112) 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 -- 172.21.15.60:0/1708804019 learned_addr learned my addr 172.21.15.60:0/1708804019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb5400640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb00cc8d0 0x7fbfb00b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb5e00640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb0073ee0 0x7fbfb015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 -- 172.21.15.60:0/1708804019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb0073ee0 msgr2=0x7fbfb015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb0073ee0 0x7fbfb015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 -- 172.21.15.60:0/1708804019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb00cc8d0 msgr2=0x7fbfb00b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb00cc8d0 0x7fbfb00b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 -- 172.21.15.60:0/1708804019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb01f8210 con 0x7fbfb00d1620 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.383+0000 7fbfb6800640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 0x7fbfb00c5e80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c7b0 tx=0x7fbfac00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.384+0000 7fbfaa000640 1 -- 172.21.15.60:0/1708804019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004960 con 0x7fbfb00d1620 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.384+0000 7fbfaa000640 1 -- 172.21.15.60:0/1708804019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfac004bf0 con 0x7fbfb00d1620 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.384+0000 7fbfaa000640 1 -- 172.21.15.60:0/1708804019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00a850 con 0x7fbfb00d1620 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.384+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb01f84b0 con 0x7fbfb00d1620 2024-09-16T15:47:13.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.384+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb01f8a70 con 0x7fbfb00d1620 2024-09-16T15:47:13.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.385+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb0070080 con 0x7fbfb00d1620 2024-09-16T15:47:13.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.386+0000 7fbfaa000640 1 -- 172.21.15.60:0/1708804019 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fbfac00a9f0 con 0x7fbfb00d1620 2024-09-16T15:47:13.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.387+0000 7fbfaa000640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbf940790f0 0x7fbf9407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:13.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.387+0000 7fbfaa000640 1 -- 172.21.15.60:0/1708804019 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5737+0+0 (secure 0 0 0) 0x7fbfac09bd10 con 0x7fbfb00d1620 2024-09-16T15:47:13.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.387+0000 7fbfb5e00640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbf940790f0 0x7fbf9407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.388+0000 7fbfb5e00640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbf940790f0 0x7fbf9407b5b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009990 tx=0x7fbfa4009110 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:13.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.391+0000 7fbfaa000640 1 -- 172.21.15.60:0/1708804019 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbfac020070 con 0x7fbfb00d1620 2024-09-16T15:47:13.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.623+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbfb00cb0f0 con 0x7fbfb00d1620 2024-09-16T15:47:13.625 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:47:13.624+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:47:13.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.626+0000 7fbfaa000640 1 -- 172.21.15.60:0/1708804019 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) ==== 161+0+0 (secure 0 0 0) 0x7fbfac063330 con 0x7fbfb00d1620 2024-09-16T15:47:13.627 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:47:13.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.629+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbf940790f0 msgr2=0x7fbf9407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.629+0000 7fbfb7200640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbf940790f0 0x7fbf9407b5b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009990 tx=0x7fbfa4009110 comp rx=0 tx=0).stop 2024-09-16T15:47:13.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.629+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 msgr2=0x7fbfb00c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.629+0000 7fbfb7200640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 0x7fbfb00c5e80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c7b0 tx=0x7fbfac00cbb0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.630+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 shutdown_connections 2024-09-16T15:47:13.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.630+0000 7fbfb7200640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbf940790f0 0x7fbf9407b5b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.630+0000 7fbfb7200640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbfb00d1620 0x7fbfb00c5e80 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.630+0000 7fbfb7200640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbfb00cc8d0 0x7fbfb00b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.630+0000 7fbfb7200640 1 --2- 172.21.15.60:0/1708804019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbfb0073ee0 0x7fbfb015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.630+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 >> 172.21.15.60:0/1708804019 conn(0x7fbfb00d0b30 msgr2=0x7fbfb0152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:13.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.630+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 shutdown_connections 2024-09-16T15:47:13.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.630+0000 7fbfb7200640 1 -- 172.21.15.60:0/1708804019 wait complete. 2024-09-16T15:47:13.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:47:13.697 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.812+0000 7f7bd9c00640 1 -- 172.21.15.60:0/2169417271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 msgr2=0x7f7bd4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.812+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/2169417271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 0x7f7bd4073440 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40cc550 tx=0x7f7bc402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.812+0000 7f7bd9c00640 1 -- 172.21.15.60:0/2169417271 shutdown_connections 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.812+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/2169417271 >> v2:172.21.15.100:3300/0 conn(0x7f7bd40ccb00 0x7f7bd415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.812+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/2169417271 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bd406efa0 0x7f7bd40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.812+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/2169417271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 0x7f7bd4073440 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.812+0000 7f7bd9c00640 1 -- 172.21.15.60:0/2169417271 >> 172.21.15.60:0/2169417271 conn(0x7f7bd40b1840 msgr2=0x7f7bd40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.812+0000 7f7bd9c00640 1 -- 172.21.15.60:0/2169417271 shutdown_connections 2024-09-16T15:47:13.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.813+0000 7f7bd9c00640 1 -- 172.21.15.60:0/2169417271 wait complete. 2024-09-16T15:47:13.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.814+0000 7f7bd9c00640 1 Processor -- start 2024-09-16T15:47:13.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.814+0000 7f7bd9c00640 1 -- start start 2024-09-16T15:47:13.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd9c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bd406efa0 0x7f7bd40c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd9c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bd40ccb00 0x7f7bd4140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd9c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 0x7f7bd415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd9c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd41600b0 con 0x7f7bd4145750 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd9c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd415ff30 con 0x7f7bd40ccb00 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd9c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd4160230 con 0x7f7bd406efa0 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 0x7f7bd415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 0x7f7bd415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52138/0 (socket says 172.21.15.60:52138) 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.815+0000 7f7bd3400640 1 -- 172.21.15.60:0/1637689827 learned_addr learned my addr 172.21.15.60:0/1637689827 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bca000640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bd40ccb00 0x7f7bd4140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd2a00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bd406efa0 0x7f7bd40c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd3400640 1 -- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bd406efa0 msgr2=0x7f7bd40c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd3400640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bd406efa0 0x7f7bd40c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd3400640 1 -- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bd40ccb00 msgr2=0x7f7bd4140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd3400640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bd40ccb00 0x7f7bd4140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd3400640 1 -- 172.21.15.60:0/1637689827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd40cbe70 con 0x7f7bd4145750 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bca000640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bd40ccb00 0x7f7bd4140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd2a00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bd406efa0 0x7f7bd40c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd3400640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 0x7f7bd415d090 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00cce0 tx=0x7f7bcc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bcb400640 1 -- 172.21.15.60:0/1637689827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc00f450 con 0x7f7bd4145750 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bcb400640 1 -- 172.21.15.60:0/1637689827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bcc00f700 con 0x7f7bd4145750 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bcb400640 1 -- 172.21.15.60:0/1637689827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc00fb50 con 0x7f7bd4145750 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.816+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd40cc110 con 0x7f7bd4145750 2024-09-16T15:47:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.817+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd40c5840 con 0x7f7bd4145750 2024-09-16T15:47:13.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.818+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd40c5a40 con 0x7f7bd4145750 2024-09-16T15:47:13.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.819+0000 7f7bcb400640 1 -- 172.21.15.60:0/1637689827 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f7bcc00fcf0 con 0x7f7bd4145750 2024-09-16T15:47:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.820+0000 7f7bcb400640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bb00790c0 0x7f7bb007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.820+0000 7f7bcb400640 1 -- 172.21.15.60:0/1637689827 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5823+0+0 (secure 0 0 0) 0x7f7bcc09c9d0 con 0x7f7bd4145750 2024-09-16T15:47:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.820+0000 7f7bd2a00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bb00790c0 0x7f7bb007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.821+0000 7f7bd2a00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bb00790c0 0x7f7bb007b580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4033000 tx=0x7f7bc4004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:13.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:13.824+0000 7f7bcb400640 1 -- 172.21.15.60:0/1637689827 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7bd40c5a40 con 0x7f7bd4145750 2024-09-16T15:47:14.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.064+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7bd4160710 con 0x7f7bd4145750 2024-09-16T15:47:14.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.654+0000 7f7bcb400640 1 -- 172.21.15.60:0/1637689827 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v105) ==== 104+0+0 (secure 0 0 0) 0x7f7bcc014030 con 0x7f7bd4145750 2024-09-16T15:47:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bb00790c0 msgr2=0x7f7bb007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bb00790c0 0x7f7bb007b580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4033000 tx=0x7f7bc4004730 comp rx=0 tx=0).stop 2024-09-16T15:47:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 msgr2=0x7f7bd415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 0x7f7bd415d090 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00cce0 tx=0x7f7bcc00a620 comp rx=0 tx=0).stop 2024-09-16T15:47:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 shutdown_connections 2024-09-16T15:47:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7bb00790c0 0x7f7bb007b580 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bd4145750 0x7f7bd415d090 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bd40ccb00 0x7f7bd4140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 --2- 172.21.15.60:0/1637689827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bd406efa0 0x7f7bd40c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 >> 172.21.15.60:0/1637689827 conn(0x7f7bd40b1840 msgr2=0x7f7bd40cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 shutdown_connections 2024-09-16T15:47:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.657+0000 7f7bd9c00640 1 -- 172.21.15.60:0/1637689827 wait complete. 2024-09-16T15:47:14.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:47:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.847+0000 7fcb9be00640 1 -- 172.21.15.60:0/3962275454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c15abd0 msgr2=0x7fcb9c072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:14.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.847+0000 7fcb9be00640 1 --2- 172.21.15.60:0/3962275454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c15abd0 0x7fcb9c072d00 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005fd0 tx=0x7fcb8c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:47:14.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.847+0000 7fcb9be00640 1 -- 172.21.15.60:0/3962275454 shutdown_connections 2024-09-16T15:47:14.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.847+0000 7fcb9be00640 1 --2- 172.21.15.60:0/3962275454 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c15abd0 0x7fcb9c072d00 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:14.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.847+0000 7fcb9be00640 1 -- 172.21.15.60:0/3962275454 >> 172.21.15.60:0/3962275454 conn(0x7fcb9c0cfa70 msgr2=0x7fcb9c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:14.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.847+0000 7fcb9be00640 1 -- 172.21.15.60:0/3962275454 shutdown_connections 2024-09-16T15:47:14.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.847+0000 7fcb9be00640 1 -- 172.21.15.60:0/3962275454 wait complete. 2024-09-16T15:47:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.849+0000 7fcb9be00640 1 Processor -- start 2024-09-16T15:47:14.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.849+0000 7fcb9be00640 1 -- start start 2024-09-16T15:47:14.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c073cf0 0x7fcb9c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb9c15abd0 0x7fcb9c0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c0c32d0 0x7fcb9c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c0cd9d0 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c0cd850 con 0x7fcb9c073cf0 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9be00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9c0cdb50 con 0x7fcb9c15abd0 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c0c32d0 0x7fcb9c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c0c32d0 0x7fcb9c0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52152/0 (socket says 172.21.15.60:52152) 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9b400640 1 -- 172.21.15.60:0/1623292136 learned_addr learned my addr 172.21.15.60:0/1623292136 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9aa00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c073cf0 0x7fcb9c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9a000640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb9c15abd0 0x7fcb9c0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9b400640 1 -- 172.21.15.60:0/1623292136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb9c15abd0 msgr2=0x7fcb9c0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.850+0000 7fcb9b400640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb9c15abd0 0x7fcb9c0c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:14.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb9b400640 1 -- 172.21.15.60:0/1623292136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c073cf0 msgr2=0x7fcb9c155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:14.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb9b400640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c073cf0 0x7fcb9c155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:14.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb9b400640 1 -- 172.21.15.60:0/1623292136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c0c42e0 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb9b400640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c0c32d0 0x7fcb9c0b05f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c7d0 tx=0x7fcb9400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb92a00640 1 -- 172.21.15.60:0/1623292136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94003b70 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb92a00640 1 -- 172.21.15.60:0/1623292136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb94003d10 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c0c3d20 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb92a00640 1 -- 172.21.15.60:0/1623292136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400aab0 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.851+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c0c4ac0 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.852+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9c0cdfd0 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.853+0000 7fcb92a00640 1 -- 172.21.15.60:0/1623292136 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fcb9400ac50 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.855+0000 7fcb92a00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcb7c0790c0 0x7fcb7c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:14.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.855+0000 7fcb92a00640 1 -- 172.21.15.60:0/1623292136 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5823+0+0 (secure 0 0 0) 0x7fcb9409b990 con 0x7fcb9c0c32d0 2024-09-16T15:47:14.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.855+0000 7fcb9aa00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcb7c0790c0 0x7fcb7c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:14.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.856+0000 7fcb9aa00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcb7c0790c0 0x7fcb7c07b580 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c033000 tx=0x7fcb8c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:14.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:14.859+0000 7fcb92a00640 1 -- 172.21.15.60:0/1623292136 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcb94062910 con 0x7fcb9c0c32d0 2024-09-16T15:47:15.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.057+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcb9c0cff80 con 0x7fcb9c0c32d0 2024-09-16T15:47:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.656+0000 7fcb92a00640 1 -- 172.21.15.60:0/1623292136 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) ==== 181+0+0 (secure 0 0 0) 0x7fcb94067340 con 0x7fcb9c0c32d0 2024-09-16T15:47:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:47:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.660+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcb7c0790c0 msgr2=0x7fcb7c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.660+0000 7fcb9be00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcb7c0790c0 0x7fcb7c07b580 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c033000 tx=0x7fcb8c004730 comp rx=0 tx=0).stop 2024-09-16T15:47:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.660+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c0c32d0 msgr2=0x7fcb9c0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.660+0000 7fcb9be00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c0c32d0 0x7fcb9c0b05f0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c7d0 tx=0x7fcb9400cc90 comp rx=0 tx=0).stop 2024-09-16T15:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.661+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 shutdown_connections 2024-09-16T15:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.661+0000 7fcb9be00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fcb7c0790c0 0x7fcb7c07b580 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.661+0000 7fcb9be00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb9c0c32d0 0x7fcb9c0b05f0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.661+0000 7fcb9be00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb9c15abd0 0x7fcb9c0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.661+0000 7fcb9be00640 1 --2- 172.21.15.60:0/1623292136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb9c073cf0 0x7fcb9c155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.661+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 >> 172.21.15.60:0/1623292136 conn(0x7fcb9c0cfa70 msgr2=0x7fcb9c159400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.661+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 shutdown_connections 2024-09-16T15:47:15.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.661+0000 7fcb9be00640 1 -- 172.21.15.60:0/1623292136 wait complete. 2024-09-16T15:47:15.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:47:15.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.851+0000 7ff11a400640 1 -- 172.21.15.60:0/268816912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0f8005210 msgr2=0x7ff0f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:15.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.851+0000 7ff11a400640 1 --2- 172.21.15.60:0/268816912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0f8005210 0x7ff0f800b9c0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7ff10c001d90 tx=0x7ff0f8005830 comp rx=0 tx=0).stop 2024-09-16T15:47:15.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.851+0000 7ff11a400640 1 -- 172.21.15.60:0/268816912 shutdown_connections 2024-09-16T15:47:15.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.851+0000 7ff11a400640 1 --2- 172.21.15.60:0/268816912 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0f8005210 0x7ff0f800b9c0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:15.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.851+0000 7ff11a400640 1 -- 172.21.15.60:0/268816912 >> 172.21.15.60:0/268816912 conn(0x7ff1140cfa70 msgr2=0x7ff1140cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:15.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.851+0000 7ff11a400640 1 -- 172.21.15.60:0/268816912 shutdown_connections 2024-09-16T15:47:15.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.851+0000 7ff11a400640 1 -- 172.21.15.60:0/268816912 wait complete. 2024-09-16T15:47:15.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.853+0000 7ff11a400640 1 Processor -- start 2024-09-16T15:47:15.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.853+0000 7ff11a400640 1 -- start start 2024-09-16T15:47:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff11a400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0f8005210 0x7ff1141f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff11a400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff11415f290 0x7ff1140c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff11a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff11415d8e0 0x7ff11414f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff11a400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff1140701d0 con 0x7ff11415d8e0 2024-09-16T15:47:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff11a400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff114070050 con 0x7ff11415f290 2024-09-16T15:47:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff11a400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff114073010 con 0x7ff0f8005210 2024-09-16T15:47:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff11415d8e0 0x7ff11414f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff11415d8e0 0x7ff11414f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52170/0 (socket says 172.21.15.60:52170) 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119a00640 1 -- 172.21.15.60:0/1390771778 learned_addr learned my addr 172.21.15.60:0/1390771778 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119000640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0f8005210 0x7ff1141f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff10be00640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff11415f290 0x7ff1140c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119a00640 1 -- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0f8005210 msgr2=0x7ff1141f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119a00640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0f8005210 0x7ff1141f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119a00640 1 -- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff11415f290 msgr2=0x7ff1140c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119a00640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff11415f290 0x7ff1140c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff119a00640 1 -- 172.21.15.60:0/1390771778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11415e1f0 con 0x7ff11415d8e0 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.854+0000 7ff10be00640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff11415f290 0x7ff1140c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.855+0000 7ff119a00640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff11415d8e0 0x7ff11414f9c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff10c001d90 tx=0x7ff10c006d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.855+0000 7ff111600640 1 -- 172.21.15.60:0/1390771778 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c003d80 con 0x7ff11415d8e0 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.855+0000 7ff111600640 1 -- 172.21.15.60:0/1390771778 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff10c005b80 con 0x7ff11415d8e0 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.855+0000 7ff111600640 1 -- 172.21.15.60:0/1390771778 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c005fd0 con 0x7ff11415d8e0 2024-09-16T15:47:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.855+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11415e4d0 con 0x7ff11415d8e0 2024-09-16T15:47:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.855+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11415ea90 con 0x7ff11415d8e0 2024-09-16T15:47:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.856+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff114073570 con 0x7ff11415d8e0 2024-09-16T15:47:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.857+0000 7ff111600640 1 -- 172.21.15.60:0/1390771778 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7ff10c005030 con 0x7ff11415d8e0 2024-09-16T15:47:15.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.858+0000 7ff111600640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff0f00790c0 0x7ff0f007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:15.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.858+0000 7ff111600640 1 -- 172.21.15.60:0/1390771778 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5385+0+0 (secure 0 0 0) 0x7ff10c0bb200 con 0x7ff11415d8e0 2024-09-16T15:47:15.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.859+0000 7ff119000640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff0f00790c0 0x7ff0f007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:15.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.859+0000 7ff119000640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff0f00790c0 0x7ff0f007b580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff104001cc0 tx=0x7ff104006720 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:15.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:15.863+0000 7ff111600640 1 -- 172.21.15.60:0/1390771778 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff10c082340 con 0x7ff11415d8e0 2024-09-16T15:47:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.058+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff1140b0560 con 0x7ff11415d8e0 2024-09-16T15:47:16.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.668+0000 7ff111600640 1 -- 172.21.15.60:0/1390771778 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) ==== 169+0+0 (secure 0 0 0) 0x7ff10c086d70 con 0x7ff11415d8e0 2024-09-16T15:47:16.670 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:47:16.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff0f00790c0 msgr2=0x7ff0f007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:16.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff0f00790c0 0x7ff0f007b580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff104001cc0 tx=0x7ff104006720 comp rx=0 tx=0).stop 2024-09-16T15:47:16.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff11415d8e0 msgr2=0x7ff11414f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:16.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff11415d8e0 0x7ff11414f9c0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7ff10c001d90 tx=0x7ff10c006d40 comp rx=0 tx=0).stop 2024-09-16T15:47:16.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 shutdown_connections 2024-09-16T15:47:16.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff0f00790c0 0x7ff0f007b580 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:16.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff11415d8e0 0x7ff11414f9c0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:16.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff11415f290 0x7ff1140c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:16.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 --2- 172.21.15.60:0/1390771778 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0f8005210 0x7ff1141f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:16.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.673+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 >> 172.21.15.60:0/1390771778 conn(0x7ff1140cfa70 msgr2=0x7ff1140ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:16.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.674+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 shutdown_connections 2024-09-16T15:47:16.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.674+0000 7ff11a400640 1 -- 172.21.15.60:0/1390771778 wait complete. 2024-09-16T15:47:16.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:47:16.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.862+0000 7ffb55800640 1 -- 172.21.15.60:0/1550942192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb501445d0 msgr2=0x7ffb500cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:16.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.862+0000 7ffb55800640 1 --2- 172.21.15.60:0/1550942192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb501445d0 0x7ffb500cebf0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005fd0 tx=0x7ffb4002a610 comp rx=0 tx=0).stop 2024-09-16T15:47:16.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.862+0000 7ffb55800640 1 -- 172.21.15.60:0/1550942192 shutdown_connections 2024-09-16T15:47:16.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.862+0000 7ffb55800640 1 --2- 172.21.15.60:0/1550942192 >> v2:172.21.15.100:3300/0 conn(0x7ffb500ccfd0 0x7ffb500cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:16.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.862+0000 7ffb55800640 1 --2- 172.21.15.60:0/1550942192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb501445d0 0x7ffb500cebf0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:16.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.862+0000 7ffb55800640 1 -- 172.21.15.60:0/1550942192 >> 172.21.15.60:0/1550942192 conn(0x7ffb500b1950 msgr2=0x7ffb500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:16.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.863+0000 7ffb55800640 1 -- 172.21.15.60:0/1550942192 shutdown_connections 2024-09-16T15:47:16.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.863+0000 7ffb55800640 1 -- 172.21.15.60:0/1550942192 wait complete. 2024-09-16T15:47:16.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.864+0000 7ffb55800640 1 Processor -- start 2024-09-16T15:47:16.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.864+0000 7ffb55800640 1 -- start start 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.865+0000 7ffb55800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb500b4ca0 0x7ffb5013e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.865+0000 7ffb55800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb500ccfd0 0x7ffb5015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.865+0000 7ffb55800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb501445d0 0x7ffb500ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.865+0000 7ffb55800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5015f080 con 0x7ffb500b4ca0 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.865+0000 7ffb55800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffb50073bc0 con 0x7ffb501445d0 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.865+0000 7ffb55800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5015f200 con 0x7ffb500ccfd0 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.865+0000 7ffb4ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb500b4ca0 0x7ffb5013e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb500b4ca0 0x7ffb5013e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52186/0 (socket says 172.21.15.60:52186) 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4ea00640 1 -- 172.21.15.60:0/340159590 learned_addr learned my addr 172.21.15.60:0/340159590 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb46000640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb500ccfd0 0x7ffb5015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4f400640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb501445d0 0x7ffb500ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4ea00640 1 -- 172.21.15.60:0/340159590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb500ccfd0 msgr2=0x7ffb5015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4ea00640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb500ccfd0 0x7ffb5015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4ea00640 1 -- 172.21.15.60:0/340159590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb501445d0 msgr2=0x7ffb500ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4ea00640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb501445d0 0x7ffb500ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4ea00640 1 -- 172.21.15.60:0/340159590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5013eb90 con 0x7ffb500b4ca0 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.866+0000 7ffb4ea00640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb500b4ca0 0x7ffb5013e650 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ffb40034040 tx=0x7ffb400038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.867+0000 7ffb47400640 1 -- 172.21.15.60:0/340159590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4003e030 con 0x7ffb500b4ca0 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.867+0000 7ffb47400640 1 -- 172.21.15.60:0/340159590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb40003ca0 con 0x7ffb500b4ca0 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.867+0000 7ffb47400640 1 -- 172.21.15.60:0/340159590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40035040 con 0x7ffb500b4ca0 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.867+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5015b840 con 0x7ffb500b4ca0 2024-09-16T15:47:16.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.867+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb501f3400 con 0x7ffb500b4ca0 2024-09-16T15:47:16.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.868+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb501f2f50 con 0x7ffb500b4ca0 2024-09-16T15:47:16.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.869+0000 7ffb47400640 1 -- 172.21.15.60:0/340159590 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7ffb40041030 con 0x7ffb500b4ca0 2024-09-16T15:47:16.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.871+0000 7ffb47400640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb2c0790c0 0x7ffb2c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:16.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.871+0000 7ffb47400640 1 -- 172.21.15.60:0/340159590 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7ffb400bbcc0 con 0x7ffb500b4ca0 2024-09-16T15:47:16.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.871+0000 7ffb46000640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb2c0790c0 0x7ffb2c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:16.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.871+0000 7ffb46000640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb2c0790c0 0x7ffb2c07b580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffb5006add0 tx=0x7ffb38009110 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:16.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:16.875+0000 7ffb47400640 1 -- 172.21.15.60:0/340159590 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffb5013eb90 con 0x7ffb500b4ca0 2024-09-16T15:47:17.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.072+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7ffb50070a20 con 0x7ffb500b4ca0 2024-09-16T15:47:17.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.076+0000 7ffb47400640 1 -- 172.21.15.60:0/340159590 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v45) ==== 71+0+0 (secure 0 0 0) 0x7ffb40083020 con 0x7ffb500b4ca0 2024-09-16T15:47:17.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb2c0790c0 msgr2=0x7ffb2c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb2c0790c0 0x7ffb2c07b580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffb5006add0 tx=0x7ffb38009110 comp rx=0 tx=0).stop 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb500b4ca0 msgr2=0x7ffb5013e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb500b4ca0 0x7ffb5013e650 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ffb40034040 tx=0x7ffb400038a0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 shutdown_connections 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ffb2c0790c0 0x7ffb2c07b580 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb501445d0 0x7ffb500ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb500ccfd0 0x7ffb5015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 --2- 172.21.15.60:0/340159590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb500b4ca0 0x7ffb5013e650 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 >> 172.21.15.60:0/340159590 conn(0x7ffb500b1950 msgr2=0x7ffb500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 shutdown_connections 2024-09-16T15:47:17.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.078+0000 7ffb55800640 1 -- 172.21.15.60:0/340159590 wait complete. 2024-09-16T15:47:17.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-16T15:47:17.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.265+0000 7f280b400640 1 -- 172.21.15.60:0/2512036984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28040ccf00 msgr2=0x7f28040b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.265+0000 7f280b400640 1 --2- 172.21.15.60:0/2512036984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28040ccf00 0x7f28040b2440 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f27ec005fd0 tx=0x7f27ec02a540 comp rx=0 tx=0).stop 2024-09-16T15:47:17.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.265+0000 7f280b400640 1 -- 172.21.15.60:0/2512036984 shutdown_connections 2024-09-16T15:47:17.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.265+0000 7f280b400640 1 --2- 172.21.15.60:0/2512036984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28040ccf00 0x7f28040b2440 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.265+0000 7f280b400640 1 --2- 172.21.15.60:0/2512036984 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2804073fb0 0x7f2804074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.265+0000 7f280b400640 1 -- 172.21.15.60:0/2512036984 >> 172.21.15.60:0/2512036984 conn(0x7f28040b1950 msgr2=0x7f28040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:17.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.265+0000 7f280b400640 1 -- 172.21.15.60:0/2512036984 shutdown_connections 2024-09-16T15:47:17.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.265+0000 7f280b400640 1 -- 172.21.15.60:0/2512036984 wait complete. 2024-09-16T15:47:17.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.267+0000 7f280b400640 1 Processor -- start 2024-09-16T15:47:17.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.267+0000 7f280b400640 1 -- start start 2024-09-16T15:47:17.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f280b400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804073fb0 0x7f28041efd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:17.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f280b400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28040ccf00 0x7f2804156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:17.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f280b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280415f5a0 0x7f28041565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:17.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f280b400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f280406f2e0 con 0x7f280415f5a0 2024-09-16T15:47:17.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f280b400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f280406f160 con 0x7f2804073fb0 2024-09-16T15:47:17.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f280b400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f280406f460 con 0x7f28040ccf00 2024-09-16T15:47:17.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f2809400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280415f5a0 0x7f28041565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:17.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f2808a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804073fb0 0x7f28041efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f2808a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804073fb0 0x7f28041efd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41304/0 (socket says 172.21.15.60:41304) 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f2808a00640 1 -- 172.21.15.60:0/1275381228 learned_addr learned my addr 172.21.15.60:0/1275381228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.268+0000 7f27fbe00640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28040ccf00 0x7f2804156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2808a00640 1 -- 172.21.15.60:0/1275381228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28040ccf00 msgr2=0x7f2804156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2808a00640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28040ccf00 0x7f2804156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2808a00640 1 -- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280415f5a0 msgr2=0x7f28041565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2808a00640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280415f5a0 0x7f28041565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2808a00640 1 -- 172.21.15.60:0/1275381228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28041f02c0 con 0x7f2804073fb0 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2809400640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280415f5a0 0x7f28041565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:47:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2808a00640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804073fb0 0x7f28041efd80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f27f400c3a0 tx=0x7f27f400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:17.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2801600640 1 -- 172.21.15.60:0/1275381228 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4017070 con 0x7f2804073fb0 2024-09-16T15:47:17.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2801600640 1 -- 172.21.15.60:0/1275381228 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27f40053c0 con 0x7f2804073fb0 2024-09-16T15:47:17.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f2801600640 1 -- 172.21.15.60:0/1275381228 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4003850 con 0x7f2804073fb0 2024-09-16T15:47:17.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.269+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28040cfdc0 con 0x7f2804073fb0 2024-09-16T15:47:17.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.270+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28040d0380 con 0x7f2804073fb0 2024-09-16T15:47:17.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.270+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280415d380 con 0x7f2804073fb0 2024-09-16T15:47:17.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.272+0000 7f2801600640 1 -- 172.21.15.60:0/1275381228 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f27f401a070 con 0x7f2804073fb0 2024-09-16T15:47:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.273+0000 7f2801600640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f27e00790c0 0x7f27e007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.273+0000 7f2801600640 1 -- 172.21.15.60:0/1275381228 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7f27f4098650 con 0x7f2804073fb0 2024-09-16T15:47:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.273+0000 7f27fbe00640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f27e00790c0 0x7f27e007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.274+0000 7f27fbe00640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f27e00790c0 0x7f27e007b580 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f27ec033000 tx=0x7f27ec004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:17.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.276+0000 7f2801600640 1 -- 172.21.15.60:0/1275381228 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f280415d380 con 0x7f2804073fb0 2024-09-16T15:47:17.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.548+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f280406f980 con 0x7f2804073fb0 2024-09-16T15:47:17.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.673+0000 7f2801600640 1 -- 172.21.15.60:0/1275381228 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v480) ==== 149+0+0 (secure 0 0 0) 0x7f27f4069e20 con 0x7f2804073fb0 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f27e00790c0 msgr2=0x7f27e007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f27e00790c0 0x7f27e007b580 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f27ec033000 tx=0x7f27ec004730 comp rx=0 tx=0).stop 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804073fb0 msgr2=0x7f28041efd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804073fb0 0x7f28041efd80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f27f400c3a0 tx=0x7f27f400c860 comp rx=0 tx=0).stop 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 shutdown_connections 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f27e00790c0 0x7f27e007b580 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280415f5a0 0x7f28041565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28040ccf00 0x7f2804156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 --2- 172.21.15.60:0/1275381228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2804073fb0 0x7f28041efd80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 >> 172.21.15.60:0/1275381228 conn(0x7f28040b1950 msgr2=0x7f28040ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:17.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 shutdown_connections 2024-09-16T15:47:17.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.676+0000 7f280b400640 1 -- 172.21.15.60:0/1275381228 wait complete. 2024-09-16T15:47:17.694 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-16T15:47:17.695 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-16T15:47:17.695 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:47:17.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.855+0000 7f4d9be00640 1 -- 172.21.15.60:0/1779724909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 msgr2=0x7f4d9c06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.855+0000 7f4d9be00640 1 --2- 172.21.15.60:0/1779724909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 0x7f4d9c06ff70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c158920 tx=0x7f4d8c02a600 comp rx=0 tx=0).stop 2024-09-16T15:47:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.856+0000 7f4d9be00640 1 -- 172.21.15.60:0/1779724909 shutdown_connections 2024-09-16T15:47:17.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.856+0000 7f4d9be00640 1 --2- 172.21.15.60:0/1779724909 >> v2:172.21.15.100:3300/0 conn(0x7f4d9c0ad880 0x7f4d9c0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.856+0000 7f4d9be00640 1 --2- 172.21.15.60:0/1779724909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 0x7f4d9c06ff70 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.856+0000 7f4d9be00640 1 -- 172.21.15.60:0/1779724909 >> 172.21.15.60:0/1779724909 conn(0x7f4d9c0cfa20 msgr2=0x7f4d9c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.856+0000 7f4d9be00640 1 -- 172.21.15.60:0/1779724909 shutdown_connections 2024-09-16T15:47:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.856+0000 7f4d9be00640 1 -- 172.21.15.60:0/1779724909 wait complete. 2024-09-16T15:47:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.857+0000 7f4d9be00640 1 Processor -- start 2024-09-16T15:47:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.857+0000 7f4d9be00640 1 -- start start 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.858+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 0x7f4d9c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.858+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d9c0ad880 0x7f4d9c0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.858+0000 7f4d9be00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d9c0cd5f0 0x7f4d9c1f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.858+0000 7f4d9be00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c15ef00 con 0x7f4d9c06fb90 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.858+0000 7f4d9be00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c15ed80 con 0x7f4d9c0cd5f0 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.858+0000 7f4d9be00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9c15f080 con 0x7f4d9c0ad880 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.858+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 0x7f4d9c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 0x7f4d9c0b50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52236/0 (socket says 172.21.15.60:52236) 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9aa00640 1 -- 172.21.15.60:0/2135101131 learned_addr learned my addr 172.21.15.60:0/2135101131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9b400640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d9c0cd5f0 0x7f4d9c1f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d92000640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d9c0ad880 0x7f4d9c0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9aa00640 1 -- 172.21.15.60:0/2135101131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d9c0ad880 msgr2=0x7f4d9c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9aa00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d9c0ad880 0x7f4d9c0c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9aa00640 1 -- 172.21.15.60:0/2135101131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d9c0cd5f0 msgr2=0x7f4d9c1f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9aa00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d9c0cd5f0 0x7f4d9c1f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9aa00640 1 -- 172.21.15.60:0/2135101131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9c0b2990 con 0x7f4d9c06fb90 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.859+0000 7f4d9aa00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 0x7f4d9c0b50e0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c034040 tx=0x7f4d8c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.860+0000 7f4d93400640 1 -- 172.21.15.60:0/2135101131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c03e030 con 0x7f4d9c06fb90 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.860+0000 7f4d93400640 1 -- 172.21.15.60:0/2135101131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d8c0047a0 con 0x7f4d9c06fb90 2024-09-16T15:47:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.860+0000 7f4d93400640 1 -- 172.21.15.60:0/2135101131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c035040 con 0x7f4d9c06fb90 2024-09-16T15:47:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.860+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d9c0b2c70 con 0x7f4d9c06fb90 2024-09-16T15:47:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.860+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d9c0b56c0 con 0x7f4d9c06fb90 2024-09-16T15:47:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.861+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9c06ff70 con 0x7f4d9c06fb90 2024-09-16T15:47:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.862+0000 7f4d93400640 1 -- 172.21.15.60:0/2135101131 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f4d8c041030 con 0x7f4d9c06fb90 2024-09-16T15:47:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.863+0000 7f4d93400640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d780790c0 0x7f4d7807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.863+0000 7f4d93400640 1 -- 172.21.15.60:0/2135101131 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7f4d8c0bbfc0 con 0x7f4d9c06fb90 2024-09-16T15:47:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.863+0000 7f4d92000640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d780790c0 0x7f4d7807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:17.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.864+0000 7f4d92000640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d780790c0 0x7f4d7807b580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c1587e0 tx=0x7f4d84008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:17.867+0000 7f4d93400640 1 -- 172.21.15.60:0/2135101131 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4d8c02ae20 con 0x7f4d9c06fb90 2024-09-16T15:47:18.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.063+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d9c0c68b0 con 0x7f4d9c06fb90 2024-09-16T15:47:18.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.063+0000 7f4d93400640 1 -- 172.21.15.60:0/2135101131 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+13086 (secure 0 0 0) 0x7f4d8c083320 con 0x7f4d9c06fb90 2024-09-16T15:47:18.065 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:18.065 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":152,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:47:16.663762+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2397719730":"2024-09-17T15:47:13.624863+0000","172.21.15.60:6833/2397719730":"2024-09-17T15:47:13.624863+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:18.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d780790c0 msgr2=0x7f4d7807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d780790c0 0x7f4d7807b580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c1587e0 tx=0x7f4d84008040 comp rx=0 tx=0).stop 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 msgr2=0x7f4d9c0b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 0x7f4d9c0b50e0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c034040 tx=0x7f4d8c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 shutdown_connections 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d780790c0 0x7f4d7807b580 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d9c0cd5f0 0x7f4d9c1f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d9c0ad880 0x7f4d9c0c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 --2- 172.21.15.60:0/2135101131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d9c06fb90 0x7f4d9c0b50e0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 >> 172.21.15.60:0/2135101131 conn(0x7f4d9c0cfa20 msgr2=0x7f4d9c160880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 shutdown_connections 2024-09-16T15:47:18.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.066+0000 7f4d9be00640 1 -- 172.21.15.60:0/2135101131 wait complete. 2024-09-16T15:47:18.084 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:47:18.084 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:47:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.246+0000 7f55a2e00640 1 -- 172.21.15.60:0/4129891526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0d0380 msgr2=0x7f559c070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.246+0000 7f55a2e00640 1 --2- 172.21.15.60:0/4129891526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0d0380 0x7f559c070c20 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5588005fd0 tx=0x7f558802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.246+0000 7f55a2e00640 1 -- 172.21.15.60:0/4129891526 shutdown_connections 2024-09-16T15:47:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.246+0000 7f55a2e00640 1 --2- 172.21.15.60:0/4129891526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0d0380 0x7f559c070c20 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.246+0000 7f55a2e00640 1 --2- 172.21.15.60:0/4129891526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f559c0cffa0 0x7f559c0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.246+0000 7f55a2e00640 1 -- 172.21.15.60:0/4129891526 >> 172.21.15.60:0/4129891526 conn(0x7f559c0b2600 msgr2=0x7f559c0b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.246+0000 7f55a2e00640 1 -- 172.21.15.60:0/4129891526 shutdown_connections 2024-09-16T15:47:18.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.246+0000 7f55a2e00640 1 -- 172.21.15.60:0/4129891526 wait complete. 2024-09-16T15:47:18.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.248+0000 7f55a2e00640 1 Processor -- start 2024-09-16T15:47:18.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.248+0000 7f55a2e00640 1 -- start start 2024-09-16T15:47:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a2e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f559c071160 0x7f559c0b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a2e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0cffa0 0x7f559c0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a2e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559c0d0380 0x7f559c14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a2e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f559c0af4d0 con 0x7f559c0cffa0 2024-09-16T15:47:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a2e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f559c06d490 con 0x7f559c0d0380 2024-09-16T15:47:18.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a2e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f559c0af650 con 0x7f559c071160 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0cffa0 0x7f559c0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a1000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0cffa0 0x7f559c0c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52240/0 (socket says 172.21.15.60:52240) 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a1000640 1 -- 172.21.15.60:0/239114054 learned_addr learned my addr 172.21.15.60:0/239114054 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f55a1a00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f559c071160 0x7f559c0b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.249+0000 7f55a2400640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559c0d0380 0x7f559c14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f55a1000640 1 -- 172.21.15.60:0/239114054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f559c071160 msgr2=0x7f559c0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f55a1000640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f559c071160 0x7f559c0b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f55a1000640 1 -- 172.21.15.60:0/239114054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559c0d0380 msgr2=0x7f559c14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f55a1000640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559c0d0380 0x7f559c14ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f55a1000640 1 -- 172.21.15.60:0/239114054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c0ca720 con 0x7f559c0cffa0 2024-09-16T15:47:18.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f55a1000640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0cffa0 0x7f559c0c45a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5588034040 tx=0x7f55880035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f5599600640 1 -- 172.21.15.60:0/239114054 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558803e030 con 0x7f559c0cffa0 2024-09-16T15:47:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.250+0000 7f5599600640 1 -- 172.21.15.60:0/239114054 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5588003ae0 con 0x7f559c0cffa0 2024-09-16T15:47:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.251+0000 7f5599600640 1 -- 172.21.15.60:0/239114054 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588035040 con 0x7f559c0cffa0 2024-09-16T15:47:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.251+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c0ca9e0 con 0x7f559c0cffa0 2024-09-16T15:47:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.251+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c0cc0d0 con 0x7f559c0cffa0 2024-09-16T15:47:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.252+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c0cbc10 con 0x7f559c0cffa0 2024-09-16T15:47:18.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.253+0000 7f5599600640 1 -- 172.21.15.60:0/239114054 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f5588041030 con 0x7f559c0cffa0 2024-09-16T15:47:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.254+0000 7f5599600640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f557c0790c0 0x7f557c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.254+0000 7f5599600640 1 -- 172.21.15.60:0/239114054 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7f55880bc020 con 0x7f559c0cffa0 2024-09-16T15:47:18.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.254+0000 7f55a1a00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f557c0790c0 0x7f557c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:18.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.255+0000 7f55a1a00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f557c0790c0 0x7f557c07b580 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5590009bd0 tx=0x7f5590005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:18.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.258+0000 7f5599600640 1 -- 172.21.15.60:0/239114054 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f559c0cbc10 con 0x7f559c0cffa0 2024-09-16T15:47:18.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.457+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f559c0ce970 con 0x7f559c0cffa0 2024-09-16T15:47:18.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.457+0000 7f5599600640 1 -- 172.21.15.60:0/239114054 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) ==== 93+0+10 (secure 0 0 0) 0x7f5588083380 con 0x7f559c0cffa0 2024-09-16T15:47:18.459 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:47:18.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f557c0790c0 msgr2=0x7f557c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f557c0790c0 0x7f557c07b580 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5590009bd0 tx=0x7f5590005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0cffa0 msgr2=0x7f559c0c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0cffa0 0x7f559c0c45a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5588034040 tx=0x7f55880035f0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 shutdown_connections 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f557c0790c0 0x7f557c07b580 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f559c0d0380 0x7f559c14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f559c0cffa0 0x7f559c0c45a0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 --2- 172.21.15.60:0/239114054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f559c071160 0x7f559c0b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 >> 172.21.15.60:0/239114054 conn(0x7f559c0b2600 msgr2=0x7f559c15ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 shutdown_connections 2024-09-16T15:47:18.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.460+0000 7f55a2e00640 1 -- 172.21.15.60:0/239114054 wait complete. 2024-09-16T15:47:18.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-16T15:47:18.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.637+0000 7f7476600640 1 -- 172.21.15.60:0/3505669872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f747015bc30 msgr2=0x7f74700b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.637+0000 7f7476600640 1 --2- 172.21.15.60:0/3505669872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f747015bc30 0x7f74700b28f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f7460005fd0 tx=0x7f746002a670 comp rx=0 tx=0).stop 2024-09-16T15:47:18.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.637+0000 7f7476600640 1 -- 172.21.15.60:0/3505669872 shutdown_connections 2024-09-16T15:47:18.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.637+0000 7f7476600640 1 --2- 172.21.15.60:0/3505669872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f747015bc30 0x7f74700b28f0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.637+0000 7f7476600640 1 -- 172.21.15.60:0/3505669872 >> 172.21.15.60:0/3505669872 conn(0x7f74700cfa20 msgr2=0x7f74700cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:18.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.637+0000 7f7476600640 1 -- 172.21.15.60:0/3505669872 shutdown_connections 2024-09-16T15:47:18.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.637+0000 7f7476600640 1 -- 172.21.15.60:0/3505669872 wait complete. 2024-09-16T15:47:18.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.639+0000 7f7476600640 1 Processor -- start 2024-09-16T15:47:18.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.639+0000 7f7476600640 1 -- start start 2024-09-16T15:47:18.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.640+0000 7f7476600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74700d0540 0x7f747014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:18.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.640+0000 7f7476600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f747015bc30 0x7f74700aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:18.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.640+0000 7f7476600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700aabd0 0x7f74700ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:18.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.640+0000 7f7476600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7470074660 con 0x7f74700aabd0 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.640+0000 7f7476600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f747006fa30 con 0x7f74700d0540 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.640+0000 7f7476600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74700a9f20 con 0x7f747015bc30 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700aabd0 0x7f74700ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700aabd0 0x7f74700ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52272/0 (socket says 172.21.15.60:52272) 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 -- 172.21.15.60:0/3357604390 learned_addr learned my addr 172.21.15.60:0/3357604390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7467e00640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f747015bc30 0x7f74700aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475200640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74700d0540 0x7f747014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 -- 172.21.15.60:0/3357604390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f747015bc30 msgr2=0x7f74700aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f747015bc30 0x7f74700aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 -- 172.21.15.60:0/3357604390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74700d0540 msgr2=0x7f747014d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:18.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74700d0540 0x7f747014d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 -- 172.21.15.60:0/3357604390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74700a95f0 con 0x7f74700aabd0 2024-09-16T15:47:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.641+0000 7f7475c00640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700aabd0 0x7f74700ab060 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f746800c7a0 tx=0x7f746800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.642+0000 7f746d600640 1 -- 172.21.15.60:0/3357604390 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468003b70 con 0x7f74700aabd0 2024-09-16T15:47:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.642+0000 7f746d600640 1 -- 172.21.15.60:0/3357604390 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7468003d10 con 0x7f74700aabd0 2024-09-16T15:47:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.642+0000 7f746d600640 1 -- 172.21.15.60:0/3357604390 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746800aab0 con 0x7f74700aabd0 2024-09-16T15:47:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.642+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74700a98d0 con 0x7f74700aabd0 2024-09-16T15:47:18.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.642+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74701f4a10 con 0x7f74700aabd0 2024-09-16T15:47:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.643+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74700b28f0 con 0x7f74700aabd0 2024-09-16T15:47:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.644+0000 7f746d600640 1 -- 172.21.15.60:0/3357604390 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f746800ac50 con 0x7f74700aabd0 2024-09-16T15:47:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.645+0000 7f746d600640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f744c0790f0 0x7f744c07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:18.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.645+0000 7f746d600640 1 -- 172.21.15.60:0/3357604390 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7f746809b770 con 0x7f74700aabd0 2024-09-16T15:47:18.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.645+0000 7f7475200640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f744c0790f0 0x7f744c07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:18.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.646+0000 7f7475200640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f744c0790f0 0x7f744c07b5b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7460033000 tx=0x7f7460004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:18.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.649+0000 7f746d600640 1 -- 172.21.15.60:0/3357604390 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7468063090 con 0x7f74700aabd0 2024-09-16T15:47:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:18.918+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f74701589f0 con 0x7f74700aabd0 2024-09-16T15:47:19.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.678+0000 7f746d600640 1 -- 172.21.15.60:0/3357604390 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v482) ==== 148+0+0 (secure 0 0 0) 0x7f7468062990 con 0x7f74700aabd0 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f744c0790f0 msgr2=0x7f744c07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f744c0790f0 0x7f744c07b5b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7460033000 tx=0x7f7460004730 comp rx=0 tx=0).stop 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700aabd0 msgr2=0x7f74700ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700aabd0 0x7f74700ab060 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f746800c7a0 tx=0x7f746800cc60 comp rx=0 tx=0).stop 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 shutdown_connections 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f744c0790f0 0x7f744c07b5b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74700aabd0 0x7f74700ab060 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f747015bc30 0x7f74700aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 --2- 172.21.15.60:0/3357604390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74700d0540 0x7f747014d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:19.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.681+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 >> 172.21.15.60:0/3357604390 conn(0x7f74700cfa20 msgr2=0x7f7470157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:19.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.682+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 shutdown_connections 2024-09-16T15:47:19.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.682+0000 7f7476600640 1 -- 172.21.15.60:0/3357604390 wait complete. 2024-09-16T15:47:19.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:47:19.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.866+0000 7fce86200640 1 -- 172.21.15.60:0/2725229439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce8015e0f0 msgr2=0x7fce800b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:19.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.866+0000 7fce86200640 1 --2- 172.21.15.60:0/2725229439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce8015e0f0 0x7fce800b1020 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fce74005fd0 tx=0x7fce7402a620 comp rx=0 tx=0).stop 2024-09-16T15:47:19.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.866+0000 7fce86200640 1 -- 172.21.15.60:0/2725229439 shutdown_connections 2024-09-16T15:47:19.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.866+0000 7fce86200640 1 --2- 172.21.15.60:0/2725229439 >> v2:172.21.15.100:3300/0 conn(0x7fce800cbb30 0x7fce800b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:19.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.866+0000 7fce86200640 1 --2- 172.21.15.60:0/2725229439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce8015e0f0 0x7fce800b1020 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:19.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.866+0000 7fce86200640 1 -- 172.21.15.60:0/2725229439 >> 172.21.15.60:0/2725229439 conn(0x7fce800b1e90 msgr2=0x7fce800b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:19.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.867+0000 7fce86200640 1 -- 172.21.15.60:0/2725229439 shutdown_connections 2024-09-16T15:47:19.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.867+0000 7fce86200640 1 -- 172.21.15.60:0/2725229439 wait complete. 2024-09-16T15:47:19.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.868+0000 7fce86200640 1 Processor -- start 2024-09-16T15:47:19.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.868+0000 7fce86200640 1 -- start start 2024-09-16T15:47:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce86200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce800cbb30 0x7fce8015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:19.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce86200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce800cc750 0x7fce801f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce86200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce8015e0f0 0x7fce800c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce86200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fce800cc380 con 0x7fce800cbb30 2024-09-16T15:47:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce86200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce800cc200 con 0x7fce800cc750 2024-09-16T15:47:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce86200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fce800cc500 con 0x7fce8015e0f0 2024-09-16T15:47:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce84e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce800cbb30 0x7fce8015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce84e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce800cbb30 0x7fce8015b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55548/0 (socket says 172.21.15.60:55548) 2024-09-16T15:47:19.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.869+0000 7fce84e00640 1 -- 172.21.15.60:0/2146036485 learned_addr learned my addr 172.21.15.60:0/2146036485 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.870+0000 7fce84e00640 1 -- 172.21.15.60:0/2146036485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce8015e0f0 msgr2=0x7fce800c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.870+0000 7fce84e00640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce8015e0f0 0x7fce800c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.870+0000 7fce84e00640 1 -- 172.21.15.60:0/2146036485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce800cc750 msgr2=0x7fce801f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.870+0000 7fce84e00640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce800cc750 0x7fce801f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:19.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.870+0000 7fce84e00640 1 -- 172.21.15.60:0/2146036485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce801f4c50 con 0x7fce800cbb30 2024-09-16T15:47:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.870+0000 7fce84e00640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce800cbb30 0x7fce8015b6b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fce74034040 tx=0x7fce7402ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.870+0000 7fce7cc00640 1 -- 172.21.15.60:0/2146036485 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7403e030 con 0x7fce800cbb30 2024-09-16T15:47:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.871+0000 7fce7cc00640 1 -- 172.21.15.60:0/2146036485 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce74004b20 con 0x7fce800cbb30 2024-09-16T15:47:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.871+0000 7fce7cc00640 1 -- 172.21.15.60:0/2146036485 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74035040 con 0x7fce800cbb30 2024-09-16T15:47:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.871+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce801f4f30 con 0x7fce800cbb30 2024-09-16T15:47:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.871+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce800ae080 con 0x7fce800cbb30 2024-09-16T15:47:19.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.872+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce80074c80 con 0x7fce800cbb30 2024-09-16T15:47:19.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.873+0000 7fce7cc00640 1 -- 172.21.15.60:0/2146036485 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fce74041030 con 0x7fce800cbb30 2024-09-16T15:47:19.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.874+0000 7fce7cc00640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fce600790f0 0x7fce6007b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:19.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.874+0000 7fce7cc00640 1 -- 172.21.15.60:0/2146036485 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7fce740bc160 con 0x7fce800cbb30 2024-09-16T15:47:19.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.874+0000 7fce7fe00640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fce600790f0 0x7fce6007b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:19.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.875+0000 7fce7fe00640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fce600790f0 0x7fce6007b5b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce6c000f60 tx=0x7fce6c0079b0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:19.878+0000 7fce7cc00640 1 -- 172.21.15.60:0/2146036485 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fce74083a90 con 0x7fce800cbb30 2024-09-16T15:47:20.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.079+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fce800ae250 con 0x7fce800cbb30 2024-09-16T15:47:20.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.080+0000 7fce7cc00640 1 -- 172.21.15.60:0/2146036485 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v45) ==== 59+0+1998 (secure 0 0 0) 0x7fce74083390 con 0x7fce800cbb30 2024-09-16T15:47:20.081 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:47:20.081 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:47:20.081 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:47:20.081 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:47:20.081 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:47:20.082 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:47:20.083 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:47:20.083 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:47:20.083 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:47:20.083 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:47:20.083 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:47:20.083 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:47:20.083 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:47:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fce600790f0 msgr2=0x7fce6007b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fce600790f0 0x7fce6007b5b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce6c000f60 tx=0x7fce6c0079b0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce800cbb30 msgr2=0x7fce8015b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce800cbb30 0x7fce8015b6b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fce74034040 tx=0x7fce7402ae70 comp rx=0 tx=0).stop 2024-09-16T15:47:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 shutdown_connections 2024-09-16T15:47:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fce600790f0 0x7fce6007b5b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce8015e0f0 0x7fce800c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce800cc750 0x7fce801f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 --2- 172.21.15.60:0/2146036485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce800cbb30 0x7fce8015b6b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 >> 172.21.15.60:0/2146036485 conn(0x7fce800b1e90 msgr2=0x7fce800b52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:20.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 shutdown_connections 2024-09-16T15:47:20.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.082+0000 7fce86200640 1 -- 172.21.15.60:0/2146036485 wait complete. 2024-09-16T15:47:20.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:47:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.264+0000 7f88c1400640 1 -- 172.21.15.60:0/883832295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a0003a50 msgr2=0x7f88a0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.264+0000 7f88c1400640 1 --2- 172.21.15.60:0/883832295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a0003a50 0x7f88a0003e70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f88ac001690 tx=0x7f88a0005830 comp rx=0 tx=0).stop 2024-09-16T15:47:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.264+0000 7f88c1400640 1 -- 172.21.15.60:0/883832295 shutdown_connections 2024-09-16T15:47:20.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.264+0000 7f88c1400640 1 --2- 172.21.15.60:0/883832295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88a0003a50 0x7f88a0003e70 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.264+0000 7f88c1400640 1 -- 172.21.15.60:0/883832295 >> 172.21.15.60:0/883832295 conn(0x7f88bc09d0b0 msgr2=0x7f88bc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.264+0000 7f88c1400640 1 -- 172.21.15.60:0/883832295 shutdown_connections 2024-09-16T15:47:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.264+0000 7f88c1400640 1 -- 172.21.15.60:0/883832295 wait complete. 2024-09-16T15:47:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.265+0000 7f88c1400640 1 Processor -- start 2024-09-16T15:47:20.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.265+0000 7f88c1400640 1 -- start start 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.266+0000 7f88c1400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0003a50 0x7f88bc15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.266+0000 7f88c1400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88bc1f43c0 0x7f88bc0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.266+0000 7f88c1400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88bc0b2560 0x7f88bc0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.266+0000 7f88c1400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc0b50c0 con 0x7f88bc1f43c0 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.266+0000 7f88c1400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc0b4f40 con 0x7f88bc0b2560 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.266+0000 7f88c1400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f88bc0d22f0 con 0x7f88a0003a50 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88bc1f43c0 0x7f88bc0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88bc1f43c0 0x7f88bc0c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55564/0 (socket says 172.21.15.60:55564) 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 -- 172.21.15.60:0/444489327 learned_addr learned my addr 172.21.15.60:0/444489327 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bbe00640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0003a50 0x7f88bc15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:20.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88c0a00640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88bc0b2560 0x7f88bc0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 -- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0003a50 msgr2=0x7f88bc15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0003a50 0x7f88bc15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 -- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88bc0b2560 msgr2=0x7f88bc0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88bc0b2560 0x7f88bc0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 -- 172.21.15.60:0/444489327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc1f4960 con 0x7f88bc1f43c0 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88c0a00640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88bc0b2560 0x7f88bc0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.267+0000 7f88bb400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88bc1f43c0 0x7f88bc0c7920 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f88a8006630 tx=0x7f88a80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.268+0000 7f88b3e00640 1 -- 172.21.15.60:0/444489327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8015070 con 0x7f88bc1f43c0 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.268+0000 7f88b3e00640 1 -- 172.21.15.60:0/444489327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88a8011860 con 0x7f88bc1f43c0 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.268+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc1f4c40 con 0x7f88bc1f43c0 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.268+0000 7f88b3e00640 1 -- 172.21.15.60:0/444489327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8011cb0 con 0x7f88bc1f43c0 2024-09-16T15:47:20.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.268+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc1f5200 con 0x7f88bc1f43c0 2024-09-16T15:47:20.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.269+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88bc0ce820 con 0x7f88bc1f43c0 2024-09-16T15:47:20.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.270+0000 7f88b3e00640 1 -- 172.21.15.60:0/444489327 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f88a8018070 con 0x7f88bc1f43c0 2024-09-16T15:47:20.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.271+0000 7f88b3e00640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f889c0790c0 0x7f889c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:20.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.271+0000 7f88b3e00640 1 -- 172.21.15.60:0/444489327 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7f88a809c550 con 0x7f88bc1f43c0 2024-09-16T15:47:20.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.271+0000 7f88bbe00640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f889c0790c0 0x7f889c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:20.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.272+0000 7f88bbe00640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f889c0790c0 0x7f889c07b580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f88ac032000 tx=0x7f88ac02c040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:20.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.275+0000 7f88b3e00640 1 -- 172.21.15.60:0/444489327 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f88a8063830 con 0x7f88bc1f43c0 2024-09-16T15:47:20.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.474+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f88bc160830 con 0x7f88bc1f43c0 2024-09-16T15:47:20.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.477+0000 7f88b3e00640 1 -- 172.21.15.60:0/444489327 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f88a8068260 con 0x7f88bc1f43c0 2024-09-16T15:47:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f889c0790c0 msgr2=0x7f889c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f889c0790c0 0x7f889c07b580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f88ac032000 tx=0x7f88ac02c040 comp rx=0 tx=0).stop 2024-09-16T15:47:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88bc1f43c0 msgr2=0x7f88bc0c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88bc1f43c0 0x7f88bc0c7920 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f88a8006630 tx=0x7f88a80039c0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 shutdown_connections 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f889c0790c0 0x7f889c07b580 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88bc0b2560 0x7f88bc0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f88bc1f43c0 0x7f88bc0c7920 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 --2- 172.21.15.60:0/444489327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88a0003a50 0x7f88bc15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 >> 172.21.15.60:0/444489327 conn(0x7f88bc09d0b0 msgr2=0x7f88bc0cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 shutdown_connections 2024-09-16T15:47:20.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.480+0000 7f88c1400640 1 -- 172.21.15.60:0/444489327 wait complete. 2024-09-16T15:47:20.497 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:47:20.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:47:20.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:47:20.676 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:47:20.677 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:20.677 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:20.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:47:20.713 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:20.713 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:20.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:47:20.750 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:20.750 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:20.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:47:20.789 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:20.789 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:20.827 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:47:20.827 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:47:20.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:20.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.989+0000 7fd050e00640 1 -- 172.21.15.60:0/681386657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c0ccf00 msgr2=0x7fd04c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.989+0000 7fd050e00640 1 --2- 172.21.15.60:0/681386657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c0ccf00 0x7fd04c0acd30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd034009830 tx=0x7fd03402a490 comp rx=0 tx=0).stop 2024-09-16T15:47:20.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.989+0000 7fd050e00640 1 -- 172.21.15.60:0/681386657 shutdown_connections 2024-09-16T15:47:20.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.989+0000 7fd050e00640 1 --2- 172.21.15.60:0/681386657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c0ccf00 0x7fd04c0acd30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.989+0000 7fd050e00640 1 --2- 172.21.15.60:0/681386657 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04c15f5a0 0x7fd04c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.989+0000 7fd050e00640 1 -- 172.21.15.60:0/681386657 >> 172.21.15.60:0/681386657 conn(0x7fd04c0b1950 msgr2=0x7fd04c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:20.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.989+0000 7fd050e00640 1 -- 172.21.15.60:0/681386657 shutdown_connections 2024-09-16T15:47:20.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.989+0000 7fd050e00640 1 -- 172.21.15.60:0/681386657 wait complete. 2024-09-16T15:47:20.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.990+0000 7fd050e00640 1 Processor -- start 2024-09-16T15:47:20.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.990+0000 7fd050e00640 1 -- start start 2024-09-16T15:47:20.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.991+0000 7fd050e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04c0ad270 0x7fd04c09d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:20.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.991+0000 7fd050e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c0ccf00 0x7fd04c1f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:20.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.991+0000 7fd050e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04c15f5a0 0x7fd04c1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:20.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.991+0000 7fd050e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c06f2e0 con 0x7fd04c15f5a0 2024-09-16T15:47:20.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.991+0000 7fd050e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c06f160 con 0x7fd04c0ccf00 2024-09-16T15:47:20.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.991+0000 7fd050e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c06f460 con 0x7fd04c0ad270 2024-09-16T15:47:20.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04c15f5a0 0x7fd04c1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:20.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04c15f5a0 0x7fd04c1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55566/0 (socket says 172.21.15.60:55566) 2024-09-16T15:47:20.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 -- 172.21.15.60:0/456650238 learned_addr learned my addr 172.21.15.60:0/456650238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:20.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd049e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04c0ad270 0x7fd04c09d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:20.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd041400640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c0ccf00 0x7fd04c1f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:20.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 -- 172.21.15.60:0/456650238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04c0ad270 msgr2=0x7fd04c09d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04c0ad270 0x7fd04c09d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 -- 172.21.15.60:0/456650238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c0ccf00 msgr2=0x7fd04c1f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:20.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c0ccf00 0x7fd04c1f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:20.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 -- 172.21.15.60:0/456650238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c0cf580 con 0x7fd04c15f5a0 2024-09-16T15:47:20.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.992+0000 7fd04a800640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04c15f5a0 0x7fd04c1582c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd04400cb30 tx=0x7fd04400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:20.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.993+0000 7fd042a00640 1 -- 172.21.15.60:0/456650238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044016020 con 0x7fd04c15f5a0 2024-09-16T15:47:20.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.993+0000 7fd042a00640 1 -- 172.21.15.60:0/456650238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd04400ad90 con 0x7fd04c15f5a0 2024-09-16T15:47:20.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.993+0000 7fd042a00640 1 -- 172.21.15.60:0/456650238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044012680 con 0x7fd04c15f5a0 2024-09-16T15:47:20.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.993+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c0cf840 con 0x7fd04c15f5a0 2024-09-16T15:47:20.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.993+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c0cfe00 con 0x7fd04c15f5a0 2024-09-16T15:47:20.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.994+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c06f980 con 0x7fd04c15f5a0 2024-09-16T15:47:20.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.995+0000 7fd042a00640 1 -- 172.21.15.60:0/456650238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fd044019020 con 0x7fd04c15f5a0 2024-09-16T15:47:20.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.996+0000 7fd042a00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0280790c0 0x7fd02807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:20.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.996+0000 7fd042a00640 1 -- 172.21.15.60:0/456650238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7fd04409be70 con 0x7fd04c15f5a0 2024-09-16T15:47:20.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.997+0000 7fd049e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0280790c0 0x7fd02807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:20.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:20.997+0000 7fd049e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0280790c0 0x7fd02807b580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009b30 tx=0x7fd03c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.000+0000 7fd042a00640 1 -- 172.21.15.60:0/456650238 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd044020020 con 0x7fd04c15f5a0 2024-09-16T15:47:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.232+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04c0b47b0 con 0x7fd04c15f5a0 2024-09-16T15:47:21.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.232+0000 7fd042a00640 1 -- 172.21.15.60:0/456650238 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+3913 (secure 0 0 0) 0x7fd044063140 con 0x7fd04c15f5a0 2024-09-16T15:47:21.234 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:21.234 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":105,"btime":"2024-09-16T15:47:14:649304+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7133,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/822830116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":822830116},{"type":"v1","addr":"172.21.15.100:6834","nonce":822830116}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7136,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/467713690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":467713690},{"type":"v1","addr":"172.21.15.100:6835","nonce":467713690}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":7981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2740321673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2740321673},{"type":"v1","addr":"172.21.15.60:6834","nonce":2740321673}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3774199450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3774199450},{"type":"v1","addr":"172.21.15.60:6835","nonce":3774199450}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[]} 2024-09-16T15:47:21.234 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 105 2024-09-16T15:47:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0280790c0 msgr2=0x7fd02807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0280790c0 0x7fd02807b580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009b30 tx=0x7fd03c005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04c15f5a0 msgr2=0x7fd04c1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04c15f5a0 0x7fd04c1582c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd04400cb30 tx=0x7fd04400a060 comp rx=0 tx=0).stop 2024-09-16T15:47:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 shutdown_connections 2024-09-16T15:47:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0280790c0 0x7fd02807b580 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04c15f5a0 0x7fd04c1582c0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd04c0ccf00 0x7fd04c1f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 --2- 172.21.15.60:0/456650238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04c0ad270 0x7fd04c09d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 >> 172.21.15.60:0/456650238 conn(0x7fd04c0b1950 msgr2=0x7fd04c0ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 shutdown_connections 2024-09-16T15:47:21.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.235+0000 7fd050e00640 1 -- 172.21.15.60:0/456650238 wait complete. 2024-09-16T15:47:21.252 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:47:21.252 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:47:21.252 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:47:21.252 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:47:21.252 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:47:21.252 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:47:21.253 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:47:21.253 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:47:21.253 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:47:21.253 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:47:21.253 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:47:21.253 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:47:21.253 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:47:21.253 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:47:21.254 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:47:21.254 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:47:21.268 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:47:21.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:47:21.269 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:47:21.269 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:47:21.270 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:47:21.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:47:21.271 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:47:21.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:47:21.313 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:47:21.313 DEBUG:teuthology.parallel:result is None 2024-09-16T15:47:21.315 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:47:21.315 DEBUG:teuthology.parallel:result is None 2024-09-16T15:47:21.316 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:47:21.316 DEBUG:teuthology.parallel:result is None 2024-09-16T15:47:21.317 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:47:21.318 DEBUG:teuthology.parallel:result is None 2024-09-16T15:47:21.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:47:21.442 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:47:21.443 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:47:21.448 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:47:21.448 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:47:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.484+0000 7f7961400640 1 -- 172.21.15.60:0/176310095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d2050 msgr2=0x7f795c0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.484+0000 7f7961400640 1 --2- 172.21.15.60:0/176310095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d2050 0x7f795c0d10d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7948005fd0 tx=0x7f794802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.485+0000 7f7961400640 1 -- 172.21.15.60:0/176310095 shutdown_connections 2024-09-16T15:47:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.485+0000 7f7961400640 1 --2- 172.21.15.60:0/176310095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d2050 0x7f795c0d10d0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.485+0000 7f7961400640 1 --2- 172.21.15.60:0/176310095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f795c0d1610 0x7f795c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.485+0000 7f7961400640 1 -- 172.21.15.60:0/176310095 >> 172.21.15.60:0/176310095 conn(0x7f795c0d3fe0 msgr2=0x7f795c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.485+0000 7f7961400640 1 -- 172.21.15.60:0/176310095 shutdown_connections 2024-09-16T15:47:21.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.485+0000 7f7961400640 1 -- 172.21.15.60:0/176310095 wait complete. 2024-09-16T15:47:21.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.486+0000 7f7961400640 1 Processor -- start 2024-09-16T15:47:21.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.486+0000 7f7961400640 1 -- start start 2024-09-16T15:47:21.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f7961400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d1610 0x7f795c15a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:21.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f7961400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f795c0d2050 0x7f795c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f7961400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795c15e0d0 0x7f795c0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f7961400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f795c0cf5d0 con 0x7f795c0d1610 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f7961400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f795c0cf450 con 0x7f795c15e0d0 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f7961400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f795c0cf750 con 0x7f795c0d2050 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f795a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d1610 0x7f795c15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f795a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d1610 0x7f795c15a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55610/0 (socket says 172.21.15.60:55610) 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.487+0000 7f795a800640 1 -- 172.21.15.60:0/1824232945 learned_addr learned my addr 172.21.15.60:0/1824232945 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f7959e00640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f795c0d2050 0x7f795c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f795b200640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795c15e0d0 0x7f795c0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f795a800640 1 -- 172.21.15.60:0/1824232945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f795c0d2050 msgr2=0x7f795c0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f795a800640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f795c0d2050 0x7f795c0cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f795a800640 1 -- 172.21.15.60:0/1824232945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795c15e0d0 msgr2=0x7f795c0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f795a800640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795c15e0d0 0x7f795c0cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f795a800640 1 -- 172.21.15.60:0/1824232945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c1f4f70 con 0x7f795c0d1610 2024-09-16T15:47:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f795a800640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d1610 0x7f795c15a470 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c430 tx=0x7f794c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f7952a00640 1 -- 172.21.15.60:0/1824232945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c017070 con 0x7f795c0d1610 2024-09-16T15:47:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f7952a00640 1 -- 172.21.15.60:0/1824232945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f794c0053e0 con 0x7f795c0d1610 2024-09-16T15:47:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f7952a00640 1 -- 172.21.15.60:0/1824232945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c003850 con 0x7f795c0d1610 2024-09-16T15:47:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.488+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c1f5250 con 0x7f795c0d1610 2024-09-16T15:47:21.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.489+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c1f5810 con 0x7f795c0d1610 2024-09-16T15:47:21.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.490+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c0cfc70 con 0x7f795c0d1610 2024-09-16T15:47:21.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.491+0000 7f7952a00640 1 -- 172.21.15.60:0/1824232945 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f794c01a070 con 0x7f795c0d1610 2024-09-16T15:47:21.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.492+0000 7f7952a00640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f793c0790c0 0x7f793c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:21.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.492+0000 7f7952a00640 1 -- 172.21.15.60:0/1824232945 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4971+0+0 (secure 0 0 0) 0x7f794c09abb0 con 0x7f795c0d1610 2024-09-16T15:47:21.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.492+0000 7f7959e00640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f793c0790c0 0x7f793c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:21.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.493+0000 7f7959e00640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f793c0790c0 0x7f793c07b580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7948033000 tx=0x7f7948004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:21.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.496+0000 7f7952a00640 1 -- 172.21.15.60:0/1824232945 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f794c061f10 con 0x7f795c0d1610 2024-09-16T15:47:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:21.707+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f795c0b9700 con 0x7f795c0d1610 2024-09-16T15:47:22.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.713+0000 7f7952a00640 1 -- 172.21.15.60:0/1824232945 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v157) ==== 97+0+0 (secure 0 0 0) 0x7f794c066940 con 0x7f795c0d1610 2024-09-16T15:47:22.715 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:47:22.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f793c0790c0 msgr2=0x7f793c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:22.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f793c0790c0 0x7f793c07b580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7948033000 tx=0x7f7948004730 comp rx=0 tx=0).stop 2024-09-16T15:47:22.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d1610 msgr2=0x7f795c15a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:22.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d1610 0x7f795c15a470 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c430 tx=0x7f794c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 shutdown_connections 2024-09-16T15:47:22.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f793c0790c0 0x7f793c07b580 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f795c15e0d0 0x7f795c0cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f795c0d2050 0x7f795c0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 --2- 172.21.15.60:0/1824232945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f795c0d1610 0x7f795c15a470 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 >> 172.21.15.60:0/1824232945 conn(0x7f795c0d3fe0 msgr2=0x7f795c15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.718+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 shutdown_connections 2024-09-16T15:47:22.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.719+0000 7f7961400640 1 -- 172.21.15.60:0/1824232945 wait complete. 2024-09-16T15:47:22.740 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:47:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.904+0000 7f90bd800640 1 -- 172.21.15.60:0/1317940306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90b80ccf00 msgr2=0x7f90b80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.904+0000 7f90bd800640 1 --2- 172.21.15.60:0/1317940306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90b80ccf00 0x7f90b80b2440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009860 tx=0x7f90a402a4a0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.904+0000 7f90bd800640 1 -- 172.21.15.60:0/1317940306 shutdown_connections 2024-09-16T15:47:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.904+0000 7f90bd800640 1 --2- 172.21.15.60:0/1317940306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90b80ccf00 0x7f90b80b2440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.904+0000 7f90bd800640 1 -- 172.21.15.60:0/1317940306 >> 172.21.15.60:0/1317940306 conn(0x7f90b80b1950 msgr2=0x7f90b80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.904+0000 7f90bd800640 1 -- 172.21.15.60:0/1317940306 shutdown_connections 2024-09-16T15:47:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.904+0000 7f90bd800640 1 -- 172.21.15.60:0/1317940306 wait complete. 2024-09-16T15:47:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.906+0000 7f90bd800640 1 Processor -- start 2024-09-16T15:47:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.906+0000 7f90bd800640 1 -- start start 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90bd800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90b8073fb0 0x7f90b81f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90bd800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90b80ccf00 0x7f90b81e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90bd800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90b815f5a0 0x7f90b81f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90bd800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f90b806ef30 con 0x7f90b815f5a0 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90bd800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f90b806fa80 con 0x7f90b8073fb0 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90bd800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f90b8074440 con 0x7f90b80ccf00 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90b7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90b815f5a0 0x7f90b81f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90b7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90b815f5a0 0x7f90b81f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55614/0 (socket says 172.21.15.60:55614) 2024-09-16T15:47:22.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.907+0000 7f90b7400640 1 -- 172.21.15.60:0/2608915290 learned_addr learned my addr 172.21.15.60:0/2608915290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90b6a00640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90b8073fb0 0x7f90b81f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90b6000640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90b80ccf00 0x7f90b81e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90b7400640 1 -- 172.21.15.60:0/2608915290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90b80ccf00 msgr2=0x7f90b81e4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90b7400640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90b80ccf00 0x7f90b81e4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90b7400640 1 -- 172.21.15.60:0/2608915290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90b8073fb0 msgr2=0x7f90b81f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90b7400640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90b8073fb0 0x7f90b81f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90b7400640 1 -- 172.21.15.60:0/2608915290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b81f4aa0 con 0x7f90b815f5a0 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90b7400640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90b815f5a0 0x7f90b81f5240 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f90b0001ed0 tx=0x7f90b000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90aea00640 1 -- 172.21.15.60:0/2608915290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0016020 con 0x7f90b815f5a0 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90aea00640 1 -- 172.21.15.60:0/2608915290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90b0012420 con 0x7f90b815f5a0 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90aea00640 1 -- 172.21.15.60:0/2608915290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0012830 con 0x7f90b815f5a0 2024-09-16T15:47:22.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.908+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b81f85d0 con 0x7f90b815f5a0 2024-09-16T15:47:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.909+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b81f8850 con 0x7f90b815f5a0 2024-09-16T15:47:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.909+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b80b2440 con 0x7f90b815f5a0 2024-09-16T15:47:22.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.911+0000 7f90aea00640 1 -- 172.21.15.60:0/2608915290 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f90b0019020 con 0x7f90b815f5a0 2024-09-16T15:47:22.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.912+0000 7f90aea00640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90980790c0 0x7f909807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.912+0000 7f90aea00640 1 -- 172.21.15.60:0/2608915290 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 4885+0+0 (secure 0 0 0) 0x7f90b009be80 con 0x7f90b815f5a0 2024-09-16T15:47:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.913+0000 7f90b6a00640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90980790c0 0x7f909807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.913+0000 7f90b6a00640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90980790c0 0x7f909807b580 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f90a8001120 tx=0x7f90a8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:22.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:22.917+0000 7f90aea00640 1 -- 172.21.15.60:0/2608915290 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f90b0020030 con 0x7f90b815f5a0 2024-09-16T15:47:23.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.182+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f90b80cc1d0 con 0x7f90b815f5a0 2024-09-16T15:47:23.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.182+0000 7f90aea00640 1 -- 172.21.15.60:0/2608915290 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f90b81f85d0 con 0x7f90b815f5a0 2024-09-16T15:47:23.184 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:23.184 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:47:23.184 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:47:23.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.186 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:47:23.187 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:47:23.188 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.189 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.190 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.191 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.192 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.193 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.194 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:47:23.195 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:47:23.197 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.198 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:47:23.199 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.200 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.200 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:47:23.200 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90980790c0 msgr2=0x7f909807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90980790c0 0x7f909807b580 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f90a8001120 tx=0x7f90a8009040 comp rx=0 tx=0).stop 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90b815f5a0 msgr2=0x7f90b81f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90b815f5a0 0x7f90b81f5240 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f90b0001ed0 tx=0x7f90b000a430 comp rx=0 tx=0).stop 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 shutdown_connections 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90980790c0 0x7f909807b580 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90b815f5a0 0x7f90b81f5240 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90b80ccf00 0x7f90b81e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 --2- 172.21.15.60:0/2608915290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90b8073fb0 0x7f90b81f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 >> 172.21.15.60:0/2608915290 conn(0x7f90b80b1950 msgr2=0x7f90b80ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:23.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 shutdown_connections 2024-09-16T15:47:23.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.185+0000 7f90bd800640 1 -- 172.21.15.60:0/2608915290 wait complete. 2024-09-16T15:47:23.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:47:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.364+0000 7f0cd5e00640 1 -- 172.21.15.60:0/2679794274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd01438b0 msgr2=0x7f0cd006ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.364+0000 7f0cd5e00640 1 --2- 172.21.15.60:0/2679794274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd01438b0 0x7f0cd006ff60 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009830 tx=0x7f0cb802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.364+0000 7f0cd5e00640 1 -- 172.21.15.60:0/2679794274 shutdown_connections 2024-09-16T15:47:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.364+0000 7f0cd5e00640 1 --2- 172.21.15.60:0/2679794274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd01438b0 0x7f0cd006ff60 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.364+0000 7f0cd5e00640 1 -- 172.21.15.60:0/2679794274 >> 172.21.15.60:0/2679794274 conn(0x7f0cd00b1950 msgr2=0x7f0cd00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.364+0000 7f0cd5e00640 1 -- 172.21.15.60:0/2679794274 shutdown_connections 2024-09-16T15:47:23.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.365+0000 7f0cd5e00640 1 -- 172.21.15.60:0/2679794274 wait complete. 2024-09-16T15:47:23.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.366+0000 7f0cd5e00640 1 Processor -- start 2024-09-16T15:47:23.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.366+0000 7f0cd5e00640 1 -- start start 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0cd5e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd0070920 0x7f0cd013f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0cd5e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cd00b3f00 0x7f0cd015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0cd5e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cd01438b0 0x7f0cd00b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0cd5e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd00b4fa0 con 0x7f0cd0070920 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0cd5e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd00b4e20 con 0x7f0cd00b3f00 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0cd5e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd00cc5f0 con 0x7f0cd01438b0 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0ccea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd0070920 0x7f0cd013f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0ccea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd0070920 0x7f0cd013f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55634/0 (socket says 172.21.15.60:55634) 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0ccea00640 1 -- 172.21.15.60:0/3720493946 learned_addr learned my addr 172.21.15.60:0/3720493946 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:23.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.367+0000 7f0cc6000640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cd00b3f00 0x7f0cd015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0ccf400640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cd01438b0 0x7f0cd00b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0ccea00640 1 -- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cd01438b0 msgr2=0x7f0cd00b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0ccea00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cd01438b0 0x7f0cd00b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0ccea00640 1 -- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cd00b3f00 msgr2=0x7f0cd015aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0ccea00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cd00b3f00 0x7f0cd015aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0ccea00640 1 -- 172.21.15.60:0/3720493946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd013bbf0 con 0x7f0cd0070920 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0ccf400640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cd01438b0 0x7f0cd00b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0ccea00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd0070920 0x7f0cd013f610 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c760 tx=0x7f0cc000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0cc7e00640 1 -- 172.21.15.60:0/3720493946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0017070 con 0x7f0cd0070920 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0cc7e00640 1 -- 172.21.15.60:0/3720493946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cc0006b80 con 0x7f0cd0070920 2024-09-16T15:47:23.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0cc7e00640 1 -- 172.21.15.60:0/3720493946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0007030 con 0x7f0cd0070920 2024-09-16T15:47:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.368+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd013bed0 con 0x7f0cd0070920 2024-09-16T15:47:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.369+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd013dc70 con 0x7f0cd0070920 2024-09-16T15:47:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.370+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd00cca70 con 0x7f0cd0070920 2024-09-16T15:47:23.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.371+0000 7f0cc7e00640 1 -- 172.21.15.60:0/3720493946 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f0cc001a070 con 0x7f0cd0070920 2024-09-16T15:47:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.372+0000 7f0cc7e00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0cac0790f0 0x7f0cac07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.373+0000 7f0cc7e00640 1 -- 172.21.15.60:0/3720493946 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 4885+0+0 (secure 0 0 0) 0x7f0cc009acc0 con 0x7f0cd0070920 2024-09-16T15:47:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.373+0000 7f0cc6000640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0cac0790f0 0x7f0cac07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:23.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.373+0000 7f0cc6000640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0cac0790f0 0x7f0cac07b5b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8034000 tx=0x7f0cb8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:23.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.377+0000 7f0cc7e00640 1 -- 172.21.15.60:0/3720493946 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0cd00cdae0 con 0x7f0cd0070920 2024-09-16T15:47:23.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.577+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0cd0140490 con 0x7f0cd0070920 2024-09-16T15:47:23.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.577+0000 7f0cc7e00640 1 -- 172.21.15.60:0/3720493946 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v157) ==== 84+0+65 (secure 0 0 0) 0x7f0cc0061fe0 con 0x7f0cd0070920 2024-09-16T15:47:23.579 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:23.579 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:47:23.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:23.579 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:47:23.579 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:47:23.579 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:23.579 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0cac0790f0 msgr2=0x7f0cac07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0cac0790f0 0x7f0cac07b5b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8034000 tx=0x7f0cb8004730 comp rx=0 tx=0).stop 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd0070920 msgr2=0x7f0cd013f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd0070920 0x7f0cd013f610 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c760 tx=0x7f0cc000cc20 comp rx=0 tx=0).stop 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 shutdown_connections 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0cac0790f0 0x7f0cac07b5b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cd01438b0 0x7f0cd00b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cd00b3f00 0x7f0cd015aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 --2- 172.21.15.60:0/3720493946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cd0070920 0x7f0cd013f610 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 >> 172.21.15.60:0/3720493946 conn(0x7f0cd00b1950 msgr2=0x7f0cd0075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 shutdown_connections 2024-09-16T15:47:23.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.580+0000 7f0cd5e00640 1 -- 172.21.15.60:0/3720493946 wait complete. 2024-09-16T15:47:23.597 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:47:23.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:47:23.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.762+0000 7f4813200640 1 -- 172.21.15.60:0/3594111159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f481415b9b0 msgr2=0x7f481415ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.762+0000 7f4813200640 1 --2- 172.21.15.60:0/3594111159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f481415b9b0 0x7f481415ee00 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f4804005fd0 tx=0x7f480402a610 comp rx=0 tx=0).stop 2024-09-16T15:47:23.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.762+0000 7f4813200640 1 -- 172.21.15.60:0/3594111159 shutdown_connections 2024-09-16T15:47:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.762+0000 7f4813200640 1 --2- 172.21.15.60:0/3594111159 >> v2:172.21.15.100:3300/0 conn(0x7f48140caa90 0x7f48140ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.762+0000 7f4813200640 1 --2- 172.21.15.60:0/3594111159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f481415b9b0 0x7f481415ee00 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.762+0000 7f4813200640 1 -- 172.21.15.60:0/3594111159 >> 172.21.15.60:0/3594111159 conn(0x7f48140cfa70 msgr2=0x7f48140cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.762+0000 7f4813200640 1 -- 172.21.15.60:0/3594111159 shutdown_connections 2024-09-16T15:47:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.762+0000 7f4813200640 1 -- 172.21.15.60:0/3594111159 wait complete. 2024-09-16T15:47:23.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.764+0000 7f4813200640 1 Processor -- start 2024-09-16T15:47:23.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.764+0000 7f4813200640 1 -- start start 2024-09-16T15:47:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4813200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48140caa90 0x7f48140b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4813200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48140ce7f0 0x7f48140b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4813200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481415b9b0 0x7f481406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4813200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f481415bf50 con 0x7f48140caa90 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4813200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f48140cf390 con 0x7f48140ce7f0 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4813200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f48140addc0 con 0x7f481415b9b0 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4811e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48140caa90 0x7f48140b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4811e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48140caa90 0x7f48140b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55658/0 (socket says 172.21.15.60:55658) 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4811e00640 1 -- 172.21.15.60:0/1306537388 learned_addr learned my addr 172.21.15.60:0/1306537388 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4809400640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48140ce7f0 0x7f48140b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.765+0000 7f4812800640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481415b9b0 0x7f481406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f4811e00640 1 -- 172.21.15.60:0/1306537388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481415b9b0 msgr2=0x7f481406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f4811e00640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481415b9b0 0x7f481406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f4811e00640 1 -- 172.21.15.60:0/1306537388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48140ce7f0 msgr2=0x7f48140b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f4811e00640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48140ce7f0 0x7f48140b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f4811e00640 1 -- 172.21.15.60:0/1306537388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48141f49d0 con 0x7f48140caa90 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f4811e00640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48140caa90 0x7f48140b40b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4804034040 tx=0x7f48040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f480aa00640 1 -- 172.21.15.60:0/1306537388 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480403e030 con 0x7f48140caa90 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f480aa00640 1 -- 172.21.15.60:0/1306537388 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4804003ca0 con 0x7f48140caa90 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f480aa00640 1 -- 172.21.15.60:0/1306537388 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804035040 con 0x7f48140caa90 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48141f4cb0 con 0x7f48140caa90 2024-09-16T15:47:23.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.766+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48141f5270 con 0x7f48140caa90 2024-09-16T15:47:23.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.767+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48140ad880 con 0x7f48140caa90 2024-09-16T15:47:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.769+0000 7f480aa00640 1 -- 172.21.15.60:0/1306537388 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f4804041030 con 0x7f48140caa90 2024-09-16T15:47:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.769+0000 7f480aa00640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f47f00790c0 0x7f47f007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.770+0000 7f480aa00640 1 -- 172.21.15.60:0/1306537388 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 4885+0+0 (secure 0 0 0) 0x7f48040bbcd0 con 0x7f48140caa90 2024-09-16T15:47:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.770+0000 7f4809400640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f47f00790c0 0x7f47f007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.770+0000 7f4809400640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f47f00790c0 0x7f47f007b580 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f481415ab60 tx=0x7f47fc007e30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:23.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.774+0000 7f480aa00640 1 -- 172.21.15.60:0/1306537388 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4804082ff0 con 0x7f48140caa90 2024-09-16T15:47:23.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:23.972+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f48140b59f0 con 0x7f48140caa90 2024-09-16T15:47:24.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.706+0000 7f480aa00640 1 -- 172.21.15.60:0/1306537388 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v158) ==== 138+0+0 (secure 0 0 0) 0x7f4804087a20 con 0x7f48140caa90 2024-09-16T15:47:24.707 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:47:24.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f47f00790c0 msgr2=0x7f47f007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:24.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f47f00790c0 0x7f47f007b580 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f481415ab60 tx=0x7f47fc007e30 comp rx=0 tx=0).stop 2024-09-16T15:47:24.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48140caa90 msgr2=0x7f48140b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:24.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48140caa90 0x7f48140b40b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4804034040 tx=0x7f48040038a0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 shutdown_connections 2024-09-16T15:47:24.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f47f00790c0 0x7f47f007b580 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f481415b9b0 0x7f481406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48140ce7f0 0x7f48140b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 --2- 172.21.15.60:0/1306537388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48140caa90 0x7f48140b40b0 secure :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4804034040 tx=0x7f48040038a0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.709+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 >> 172.21.15.60:0/1306537388 conn(0x7f48140cfa70 msgr2=0x7f48140aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:24.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.710+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 shutdown_connections 2024-09-16T15:47:24.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.710+0000 7f4813200640 1 -- 172.21.15.60:0/1306537388 wait complete. 2024-09-16T15:47:24.734 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:47:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.905+0000 7f6677200640 1 -- 172.21.15.60:0/1307192710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 msgr2=0x7f66780b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.905+0000 7f6677200640 1 --2- 172.21.15.60:0/1307192710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 0x7f66780b7f90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6670009ca0 tx=0x7f667002aac0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.905+0000 7f6677200640 1 -- 172.21.15.60:0/1307192710 shutdown_connections 2024-09-16T15:47:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.905+0000 7f6677200640 1 --2- 172.21.15.60:0/1307192710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 0x7f66780b7f90 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.905+0000 7f6677200640 1 --2- 172.21.15.60:0/1307192710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66780d1f00 0x7f66780b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.905+0000 7f6677200640 1 -- 172.21.15.60:0/1307192710 >> 172.21.15.60:0/1307192710 conn(0x7f66780d3e60 msgr2=0x7f66780d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.905+0000 7f6677200640 1 -- 172.21.15.60:0/1307192710 shutdown_connections 2024-09-16T15:47:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.905+0000 7f6677200640 1 -- 172.21.15.60:0/1307192710 wait complete. 2024-09-16T15:47:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.906+0000 7f6677200640 1 Processor -- start 2024-09-16T15:47:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.907+0000 7f6677200640 1 -- start start 2024-09-16T15:47:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6677200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66780d0870 0x7f667815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6677200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 0x7f667809db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6677200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66780d1f00 0x7f66780cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6677200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f66780b8860 con 0x7f66780d1360 2024-09-16T15:47:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6677200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66780b86e0 con 0x7f66780d0870 2024-09-16T15:47:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6677200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f66780b89e0 con 0x7f66780d1f00 2024-09-16T15:47:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f666d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 0x7f667809db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f666d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 0x7f667809db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55678/0 (socket says 172.21.15.60:55678) 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f666d400640 1 -- 172.21.15.60:0/1119507664 learned_addr learned my addr 172.21.15.60:0/1119507664 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6676800640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66780d1f00 0x7f66780cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6675e00640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66780d0870 0x7f667815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f666d400640 1 -- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66780d1f00 msgr2=0x7f66780cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f666d400640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66780d1f00 0x7f66780cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f666d400640 1 -- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66780d0870 msgr2=0x7f667815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f666d400640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66780d0870 0x7f667815cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f666d400640 1 -- 172.21.15.60:0/1119507664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667815d510 con 0x7f66780d1360 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.908+0000 7f6675e00640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66780d0870 0x7f667815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.909+0000 7f666d400640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 0x7f667809db50 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f666000d750 tx=0x7f666000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.909+0000 7f666ea00640 1 -- 172.21.15.60:0/1119507664 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660017070 con 0x7f66780d1360 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.909+0000 7f666ea00640 1 -- 172.21.15.60:0/1119507664 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6660009b20 con 0x7f66780d1360 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.909+0000 7f666ea00640 1 -- 172.21.15.60:0/1119507664 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660010460 con 0x7f66780d1360 2024-09-16T15:47:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.909+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667809e450 con 0x7f66780d1360 2024-09-16T15:47:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.909+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66781f4db0 con 0x7f66780d1360 2024-09-16T15:47:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.910+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66780d0c50 con 0x7f66780d1360 2024-09-16T15:47:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.912+0000 7f666ea00640 1 -- 172.21.15.60:0/1119507664 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f666001a070 con 0x7f66780d1360 2024-09-16T15:47:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.913+0000 7f666ea00640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66540790c0 0x7f665407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.913+0000 7f666ea00640 1 -- 172.21.15.60:0/1119507664 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5255+0+0 (secure 0 0 0) 0x7f666009bb60 con 0x7f66780d1360 2024-09-16T15:47:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.913+0000 7f6675e00640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66540790c0 0x7f665407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.914+0000 7f6675e00640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66540790c0 0x7f665407b580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6668007fd0 tx=0x7f6668009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:24.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:24.918+0000 7f666ea00640 1 -- 172.21.15.60:0/1119507664 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66600633f0 con 0x7f66780d1360 2024-09-16T15:47:25.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.113+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f66780d0650 con 0x7f66780d1360 2024-09-16T15:47:25.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.714+0000 7f666ea00640 1 -- 172.21.15.60:0/1119507664 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v159) ==== 170+0+0 (secure 0 0 0) 0x7f6660062cf0 con 0x7f66780d1360 2024-09-16T15:47:25.715 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:47:25.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66540790c0 msgr2=0x7f665407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:25.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66540790c0 0x7f665407b580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6668007fd0 tx=0x7f6668009040 comp rx=0 tx=0).stop 2024-09-16T15:47:25.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 msgr2=0x7f667809db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:25.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 0x7f667809db50 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f666000d750 tx=0x7f666000dc10 comp rx=0 tx=0).stop 2024-09-16T15:47:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 shutdown_connections 2024-09-16T15:47:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f66540790c0 0x7f665407b580 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66780d1f00 0x7f66780cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66780d1360 0x7f667809db50 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 --2- 172.21.15.60:0/1119507664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66780d0870 0x7f667815cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.717+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 >> 172.21.15.60:0/1119507664 conn(0x7f66780d3e60 msgr2=0x7f667806e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:25.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.718+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 shutdown_connections 2024-09-16T15:47:25.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.718+0000 7f6677200640 1 -- 172.21.15.60:0/1119507664 wait complete. 2024-09-16T15:47:25.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.916+0000 7f4e3f400640 1 -- 172.21.15.60:0/2246756748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380cec60 msgr2=0x7f4e380cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.916+0000 7f4e3f400640 1 --2- 172.21.15.60:0/2246756748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380cec60 0x7f4e380cf090 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4e38074ff0 tx=0x7f4e3002a980 comp rx=0 tx=0).stop 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.917+0000 7f4e3f400640 1 -- 172.21.15.60:0/2246756748 shutdown_connections 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.917+0000 7f4e3f400640 1 --2- 172.21.15.60:0/2246756748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380cec60 0x7f4e380cf090 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.917+0000 7f4e3f400640 1 --2- 172.21.15.60:0/2246756748 >> v2:172.21.15.100:3300/0 conn(0x7f4e380af190 0x7f4e380b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.917+0000 7f4e3f400640 1 --2- 172.21.15.60:0/2246756748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e38161760 0x7f4e380ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.917+0000 7f4e3f400640 1 -- 172.21.15.60:0/2246756748 >> 172.21.15.60:0/2246756748 conn(0x7f4e380b1840 msgr2=0x7f4e380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.917+0000 7f4e3f400640 1 -- 172.21.15.60:0/2246756748 shutdown_connections 2024-09-16T15:47:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.917+0000 7f4e3f400640 1 -- 172.21.15.60:0/2246756748 wait complete. 2024-09-16T15:47:25.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.918+0000 7f4e3f400640 1 Processor -- start 2024-09-16T15:47:25.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.918+0000 7f4e3f400640 1 -- start start 2024-09-16T15:47:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.919+0000 7f4e3f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380af190 0x7f4e381513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.919+0000 7f4e3f400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e380cec60 0x7f4e3815f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.919+0000 7f4e3f400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38161760 0x7f4e381f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.919+0000 7f4e3f400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4e380b3200 con 0x7f4e380af190 2024-09-16T15:47:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.919+0000 7f4e3f400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e380b3080 con 0x7f4e38161760 2024-09-16T15:47:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.919+0000 7f4e3f400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4e38162190 con 0x7f4e380cec60 2024-09-16T15:47:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380af190 0x7f4e381513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380af190 0x7f4e381513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55698/0 (socket says 172.21.15.60:55698) 2024-09-16T15:47:25.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 -- 172.21.15.60:0/332315650 learned_addr learned my addr 172.21.15.60:0/332315650 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:25.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 -- 172.21.15.60:0/332315650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e380cec60 msgr2=0x7f4e3815f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:25.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e380cec60 0x7f4e3815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 -- 172.21.15.60:0/332315650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38161760 msgr2=0x7f4e381f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:25.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38161760 0x7f4e381f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:25.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 -- 172.21.15.60:0/332315650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3815fca0 con 0x7f4e380af190 2024-09-16T15:47:25.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.920+0000 7f4e3ca00640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380af190 0x7f4e381513e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800d740 tx=0x7f4e2800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:25.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.921+0000 7f4e34c00640 1 -- 172.21.15.60:0/332315650 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28017070 con 0x7f4e380af190 2024-09-16T15:47:25.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.921+0000 7f4e34c00640 1 -- 172.21.15.60:0/332315650 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e28009d20 con 0x7f4e380af190 2024-09-16T15:47:25.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.921+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e381f89d0 con 0x7f4e380af190 2024-09-16T15:47:25.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.921+0000 7f4e34c00640 1 -- 172.21.15.60:0/332315650 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280106c0 con 0x7f4e380af190 2024-09-16T15:47:25.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.921+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3815ef60 con 0x7f4e380af190 2024-09-16T15:47:25.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.922+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3815ed50 con 0x7f4e380af190 2024-09-16T15:47:25.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.924+0000 7f4e34c00640 1 -- 172.21.15.60:0/332315650 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f4e2801a070 con 0x7f4e380af190 2024-09-16T15:47:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.925+0000 7f4e34c00640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e180790c0 0x7f4e1807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.925+0000 7f4e34c00640 1 -- 172.21.15.60:0/332315650 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5637+0+0 (secure 0 0 0) 0x7f4e2809bdb0 con 0x7f4e380af190 2024-09-16T15:47:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.925+0000 7f4e37e00640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e180790c0 0x7f4e1807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.926+0000 7f4e37e00640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e180790c0 0x7f4e1807b580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005d80 tx=0x7f4e24005c90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:25.928+0000 7f4e34c00640 1 -- 172.21.15.60:0/332315650 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e280638c0 con 0x7f4e380af190 2024-09-16T15:47:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.163+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4e380cf090 con 0x7f4e380af190 2024-09-16T15:47:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.725+0000 7f4e34c00640 1 -- 172.21.15.60:0/332315650 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v108) ==== 174+0+0 (secure 0 0 0) 0x7f4e280631c0 con 0x7f4e380af190 2024-09-16T15:47:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 24 and data pool 25 2024-09-16T15:47:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.731+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e180790c0 msgr2=0x7f4e1807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.731+0000 7f4e3f400640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e180790c0 0x7f4e1807b580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005d80 tx=0x7f4e24005c90 comp rx=0 tx=0).stop 2024-09-16T15:47:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.731+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380af190 msgr2=0x7f4e381513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.731+0000 7f4e3f400640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380af190 0x7f4e381513e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800d740 tx=0x7f4e2800dc00 comp rx=0 tx=0).stop 2024-09-16T15:47:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.732+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 shutdown_connections 2024-09-16T15:47:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.732+0000 7f4e3f400640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4e180790c0 0x7f4e1807b580 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.732+0000 7f4e3f400640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e38161760 0x7f4e381f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.732+0000 7f4e3f400640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e380cec60 0x7f4e3815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.732+0000 7f4e3f400640 1 --2- 172.21.15.60:0/332315650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e380af190 0x7f4e381513e0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:26.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.732+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 >> 172.21.15.60:0/332315650 conn(0x7f4e380b1840 msgr2=0x7f4e38073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.733+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 shutdown_connections 2024-09-16T15:47:26.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.733+0000 7f4e3f400640 1 -- 172.21.15.60:0/332315650 wait complete. 2024-09-16T15:47:26.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:26.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.926+0000 7fdbc5600640 1 -- 172.21.15.60:0/2692303365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 msgr2=0x7fdbc00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.926+0000 7fdbc5600640 1 --2- 172.21.15.60:0/2692303365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 0x7fdbc00b2440 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0005fd0 tx=0x7fdbb002a540 comp rx=0 tx=0).stop 2024-09-16T15:47:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.926+0000 7fdbc5600640 1 -- 172.21.15.60:0/2692303365 shutdown_connections 2024-09-16T15:47:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.926+0000 7fdbc5600640 1 --2- 172.21.15.60:0/2692303365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbc00ccf00 0x7fdbc00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.926+0000 7fdbc5600640 1 --2- 172.21.15.60:0/2692303365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 0x7fdbc00b2440 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.926+0000 7fdbc5600640 1 -- 172.21.15.60:0/2692303365 >> 172.21.15.60:0/2692303365 conn(0x7fdbc00b1950 msgr2=0x7fdbc00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.926+0000 7fdbc5600640 1 -- 172.21.15.60:0/2692303365 shutdown_connections 2024-09-16T15:47:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.926+0000 7fdbc5600640 1 -- 172.21.15.60:0/2692303365 wait complete. 2024-09-16T15:47:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.928+0000 7fdbc5600640 1 Processor -- start 2024-09-16T15:47:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.928+0000 7fdbc5600640 1 -- start start 2024-09-16T15:47:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbc5600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbc00ad270 0x7fdbc00c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbc5600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbc00ccf00 0x7fdbc01f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbc5600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 0x7fdbc00cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbc5600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc006f2e0 con 0x7fdbc015f5a0 2024-09-16T15:47:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbc5600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc006f160 con 0x7fdbc00ad270 2024-09-16T15:47:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbc5600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc006f460 con 0x7fdbc00ccf00 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 0x7fdbc00cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 0x7fdbc00cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55708/0 (socket says 172.21.15.60:55708) 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbf400640 1 -- 172.21.15.60:0/3200466345 learned_addr learned my addr 172.21.15.60:0/3200466345 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbe000640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbc00ccf00 0x7fdbc01f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbea00640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbc00ad270 0x7fdbc00c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbf400640 1 -- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbc00ccf00 msgr2=0x7fdbc01f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbf400640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbc00ccf00 0x7fdbc01f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbf400640 1 -- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbc00ad270 msgr2=0x7fdbc00c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbf400640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbc00ad270 0x7fdbc00c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbf400640 1 -- 172.21.15.60:0/3200466345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc01e5650 con 0x7fdbc015f5a0 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.929+0000 7fdbbea00640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbc00ad270 0x7fdbc00c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.930+0000 7fdbbf400640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 0x7fdbc00cbfc0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c7d0 tx=0x7fdbb800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.930+0000 7fdbb6a00640 1 -- 172.21.15.60:0/3200466345 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8003b70 con 0x7fdbc015f5a0 2024-09-16T15:47:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.930+0000 7fdbb6a00640 1 -- 172.21.15.60:0/3200466345 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbb8003d10 con 0x7fdbc015f5a0 2024-09-16T15:47:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.930+0000 7fdbb6a00640 1 -- 172.21.15.60:0/3200466345 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800aab0 con 0x7fdbc015f5a0 2024-09-16T15:47:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.930+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc01e5930 con 0x7fdbc015f5a0 2024-09-16T15:47:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.930+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc00c92b0 con 0x7fdbc015f5a0 2024-09-16T15:47:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.931+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc006f9f0 con 0x7fdbc015f5a0 2024-09-16T15:47:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.932+0000 7fdbb6a00640 1 -- 172.21.15.60:0/3200466345 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fdbb800ac50 con 0x7fdbc015f5a0 2024-09-16T15:47:26.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.933+0000 7fdbb6a00640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdba00790c0 0x7fdba007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:26.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.933+0000 7fdbb6a00640 1 -- 172.21.15.60:0/3200466345 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5737+0+0 (secure 0 0 0) 0x7fdbb809ba90 con 0x7fdbc015f5a0 2024-09-16T15:47:26.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.934+0000 7fdbbea00640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdba00790c0 0x7fdba007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:26.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.934+0000 7fdbbea00640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdba00790c0 0x7fdba007b580 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0033000 tx=0x7fdbb0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:26.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:26.937+0000 7fdbb6a00640 1 -- 172.21.15.60:0/3200466345 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdbb80630b0 con 0x7fdbc015f5a0 2024-09-16T15:47:27.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.134+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbc00af5d0 con 0x7fdbc015f5a0 2024-09-16T15:47:27.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.136+0000 7fdbb6a00640 1 -- 172.21.15.60:0/3200466345 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) ==== 74+0+16796 (secure 0 0 0) 0x7fdbb80629b0 con 0x7fdbc015f5a0 2024-09-16T15:47:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:27.137 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":160,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:47:26.165043+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:47:23.982547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-16T15:47:25.124723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdba00790c0 msgr2=0x7fdba007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdba00790c0 0x7fdba007b580 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0033000 tx=0x7fdbb0004730 comp rx=0 tx=0).stop 2024-09-16T15:47:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 msgr2=0x7fdbc00cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 0x7fdbc00cbfc0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c7d0 tx=0x7fdbb800cc90 comp rx=0 tx=0).stop 2024-09-16T15:47:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 shutdown_connections 2024-09-16T15:47:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdba00790c0 0x7fdba007b580 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdbc015f5a0 0x7fdbc00cbfc0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdbc00ccf00 0x7fdbc01f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 --2- 172.21.15.60:0/3200466345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbc00ad270 0x7fdbc00c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 >> 172.21.15.60:0/3200466345 conn(0x7fdbc00b1950 msgr2=0x7fdbc00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:27.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 shutdown_connections 2024-09-16T15:47:27.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.138+0000 7fdbc5600640 1 -- 172.21.15.60:0/3200466345 wait complete. 2024-09-16T15:47:27.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:27.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.316+0000 7fd6d9400640 1 -- 172.21.15.60:0/358071471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004800 msgr2=0x7fd6b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.316+0000 7fd6d9400640 1 --2- 172.21.15.60:0/358071471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004800 0x7fd6b8004c80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009b20 tx=0x7fd6b8005830 comp rx=0 tx=0).stop 2024-09-16T15:47:27.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.316+0000 7fd6d9400640 1 -- 172.21.15.60:0/358071471 shutdown_connections 2024-09-16T15:47:27.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.316+0000 7fd6d9400640 1 --2- 172.21.15.60:0/358071471 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6b8004800 0x7fd6b8004c80 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.316+0000 7fd6d9400640 1 -- 172.21.15.60:0/358071471 >> 172.21.15.60:0/358071471 conn(0x7fd6d40cfa20 msgr2=0x7fd6d40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:27.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.316+0000 7fd6d9400640 1 -- 172.21.15.60:0/358071471 shutdown_connections 2024-09-16T15:47:27.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.316+0000 7fd6d9400640 1 -- 172.21.15.60:0/358071471 wait complete. 2024-09-16T15:47:27.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.317+0000 7fd6d9400640 1 Processor -- start 2024-09-16T15:47:27.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.318+0000 7fd6d9400640 1 -- start start 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.318+0000 7fd6d9400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b8004800 0x7fd6d41525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.318+0000 7fd6d9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d41506e0 0x7fd6d41f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.318+0000 7fd6d9400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d415f3b0 0x7fd6d41f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.318+0000 7fd6d9400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d40b2c80 con 0x7fd6d415f3b0 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.318+0000 7fd6d9400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d40b2b00 con 0x7fd6d41506e0 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.318+0000 7fd6d9400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d4073d40 con 0x7fd6b8004800 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d415f3b0 0x7fd6d41f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d415f3b0 0x7fd6d41f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55728/0 (socket says 172.21.15.60:55728) 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 -- 172.21.15.60:0/3559993760 learned_addr learned my addr 172.21.15.60:0/3559993760 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:27.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d3e00640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b8004800 0x7fd6d41525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6cb400640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d41506e0 0x7fd6d41f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 -- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b8004800 msgr2=0x7fd6d41525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b8004800 0x7fd6d41525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 -- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d41506e0 msgr2=0x7fd6d41f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d41506e0 0x7fd6d41f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 -- 172.21.15.60:0/3559993760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d4153a20 con 0x7fd6d415f3b0 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6cb400640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d41506e0 0x7fd6d41f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.319+0000 7fd6d8a00640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d415f3b0 0x7fd6d41f8190 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4072a50 tx=0x7fd6cc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.320+0000 7fd6d0c00640 1 -- 172.21.15.60:0/3559993760 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00c990 con 0x7fd6d415f3b0 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.320+0000 7fd6d0c00640 1 -- 172.21.15.60:0/3559993760 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6cc018070 con 0x7fd6d415f3b0 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.320+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d4150b70 con 0x7fd6d415f3b0 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.320+0000 7fd6d0c00640 1 -- 172.21.15.60:0/3559993760 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00ce80 con 0x7fd6d415f3b0 2024-09-16T15:47:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.320+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d4153240 con 0x7fd6d415f3b0 2024-09-16T15:47:27.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.321+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d40742a0 con 0x7fd6d415f3b0 2024-09-16T15:47:27.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.322+0000 7fd6d0c00640 1 -- 172.21.15.60:0/3559993760 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fd6cc01d460 con 0x7fd6d415f3b0 2024-09-16T15:47:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.323+0000 7fd6d0c00640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd6b00790c0 0x7fd6b007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.323+0000 7fd6d0c00640 1 -- 172.21.15.60:0/3559993760 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5737+0+0 (secure 0 0 0) 0x7fd6cc09c060 con 0x7fd6d415f3b0 2024-09-16T15:47:27.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.323+0000 7fd6d3e00640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd6b00790c0 0x7fd6b007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:27.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.324+0000 7fd6d3e00640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd6b00790c0 0x7fd6b007b580 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4000c00 tx=0x7fd6c4006b10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:27.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.327+0000 7fd6d0c00640 1 -- 172.21.15.60:0/3559993760 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd6cc063710 con 0x7fd6d415f3b0 2024-09-16T15:47:27.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.525+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6d4157380 con 0x7fd6d415f3b0 2024-09-16T15:47:27.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.526+0000 7fd6d0c00640 1 -- 172.21.15.60:0/3559993760 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) ==== 74+0+16796 (secure 0 0 0) 0x7fd6cc063010 con 0x7fd6d415f3b0 2024-09-16T15:47:27.528 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:27.528 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":160,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:47:26.165043+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:47:23.982547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-16T15:47:25.124723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd6b00790c0 msgr2=0x7fd6b007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd6b00790c0 0x7fd6b007b580 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4000c00 tx=0x7fd6c4006b10 comp rx=0 tx=0).stop 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d415f3b0 msgr2=0x7fd6d41f8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d415f3b0 0x7fd6d41f8190 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4072a50 tx=0x7fd6cc00ec50 comp rx=0 tx=0).stop 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 shutdown_connections 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd6b00790c0 0x7fd6b007b580 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6d415f3b0 0x7fd6d41f8190 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6d41506e0 0x7fd6d41f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 --2- 172.21.15.60:0/3559993760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6b8004800 0x7fd6d41525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 >> 172.21.15.60:0/3559993760 conn(0x7fd6d40cfa20 msgr2=0x7fd6d41578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 shutdown_connections 2024-09-16T15:47:27.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.529+0000 7fd6d9400640 1 -- 172.21.15.60:0/3559993760 wait complete. 2024-09-16T15:47:27.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:47:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.709+0000 7f3087c00640 1 -- 172.21.15.60:0/4014154106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30800cccd0 msgr2=0x7f3080073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.709+0000 7f3087c00640 1 --2- 172.21.15.60:0/4014154106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30800cccd0 0x7f3080073440 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f306c005fd0 tx=0x7f306c02a540 comp rx=0 tx=0).stop 2024-09-16T15:47:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.709+0000 7f3087c00640 1 -- 172.21.15.60:0/4014154106 shutdown_connections 2024-09-16T15:47:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.709+0000 7f3087c00640 1 --2- 172.21.15.60:0/4014154106 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f308006efa0 0x7f30800cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.709+0000 7f3087c00640 1 --2- 172.21.15.60:0/4014154106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30800cccd0 0x7f3080073440 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.709+0000 7f3087c00640 1 -- 172.21.15.60:0/4014154106 >> 172.21.15.60:0/4014154106 conn(0x7f30800b1840 msgr2=0x7f30800b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.709+0000 7f3087c00640 1 -- 172.21.15.60:0/4014154106 shutdown_connections 2024-09-16T15:47:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.709+0000 7f3087c00640 1 -- 172.21.15.60:0/4014154106 wait complete. 2024-09-16T15:47:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.710+0000 7f3087c00640 1 Processor -- start 2024-09-16T15:47:27.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.711+0000 7f3087c00640 1 -- start start 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3087c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f308006efa0 0x7f30801584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3087c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30800cccd0 0x7f30800c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3087c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3080145750 0x7f30800ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3087c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f30800ce480 con 0x7f3080145750 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3087c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f30800ce300 con 0x7f308006efa0 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3087c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f30800ce600 con 0x7f30800cccd0 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3080145750 0x7f30800ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3080145750 0x7f30800ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55744/0 (socket says 172.21.15.60:55744) 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085c00640 1 -- 172.21.15.60:0/2151852023 learned_addr learned my addr 172.21.15.60:0/2151852023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085200640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f308006efa0 0x7f30801584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f307fe00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30800cccd0 0x7f30800c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085c00640 1 -- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30800cccd0 msgr2=0x7f30800c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30800cccd0 0x7f30800c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085c00640 1 -- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f308006efa0 msgr2=0x7f30801584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f308006efa0 0x7f30801584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085c00640 1 -- 172.21.15.60:0/2151852023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30800c5560 con 0x7f3080145750 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.712+0000 7f3085200640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f308006efa0 0x7f30801584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.713+0000 7f3085c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3080145750 0x7f30800ad870 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f307800da70 tx=0x7f307800df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.713+0000 7f307cc00640 1 -- 172.21.15.60:0/2151852023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078017070 con 0x7f3080145750 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.713+0000 7f307cc00640 1 -- 172.21.15.60:0/2151852023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3078003970 con 0x7f3080145750 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.713+0000 7f307cc00640 1 -- 172.21.15.60:0/2151852023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078003dc0 con 0x7f3080145750 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.713+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30800c5840 con 0x7f3080145750 2024-09-16T15:47:27.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.713+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30800c60e0 con 0x7f3080145750 2024-09-16T15:47:27.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.714+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30800cdbb0 con 0x7f3080145750 2024-09-16T15:47:27.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.715+0000 7f307cc00640 1 -- 172.21.15.60:0/2151852023 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f3078003f60 con 0x7f3080145750 2024-09-16T15:47:27.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.716+0000 7f307cc00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f30600790c0 0x7f306007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:27.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.716+0000 7f307cc00640 1 -- 172.21.15.60:0/2151852023 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5737+0+0 (secure 0 0 0) 0x7f307809c0c0 con 0x7f3080145750 2024-09-16T15:47:27.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.717+0000 7f3085200640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f30600790c0 0x7f306007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:27.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.717+0000 7f3085200640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f30600790c0 0x7f306007b580 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3074000f60 tx=0x7f3074009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:27.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.721+0000 7f307cc00640 1 -- 172.21.15.60:0/2151852023 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3078063770 con 0x7f3080145750 2024-09-16T15:47:27.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:27.960+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f308006ece0 con 0x7f3080145750 2024-09-16T15:47:28.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.737+0000 7f307cc00640 1 -- 172.21.15.60:0/2151852023 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v111) ==== 118+0+0 (secure 0 0 0) 0x7f3078063070 con 0x7f3080145750 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f30600790c0 msgr2=0x7f306007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f30600790c0 0x7f306007b580 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3074000f60 tx=0x7f3074009040 comp rx=0 tx=0).stop 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3080145750 msgr2=0x7f30800ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3080145750 0x7f30800ad870 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f307800da70 tx=0x7f307800df30 comp rx=0 tx=0).stop 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 shutdown_connections 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f30600790c0 0x7f306007b580 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3080145750 0x7f30800ad870 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30800cccd0 0x7f30800c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.740+0000 7f3087c00640 1 --2- 172.21.15.60:0/2151852023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f308006efa0 0x7f30801584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.741+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 >> 172.21.15.60:0/2151852023 conn(0x7f30800b1840 msgr2=0x7f30800aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:28.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.741+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 shutdown_connections 2024-09-16T15:47:28.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.741+0000 7f3087c00640 1 -- 172.21.15.60:0/2151852023 wait complete. 2024-09-16T15:47:28.761 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:47:28.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:47:28.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.927+0000 7ff2bca00640 1 -- 172.21.15.60:0/1798730803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80af190 msgr2=0x7ff2b80b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.927+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1798730803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80af190 0x7ff2b80b2b40 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0005fd0 tx=0x7ff2a002a540 comp rx=0 tx=0).stop 2024-09-16T15:47:28.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.927+0000 7ff2bca00640 1 -- 172.21.15.60:0/1798730803 shutdown_connections 2024-09-16T15:47:28.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.927+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1798730803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80af190 0x7ff2b80b2b40 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.927+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1798730803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2b8161760 0x7ff2b80ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.927+0000 7ff2bca00640 1 -- 172.21.15.60:0/1798730803 >> 172.21.15.60:0/1798730803 conn(0x7ff2b80b1840 msgr2=0x7ff2b80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:28.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.928+0000 7ff2bca00640 1 -- 172.21.15.60:0/1798730803 shutdown_connections 2024-09-16T15:47:28.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.928+0000 7ff2bca00640 1 -- 172.21.15.60:0/1798730803 wait complete. 2024-09-16T15:47:28.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.929+0000 7ff2bca00640 1 Processor -- start 2024-09-16T15:47:28.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.929+0000 7ff2bca00640 1 -- start start 2024-09-16T15:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2bca00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b80af190 0x7ff2b81580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2bca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80cec60 0x7ff2b81f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2bca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2b8161760 0x7ff2b80cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2bca00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8162a20 con 0x7ff2b80cec60 2024-09-16T15:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2bca00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81628a0 con 0x7ff2b80af190 2024-09-16T15:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2bca00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b8162ba0 con 0x7ff2b8161760 2024-09-16T15:47:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2ad400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80cec60 0x7ff2b81f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2ad400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80cec60 0x7ff2b81f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44364/0 (socket says 172.21.15.60:44364) 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.930+0000 7ff2ad400640 1 -- 172.21.15.60:0/1378007672 learned_addr learned my addr 172.21.15.60:0/1378007672 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2b5e00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b80af190 0x7ff2b81580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2b6800640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2b8161760 0x7ff2b80cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2ad400640 1 -- 172.21.15.60:0/1378007672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2b8161760 msgr2=0x7ff2b80cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2ad400640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2b8161760 0x7ff2b80cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2ad400640 1 -- 172.21.15.60:0/1378007672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b80af190 msgr2=0x7ff2b81580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2ad400640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b80af190 0x7ff2b81580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2ad400640 1 -- 172.21.15.60:0/1378007672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b81f4ce0 con 0x7ff2b80cec60 2024-09-16T15:47:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2ad400640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80cec60 0x7ff2b81f5480 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009850 tx=0x7ff2a00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.931+0000 7ff2aea00640 1 -- 172.21.15.60:0/1378007672 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a003f070 con 0x7ff2b80cec60 2024-09-16T15:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.932+0000 7ff2aea00640 1 -- 172.21.15.60:0/1378007672 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2a00047a0 con 0x7ff2b80cec60 2024-09-16T15:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.932+0000 7ff2aea00640 1 -- 172.21.15.60:0/1378007672 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0035030 con 0x7ff2b80cec60 2024-09-16T15:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.932+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b81559f0 con 0x7ff2b80cec60 2024-09-16T15:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.932+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b81f4f80 con 0x7ff2b80cec60 2024-09-16T15:47:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.933+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b815e380 con 0x7ff2b80cec60 2024-09-16T15:47:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.934+0000 7ff2aea00640 1 -- 172.21.15.60:0/1378007672 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7ff2a0042070 con 0x7ff2b80cec60 2024-09-16T15:47:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.935+0000 7ff2aea00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2940790c0 0x7ff29407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.936+0000 7ff2aea00640 1 -- 172.21.15.60:0/1378007672 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7ff2a00bc9f0 con 0x7ff2b80cec60 2024-09-16T15:47:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.936+0000 7ff2b5e00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2940790c0 0x7ff29407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.936+0000 7ff2b5e00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2940790c0 0x7ff29407b580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800bb50 tx=0x7ff2a800b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:28.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:28.940+0000 7ff2aea00640 1 -- 172.21.15.60:0/1378007672 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff2a0083a50 con 0x7ff2b80cec60 2024-09-16T15:47:29.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.175+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff2b815e0a0 con 0x7ff2b80cec60 2024-09-16T15:47:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.739+0000 7ff2aea00640 1 -- 172.21.15.60:0/1378007672 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v112) ==== 122+0+0 (secure 0 0 0) 0x7ff2a0088480 con 0x7ff2b80cec60 2024-09-16T15:47:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.741+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2940790c0 msgr2=0x7ff29407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.741+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2940790c0 0x7ff29407b580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800bb50 tx=0x7ff2a800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.741+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80cec60 msgr2=0x7ff2b81f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.741+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80cec60 0x7ff2b81f5480 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009850 tx=0x7ff2a00038a0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.742+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 shutdown_connections 2024-09-16T15:47:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.742+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff2940790c0 0x7ff29407b580 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.742+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2b8161760 0x7ff2b80cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.742+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2b80cec60 0x7ff2b81f5480 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.742+0000 7ff2bca00640 1 --2- 172.21.15.60:0/1378007672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2b80af190 0x7ff2b81580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.742+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 >> 172.21.15.60:0/1378007672 conn(0x7ff2b80b1840 msgr2=0x7ff2b80ae070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.742+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 shutdown_connections 2024-09-16T15:47:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.742+0000 7ff2bca00640 1 -- 172.21.15.60:0/1378007672 wait complete. 2024-09-16T15:47:29.761 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:47:29.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:29.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.925+0000 7f9087000640 1 -- 172.21.15.60:0/3954500842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9068003f70 msgr2=0x7f9068004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.925+0000 7f9087000640 1 --2- 172.21.15.60:0/3954500842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9068003f70 0x7f9068004800 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f906c001eb0 tx=0x7f9068005830 comp rx=0 tx=0).stop 2024-09-16T15:47:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.925+0000 7f9086600640 1 -- 172.21.15.60:0/3954500842 reap_dead start 2024-09-16T15:47:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.925+0000 7f9087000640 1 -- 172.21.15.60:0/3954500842 shutdown_connections 2024-09-16T15:47:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.925+0000 7f9087000640 1 -- 172.21.15.60:0/3954500842 >> 172.21.15.60:0/3954500842 conn(0x7f90800cfa20 msgr2=0x7f90800cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.925+0000 7f9087000640 1 -- 172.21.15.60:0/3954500842 shutdown_connections 2024-09-16T15:47:29.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.925+0000 7f9087000640 1 -- 172.21.15.60:0/3954500842 wait complete. 2024-09-16T15:47:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.927+0000 7f9087000640 1 Processor -- start 2024-09-16T15:47:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.927+0000 7f9087000640 1 -- start start 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9087000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9068003f70 0x7f908015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9087000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90800cd4f0 0x7f908015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9087000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9080150290 0x7f9080157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9087000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9080074660 con 0x7f9080150290 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9087000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f90800b28f0 con 0x7f90800cd4f0 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9087000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f90801f4940 con 0x7f9068003f70 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9086600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9080150290 0x7f9080157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9086600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9080150290 0x7f9080157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44376/0 (socket says 172.21.15.60:44376) 2024-09-16T15:47:29.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9086600640 1 -- 172.21.15.60:0/2757126262 learned_addr learned my addr 172.21.15.60:0/2757126262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9085200640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90800cd4f0 0x7f908015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.928+0000 7f9085c00640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9068003f70 0x7f908015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f9086600640 1 -- 172.21.15.60:0/2757126262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9068003f70 msgr2=0x7f908015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f9086600640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9068003f70 0x7f908015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f9086600640 1 -- 172.21.15.60:0/2757126262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90800cd4f0 msgr2=0x7f908015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f9086600640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90800cd4f0 0x7f908015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f9086600640 1 -- 172.21.15.60:0/2757126262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90801f8f80 con 0x7f9080150290 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f9086600640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9080150290 0x7f9080157b40 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9080070640 tx=0x7f907800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f907d600640 1 -- 172.21.15.60:0/2757126262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078004b20 con 0x7f9080150290 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f907d600640 1 -- 172.21.15.60:0/2757126262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9078004cc0 con 0x7f9080150290 2024-09-16T15:47:29.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90800cdf90 con 0x7f9080150290 2024-09-16T15:47:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f907d600640 1 -- 172.21.15.60:0/2757126262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078003bd0 con 0x7f9080150290 2024-09-16T15:47:29.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.929+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9080153130 con 0x7f9080150290 2024-09-16T15:47:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.930+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9080073d40 con 0x7f9080150290 2024-09-16T15:47:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.931+0000 7f907d600640 1 -- 172.21.15.60:0/2757126262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f9078004030 con 0x7f9080150290 2024-09-16T15:47:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.932+0000 7f907d600640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90600790f0 0x7f906007b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.933+0000 7f907d600640 1 -- 172.21.15.60:0/2757126262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f907809c4d0 con 0x7f9080150290 2024-09-16T15:47:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.933+0000 7f9085c00640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90600790f0 0x7f906007b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.933+0000 7f9085c00640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90600790f0 0x7f906007b5b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9074006fd0 tx=0x7f9074006b10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:29.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:29.937+0000 7f907d600640 1 -- 172.21.15.60:0/2757126262 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f90780634b0 con 0x7f9080150290 2024-09-16T15:47:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.170+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9080159ff0 con 0x7f9080150290 2024-09-16T15:47:30.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.171+0000 7f907d600640 1 -- 172.21.15.60:0/2757126262 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5239 (secure 0 0 0) 0x7f9078067ee0 con 0x7f9080150290 2024-09-16T15:47:30.172 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:30.173 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"btime":"2024-09-16T15:47:29:738014+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7361,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1464936997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1464936997},{"type":"v1","addr":"172.21.15.60:6834","nonce":1464936997}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":7364,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1611348877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1611348877},{"type":"v1","addr":"172.21.15.60:6835","nonce":1611348877}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2321854052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2321854052},{"type":"v1","addr":"172.21.15.100:6834","nonce":2321854052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:47:26.164998+0000","modified":"2024-09-16T15:47:29.176697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"d","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3325202304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":3325202304},{"type":"v1","addr":"172.21.15.100:6835","nonce":3325202304}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":12}]} 2024-09-16T15:47:30.173 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 112 2024-09-16T15:47:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.173+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90600790f0 msgr2=0x7f906007b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.173+0000 7f9087000640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90600790f0 0x7f906007b5b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9074006fd0 tx=0x7f9074006b10 comp rx=0 tx=0).stop 2024-09-16T15:47:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.173+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9080150290 msgr2=0x7f9080157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.173+0000 7f9087000640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9080150290 0x7f9080157b40 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9080070640 tx=0x7f907800f3c0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.174+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 shutdown_connections 2024-09-16T15:47:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.174+0000 7f9087000640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f90600790f0 0x7f906007b5b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.174+0000 7f9087000640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9080150290 0x7f9080157b40 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.174+0000 7f9087000640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90800cd4f0 0x7f908015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.174+0000 7f9087000640 1 --2- 172.21.15.60:0/2757126262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9068003f70 0x7f908015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.174+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 >> 172.21.15.60:0/2757126262 conn(0x7f90800cfa20 msgr2=0x7f9080150900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.174+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 shutdown_connections 2024-09-16T15:47:30.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.174+0000 7f9087000640 1 -- 172.21.15.60:0/2757126262 wait complete. 2024-09-16T15:47:30.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.351+0000 7f1897000640 1 -- 172.21.15.60:0/4027590245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 msgr2=0x7f18900b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.351+0000 7f1897000640 1 --2- 172.21.15.60:0/4027590245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 0x7f18900b2440 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1880009860 tx=0x7f188002a510 comp rx=0 tx=0).stop 2024-09-16T15:47:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.351+0000 7f1897000640 1 -- 172.21.15.60:0/4027590245 shutdown_connections 2024-09-16T15:47:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.351+0000 7f1897000640 1 --2- 172.21.15.60:0/4027590245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 0x7f18900b2440 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.351+0000 7f1897000640 1 -- 172.21.15.60:0/4027590245 >> 172.21.15.60:0/4027590245 conn(0x7f18900b1950 msgr2=0x7f18900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.351+0000 7f1897000640 1 -- 172.21.15.60:0/4027590245 shutdown_connections 2024-09-16T15:47:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.351+0000 7f1897000640 1 -- 172.21.15.60:0/4027590245 wait complete. 2024-09-16T15:47:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.353+0000 7f1897000640 1 Processor -- start 2024-09-16T15:47:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.353+0000 7f1897000640 1 -- start start 2024-09-16T15:47:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f1897000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1890073fb0 0x7f189014a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f1897000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900ccf00 0x7f18900c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f1897000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 0x7f18900c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f1897000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f189006ef30 con 0x7f18900ccf00 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f1897000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f189006fa80 con 0x7f1890073fb0 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f1897000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1890074460 con 0x7f189015f5a0 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f188f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900ccf00 0x7f18900c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f188f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900ccf00 0x7f18900c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44392/0 (socket says 172.21.15.60:44392) 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f188f400640 1 -- 172.21.15.60:0/4002147066 learned_addr learned my addr 172.21.15.60:0/4002147066 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f188fe00640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1890073fb0 0x7f189014a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f1895000640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 0x7f18900c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.354+0000 7f188f400640 1 -- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 msgr2=0x7f18900c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f188f400640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 0x7f18900c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f188f400640 1 -- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1890073fb0 msgr2=0x7f189014a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f188f400640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1890073fb0 0x7f189014a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f188f400640 1 -- 172.21.15.60:0/4002147066 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18900c4bc0 con 0x7f18900ccf00 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f1895000640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 0x7f18900c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f188f400640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900ccf00 0x7f18900c6cf0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c430 tx=0x7f187c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f1887e00640 1 -- 172.21.15.60:0/4002147066 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c017070 con 0x7f18900ccf00 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f1887e00640 1 -- 172.21.15.60:0/4002147066 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f187c0053e0 con 0x7f18900ccf00 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f1887e00640 1 -- 172.21.15.60:0/4002147066 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c003850 con 0x7f18900ccf00 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18900c4ea0 con 0x7f18900ccf00 2024-09-16T15:47:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.355+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18900c5460 con 0x7f18900ccf00 2024-09-16T15:47:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.356+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18900b2440 con 0x7f18900ccf00 2024-09-16T15:47:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.358+0000 7f1887e00640 1 -- 172.21.15.60:0/4002147066 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f187c01a070 con 0x7f18900ccf00 2024-09-16T15:47:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.358+0000 7f1887e00640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f18700790c0 0x7f187007b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:30.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.359+0000 7f1887e00640 1 -- 172.21.15.60:0/4002147066 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f187c0984d0 con 0x7f18900ccf00 2024-09-16T15:47:30.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.359+0000 7f188fe00640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f18700790c0 0x7f187007b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:30.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.359+0000 7f188fe00640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f18700790c0 0x7f187007b580 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1880034000 tx=0x7f1880004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:30.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.363+0000 7f1887e00640 1 -- 172.21.15.60:0/4002147066 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f187c061fa0 con 0x7f18900ccf00 2024-09-16T15:47:30.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.560+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f189014a560 con 0x7f18900ccf00 2024-09-16T15:47:30.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.561+0000 7f1887e00640 1 -- 172.21.15.60:0/4002147066 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+16783 (secure 0 0 0) 0x7f187c0669d0 con 0x7f18900ccf00 2024-09-16T15:47:30.562 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:30.563 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":161,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:47:27.726082+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:47:23.982547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-16T15:47:25.124723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f18700790c0 msgr2=0x7f187007b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f18700790c0 0x7f187007b580 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1880034000 tx=0x7f1880004730 comp rx=0 tx=0).stop 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900ccf00 msgr2=0x7f18900c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900ccf00 0x7f18900c6cf0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c430 tx=0x7f187c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 shutdown_connections 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f18700790c0 0x7f187007b580 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015f5a0 0x7f18900c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900ccf00 0x7f18900c6cf0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 --2- 172.21.15.60:0/4002147066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1890073fb0 0x7f189014a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 >> 172.21.15.60:0/4002147066 conn(0x7f18900b1950 msgr2=0x7f189009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.563+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 shutdown_connections 2024-09-16T15:47:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.564+0000 7f1897000640 1 -- 172.21.15.60:0/4002147066 wait complete. 2024-09-16T15:47:30.581 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:47:30.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:47:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.741+0000 7f7120000640 1 -- 172.21.15.60:0/3074336070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7118073fb0 msgr2=0x7f7118074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.741+0000 7f7120000640 1 --2- 172.21.15.60:0/3074336070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7118073fb0 0x7f7118074440 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7110005fd0 tx=0x7f711002a820 comp rx=0 tx=0).stop 2024-09-16T15:47:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.741+0000 7f7120000640 1 -- 172.21.15.60:0/3074336070 shutdown_connections 2024-09-16T15:47:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.741+0000 7f7120000640 1 --2- 172.21.15.60:0/3074336070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7118073fb0 0x7f7118074440 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.741+0000 7f7120000640 1 --2- 172.21.15.60:0/3074336070 >> v2:172.21.15.100:3300/0 conn(0x7f711815f5a0 0x7f71180b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.741+0000 7f7120000640 1 -- 172.21.15.60:0/3074336070 >> 172.21.15.60:0/3074336070 conn(0x7f71180b1950 msgr2=0x7f71180b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.741+0000 7f7120000640 1 -- 172.21.15.60:0/3074336070 shutdown_connections 2024-09-16T15:47:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.741+0000 7f7120000640 1 -- 172.21.15.60:0/3074336070 wait complete. 2024-09-16T15:47:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.742+0000 7f7120000640 1 Processor -- start 2024-09-16T15:47:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.742+0000 7f7120000640 1 -- start start 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.743+0000 7f7120000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7118073fb0 0x7f7118153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.743+0000 7f7120000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71180ccf00 0x7f711815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.743+0000 7f7120000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711815f5a0 0x7f711815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.743+0000 7f7120000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f711806f2e0 con 0x7f7118073fb0 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.743+0000 7f7120000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f711806f160 con 0x7f711815f5a0 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.743+0000 7f7120000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f711806f460 con 0x7f71180ccf00 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7118073fb0 0x7f7118153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711d600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7118073fb0 0x7f7118153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44402/0 (socket says 172.21.15.60:44402) 2024-09-16T15:47:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711d600640 1 -- 172.21.15.60:0/374800954 learned_addr learned my addr 172.21.15.60:0/374800954 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711e000640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711815f5a0 0x7f711815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711cc00640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71180ccf00 0x7f711815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711d600640 1 -- 172.21.15.60:0/374800954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71180ccf00 msgr2=0x7f711815b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711d600640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71180ccf00 0x7f711815b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711d600640 1 -- 172.21.15.60:0/374800954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711815f5a0 msgr2=0x7f711815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711d600640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711815f5a0 0x7f711815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.744+0000 7f711d600640 1 -- 172.21.15.60:0/374800954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71181f5700 con 0x7f7118073fb0 2024-09-16T15:47:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.745+0000 7f711d600640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7118073fb0 0x7f7118153950 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f710c00e7f0 tx=0x7f710c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.747+0000 7f7115600640 1 -- 172.21.15.60:0/374800954 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c00ccc0 con 0x7f7118073fb0 2024-09-16T15:47:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.747+0000 7f7115600640 1 -- 172.21.15.60:0/374800954 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f710c00ce80 con 0x7f7118073fb0 2024-09-16T15:47:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.747+0000 7f7115600640 1 -- 172.21.15.60:0/374800954 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c005280 con 0x7f7118073fb0 2024-09-16T15:47:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.747+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711815df30 con 0x7f7118073fb0 2024-09-16T15:47:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.747+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711815e4f0 con 0x7f7118073fb0 2024-09-16T15:47:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.748+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711806f9f0 con 0x7f7118073fb0 2024-09-16T15:47:30.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.749+0000 7f7115600640 1 -- 172.21.15.60:0/374800954 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f710c005420 con 0x7f7118073fb0 2024-09-16T15:47:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.750+0000 7f7115600640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f70f80790c0 0x7f70f807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.750+0000 7f7115600640 1 -- 172.21.15.60:0/374800954 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f710c09c4d0 con 0x7f7118073fb0 2024-09-16T15:47:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.751+0000 7f711cc00640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f70f80790c0 0x7f70f807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.751+0000 7f711cc00640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f70f80790c0 0x7f70f807b580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7104007fd0 tx=0x7f7104009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:30.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.756+0000 7f7115600640 1 -- 172.21.15.60:0/374800954 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f710c0634a0 con 0x7f7118073fb0 2024-09-16T15:47:30.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.948+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7118153e90 con 0x7f70f80790c0 2024-09-16T15:47:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.952+0000 7f7115600640 1 -- 172.21.15.60:0/374800954 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252470 (secure 0 0 0) 0x7f71040014d0 con 0x7f70f80790c0 2024-09-16T15:47:30.956 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:47:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f70f80790c0 msgr2=0x7f70f807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f70f80790c0 0x7f70f807b580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7104007fd0 tx=0x7f7104009040 comp rx=0 tx=0).stop 2024-09-16T15:47:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7118073fb0 msgr2=0x7f7118153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7118073fb0 0x7f7118153950 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f710c00e7f0 tx=0x7f710c00ecb0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 shutdown_connections 2024-09-16T15:47:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f70f80790c0 0x7f70f807b580 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711815f5a0 0x7f711815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71180ccf00 0x7f711815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 --2- 172.21.15.60:0/374800954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7118073fb0 0x7f7118153950 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 >> 172.21.15.60:0/374800954 conn(0x7f71180b1950 msgr2=0x7f71180cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 shutdown_connections 2024-09-16T15:47:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:30.956+0000 7f7120000640 1 -- 172.21.15.60:0/374800954 wait complete. 2024-09-16T15:47:30.973 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-16T15:47:30.217395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283020,"kb_used_data":16852,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691700,"statfs":{"total":773094113280,"available":772804300800,"internally_reserved":0,"allocated":17256448,"data_stored":8252513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.510230"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.754891+0000","last_change":"2024-09-16T15:47:26.753571+0000","last_active":"2024-09-16T15:47:26.754891+0000","last_peered":"2024-09-16T15:47:26.754891+0000","last_clean":"2024-09-16T15:47:26.754891+0000","last_became_active":"2024-09-16T15:47:26.753260+0000","last_became_peered":"2024-09-16T15:47:26.753260+0000","last_unstale":"2024-09-16T15:47:26.754891+0000","last_undegraded":"2024-09-16T15:47:26.754891+0000","last_fullsized":"2024-09-16T15:47:26.754891+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:57.730928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752076+0000","last_change":"2024-09-16T15:47:26.744170+0000","last_active":"2024-09-16T15:47:26.752076+0000","last_peered":"2024-09-16T15:47:26.752076+0000","last_clean":"2024-09-16T15:47:26.752076+0000","last_became_active":"2024-09-16T15:47:26.743832+0000","last_became_peered":"2024-09-16T15:47:26.743832+0000","last_unstale":"2024-09-16T15:47:26.752076+0000","last_undegraded":"2024-09-16T15:47:26.752076+0000","last_fullsized":"2024-09-16T15:47:26.752076+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:13:20.684223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742240+0000","last_change":"2024-09-16T15:47:26.738179+0000","last_active":"2024-09-16T15:47:26.742240+0000","last_peered":"2024-09-16T15:47:26.742240+0000","last_clean":"2024-09-16T15:47:26.742240+0000","last_became_active":"2024-09-16T15:47:26.737893+0000","last_became_peered":"2024-09-16T15:47:26.737893+0000","last_unstale":"2024-09-16T15:47:26.742240+0000","last_undegraded":"2024-09-16T15:47:26.742240+0000","last_fullsized":"2024-09-16T15:47:26.742240+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:55.621116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752123+0000","last_change":"2024-09-16T15:47:26.743437+0000","last_active":"2024-09-16T15:47:26.752123+0000","last_peered":"2024-09-16T15:47:26.752123+0000","last_clean":"2024-09-16T15:47:26.752123+0000","last_became_active":"2024-09-16T15:47:26.743096+0000","last_became_peered":"2024-09-16T15:47:26.743096+0000","last_unstale":"2024-09-16T15:47:26.752123+0000","last_undegraded":"2024-09-16T15:47:26.752123+0000","last_fullsized":"2024-09-16T15:47:26.752123+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:03.093584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742514+0000","last_change":"2024-09-16T15:47:26.740088+0000","last_active":"2024-09-16T15:47:26.742514+0000","last_peered":"2024-09-16T15:47:26.742514+0000","last_clean":"2024-09-16T15:47:26.742514+0000","last_became_active":"2024-09-16T15:47:26.739741+0000","last_became_peered":"2024-09-16T15:47:26.739741+0000","last_unstale":"2024-09-16T15:47:26.742514+0000","last_undegraded":"2024-09-16T15:47:26.742514+0000","last_fullsized":"2024-09-16T15:47:26.742514+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:37.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.738170+0000","last_change":"2024-09-16T15:47:26.736173+0000","last_active":"2024-09-16T15:47:26.738170+0000","last_peered":"2024-09-16T15:47:26.738170+0000","last_clean":"2024-09-16T15:47:26.738170+0000","last_became_active":"2024-09-16T15:47:26.735702+0000","last_became_peered":"2024-09-16T15:47:26.735702+0000","last_unstale":"2024-09-16T15:47:26.738170+0000","last_undegraded":"2024-09-16T15:47:26.738170+0000","last_fullsized":"2024-09-16T15:47:26.738170+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:26.299600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744892+0000","last_change":"2024-09-16T15:47:26.738143+0000","last_active":"2024-09-16T15:47:26.744892+0000","last_peered":"2024-09-16T15:47:26.744892+0000","last_clean":"2024-09-16T15:47:26.744892+0000","last_became_active":"2024-09-16T15:47:26.737859+0000","last_became_peered":"2024-09-16T15:47:26.737859+0000","last_unstale":"2024-09-16T15:47:26.744892+0000","last_undegraded":"2024-09-16T15:47:26.744892+0000","last_fullsized":"2024-09-16T15:47:26.744892+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:47:09.265463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746558+0000","last_change":"2024-09-16T15:47:26.740986+0000","last_active":"2024-09-16T15:47:26.746558+0000","last_peered":"2024-09-16T15:47:26.746558+0000","last_clean":"2024-09-16T15:47:26.746558+0000","last_became_active":"2024-09-16T15:47:26.740610+0000","last_became_peered":"2024-09-16T15:47:26.740610+0000","last_unstale":"2024-09-16T15:47:26.746558+0000","last_undegraded":"2024-09-16T15:47:26.746558+0000","last_fullsized":"2024-09-16T15:47:26.746558+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:46.136548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.741063+0000","last_change":"2024-09-16T15:47:26.733836+0000","last_active":"2024-09-16T15:47:26.741063+0000","last_peered":"2024-09-16T15:47:26.741063+0000","last_clean":"2024-09-16T15:47:26.741063+0000","last_became_active":"2024-09-16T15:47:26.733570+0000","last_became_peered":"2024-09-16T15:47:26.733570+0000","last_unstale":"2024-09-16T15:47:26.741063+0000","last_undegraded":"2024-09-16T15:47:26.741063+0000","last_fullsized":"2024-09-16T15:47:26.741063+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:47.030189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.747439+0000","last_change":"2024-09-16T15:47:26.742278+0000","last_active":"2024-09-16T15:47:26.747439+0000","last_peered":"2024-09-16T15:47:26.747439+0000","last_clean":"2024-09-16T15:47:26.747439+0000","last_became_active":"2024-09-16T15:47:26.737702+0000","last_became_peered":"2024-09-16T15:47:26.737702+0000","last_unstale":"2024-09-16T15:47:26.747439+0000","last_undegraded":"2024-09-16T15:47:26.747439+0000","last_fullsized":"2024-09-16T15:47:26.747439+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:02.578133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.748339+0000","last_change":"2024-09-16T15:47:26.743163+0000","last_active":"2024-09-16T15:47:26.748339+0000","last_peered":"2024-09-16T15:47:26.748339+0000","last_clean":"2024-09-16T15:47:26.748339+0000","last_became_active":"2024-09-16T15:47:26.742891+0000","last_became_peered":"2024-09-16T15:47:26.742891+0000","last_unstale":"2024-09-16T15:47:26.748339+0000","last_undegraded":"2024-09-16T15:47:26.748339+0000","last_fullsized":"2024-09-16T15:47:26.748339+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:34.079341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.743752+0000","last_change":"2024-09-16T15:47:26.738168+0000","last_active":"2024-09-16T15:47:26.743752+0000","last_peered":"2024-09-16T15:47:26.743752+0000","last_clean":"2024-09-16T15:47:26.743752+0000","last_became_active":"2024-09-16T15:47:26.737818+0000","last_became_peered":"2024-09-16T15:47:26.737818+0000","last_unstale":"2024-09-16T15:47:26.743752+0000","last_undegraded":"2024-09-16T15:47:26.743752+0000","last_fullsized":"2024-09-16T15:47:26.743752+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:46.073366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744064+0000","last_change":"2024-09-16T15:47:26.735341+0000","last_active":"2024-09-16T15:47:26.744064+0000","last_peered":"2024-09-16T15:47:26.744064+0000","last_clean":"2024-09-16T15:47:26.744064+0000","last_became_active":"2024-09-16T15:47:26.734979+0000","last_became_peered":"2024-09-16T15:47:26.734979+0000","last_unstale":"2024-09-16T15:47:26.744064+0000","last_undegraded":"2024-09-16T15:47:26.744064+0000","last_fullsized":"2024-09-16T15:47:26.744064+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:23.165921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.745852+0000","last_change":"2024-09-16T15:47:26.738951+0000","last_active":"2024-09-16T15:47:26.745852+0000","last_peered":"2024-09-16T15:47:26.745852+0000","last_clean":"2024-09-16T15:47:26.745852+0000","last_became_active":"2024-09-16T15:47:26.738640+0000","last_became_peered":"2024-09-16T15:47:26.738640+0000","last_unstale":"2024-09-16T15:47:26.745852+0000","last_undegraded":"2024-09-16T15:47:26.745852+0000","last_fullsized":"2024-09-16T15:47:26.745852+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:14.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.748182+0000","last_change":"2024-09-16T15:47:26.740431+0000","last_active":"2024-09-16T15:47:26.748182+0000","last_peered":"2024-09-16T15:47:26.748182+0000","last_clean":"2024-09-16T15:47:26.748182+0000","last_became_active":"2024-09-16T15:47:26.739635+0000","last_became_peered":"2024-09-16T15:47:26.739635+0000","last_unstale":"2024-09-16T15:47:26.748182+0000","last_undegraded":"2024-09-16T15:47:26.748182+0000","last_fullsized":"2024-09-16T15:47:26.748182+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:27.641894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746869+0000","last_change":"2024-09-16T15:47:26.741613+0000","last_active":"2024-09-16T15:47:26.746869+0000","last_peered":"2024-09-16T15:47:26.746869+0000","last_clean":"2024-09-16T15:47:26.746869+0000","last_became_active":"2024-09-16T15:47:26.741331+0000","last_became_peered":"2024-09-16T15:47:26.741331+0000","last_unstale":"2024-09-16T15:47:26.746869+0000","last_undegraded":"2024-09-16T15:47:26.746869+0000","last_fullsized":"2024-09-16T15:47:26.746869+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:42.815044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.756500+0000","last_change":"2024-09-16T15:47:26.748721+0000","last_active":"2024-09-16T15:47:26.756500+0000","last_peered":"2024-09-16T15:47:26.756500+0000","last_clean":"2024-09-16T15:47:26.756500+0000","last_became_active":"2024-09-16T15:47:26.748169+0000","last_became_peered":"2024-09-16T15:47:26.748169+0000","last_unstale":"2024-09-16T15:47:26.756500+0000","last_undegraded":"2024-09-16T15:47:26.756500+0000","last_fullsized":"2024-09-16T15:47:26.756500+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:33:32.963714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.754228+0000","last_change":"2024-09-16T15:47:26.750890+0000","last_active":"2024-09-16T15:47:26.754228+0000","last_peered":"2024-09-16T15:47:26.754228+0000","last_clean":"2024-09-16T15:47:26.754228+0000","last_became_active":"2024-09-16T15:47:26.750593+0000","last_became_peered":"2024-09-16T15:47:26.750593+0000","last_unstale":"2024-09-16T15:47:26.754228+0000","last_undegraded":"2024-09-16T15:47:26.754228+0000","last_fullsized":"2024-09-16T15:47:26.754228+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:59:45.858811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744117+0000","last_change":"2024-09-16T15:47:26.733574+0000","last_active":"2024-09-16T15:47:26.744117+0000","last_peered":"2024-09-16T15:47:26.744117+0000","last_clean":"2024-09-16T15:47:26.744117+0000","last_became_active":"2024-09-16T15:47:26.733291+0000","last_became_peered":"2024-09-16T15:47:26.733291+0000","last_unstale":"2024-09-16T15:47:26.744117+0000","last_undegraded":"2024-09-16T15:47:26.744117+0000","last_fullsized":"2024-09-16T15:47:26.744117+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:00:02.550196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.745997+0000","last_change":"2024-09-16T15:47:26.741878+0000","last_active":"2024-09-16T15:47:26.745997+0000","last_peered":"2024-09-16T15:47:26.745997+0000","last_clean":"2024-09-16T15:47:26.745997+0000","last_became_active":"2024-09-16T15:47:26.741509+0000","last_became_peered":"2024-09-16T15:47:26.741509+0000","last_unstale":"2024-09-16T15:47:26.745997+0000","last_undegraded":"2024-09-16T15:47:26.745997+0000","last_fullsized":"2024-09-16T15:47:26.745997+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:49.061178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.745719+0000","last_change":"2024-09-16T15:47:26.743256+0000","last_active":"2024-09-16T15:47:26.745719+0000","last_peered":"2024-09-16T15:47:26.745719+0000","last_clean":"2024-09-16T15:47:26.745719+0000","last_became_active":"2024-09-16T15:47:26.742731+0000","last_became_peered":"2024-09-16T15:47:26.742731+0000","last_unstale":"2024-09-16T15:47:26.745719+0000","last_undegraded":"2024-09-16T15:47:26.745719+0000","last_fullsized":"2024-09-16T15:47:26.745719+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:02.035214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.751080+0000","last_change":"2024-09-16T15:47:26.742963+0000","last_active":"2024-09-16T15:47:26.751080+0000","last_peered":"2024-09-16T15:47:26.751080+0000","last_clean":"2024-09-16T15:47:26.751080+0000","last_became_active":"2024-09-16T15:47:26.739510+0000","last_became_peered":"2024-09-16T15:47:26.739510+0000","last_unstale":"2024-09-16T15:47:26.751080+0000","last_undegraded":"2024-09-16T15:47:26.751080+0000","last_fullsized":"2024-09-16T15:47:26.751080+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:57.594165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752275+0000","last_change":"2024-09-16T15:47:26.749057+0000","last_active":"2024-09-16T15:47:26.752275+0000","last_peered":"2024-09-16T15:47:26.752275+0000","last_clean":"2024-09-16T15:47:26.752275+0000","last_became_active":"2024-09-16T15:47:26.748806+0000","last_became_peered":"2024-09-16T15:47:26.748806+0000","last_unstale":"2024-09-16T15:47:26.752275+0000","last_undegraded":"2024-09-16T15:47:26.752275+0000","last_fullsized":"2024-09-16T15:47:26.752275+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:45:34.284904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746529+0000","last_change":"2024-09-16T15:47:26.740324+0000","last_active":"2024-09-16T15:47:26.746529+0000","last_peered":"2024-09-16T15:47:26.746529+0000","last_clean":"2024-09-16T15:47:26.746529+0000","last_became_active":"2024-09-16T15:47:26.740070+0000","last_became_peered":"2024-09-16T15:47:26.740070+0000","last_unstale":"2024-09-16T15:47:26.746529+0000","last_undegraded":"2024-09-16T15:47:26.746529+0000","last_fullsized":"2024-09-16T15:47:26.746529+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:57.681320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.743411+0000","last_change":"2024-09-16T15:47:26.729352+0000","last_active":"2024-09-16T15:47:26.743411+0000","last_peered":"2024-09-16T15:47:26.743411+0000","last_clean":"2024-09-16T15:47:26.743411+0000","last_became_active":"2024-09-16T15:47:26.729029+0000","last_became_peered":"2024-09-16T15:47:26.729029+0000","last_unstale":"2024-09-16T15:47:26.743411+0000","last_undegraded":"2024-09-16T15:47:26.743411+0000","last_fullsized":"2024-09-16T15:47:26.743411+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:39.365473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742595+0000","last_change":"2024-09-16T15:47:26.735264+0000","last_active":"2024-09-16T15:47:26.742595+0000","last_peered":"2024-09-16T15:47:26.742595+0000","last_clean":"2024-09-16T15:47:26.742595+0000","last_became_active":"2024-09-16T15:47:26.734978+0000","last_became_peered":"2024-09-16T15:47:26.734978+0000","last_unstale":"2024-09-16T15:47:26.742595+0000","last_undegraded":"2024-09-16T15:47:26.742595+0000","last_fullsized":"2024-09-16T15:47:26.742595+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:04.602404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.754950+0000","last_change":"2024-09-16T15:47:26.750922+0000","last_active":"2024-09-16T15:47:26.754950+0000","last_peered":"2024-09-16T15:47:26.754950+0000","last_clean":"2024-09-16T15:47:26.754950+0000","last_became_active":"2024-09-16T15:47:26.746538+0000","last_became_peered":"2024-09-16T15:47:26.746538+0000","last_unstale":"2024-09-16T15:47:26.754950+0000","last_undegraded":"2024-09-16T15:47:26.754950+0000","last_fullsized":"2024-09-16T15:47:26.754950+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:40:29.106016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.741690+0000","last_change":"2024-09-16T15:47:26.739824+0000","last_active":"2024-09-16T15:47:26.741690+0000","last_peered":"2024-09-16T15:47:26.741690+0000","last_clean":"2024-09-16T15:47:26.741690+0000","last_became_active":"2024-09-16T15:47:26.738236+0000","last_became_peered":"2024-09-16T15:47:26.738236+0000","last_unstale":"2024-09-16T15:47:26.741690+0000","last_undegraded":"2024-09-16T15:47:26.741690+0000","last_fullsized":"2024-09-16T15:47:26.741690+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:58.684833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744090+0000","last_change":"2024-09-16T15:47:26.736657+0000","last_active":"2024-09-16T15:47:26.744090+0000","last_peered":"2024-09-16T15:47:26.744090+0000","last_clean":"2024-09-16T15:47:26.744090+0000","last_became_active":"2024-09-16T15:47:26.736391+0000","last_became_peered":"2024-09-16T15:47:26.736391+0000","last_unstale":"2024-09-16T15:47:26.744090+0000","last_undegraded":"2024-09-16T15:47:26.744090+0000","last_fullsized":"2024-09-16T15:47:26.744090+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:18.710208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.751225+0000","last_change":"2024-09-16T15:47:26.743195+0000","last_active":"2024-09-16T15:47:26.751225+0000","last_peered":"2024-09-16T15:47:26.751225+0000","last_clean":"2024-09-16T15:47:26.751225+0000","last_became_active":"2024-09-16T15:47:26.742925+0000","last_became_peered":"2024-09-16T15:47:26.742925+0000","last_unstale":"2024-09-16T15:47:26.751225+0000","last_undegraded":"2024-09-16T15:47:26.751225+0000","last_fullsized":"2024-09-16T15:47:26.751225+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:59.635093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.737392+0000","last_change":"2024-09-16T15:47:26.735156+0000","last_active":"2024-09-16T15:47:26.737392+0000","last_peered":"2024-09-16T15:47:26.737392+0000","last_clean":"2024-09-16T15:47:26.737392+0000","last_became_active":"2024-09-16T15:47:26.734704+0000","last_became_peered":"2024-09-16T15:47:26.734704+0000","last_unstale":"2024-09-16T15:47:26.737392+0000","last_undegraded":"2024-09-16T15:47:26.737392+0000","last_fullsized":"2024-09-16T15:47:26.737392+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:56.067533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.740215+0000","last_change":"2024-09-16T15:47:26.738808+0000","last_active":"2024-09-16T15:47:26.740215+0000","last_peered":"2024-09-16T15:47:26.740215+0000","last_clean":"2024-09-16T15:47:26.740215+0000","last_became_active":"2024-09-16T15:47:26.737472+0000","last_became_peered":"2024-09-16T15:47:26.737472+0000","last_unstale":"2024-09-16T15:47:26.740215+0000","last_undegraded":"2024-09-16T15:47:26.740215+0000","last_fullsized":"2024-09-16T15:47:26.740215+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:30.535633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744409+0000","last_change":"2024-09-16T15:47:26.738481+0000","last_active":"2024-09-16T15:47:26.744409+0000","last_peered":"2024-09-16T15:47:26.744409+0000","last_clean":"2024-09-16T15:47:26.744409+0000","last_became_active":"2024-09-16T15:47:26.738224+0000","last_became_peered":"2024-09-16T15:47:26.738224+0000","last_unstale":"2024-09-16T15:47:26.744409+0000","last_undegraded":"2024-09-16T15:47:26.744409+0000","last_fullsized":"2024-09-16T15:47:26.744409+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:00:33.411582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"160'4","reported_seq":17,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.759137+0000","last_change":"2024-09-16T15:47:25.735971+0000","last_active":"2024-09-16T15:47:26.759137+0000","last_peered":"2024-09-16T15:47:26.759137+0000","last_clean":"2024-09-16T15:47:26.759137+0000","last_became_active":"2024-09-16T15:47:25.732985+0000","last_became_peered":"2024-09-16T15:47:25.732985+0000","last_unstale":"2024-09-16T15:47:26.759137+0000","last_undegraded":"2024-09-16T15:47:26.759137+0000","last_fullsized":"2024-09-16T15:47:26.759137+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:25.835603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757218+0000","last_change":"2024-09-16T15:47:26.748491+0000","last_active":"2024-09-16T15:47:26.757218+0000","last_peered":"2024-09-16T15:47:26.757218+0000","last_clean":"2024-09-16T15:47:26.757218+0000","last_became_active":"2024-09-16T15:47:26.748114+0000","last_became_peered":"2024-09-16T15:47:26.748114+0000","last_unstale":"2024-09-16T15:47:26.757218+0000","last_undegraded":"2024-09-16T15:47:26.757218+0000","last_fullsized":"2024-09-16T15:47:26.757218+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:04.794001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.753488+0000","last_change":"2024-09-16T15:47:25.732835+0000","last_active":"2024-09-16T15:47:26.753488+0000","last_peered":"2024-09-16T15:47:26.753488+0000","last_clean":"2024-09-16T15:47:26.753488+0000","last_became_active":"2024-09-16T15:47:25.731930+0000","last_became_peered":"2024-09-16T15:47:25.731930+0000","last_unstale":"2024-09-16T15:47:26.753488+0000","last_undegraded":"2024-09-16T15:47:26.753488+0000","last_fullsized":"2024-09-16T15:47:26.753488+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:05.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.749807+0000","last_change":"2024-09-16T15:47:26.747282+0000","last_active":"2024-09-16T15:47:26.749807+0000","last_peered":"2024-09-16T15:47:26.749807+0000","last_clean":"2024-09-16T15:47:26.749807+0000","last_became_active":"2024-09-16T15:47:26.746811+0000","last_became_peered":"2024-09-16T15:47:26.746811+0000","last_unstale":"2024-09-16T15:47:26.749807+0000","last_undegraded":"2024-09-16T15:47:26.749807+0000","last_fullsized":"2024-09-16T15:47:26.749807+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:54.122177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758609+0000","last_change":"2024-09-16T15:47:25.741179+0000","last_active":"2024-09-16T15:47:26.758609+0000","last_peered":"2024-09-16T15:47:26.758609+0000","last_clean":"2024-09-16T15:47:26.758609+0000","last_became_active":"2024-09-16T15:47:25.738798+0000","last_became_peered":"2024-09-16T15:47:25.738798+0000","last_unstale":"2024-09-16T15:47:26.758609+0000","last_undegraded":"2024-09-16T15:47:26.758609+0000","last_fullsized":"2024-09-16T15:47:26.758609+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:42:09.276104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746063+0000","last_change":"2024-09-16T15:47:26.739271+0000","last_active":"2024-09-16T15:47:26.746063+0000","last_peered":"2024-09-16T15:47:26.746063+0000","last_clean":"2024-09-16T15:47:26.746063+0000","last_became_active":"2024-09-16T15:47:26.738752+0000","last_became_peered":"2024-09-16T15:47:26.738752+0000","last_unstale":"2024-09-16T15:47:26.746063+0000","last_undegraded":"2024-09-16T15:47:26.746063+0000","last_fullsized":"2024-09-16T15:47:26.746063+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:21.130447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.754665+0000","last_change":"2024-09-16T15:47:25.730098+0000","last_active":"2024-09-16T15:47:26.754665+0000","last_peered":"2024-09-16T15:47:26.754665+0000","last_clean":"2024-09-16T15:47:26.754665+0000","last_became_active":"2024-09-16T15:47:25.729591+0000","last_became_peered":"2024-09-16T15:47:25.729591+0000","last_unstale":"2024-09-16T15:47:26.754665+0000","last_undegraded":"2024-09-16T15:47:26.754665+0000","last_fullsized":"2024-09-16T15:47:26.754665+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:20.008186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.751509+0000","last_change":"2024-09-16T15:47:26.743607+0000","last_active":"2024-09-16T15:47:26.751509+0000","last_peered":"2024-09-16T15:47:26.751509+0000","last_clean":"2024-09-16T15:47:26.751509+0000","last_became_active":"2024-09-16T15:47:26.740918+0000","last_became_peered":"2024-09-16T15:47:26.740918+0000","last_unstale":"2024-09-16T15:47:26.751509+0000","last_undegraded":"2024-09-16T15:47:26.751509+0000","last_fullsized":"2024-09-16T15:47:26.751509+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:02.953258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758839+0000","last_change":"2024-09-16T15:47:25.739585+0000","last_active":"2024-09-16T15:47:26.758839+0000","last_peered":"2024-09-16T15:47:26.758839+0000","last_clean":"2024-09-16T15:47:26.758839+0000","last_became_active":"2024-09-16T15:47:25.739322+0000","last_became_peered":"2024-09-16T15:47:25.739322+0000","last_unstale":"2024-09-16T15:47:26.758839+0000","last_undegraded":"2024-09-16T15:47:26.758839+0000","last_fullsized":"2024-09-16T15:47:26.758839+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:18:34.714937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.750913+0000","last_change":"2024-09-16T15:47:26.746558+0000","last_active":"2024-09-16T15:47:26.750913+0000","last_peered":"2024-09-16T15:47:26.750913+0000","last_clean":"2024-09-16T15:47:26.750913+0000","last_became_active":"2024-09-16T15:47:26.741504+0000","last_became_peered":"2024-09-16T15:47:26.741504+0000","last_unstale":"2024-09-16T15:47:26.750913+0000","last_undegraded":"2024-09-16T15:47:26.750913+0000","last_fullsized":"2024-09-16T15:47:26.750913+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:19.931079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757792+0000","last_change":"2024-09-16T15:47:25.730194+0000","last_active":"2024-09-16T15:47:26.757792+0000","last_peered":"2024-09-16T15:47:26.757792+0000","last_clean":"2024-09-16T15:47:26.757792+0000","last_became_active":"2024-09-16T15:47:25.729739+0000","last_became_peered":"2024-09-16T15:47:25.729739+0000","last_unstale":"2024-09-16T15:47:26.757792+0000","last_undegraded":"2024-09-16T15:47:26.757792+0000","last_fullsized":"2024-09-16T15:47:26.757792+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:35.834157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.748596+0000","last_change":"2024-09-16T15:47:26.741342+0000","last_active":"2024-09-16T15:47:26.748596+0000","last_peered":"2024-09-16T15:47:26.748596+0000","last_clean":"2024-09-16T15:47:26.748596+0000","last_became_active":"2024-09-16T15:47:26.741017+0000","last_became_peered":"2024-09-16T15:47:26.741017+0000","last_unstale":"2024-09-16T15:47:26.748596+0000","last_undegraded":"2024-09-16T15:47:26.748596+0000","last_fullsized":"2024-09-16T15:47:26.748596+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:10.927948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.732886+0000","last_change":"2024-09-16T15:47:25.734026+0000","last_active":"2024-09-16T15:47:26.732886+0000","last_peered":"2024-09-16T15:47:26.732886+0000","last_clean":"2024-09-16T15:47:26.732886+0000","last_became_active":"2024-09-16T15:47:25.732937+0000","last_became_peered":"2024-09-16T15:47:25.732937+0000","last_unstale":"2024-09-16T15:47:26.732886+0000","last_undegraded":"2024-09-16T15:47:26.732886+0000","last_fullsized":"2024-09-16T15:47:26.732886+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:41.808076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742318+0000","last_change":"2024-09-16T15:47:26.738799+0000","last_active":"2024-09-16T15:47:26.742318+0000","last_peered":"2024-09-16T15:47:26.742318+0000","last_clean":"2024-09-16T15:47:26.742318+0000","last_became_active":"2024-09-16T15:47:26.738193+0000","last_became_peered":"2024-09-16T15:47:26.738193+0000","last_unstale":"2024-09-16T15:47:26.742318+0000","last_undegraded":"2024-09-16T15:47:26.742318+0000","last_fullsized":"2024-09-16T15:47:26.742318+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:44.952725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.722827+0000","last_change":"2024-09-16T15:47:25.736151+0000","last_active":"2024-09-16T15:47:26.722827+0000","last_peered":"2024-09-16T15:47:26.722827+0000","last_clean":"2024-09-16T15:47:26.722827+0000","last_became_active":"2024-09-16T15:47:25.735301+0000","last_became_peered":"2024-09-16T15:47:25.735301+0000","last_unstale":"2024-09-16T15:47:26.722827+0000","last_undegraded":"2024-09-16T15:47:26.722827+0000","last_fullsized":"2024-09-16T15:47:26.722827+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:18.390003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.749720+0000","last_change":"2024-09-16T15:47:26.744761+0000","last_active":"2024-09-16T15:47:26.749720+0000","last_peered":"2024-09-16T15:47:26.749720+0000","last_clean":"2024-09-16T15:47:26.749720+0000","last_became_active":"2024-09-16T15:47:26.743460+0000","last_became_peered":"2024-09-16T15:47:26.743460+0000","last_unstale":"2024-09-16T15:47:26.749720+0000","last_undegraded":"2024-09-16T15:47:26.749720+0000","last_fullsized":"2024-09-16T15:47:26.749720+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:15.409871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.732544+0000","last_change":"2024-09-16T15:47:25.733805+0000","last_active":"2024-09-16T15:47:26.732544+0000","last_peered":"2024-09-16T15:47:26.732544+0000","last_clean":"2024-09-16T15:47:26.732544+0000","last_became_active":"2024-09-16T15:47:25.733550+0000","last_became_peered":"2024-09-16T15:47:25.733550+0000","last_unstale":"2024-09-16T15:47:26.732544+0000","last_undegraded":"2024-09-16T15:47:26.732544+0000","last_fullsized":"2024-09-16T15:47:26.732544+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:17.610356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.736557+0000","last_change":"2024-09-16T15:47:26.730945+0000","last_active":"2024-09-16T15:47:26.736557+0000","last_peered":"2024-09-16T15:47:26.736557+0000","last_clean":"2024-09-16T15:47:26.736557+0000","last_became_active":"2024-09-16T15:47:26.730661+0000","last_became_peered":"2024-09-16T15:47:26.730661+0000","last_unstale":"2024-09-16T15:47:26.736557+0000","last_undegraded":"2024-09-16T15:47:26.736557+0000","last_fullsized":"2024-09-16T15:47:26.736557+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:28.879178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"160'3","reported_seq":16,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757444+0000","last_change":"2024-09-16T15:47:25.734701+0000","last_active":"2024-09-16T15:47:26.757444+0000","last_peered":"2024-09-16T15:47:26.757444+0000","last_clean":"2024-09-16T15:47:26.757444+0000","last_became_active":"2024-09-16T15:47:25.734408+0000","last_became_peered":"2024-09-16T15:47:25.734408+0000","last_unstale":"2024-09-16T15:47:26.757444+0000","last_undegraded":"2024-09-16T15:47:26.757444+0000","last_fullsized":"2024-09-16T15:47:26.757444+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:47:37.541285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.751381+0000","last_change":"2024-09-16T15:47:26.743924+0000","last_active":"2024-09-16T15:47:26.751381+0000","last_peered":"2024-09-16T15:47:26.751381+0000","last_clean":"2024-09-16T15:47:26.751381+0000","last_became_active":"2024-09-16T15:47:26.741579+0000","last_became_peered":"2024-09-16T15:47:26.741579+0000","last_unstale":"2024-09-16T15:47:26.751381+0000","last_undegraded":"2024-09-16T15:47:26.751381+0000","last_fullsized":"2024-09-16T15:47:26.751381+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:47:46.994742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"160'6","reported_seq":19,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.763267+0000","last_change":"2024-09-16T15:47:25.731148+0000","last_active":"2024-09-16T15:47:26.763267+0000","last_peered":"2024-09-16T15:47:26.763267+0000","last_clean":"2024-09-16T15:47:26.763267+0000","last_became_active":"2024-09-16T15:47:25.730832+0000","last_became_peered":"2024-09-16T15:47:25.730832+0000","last_unstale":"2024-09-16T15:47:26.763267+0000","last_undegraded":"2024-09-16T15:47:26.763267+0000","last_fullsized":"2024-09-16T15:47:26.763267+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:52.104710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.738200+0000","last_change":"2024-09-16T15:47:26.736931+0000","last_active":"2024-09-16T15:47:26.738200+0000","last_peered":"2024-09-16T15:47:26.738200+0000","last_clean":"2024-09-16T15:47:26.738200+0000","last_became_active":"2024-09-16T15:47:26.736638+0000","last_became_peered":"2024-09-16T15:47:26.736638+0000","last_unstale":"2024-09-16T15:47:26.738200+0000","last_undegraded":"2024-09-16T15:47:26.738200+0000","last_fullsized":"2024-09-16T15:47:26.738200+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:02.909340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758636+0000","last_change":"2024-09-16T15:47:25.731456+0000","last_active":"2024-09-16T15:47:26.758636+0000","last_peered":"2024-09-16T15:47:26.758636+0000","last_clean":"2024-09-16T15:47:26.758636+0000","last_became_active":"2024-09-16T15:47:25.731162+0000","last_became_peered":"2024-09-16T15:47:25.731162+0000","last_unstale":"2024-09-16T15:47:26.758636+0000","last_undegraded":"2024-09-16T15:47:26.758636+0000","last_fullsized":"2024-09-16T15:47:26.758636+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:17.257643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744342+0000","last_change":"2024-09-16T15:47:26.736789+0000","last_active":"2024-09-16T15:47:26.744342+0000","last_peered":"2024-09-16T15:47:26.744342+0000","last_clean":"2024-09-16T15:47:26.744342+0000","last_became_active":"2024-09-16T15:47:26.736551+0000","last_became_peered":"2024-09-16T15:47:26.736551+0000","last_unstale":"2024-09-16T15:47:26.744342+0000","last_undegraded":"2024-09-16T15:47:26.744342+0000","last_fullsized":"2024-09-16T15:47:26.744342+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:20.838172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.719517+0000","last_change":"2024-09-16T15:47:25.743802+0000","last_active":"2024-09-16T15:47:26.719517+0000","last_peered":"2024-09-16T15:47:26.719517+0000","last_clean":"2024-09-16T15:47:26.719517+0000","last_became_active":"2024-09-16T15:47:25.743331+0000","last_became_peered":"2024-09-16T15:47:25.743331+0000","last_unstale":"2024-09-16T15:47:26.719517+0000","last_undegraded":"2024-09-16T15:47:26.719517+0000","last_fullsized":"2024-09-16T15:47:26.719517+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:52.190872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.745956+0000","last_change":"2024-09-16T15:47:26.738649+0000","last_active":"2024-09-16T15:47:26.745956+0000","last_peered":"2024-09-16T15:47:26.745956+0000","last_clean":"2024-09-16T15:47:26.745956+0000","last_became_active":"2024-09-16T15:47:26.738336+0000","last_became_peered":"2024-09-16T15:47:26.738336+0000","last_unstale":"2024-09-16T15:47:26.745956+0000","last_undegraded":"2024-09-16T15:47:26.745956+0000","last_fullsized":"2024-09-16T15:47:26.745956+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:01.208576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.728925+0000","last_change":"2024-09-16T15:47:25.731018+0000","last_active":"2024-09-16T15:47:26.728925+0000","last_peered":"2024-09-16T15:47:26.728925+0000","last_clean":"2024-09-16T15:47:26.728925+0000","last_became_active":"2024-09-16T15:47:25.730771+0000","last_became_peered":"2024-09-16T15:47:25.730771+0000","last_unstale":"2024-09-16T15:47:26.728925+0000","last_undegraded":"2024-09-16T15:47:26.728925+0000","last_fullsized":"2024-09-16T15:47:26.728925+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:28.556954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.736791+0000","last_change":"2024-09-16T15:47:26.728974+0000","last_active":"2024-09-16T15:47:26.736791+0000","last_peered":"2024-09-16T15:47:26.736791+0000","last_clean":"2024-09-16T15:47:26.736791+0000","last_became_active":"2024-09-16T15:47:26.727711+0000","last_became_peered":"2024-09-16T15:47:26.727711+0000","last_unstale":"2024-09-16T15:47:26.736791+0000","last_undegraded":"2024-09-16T15:47:26.736791+0000","last_fullsized":"2024-09-16T15:47:26.736791+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:57.181888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"160'8","reported_seq":21,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758527+0000","last_change":"2024-09-16T15:47:25.733393+0000","last_active":"2024-09-16T15:47:26.758527+0000","last_peered":"2024-09-16T15:47:26.758527+0000","last_clean":"2024-09-16T15:47:26.758527+0000","last_became_active":"2024-09-16T15:47:25.732964+0000","last_became_peered":"2024-09-16T15:47:25.732964+0000","last_unstale":"2024-09-16T15:47:26.758527+0000","last_undegraded":"2024-09-16T15:47:26.758527+0000","last_fullsized":"2024-09-16T15:47:26.758527+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:10.318582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742182+0000","last_change":"2024-09-16T15:47:26.733206+0000","last_active":"2024-09-16T15:47:26.742182+0000","last_peered":"2024-09-16T15:47:26.742182+0000","last_clean":"2024-09-16T15:47:26.742182+0000","last_became_active":"2024-09-16T15:47:26.732932+0000","last_became_peered":"2024-09-16T15:47:26.732932+0000","last_unstale":"2024-09-16T15:47:26.742182+0000","last_undegraded":"2024-09-16T15:47:26.742182+0000","last_fullsized":"2024-09-16T15:47:26.742182+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:13:28.912805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.759281+0000","last_change":"2024-09-16T15:47:25.741726+0000","last_active":"2024-09-16T15:47:26.759281+0000","last_peered":"2024-09-16T15:47:26.759281+0000","last_clean":"2024-09-16T15:47:26.759281+0000","last_became_active":"2024-09-16T15:47:25.741038+0000","last_became_peered":"2024-09-16T15:47:25.741038+0000","last_unstale":"2024-09-16T15:47:26.759281+0000","last_undegraded":"2024-09-16T15:47:26.759281+0000","last_fullsized":"2024-09-16T15:47:26.759281+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:05.034213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.753485+0000","last_change":"2024-09-16T15:47:26.750397+0000","last_active":"2024-09-16T15:47:26.753485+0000","last_peered":"2024-09-16T15:47:26.753485+0000","last_clean":"2024-09-16T15:47:26.753485+0000","last_became_active":"2024-09-16T15:47:26.748909+0000","last_became_peered":"2024-09-16T15:47:26.748909+0000","last_unstale":"2024-09-16T15:47:26.753485+0000","last_undegraded":"2024-09-16T15:47:26.753485+0000","last_fullsized":"2024-09-16T15:47:26.753485+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:17.332634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.755742+0000","last_change":"2024-09-16T15:47:25.731956+0000","last_active":"2024-09-16T15:47:26.755742+0000","last_peered":"2024-09-16T15:47:26.755742+0000","last_clean":"2024-09-16T15:47:26.755742+0000","last_became_active":"2024-09-16T15:47:25.731669+0000","last_became_peered":"2024-09-16T15:47:25.731669+0000","last_unstale":"2024-09-16T15:47:26.755742+0000","last_undegraded":"2024-09-16T15:47:26.755742+0000","last_fullsized":"2024-09-16T15:47:26.755742+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:08.890275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.750531+0000","last_change":"2024-09-16T15:47:26.741549+0000","last_active":"2024-09-16T15:47:26.750531+0000","last_peered":"2024-09-16T15:47:26.750531+0000","last_clean":"2024-09-16T15:47:26.750531+0000","last_became_active":"2024-09-16T15:47:26.741264+0000","last_became_peered":"2024-09-16T15:47:26.741264+0000","last_unstale":"2024-09-16T15:47:26.750531+0000","last_undegraded":"2024-09-16T15:47:26.750531+0000","last_fullsized":"2024-09-16T15:47:26.750531+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:56:26.270706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.756604+0000","last_change":"2024-09-16T15:47:25.737911+0000","last_active":"2024-09-16T15:47:26.756604+0000","last_peered":"2024-09-16T15:47:26.756604+0000","last_clean":"2024-09-16T15:47:26.756604+0000","last_became_active":"2024-09-16T15:47:25.737601+0000","last_became_peered":"2024-09-16T15:47:25.737601+0000","last_unstale":"2024-09-16T15:47:26.756604+0000","last_undegraded":"2024-09-16T15:47:26.756604+0000","last_fullsized":"2024-09-16T15:47:26.756604+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:14.402903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752187+0000","last_change":"2024-09-16T15:47:26.744552+0000","last_active":"2024-09-16T15:47:26.752187+0000","last_peered":"2024-09-16T15:47:26.752187+0000","last_clean":"2024-09-16T15:47:26.752187+0000","last_became_active":"2024-09-16T15:47:26.744306+0000","last_became_peered":"2024-09-16T15:47:26.744306+0000","last_unstale":"2024-09-16T15:47:26.752187+0000","last_undegraded":"2024-09-16T15:47:26.752187+0000","last_fullsized":"2024-09-16T15:47:26.752187+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:28.753774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.723003+0000","last_change":"2024-09-16T15:47:25.736185+0000","last_active":"2024-09-16T15:47:26.723003+0000","last_peered":"2024-09-16T15:47:26.723003+0000","last_clean":"2024-09-16T15:47:26.723003+0000","last_became_active":"2024-09-16T15:47:25.735711+0000","last_became_peered":"2024-09-16T15:47:25.735711+0000","last_unstale":"2024-09-16T15:47:26.723003+0000","last_undegraded":"2024-09-16T15:47:26.723003+0000","last_fullsized":"2024-09-16T15:47:26.723003+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:25.146154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757290+0000","last_change":"2024-09-16T15:47:26.750853+0000","last_active":"2024-09-16T15:47:26.757290+0000","last_peered":"2024-09-16T15:47:26.757290+0000","last_clean":"2024-09-16T15:47:26.757290+0000","last_became_active":"2024-09-16T15:47:26.750460+0000","last_became_peered":"2024-09-16T15:47:26.750460+0000","last_unstale":"2024-09-16T15:47:26.757290+0000","last_undegraded":"2024-09-16T15:47:26.757290+0000","last_fullsized":"2024-09-16T15:47:26.757290+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:31.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757353+0000","last_change":"2024-09-16T15:47:25.740990+0000","last_active":"2024-09-16T15:47:26.757353+0000","last_peered":"2024-09-16T15:47:26.757353+0000","last_clean":"2024-09-16T15:47:26.757353+0000","last_became_active":"2024-09-16T15:47:25.740647+0000","last_became_peered":"2024-09-16T15:47:25.740647+0000","last_unstale":"2024-09-16T15:47:26.757353+0000","last_undegraded":"2024-09-16T15:47:26.757353+0000","last_fullsized":"2024-09-16T15:47:26.757353+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:43.737825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744312+0000","last_change":"2024-09-16T15:47:26.736168+0000","last_active":"2024-09-16T15:47:26.744312+0000","last_peered":"2024-09-16T15:47:26.744312+0000","last_clean":"2024-09-16T15:47:26.744312+0000","last_became_active":"2024-09-16T15:47:26.735472+0000","last_became_peered":"2024-09-16T15:47:26.735472+0000","last_unstale":"2024-09-16T15:47:26.744312+0000","last_undegraded":"2024-09-16T15:47:26.744312+0000","last_fullsized":"2024-09-16T15:47:26.744312+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:34:30.205847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.731446+0000","last_change":"2024-09-16T15:47:25.727858+0000","last_active":"2024-09-16T15:47:26.731446+0000","last_peered":"2024-09-16T15:47:26.731446+0000","last_clean":"2024-09-16T15:47:26.731446+0000","last_became_active":"2024-09-16T15:47:25.727601+0000","last_became_peered":"2024-09-16T15:47:25.727601+0000","last_unstale":"2024-09-16T15:47:26.731446+0000","last_undegraded":"2024-09-16T15:47:26.731446+0000","last_fullsized":"2024-09-16T15:47:26.731446+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:09.996955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.748751+0000","last_change":"2024-09-16T15:47:26.741675+0000","last_active":"2024-09-16T15:47:26.748751+0000","last_peered":"2024-09-16T15:47:26.748751+0000","last_clean":"2024-09-16T15:47:26.748751+0000","last_became_active":"2024-09-16T15:47:26.741408+0000","last_became_peered":"2024-09-16T15:47:26.741408+0000","last_unstale":"2024-09-16T15:47:26.748751+0000","last_undegraded":"2024-09-16T15:47:26.748751+0000","last_fullsized":"2024-09-16T15:47:26.748751+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:28.336349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.728599+0000","last_change":"2024-09-16T15:47:25.740622+0000","last_active":"2024-09-16T15:47:26.728599+0000","last_peered":"2024-09-16T15:47:26.728599+0000","last_clean":"2024-09-16T15:47:26.728599+0000","last_became_active":"2024-09-16T15:47:25.739380+0000","last_became_peered":"2024-09-16T15:47:25.739380+0000","last_unstale":"2024-09-16T15:47:26.728599+0000","last_undegraded":"2024-09-16T15:47:26.728599+0000","last_fullsized":"2024-09-16T15:47:26.728599+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:42.247704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.740599+0000","last_change":"2024-09-16T15:47:26.739451+0000","last_active":"2024-09-16T15:47:26.740599+0000","last_peered":"2024-09-16T15:47:26.740599+0000","last_clean":"2024-09-16T15:47:26.740599+0000","last_became_active":"2024-09-16T15:47:26.738692+0000","last_became_peered":"2024-09-16T15:47:26.738692+0000","last_unstale":"2024-09-16T15:47:26.740599+0000","last_undegraded":"2024-09-16T15:47:26.740599+0000","last_fullsized":"2024-09-16T15:47:26.740599+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:49.696373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.725503+0000","last_change":"2024-09-16T15:47:25.734530+0000","last_active":"2024-09-16T15:47:26.725503+0000","last_peered":"2024-09-16T15:47:26.725503+0000","last_clean":"2024-09-16T15:47:26.725503+0000","last_became_active":"2024-09-16T15:47:25.734113+0000","last_became_peered":"2024-09-16T15:47:25.734113+0000","last_unstale":"2024-09-16T15:47:26.725503+0000","last_undegraded":"2024-09-16T15:47:26.725503+0000","last_fullsized":"2024-09-16T15:47:26.725503+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:17.659599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.740510+0000","last_change":"2024-09-16T15:47:26.738883+0000","last_active":"2024-09-16T15:47:26.740510+0000","last_peered":"2024-09-16T15:47:26.740510+0000","last_clean":"2024-09-16T15:47:26.740510+0000","last_became_active":"2024-09-16T15:47:26.738596+0000","last_became_peered":"2024-09-16T15:47:26.738596+0000","last_unstale":"2024-09-16T15:47:26.740510+0000","last_undegraded":"2024-09-16T15:47:26.740510+0000","last_fullsized":"2024-09-16T15:47:26.740510+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:05.179535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758079+0000","last_change":"2024-09-16T15:47:25.741904+0000","last_active":"2024-09-16T15:47:26.758079+0000","last_peered":"2024-09-16T15:47:26.758079+0000","last_clean":"2024-09-16T15:47:26.758079+0000","last_became_active":"2024-09-16T15:47:25.741666+0000","last_became_peered":"2024-09-16T15:47:25.741666+0000","last_unstale":"2024-09-16T15:47:26.758079+0000","last_undegraded":"2024-09-16T15:47:26.758079+0000","last_fullsized":"2024-09-16T15:47:26.758079+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:18.924111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752565+0000","last_change":"2024-09-16T15:47:26.745919+0000","last_active":"2024-09-16T15:47:26.752565+0000","last_peered":"2024-09-16T15:47:26.752565+0000","last_clean":"2024-09-16T15:47:26.752565+0000","last_became_active":"2024-09-16T15:47:26.744056+0000","last_became_peered":"2024-09-16T15:47:26.744056+0000","last_unstale":"2024-09-16T15:47:26.752565+0000","last_undegraded":"2024-09-16T15:47:26.752565+0000","last_fullsized":"2024-09-16T15:47:26.752565+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:16.753771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.720429+0000","last_change":"2024-09-16T15:47:25.735712+0000","last_active":"2024-09-16T15:47:26.720429+0000","last_peered":"2024-09-16T15:47:26.720429+0000","last_clean":"2024-09-16T15:47:26.720429+0000","last_became_active":"2024-09-16T15:47:25.733278+0000","last_became_peered":"2024-09-16T15:47:25.733278+0000","last_unstale":"2024-09-16T15:47:26.720429+0000","last_undegraded":"2024-09-16T15:47:26.720429+0000","last_fullsized":"2024-09-16T15:47:26.720429+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.388104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746640+0000","last_change":"2024-09-16T15:47:26.744748+0000","last_active":"2024-09-16T15:47:26.746640+0000","last_peered":"2024-09-16T15:47:26.746640+0000","last_clean":"2024-09-16T15:47:26.746640+0000","last_became_active":"2024-09-16T15:47:26.744452+0000","last_became_peered":"2024-09-16T15:47:26.744452+0000","last_unstale":"2024-09-16T15:47:26.746640+0000","last_undegraded":"2024-09-16T15:47:26.746640+0000","last_fullsized":"2024-09-16T15:47:26.746640+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:11.325550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.719856+0000","last_change":"2024-09-16T15:47:25.733325+0000","last_active":"2024-09-16T15:47:26.719856+0000","last_peered":"2024-09-16T15:47:26.719856+0000","last_clean":"2024-09-16T15:47:26.719856+0000","last_became_active":"2024-09-16T15:47:25.733034+0000","last_became_peered":"2024-09-16T15:47:25.733034+0000","last_unstale":"2024-09-16T15:47:26.719856+0000","last_undegraded":"2024-09-16T15:47:26.719856+0000","last_fullsized":"2024-09-16T15:47:26.719856+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:34.867012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742567+0000","last_change":"2024-09-16T15:47:26.737216+0000","last_active":"2024-09-16T15:47:26.742567+0000","last_peered":"2024-09-16T15:47:26.742567+0000","last_clean":"2024-09-16T15:47:26.742567+0000","last_became_active":"2024-09-16T15:47:26.735395+0000","last_became_peered":"2024-09-16T15:47:26.735395+0000","last_unstale":"2024-09-16T15:47:26.742567+0000","last_undegraded":"2024-09-16T15:47:26.742567+0000","last_fullsized":"2024-09-16T15:47:26.742567+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:36.077243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.728074+0000","last_change":"2024-09-16T15:47:25.732565+0000","last_active":"2024-09-16T15:47:26.728074+0000","last_peered":"2024-09-16T15:47:26.728074+0000","last_clean":"2024-09-16T15:47:26.728074+0000","last_became_active":"2024-09-16T15:47:25.731334+0000","last_became_peered":"2024-09-16T15:47:25.731334+0000","last_unstale":"2024-09-16T15:47:26.728074+0000","last_undegraded":"2024-09-16T15:47:26.728074+0000","last_fullsized":"2024-09-16T15:47:26.728074+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:28.672670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752792+0000","last_change":"2024-09-16T15:47:26.748025+0000","last_active":"2024-09-16T15:47:26.752792+0000","last_peered":"2024-09-16T15:47:26.752792+0000","last_clean":"2024-09-16T15:47:26.752792+0000","last_became_active":"2024-09-16T15:47:26.747658+0000","last_became_peered":"2024-09-16T15:47:26.747658+0000","last_unstale":"2024-09-16T15:47:26.752792+0000","last_undegraded":"2024-09-16T15:47:26.752792+0000","last_fullsized":"2024-09-16T15:47:26.752792+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:56.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.725678+0000","last_change":"2024-09-16T15:47:25.737685+0000","last_active":"2024-09-16T15:47:26.725678+0000","last_peered":"2024-09-16T15:47:26.725678+0000","last_clean":"2024-09-16T15:47:26.725678+0000","last_became_active":"2024-09-16T15:47:25.737280+0000","last_became_peered":"2024-09-16T15:47:25.737280+0000","last_unstale":"2024-09-16T15:47:26.725678+0000","last_undegraded":"2024-09-16T15:47:26.725678+0000","last_fullsized":"2024-09-16T15:47:26.725678+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:19:34.957214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.753044+0000","last_change":"2024-09-16T15:47:26.749089+0000","last_active":"2024-09-16T15:47:26.753044+0000","last_peered":"2024-09-16T15:47:26.753044+0000","last_clean":"2024-09-16T15:47:26.753044+0000","last_became_active":"2024-09-16T15:47:26.748866+0000","last_became_peered":"2024-09-16T15:47:26.748866+0000","last_unstale":"2024-09-16T15:47:26.753044+0000","last_undegraded":"2024-09-16T15:47:26.753044+0000","last_fullsized":"2024-09-16T15:47:26.753044+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:15.005176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.726169+0000","last_change":"2024-09-16T15:47:25.731489+0000","last_active":"2024-09-16T15:47:26.726169+0000","last_peered":"2024-09-16T15:47:26.726169+0000","last_clean":"2024-09-16T15:47:26.726169+0000","last_became_active":"2024-09-16T15:47:25.731230+0000","last_became_peered":"2024-09-16T15:47:25.731230+0000","last_unstale":"2024-09-16T15:47:26.726169+0000","last_undegraded":"2024-09-16T15:47:26.726169+0000","last_fullsized":"2024-09-16T15:47:26.726169+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:59.606142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752401+0000","last_change":"2024-09-16T15:47:26.745679+0000","last_active":"2024-09-16T15:47:26.752401+0000","last_peered":"2024-09-16T15:47:26.752401+0000","last_clean":"2024-09-16T15:47:26.752401+0000","last_became_active":"2024-09-16T15:47:26.744684+0000","last_became_peered":"2024-09-16T15:47:26.744684+0000","last_unstale":"2024-09-16T15:47:26.752401+0000","last_undegraded":"2024-09-16T15:47:26.752401+0000","last_fullsized":"2024-09-16T15:47:26.752401+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:36.691777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.721754+0000","last_change":"2024-09-16T15:47:25.733626+0000","last_active":"2024-09-16T15:47:26.721754+0000","last_peered":"2024-09-16T15:47:26.721754+0000","last_clean":"2024-09-16T15:47:26.721754+0000","last_became_active":"2024-09-16T15:47:25.732281+0000","last_became_peered":"2024-09-16T15:47:25.732281+0000","last_unstale":"2024-09-16T15:47:26.721754+0000","last_undegraded":"2024-09-16T15:47:26.721754+0000","last_fullsized":"2024-09-16T15:47:26.721754+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:42.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.756883+0000","last_change":"2024-09-16T15:47:26.753758+0000","last_active":"2024-09-16T15:47:26.756883+0000","last_peered":"2024-09-16T15:47:26.756883+0000","last_clean":"2024-09-16T15:47:26.756883+0000","last_became_active":"2024-09-16T15:47:26.749557+0000","last_became_peered":"2024-09-16T15:47:26.749557+0000","last_unstale":"2024-09-16T15:47:26.756883+0000","last_undegraded":"2024-09-16T15:47:26.756883+0000","last_fullsized":"2024-09-16T15:47:26.756883+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:33.369693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.728152+0000","last_change":"2024-09-16T15:47:25.735373+0000","last_active":"2024-09-16T15:47:26.728152+0000","last_peered":"2024-09-16T15:47:26.728152+0000","last_clean":"2024-09-16T15:47:26.728152+0000","last_became_active":"2024-09-16T15:47:25.735084+0000","last_became_peered":"2024-09-16T15:47:25.735084+0000","last_unstale":"2024-09-16T15:47:26.728152+0000","last_undegraded":"2024-09-16T15:47:26.728152+0000","last_fullsized":"2024-09-16T15:47:26.728152+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:41.496815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.736443+0000","last_change":"2024-09-16T15:47:26.728573+0000","last_active":"2024-09-16T15:47:26.736443+0000","last_peered":"2024-09-16T15:47:26.736443+0000","last_clean":"2024-09-16T15:47:26.736443+0000","last_became_active":"2024-09-16T15:47:26.728305+0000","last_became_peered":"2024-09-16T15:47:26.728305+0000","last_unstale":"2024-09-16T15:47:26.736443+0000","last_undegraded":"2024-09-16T15:47:26.736443+0000","last_fullsized":"2024-09-16T15:47:26.736443+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:06.725094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.731632+0000","last_change":"2024-09-16T15:47:25.731262+0000","last_active":"2024-09-16T15:47:26.731632+0000","last_peered":"2024-09-16T15:47:26.731632+0000","last_clean":"2024-09-16T15:47:26.731632+0000","last_became_active":"2024-09-16T15:47:25.730949+0000","last_became_peered":"2024-09-16T15:47:25.730949+0000","last_unstale":"2024-09-16T15:47:26.731632+0000","last_undegraded":"2024-09-16T15:47:26.731632+0000","last_fullsized":"2024-09-16T15:47:26.731632+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:45.129580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":392,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.733037+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:47:26.733037+0000","last_peered":"2024-09-16T15:47:26.733037+0000","last_clean":"2024-09-16T15:47:26.733037+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:47:26.733037+0000","last_undegraded":"2024-09-16T15:47:26.733037+0000","last_fullsized":"2024-09-16T15:47:26.733037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33744,"kb_used_data":2548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338096,"statfs":{"total":96636764160,"available":96602210304,"internally_reserved":0,"allocated":2609152,"data_stored":1476597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509553,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":886125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1994752,"data_stored":879322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2023424,"data_stored":888319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37324,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334516,"statfs":{"total":96636764160,"available":96598544384,"internally_reserved":0,"allocated":2019328,"data_stored":886319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37324,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334516,"statfs":{"total":96636764160,"available":96598544384,"internally_reserved":0,"allocated":2019328,"data_stored":886819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":1994752,"data_stored":879322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509553,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37876,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333964,"statfs":{"total":96636764160,"available":96597979136,"internally_reserved":0,"allocated":2588672,"data_stored":1469690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:47:30.977 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:47:31.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.175+0000 7f5fe2600640 1 -- 172.21.15.60:0/3522100490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 msgr2=0x7f5fc000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.175+0000 7f5fe2600640 1 --2- 172.21.15.60:0/3522100490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 0x7f5fc000b9c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4001d90 tx=0x7f5fc0005830 comp rx=0 tx=0).stop 2024-09-16T15:47:31.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.175+0000 7f5fe2600640 1 -- 172.21.15.60:0/3522100490 shutdown_connections 2024-09-16T15:47:31.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.175+0000 7f5fe2600640 1 --2- 172.21.15.60:0/3522100490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 0x7f5fc000b9c0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.175+0000 7f5fe2600640 1 -- 172.21.15.60:0/3522100490 >> 172.21.15.60:0/3522100490 conn(0x7f5fdc0b1e90 msgr2=0x7f5fdc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:31.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.175+0000 7f5fe2600640 1 -- 172.21.15.60:0/3522100490 shutdown_connections 2024-09-16T15:47:31.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.175+0000 7f5fe2600640 1 -- 172.21.15.60:0/3522100490 wait complete. 2024-09-16T15:47:31.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.177+0000 7f5fe2600640 1 Processor -- start 2024-09-16T15:47:31.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.177+0000 7f5fe2600640 1 -- start start 2024-09-16T15:47:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.178+0000 7f5fe2600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 0x7f5fdc158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.178+0000 7f5fe2600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fdc1f4b00 0x7f5fdc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.178+0000 7f5fe2600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fdc0b5c90 0x7f5fdc0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.178+0000 7f5fe2600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc0ce9e0 con 0x7f5fc00052e0 2024-09-16T15:47:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.178+0000 7f5fe2600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc0ce860 con 0x7f5fdc0b5c90 2024-09-16T15:47:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.178+0000 7f5fe2600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5fdc0cab90 con 0x7f5fdc1f4b00 2024-09-16T15:47:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.178+0000 7f5fe1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 0x7f5fdc158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 0x7f5fdc158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44426/0 (socket says 172.21.15.60:44426) 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1200640 1 -- 172.21.15.60:0/2755046371 learned_addr learned my addr 172.21.15.60:0/2755046371 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fd3e00640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fdc1f4b00 0x7f5fdc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1c00640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fdc0b5c90 0x7f5fdc0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1200640 1 -- 172.21.15.60:0/2755046371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fdc1f4b00 msgr2=0x7f5fdc0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1200640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fdc1f4b00 0x7f5fdc0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1200640 1 -- 172.21.15.60:0/2755046371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fdc0b5c90 msgr2=0x7f5fdc0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1200640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fdc0b5c90 0x7f5fdc0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1200640 1 -- 172.21.15.60:0/2755046371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc1f4f90 con 0x7f5fc00052e0 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fe1200640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 0x7f5fdc158d80 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00e610 tx=0x7f5fcc00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.179+0000 7f5fd9600640 1 -- 172.21.15.60:0/2755046371 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc015020 con 0x7f5fc00052e0 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.180+0000 7f5fd9600640 1 -- 172.21.15.60:0/2755046371 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fcc00ca60 con 0x7f5fc00052e0 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.180+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc1f5270 con 0x7f5fc00052e0 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.180+0000 7f5fd9600640 1 -- 172.21.15.60:0/2755046371 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc011460 con 0x7f5fc00052e0 2024-09-16T15:47:31.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.180+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1f5830 con 0x7f5fc00052e0 2024-09-16T15:47:31.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.181+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc0cb060 con 0x7f5fc00052e0 2024-09-16T15:47:31.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.182+0000 7f5fd9600640 1 -- 172.21.15.60:0/2755046371 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f5fcc018020 con 0x7f5fc00052e0 2024-09-16T15:47:31.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.183+0000 7f5fd9600640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fb80790c0 0x7f5fb807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.183+0000 7f5fd9600640 1 -- 172.21.15.60:0/2755046371 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f5fcc099660 con 0x7f5fc00052e0 2024-09-16T15:47:31.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.183+0000 7f5fd3e00640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fb80790c0 0x7f5fb807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.184+0000 7f5fd3e00640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fb80790c0 0x7f5fb807b580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0b4ee0 tx=0x7f5fc400b260 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:31.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.187+0000 7f5fd9600640 1 -- 172.21.15.60:0/2755046371 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5fcc01f020 con 0x7f5fc00052e0 2024-09-16T15:47:31.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.379+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5fdc0749a0 con 0x7f5fb80790c0 2024-09-16T15:47:31.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.383+0000 7f5fd9600640 1 -- 172.21.15.60:0/2755046371 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252470 (secure 0 0 0) 0x7f5fdc0749a0 con 0x7f5fb80790c0 2024-09-16T15:47:31.387 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:31.388 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:47:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fb80790c0 msgr2=0x7f5fb807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fb80790c0 0x7f5fb807b580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0b4ee0 tx=0x7f5fc400b260 comp rx=0 tx=0).stop 2024-09-16T15:47:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 msgr2=0x7f5fdc158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 0x7f5fdc158d80 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00e610 tx=0x7f5fcc00ead0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 shutdown_connections 2024-09-16T15:47:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5fb80790c0 0x7f5fb807b580 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fdc0b5c90 0x7f5fdc0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fdc1f4b00 0x7f5fdc0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 --2- 172.21.15.60:0/2755046371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fc00052e0 0x7f5fdc158d80 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 >> 172.21.15.60:0/2755046371 conn(0x7f5fdc0b1e90 msgr2=0x7f5fdc0b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 shutdown_connections 2024-09-16T15:47:31.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.387+0000 7f5fe2600640 1 -- 172.21.15.60:0/2755046371 wait complete. 2024-09-16T15:47:31.404 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-16T15:47:30.217395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283020,"kb_used_data":16852,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754691700,"statfs":{"total":773094113280,"available":772804300800,"internally_reserved":0,"allocated":17256448,"data_stored":8252513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.510230"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.754891+0000","last_change":"2024-09-16T15:47:26.753571+0000","last_active":"2024-09-16T15:47:26.754891+0000","last_peered":"2024-09-16T15:47:26.754891+0000","last_clean":"2024-09-16T15:47:26.754891+0000","last_became_active":"2024-09-16T15:47:26.753260+0000","last_became_peered":"2024-09-16T15:47:26.753260+0000","last_unstale":"2024-09-16T15:47:26.754891+0000","last_undegraded":"2024-09-16T15:47:26.754891+0000","last_fullsized":"2024-09-16T15:47:26.754891+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:57.730928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752076+0000","last_change":"2024-09-16T15:47:26.744170+0000","last_active":"2024-09-16T15:47:26.752076+0000","last_peered":"2024-09-16T15:47:26.752076+0000","last_clean":"2024-09-16T15:47:26.752076+0000","last_became_active":"2024-09-16T15:47:26.743832+0000","last_became_peered":"2024-09-16T15:47:26.743832+0000","last_unstale":"2024-09-16T15:47:26.752076+0000","last_undegraded":"2024-09-16T15:47:26.752076+0000","last_fullsized":"2024-09-16T15:47:26.752076+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:13:20.684223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742240+0000","last_change":"2024-09-16T15:47:26.738179+0000","last_active":"2024-09-16T15:47:26.742240+0000","last_peered":"2024-09-16T15:47:26.742240+0000","last_clean":"2024-09-16T15:47:26.742240+0000","last_became_active":"2024-09-16T15:47:26.737893+0000","last_became_peered":"2024-09-16T15:47:26.737893+0000","last_unstale":"2024-09-16T15:47:26.742240+0000","last_undegraded":"2024-09-16T15:47:26.742240+0000","last_fullsized":"2024-09-16T15:47:26.742240+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:55.621116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752123+0000","last_change":"2024-09-16T15:47:26.743437+0000","last_active":"2024-09-16T15:47:26.752123+0000","last_peered":"2024-09-16T15:47:26.752123+0000","last_clean":"2024-09-16T15:47:26.752123+0000","last_became_active":"2024-09-16T15:47:26.743096+0000","last_became_peered":"2024-09-16T15:47:26.743096+0000","last_unstale":"2024-09-16T15:47:26.752123+0000","last_undegraded":"2024-09-16T15:47:26.752123+0000","last_fullsized":"2024-09-16T15:47:26.752123+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:03.093584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742514+0000","last_change":"2024-09-16T15:47:26.740088+0000","last_active":"2024-09-16T15:47:26.742514+0000","last_peered":"2024-09-16T15:47:26.742514+0000","last_clean":"2024-09-16T15:47:26.742514+0000","last_became_active":"2024-09-16T15:47:26.739741+0000","last_became_peered":"2024-09-16T15:47:26.739741+0000","last_unstale":"2024-09-16T15:47:26.742514+0000","last_undegraded":"2024-09-16T15:47:26.742514+0000","last_fullsized":"2024-09-16T15:47:26.742514+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:37.328585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.738170+0000","last_change":"2024-09-16T15:47:26.736173+0000","last_active":"2024-09-16T15:47:26.738170+0000","last_peered":"2024-09-16T15:47:26.738170+0000","last_clean":"2024-09-16T15:47:26.738170+0000","last_became_active":"2024-09-16T15:47:26.735702+0000","last_became_peered":"2024-09-16T15:47:26.735702+0000","last_unstale":"2024-09-16T15:47:26.738170+0000","last_undegraded":"2024-09-16T15:47:26.738170+0000","last_fullsized":"2024-09-16T15:47:26.738170+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:26.299600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744892+0000","last_change":"2024-09-16T15:47:26.738143+0000","last_active":"2024-09-16T15:47:26.744892+0000","last_peered":"2024-09-16T15:47:26.744892+0000","last_clean":"2024-09-16T15:47:26.744892+0000","last_became_active":"2024-09-16T15:47:26.737859+0000","last_became_peered":"2024-09-16T15:47:26.737859+0000","last_unstale":"2024-09-16T15:47:26.744892+0000","last_undegraded":"2024-09-16T15:47:26.744892+0000","last_fullsized":"2024-09-16T15:47:26.744892+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:47:09.265463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746558+0000","last_change":"2024-09-16T15:47:26.740986+0000","last_active":"2024-09-16T15:47:26.746558+0000","last_peered":"2024-09-16T15:47:26.746558+0000","last_clean":"2024-09-16T15:47:26.746558+0000","last_became_active":"2024-09-16T15:47:26.740610+0000","last_became_peered":"2024-09-16T15:47:26.740610+0000","last_unstale":"2024-09-16T15:47:26.746558+0000","last_undegraded":"2024-09-16T15:47:26.746558+0000","last_fullsized":"2024-09-16T15:47:26.746558+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:46.136548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.741063+0000","last_change":"2024-09-16T15:47:26.733836+0000","last_active":"2024-09-16T15:47:26.741063+0000","last_peered":"2024-09-16T15:47:26.741063+0000","last_clean":"2024-09-16T15:47:26.741063+0000","last_became_active":"2024-09-16T15:47:26.733570+0000","last_became_peered":"2024-09-16T15:47:26.733570+0000","last_unstale":"2024-09-16T15:47:26.741063+0000","last_undegraded":"2024-09-16T15:47:26.741063+0000","last_fullsized":"2024-09-16T15:47:26.741063+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:47.030189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.747439+0000","last_change":"2024-09-16T15:47:26.742278+0000","last_active":"2024-09-16T15:47:26.747439+0000","last_peered":"2024-09-16T15:47:26.747439+0000","last_clean":"2024-09-16T15:47:26.747439+0000","last_became_active":"2024-09-16T15:47:26.737702+0000","last_became_peered":"2024-09-16T15:47:26.737702+0000","last_unstale":"2024-09-16T15:47:26.747439+0000","last_undegraded":"2024-09-16T15:47:26.747439+0000","last_fullsized":"2024-09-16T15:47:26.747439+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:02.578133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.748339+0000","last_change":"2024-09-16T15:47:26.743163+0000","last_active":"2024-09-16T15:47:26.748339+0000","last_peered":"2024-09-16T15:47:26.748339+0000","last_clean":"2024-09-16T15:47:26.748339+0000","last_became_active":"2024-09-16T15:47:26.742891+0000","last_became_peered":"2024-09-16T15:47:26.742891+0000","last_unstale":"2024-09-16T15:47:26.748339+0000","last_undegraded":"2024-09-16T15:47:26.748339+0000","last_fullsized":"2024-09-16T15:47:26.748339+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:34.079341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.743752+0000","last_change":"2024-09-16T15:47:26.738168+0000","last_active":"2024-09-16T15:47:26.743752+0000","last_peered":"2024-09-16T15:47:26.743752+0000","last_clean":"2024-09-16T15:47:26.743752+0000","last_became_active":"2024-09-16T15:47:26.737818+0000","last_became_peered":"2024-09-16T15:47:26.737818+0000","last_unstale":"2024-09-16T15:47:26.743752+0000","last_undegraded":"2024-09-16T15:47:26.743752+0000","last_fullsized":"2024-09-16T15:47:26.743752+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:46.073366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744064+0000","last_change":"2024-09-16T15:47:26.735341+0000","last_active":"2024-09-16T15:47:26.744064+0000","last_peered":"2024-09-16T15:47:26.744064+0000","last_clean":"2024-09-16T15:47:26.744064+0000","last_became_active":"2024-09-16T15:47:26.734979+0000","last_became_peered":"2024-09-16T15:47:26.734979+0000","last_unstale":"2024-09-16T15:47:26.744064+0000","last_undegraded":"2024-09-16T15:47:26.744064+0000","last_fullsized":"2024-09-16T15:47:26.744064+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:23.165921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.745852+0000","last_change":"2024-09-16T15:47:26.738951+0000","last_active":"2024-09-16T15:47:26.745852+0000","last_peered":"2024-09-16T15:47:26.745852+0000","last_clean":"2024-09-16T15:47:26.745852+0000","last_became_active":"2024-09-16T15:47:26.738640+0000","last_became_peered":"2024-09-16T15:47:26.738640+0000","last_unstale":"2024-09-16T15:47:26.745852+0000","last_undegraded":"2024-09-16T15:47:26.745852+0000","last_fullsized":"2024-09-16T15:47:26.745852+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:14.490072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.748182+0000","last_change":"2024-09-16T15:47:26.740431+0000","last_active":"2024-09-16T15:47:26.748182+0000","last_peered":"2024-09-16T15:47:26.748182+0000","last_clean":"2024-09-16T15:47:26.748182+0000","last_became_active":"2024-09-16T15:47:26.739635+0000","last_became_peered":"2024-09-16T15:47:26.739635+0000","last_unstale":"2024-09-16T15:47:26.748182+0000","last_undegraded":"2024-09-16T15:47:26.748182+0000","last_fullsized":"2024-09-16T15:47:26.748182+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:27.641894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746869+0000","last_change":"2024-09-16T15:47:26.741613+0000","last_active":"2024-09-16T15:47:26.746869+0000","last_peered":"2024-09-16T15:47:26.746869+0000","last_clean":"2024-09-16T15:47:26.746869+0000","last_became_active":"2024-09-16T15:47:26.741331+0000","last_became_peered":"2024-09-16T15:47:26.741331+0000","last_unstale":"2024-09-16T15:47:26.746869+0000","last_undegraded":"2024-09-16T15:47:26.746869+0000","last_fullsized":"2024-09-16T15:47:26.746869+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:42.815044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.756500+0000","last_change":"2024-09-16T15:47:26.748721+0000","last_active":"2024-09-16T15:47:26.756500+0000","last_peered":"2024-09-16T15:47:26.756500+0000","last_clean":"2024-09-16T15:47:26.756500+0000","last_became_active":"2024-09-16T15:47:26.748169+0000","last_became_peered":"2024-09-16T15:47:26.748169+0000","last_unstale":"2024-09-16T15:47:26.756500+0000","last_undegraded":"2024-09-16T15:47:26.756500+0000","last_fullsized":"2024-09-16T15:47:26.756500+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:33:32.963714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.754228+0000","last_change":"2024-09-16T15:47:26.750890+0000","last_active":"2024-09-16T15:47:26.754228+0000","last_peered":"2024-09-16T15:47:26.754228+0000","last_clean":"2024-09-16T15:47:26.754228+0000","last_became_active":"2024-09-16T15:47:26.750593+0000","last_became_peered":"2024-09-16T15:47:26.750593+0000","last_unstale":"2024-09-16T15:47:26.754228+0000","last_undegraded":"2024-09-16T15:47:26.754228+0000","last_fullsized":"2024-09-16T15:47:26.754228+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:59:45.858811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744117+0000","last_change":"2024-09-16T15:47:26.733574+0000","last_active":"2024-09-16T15:47:26.744117+0000","last_peered":"2024-09-16T15:47:26.744117+0000","last_clean":"2024-09-16T15:47:26.744117+0000","last_became_active":"2024-09-16T15:47:26.733291+0000","last_became_peered":"2024-09-16T15:47:26.733291+0000","last_unstale":"2024-09-16T15:47:26.744117+0000","last_undegraded":"2024-09-16T15:47:26.744117+0000","last_fullsized":"2024-09-16T15:47:26.744117+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:00:02.550196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.745997+0000","last_change":"2024-09-16T15:47:26.741878+0000","last_active":"2024-09-16T15:47:26.745997+0000","last_peered":"2024-09-16T15:47:26.745997+0000","last_clean":"2024-09-16T15:47:26.745997+0000","last_became_active":"2024-09-16T15:47:26.741509+0000","last_became_peered":"2024-09-16T15:47:26.741509+0000","last_unstale":"2024-09-16T15:47:26.745997+0000","last_undegraded":"2024-09-16T15:47:26.745997+0000","last_fullsized":"2024-09-16T15:47:26.745997+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:49.061178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.745719+0000","last_change":"2024-09-16T15:47:26.743256+0000","last_active":"2024-09-16T15:47:26.745719+0000","last_peered":"2024-09-16T15:47:26.745719+0000","last_clean":"2024-09-16T15:47:26.745719+0000","last_became_active":"2024-09-16T15:47:26.742731+0000","last_became_peered":"2024-09-16T15:47:26.742731+0000","last_unstale":"2024-09-16T15:47:26.745719+0000","last_undegraded":"2024-09-16T15:47:26.745719+0000","last_fullsized":"2024-09-16T15:47:26.745719+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:02.035214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.751080+0000","last_change":"2024-09-16T15:47:26.742963+0000","last_active":"2024-09-16T15:47:26.751080+0000","last_peered":"2024-09-16T15:47:26.751080+0000","last_clean":"2024-09-16T15:47:26.751080+0000","last_became_active":"2024-09-16T15:47:26.739510+0000","last_became_peered":"2024-09-16T15:47:26.739510+0000","last_unstale":"2024-09-16T15:47:26.751080+0000","last_undegraded":"2024-09-16T15:47:26.751080+0000","last_fullsized":"2024-09-16T15:47:26.751080+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:57.594165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752275+0000","last_change":"2024-09-16T15:47:26.749057+0000","last_active":"2024-09-16T15:47:26.752275+0000","last_peered":"2024-09-16T15:47:26.752275+0000","last_clean":"2024-09-16T15:47:26.752275+0000","last_became_active":"2024-09-16T15:47:26.748806+0000","last_became_peered":"2024-09-16T15:47:26.748806+0000","last_unstale":"2024-09-16T15:47:26.752275+0000","last_undegraded":"2024-09-16T15:47:26.752275+0000","last_fullsized":"2024-09-16T15:47:26.752275+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:45:34.284904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746529+0000","last_change":"2024-09-16T15:47:26.740324+0000","last_active":"2024-09-16T15:47:26.746529+0000","last_peered":"2024-09-16T15:47:26.746529+0000","last_clean":"2024-09-16T15:47:26.746529+0000","last_became_active":"2024-09-16T15:47:26.740070+0000","last_became_peered":"2024-09-16T15:47:26.740070+0000","last_unstale":"2024-09-16T15:47:26.746529+0000","last_undegraded":"2024-09-16T15:47:26.746529+0000","last_fullsized":"2024-09-16T15:47:26.746529+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:57.681320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.743411+0000","last_change":"2024-09-16T15:47:26.729352+0000","last_active":"2024-09-16T15:47:26.743411+0000","last_peered":"2024-09-16T15:47:26.743411+0000","last_clean":"2024-09-16T15:47:26.743411+0000","last_became_active":"2024-09-16T15:47:26.729029+0000","last_became_peered":"2024-09-16T15:47:26.729029+0000","last_unstale":"2024-09-16T15:47:26.743411+0000","last_undegraded":"2024-09-16T15:47:26.743411+0000","last_fullsized":"2024-09-16T15:47:26.743411+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:40:39.365473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742595+0000","last_change":"2024-09-16T15:47:26.735264+0000","last_active":"2024-09-16T15:47:26.742595+0000","last_peered":"2024-09-16T15:47:26.742595+0000","last_clean":"2024-09-16T15:47:26.742595+0000","last_became_active":"2024-09-16T15:47:26.734978+0000","last_became_peered":"2024-09-16T15:47:26.734978+0000","last_unstale":"2024-09-16T15:47:26.742595+0000","last_undegraded":"2024-09-16T15:47:26.742595+0000","last_fullsized":"2024-09-16T15:47:26.742595+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:54:04.602404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.754950+0000","last_change":"2024-09-16T15:47:26.750922+0000","last_active":"2024-09-16T15:47:26.754950+0000","last_peered":"2024-09-16T15:47:26.754950+0000","last_clean":"2024-09-16T15:47:26.754950+0000","last_became_active":"2024-09-16T15:47:26.746538+0000","last_became_peered":"2024-09-16T15:47:26.746538+0000","last_unstale":"2024-09-16T15:47:26.754950+0000","last_undegraded":"2024-09-16T15:47:26.754950+0000","last_fullsized":"2024-09-16T15:47:26.754950+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:40:29.106016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.741690+0000","last_change":"2024-09-16T15:47:26.739824+0000","last_active":"2024-09-16T15:47:26.741690+0000","last_peered":"2024-09-16T15:47:26.741690+0000","last_clean":"2024-09-16T15:47:26.741690+0000","last_became_active":"2024-09-16T15:47:26.738236+0000","last_became_peered":"2024-09-16T15:47:26.738236+0000","last_unstale":"2024-09-16T15:47:26.741690+0000","last_undegraded":"2024-09-16T15:47:26.741690+0000","last_fullsized":"2024-09-16T15:47:26.741690+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:58.684833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744090+0000","last_change":"2024-09-16T15:47:26.736657+0000","last_active":"2024-09-16T15:47:26.744090+0000","last_peered":"2024-09-16T15:47:26.744090+0000","last_clean":"2024-09-16T15:47:26.744090+0000","last_became_active":"2024-09-16T15:47:26.736391+0000","last_became_peered":"2024-09-16T15:47:26.736391+0000","last_unstale":"2024-09-16T15:47:26.744090+0000","last_undegraded":"2024-09-16T15:47:26.744090+0000","last_fullsized":"2024-09-16T15:47:26.744090+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:18.710208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.751225+0000","last_change":"2024-09-16T15:47:26.743195+0000","last_active":"2024-09-16T15:47:26.751225+0000","last_peered":"2024-09-16T15:47:26.751225+0000","last_clean":"2024-09-16T15:47:26.751225+0000","last_became_active":"2024-09-16T15:47:26.742925+0000","last_became_peered":"2024-09-16T15:47:26.742925+0000","last_unstale":"2024-09-16T15:47:26.751225+0000","last_undegraded":"2024-09-16T15:47:26.751225+0000","last_fullsized":"2024-09-16T15:47:26.751225+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:59.635093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.737392+0000","last_change":"2024-09-16T15:47:26.735156+0000","last_active":"2024-09-16T15:47:26.737392+0000","last_peered":"2024-09-16T15:47:26.737392+0000","last_clean":"2024-09-16T15:47:26.737392+0000","last_became_active":"2024-09-16T15:47:26.734704+0000","last_became_peered":"2024-09-16T15:47:26.734704+0000","last_unstale":"2024-09-16T15:47:26.737392+0000","last_undegraded":"2024-09-16T15:47:26.737392+0000","last_fullsized":"2024-09-16T15:47:26.737392+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:56.067533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.740215+0000","last_change":"2024-09-16T15:47:26.738808+0000","last_active":"2024-09-16T15:47:26.740215+0000","last_peered":"2024-09-16T15:47:26.740215+0000","last_clean":"2024-09-16T15:47:26.740215+0000","last_became_active":"2024-09-16T15:47:26.737472+0000","last_became_peered":"2024-09-16T15:47:26.737472+0000","last_unstale":"2024-09-16T15:47:26.740215+0000","last_undegraded":"2024-09-16T15:47:26.740215+0000","last_fullsized":"2024-09-16T15:47:26.740215+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:30.535633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744409+0000","last_change":"2024-09-16T15:47:26.738481+0000","last_active":"2024-09-16T15:47:26.744409+0000","last_peered":"2024-09-16T15:47:26.744409+0000","last_clean":"2024-09-16T15:47:26.744409+0000","last_became_active":"2024-09-16T15:47:26.738224+0000","last_became_peered":"2024-09-16T15:47:26.738224+0000","last_unstale":"2024-09-16T15:47:26.744409+0000","last_undegraded":"2024-09-16T15:47:26.744409+0000","last_fullsized":"2024-09-16T15:47:26.744409+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:00:33.411582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"160'4","reported_seq":17,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.759137+0000","last_change":"2024-09-16T15:47:25.735971+0000","last_active":"2024-09-16T15:47:26.759137+0000","last_peered":"2024-09-16T15:47:26.759137+0000","last_clean":"2024-09-16T15:47:26.759137+0000","last_became_active":"2024-09-16T15:47:25.732985+0000","last_became_peered":"2024-09-16T15:47:25.732985+0000","last_unstale":"2024-09-16T15:47:26.759137+0000","last_undegraded":"2024-09-16T15:47:26.759137+0000","last_fullsized":"2024-09-16T15:47:26.759137+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:25.835603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757218+0000","last_change":"2024-09-16T15:47:26.748491+0000","last_active":"2024-09-16T15:47:26.757218+0000","last_peered":"2024-09-16T15:47:26.757218+0000","last_clean":"2024-09-16T15:47:26.757218+0000","last_became_active":"2024-09-16T15:47:26.748114+0000","last_became_peered":"2024-09-16T15:47:26.748114+0000","last_unstale":"2024-09-16T15:47:26.757218+0000","last_undegraded":"2024-09-16T15:47:26.757218+0000","last_fullsized":"2024-09-16T15:47:26.757218+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:04.794001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.753488+0000","last_change":"2024-09-16T15:47:25.732835+0000","last_active":"2024-09-16T15:47:26.753488+0000","last_peered":"2024-09-16T15:47:26.753488+0000","last_clean":"2024-09-16T15:47:26.753488+0000","last_became_active":"2024-09-16T15:47:25.731930+0000","last_became_peered":"2024-09-16T15:47:25.731930+0000","last_unstale":"2024-09-16T15:47:26.753488+0000","last_undegraded":"2024-09-16T15:47:26.753488+0000","last_fullsized":"2024-09-16T15:47:26.753488+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:05.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.749807+0000","last_change":"2024-09-16T15:47:26.747282+0000","last_active":"2024-09-16T15:47:26.749807+0000","last_peered":"2024-09-16T15:47:26.749807+0000","last_clean":"2024-09-16T15:47:26.749807+0000","last_became_active":"2024-09-16T15:47:26.746811+0000","last_became_peered":"2024-09-16T15:47:26.746811+0000","last_unstale":"2024-09-16T15:47:26.749807+0000","last_undegraded":"2024-09-16T15:47:26.749807+0000","last_fullsized":"2024-09-16T15:47:26.749807+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:54.122177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758609+0000","last_change":"2024-09-16T15:47:25.741179+0000","last_active":"2024-09-16T15:47:26.758609+0000","last_peered":"2024-09-16T15:47:26.758609+0000","last_clean":"2024-09-16T15:47:26.758609+0000","last_became_active":"2024-09-16T15:47:25.738798+0000","last_became_peered":"2024-09-16T15:47:25.738798+0000","last_unstale":"2024-09-16T15:47:26.758609+0000","last_undegraded":"2024-09-16T15:47:26.758609+0000","last_fullsized":"2024-09-16T15:47:26.758609+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:42:09.276104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746063+0000","last_change":"2024-09-16T15:47:26.739271+0000","last_active":"2024-09-16T15:47:26.746063+0000","last_peered":"2024-09-16T15:47:26.746063+0000","last_clean":"2024-09-16T15:47:26.746063+0000","last_became_active":"2024-09-16T15:47:26.738752+0000","last_became_peered":"2024-09-16T15:47:26.738752+0000","last_unstale":"2024-09-16T15:47:26.746063+0000","last_undegraded":"2024-09-16T15:47:26.746063+0000","last_fullsized":"2024-09-16T15:47:26.746063+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:21.130447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.754665+0000","last_change":"2024-09-16T15:47:25.730098+0000","last_active":"2024-09-16T15:47:26.754665+0000","last_peered":"2024-09-16T15:47:26.754665+0000","last_clean":"2024-09-16T15:47:26.754665+0000","last_became_active":"2024-09-16T15:47:25.729591+0000","last_became_peered":"2024-09-16T15:47:25.729591+0000","last_unstale":"2024-09-16T15:47:26.754665+0000","last_undegraded":"2024-09-16T15:47:26.754665+0000","last_fullsized":"2024-09-16T15:47:26.754665+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:20.008186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.751509+0000","last_change":"2024-09-16T15:47:26.743607+0000","last_active":"2024-09-16T15:47:26.751509+0000","last_peered":"2024-09-16T15:47:26.751509+0000","last_clean":"2024-09-16T15:47:26.751509+0000","last_became_active":"2024-09-16T15:47:26.740918+0000","last_became_peered":"2024-09-16T15:47:26.740918+0000","last_unstale":"2024-09-16T15:47:26.751509+0000","last_undegraded":"2024-09-16T15:47:26.751509+0000","last_fullsized":"2024-09-16T15:47:26.751509+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:02.953258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758839+0000","last_change":"2024-09-16T15:47:25.739585+0000","last_active":"2024-09-16T15:47:26.758839+0000","last_peered":"2024-09-16T15:47:26.758839+0000","last_clean":"2024-09-16T15:47:26.758839+0000","last_became_active":"2024-09-16T15:47:25.739322+0000","last_became_peered":"2024-09-16T15:47:25.739322+0000","last_unstale":"2024-09-16T15:47:26.758839+0000","last_undegraded":"2024-09-16T15:47:26.758839+0000","last_fullsized":"2024-09-16T15:47:26.758839+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:18:34.714937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.750913+0000","last_change":"2024-09-16T15:47:26.746558+0000","last_active":"2024-09-16T15:47:26.750913+0000","last_peered":"2024-09-16T15:47:26.750913+0000","last_clean":"2024-09-16T15:47:26.750913+0000","last_became_active":"2024-09-16T15:47:26.741504+0000","last_became_peered":"2024-09-16T15:47:26.741504+0000","last_unstale":"2024-09-16T15:47:26.750913+0000","last_undegraded":"2024-09-16T15:47:26.750913+0000","last_fullsized":"2024-09-16T15:47:26.750913+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:19.931079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757792+0000","last_change":"2024-09-16T15:47:25.730194+0000","last_active":"2024-09-16T15:47:26.757792+0000","last_peered":"2024-09-16T15:47:26.757792+0000","last_clean":"2024-09-16T15:47:26.757792+0000","last_became_active":"2024-09-16T15:47:25.729739+0000","last_became_peered":"2024-09-16T15:47:25.729739+0000","last_unstale":"2024-09-16T15:47:26.757792+0000","last_undegraded":"2024-09-16T15:47:26.757792+0000","last_fullsized":"2024-09-16T15:47:26.757792+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:35.834157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.748596+0000","last_change":"2024-09-16T15:47:26.741342+0000","last_active":"2024-09-16T15:47:26.748596+0000","last_peered":"2024-09-16T15:47:26.748596+0000","last_clean":"2024-09-16T15:47:26.748596+0000","last_became_active":"2024-09-16T15:47:26.741017+0000","last_became_peered":"2024-09-16T15:47:26.741017+0000","last_unstale":"2024-09-16T15:47:26.748596+0000","last_undegraded":"2024-09-16T15:47:26.748596+0000","last_fullsized":"2024-09-16T15:47:26.748596+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:10.927948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.732886+0000","last_change":"2024-09-16T15:47:25.734026+0000","last_active":"2024-09-16T15:47:26.732886+0000","last_peered":"2024-09-16T15:47:26.732886+0000","last_clean":"2024-09-16T15:47:26.732886+0000","last_became_active":"2024-09-16T15:47:25.732937+0000","last_became_peered":"2024-09-16T15:47:25.732937+0000","last_unstale":"2024-09-16T15:47:26.732886+0000","last_undegraded":"2024-09-16T15:47:26.732886+0000","last_fullsized":"2024-09-16T15:47:26.732886+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:41.808076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742318+0000","last_change":"2024-09-16T15:47:26.738799+0000","last_active":"2024-09-16T15:47:26.742318+0000","last_peered":"2024-09-16T15:47:26.742318+0000","last_clean":"2024-09-16T15:47:26.742318+0000","last_became_active":"2024-09-16T15:47:26.738193+0000","last_became_peered":"2024-09-16T15:47:26.738193+0000","last_unstale":"2024-09-16T15:47:26.742318+0000","last_undegraded":"2024-09-16T15:47:26.742318+0000","last_fullsized":"2024-09-16T15:47:26.742318+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:44.952725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.722827+0000","last_change":"2024-09-16T15:47:25.736151+0000","last_active":"2024-09-16T15:47:26.722827+0000","last_peered":"2024-09-16T15:47:26.722827+0000","last_clean":"2024-09-16T15:47:26.722827+0000","last_became_active":"2024-09-16T15:47:25.735301+0000","last_became_peered":"2024-09-16T15:47:25.735301+0000","last_unstale":"2024-09-16T15:47:26.722827+0000","last_undegraded":"2024-09-16T15:47:26.722827+0000","last_fullsized":"2024-09-16T15:47:26.722827+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:18.390003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.749720+0000","last_change":"2024-09-16T15:47:26.744761+0000","last_active":"2024-09-16T15:47:26.749720+0000","last_peered":"2024-09-16T15:47:26.749720+0000","last_clean":"2024-09-16T15:47:26.749720+0000","last_became_active":"2024-09-16T15:47:26.743460+0000","last_became_peered":"2024-09-16T15:47:26.743460+0000","last_unstale":"2024-09-16T15:47:26.749720+0000","last_undegraded":"2024-09-16T15:47:26.749720+0000","last_fullsized":"2024-09-16T15:47:26.749720+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:15.409871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.732544+0000","last_change":"2024-09-16T15:47:25.733805+0000","last_active":"2024-09-16T15:47:26.732544+0000","last_peered":"2024-09-16T15:47:26.732544+0000","last_clean":"2024-09-16T15:47:26.732544+0000","last_became_active":"2024-09-16T15:47:25.733550+0000","last_became_peered":"2024-09-16T15:47:25.733550+0000","last_unstale":"2024-09-16T15:47:26.732544+0000","last_undegraded":"2024-09-16T15:47:26.732544+0000","last_fullsized":"2024-09-16T15:47:26.732544+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:17.610356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.736557+0000","last_change":"2024-09-16T15:47:26.730945+0000","last_active":"2024-09-16T15:47:26.736557+0000","last_peered":"2024-09-16T15:47:26.736557+0000","last_clean":"2024-09-16T15:47:26.736557+0000","last_became_active":"2024-09-16T15:47:26.730661+0000","last_became_peered":"2024-09-16T15:47:26.730661+0000","last_unstale":"2024-09-16T15:47:26.736557+0000","last_undegraded":"2024-09-16T15:47:26.736557+0000","last_fullsized":"2024-09-16T15:47:26.736557+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:28.879178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"160'3","reported_seq":16,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757444+0000","last_change":"2024-09-16T15:47:25.734701+0000","last_active":"2024-09-16T15:47:26.757444+0000","last_peered":"2024-09-16T15:47:26.757444+0000","last_clean":"2024-09-16T15:47:26.757444+0000","last_became_active":"2024-09-16T15:47:25.734408+0000","last_became_peered":"2024-09-16T15:47:25.734408+0000","last_unstale":"2024-09-16T15:47:26.757444+0000","last_undegraded":"2024-09-16T15:47:26.757444+0000","last_fullsized":"2024-09-16T15:47:26.757444+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:47:37.541285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.751381+0000","last_change":"2024-09-16T15:47:26.743924+0000","last_active":"2024-09-16T15:47:26.751381+0000","last_peered":"2024-09-16T15:47:26.751381+0000","last_clean":"2024-09-16T15:47:26.751381+0000","last_became_active":"2024-09-16T15:47:26.741579+0000","last_became_peered":"2024-09-16T15:47:26.741579+0000","last_unstale":"2024-09-16T15:47:26.751381+0000","last_undegraded":"2024-09-16T15:47:26.751381+0000","last_fullsized":"2024-09-16T15:47:26.751381+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:47:46.994742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"160'6","reported_seq":19,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.763267+0000","last_change":"2024-09-16T15:47:25.731148+0000","last_active":"2024-09-16T15:47:26.763267+0000","last_peered":"2024-09-16T15:47:26.763267+0000","last_clean":"2024-09-16T15:47:26.763267+0000","last_became_active":"2024-09-16T15:47:25.730832+0000","last_became_peered":"2024-09-16T15:47:25.730832+0000","last_unstale":"2024-09-16T15:47:26.763267+0000","last_undegraded":"2024-09-16T15:47:26.763267+0000","last_fullsized":"2024-09-16T15:47:26.763267+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:44:52.104710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.738200+0000","last_change":"2024-09-16T15:47:26.736931+0000","last_active":"2024-09-16T15:47:26.738200+0000","last_peered":"2024-09-16T15:47:26.738200+0000","last_clean":"2024-09-16T15:47:26.738200+0000","last_became_active":"2024-09-16T15:47:26.736638+0000","last_became_peered":"2024-09-16T15:47:26.736638+0000","last_unstale":"2024-09-16T15:47:26.738200+0000","last_undegraded":"2024-09-16T15:47:26.738200+0000","last_fullsized":"2024-09-16T15:47:26.738200+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:02.909340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758636+0000","last_change":"2024-09-16T15:47:25.731456+0000","last_active":"2024-09-16T15:47:26.758636+0000","last_peered":"2024-09-16T15:47:26.758636+0000","last_clean":"2024-09-16T15:47:26.758636+0000","last_became_active":"2024-09-16T15:47:25.731162+0000","last_became_peered":"2024-09-16T15:47:25.731162+0000","last_unstale":"2024-09-16T15:47:26.758636+0000","last_undegraded":"2024-09-16T15:47:26.758636+0000","last_fullsized":"2024-09-16T15:47:26.758636+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:17.257643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744342+0000","last_change":"2024-09-16T15:47:26.736789+0000","last_active":"2024-09-16T15:47:26.744342+0000","last_peered":"2024-09-16T15:47:26.744342+0000","last_clean":"2024-09-16T15:47:26.744342+0000","last_became_active":"2024-09-16T15:47:26.736551+0000","last_became_peered":"2024-09-16T15:47:26.736551+0000","last_unstale":"2024-09-16T15:47:26.744342+0000","last_undegraded":"2024-09-16T15:47:26.744342+0000","last_fullsized":"2024-09-16T15:47:26.744342+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:20.838172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.719517+0000","last_change":"2024-09-16T15:47:25.743802+0000","last_active":"2024-09-16T15:47:26.719517+0000","last_peered":"2024-09-16T15:47:26.719517+0000","last_clean":"2024-09-16T15:47:26.719517+0000","last_became_active":"2024-09-16T15:47:25.743331+0000","last_became_peered":"2024-09-16T15:47:25.743331+0000","last_unstale":"2024-09-16T15:47:26.719517+0000","last_undegraded":"2024-09-16T15:47:26.719517+0000","last_fullsized":"2024-09-16T15:47:26.719517+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:52.190872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.745956+0000","last_change":"2024-09-16T15:47:26.738649+0000","last_active":"2024-09-16T15:47:26.745956+0000","last_peered":"2024-09-16T15:47:26.745956+0000","last_clean":"2024-09-16T15:47:26.745956+0000","last_became_active":"2024-09-16T15:47:26.738336+0000","last_became_peered":"2024-09-16T15:47:26.738336+0000","last_unstale":"2024-09-16T15:47:26.745956+0000","last_undegraded":"2024-09-16T15:47:26.745956+0000","last_fullsized":"2024-09-16T15:47:26.745956+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:01.208576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.728925+0000","last_change":"2024-09-16T15:47:25.731018+0000","last_active":"2024-09-16T15:47:26.728925+0000","last_peered":"2024-09-16T15:47:26.728925+0000","last_clean":"2024-09-16T15:47:26.728925+0000","last_became_active":"2024-09-16T15:47:25.730771+0000","last_became_peered":"2024-09-16T15:47:25.730771+0000","last_unstale":"2024-09-16T15:47:26.728925+0000","last_undegraded":"2024-09-16T15:47:26.728925+0000","last_fullsized":"2024-09-16T15:47:26.728925+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:28.556954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.736791+0000","last_change":"2024-09-16T15:47:26.728974+0000","last_active":"2024-09-16T15:47:26.736791+0000","last_peered":"2024-09-16T15:47:26.736791+0000","last_clean":"2024-09-16T15:47:26.736791+0000","last_became_active":"2024-09-16T15:47:26.727711+0000","last_became_peered":"2024-09-16T15:47:26.727711+0000","last_unstale":"2024-09-16T15:47:26.736791+0000","last_undegraded":"2024-09-16T15:47:26.736791+0000","last_fullsized":"2024-09-16T15:47:26.736791+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:57.181888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"160'8","reported_seq":21,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758527+0000","last_change":"2024-09-16T15:47:25.733393+0000","last_active":"2024-09-16T15:47:26.758527+0000","last_peered":"2024-09-16T15:47:26.758527+0000","last_clean":"2024-09-16T15:47:26.758527+0000","last_became_active":"2024-09-16T15:47:25.732964+0000","last_became_peered":"2024-09-16T15:47:25.732964+0000","last_unstale":"2024-09-16T15:47:26.758527+0000","last_undegraded":"2024-09-16T15:47:26.758527+0000","last_fullsized":"2024-09-16T15:47:26.758527+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:10.318582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742182+0000","last_change":"2024-09-16T15:47:26.733206+0000","last_active":"2024-09-16T15:47:26.742182+0000","last_peered":"2024-09-16T15:47:26.742182+0000","last_clean":"2024-09-16T15:47:26.742182+0000","last_became_active":"2024-09-16T15:47:26.732932+0000","last_became_peered":"2024-09-16T15:47:26.732932+0000","last_unstale":"2024-09-16T15:47:26.742182+0000","last_undegraded":"2024-09-16T15:47:26.742182+0000","last_fullsized":"2024-09-16T15:47:26.742182+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:13:28.912805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.759281+0000","last_change":"2024-09-16T15:47:25.741726+0000","last_active":"2024-09-16T15:47:26.759281+0000","last_peered":"2024-09-16T15:47:26.759281+0000","last_clean":"2024-09-16T15:47:26.759281+0000","last_became_active":"2024-09-16T15:47:25.741038+0000","last_became_peered":"2024-09-16T15:47:25.741038+0000","last_unstale":"2024-09-16T15:47:26.759281+0000","last_undegraded":"2024-09-16T15:47:26.759281+0000","last_fullsized":"2024-09-16T15:47:26.759281+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:38:05.034213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.753485+0000","last_change":"2024-09-16T15:47:26.750397+0000","last_active":"2024-09-16T15:47:26.753485+0000","last_peered":"2024-09-16T15:47:26.753485+0000","last_clean":"2024-09-16T15:47:26.753485+0000","last_became_active":"2024-09-16T15:47:26.748909+0000","last_became_peered":"2024-09-16T15:47:26.748909+0000","last_unstale":"2024-09-16T15:47:26.753485+0000","last_undegraded":"2024-09-16T15:47:26.753485+0000","last_fullsized":"2024-09-16T15:47:26.753485+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:27:17.332634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.755742+0000","last_change":"2024-09-16T15:47:25.731956+0000","last_active":"2024-09-16T15:47:26.755742+0000","last_peered":"2024-09-16T15:47:26.755742+0000","last_clean":"2024-09-16T15:47:26.755742+0000","last_became_active":"2024-09-16T15:47:25.731669+0000","last_became_peered":"2024-09-16T15:47:25.731669+0000","last_unstale":"2024-09-16T15:47:26.755742+0000","last_undegraded":"2024-09-16T15:47:26.755742+0000","last_fullsized":"2024-09-16T15:47:26.755742+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:08.890275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.750531+0000","last_change":"2024-09-16T15:47:26.741549+0000","last_active":"2024-09-16T15:47:26.750531+0000","last_peered":"2024-09-16T15:47:26.750531+0000","last_clean":"2024-09-16T15:47:26.750531+0000","last_became_active":"2024-09-16T15:47:26.741264+0000","last_became_peered":"2024-09-16T15:47:26.741264+0000","last_unstale":"2024-09-16T15:47:26.750531+0000","last_undegraded":"2024-09-16T15:47:26.750531+0000","last_fullsized":"2024-09-16T15:47:26.750531+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:56:26.270706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"160'1","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.756604+0000","last_change":"2024-09-16T15:47:25.737911+0000","last_active":"2024-09-16T15:47:26.756604+0000","last_peered":"2024-09-16T15:47:26.756604+0000","last_clean":"2024-09-16T15:47:26.756604+0000","last_became_active":"2024-09-16T15:47:25.737601+0000","last_became_peered":"2024-09-16T15:47:25.737601+0000","last_unstale":"2024-09-16T15:47:26.756604+0000","last_undegraded":"2024-09-16T15:47:26.756604+0000","last_fullsized":"2024-09-16T15:47:26.756604+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:14.402903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752187+0000","last_change":"2024-09-16T15:47:26.744552+0000","last_active":"2024-09-16T15:47:26.752187+0000","last_peered":"2024-09-16T15:47:26.752187+0000","last_clean":"2024-09-16T15:47:26.752187+0000","last_became_active":"2024-09-16T15:47:26.744306+0000","last_became_peered":"2024-09-16T15:47:26.744306+0000","last_unstale":"2024-09-16T15:47:26.752187+0000","last_undegraded":"2024-09-16T15:47:26.752187+0000","last_fullsized":"2024-09-16T15:47:26.752187+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:28.753774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.723003+0000","last_change":"2024-09-16T15:47:25.736185+0000","last_active":"2024-09-16T15:47:26.723003+0000","last_peered":"2024-09-16T15:47:26.723003+0000","last_clean":"2024-09-16T15:47:26.723003+0000","last_became_active":"2024-09-16T15:47:25.735711+0000","last_became_peered":"2024-09-16T15:47:25.735711+0000","last_unstale":"2024-09-16T15:47:26.723003+0000","last_undegraded":"2024-09-16T15:47:26.723003+0000","last_fullsized":"2024-09-16T15:47:26.723003+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:25.146154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757290+0000","last_change":"2024-09-16T15:47:26.750853+0000","last_active":"2024-09-16T15:47:26.757290+0000","last_peered":"2024-09-16T15:47:26.757290+0000","last_clean":"2024-09-16T15:47:26.757290+0000","last_became_active":"2024-09-16T15:47:26.750460+0000","last_became_peered":"2024-09-16T15:47:26.750460+0000","last_unstale":"2024-09-16T15:47:26.757290+0000","last_undegraded":"2024-09-16T15:47:26.757290+0000","last_fullsized":"2024-09-16T15:47:26.757290+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:31.569552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.757353+0000","last_change":"2024-09-16T15:47:25.740990+0000","last_active":"2024-09-16T15:47:26.757353+0000","last_peered":"2024-09-16T15:47:26.757353+0000","last_clean":"2024-09-16T15:47:26.757353+0000","last_became_active":"2024-09-16T15:47:25.740647+0000","last_became_peered":"2024-09-16T15:47:25.740647+0000","last_unstale":"2024-09-16T15:47:26.757353+0000","last_undegraded":"2024-09-16T15:47:26.757353+0000","last_fullsized":"2024-09-16T15:47:26.757353+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:43.737825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.744312+0000","last_change":"2024-09-16T15:47:26.736168+0000","last_active":"2024-09-16T15:47:26.744312+0000","last_peered":"2024-09-16T15:47:26.744312+0000","last_clean":"2024-09-16T15:47:26.744312+0000","last_became_active":"2024-09-16T15:47:26.735472+0000","last_became_peered":"2024-09-16T15:47:26.735472+0000","last_unstale":"2024-09-16T15:47:26.744312+0000","last_undegraded":"2024-09-16T15:47:26.744312+0000","last_fullsized":"2024-09-16T15:47:26.744312+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:34:30.205847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.731446+0000","last_change":"2024-09-16T15:47:25.727858+0000","last_active":"2024-09-16T15:47:26.731446+0000","last_peered":"2024-09-16T15:47:26.731446+0000","last_clean":"2024-09-16T15:47:26.731446+0000","last_became_active":"2024-09-16T15:47:25.727601+0000","last_became_peered":"2024-09-16T15:47:25.727601+0000","last_unstale":"2024-09-16T15:47:26.731446+0000","last_undegraded":"2024-09-16T15:47:26.731446+0000","last_fullsized":"2024-09-16T15:47:26.731446+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:09.996955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.748751+0000","last_change":"2024-09-16T15:47:26.741675+0000","last_active":"2024-09-16T15:47:26.748751+0000","last_peered":"2024-09-16T15:47:26.748751+0000","last_clean":"2024-09-16T15:47:26.748751+0000","last_became_active":"2024-09-16T15:47:26.741408+0000","last_became_peered":"2024-09-16T15:47:26.741408+0000","last_unstale":"2024-09-16T15:47:26.748751+0000","last_undegraded":"2024-09-16T15:47:26.748751+0000","last_fullsized":"2024-09-16T15:47:26.748751+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:28.336349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.728599+0000","last_change":"2024-09-16T15:47:25.740622+0000","last_active":"2024-09-16T15:47:26.728599+0000","last_peered":"2024-09-16T15:47:26.728599+0000","last_clean":"2024-09-16T15:47:26.728599+0000","last_became_active":"2024-09-16T15:47:25.739380+0000","last_became_peered":"2024-09-16T15:47:25.739380+0000","last_unstale":"2024-09-16T15:47:26.728599+0000","last_undegraded":"2024-09-16T15:47:26.728599+0000","last_fullsized":"2024-09-16T15:47:26.728599+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:42.247704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.740599+0000","last_change":"2024-09-16T15:47:26.739451+0000","last_active":"2024-09-16T15:47:26.740599+0000","last_peered":"2024-09-16T15:47:26.740599+0000","last_clean":"2024-09-16T15:47:26.740599+0000","last_became_active":"2024-09-16T15:47:26.738692+0000","last_became_peered":"2024-09-16T15:47:26.738692+0000","last_unstale":"2024-09-16T15:47:26.740599+0000","last_undegraded":"2024-09-16T15:47:26.740599+0000","last_fullsized":"2024-09-16T15:47:26.740599+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:49.696373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.725503+0000","last_change":"2024-09-16T15:47:25.734530+0000","last_active":"2024-09-16T15:47:26.725503+0000","last_peered":"2024-09-16T15:47:26.725503+0000","last_clean":"2024-09-16T15:47:26.725503+0000","last_became_active":"2024-09-16T15:47:25.734113+0000","last_became_peered":"2024-09-16T15:47:25.734113+0000","last_unstale":"2024-09-16T15:47:26.725503+0000","last_undegraded":"2024-09-16T15:47:26.725503+0000","last_fullsized":"2024-09-16T15:47:26.725503+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:17.659599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.740510+0000","last_change":"2024-09-16T15:47:26.738883+0000","last_active":"2024-09-16T15:47:26.740510+0000","last_peered":"2024-09-16T15:47:26.740510+0000","last_clean":"2024-09-16T15:47:26.740510+0000","last_became_active":"2024-09-16T15:47:26.738596+0000","last_became_peered":"2024-09-16T15:47:26.738596+0000","last_unstale":"2024-09-16T15:47:26.740510+0000","last_undegraded":"2024-09-16T15:47:26.740510+0000","last_fullsized":"2024-09-16T15:47:26.740510+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:05.179535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"160'2","reported_seq":15,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.758079+0000","last_change":"2024-09-16T15:47:25.741904+0000","last_active":"2024-09-16T15:47:26.758079+0000","last_peered":"2024-09-16T15:47:26.758079+0000","last_clean":"2024-09-16T15:47:26.758079+0000","last_became_active":"2024-09-16T15:47:25.741666+0000","last_became_peered":"2024-09-16T15:47:25.741666+0000","last_unstale":"2024-09-16T15:47:26.758079+0000","last_undegraded":"2024-09-16T15:47:26.758079+0000","last_fullsized":"2024-09-16T15:47:26.758079+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:18.924111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752565+0000","last_change":"2024-09-16T15:47:26.745919+0000","last_active":"2024-09-16T15:47:26.752565+0000","last_peered":"2024-09-16T15:47:26.752565+0000","last_clean":"2024-09-16T15:47:26.752565+0000","last_became_active":"2024-09-16T15:47:26.744056+0000","last_became_peered":"2024-09-16T15:47:26.744056+0000","last_unstale":"2024-09-16T15:47:26.752565+0000","last_undegraded":"2024-09-16T15:47:26.752565+0000","last_fullsized":"2024-09-16T15:47:26.752565+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:16.753771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.720429+0000","last_change":"2024-09-16T15:47:25.735712+0000","last_active":"2024-09-16T15:47:26.720429+0000","last_peered":"2024-09-16T15:47:26.720429+0000","last_clean":"2024-09-16T15:47:26.720429+0000","last_became_active":"2024-09-16T15:47:25.733278+0000","last_became_peered":"2024-09-16T15:47:25.733278+0000","last_unstale":"2024-09-16T15:47:26.720429+0000","last_undegraded":"2024-09-16T15:47:26.720429+0000","last_fullsized":"2024-09-16T15:47:26.720429+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:35.388104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.746640+0000","last_change":"2024-09-16T15:47:26.744748+0000","last_active":"2024-09-16T15:47:26.746640+0000","last_peered":"2024-09-16T15:47:26.746640+0000","last_clean":"2024-09-16T15:47:26.746640+0000","last_became_active":"2024-09-16T15:47:26.744452+0000","last_became_peered":"2024-09-16T15:47:26.744452+0000","last_unstale":"2024-09-16T15:47:26.746640+0000","last_undegraded":"2024-09-16T15:47:26.746640+0000","last_fullsized":"2024-09-16T15:47:26.746640+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:11.325550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.719856+0000","last_change":"2024-09-16T15:47:25.733325+0000","last_active":"2024-09-16T15:47:26.719856+0000","last_peered":"2024-09-16T15:47:26.719856+0000","last_clean":"2024-09-16T15:47:26.719856+0000","last_became_active":"2024-09-16T15:47:25.733034+0000","last_became_peered":"2024-09-16T15:47:25.733034+0000","last_unstale":"2024-09-16T15:47:26.719856+0000","last_undegraded":"2024-09-16T15:47:26.719856+0000","last_fullsized":"2024-09-16T15:47:26.719856+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:34.867012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.742567+0000","last_change":"2024-09-16T15:47:26.737216+0000","last_active":"2024-09-16T15:47:26.742567+0000","last_peered":"2024-09-16T15:47:26.742567+0000","last_clean":"2024-09-16T15:47:26.742567+0000","last_became_active":"2024-09-16T15:47:26.735395+0000","last_became_peered":"2024-09-16T15:47:26.735395+0000","last_unstale":"2024-09-16T15:47:26.742567+0000","last_undegraded":"2024-09-16T15:47:26.742567+0000","last_fullsized":"2024-09-16T15:47:26.742567+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:36.077243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.728074+0000","last_change":"2024-09-16T15:47:25.732565+0000","last_active":"2024-09-16T15:47:26.728074+0000","last_peered":"2024-09-16T15:47:26.728074+0000","last_clean":"2024-09-16T15:47:26.728074+0000","last_became_active":"2024-09-16T15:47:25.731334+0000","last_became_peered":"2024-09-16T15:47:25.731334+0000","last_unstale":"2024-09-16T15:47:26.728074+0000","last_undegraded":"2024-09-16T15:47:26.728074+0000","last_fullsized":"2024-09-16T15:47:26.728074+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:28.672670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752792+0000","last_change":"2024-09-16T15:47:26.748025+0000","last_active":"2024-09-16T15:47:26.752792+0000","last_peered":"2024-09-16T15:47:26.752792+0000","last_clean":"2024-09-16T15:47:26.752792+0000","last_became_active":"2024-09-16T15:47:26.747658+0000","last_became_peered":"2024-09-16T15:47:26.747658+0000","last_unstale":"2024-09-16T15:47:26.752792+0000","last_undegraded":"2024-09-16T15:47:26.752792+0000","last_fullsized":"2024-09-16T15:47:26.752792+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:56.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.725678+0000","last_change":"2024-09-16T15:47:25.737685+0000","last_active":"2024-09-16T15:47:26.725678+0000","last_peered":"2024-09-16T15:47:26.725678+0000","last_clean":"2024-09-16T15:47:26.725678+0000","last_became_active":"2024-09-16T15:47:25.737280+0000","last_became_peered":"2024-09-16T15:47:25.737280+0000","last_unstale":"2024-09-16T15:47:26.725678+0000","last_undegraded":"2024-09-16T15:47:26.725678+0000","last_fullsized":"2024-09-16T15:47:26.725678+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:19:34.957214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.753044+0000","last_change":"2024-09-16T15:47:26.749089+0000","last_active":"2024-09-16T15:47:26.753044+0000","last_peered":"2024-09-16T15:47:26.753044+0000","last_clean":"2024-09-16T15:47:26.753044+0000","last_became_active":"2024-09-16T15:47:26.748866+0000","last_became_peered":"2024-09-16T15:47:26.748866+0000","last_unstale":"2024-09-16T15:47:26.753044+0000","last_undegraded":"2024-09-16T15:47:26.753044+0000","last_fullsized":"2024-09-16T15:47:26.753044+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:15.005176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.726169+0000","last_change":"2024-09-16T15:47:25.731489+0000","last_active":"2024-09-16T15:47:26.726169+0000","last_peered":"2024-09-16T15:47:26.726169+0000","last_clean":"2024-09-16T15:47:26.726169+0000","last_became_active":"2024-09-16T15:47:25.731230+0000","last_became_peered":"2024-09-16T15:47:25.731230+0000","last_unstale":"2024-09-16T15:47:26.726169+0000","last_undegraded":"2024-09-16T15:47:26.726169+0000","last_fullsized":"2024-09-16T15:47:26.726169+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:59.606142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.752401+0000","last_change":"2024-09-16T15:47:26.745679+0000","last_active":"2024-09-16T15:47:26.752401+0000","last_peered":"2024-09-16T15:47:26.752401+0000","last_clean":"2024-09-16T15:47:26.752401+0000","last_became_active":"2024-09-16T15:47:26.744684+0000","last_became_peered":"2024-09-16T15:47:26.744684+0000","last_unstale":"2024-09-16T15:47:26.752401+0000","last_undegraded":"2024-09-16T15:47:26.752401+0000","last_fullsized":"2024-09-16T15:47:26.752401+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:36.691777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.721754+0000","last_change":"2024-09-16T15:47:25.733626+0000","last_active":"2024-09-16T15:47:26.721754+0000","last_peered":"2024-09-16T15:47:26.721754+0000","last_clean":"2024-09-16T15:47:26.721754+0000","last_became_active":"2024-09-16T15:47:25.732281+0000","last_became_peered":"2024-09-16T15:47:25.732281+0000","last_unstale":"2024-09-16T15:47:26.721754+0000","last_undegraded":"2024-09-16T15:47:26.721754+0000","last_fullsized":"2024-09-16T15:47:26.721754+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:42.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.756883+0000","last_change":"2024-09-16T15:47:26.753758+0000","last_active":"2024-09-16T15:47:26.756883+0000","last_peered":"2024-09-16T15:47:26.756883+0000","last_clean":"2024-09-16T15:47:26.756883+0000","last_became_active":"2024-09-16T15:47:26.749557+0000","last_became_peered":"2024-09-16T15:47:26.749557+0000","last_unstale":"2024-09-16T15:47:26.756883+0000","last_undegraded":"2024-09-16T15:47:26.756883+0000","last_fullsized":"2024-09-16T15:47:26.756883+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:52:33.369693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.728152+0000","last_change":"2024-09-16T15:47:25.735373+0000","last_active":"2024-09-16T15:47:26.728152+0000","last_peered":"2024-09-16T15:47:26.728152+0000","last_clean":"2024-09-16T15:47:26.728152+0000","last_became_active":"2024-09-16T15:47:25.735084+0000","last_became_peered":"2024-09-16T15:47:25.735084+0000","last_unstale":"2024-09-16T15:47:26.728152+0000","last_undegraded":"2024-09-16T15:47:26.728152+0000","last_fullsized":"2024-09-16T15:47:26.728152+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:41.496815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.736443+0000","last_change":"2024-09-16T15:47:26.728573+0000","last_active":"2024-09-16T15:47:26.736443+0000","last_peered":"2024-09-16T15:47:26.736443+0000","last_clean":"2024-09-16T15:47:26.736443+0000","last_became_active":"2024-09-16T15:47:26.728305+0000","last_became_peered":"2024-09-16T15:47:26.728305+0000","last_unstale":"2024-09-16T15:47:26.736443+0000","last_undegraded":"2024-09-16T15:47:26.736443+0000","last_fullsized":"2024-09-16T15:47:26.736443+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:25.707049+0000","last_clean_scrub_stamp":"2024-09-16T15:47:25.707049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:06.725094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":13,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.731632+0000","last_change":"2024-09-16T15:47:25.731262+0000","last_active":"2024-09-16T15:47:26.731632+0000","last_peered":"2024-09-16T15:47:26.731632+0000","last_clean":"2024-09-16T15:47:26.731632+0000","last_became_active":"2024-09-16T15:47:25.730949+0000","last_became_peered":"2024-09-16T15:47:25.730949+0000","last_unstale":"2024-09-16T15:47:26.731632+0000","last_undegraded":"2024-09-16T15:47:26.731632+0000","last_fullsized":"2024-09-16T15:47:26.731632+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:47:24.700081+0000","last_clean_scrub_stamp":"2024-09-16T15:47:24.700081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:45.129580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":392,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-16T15:47:26.733037+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:47:26.733037+0000","last_peered":"2024-09-16T15:47:26.733037+0000","last_clean":"2024-09-16T15:47:26.733037+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:47:26.733037+0000","last_undegraded":"2024-09-16T15:47:26.733037+0000","last_fullsized":"2024-09-16T15:47:26.733037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33744,"kb_used_data":2548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338096,"statfs":{"total":96636764160,"available":96602210304,"internally_reserved":0,"allocated":2609152,"data_stored":1476597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509553,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":886125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1994752,"data_stored":879322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2023424,"data_stored":888319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37324,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334516,"statfs":{"total":96636764160,"available":96598544384,"internally_reserved":0,"allocated":2019328,"data_stored":886319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37324,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334516,"statfs":{"total":96636764160,"available":96598544384,"internally_reserved":0,"allocated":2019328,"data_stored":886819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37300,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334540,"statfs":{"total":96636764160,"available":96598568960,"internally_reserved":0,"allocated":1994752,"data_stored":879322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509553,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37876,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333964,"statfs":{"total":96636764160,"available":96597979136,"internally_reserved":0,"allocated":2588672,"data_stored":1469690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:47:31.408 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:47:31.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:47:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.611+0000 7f07f6400640 1 -- 172.21.15.60:0/3526669849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 msgr2=0x7f07f00cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.611+0000 7f07f6400640 1 --2- 172.21.15.60:0/3526669849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 0x7f07f00cfae0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f07f00ce670 tx=0x7f07e802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.611+0000 7f07f6400640 1 -- 172.21.15.60:0/3526669849 shutdown_connections 2024-09-16T15:47:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.611+0000 7f07f6400640 1 --2- 172.21.15.60:0/3526669849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 0x7f07f00cfae0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.611+0000 7f07f6400640 1 --2- 172.21.15.60:0/3526669849 >> v2:172.21.15.100:3300/0 conn(0x7f07f01618c0 0x7f07f00d4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.611+0000 7f07f6400640 1 -- 172.21.15.60:0/3526669849 >> 172.21.15.60:0/3526669849 conn(0x7f07f00d40a0 msgr2=0x7f07f00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.611+0000 7f07f6400640 1 -- 172.21.15.60:0/3526669849 shutdown_connections 2024-09-16T15:47:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.611+0000 7f07f6400640 1 -- 172.21.15.60:0/3526669849 wait complete. 2024-09-16T15:47:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.612+0000 7f07f6400640 1 Processor -- start 2024-09-16T15:47:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.613+0000 7f07f6400640 1 -- start start 2024-09-16T15:47:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.613+0000 7f07f6400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 0x7f07f014a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.613+0000 7f07f6400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f01614e0 0x7f07f00c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.613+0000 7f07f6400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f01618c0 0x7f07f00c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07f6400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f07f00d24e0 con 0x7f07f00cf630 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07f6400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f07f00d2360 con 0x7f07f01614e0 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07f6400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f07f00d2660 con 0x7f07f01618c0 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07ef400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 0x7f07f014a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07ef400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 0x7f07f014a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44448/0 (socket says 172.21.15.60:44448) 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07ef400640 1 -- 172.21.15.60:0/2710850680 learned_addr learned my addr 172.21.15.60:0/2710850680 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07efe00640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f01618c0 0x7f07f00c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07e7e00640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f01614e0 0x7f07f00c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07ef400640 1 -- 172.21.15.60:0/2710850680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f01618c0 msgr2=0x7f07f00c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07ef400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f01618c0 0x7f07f00c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07ef400640 1 -- 172.21.15.60:0/2710850680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f01614e0 msgr2=0x7f07f00c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07ef400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f01614e0 0x7f07f00c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.614+0000 7f07ef400640 1 -- 172.21.15.60:0/2710850680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f00b8de0 con 0x7f07f00cf630 2024-09-16T15:47:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.615+0000 7f07ef400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 0x7f07f014a530 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c500 tx=0x7f07e000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.615+0000 7f07e7400640 1 -- 172.21.15.60:0/2710850680 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0004230 con 0x7f07f00cf630 2024-09-16T15:47:31.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.615+0000 7f07e7400640 1 -- 172.21.15.60:0/2710850680 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07e00044c0 con 0x7f07f00cf630 2024-09-16T15:47:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.615+0000 7f07e7400640 1 -- 172.21.15.60:0/2710850680 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e00038a0 con 0x7f07f00cf630 2024-09-16T15:47:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.615+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f00c9b10 con 0x7f07f00cf630 2024-09-16T15:47:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.615+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f00ca090 con 0x7f07f00cf630 2024-09-16T15:47:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.616+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f00cfae0 con 0x7f07f00cf630 2024-09-16T15:47:31.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.617+0000 7f07e7400640 1 -- 172.21.15.60:0/2710850680 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f07e0003a40 con 0x7f07f00cf630 2024-09-16T15:47:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.618+0000 7f07e7400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f07cc0790c0 0x7f07cc07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.618+0000 7f07e7400640 1 -- 172.21.15.60:0/2710850680 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f07e009bdd0 con 0x7f07f00cf630 2024-09-16T15:47:31.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.618+0000 7f07e7e00640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f07cc0790c0 0x7f07cc07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.619+0000 7f07e7e00640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f07cc0790c0 0x7f07cc07b580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f07f0156d00 tx=0x7f07d8005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:31.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.622+0000 7f07e7400640 1 -- 172.21.15.60:0/2710850680 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f07e0062db0 con 0x7f07f00cf630 2024-09-16T15:47:31.705 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-09-16T15:47:31.704+0000 7fb866800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:47:31.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:47:31.788 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:47:31.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.892+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f07f00ba020 con 0x7f07f00cf630 2024-09-16T15:47:31.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.895+0000 7f07e7400640 1 -- 172.21.15.60:0/2710850680 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f07e00677e0 con 0x7f07f00cf630 2024-09-16T15:47:31.896 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:47:31.896 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:47:31.897 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:47:31.897 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:47:31.897 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:47:31.897 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:47:31.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f07cc0790c0 msgr2=0x7f07cc07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f07cc0790c0 0x7f07cc07b580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f07f0156d00 tx=0x7f07d8005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 msgr2=0x7f07f014a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 0x7f07f014a530 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c500 tx=0x7f07e000c9c0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 shutdown_connections 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f07cc0790c0 0x7f07cc07b580 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07f01618c0 0x7f07f00c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07f01614e0 0x7f07f00c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 --2- 172.21.15.60:0/2710850680 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07f00cf630 0x7f07f014a530 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 >> 172.21.15.60:0/2710850680 conn(0x7f07f00d40a0 msgr2=0x7f07f006e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 shutdown_connections 2024-09-16T15:47:31.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:31.898+0000 7f07f6400640 1 -- 172.21.15.60:0/2710850680 wait complete. 2024-09-16T15:47:31.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.115+0000 7f94a5800640 1 -- 172.21.15.60:0/1886467175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 msgr2=0x7f94a0074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.115+0000 7f94a5800640 1 --2- 172.21.15.60:0/1886467175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 0x7f94a0074440 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f9498005fd0 tx=0x7f949802a820 comp rx=0 tx=0).stop 2024-09-16T15:47:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.116+0000 7f94a5800640 1 -- 172.21.15.60:0/1886467175 shutdown_connections 2024-09-16T15:47:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.116+0000 7f94a5800640 1 --2- 172.21.15.60:0/1886467175 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 0x7f94a0074440 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.116+0000 7f94a5800640 1 --2- 172.21.15.60:0/1886467175 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94a00ccf00 0x7f94a00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.116+0000 7f94a5800640 1 -- 172.21.15.60:0/1886467175 >> 172.21.15.60:0/1886467175 conn(0x7f94a00b1950 msgr2=0x7f94a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.116+0000 7f94a5800640 1 -- 172.21.15.60:0/1886467175 shutdown_connections 2024-09-16T15:47:32.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.116+0000 7f94a5800640 1 -- 172.21.15.60:0/1886467175 wait complete. 2024-09-16T15:47:32.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.117+0000 7f94a5800640 1 Processor -- start 2024-09-16T15:47:32.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.117+0000 7f94a5800640 1 -- start start 2024-09-16T15:47:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.118+0000 7f94a5800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 0x7f94a014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.118+0000 7f94a5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a00ccf00 0x7f94a009c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.118+0000 7f94a5800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94a015f5a0 0x7f94a009ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.118+0000 7f94a5800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f94a006f2e0 con 0x7f94a0073fb0 2024-09-16T15:47:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.118+0000 7f94a5800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94a006f160 con 0x7f94a00ccf00 2024-09-16T15:47:32.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.118+0000 7f94a5800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f94a006f460 con 0x7f94a015f5a0 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 0x7f94a014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 0x7f94a014cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44466/0 (socket says 172.21.15.60:44466) 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 -- 172.21.15.60:0/4221559456 learned_addr learned my addr 172.21.15.60:0/4221559456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949f400640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94a015f5a0 0x7f94a009ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f9496000640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a00ccf00 0x7f94a009c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 -- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94a015f5a0 msgr2=0x7f94a009ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94a015f5a0 0x7f94a009ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 -- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a00ccf00 msgr2=0x7f94a009c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a00ccf00 0x7f94a009c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 -- 172.21.15.60:0/4221559456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a00cdd30 con 0x7f94a0073fb0 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949f400640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94a015f5a0 0x7f94a009ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f9496000640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a00ccf00 0x7f94a009c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.119+0000 7f949ea00640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 0x7f94a014cd10 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f949000c430 tx=0x7f949000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.120+0000 7f9497400640 1 -- 172.21.15.60:0/4221559456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490011790 con 0x7f94a0073fb0 2024-09-16T15:47:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.120+0000 7f9497400640 1 -- 172.21.15.60:0/4221559456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9490017070 con 0x7f94a0073fb0 2024-09-16T15:47:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.120+0000 7f9497400640 1 -- 172.21.15.60:0/4221559456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490011ce0 con 0x7f94a0073fb0 2024-09-16T15:47:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.120+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a00ce010 con 0x7f94a0073fb0 2024-09-16T15:47:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.120+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a00ce5d0 con 0x7f94a0073fb0 2024-09-16T15:47:32.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.121+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a00ce7d0 con 0x7f94a0073fb0 2024-09-16T15:47:32.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.122+0000 7f9497400640 1 -- 172.21.15.60:0/4221559456 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f949001c460 con 0x7f94a0073fb0 2024-09-16T15:47:32.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.123+0000 7f9497400640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f947c0790c0 0x7f947c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.123+0000 7f9497400640 1 -- 172.21.15.60:0/4221559456 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f949009b910 con 0x7f94a0073fb0 2024-09-16T15:47:32.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.123+0000 7f9496000640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f947c0790c0 0x7f947c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.124+0000 7f9496000640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f947c0790c0 0x7f947c07b580 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f94a01555b0 tx=0x7f9488005e50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.127+0000 7f9497400640 1 -- 172.21.15.60:0/4221559456 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f94a00ce7d0 con 0x7f94a0073fb0 2024-09-16T15:47:32.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.361+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94a006f990 con 0x7f94a0073fb0 2024-09-16T15:47:32.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.361+0000 7f9497400640 1 -- 172.21.15.60:0/4221559456 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5239 (secure 0 0 0) 0x7f9490062fc0 con 0x7f94a0073fb0 2024-09-16T15:47:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:32.363 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"btime":"2024-09-16T15:47:29:738014+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7361,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1464936997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1464936997},{"type":"v1","addr":"172.21.15.60:6834","nonce":1464936997}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":7364,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1611348877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1611348877},{"type":"v1","addr":"172.21.15.60:6835","nonce":1611348877}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2321854052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2321854052},{"type":"v1","addr":"172.21.15.100:6834","nonce":2321854052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:47:26.164998+0000","modified":"2024-09-16T15:47:29.176697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"d","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3325202304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":3325202304},{"type":"v1","addr":"172.21.15.100:6835","nonce":3325202304}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":12}]} 2024-09-16T15:47:32.363 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 112 2024-09-16T15:47:32.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f947c0790c0 msgr2=0x7f947c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f947c0790c0 0x7f947c07b580 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f94a01555b0 tx=0x7f9488005e50 comp rx=0 tx=0).stop 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 msgr2=0x7f94a014cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 0x7f94a014cd10 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f949000c430 tx=0x7f949000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 shutdown_connections 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f947c0790c0 0x7f947c07b580 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94a015f5a0 0x7f94a009ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94a00ccf00 0x7f94a009c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 --2- 172.21.15.60:0/4221559456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94a0073fb0 0x7f94a014cd10 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 >> 172.21.15.60:0/4221559456 conn(0x7f94a00b1950 msgr2=0x7f94a015de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 shutdown_connections 2024-09-16T15:47:32.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.364+0000 7f94a5800640 1 -- 172.21.15.60:0/4221559456 wait complete. 2024-09-16T15:47:32.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:47:26.164998+0000', 'modified': '2024-09-16T15:47:29.176697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8853}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8853': {'gid': 8853, 'name': 'd', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6835/3325202304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6801', 'nonce': 3325202304}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 3325202304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8853, 'qdb_cluster': [8853]} 2024-09-16T15:47:32.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:47:32.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-16T15:47:32.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.946+0000 7fd550c00640 1 -- 172.21.15.60:0/1085220213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c0725d0 msgr2=0x7fd54c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.946+0000 7fd550c00640 1 --2- 172.21.15.60:0/1085220213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c0725d0 0x7fd54c0b28f0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd538005fd0 tx=0x7fd53802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.946+0000 7fd550c00640 1 -- 172.21.15.60:0/1085220213 shutdown_connections 2024-09-16T15:47:32.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.946+0000 7fd550c00640 1 --2- 172.21.15.60:0/1085220213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd54c0d0540 0x7fd54c06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.946+0000 7fd550c00640 1 --2- 172.21.15.60:0/1085220213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c0725d0 0x7fd54c0b28f0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.946+0000 7fd550c00640 1 -- 172.21.15.60:0/1085220213 >> 172.21.15.60:0/1085220213 conn(0x7fd54c0cfa20 msgr2=0x7fd54c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.946+0000 7fd550c00640 1 -- 172.21.15.60:0/1085220213 shutdown_connections 2024-09-16T15:47:32.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.946+0000 7fd550c00640 1 -- 172.21.15.60:0/1085220213 wait complete. 2024-09-16T15:47:32.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.947+0000 7fd550c00640 1 Processor -- start 2024-09-16T15:47:32.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.948+0000 7fd550c00640 1 -- start start 2024-09-16T15:47:32.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.948+0000 7fd550c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd54c0d0540 0x7fd54c0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.948+0000 7fd550c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c15bc30 0x7fd54c14bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.948+0000 7fd550c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd54c14dc90 0x7fd54c14c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.948+0000 7fd550c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c074660 con 0x7fd54c15bc30 2024-09-16T15:47:32.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.948+0000 7fd550c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c06f860 con 0x7fd54c0d0540 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.948+0000 7fd550c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c155960 con 0x7fd54c14dc90 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c15bc30 0x7fd54c14bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c15bc30 0x7fd54c14bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44482/0 (socket says 172.21.15.60:44482) 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 -- 172.21.15.60:0/1924101732 learned_addr learned my addr 172.21.15.60:0/1924101732 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54b400640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd54c0d0540 0x7fd54c0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54be00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd54c14dc90 0x7fd54c14c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 -- 172.21.15.60:0/1924101732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd54c14dc90 msgr2=0x7fd54c14c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd54c14dc90 0x7fd54c14c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 -- 172.21.15.60:0/1924101732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd54c0d0540 msgr2=0x7fd54c0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd54c0d0540 0x7fd54c0c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 -- 172.21.15.60:0/1924101732 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c155bb0 con 0x7fd54c15bc30 2024-09-16T15:47:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.949+0000 7fd54aa00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c15bc30 0x7fd54c14bba0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd538009850 tx=0x7fd5380038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.950+0000 7fd543400640 1 -- 172.21.15.60:0/1924101732 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53803f070 con 0x7fd54c15bc30 2024-09-16T15:47:32.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.950+0000 7fd543400640 1 -- 172.21.15.60:0/1924101732 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5380047a0 con 0x7fd54c15bc30 2024-09-16T15:47:32.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.950+0000 7fd543400640 1 -- 172.21.15.60:0/1924101732 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538035030 con 0x7fd54c15bc30 2024-09-16T15:47:32.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.950+0000 7fd550c00640 1 -- 172.21.15.60:0/1924101732 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c155e90 con 0x7fd54c15bc30 2024-09-16T15:47:32.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.950+0000 7fd550c00640 1 -- 172.21.15.60:0/1924101732 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c156450 con 0x7fd54c15bc30 2024-09-16T15:47:32.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.952+0000 7fd543400640 1 -- 172.21.15.60:0/1924101732 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fd538042070 con 0x7fd54c15bc30 2024-09-16T15:47:32.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.954+0000 7fd543400640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd52c0790c0 0x7fd52c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.954+0000 7fd543400640 1 -- 172.21.15.60:0/1924101732 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fd5380bca20 con 0x7fd54c15bc30 2024-09-16T15:47:32.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.954+0000 7fd54b400640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd52c0790c0 0x7fd52c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.954+0000 7fd5550be740 1 Processor -- start 2024-09-16T15:47:32.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.954+0000 7fd5550be740 1 -- start start 2024-09-16T15:47:32.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd54b400640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd52c0790c0 0x7fd52c07b580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0015e0 tx=0x7fd53c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd5550be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff67e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd5550be740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff22c10 0x563eeff66580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd5550be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeff66ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd5550be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563eefea5430 con 0x563eeff67aa0 2024-09-16T15:47:32.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd5550be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563eefe955d0 con 0x563eeff23190 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd5550be740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x563eefe84a00 con 0x563eeff22c10 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd54b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff67e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd54b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff67e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44488/0 (socket says 172.21.15.60:44488) 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.955+0000 7fd54b400640 1 -- 172.21.15.60:0/3320053107 learned_addr learned my addr 172.21.15.60:0/3320053107 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd54be00640 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeff66ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd54aa00640 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff22c10 0x563eeff66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd54b400640 1 -- 172.21.15.60:0/3320053107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff22c10 msgr2=0x563eeff66580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd54b400640 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff22c10 0x563eeff66580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd54b400640 1 -- 172.21.15.60:0/3320053107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 msgr2=0x563eeff66ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd54b400640 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeff66ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd54b400640 1 -- 172.21.15.60:0/3320053107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eefe4d520 con 0x563eeff67aa0 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd54b400640 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff67e80 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd53c005280 tx=0x7fd53c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a0b800bea18161 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd537400640 1 -- 172.21.15.60:0/3320053107 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00e4b0 con 0x563eeff67aa0 2024-09-16T15:47:32.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd537400640 1 -- 172.21.15.60:0/3320053107 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd53c00e740 con 0x563eeff67aa0 2024-09-16T15:47:32.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.956+0000 7fd537400640 1 -- 172.21.15.60:0/3320053107 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00eb30 con 0x563eeff67aa0 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 -- 172.21.15.60:0/3320053107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 msgr2=0x563eeff67e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff67e80 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd53c005280 tx=0x7fd53c011000 comp rx=0 tx=0).stop 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 -- 172.21.15.60:0/3320053107 shutdown_connections 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeff66ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff22c10 0x563eeff66580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 --2- 172.21.15.60:0/3320053107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff67e80 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 -- 172.21.15.60:0/3320053107 >> 172.21.15.60:0/3320053107 conn(0x563eeff1e1e0 msgr2=0x563eeff21c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 -- 172.21.15.60:0/3320053107 shutdown_connections 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.957+0000 7fd5550be740 1 -- 172.21.15.60:0/3320053107 wait complete. 2024-09-16T15:47:32.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.958+0000 7fd5550be740 1 -- start start 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.958+0000 7fd5550be740 1 Processor -- start 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd5550be740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff23190 0x563eeff66960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd5550be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff63fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd5550be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 0x563eeff4aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd5550be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563eefe2b330 con 0x563eeff67aa0 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd5550be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563eefe88810 con 0x563eeff644f0 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd5550be740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x563eefbf5620 con 0x563eeff23190 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd5550be740 10 client.0 _ll_register_callbacks cb 0x563eefe30420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd54aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff63fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd54aa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff63fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44498/0 (socket says 172.21.15.60:44498) 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd54b400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff23190 0x563eeff66960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd54aa00640 1 -- 172.21.15.60:0/708924877 learned_addr learned my addr 172.21.15.60:0/708924877 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:32.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.959+0000 7fd54be00640 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 0x563eeff4aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd54aa00640 1 -- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff23190 msgr2=0x563eeff66960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd54aa00640 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff23190 0x563eeff66960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd54aa00640 1 -- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 msgr2=0x563eeff4aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd54aa00640 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 0x563eeff4aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd54aa00640 1 -- 172.21.15.60:0/708924877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eefbe6850 con 0x563eeff67aa0 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd54aa00640 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff63fb0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd538083640 tx=0x7fd538083ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd061856023a7e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd52be00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd52be00640 1 -- 172.21.15.60:0/708924877 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5380a9020 con 0x563eeff67aa0 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd52be00640 1 -- 172.21.15.60:0/708924877 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fd538082da0 con 0x563eeff67aa0 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd5550be740 1 -- 172.21.15.60:0/708924877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd53c00e740 con 0x563eeff67aa0 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd52be00640 1 -- 172.21.15.60:0/708924877 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fd538049040 con 0x563eeff67aa0 2024-09-16T15:47:32.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd5550be740 10 client.8979 fetch_fsmap learned FSMap version 112 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd5550be740 1 -- 172.21.15.60:0/708924877 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x563eefc5d6b0 con 0x563eeff67aa0 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.960+0000 7fd52be00640 1 -- 172.21.15.60:0/708924877 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd53c00e740 con 0x563eeff67aa0 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.961+0000 7fd52be00640 1 -- 172.21.15.60:0/708924877 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 112) ==== 3502+0+0 (secure 0 0 0) 0x7fd53807f290 con 0x563eeff67aa0 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.961+0000 7fd5550be740 10 client.8979 fetch_fsmap finished waiting for FSMap version 112 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.961+0000 7fd5550be740 10 client.8979 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.961+0000 7fd5550be740 20 client.8979 populate_metadata read hostname 'smithi060' 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.961+0000 7fd5550be740 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] conn(0x563eeff825a0 0x563eeff84990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.961+0000 7fd5550be740 4 client.8979 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.961+0000 7fd5550be740 1 -- 172.21.15.60:0/708924877 --> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eefe97520 con 0x563eeff825a0 2024-09-16T15:47:32.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.961+0000 7fd54b400640 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] conn(0x563eeff825a0 0x563eeff84990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.962+0000 7fd54b400640 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] conn(0x563eeff825a0 0x563eeff84990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e349db067e8b038 server_cookie=d6b463ccdf3a2cfb in_seq=0 out_seq=0 2024-09-16T15:47:32.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.962+0000 7fd52be00640 10 client.8979 ms_handle_connect on v2:172.21.15.100:6801/3325202304 2024-09-16T15:47:32.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd52be00640 1 -- 172.21.15.60:0/708924877 <== mds.0 v2:172.21.15.100:6801/3325202304 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563eefe97520 con 0x563eeff825a0 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd52be00640 10 client.8979 handle_command_reply: tid=0 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 client.8979 shutdown 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 -- 172.21.15.60:0/708924877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 msgr2=0x563eeff63fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff63fb0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd538083640 tx=0x7fd538083ea0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 -- 172.21.15.60:0/708924877 shutdown_connections 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 0x563eeff4aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff63fb0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] conn(0x563eeff825a0 0x563eeff84990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 --2- 172.21.15.60:0/708924877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff23190 0x563eeff66960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 -- 172.21.15.60:0/708924877 >> 172.21.15.60:0/708924877 conn(0x563eeff22c10 msgr2=0x563eeff1cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd52be00640 0 client.8979 ms_handle_reset on v2:172.21.15.100:6801/3325202304 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 -- 172.21.15.60:0/708924877 shutdown_connections 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 1 -- 172.21.15.60:0/708924877 wait complete. 2024-09-16T15:47:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.963+0000 7fd5550be740 20 client.8979 trim_cache size 0 max 16384 2024-09-16T15:47:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.982+0000 7fd5550be740 1 Processor -- start 2024-09-16T15:47:32.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.983+0000 7fd5550be740 1 -- start start 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.983+0000 7fd5550be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff23190 0x563eeff5d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.983+0000 7fd5550be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 0x563eeff65da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.983+0000 7fd5550be740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff67aa0 0x563eeff662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.983+0000 7fd5550be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0ceca0 con 0x563eeff23190 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.983+0000 7fd5550be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c15e8d0 con 0x563eeff644f0 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.983+0000 7fd5550be740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c15ef20 con 0x563eeff67aa0 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff23190 0x563eeff5d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff23190 0x563eeff5d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44508/0 (socket says 172.21.15.60:44508) 2024-09-16T15:47:32.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 -- 172.21.15.60:0/734555327 learned_addr learned my addr 172.21.15.60:0/734555327 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 -- 172.21.15.60:0/734555327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff67aa0 msgr2=0x563eeff662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:47:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 --2- 172.21.15.60:0/734555327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff67aa0 0x563eeff662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 -- 172.21.15.60:0/734555327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 msgr2=0x563eeff65da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:47:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 --2- 172.21.15.60:0/734555327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 0x563eeff65da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 -- 172.21.15.60:0/734555327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eeff39aa0 con 0x563eeff23190 2024-09-16T15:47:32.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd54b400640 1 --2- 172.21.15.60:0/734555327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff23190 0x563eeff5d010 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd53c005040 tx=0x7fd53c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c939e67f4bb2fc9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd536a00640 1 -- 172.21.15.60:0/734555327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00e740 con 0x563eeff23190 2024-09-16T15:47:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd536a00640 1 -- 172.21.15.60:0/734555327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd53c00e9f0 con 0x563eeff23190 2024-09-16T15:47:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.984+0000 7fd536a00640 1 -- 172.21.15.60:0/734555327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00ee40 con 0x563eeff23190 2024-09-16T15:47:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 -- 172.21.15.60:0/734555327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff23190 msgr2=0x563eeff5d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 --2- 172.21.15.60:0/734555327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff23190 0x563eeff5d010 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd53c005040 tx=0x7fd53c012000 comp rx=0 tx=0).stop 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 -- 172.21.15.60:0/734555327 shutdown_connections 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 --2- 172.21.15.60:0/734555327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff67aa0 0x563eeff662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 --2- 172.21.15.60:0/734555327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff644f0 0x563eeff65da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 --2- 172.21.15.60:0/734555327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff23190 0x563eeff5d010 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 -- 172.21.15.60:0/734555327 >> 172.21.15.60:0/734555327 conn(0x563eeff22c10 msgr2=0x563eeff1c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 -- 172.21.15.60:0/734555327 shutdown_connections 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.985+0000 7fd5550be740 1 -- 172.21.15.60:0/734555327 wait complete. 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.986+0000 7fd5550be740 1 -- start start 2024-09-16T15:47:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.986+0000 7fd5550be740 1 Processor -- start 2024-09-16T15:47:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd5550be740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeffe5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd5550be740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff644f0 0x563eeff48d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd5550be740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd5550be740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x563eeff4b9b0 con 0x563eeff67aa0 2024-09-16T15:47:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd5550be740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563eeff66ea0 con 0x563eeff23190 2024-09-16T15:47:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd5550be740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x563eeff39570 con 0x563eeff644f0 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd5550be740 10 client.0 _ll_register_callbacks cb 0x7fd538005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44522/0 (socket says 172.21.15.60:44522) 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54be00640 1 -- 172.21.15.60:0/35868024 learned_addr learned my addr 172.21.15.60:0/35868024 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54aa00640 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff644f0 0x563eeff48d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54b400640 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeffe5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54be00640 1 -- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff644f0 msgr2=0x563eeff48d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54be00640 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff644f0 0x563eeff48d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54be00640 1 -- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 msgr2=0x563eeffe5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54be00640 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeffe5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.987+0000 7fd54be00640 1 -- 172.21.15.60:0/35868024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eeff4bf70 con 0x563eeff67aa0 2024-09-16T15:47:32.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd54b400640 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeffe5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd54be00640 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff492a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd544011a60 tx=0x7fd544011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595b5a5d93a759c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd52be00640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd52be00640 1 -- 172.21.15.60:0/35868024 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544010c40 con 0x563eeff67aa0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd52be00640 1 -- 172.21.15.60:0/35868024 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fd544010de0 con 0x563eeff67aa0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd5550be740 1 -- 172.21.15.60:0/35868024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd53c00e9f0 con 0x563eeff67aa0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd52be00640 1 -- 172.21.15.60:0/35868024 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fd544023050 con 0x563eeff67aa0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd5550be740 10 client.8985 fetch_fsmap learned FSMap version 112 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.988+0000 7fd5550be740 1 -- 172.21.15.60:0/35868024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x563eeff64010 con 0x563eeff67aa0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd52be00640 1 -- 172.21.15.60:0/35868024 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd53c00e9f0 con 0x563eeff67aa0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd52be00640 1 -- 172.21.15.60:0/35868024 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 112) ==== 3502+0+0 (secure 0 0 0) 0x7fd544026020 con 0x563eeff67aa0 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd5550be740 10 client.8985 fetch_fsmap finished waiting for FSMap version 112 2024-09-16T15:47:32.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd5550be740 10 client.8985 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T15:47:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd5550be740 20 client.8985 populate_metadata read hostname 'smithi060' 2024-09-16T15:47:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd5550be740 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] conn(0x563eeffe8c80 0x563eeff35600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd5550be740 4 client.8985 mds_command: new command op to 8853 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:47:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd5550be740 1 -- 172.21.15.60:0/35868024 --> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] -- command(tid 0: {"prefix": "status"}) -- 0x563eefe92fd0 con 0x563eeffe8c80 2024-09-16T15:47:32.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.989+0000 7fd54b400640 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] conn(0x563eeffe8c80 0x563eeff35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd54b400640 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] conn(0x563eeffe8c80 0x563eeff35600 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995d43460e64ad17 server_cookie=ef5317aeb326eb90 in_seq=0 out_seq=0 2024-09-16T15:47:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd52be00640 10 client.8985 ms_handle_connect on v2:172.21.15.100:6801/3325202304 2024-09-16T15:47:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd52be00640 1 -- 172.21.15.60:0/35868024 <== mds.0 v2:172.21.15.100:6801/3325202304 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x563eefe92fd0 con 0x563eeffe8c80 2024-09-16T15:47:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd52be00640 10 client.8985 handle_command_reply: tid=0 2024-09-16T15:47:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd5550be740 1 client.8985 shutdown 2024-09-16T15:47:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd5550be740 1 -- 172.21.15.60:0/35868024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 msgr2=0x563eeff492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd5550be740 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff492a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd544011a60 tx=0x7fd544011f20 comp rx=0 tx=0).stop 2024-09-16T15:47:32.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd5550be740 1 -- 172.21.15.60:0/35868024 shutdown_connections 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd5550be740 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563eeff67aa0 0x563eeff492a0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd5550be740 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:6801/3325202304,v1:172.21.15.100:6835/3325202304] conn(0x563eeffe8c80 0x563eeff35600 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd5550be740 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563eeff644f0 0x563eeff48d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.990+0000 7fd5550be740 1 --2- 172.21.15.60:0/35868024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563eeff23190 0x563eeffe5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd5550be740 1 -- 172.21.15.60:0/35868024 >> 172.21.15.60:0/35868024 conn(0x563eeff22c10 msgr2=0x563eeff65d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd52be00640 0 client.8985 ms_handle_reset on v2:172.21.15.100:6801/3325202304 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd5550be740 1 -- 172.21.15.60:0/35868024 shutdown_connections 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd5550be740 1 -- 172.21.15.60:0/35868024 wait complete. 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd5550be740 20 client.8985 trim_cache size 0 max 16384 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:47:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8853, 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2633729489999999, 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 112, 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 161, 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 161, 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.546482674 2024-09-16T15:47:32.996 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:47:32.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 -- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd52c0790c0 msgr2=0x7fd52c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd52c0790c0 0x7fd52c07b580 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0015e0 tx=0x7fd53c009040 comp rx=0 tx=0).stop 2024-09-16T15:47:32.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 -- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c15bc30 msgr2=0x7fd54c14bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c15bc30 0x7fd54c14bba0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd538009850 tx=0x7fd5380038a0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 -- 172.21.15.60:0/1924101732 shutdown_connections 2024-09-16T15:47:32.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd52c0790c0 0x7fd52c07b580 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd54c14dc90 0x7fd54c14c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd54c15bc30 0x7fd54c14bba0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 --2- 172.21.15.60:0/1924101732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd54c0d0540 0x7fd54c0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.991+0000 7fd550c00640 1 -- 172.21.15.60:0/1924101732 >> 172.21.15.60:0/1924101732 conn(0x7fd54c0cfa20 msgr2=0x7fd54c0b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.992+0000 7fd550c00640 1 -- 172.21.15.60:0/1924101732 shutdown_connections 2024-09-16T15:47:32.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:32.992+0000 7fd550c00640 1 -- 172.21.15.60:0/1924101732 wait complete. 2024-09-16T15:47:33.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:47:33.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.168+0000 7f528fa00640 1 -- 172.21.15.60:0/3354177404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c003a00 msgr2=0x7f526c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:33.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.168+0000 7f528fa00640 1 --2- 172.21.15.60:0/3354177404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f526c003a00 0x7f526c003e20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f5278009b50 tx=0x7f526c005830 comp rx=0 tx=0).stop 2024-09-16T15:47:33.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.168+0000 7f528da00640 1 -- 172.21.15.60:0/3354177404 reap_dead start 2024-09-16T15:47:33.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.169+0000 7f528fa00640 1 -- 172.21.15.60:0/3354177404 shutdown_connections 2024-09-16T15:47:33.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.169+0000 7f528fa00640 1 -- 172.21.15.60:0/3354177404 >> 172.21.15.60:0/3354177404 conn(0x7f52880b1950 msgr2=0x7f52880b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:33.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.169+0000 7f528fa00640 1 -- 172.21.15.60:0/3354177404 shutdown_connections 2024-09-16T15:47:33.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.169+0000 7f528fa00640 1 -- 172.21.15.60:0/3354177404 wait complete. 2024-09-16T15:47:33.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.171+0000 7f528fa00640 1 Processor -- start 2024-09-16T15:47:33.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.171+0000 7f528fa00640 1 -- start start 2024-09-16T15:47:33.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528fa00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c003a00 0x7f52880cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528fa00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5288158570 0x7f528815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528fa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52881572d0 0x7f5288157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528fa00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f52880745c0 con 0x7f52881572d0 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528fa00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5288074440 con 0x7f526c003a00 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528fa00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5288074740 con 0x7f5288158570 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528da00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52881572d0 0x7f5288157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c003a00 0x7f52880cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528da00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52881572d0 0x7f5288157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44542/0 (socket says 172.21.15.60:44542) 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f527fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5288158570 0x7f528815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528da00640 1 -- 172.21.15.60:0/1630977068 learned_addr learned my addr 172.21.15.60:0/1630977068 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528da00640 1 -- 172.21.15.60:0/1630977068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5288158570 msgr2=0x7f528815a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:33.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528da00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5288158570 0x7f528815a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528da00640 1 -- 172.21.15.60:0/1630977068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c003a00 msgr2=0x7f52880cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528da00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c003a00 0x7f52880cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.172+0000 7f528da00640 1 -- 172.21.15.60:0/1630977068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528815b390 con 0x7f52881572d0 2024-09-16T15:47:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.173+0000 7f528da00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52881572d0 0x7f5288157760 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f528806e130 tx=0x7f5280010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.173+0000 7f5285600640 1 -- 172.21.15.60:0/1630977068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280003970 con 0x7f52881572d0 2024-09-16T15:47:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.173+0000 7f5285600640 1 -- 172.21.15.60:0/1630977068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5280015070 con 0x7f52881572d0 2024-09-16T15:47:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.173+0000 7f5285600640 1 -- 172.21.15.60:0/1630977068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280004030 con 0x7f52881572d0 2024-09-16T15:47:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.173+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528815b670 con 0x7f52881572d0 2024-09-16T15:47:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.173+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52880cde60 con 0x7f52881572d0 2024-09-16T15:47:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.174+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52880b2440 con 0x7f52881572d0 2024-09-16T15:47:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.175+0000 7f5285600640 1 -- 172.21.15.60:0/1630977068 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f52800041d0 con 0x7f52881572d0 2024-09-16T15:47:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.176+0000 7f5285600640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f52640790f0 0x7f526407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.176+0000 7f5285600640 1 -- 172.21.15.60:0/1630977068 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f528009c690 con 0x7f52881572d0 2024-09-16T15:47:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.177+0000 7f528d000640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f52640790f0 0x7f526407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.177+0000 7f528d000640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f52640790f0 0x7f526407b5b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5278035000 tx=0x7f527802d040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.181+0000 7f5285600640 1 -- 172.21.15.60:0/1630977068 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f52800636a0 con 0x7f52881572d0 2024-09-16T15:47:33.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.415+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52880aea60 con 0x7f52881572d0 2024-09-16T15:47:33.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.415+0000 7f5285600640 1 -- 172.21.15.60:0/1630977068 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) ==== 53+0+73 (secure 0 0 0) 0x7f52800680d0 con 0x7f52881572d0 2024-09-16T15:47:33.416 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:47:33.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f52640790f0 msgr2=0x7f526407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:33.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f52640790f0 0x7f526407b5b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5278035000 tx=0x7f527802d040 comp rx=0 tx=0).stop 2024-09-16T15:47:33.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52881572d0 msgr2=0x7f5288157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52881572d0 0x7f5288157760 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f528806e130 tx=0x7f5280010f40 comp rx=0 tx=0).stop 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 shutdown_connections 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f52640790f0 0x7f526407b5b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52881572d0 0x7f5288157760 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5288158570 0x7f528815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 --2- 172.21.15.60:0/1630977068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f526c003a00 0x7f52880cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 >> 172.21.15.60:0/1630977068 conn(0x7f52880b1950 msgr2=0x7f528809cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 shutdown_connections 2024-09-16T15:47:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33.418+0000 7f528fa00640 1 -- 172.21.15.60:0/1630977068 wait complete. 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:47:33.435 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:47:33.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:33.436 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:47:33.471 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:47:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:47:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:47:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42196sec preferred_lft 42196sec 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:47:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:47:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:33.474 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:47:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:47:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:47:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:33.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:33.474 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:47:33.565 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:47:33.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:33.568 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:47:33.604 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:47:33.605 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:47:33.605 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:47:33.605 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:47:33.605 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:47:33.605 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:47:33.605 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:47:33.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:47:33.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:47:33.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:47:33.605 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:47:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:47:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:47:33.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:33.805 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:47:33.840 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:47:33.840 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:47:33.841 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:47:33.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:33.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:47:33.936 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:47:33.937 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:33.971 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:47:33.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:47:33.972 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:34.046 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:47:34.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:34.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:47:34.174 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:47:34.175 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:47:34.203 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:47:34.203 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:47:34.203 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:47:34.203 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:47:34.203 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:47:34.206 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:47:34.206 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:47:34.206 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:47:34.206 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:47:34.206 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:47:34.209 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:47:34.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:34.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:34.265 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:47:34.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:47:34.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:47:34.369 INFO:teuthology.orchestra.run.smithi060.stdout:client7460 2024-09-16T15:47:34.370 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:34.370 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:34.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:47:34.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.607+0000 7f0d29800640 1 -- 172.21.15.60:0/2045895295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d08004d40 msgr2=0x7f0d080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.607+0000 7f0d29800640 1 --2- 172.21.15.60:0/2045895295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d08004d40 0x7f0d080051c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c005fd0 tx=0x7f0d08005830 comp rx=0 tx=0).stop 2024-09-16T15:47:34.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.607+0000 7f0d28e00640 1 -- 172.21.15.60:0/2045895295 reap_dead start 2024-09-16T15:47:34.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.607+0000 7f0d29800640 1 -- 172.21.15.60:0/2045895295 shutdown_connections 2024-09-16T15:47:34.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.607+0000 7f0d29800640 1 -- 172.21.15.60:0/2045895295 >> 172.21.15.60:0/2045895295 conn(0x7f0d240cfa20 msgr2=0x7f0d240cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:34.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.607+0000 7f0d29800640 1 -- 172.21.15.60:0/2045895295 shutdown_connections 2024-09-16T15:47:34.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.607+0000 7f0d29800640 1 -- 172.21.15.60:0/2045895295 wait complete. 2024-09-16T15:47:34.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.609+0000 7f0d29800640 1 Processor -- start 2024-09-16T15:47:34.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.609+0000 7f0d29800640 1 -- start start 2024-09-16T15:47:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d29800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d08004d40 0x7f0d241f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d29800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d24158170 0x7f0d24156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d29800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d24157470 0x7f0d240c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d29800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2415e670 con 0x7f0d24157470 2024-09-16T15:47:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d29800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240ae020 con 0x7f0d08004d40 2024-09-16T15:47:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d29800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2415e7f0 con 0x7f0d24158170 2024-09-16T15:47:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d28e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d24157470 0x7f0d240c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d28e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d24157470 0x7f0d240c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44570/0 (socket says 172.21.15.60:44570) 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d28e00640 1 -- 172.21.15.60:0/524241586 learned_addr learned my addr 172.21.15.60:0/524241586 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d23e00640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d08004d40 0x7f0d241f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d23400640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d24158170 0x7f0d24156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d28e00640 1 -- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d24158170 msgr2=0x7f0d24156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d28e00640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d24158170 0x7f0d24156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d28e00640 1 -- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d08004d40 msgr2=0x7f0d241f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d28e00640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d08004d40 0x7f0d241f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d28e00640 1 -- 172.21.15.60:0/524241586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d241f7d30 con 0x7f0d24157470 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.610+0000 7f0d23400640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d24158170 0x7f0d24156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.611+0000 7f0d28e00640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d24157470 0x7f0d240c5bd0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c02aea0 tx=0x7f0d1c033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.611+0000 7f0d1be00640 1 -- 172.21.15.60:0/524241586 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c042070 con 0x7f0d24157470 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.611+0000 7f0d1be00640 1 -- 172.21.15.60:0/524241586 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d1c003970 con 0x7f0d24157470 2024-09-16T15:47:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.611+0000 7f0d1be00640 1 -- 172.21.15.60:0/524241586 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c038040 con 0x7f0d24157470 2024-09-16T15:47:34.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.611+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d240c36f0 con 0x7f0d24157470 2024-09-16T15:47:34.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.611+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d240c3cb0 con 0x7f0d24157470 2024-09-16T15:47:34.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.612+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2415ed80 con 0x7f0d24157470 2024-09-16T15:47:34.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.614+0000 7f0d1be00640 1 -- 172.21.15.60:0/524241586 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f0d1c004030 con 0x7f0d24157470 2024-09-16T15:47:34.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.614+0000 7f0d1be00640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d040790f0 0x7f0d0407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:34.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.615+0000 7f0d1be00640 1 -- 172.21.15.60:0/524241586 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f0d1c0b9200 con 0x7f0d24157470 2024-09-16T15:47:34.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.615+0000 7f0d23e00640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d040790f0 0x7f0d0407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:34.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.615+0000 7f0d23e00640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d040790f0 0x7f0d0407b5b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14006b70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:34.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.619+0000 7f0d1be00640 1 -- 172.21.15.60:0/524241586 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0d241f7d30 con 0x7f0d24157470 2024-09-16T15:47:34.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.859+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0d240afc40 con 0x7f0d24157470 2024-09-16T15:47:34.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.859+0000 7f0d1be00640 1 -- 172.21.15.60:0/524241586 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) ==== 53+0+73 (secure 0 0 0) 0x7f0d1c082c40 con 0x7f0d24157470 2024-09-16T15:47:34.860 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:47:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d040790f0 msgr2=0x7f0d0407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d040790f0 0x7f0d0407b5b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0d14000c00 tx=0x7f0d14006b70 comp rx=0 tx=0).stop 2024-09-16T15:47:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d24157470 msgr2=0x7f0d240c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d24157470 0x7f0d240c5bd0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c02aea0 tx=0x7f0d1c033c10 comp rx=0 tx=0).stop 2024-09-16T15:47:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 shutdown_connections 2024-09-16T15:47:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0d040790f0 0x7f0d0407b5b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d24157470 0x7f0d240c5bd0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d24158170 0x7f0d24156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 --2- 172.21.15.60:0/524241586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d08004d40 0x7f0d241f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 >> 172.21.15.60:0/524241586 conn(0x7f0d240cfa20 msgr2=0x7f0d241618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 shutdown_connections 2024-09-16T15:47:34.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:34.862+0000 7f0d29800640 1 -- 172.21.15.60:0/524241586 wait complete. 2024-09-16T15:47:34.878 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:47:34.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:47:34.879 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:47:34.879 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:47:34.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:47:34.879 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:47:34.879 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:47:34.879 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:47:34.879 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:47:34.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:34.879 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:47:34.919 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:47:34.919 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:47:34.920 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42194sec preferred_lft 42194sec 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:34.921 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:47:34.922 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:47:34.922 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:47:34.922 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:47:34.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:34.923 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:47:35.020 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:47:35.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:35.024 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:47:35.061 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:47:35.061 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:47:35.061 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:47:35.061 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:47:35.061 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:47:35.061 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:47:35.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:47:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:47:35.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:47:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:47:35.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:35.257 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:47:35.292 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:47:35.292 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:47:35.292 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:47:35.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:35.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:47:35.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:47:35.388 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:47:35.423 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:47:35.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:47:35.424 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:47:35.498 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:47:35.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:35.498 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:47:35.625 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:47:35.626 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:47:35.652 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:47:35.652 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:47:35.652 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:47:35.652 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:47:35.652 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:47:35.654 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:47:35.654 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:47:35.654 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:47:35.654 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:47:35.654 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:47:35.657 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:47:35.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:35.657 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:47:35.712 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:47:35.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:47:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:47:35.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:47:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:47:35.802 INFO:teuthology.orchestra.run.smithi100.stdout:client7469 2024-09-16T15:47:35.803 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:35.803 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:35.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:47:36.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.012+0000 7f1c2de00640 1 -- 172.21.15.60:0/2711039327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815aba0 msgr2=0x7f1c2815af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.012+0000 7f1c2de00640 1 --2- 172.21.15.60:0/2711039327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815aba0 0x7f1c2815af80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005fd0 tx=0x7f1c1802a900 comp rx=0 tx=0).stop 2024-09-16T15:47:36.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.012+0000 7f1c2de00640 1 -- 172.21.15.60:0/2711039327 shutdown_connections 2024-09-16T15:47:36.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.012+0000 7f1c2de00640 1 --2- 172.21.15.60:0/2711039327 >> v2:172.21.15.100:3300/0 conn(0x7f1c28073780 0x7f1c2806f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.012+0000 7f1c2de00640 1 --2- 172.21.15.60:0/2711039327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c2815aba0 0x7f1c2815af80 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.012+0000 7f1c2de00640 1 -- 172.21.15.60:0/2711039327 >> 172.21.15.60:0/2711039327 conn(0x7f1c2806cb70 msgr2=0x7f1c2806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:36.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.012+0000 7f1c2de00640 1 -- 172.21.15.60:0/2711039327 shutdown_connections 2024-09-16T15:47:36.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.012+0000 7f1c2de00640 1 -- 172.21.15.60:0/2711039327 wait complete. 2024-09-16T15:47:36.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.014+0000 7f1c2de00640 1 Processor -- start 2024-09-16T15:47:36.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.014+0000 7f1c2de00640 1 -- start start 2024-09-16T15:47:36.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c28073780 0x7f1c2814c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c28073fd0 0x7f1c2814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815aba0 0x7f1c280d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c2de00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2806f960 con 0x7f1c28073780 2024-09-16T15:47:36.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c2de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2806f7e0 con 0x7f1c28073fd0 2024-09-16T15:47:36.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c2de00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2806fae0 con 0x7f1c2815aba0 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c26a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c28073780 0x7f1c2814c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c26a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c28073780 0x7f1c2814c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44594/0 (socket says 172.21.15.60:44594) 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c26a00640 1 -- 172.21.15.60:0/1824464537 learned_addr learned my addr 172.21.15.60:0/1824464537 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c1e000640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c28073fd0 0x7f1c2814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c27400640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815aba0 0x7f1c280d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c26a00640 1 -- 172.21.15.60:0/1824464537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815aba0 msgr2=0x7f1c280d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c26a00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815aba0 0x7f1c280d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c26a00640 1 -- 172.21.15.60:0/1824464537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c28073fd0 msgr2=0x7f1c2814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c26a00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c28073fd0 0x7f1c2814cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.015+0000 7f1c26a00640 1 -- 172.21.15.60:0/1824464537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c280caf60 con 0x7f1c28073780 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.016+0000 7f1c26a00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c28073780 0x7f1c2814c620 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1c18000f80 tx=0x7f1c18003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.016+0000 7f1c1fe00640 1 -- 172.21.15.60:0/1824464537 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18031db0 con 0x7f1c28073780 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.016+0000 7f1c1fe00640 1 -- 172.21.15.60:0/1824464537 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c180330d0 con 0x7f1c28073780 2024-09-16T15:47:36.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.016+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c280cb240 con 0x7f1c28073780 2024-09-16T15:47:36.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.016+0000 7f1c1fe00640 1 -- 172.21.15.60:0/1824464537 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18033520 con 0x7f1c28073780 2024-09-16T15:47:36.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.016+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c280cb800 con 0x7f1c28073780 2024-09-16T15:47:36.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.017+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2806ff90 con 0x7f1c28073780 2024-09-16T15:47:36.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.018+0000 7f1c1fe00640 1 -- 172.21.15.60:0/1824464537 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f1c180336c0 con 0x7f1c28073780 2024-09-16T15:47:36.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.019+0000 7f1c1fe00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1c040790c0 0x7f1c0407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.019+0000 7f1c1fe00640 1 -- 172.21.15.60:0/1824464537 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f1c180bd0a0 con 0x7f1c28073780 2024-09-16T15:47:36.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.020+0000 7f1c1e000640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1c040790c0 0x7f1c0407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.020+0000 7f1c1e000640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1c040790c0 0x7f1c0407b580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c100017c0 tx=0x7f1c10007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:36.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.024+0000 7f1c1fe00640 1 -- 172.21.15.60:0/1824464537 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1c1803f030 con 0x7f1c28073780 2024-09-16T15:47:36.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.260+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f1c28075080 con 0x7f1c28073780 2024-09-16T15:47:36.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.762+0000 7f1c1fe00640 1 -- 172.21.15.60:0/1824464537 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v113) ==== 136+0+0 (secure 0 0 0) 0x7f1c180317c0 con 0x7f1c28073780 2024-09-16T15:47:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1c040790c0 msgr2=0x7f1c0407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1c040790c0 0x7f1c0407b580 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c100017c0 tx=0x7f1c10007e90 comp rx=0 tx=0).stop 2024-09-16T15:47:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c28073780 msgr2=0x7f1c2814c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c28073780 0x7f1c2814c620 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1c18000f80 tx=0x7f1c18003c20 comp rx=0 tx=0).stop 2024-09-16T15:47:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 shutdown_connections 2024-09-16T15:47:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1c040790c0 0x7f1c0407b580 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c2815aba0 0x7f1c280d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c28073fd0 0x7f1c2814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 --2- 172.21.15.60:0/1824464537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c28073780 0x7f1c2814c620 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 >> 172.21.15.60:0/1824464537 conn(0x7f1c2806cb70 msgr2=0x7f1c280aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 shutdown_connections 2024-09-16T15:47:36.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.765+0000 7f1c2de00640 1 -- 172.21.15.60:0/1824464537 wait complete. 2024-09-16T15:47:36.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:47:36.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.950+0000 7fa037e00640 1 -- 172.21.15.60:0/2667328483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa01c004000 msgr2=0x7fa01c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.950+0000 7fa037e00640 1 --2- 172.21.15.60:0/2667328483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa01c004000 0x7fa01c004800 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa020009b50 tx=0x7fa01c005830 comp rx=0 tx=0).stop 2024-09-16T15:47:36.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.950+0000 7fa037400640 1 -- 172.21.15.60:0/2667328483 reap_dead start 2024-09-16T15:47:36.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.950+0000 7fa037e00640 1 -- 172.21.15.60:0/2667328483 shutdown_connections 2024-09-16T15:47:36.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.950+0000 7fa037e00640 1 -- 172.21.15.60:0/2667328483 >> 172.21.15.60:0/2667328483 conn(0x7fa0380d3df0 msgr2=0x7fa0380d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:36.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.950+0000 7fa037e00640 1 -- 172.21.15.60:0/2667328483 shutdown_connections 2024-09-16T15:47:36.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.950+0000 7fa037e00640 1 -- 172.21.15.60:0/2667328483 wait complete. 2024-09-16T15:47:36.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.952+0000 7fa037e00640 1 Processor -- start 2024-09-16T15:47:36.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.952+0000 7fa037e00640 1 -- start start 2024-09-16T15:47:36.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa037e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa01c004000 0x7fa0380b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa037e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0380ca090 0x7fa0381f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa037e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0380ca510 0x7fa0381f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa037e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa0380b3f50 con 0x7fa0380ca090 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa037e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0380b3dd0 con 0x7fa0380ca510 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa037e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa0380b1e80 con 0x7fa01c004000 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa02e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0380ca090 0x7fa0381f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa02e000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0380ca090 0x7fa0381f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44612/0 (socket says 172.21.15.60:44612) 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa02e000640 1 -- 172.21.15.60:0/2617202107 learned_addr learned my addr 172.21.15.60:0/2617202107 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa037400640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0380ca510 0x7fa0381f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.953+0000 7fa036a00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa01c004000 0x7fa0380b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02e000640 1 -- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa01c004000 msgr2=0x7fa0380b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02e000640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa01c004000 0x7fa0380b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02e000640 1 -- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0380ca510 msgr2=0x7fa0381f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02e000640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0380ca510 0x7fa0381f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02e000640 1 -- 172.21.15.60:0/2617202107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0381f5700 con 0x7fa0380ca090 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa037400640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0380ca510 0x7fa0381f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa036a00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa01c004000 0x7fa0380b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02e000640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0380ca090 0x7fa0381f47a0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa020009a70 tx=0x7fa020033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02f400640 1 -- 172.21.15.60:0/2617202107 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020038040 con 0x7fa0380ca090 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02f400640 1 -- 172.21.15.60:0/2617202107 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa020033950 con 0x7fa0380ca090 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0381f5420 con 0x7fa0380ca090 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa02f400640 1 -- 172.21.15.60:0/2617202107 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020033da0 con 0x7fa0380ca090 2024-09-16T15:47:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.954+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0380b86a0 con 0x7fa0380ca090 2024-09-16T15:47:36.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.955+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0380b1770 con 0x7fa0380ca090 2024-09-16T15:47:36.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.957+0000 7fa02f400640 1 -- 172.21.15.60:0/2617202107 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7fa020043020 con 0x7fa0380ca090 2024-09-16T15:47:36.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.958+0000 7fa02f400640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa0140790c0 0x7fa01407b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.958+0000 7fa02f400640 1 -- 172.21.15.60:0/2617202107 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fa0200bbc10 con 0x7fa0380ca090 2024-09-16T15:47:36.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.958+0000 7fa036a00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa0140790c0 0x7fa01407b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.959+0000 7fa036a00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa0140790c0 0x7fa01407b580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa028000c00 tx=0x7fa028006c20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:36.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:36.962+0000 7fa02f400640 1 -- 172.21.15.60:0/2617202107 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa02004a020 con 0x7fa0380ca090 2024-09-16T15:47:37.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.184+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa03815d400 con 0x7fa0140790c0 2024-09-16T15:47:37.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.185+0000 7fa02f400640 1 -- 172.21.15.60:0/2617202107 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa03815d400 con 0x7fa0140790c0 2024-09-16T15:47:37.187 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:47:37.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:47:37.188 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:47:37.188 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:47:37.188 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.187+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa0140790c0 msgr2=0x7fa01407b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.187+0000 7fa037e00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa0140790c0 0x7fa01407b580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa028000c00 tx=0x7fa028006c20 comp rx=0 tx=0).stop 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0380ca090 msgr2=0x7fa0381f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0380ca090 0x7fa0381f47a0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa020009a70 tx=0x7fa020033670 comp rx=0 tx=0).stop 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 shutdown_connections 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa0140790c0 0x7fa01407b580 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0380ca510 0x7fa0381f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0380ca090 0x7fa0381f47a0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 --2- 172.21.15.60:0/2617202107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa01c004000 0x7fa0380b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 >> 172.21.15.60:0/2617202107 conn(0x7fa0380d3df0 msgr2=0x7fa03815ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:37.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 shutdown_connections 2024-09-16T15:47:37.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.188+0000 7fa037e00640 1 -- 172.21.15.60:0/2617202107 wait complete. 2024-09-16T15:47:37.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:47:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.368+0000 7f2cfb000640 1 -- 172.21.15.60:0/609962277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf406efa0 msgr2=0x7f2cf4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.368+0000 7f2cfb000640 1 --2- 172.21.15.60:0/609962277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf406efa0 0x7f2cf4073440 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005fd0 tx=0x7f2cec02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.368+0000 7f2cfb000640 1 -- 172.21.15.60:0/609962277 shutdown_connections 2024-09-16T15:47:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.368+0000 7f2cfb000640 1 --2- 172.21.15.60:0/609962277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf406efa0 0x7f2cf4073440 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.368+0000 7f2cfb000640 1 -- 172.21.15.60:0/609962277 >> 172.21.15.60:0/609962277 conn(0x7f2cf40b1840 msgr2=0x7f2cf40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.368+0000 7f2cfb000640 1 -- 172.21.15.60:0/609962277 shutdown_connections 2024-09-16T15:47:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.368+0000 7f2cfb000640 1 -- 172.21.15.60:0/609962277 wait complete. 2024-09-16T15:47:37.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.369+0000 7f2cfb000640 1 Processor -- start 2024-09-16T15:47:37.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.369+0000 7f2cfb000640 1 -- start start 2024-09-16T15:47:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.370+0000 7f2cfb000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cf406efa0 0x7f2cf41e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.370+0000 7f2cfb000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf40cccd0 0x7f2cf4153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.370+0000 7f2cfb000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf4145750 0x7f2cf4144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.370+0000 7f2cfb000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf40ce6a0 con 0x7f2cf40cccd0 2024-09-16T15:47:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.370+0000 7f2cfb000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf40ce520 con 0x7f2cf4145750 2024-09-16T15:47:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.370+0000 7f2cfb000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf40ce820 con 0x7f2cf406efa0 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf40cccd0 0x7f2cf4153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf40cccd0 0x7f2cf4153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44622/0 (socket says 172.21.15.60:44622) 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 -- 172.21.15.60:0/131800729 learned_addr learned my addr 172.21.15.60:0/131800729 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf9000640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf4145750 0x7f2cf4144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3e00640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cf406efa0 0x7f2cf41e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 -- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cf406efa0 msgr2=0x7f2cf41e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cf406efa0 0x7f2cf41e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 -- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf4145750 msgr2=0x7f2cf4144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf4145750 0x7f2cf4144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 -- 172.21.15.60:0/131800729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf41e58d0 con 0x7f2cf40cccd0 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf9000640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf4145750 0x7f2cf4144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3e00640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cf406efa0 0x7f2cf41e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:37.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.371+0000 7f2cf3400640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf40cccd0 0x7f2cf4153640 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c490 tx=0x7f2ce000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.372+0000 7f2cebe00640 1 -- 172.21.15.60:0/131800729 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0016020 con 0x7f2cf40cccd0 2024-09-16T15:47:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.372+0000 7f2cebe00640 1 -- 172.21.15.60:0/131800729 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ce0012940 con 0x7f2cf40cccd0 2024-09-16T15:47:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.372+0000 7f2cebe00640 1 -- 172.21.15.60:0/131800729 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0012d90 con 0x7f2cf40cccd0 2024-09-16T15:47:37.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.372+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf41e5bb0 con 0x7f2cf40cccd0 2024-09-16T15:47:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.372+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf41e6170 con 0x7f2cf40cccd0 2024-09-16T15:47:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.373+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf40cecd0 con 0x7f2cf40cccd0 2024-09-16T15:47:37.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.374+0000 7f2cebe00640 1 -- 172.21.15.60:0/131800729 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f2ce0019020 con 0x7f2cf40cccd0 2024-09-16T15:47:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.375+0000 7f2cebe00640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2cd40790f0 0x7f2cd407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.375+0000 7f2cebe00640 1 -- 172.21.15.60:0/131800729 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f2ce009b5c0 con 0x7f2cf40cccd0 2024-09-16T15:47:37.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.375+0000 7f2cf3e00640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2cd40790f0 0x7f2cd407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.376+0000 7f2cf3e00640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2cd40790f0 0x7f2cd407b5b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4007fd0 tx=0x7f2ce4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:37.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.379+0000 7f2cebe00640 1 -- 172.21.15.60:0/131800729 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ce0020070 con 0x7f2cf40cccd0 2024-09-16T15:47:37.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.574+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2cf4069a10 con 0x7f2cf40cccd0 2024-09-16T15:47:37.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.575+0000 7f2cebe00640 1 -- 172.21.15.60:0/131800729 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f2ce0062c70 con 0x7f2cf40cccd0 2024-09-16T15:47:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2cd40790f0 msgr2=0x7f2cd407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2cd40790f0 0x7f2cd407b5b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4007fd0 tx=0x7f2ce4009040 comp rx=0 tx=0).stop 2024-09-16T15:47:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf40cccd0 msgr2=0x7f2cf4153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf40cccd0 0x7f2cf4153640 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c490 tx=0x7f2ce000c950 comp rx=0 tx=0).stop 2024-09-16T15:47:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 shutdown_connections 2024-09-16T15:47:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2cd40790f0 0x7f2cd407b5b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cf4145750 0x7f2cf4144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2cf40cccd0 0x7f2cf4153640 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 --2- 172.21.15.60:0/131800729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cf406efa0 0x7f2cf41e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 >> 172.21.15.60:0/131800729 conn(0x7f2cf40b1840 msgr2=0x7f2cf40b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 shutdown_connections 2024-09-16T15:47:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:37.577+0000 7f2cfb000640 1 -- 172.21.15.60:0/131800729 wait complete. 2024-09-16T15:47:37.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:37.596 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol14660/3eeb1382-6b4c-4a0e-b6fd-073a5a849dc4) 2024-09-16T15:47:37.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:37.662 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:47:37.699 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:47:37.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:37.700 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol14660/3eeb1382-6b4c-4a0e-b6fd-073a5a849dc4) 2024-09-16T15:47:37.803 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:37.803 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol14660/.meta 2024-09-16T15:47:37.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:37.863 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp25450/subvol27801/ea75e1c2-c266-4f55-91c8-3c8dab747146) 2024-09-16T15:47:37.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:37.963 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:47:37.999 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:47:37.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:37.999 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp25450/subvol27801/ea75e1c2-c266-4f55-91c8-3c8dab747146) 2024-09-16T15:47:38.102 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:38.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp25450/subvol27801/.meta 2024-09-16T15:47:38.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol14660 2024-09-16T15:47:38.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.360+0000 7f74a5200640 1 -- 172.21.15.60:0/1453755410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 msgr2=0x7f74a00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.360+0000 7f74a5200640 1 --2- 172.21.15.60:0/1453755410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 0x7f74a00cdbb0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f74a00af780 tx=0x7f749802a980 comp rx=0 tx=0).stop 2024-09-16T15:47:38.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.360+0000 7f74a5200640 1 -- 172.21.15.60:0/1453755410 shutdown_connections 2024-09-16T15:47:38.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.360+0000 7f74a5200640 1 --2- 172.21.15.60:0/1453755410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 0x7f74a00cdbb0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.360+0000 7f74a5200640 1 --2- 172.21.15.60:0/1453755410 >> v2:172.21.15.100:3300/0 conn(0x7f74a00cccd0 0x7f74a0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.360+0000 7f74a5200640 1 -- 172.21.15.60:0/1453755410 >> 172.21.15.60:0/1453755410 conn(0x7f74a00b1840 msgr2=0x7f74a00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:38.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.360+0000 7f74a5200640 1 -- 172.21.15.60:0/1453755410 shutdown_connections 2024-09-16T15:47:38.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.360+0000 7f74a5200640 1 -- 172.21.15.60:0/1453755410 wait complete. 2024-09-16T15:47:38.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.362+0000 7f74a5200640 1 Processor -- start 2024-09-16T15:47:38.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.362+0000 7f74a5200640 1 -- start start 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f74a5200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 0x7f74a00ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f74a5200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a00cccd0 0x7f74a00ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f74a5200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a0145750 0x7f74a01f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f74a5200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f74a00ce480 con 0x7f74a006efa0 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f74a5200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74a00ce300 con 0x7f74a0145750 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f74a5200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f74a00ce600 con 0x7f74a00cccd0 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f749de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 0x7f74a00ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f749de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 0x7f74a00ab020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44630/0 (socket says 172.21.15.60:44630) 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f749de00640 1 -- 172.21.15.60:0/3504140986 learned_addr learned my addr 172.21.15.60:0/3504140986 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f749e800640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a0145750 0x7f74a01f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.363+0000 7f7495400640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a00cccd0 0x7f74a00ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f749de00640 1 -- 172.21.15.60:0/3504140986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a00cccd0 msgr2=0x7f74a00ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f749de00640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a00cccd0 0x7f74a00ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f749de00640 1 -- 172.21.15.60:0/3504140986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a0145750 msgr2=0x7f74a01f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f749de00640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a0145750 0x7f74a01f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f749de00640 1 -- 172.21.15.60:0/3504140986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a00ad850 con 0x7f74a006efa0 2024-09-16T15:47:38.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f749de00640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 0x7f74a00ab020 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f749000b3f0 tx=0x7f749000b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f7496a00640 1 -- 172.21.15.60:0/3504140986 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490015020 con 0x7f74a006efa0 2024-09-16T15:47:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f7496a00640 1 -- 172.21.15.60:0/3504140986 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7490011920 con 0x7f74a006efa0 2024-09-16T15:47:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f7496a00640 1 -- 172.21.15.60:0/3504140986 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490011db0 con 0x7f74a006efa0 2024-09-16T15:47:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a00adb30 con 0x7f74a006efa0 2024-09-16T15:47:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.364+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a00ae0f0 con 0x7f74a006efa0 2024-09-16T15:47:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.365+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a00cdbb0 con 0x7f74a006efa0 2024-09-16T15:47:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.367+0000 7f7496a00640 1 -- 172.21.15.60:0/3504140986 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f7490018020 con 0x7f74a006efa0 2024-09-16T15:47:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.368+0000 7f7496a00640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f747c0790c0 0x7f747c07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.368+0000 7f7496a00640 1 -- 172.21.15.60:0/3504140986 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f749009b650 con 0x7f74a006efa0 2024-09-16T15:47:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.368+0000 7f7495400640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f747c0790c0 0x7f747c07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.369+0000 7f7495400640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f747c0790c0 0x7f747c07b580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f74a00cc6e0 tx=0x7f748800a2d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:38.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.372+0000 7f7496a00640 1 -- 172.21.15.60:0/3504140986 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f749001f030 con 0x7f74a006efa0 2024-09-16T15:47:38.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.597+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol14660", "target": ["mon-mgr", ""]}) -- 0x7f74a015f2d0 con 0x7f747c0790c0 2024-09-16T15:47:38.606 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:38.604+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:38.606 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:38.604+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:38.606 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:38.604+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:38.606 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:38.604+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:38.606 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:38.604+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:38.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.663+0000 7f7496a00640 1 -- 172.21.15.60:0/3504140986 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f74a015f2d0 con 0x7f747c0790c0 2024-09-16T15:47:38.665 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/_nogroup/subvol14660/3eeb1382-6b4c-4a0e-b6fd-073a5a849dc4 2024-09-16T15:47:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f747c0790c0 msgr2=0x7f747c07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f747c0790c0 0x7f747c07b580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f74a00cc6e0 tx=0x7f748800a2d0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 msgr2=0x7f74a00ab020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 0x7f74a00ab020 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f749000b3f0 tx=0x7f749000b7f0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 shutdown_connections 2024-09-16T15:47:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f747c0790c0 0x7f747c07b580 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74a0145750 0x7f74a01f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74a00cccd0 0x7f74a00ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 --2- 172.21.15.60:0/3504140986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74a006efa0 0x7f74a00ab020 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 >> 172.21.15.60:0/3504140986 conn(0x7f74a00b1840 msgr2=0x7f74a00aa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 shutdown_connections 2024-09-16T15:47:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.666+0000 7f74a5200640 1 -- 172.21.15.60:0/3504140986 wait complete. 2024-09-16T15:47:38.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol27801 subvol_grp25450 2024-09-16T15:47:38.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.848+0000 7f9830a00640 1 -- 172.21.15.60:0/2096318770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c06e590 msgr2=0x7f982c06fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.848+0000 7f9830a00640 1 --2- 172.21.15.60:0/2096318770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c06e590 0x7f982c06fe40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9814009830 tx=0x7f981402a520 comp rx=0 tx=0).stop 2024-09-16T15:47:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.848+0000 7f9830a00640 1 -- 172.21.15.60:0/2096318770 shutdown_connections 2024-09-16T15:47:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.848+0000 7f9830a00640 1 --2- 172.21.15.60:0/2096318770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f982c070380 0x7f982c0cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.848+0000 7f9830a00640 1 --2- 172.21.15.60:0/2096318770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c06e590 0x7f982c06fe40 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.848+0000 7f9830a00640 1 -- 172.21.15.60:0/2096318770 >> 172.21.15.60:0/2096318770 conn(0x7f982c0cfa70 msgr2=0x7f982c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.848+0000 7f9830a00640 1 -- 172.21.15.60:0/2096318770 shutdown_connections 2024-09-16T15:47:38.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.848+0000 7f9830a00640 1 -- 172.21.15.60:0/2096318770 wait complete. 2024-09-16T15:47:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.850+0000 7f9830a00640 1 Processor -- start 2024-09-16T15:47:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.850+0000 7f9830a00640 1 -- start start 2024-09-16T15:47:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f9830a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f982c06e590 0x7f982c15ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f9830a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f982c070380 0x7f982c1f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f9830a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c15bda0 0x7f982c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f9830a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0cba90 con 0x7f982c15bda0 2024-09-16T15:47:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f9830a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0cb910 con 0x7f982c070380 2024-09-16T15:47:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f9830a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f982c0cbc10 con 0x7f982c06e590 2024-09-16T15:47:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c15bda0 0x7f982c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982b200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c15bda0 0x7f982c0cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58116/0 (socket says 172.21.15.60:58116) 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982b200640 1 -- 172.21.15.60:0/1796730859 learned_addr learned my addr 172.21.15.60:0/1796730859 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982a800640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f982c06e590 0x7f982c15ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f9821e00640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f982c070380 0x7f982c1f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982b200640 1 -- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f982c06e590 msgr2=0x7f982c15ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982b200640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f982c06e590 0x7f982c15ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982b200640 1 -- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f982c070380 msgr2=0x7f982c1f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982b200640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f982c070380 0x7f982c1f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f982b200640 1 -- 172.21.15.60:0/1796730859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c1f4c90 con 0x7f982c15bda0 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.851+0000 7f9821e00640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f982c070380 0x7f982c1f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.852+0000 7f982b200640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c15bda0 0x7f982c0cc620 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f982400cce0 tx=0x7f982400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.852+0000 7f9823400640 1 -- 172.21.15.60:0/1796730859 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982400f450 con 0x7f982c15bda0 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.852+0000 7f9823400640 1 -- 172.21.15.60:0/1796730859 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f982400f700 con 0x7f982c15bda0 2024-09-16T15:47:38.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.852+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c1f4f10 con 0x7f982c15bda0 2024-09-16T15:47:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.852+0000 7f9823400640 1 -- 172.21.15.60:0/1796730859 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982400fb50 con 0x7f982c15bda0 2024-09-16T15:47:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.852+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c1f54d0 con 0x7f982c15bda0 2024-09-16T15:47:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.853+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c0cc1a0 con 0x7f982c15bda0 2024-09-16T15:47:38.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.854+0000 7f9823400640 1 -- 172.21.15.60:0/1796730859 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f982400fcf0 con 0x7f982c15bda0 2024-09-16T15:47:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.855+0000 7f9823400640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f98080790c0 0x7f980807b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.855+0000 7f9823400640 1 -- 172.21.15.60:0/1796730859 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f982409c6c0 con 0x7f982c15bda0 2024-09-16T15:47:38.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.856+0000 7f982a800640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f98080790c0 0x7f980807b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.856+0000 7f982a800640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f98080790c0 0x7f980807b580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f981c001720 tx=0x7f981c007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:38.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:38.860+0000 7f9823400640 1 -- 172.21.15.60:0/1796730859 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f982c1f4c90 con 0x7f982c15bda0 2024-09-16T15:47:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.082+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol27801", "group_name": "subvol_grp25450", "target": ["mon-mgr", ""]}) -- 0x7f982c0cf610 con 0x7f98080790c0 2024-09-16T15:47:39.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.139+0000 7f9823400640 1 -- 172.21.15.60:0/1796730859 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7f982c0cf610 con 0x7f98080790c0 2024-09-16T15:47:39.141 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/subvol_grp25450/subvol27801/ea75e1c2-c266-4f55-91c8-3c8dab747146 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f98080790c0 msgr2=0x7f980807b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f98080790c0 0x7f980807b580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f981c001720 tx=0x7f981c007e90 comp rx=0 tx=0).stop 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c15bda0 msgr2=0x7f982c0cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c15bda0 0x7f982c0cc620 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f982400cce0 tx=0x7f982400a620 comp rx=0 tx=0).stop 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 shutdown_connections 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f98080790c0 0x7f980807b580 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f982c15bda0 0x7f982c0cc620 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f982c070380 0x7f982c1f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 --2- 172.21.15.60:0/1796730859 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f982c06e590 0x7f982c15ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.141+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 >> 172.21.15.60:0/1796730859 conn(0x7f982c0cfa70 msgr2=0x7f982c0b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:39.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.142+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 shutdown_connections 2024-09-16T15:47:39.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.142+0000 7f9830a00640 1 -- 172.21.15.60:0/1796730859 wait complete. 2024-09-16T15:47:39.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:39.160 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol14660/.meta) 2024-09-16T15:47:39.216 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:47:39.216 INFO:teuthology.orchestra.run.smithi060.stdout:version = 2 2024-09-16T15:47:39.216 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:47:39.216 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/_nogroup/subvol14660/3eeb1382-6b4c-4a0e-b6fd-073a5a849dc4 2024-09-16T15:47:39.216 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:47:39.217 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:39.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:39.219 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp25450/subvol27801/.meta) 2024-09-16T15:47:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:47:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:version = 2 2024-09-16T15:47:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:47:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/subvol_grp25450/subvol27801/ea75e1c2-c266-4f55-91c8-3c8dab747146 2024-09-16T15:47:39.275 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:47:39.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:39.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol14660 2024-09-16T15:47:39.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.440+0000 7f5e26a00640 1 -- 172.21.15.60:0/441294780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e2006efa0 msgr2=0x7f5e200cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.440+0000 7f5e26a00640 1 --2- 172.21.15.60:0/441294780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e2006efa0 0x7f5e200cdbb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5e18005fd0 tx=0x7f5e1802a540 comp rx=0 tx=0).stop 2024-09-16T15:47:39.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.440+0000 7f5e26a00640 1 -- 172.21.15.60:0/441294780 shutdown_connections 2024-09-16T15:47:39.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.440+0000 7f5e26a00640 1 --2- 172.21.15.60:0/441294780 >> v2:172.21.15.100:3300/0 conn(0x7f5e200cccd0 0x7f5e20073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.440+0000 7f5e26a00640 1 --2- 172.21.15.60:0/441294780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e2006efa0 0x7f5e200cdbb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.440+0000 7f5e26a00640 1 -- 172.21.15.60:0/441294780 >> 172.21.15.60:0/441294780 conn(0x7f5e200b1840 msgr2=0x7f5e200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:39.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.440+0000 7f5e26a00640 1 -- 172.21.15.60:0/441294780 shutdown_connections 2024-09-16T15:47:39.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.440+0000 7f5e26a00640 1 -- 172.21.15.60:0/441294780 wait complete. 2024-09-16T15:47:39.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.441+0000 7f5e26a00640 1 Processor -- start 2024-09-16T15:47:39.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.441+0000 7f5e26a00640 1 -- start start 2024-09-16T15:47:39.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.442+0000 7f5e26a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e2006efa0 0x7f5e201f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:39.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.442+0000 7f5e26a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e200cccd0 0x7f5e2013c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:39.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.442+0000 7f5e26a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20145750 0x7f5e2015e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:39.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.442+0000 7f5e26a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5e200ce480 con 0x7f5e20145750 2024-09-16T15:47:39.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.442+0000 7f5e26a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e200ce300 con 0x7f5e200cccd0 2024-09-16T15:47:39.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.442+0000 7f5e26a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5e200ce600 con 0x7f5e2006efa0 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20145750 0x7f5e2015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20145750 0x7f5e2015e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58118/0 (socket says 172.21.15.60:58118) 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 -- 172.21.15.60:0/1340971278 learned_addr learned my addr 172.21.15.60:0/1340971278 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e1fe00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e2006efa0 0x7f5e201f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e17400640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e200cccd0 0x7f5e2013c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 -- 172.21.15.60:0/1340971278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e2006efa0 msgr2=0x7f5e201f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e2006efa0 0x7f5e201f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 -- 172.21.15.60:0/1340971278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e200cccd0 msgr2=0x7f5e2013c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e200cccd0 0x7f5e2013c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 -- 172.21.15.60:0/1340971278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2013cb10 con 0x7f5e20145750 2024-09-16T15:47:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.443+0000 7f5e24a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20145750 0x7f5e2015e500 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5e18006010 tx=0x7f5e18003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:39.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.444+0000 7f5e1cc00640 1 -- 172.21.15.60:0/1340971278 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1803f070 con 0x7f5e20145750 2024-09-16T15:47:39.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.444+0000 7f5e1cc00640 1 -- 172.21.15.60:0/1340971278 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e180047a0 con 0x7f5e20145750 2024-09-16T15:47:39.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.444+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2015ea40 con 0x7f5e20145750 2024-09-16T15:47:39.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.444+0000 7f5e1cc00640 1 -- 172.21.15.60:0/1340971278 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18035030 con 0x7f5e20145750 2024-09-16T15:47:39.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.444+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e20141630 con 0x7f5e20145750 2024-09-16T15:47:39.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.445+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e200cdbb0 con 0x7f5e20145750 2024-09-16T15:47:39.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.446+0000 7f5e1cc00640 1 -- 172.21.15.60:0/1340971278 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f5e18042070 con 0x7f5e20145750 2024-09-16T15:47:39.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.447+0000 7f5e1cc00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5dfc0790c0 0x7f5dfc07b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:39.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.447+0000 7f5e1cc00640 1 -- 172.21.15.60:0/1340971278 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f5e180bc990 con 0x7f5e20145750 2024-09-16T15:47:39.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.447+0000 7f5e1fe00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5dfc0790c0 0x7f5dfc07b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:39.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.448+0000 7f5e1fe00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5dfc0790c0 0x7f5dfc07b580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5e201f58a0 tx=0x7f5e10009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.451+0000 7f5e1cc00640 1 -- 172.21.15.60:0/1340971278 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5e18084040 con 0x7f5e20145750 2024-09-16T15:47:39.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.672+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol14660", "target": ["mon-mgr", ""]}) -- 0x7f5e2015f4f0 con 0x7f5dfc0790c0 2024-09-16T15:47:39.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.690+0000 7f5e1cc00640 1 -- 172.21.15.60:0/1340971278 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e2015f4f0 con 0x7f5dfc0790c0 2024-09-16T15:47:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5dfc0790c0 msgr2=0x7f5dfc07b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5dfc0790c0 0x7f5dfc07b580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5e201f58a0 tx=0x7f5e10009040 comp rx=0 tx=0).stop 2024-09-16T15:47:39.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20145750 msgr2=0x7f5e2015e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20145750 0x7f5e2015e500 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5e18006010 tx=0x7f5e18003820 comp rx=0 tx=0).stop 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 shutdown_connections 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5dfc0790c0 0x7f5dfc07b580 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e20145750 0x7f5e2015e500 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e200cccd0 0x7f5e2013c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 --2- 172.21.15.60:0/1340971278 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e2006efa0 0x7f5e201f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 >> 172.21.15.60:0/1340971278 conn(0x7f5e200b1840 msgr2=0x7f5e200a9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 shutdown_connections 2024-09-16T15:47:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.692+0000 7f5e26a00640 1 -- 172.21.15.60:0/1340971278 wait complete. 2024-09-16T15:47:39.699 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.697+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.699 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.697+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.699 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.697+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.699 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.697+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.699 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.697+0000 7fcb87800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol27801 subvol_grp25450 2024-09-16T15:47:39.716 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.714+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.716 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.714+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.716 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.714+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.716 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.714+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.716 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:47:39.714+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.878+0000 7f4883e00640 1 -- 172.21.15.60:0/458496845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4864003fd0 msgr2=0x7f4864004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.878+0000 7f4883e00640 1 --2- 172.21.15.60:0/458496845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4864003fd0 0x7f4864004800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4868009b50 tx=0x7f4864005830 comp rx=0 tx=0).stop 2024-09-16T15:47:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.878+0000 7f4881e00640 1 -- 172.21.15.60:0/458496845 reap_dead start 2024-09-16T15:47:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.878+0000 7f4883e00640 1 -- 172.21.15.60:0/458496845 shutdown_connections 2024-09-16T15:47:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.878+0000 7f4883e00640 1 -- 172.21.15.60:0/458496845 >> 172.21.15.60:0/458496845 conn(0x7f487c0b18e0 msgr2=0x7f487c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.879+0000 7f4883e00640 1 -- 172.21.15.60:0/458496845 shutdown_connections 2024-09-16T15:47:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.879+0000 7f4883e00640 1 -- 172.21.15.60:0/458496845 wait complete. 2024-09-16T15:47:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.880+0000 7f4883e00640 1 Processor -- start 2024-09-16T15:47:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.880+0000 7f4883e00640 1 -- start start 2024-09-16T15:47:39.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.881+0000 7f4883e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4864003fd0 0x7f487c1f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:39.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.881+0000 7f4883e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c15b6f0 0x7f487c14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.881+0000 7f4883e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c0ca280 0x7f487c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.881+0000 7f4883e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f487c06ecc0 con 0x7f487c15b6f0 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.881+0000 7f4883e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f487c06eb40 con 0x7f4864003fd0 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.881+0000 7f4883e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f487c0cc190 con 0x7f487c0ca280 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.881+0000 7f4880a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c15b6f0 0x7f487c14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4880a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c15b6f0 0x7f487c14d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58164/0 (socket says 172.21.15.60:58164) 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4880a00640 1 -- 172.21.15.60:0/3823424063 learned_addr learned my addr 172.21.15.60:0/3823424063 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4881400640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4864003fd0 0x7f487c1f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4881e00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c0ca280 0x7f487c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4880a00640 1 -- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c0ca280 msgr2=0x7f487c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4880a00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c0ca280 0x7f487c0ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4880a00640 1 -- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4864003fd0 msgr2=0x7f487c1f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4880a00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4864003fd0 0x7f487c1f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4880a00640 1 -- 172.21.15.60:0/3823424063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c14de90 con 0x7f487c15b6f0 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4881400640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4864003fd0 0x7f487c1f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4881e00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c0ca280 0x7f487c0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4880a00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c15b6f0 0x7f487c14d950 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f4868009c70 tx=0x7f4868033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.882+0000 7f4879600640 1 -- 172.21.15.60:0/3823424063 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868038040 con 0x7f487c15b6f0 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.883+0000 7f4879600640 1 -- 172.21.15.60:0/3823424063 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4868043070 con 0x7f487c15b6f0 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.883+0000 7f4879600640 1 -- 172.21.15.60:0/3823424063 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868033bd0 con 0x7f487c15b6f0 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.883+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c1f82a0 con 0x7f487c15b6f0 2024-09-16T15:47:39.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.883+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c1f8860 con 0x7f487c15b6f0 2024-09-16T15:47:39.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.884+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c069360 con 0x7f487c15b6f0 2024-09-16T15:47:39.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.885+0000 7f4879600640 1 -- 172.21.15.60:0/3823424063 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 24) ==== 102052+0+0 (secure 0 0 0) 0x7f486803e440 con 0x7f487c15b6f0 2024-09-16T15:47:39.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.886+0000 7f4879600640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f485c0790f0 0x7f485c07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:39.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.887+0000 7f4879600640 1 -- 172.21.15.60:0/3823424063 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f48680bc3c0 con 0x7f487c15b6f0 2024-09-16T15:47:39.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.887+0000 7f4881400640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f485c0790f0 0x7f485c07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:39.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.887+0000 7f4881400640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f485c0790f0 0x7f485c07b5b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4870006d30 tx=0x7f4870001590 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:39.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:39.891+0000 7f4879600640 1 -- 172.21.15.60:0/3823424063 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f486804c020 con 0x7f487c15b6f0 2024-09-16T15:47:40.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:40.114+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol27801", "group_name": "subvol_grp25450", "target": ["mon-mgr", ""]}) -- 0x7f487c15b290 con 0x7f485c0790f0 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:40.786+0000 7f4879600640 1 -- 172.21.15.60:0/3823424063 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f486807f550 con 0x7f487c15b6f0 2024-09-16T15:47:46.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.463+0000 7f4879600640 1 -- 172.21.15.60:0/3823424063 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4870006d60 con 0x7f485c0790f0 2024-09-16T15:47:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f485c0790f0 msgr2=0x7f485c07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f485c0790f0 0x7f485c07b5b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4870006d30 tx=0x7f4870001590 comp rx=0 tx=0).stop 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c15b6f0 msgr2=0x7f487c14d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c15b6f0 0x7f487c14d950 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f4868009c70 tx=0x7f4868033670 comp rx=0 tx=0).stop 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 shutdown_connections 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f485c0790f0 0x7f485c07b5b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c0ca280 0x7f487c0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c15b6f0 0x7f487c14d950 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 --2- 172.21.15.60:0/3823424063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4864003fd0 0x7f487c1f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 >> 172.21.15.60:0/3823424063 conn(0x7f487c0b18e0 msgr2=0x7f487c0afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 shutdown_connections 2024-09-16T15:47:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46.466+0000 7f4883e00640 1 -- 172.21.15.60:0/3823424063 wait complete. 2024-09-16T15:47:46.488 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:47:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:47:46.620 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-16T15:47:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:47:46.629 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:47:46.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:47:46.725 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:47:46.821 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:47:46.821 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:47:46.821 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:46.821 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:46.857 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:46.857 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:46.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:47:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.131+0000 7f24f3a00640 1 -- 172.21.15.60:0/2624190630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec15f5a0 msgr2=0x7f24ec0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.131+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2624190630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec15f5a0 0x7f24ec0b2440 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0cfbb0 tx=0x7f24e002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.131+0000 7f24f3a00640 1 -- 172.21.15.60:0/2624190630 shutdown_connections 2024-09-16T15:47:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.131+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2624190630 >> v2:172.21.15.100:3300/0 conn(0x7f24ec0ccf00 0x7f24ec0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.131+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2624190630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec15f5a0 0x7f24ec0b2440 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.131+0000 7f24f3a00640 1 -- 172.21.15.60:0/2624190630 >> 172.21.15.60:0/2624190630 conn(0x7f24ec0b1950 msgr2=0x7f24ec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.131+0000 7f24f3a00640 1 -- 172.21.15.60:0/2624190630 shutdown_connections 2024-09-16T15:47:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.131+0000 7f24f3a00640 1 -- 172.21.15.60:0/2624190630 wait complete. 2024-09-16T15:47:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.133+0000 7f24f3a00640 1 Processor -- start 2024-09-16T15:47:47.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.133+0000 7f24f3a00640 1 -- start start 2024-09-16T15:47:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f3a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ad270 0x7f24ec14c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f3a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0ccf00 0x7f24ec152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f3a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15f5a0 0x7f24ec152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f3a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec06f160 con 0x7f24ec0ccf00 2024-09-16T15:47:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f3a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec06f2e0 con 0x7f24ec0ad270 2024-09-16T15:47:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f3a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f24ec06f460 con 0x7f24ec15f5a0 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24ebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0ccf00 0x7f24ec152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24ebe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0ccf00 0x7f24ec152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58182/0 (socket says 172.21.15.60:58182) 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24ebe00640 1 -- 172.21.15.60:0/2037329120 learned_addr learned my addr 172.21.15.60:0/2037329120 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f1000640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ad270 0x7f24ec14c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f1a00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15f5a0 0x7f24ec152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24ebe00640 1 -- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15f5a0 msgr2=0x7f24ec152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24ebe00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15f5a0 0x7f24ec152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24ebe00640 1 -- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ad270 msgr2=0x7f24ec14c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24ebe00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ad270 0x7f24ec14c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24ebe00640 1 -- 172.21.15.60:0/2037329120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec1f2c40 con 0x7f24ec0ccf00 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f1000640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ad270 0x7f24ec14c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.134+0000 7f24f1a00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15f5a0 0x7f24ec152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:47.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.135+0000 7f24ebe00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0ccf00 0x7f24ec152220 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9f0 tx=0x7f24d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.135+0000 7f24e8c00640 1 -- 172.21.15.60:0/2037329120 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d800cd80 con 0x7f24ec0ccf00 2024-09-16T15:47:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.135+0000 7f24e8c00640 1 -- 172.21.15.60:0/2037329120 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24d8010430 con 0x7f24ec0ccf00 2024-09-16T15:47:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.135+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ec150990 con 0x7f24ec0ccf00 2024-09-16T15:47:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.135+0000 7f24e8c00640 1 -- 172.21.15.60:0/2037329120 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8010840 con 0x7f24ec0ccf00 2024-09-16T15:47:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.135+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ec1f2ee0 con 0x7f24ec0ccf00 2024-09-16T15:47:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.136+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ec06f9f0 con 0x7f24ec0ccf00 2024-09-16T15:47:47.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.137+0000 7f24e8c00640 1 -- 172.21.15.60:0/2037329120 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f24d80109e0 con 0x7f24ec0ccf00 2024-09-16T15:47:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.138+0000 7f24e8c00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24cc079270 0x7f24cc07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:47.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.139+0000 7f24e8c00640 1 -- 172.21.15.60:0/2037329120 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f24d809a060 con 0x7f24ec0ccf00 2024-09-16T15:47:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.139+0000 7f24f1000640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24cc079270 0x7f24cc07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:47.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.139+0000 7f24f1000640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24cc079270 0x7f24cc07b730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24e0034000 tx=0x7f24e0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:47.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.143+0000 7f24e8c00640 1 -- 172.21.15.60:0/2037329120 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f24d8063fc0 con 0x7f24ec0ccf00 2024-09-16T15:47:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.354+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f24ec15e060 con 0x7f24ec0ccf00 2024-09-16T15:47:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.354+0000 7f24e8c00640 1 -- 172.21.15.60:0/2037329120 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v161) ==== 80+0+0 (secure 0 0 0) 0x7f24d80638c0 con 0x7f24ec0ccf00 2024-09-16T15:47:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:47:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24cc079270 msgr2=0x7f24cc07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24cc079270 0x7f24cc07b730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f24e0034000 tx=0x7f24e0004730 comp rx=0 tx=0).stop 2024-09-16T15:47:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0ccf00 msgr2=0x7f24ec152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0ccf00 0x7f24ec152220 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9f0 tx=0x7f24d800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 shutdown_connections 2024-09-16T15:47:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24cc079270 0x7f24cc07b730 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24ec15f5a0 0x7f24ec152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24ec0ccf00 0x7f24ec152220 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.356+0000 7f24f3a00640 1 --2- 172.21.15.60:0/2037329120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec0ad270 0x7f24ec14c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.357+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 >> 172.21.15.60:0/2037329120 conn(0x7f24ec0b1950 msgr2=0x7f24ec15e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.357+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 shutdown_connections 2024-09-16T15:47:47.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.357+0000 7f24f3a00640 1 -- 172.21.15.60:0/2037329120 wait complete. 2024-09-16T15:47:47.374 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:47:47.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:47.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:47.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:47:47.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:47.469 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:47:47.515 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:47:47.516 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:47.516 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:47.553 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:47.554 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:47.630 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:47:47.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.793+0000 7fbafd400640 1 -- 172.21.15.60:0/3920803381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf8072df0 msgr2=0x7fbaf8073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:47.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.793+0000 7fbafd400640 1 --2- 172.21.15.60:0/3920803381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf8072df0 0x7fbaf8073240 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80b4890 tx=0x7fbaf002aa10 comp rx=0 tx=0).stop 2024-09-16T15:47:47.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.794+0000 7fbafd400640 1 -- 172.21.15.60:0/3920803381 shutdown_connections 2024-09-16T15:47:47.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.794+0000 7fbafd400640 1 --2- 172.21.15.60:0/3920803381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf8072df0 0x7fbaf8073240 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.794+0000 7fbafd400640 1 --2- 172.21.15.60:0/3920803381 >> v2:172.21.15.100:3300/0 conn(0x7fbaf80af3d0 0x7fbaf806f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.794+0000 7fbafd400640 1 -- 172.21.15.60:0/3920803381 >> 172.21.15.60:0/3920803381 conn(0x7fbaf80b1950 msgr2=0x7fbaf80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:47.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.794+0000 7fbafd400640 1 -- 172.21.15.60:0/3920803381 shutdown_connections 2024-09-16T15:47:47.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.794+0000 7fbafd400640 1 -- 172.21.15.60:0/3920803381 wait complete. 2024-09-16T15:47:47.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.795+0000 7fbafd400640 1 Processor -- start 2024-09-16T15:47:47.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.796+0000 7fbafd400640 1 -- start start 2024-09-16T15:47:47.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.796+0000 7fbafd400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbaf8072df0 0x7fbaf81f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:47.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.796+0000 7fbafd400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf80af3d0 0x7fbaf8155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:47.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.796+0000 7fbafd400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf8161550 0x7fbaf81436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:47.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.796+0000 7fbafd400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8073780 con 0x7fbaf80af3d0 2024-09-16T15:47:47.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.796+0000 7fbafd400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf806fdb0 con 0x7fbaf8161550 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.796+0000 7fbafd400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf8073900 con 0x7fbaf8072df0 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf80af3d0 0x7fbaf8155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf80af3d0 0x7fbaf8155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58200/0 (socket says 172.21.15.60:58200) 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 -- 172.21.15.60:0/3119585024 learned_addr learned my addr 172.21.15.60:0/3119585024 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaf6800640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbaf8072df0 0x7fbaf81f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaf7200640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf8161550 0x7fbaf81436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 -- 172.21.15.60:0/3119585024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbaf8072df0 msgr2=0x7fbaf81f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbaf8072df0 0x7fbaf81f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 -- 172.21.15.60:0/3119585024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf8161550 msgr2=0x7fbaf81436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf8161550 0x7fbaf81436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 -- 172.21.15.60:0/3119585024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf8156700 con 0x7fbaf80af3d0 2024-09-16T15:47:47.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.797+0000 7fbaede00640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf80af3d0 0x7fbaf8155fd0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbae000e9f0 tx=0x7fbae000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:47.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.798+0000 7fbaef400640 1 -- 172.21.15.60:0/3119585024 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae000cd80 con 0x7fbaf80af3d0 2024-09-16T15:47:47.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.798+0000 7fbaef400640 1 -- 172.21.15.60:0/3119585024 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbae0010430 con 0x7fbaf80af3d0 2024-09-16T15:47:47.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.798+0000 7fbaef400640 1 -- 172.21.15.60:0/3119585024 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0010840 con 0x7fbaf80af3d0 2024-09-16T15:47:47.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.798+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf81569e0 con 0x7fbaf80af3d0 2024-09-16T15:47:47.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.798+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf80cfb40 con 0x7fbaf80af3d0 2024-09-16T15:47:47.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.799+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf8073df0 con 0x7fbaf80af3d0 2024-09-16T15:47:47.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.800+0000 7fbaef400640 1 -- 172.21.15.60:0/3119585024 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fbae00109e0 con 0x7fbaf80af3d0 2024-09-16T15:47:47.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.802+0000 7fbaef400640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbad4079270 0x7fbad407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:47.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.802+0000 7fbaef400640 1 -- 172.21.15.60:0/3119585024 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fbae0099f60 con 0x7fbaf80af3d0 2024-09-16T15:47:47.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.802+0000 7fbaf6800640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbad4079270 0x7fbad407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:47.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.802+0000 7fbaf6800640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbad4079270 0x7fbad407b730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbae8001720 tx=0x7fbae8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:47.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:47.806+0000 7fbaef400640 1 -- 172.21.15.60:0/3119585024 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbae0063870 con 0x7fbaf80af3d0 2024-09-16T15:47:48.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.013+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbaf80aeb10 con 0x7fbaf80af3d0 2024-09-16T15:47:48.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.013+0000 7fbaef400640 1 -- 172.21.15.60:0/3119585024 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v161) ==== 80+0+0 (secure 0 0 0) 0x7fbae00682a0 con 0x7fbaf80af3d0 2024-09-16T15:47:48.016 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbad4079270 msgr2=0x7fbad407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbad4079270 0x7fbad407b730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbae8001720 tx=0x7fbae8009040 comp rx=0 tx=0).stop 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf80af3d0 msgr2=0x7fbaf8155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf80af3d0 0x7fbaf8155fd0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbae000e9f0 tx=0x7fbae000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 shutdown_connections 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbad4079270 0x7fbad407b730 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbaf8161550 0x7fbaf81436e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbaf80af3d0 0x7fbaf8155fd0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 --2- 172.21.15.60:0/3119585024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbaf8072df0 0x7fbaf81f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 >> 172.21.15.60:0/3119585024 conn(0x7fbaf80b1950 msgr2=0x7fbaf80ae400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 shutdown_connections 2024-09-16T15:47:48.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.016+0000 7fbafd400640 1 -- 172.21.15.60:0/3119585024 wait complete. 2024-09-16T15:47:48.033 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:47:48.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:48.034 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:47:48.099 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:47:48.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:48.099 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:47:48.135 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:47:48.135 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:48.135 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:48.171 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:47:48.171 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:47:48.171 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:48.171 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:48.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:47:48.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.370+0000 7fb90e200640 1 -- 172.21.15.60:0/3823683138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080ccf00 msgr2=0x7fb9080b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.370+0000 7fb90e200640 1 --2- 172.21.15.60:0/3823683138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080ccf00 0x7fb9080b2440 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009830 tx=0x7fb8f002a530 comp rx=0 tx=0).stop 2024-09-16T15:47:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.370+0000 7fb90e200640 1 -- 172.21.15.60:0/3823683138 shutdown_connections 2024-09-16T15:47:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.370+0000 7fb90e200640 1 --2- 172.21.15.60:0/3823683138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080ccf00 0x7fb9080b2440 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.370+0000 7fb90e200640 1 -- 172.21.15.60:0/3823683138 >> 172.21.15.60:0/3823683138 conn(0x7fb9080b1950 msgr2=0x7fb9080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.370+0000 7fb90e200640 1 -- 172.21.15.60:0/3823683138 shutdown_connections 2024-09-16T15:47:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.370+0000 7fb90e200640 1 -- 172.21.15.60:0/3823683138 wait complete. 2024-09-16T15:47:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.372+0000 7fb90e200640 1 Processor -- start 2024-09-16T15:47:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.372+0000 7fb90e200640 1 -- start start 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb90e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb908073fb0 0x7fb9081f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb90e200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9080ccf00 0x7fb90815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb90e200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90815f5a0 0x7fb908151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb90e200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb90806f9f0 con 0x7fb908073fb0 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb90e200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb90806f870 con 0x7fb9080ccf00 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb90e200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb908074420 con 0x7fb90815f5a0 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb907400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb908073fb0 0x7fb9081f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb907400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb908073fb0 0x7fb9081f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58230/0 (socket says 172.21.15.60:58230) 2024-09-16T15:47:48.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb907400640 1 -- 172.21.15.60:0/2562784943 learned_addr learned my addr 172.21.15.60:0/2562784943 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb8fea00640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9080ccf00 0x7fb90815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb907e00640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90815f5a0 0x7fb908151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb907400640 1 -- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90815f5a0 msgr2=0x7fb908151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb907400640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90815f5a0 0x7fb908151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb907400640 1 -- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9080ccf00 msgr2=0x7fb90815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.373+0000 7fb907400640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9080ccf00 0x7fb90815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.374+0000 7fb907400640 1 -- 172.21.15.60:0/2562784943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90815c2f0 con 0x7fb908073fb0 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.374+0000 7fb907e00640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90815f5a0 0x7fb908151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.374+0000 7fb907400640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb908073fb0 0x7fb9081f6790 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c3a0 tx=0x7fb8f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.374+0000 7fb8ffe00640 1 -- 172.21.15.60:0/2562784943 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8017070 con 0x7fb908073fb0 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.374+0000 7fb8ffe00640 1 -- 172.21.15.60:0/2562784943 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8f80053c0 con 0x7fb908073fb0 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.374+0000 7fb8ffe00640 1 -- 172.21.15.60:0/2562784943 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8003850 con 0x7fb908073fb0 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.374+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90815c5b0 con 0x7fb908073fb0 2024-09-16T15:47:48.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.374+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9081f5cb0 con 0x7fb908073fb0 2024-09-16T15:47:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.375+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9080b2440 con 0x7fb908073fb0 2024-09-16T15:47:48.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.376+0000 7fb8ffe00640 1 -- 172.21.15.60:0/2562784943 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fb8f801a070 con 0x7fb908073fb0 2024-09-16T15:47:48.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.377+0000 7fb8ffe00640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb8e4079270 0x7fb8e407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.377+0000 7fb8ffe00640 1 -- 172.21.15.60:0/2562784943 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fb8f80987b0 con 0x7fb908073fb0 2024-09-16T15:47:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.378+0000 7fb8fea00640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb8e4079270 0x7fb8e407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.378+0000 7fb8fea00640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb8e4079270 0x7fb8e407b730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0034000 tx=0x7fb8f0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:48.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.381+0000 7fb8ffe00640 1 -- 172.21.15.60:0/2562784943 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb8f8062650 con 0x7fb908073fb0 2024-09-16T15:47:48.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.614+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9080cc370 con 0x7fb908073fb0 2024-09-16T15:47:48.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.615+0000 7fb8ffe00640 1 -- 172.21.15.60:0/2562784943 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7fb8f8061f50 con 0x7fb908073fb0 2024-09-16T15:47:48.617 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:48.617 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":113,"btime":"2024-09-16T15:47:36:757844+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7361,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1464936997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1464936997},{"type":"v1","addr":"172.21.15.60:6834","nonce":1464936997}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":7364,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1611348877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1611348877},{"type":"v1","addr":"172.21.15.60:6835","nonce":1611348877}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2321854052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2321854052},{"type":"v1","addr":"172.21.15.100:6834","nonce":2321854052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:47:26.164998+0000","modified":"2024-09-16T15:47:29.176697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"d","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3325202304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":3325202304},{"type":"v1","addr":"172.21.15.100:6835","nonce":3325202304}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":12}]} 2024-09-16T15:47:48.617 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 113 2024-09-16T15:47:48.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb8e4079270 msgr2=0x7fb8e407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb8e4079270 0x7fb8e407b730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0034000 tx=0x7fb8f0004730 comp rx=0 tx=0).stop 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb908073fb0 msgr2=0x7fb9081f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb908073fb0 0x7fb9081f6790 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c3a0 tx=0x7fb8f800c860 comp rx=0 tx=0).stop 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 shutdown_connections 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb8e4079270 0x7fb8e407b730 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90815f5a0 0x7fb908151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9080ccf00 0x7fb90815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 --2- 172.21.15.60:0/2562784943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb908073fb0 0x7fb9081f6790 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 >> 172.21.15.60:0/2562784943 conn(0x7fb9080b1950 msgr2=0x7fb90809cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 shutdown_connections 2024-09-16T15:47:48.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.617+0000 7fb90e200640 1 -- 172.21.15.60:0/2562784943 wait complete. 2024-09-16T15:47:48.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.795+0000 7f7020e00640 1 -- 172.21.15.60:0/2068170161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c0af3d0 msgr2=0x7f701c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.795+0000 7f7020e00640 1 --2- 172.21.15.60:0/2068170161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c0af3d0 0x7f701c06f490 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f701c06d9e0 tx=0x7f700802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.795+0000 7f7020e00640 1 -- 172.21.15.60:0/2068170161 shutdown_connections 2024-09-16T15:47:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.795+0000 7f7020e00640 1 --2- 172.21.15.60:0/2068170161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c0af3d0 0x7f701c06f490 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.795+0000 7f7020e00640 1 --2- 172.21.15.60:0/2068170161 >> v2:172.21.15.100:3300/0 conn(0x7f701c161550 0x7f701c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.795+0000 7f7020e00640 1 -- 172.21.15.60:0/2068170161 >> 172.21.15.60:0/2068170161 conn(0x7f701c0b1950 msgr2=0x7f701c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.795+0000 7f7020e00640 1 -- 172.21.15.60:0/2068170161 shutdown_connections 2024-09-16T15:47:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.795+0000 7f7020e00640 1 -- 172.21.15.60:0/2068170161 wait complete. 2024-09-16T15:47:48.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.797+0000 7f7020e00640 1 Processor -- start 2024-09-16T15:47:48.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.797+0000 7f7020e00640 1 -- start start 2024-09-16T15:47:48.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f7020e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f701c06f9d0 0x7f701c147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f7020e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f701c0af3d0 0x7f701c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f7020e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c161550 0x7f701c145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f7020e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f701c073f80 con 0x7f701c161550 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f7020e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f701c073e00 con 0x7f701c0af3d0 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f7020e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f701c074100 con 0x7f701c06f9d0 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f701a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c161550 0x7f701c145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f701a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c161550 0x7f701c145360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41588/0 (socket says 172.21.15.60:41588) 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f701a800640 1 -- 172.21.15.60:0/1170164277 learned_addr learned my addr 172.21.15.60:0/1170164277 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f7019e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f701c06f9d0 0x7f701c147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.798+0000 7f7019400640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f701c0af3d0 0x7f701c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f701a800640 1 -- 172.21.15.60:0/1170164277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f701c06f9d0 msgr2=0x7f701c147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f701a800640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f701c06f9d0 0x7f701c147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f701a800640 1 -- 172.21.15.60:0/1170164277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f701c0af3d0 msgr2=0x7f701c144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f701a800640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f701c0af3d0 0x7f701c144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f701a800640 1 -- 172.21.15.60:0/1170164277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c148340 con 0x7f701c161550 2024-09-16T15:47:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f701a800640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c161550 0x7f701c145360 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f701400cbc0 tx=0x7f701400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f7012000640 1 -- 172.21.15.60:0/1170164277 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014016020 con 0x7f701c161550 2024-09-16T15:47:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f7012000640 1 -- 172.21.15.60:0/1170164277 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f701400ae30 con 0x7f701c161550 2024-09-16T15:47:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f7012000640 1 -- 172.21.15.60:0/1170164277 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014012660 con 0x7f701c161550 2024-09-16T15:47:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.799+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c145040 con 0x7f701c161550 2024-09-16T15:47:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.800+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c0ce010 con 0x7f701c161550 2024-09-16T15:47:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.801+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c1626a0 con 0x7f701c161550 2024-09-16T15:47:48.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.802+0000 7f7012000640 1 -- 172.21.15.60:0/1170164277 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f7014019020 con 0x7f701c161550 2024-09-16T15:47:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.803+0000 7f7012000640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6ffc0792a0 0x7f6ffc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.803+0000 7f7012000640 1 -- 172.21.15.60:0/1170164277 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f7014099970 con 0x7f701c161550 2024-09-16T15:47:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.803+0000 7f7019e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6ffc0792a0 0x7f6ffc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.804+0000 7f7019e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6ffc0792a0 0x7f6ffc07b760 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f700c009b30 tx=0x7f700c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:48.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:48.807+0000 7f7012000640 1 -- 172.21.15.60:0/1170164277 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7014020020 con 0x7f701c161550 2024-09-16T15:47:49.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.039+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f701c074530 con 0x7f701c161550 2024-09-16T15:47:49.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.039+0000 7f7012000640 1 -- 172.21.15.60:0/1170164277 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7f701c148340 con 0x7f701c161550 2024-09-16T15:47:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:49.042 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":113,"btime":"2024-09-16T15:47:36:757844+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7361,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1464936997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1464936997},{"type":"v1","addr":"172.21.15.60:6834","nonce":1464936997}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":7364,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1611348877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1611348877},{"type":"v1","addr":"172.21.15.60:6835","nonce":1611348877}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2321854052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2321854052},{"type":"v1","addr":"172.21.15.100:6834","nonce":2321854052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:47:26.164998+0000","modified":"2024-09-16T15:47:29.176697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"d","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3325202304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":3325202304},{"type":"v1","addr":"172.21.15.100:6835","nonce":3325202304}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":12}]} 2024-09-16T15:47:49.042 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 113 2024-09-16T15:47:49.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6ffc0792a0 msgr2=0x7f6ffc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6ffc0792a0 0x7f6ffc07b760 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f700c009b30 tx=0x7f700c005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c161550 msgr2=0x7f701c145360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c161550 0x7f701c145360 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f701400cbc0 tx=0x7f701400a430 comp rx=0 tx=0).stop 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 shutdown_connections 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6ffc0792a0 0x7f6ffc07b760 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f701c161550 0x7f701c145360 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f701c0af3d0 0x7f701c144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 --2- 172.21.15.60:0/1170164277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f701c06f9d0 0x7f701c147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 >> 172.21.15.60:0/1170164277 conn(0x7f701c0b1950 msgr2=0x7f701c15ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 shutdown_connections 2024-09-16T15:47:49.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.042+0000 7f7020e00640 1 -- 172.21.15.60:0/1170164277 wait complete. 2024-09-16T15:47:49.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.219+0000 7fa277c00640 1 -- 172.21.15.60:0/2934353840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2701445d0 msgr2=0x7fa2700cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.219+0000 7fa277c00640 1 --2- 172.21.15.60:0/2934353840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2701445d0 0x7fa2700cebf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa260009860 tx=0x7fa26002a540 comp rx=0 tx=0).stop 2024-09-16T15:47:49.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.219+0000 7fa277c00640 1 -- 172.21.15.60:0/2934353840 shutdown_connections 2024-09-16T15:47:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.219+0000 7fa277c00640 1 --2- 172.21.15.60:0/2934353840 >> v2:172.21.15.100:3300/0 conn(0x7fa2700ccfd0 0x7fa2700cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.219+0000 7fa277c00640 1 --2- 172.21.15.60:0/2934353840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2701445d0 0x7fa2700cebf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.219+0000 7fa277c00640 1 -- 172.21.15.60:0/2934353840 >> 172.21.15.60:0/2934353840 conn(0x7fa2700b1950 msgr2=0x7fa2700b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.219+0000 7fa277c00640 1 -- 172.21.15.60:0/2934353840 shutdown_connections 2024-09-16T15:47:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.219+0000 7fa277c00640 1 -- 172.21.15.60:0/2934353840 wait complete. 2024-09-16T15:47:49.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.221+0000 7fa277c00640 1 Processor -- start 2024-09-16T15:47:49.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.221+0000 7fa277c00640 1 -- start start 2024-09-16T15:47:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa277c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2700b4ca0 0x7fa2701f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa277c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2700ccfd0 0x7fa27015e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:49.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa277c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2701445d0 0x7fa27009bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa277c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa27015f080 con 0x7fa2700b4ca0 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa277c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa270073bc0 con 0x7fa2700ccfd0 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa277c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa27015f200 con 0x7fa2701445d0 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2700b4ca0 0x7fa2701f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2700b4ca0 0x7fa2701f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41600/0 (socket says 172.21.15.60:41600) 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275200640 1 -- 172.21.15.60:0/2949253434 learned_addr learned my addr 172.21.15.60:0/2949253434 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa267e00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2700ccfd0 0x7fa27015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275c00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2701445d0 0x7fa27009bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275200640 1 -- 172.21.15.60:0/2949253434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2701445d0 msgr2=0x7fa27009bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275200640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2701445d0 0x7fa27009bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275200640 1 -- 172.21.15.60:0/2949253434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2700ccfd0 msgr2=0x7fa27015e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275200640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2700ccfd0 0x7fa27015e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.222+0000 7fa275200640 1 -- 172.21.15.60:0/2949253434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2701f91c0 con 0x7fa2700b4ca0 2024-09-16T15:47:49.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.223+0000 7fa275200640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2700b4ca0 0x7fa2701f8c80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa260000c00 tx=0x7fa260033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.223+0000 7fa26d600640 1 -- 172.21.15.60:0/2949253434 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260031d60 con 0x7fa2700b4ca0 2024-09-16T15:47:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.223+0000 7fa26d600640 1 -- 172.21.15.60:0/2949253434 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa260033e10 con 0x7fa2700b4ca0 2024-09-16T15:47:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.223+0000 7fa26d600640 1 -- 172.21.15.60:0/2949253434 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260033310 con 0x7fa2700b4ca0 2024-09-16T15:47:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.223+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2701e5170 con 0x7fa2700b4ca0 2024-09-16T15:47:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.223+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2701e5720 con 0x7fa2700b4ca0 2024-09-16T15:47:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.224+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27015f6b0 con 0x7fa2700b4ca0 2024-09-16T15:47:49.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.226+0000 7fa26d600640 1 -- 172.21.15.60:0/2949253434 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fa2600334b0 con 0x7fa2700b4ca0 2024-09-16T15:47:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.227+0000 7fa26d600640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa24c079270 0x7fa24c07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.227+0000 7fa26d600640 1 -- 172.21.15.60:0/2949253434 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fa2600ba680 con 0x7fa2700b4ca0 2024-09-16T15:47:49.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.227+0000 7fa267e00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa24c079270 0x7fa24c07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:49.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.228+0000 7fa267e00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa24c079270 0x7fa24c07b730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa27006f370 tx=0x7fa258005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:49.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.231+0000 7fa26d600640 1 -- 172.21.15.60:0/2949253434 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa2701f91c0 con 0x7fa2700b4ca0 2024-09-16T15:47:49.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.427+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa270070a20 con 0x7fa2700b4ca0 2024-09-16T15:47:49.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.428+0000 7fa26d600640 1 -- 172.21.15.60:0/2949253434 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+16783 (secure 0 0 0) 0x7fa26003f030 con 0x7fa2700b4ca0 2024-09-16T15:47:49.431 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:49.431 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":161,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:47:27.726082+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:47:23.982547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-16T15:47:25.124723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:49.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.430+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa24c079270 msgr2=0x7fa24c07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa24c079270 0x7fa24c07b730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa27006f370 tx=0x7fa258005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:49.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2700b4ca0 msgr2=0x7fa2701f8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2700b4ca0 0x7fa2701f8c80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa260000c00 tx=0x7fa260033c20 comp rx=0 tx=0).stop 2024-09-16T15:47:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 shutdown_connections 2024-09-16T15:47:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa24c079270 0x7fa24c07b730 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2701445d0 0x7fa27009bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2700ccfd0 0x7fa27015e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 --2- 172.21.15.60:0/2949253434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2700b4ca0 0x7fa2701f8c80 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 >> 172.21.15.60:0/2949253434 conn(0x7fa2700b1950 msgr2=0x7fa2700afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 shutdown_connections 2024-09-16T15:47:49.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.431+0000 7fa277c00640 1 -- 172.21.15.60:0/2949253434 wait complete. 2024-09-16T15:47:49.449 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:47:49.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:49.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.607+0000 7f3a7a800640 1 -- 172.21.15.60:0/2210459053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a74073fb0 msgr2=0x7f3a74074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.607+0000 7f3a7a800640 1 --2- 172.21.15.60:0/2210459053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a74073fb0 0x7f3a74074440 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c005fd0 tx=0x7f3a6c02a820 comp rx=0 tx=0).stop 2024-09-16T15:47:49.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.607+0000 7f3a7a800640 1 -- 172.21.15.60:0/2210459053 shutdown_connections 2024-09-16T15:47:49.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.607+0000 7f3a7a800640 1 --2- 172.21.15.60:0/2210459053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a74073fb0 0x7f3a74074440 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.607+0000 7f3a7a800640 1 --2- 172.21.15.60:0/2210459053 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a740ccf00 0x7f3a740b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.607+0000 7f3a7a800640 1 -- 172.21.15.60:0/2210459053 >> 172.21.15.60:0/2210459053 conn(0x7f3a740b1950 msgr2=0x7f3a740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:49.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.608+0000 7f3a7a800640 1 -- 172.21.15.60:0/2210459053 shutdown_connections 2024-09-16T15:47:49.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.608+0000 7f3a7a800640 1 -- 172.21.15.60:0/2210459053 wait complete. 2024-09-16T15:47:49.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.609+0000 7f3a7a800640 1 Processor -- start 2024-09-16T15:47:49.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.609+0000 7f3a7a800640 1 -- start start 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a7a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a74073fb0 0x7f3a740d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a7a800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a740ccf00 0x7f3a741f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a7a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a7415f5a0 0x7f3a7415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a7a800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7406f2e0 con 0x7f3a7415f5a0 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a7a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7406f160 con 0x7f3a74073fb0 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a7a800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7406f460 con 0x7f3a740ccf00 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a7415f5a0 0x7f3a7415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a7415f5a0 0x7f3a7415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41614/0 (socket says 172.21.15.60:41614) 2024-09-16T15:47:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.610+0000 7f3a73e00640 1 -- 172.21.15.60:0/3070434305 learned_addr learned my addr 172.21.15.60:0/3070434305 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.611+0000 7f3a6aa00640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a740ccf00 0x7f3a741f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.611+0000 7f3a73400640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a74073fb0 0x7f3a740d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.611+0000 7f3a73e00640 1 -- 172.21.15.60:0/3070434305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a740ccf00 msgr2=0x7f3a741f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.611+0000 7f3a73e00640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a740ccf00 0x7f3a741f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.611+0000 7f3a73e00640 1 -- 172.21.15.60:0/3070434305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a74073fb0 msgr2=0x7f3a740d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.611+0000 7f3a73e00640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a74073fb0 0x7f3a740d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.611+0000 7f3a73e00640 1 -- 172.21.15.60:0/3070434305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a74156550 con 0x7f3a7415f5a0 2024-09-16T15:47:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.611+0000 7f3a73e00640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a7415f5a0 0x7f3a7415b970 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009b30 tx=0x7f3a6c032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.612+0000 7f3a6be00640 1 -- 172.21.15.60:0/3070434305 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c037040 con 0x7f3a7415f5a0 2024-09-16T15:47:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.612+0000 7f3a6be00640 1 -- 172.21.15.60:0/3070434305 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a6c042070 con 0x7f3a7415f5a0 2024-09-16T15:47:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.612+0000 7f3a6be00640 1 -- 172.21.15.60:0/3070434305 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0042a0 con 0x7f3a7415f5a0 2024-09-16T15:47:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.612+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a740cf5e0 con 0x7f3a7415f5a0 2024-09-16T15:47:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.612+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a74144290 con 0x7f3a7415f5a0 2024-09-16T15:47:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.613+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7406f990 con 0x7f3a7415f5a0 2024-09-16T15:47:49.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.614+0000 7f3a6be00640 1 -- 172.21.15.60:0/3070434305 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f3a6c004440 con 0x7f3a7415f5a0 2024-09-16T15:47:49.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.615+0000 7f3a6be00640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3a50079270 0x7f3a5007b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:49.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.615+0000 7f3a6be00640 1 -- 172.21.15.60:0/3070434305 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f3a6c0ba1e0 con 0x7f3a7415f5a0 2024-09-16T15:47:49.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.615+0000 7f3a73400640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3a50079270 0x7f3a5007b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:49.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.616+0000 7f3a73400640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3a50079270 0x7f3a5007b730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3a64007fd0 tx=0x7f3a64009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:49.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.619+0000 7f3a6be00640 1 -- 172.21.15.60:0/3070434305 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3a6c084000 con 0x7f3a7415f5a0 2024-09-16T15:47:49.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.855+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a7409d570 con 0x7f3a7415f5a0 2024-09-16T15:47:49.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.856+0000 7f3a6be00640 1 -- 172.21.15.60:0/3070434305 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7f3a6c083900 con 0x7f3a7415f5a0 2024-09-16T15:47:49.858 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:49.858 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":113,"btime":"2024-09-16T15:47:36:757844+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7361,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1464936997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1464936997},{"type":"v1","addr":"172.21.15.60:6834","nonce":1464936997}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":7364,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1611348877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1611348877},{"type":"v1","addr":"172.21.15.60:6835","nonce":1611348877}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2321854052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2321854052},{"type":"v1","addr":"172.21.15.100:6834","nonce":2321854052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:47:26.164998+0000","modified":"2024-09-16T15:47:29.176697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"d","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3325202304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":3325202304},{"type":"v1","addr":"172.21.15.100:6835","nonce":3325202304}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":12}]} 2024-09-16T15:47:49.858 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 113 2024-09-16T15:47:49.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3a50079270 msgr2=0x7f3a5007b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3a50079270 0x7f3a5007b730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3a64007fd0 tx=0x7f3a64009040 comp rx=0 tx=0).stop 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a7415f5a0 msgr2=0x7f3a7415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a7415f5a0 0x7f3a7415b970 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009b30 tx=0x7f3a6c032c90 comp rx=0 tx=0).stop 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 shutdown_connections 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3a50079270 0x7f3a5007b730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a7415f5a0 0x7f3a7415b970 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a740ccf00 0x7f3a741f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 --2- 172.21.15.60:0/3070434305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a74073fb0 0x7f3a740d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 >> 172.21.15.60:0/3070434305 conn(0x7f3a740b1950 msgr2=0x7f3a7415dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 shutdown_connections 2024-09-16T15:47:49.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:49.858+0000 7f3a7a800640 1 -- 172.21.15.60:0/3070434305 wait complete. 2024-09-16T15:47:49.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:50.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.034+0000 7fe2f4200640 1 -- 172.21.15.60:0/3337527719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec0b2440 msgr2=0x7fe2ec06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.034+0000 7fe2f4200640 1 --2- 172.21.15.60:0/3337527719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec0b2440 0x7fe2ec06f160 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4005fd0 tx=0x7fe2e402a820 comp rx=0 tx=0).stop 2024-09-16T15:47:50.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.034+0000 7fe2f4200640 1 -- 172.21.15.60:0/3337527719 shutdown_connections 2024-09-16T15:47:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.034+0000 7fe2f4200640 1 --2- 172.21.15.60:0/3337527719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec0b2440 0x7fe2ec06f160 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.034+0000 7fe2f4200640 1 -- 172.21.15.60:0/3337527719 >> 172.21.15.60:0/3337527719 conn(0x7fe2ec0b1950 msgr2=0x7fe2ec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.034+0000 7fe2f4200640 1 -- 172.21.15.60:0/3337527719 shutdown_connections 2024-09-16T15:47:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.034+0000 7fe2f4200640 1 -- 172.21.15.60:0/3337527719 wait complete. 2024-09-16T15:47:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.036+0000 7fe2f4200640 1 Processor -- start 2024-09-16T15:47:50.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.036+0000 7fe2f4200640 1 -- start start 2024-09-16T15:47:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f4200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ec0b2440 0x7fe2ec1f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f4200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ec0ccf00 0x7fe2ec1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f4200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec15f5a0 0x7fe2ec09cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f4200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec074840 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f4200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec0746c0 con 0x7fe2ec0b2440 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f4200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ec06f6a0 con 0x7fe2ec0ccf00 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f2200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec15f5a0 0x7fe2ec09cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f2200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec15f5a0 0x7fe2ec09cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41640/0 (socket says 172.21.15.60:41640) 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f2200640 1 -- 172.21.15.60:0/3095737646 learned_addr learned my addr 172.21.15.60:0/3095737646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f0e00640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ec0ccf00 0x7fe2ec1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.037+0000 7fe2f1800640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ec0b2440 0x7fe2ec1f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f2200640 1 -- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ec0ccf00 msgr2=0x7fe2ec1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f2200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ec0ccf00 0x7fe2ec1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f2200640 1 -- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ec0b2440 msgr2=0x7fe2ec1f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f2200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ec0b2440 0x7fe2ec1f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f2200640 1 -- 172.21.15.60:0/3095737646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec09ba50 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f0e00640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ec0ccf00 0x7fe2ec1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f1800640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ec0b2440 0x7fe2ec1f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f2200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec15f5a0 0x7fe2ec09cdd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4006010 tx=0x7fe2e4032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2e9600640 1 -- 172.21.15.60:0/3095737646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4037040 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2e9600640 1 -- 172.21.15.60:0/3095737646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2e4042070 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2e9600640 1 -- 172.21.15.60:0/3095737646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40042a0 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec09bd30 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.038+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec09c2f0 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.039+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ec06f160 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.041+0000 7fe2e9600640 1 -- 172.21.15.60:0/3095737646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fe2e4004440 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.042+0000 7fe2e9600640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe2cc0792a0 0x7fe2cc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.042+0000 7fe2e9600640 1 -- 172.21.15.60:0/3095737646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fe2e40ba150 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.042+0000 7fe2f1800640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe2cc0792a0 0x7fe2cc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.043+0000 7fe2f1800640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe2cc0792a0 0x7fe2cc07b760 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006fd0 tx=0x7fe2e0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:50.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.046+0000 7fe2e9600640 1 -- 172.21.15.60:0/3095737646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe2e4083ff0 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.281+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2ec0ae900 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.281+0000 7fe2e9600640 1 -- 172.21.15.60:0/3095737646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7fe2e40838f0 con 0x7fe2ec15f5a0 2024-09-16T15:47:50.284 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:50.284 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":113,"btime":"2024-09-16T15:47:36:757844+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7361,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1464936997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1464936997},{"type":"v1","addr":"172.21.15.60:6834","nonce":1464936997}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":7364,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1611348877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1611348877},{"type":"v1","addr":"172.21.15.60:6835","nonce":1611348877}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2321854052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2321854052},{"type":"v1","addr":"172.21.15.100:6834","nonce":2321854052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:47:26.164998+0000","modified":"2024-09-16T15:47:29.176697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8853},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8853":{"gid":8853,"name":"d","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6835/3325202304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":3325202304},{"type":"v1","addr":"172.21.15.100:6835","nonce":3325202304}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8853,"qdb_cluster":[8853]},"id":12}]} 2024-09-16T15:47:50.284 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 113 2024-09-16T15:47:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.283+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe2cc0792a0 msgr2=0x7fe2cc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.283+0000 7fe2f4200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe2cc0792a0 0x7fe2cc07b760 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006fd0 tx=0x7fe2e0008040 comp rx=0 tx=0).stop 2024-09-16T15:47:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.283+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec15f5a0 msgr2=0x7fe2ec09cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.283+0000 7fe2f4200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec15f5a0 0x7fe2ec09cdd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4006010 tx=0x7fe2e4032c90 comp rx=0 tx=0).stop 2024-09-16T15:47:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.284+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 shutdown_connections 2024-09-16T15:47:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.284+0000 7fe2f4200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe2cc0792a0 0x7fe2cc07b760 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.284+0000 7fe2f4200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec15f5a0 0x7fe2ec09cdd0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.284+0000 7fe2f4200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2ec0ccf00 0x7fe2ec1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.284+0000 7fe2f4200640 1 --2- 172.21.15.60:0/3095737646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2ec0b2440 0x7fe2ec1f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.284+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 >> 172.21.15.60:0/3095737646 conn(0x7fe2ec0b1950 msgr2=0x7fe2ec0ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:50.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.284+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 shutdown_connections 2024-09-16T15:47:50.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.284+0000 7fe2f4200640 1 -- 172.21.15.60:0/3095737646 wait complete. 2024-09-16T15:47:50.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.460+0000 7fc7b3c00640 1 -- 172.21.15.60:0/2365450818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac0ccf00 msgr2=0x7fc7ac0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.460+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/2365450818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac0ccf00 0x7fc7ac0acd30 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac075130 tx=0x7fc79802a560 comp rx=0 tx=0).stop 2024-09-16T15:47:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.460+0000 7fc7b3c00640 1 -- 172.21.15.60:0/2365450818 shutdown_connections 2024-09-16T15:47:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.460+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/2365450818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac0ccf00 0x7fc7ac0acd30 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.460+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/2365450818 >> v2:172.21.15.100:3300/0 conn(0x7fc7ac15f5a0 0x7fc7ac0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.460+0000 7fc7b3c00640 1 -- 172.21.15.60:0/2365450818 >> 172.21.15.60:0/2365450818 conn(0x7fc7ac0b1950 msgr2=0x7fc7ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:50.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.460+0000 7fc7b3c00640 1 -- 172.21.15.60:0/2365450818 shutdown_connections 2024-09-16T15:47:50.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.460+0000 7fc7b3c00640 1 -- 172.21.15.60:0/2365450818 wait complete. 2024-09-16T15:47:50.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.462+0000 7fc7b3c00640 1 Processor -- start 2024-09-16T15:47:50.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.462+0000 7fc7b3c00640 1 -- start start 2024-09-16T15:47:50.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b3c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7ac0ad270 0x7fc7ac15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b3c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7ac0ccf00 0x7fc7ac1f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b3c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac15f5a0 0x7fc7ac1efd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b3c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac06f160 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b3c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac06f2e0 con 0x7fc7ac0ad270 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b3c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac06f460 con 0x7fc7ac0ccf00 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b1c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac15f5a0 0x7fc7ac1efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b1c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac15f5a0 0x7fc7ac1efd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41662/0 (socket says 172.21.15.60:41662) 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.463+0000 7fc7b1c00640 1 -- 172.21.15.60:0/1741276874 learned_addr learned my addr 172.21.15.60:0/1741276874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7b1200640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7ac0ad270 0x7fc7ac15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7abe00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7ac0ccf00 0x7fc7ac1f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7b1c00640 1 -- 172.21.15.60:0/1741276874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7ac0ccf00 msgr2=0x7fc7ac1f32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7b1c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7ac0ccf00 0x7fc7ac1f32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7b1c00640 1 -- 172.21.15.60:0/1741276874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7ac0ad270 msgr2=0x7fc7ac15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7b1c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7ac0ad270 0x7fc7ac15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7b1c00640 1 -- 172.21.15.60:0/1741276874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac1f3820 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7b1c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac15f5a0 0x7fc7ac1efd10 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400c7a0 tx=0x7fc7a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7a8c00640 1 -- 172.21.15.60:0/1741276874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4003b70 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7a8c00640 1 -- 172.21.15.60:0/1741276874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7a4003d30 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7a8c00640 1 -- 172.21.15.60:0/1741276874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400aab0 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.464+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac1f0250 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.465+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac0ced40 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.465+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac0ceb30 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.467+0000 7fc7a8c00640 1 -- 172.21.15.60:0/1741276874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fc7a400ac50 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.468+0000 7fc7a8c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc78c079270 0x7fc78c07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.468+0000 7fc7a8c00640 1 -- 172.21.15.60:0/1741276874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7fc7a4099350 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.468+0000 7fc7b1200640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc78c079270 0x7fc78c07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.468+0000 7fc7b1200640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc78c079270 0x7fc78c07b730 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009ac0 tx=0x7fc79c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:50.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.472+0000 7fc7a8c00640 1 -- 172.21.15.60:0/1741276874 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc7ac1f3820 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.670+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7ac06f9f0 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.671+0000 7fc7a8c00640 1 -- 172.21.15.60:0/1741276874 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+16783 (secure 0 0 0) 0x7fc7a40631f0 con 0x7fc7ac15f5a0 2024-09-16T15:47:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:50.674 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":161,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:47:27.726082+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:47:23.982547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-16T15:47:25.124723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.673+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc78c079270 msgr2=0x7fc78c07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.673+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc78c079270 0x7fc78c07b730 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009ac0 tx=0x7fc79c005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.673+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac15f5a0 msgr2=0x7fc7ac1efd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.673+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac15f5a0 0x7fc7ac1efd10 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400c7a0 tx=0x7fc7a400cc60 comp rx=0 tx=0).stop 2024-09-16T15:47:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.674+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 shutdown_connections 2024-09-16T15:47:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.674+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc78c079270 0x7fc78c07b730 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.674+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7ac15f5a0 0x7fc7ac1efd10 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.674+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7ac0ccf00 0x7fc7ac1f32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.674+0000 7fc7b3c00640 1 --2- 172.21.15.60:0/1741276874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7ac0ad270 0x7fc7ac15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.674+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 >> 172.21.15.60:0/1741276874 conn(0x7fc7ac0b1950 msgr2=0x7fc7ac15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.674+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 shutdown_connections 2024-09-16T15:47:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.674+0000 7fc7b3c00640 1 -- 172.21.15.60:0/1741276874 wait complete. 2024-09-16T15:47:50.692 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:47:50.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.855+0000 7f1061400640 1 -- 172.21.15.60:0/2738142264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 msgr2=0x7f105c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.855+0000 7f1061400640 1 --2- 172.21.15.60:0/2738142264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 0x7f105c0b2440 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f1048009830 tx=0x7f104802a480 comp rx=0 tx=0).stop 2024-09-16T15:47:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.855+0000 7f1061400640 1 -- 172.21.15.60:0/2738142264 shutdown_connections 2024-09-16T15:47:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.855+0000 7f1061400640 1 --2- 172.21.15.60:0/2738142264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 0x7f105c0b2440 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.855+0000 7f1061400640 1 -- 172.21.15.60:0/2738142264 >> 172.21.15.60:0/2738142264 conn(0x7f105c0b1950 msgr2=0x7f105c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.855+0000 7f1061400640 1 -- 172.21.15.60:0/2738142264 shutdown_connections 2024-09-16T15:47:50.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.855+0000 7f1061400640 1 -- 172.21.15.60:0/2738142264 wait complete. 2024-09-16T15:47:50.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.857+0000 7f1061400640 1 Processor -- start 2024-09-16T15:47:50.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.857+0000 7f1061400640 1 -- start start 2024-09-16T15:47:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1061400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c073fb0 0x7f105c15a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1061400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 0x7f105c0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1061400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f5a0 0x7f105c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1061400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f105c06f9f0 con 0x7f105c0ccf00 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1061400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f105c06f870 con 0x7f105c073fb0 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1061400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f105c074420 con 0x7f105c15f5a0 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1059e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 0x7f105c0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1059e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 0x7f105c0cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41684/0 (socket says 172.21.15.60:41684) 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f1059e00640 1 -- 172.21.15.60:0/2301943135 learned_addr learned my addr 172.21.15.60:0/2301943135 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f105b200640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f5a0 0x7f105c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.858+0000 7f105a800640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c073fb0 0x7f105c15a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1059e00640 1 -- 172.21.15.60:0/2301943135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f5a0 msgr2=0x7f105c1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1059e00640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f5a0 0x7f105c1572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1059e00640 1 -- 172.21.15.60:0/2301943135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c073fb0 msgr2=0x7f105c15a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1059e00640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c073fb0 0x7f105c15a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1059e00640 1 -- 172.21.15.60:0/2301943135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105c1f6370 con 0x7f105c0ccf00 2024-09-16T15:47:50.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1059e00640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 0x7f105c0cf5c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1048009950 tx=0x7f1048032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:50.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1052a00640 1 -- 172.21.15.60:0/2301943135 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048040070 con 0x7f105c0ccf00 2024-09-16T15:47:50.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1052a00640 1 -- 172.21.15.60:0/2301943135 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1048005310 con 0x7f105c0ccf00 2024-09-16T15:47:50.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1052a00640 1 -- 172.21.15.60:0/2301943135 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10480047a0 con 0x7f105c0ccf00 2024-09-16T15:47:50.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105c1f6650 con 0x7f105c0ccf00 2024-09-16T15:47:50.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.859+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105c1f28a0 con 0x7f105c0ccf00 2024-09-16T15:47:50.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.860+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105c1f6820 con 0x7f105c0ccf00 2024-09-16T15:47:50.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.862+0000 7f1052a00640 1 -- 172.21.15.60:0/2301943135 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f1048039040 con 0x7f105c0ccf00 2024-09-16T15:47:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.862+0000 7f1052a00640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c079270 0x7f103c07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.863+0000 7f1052a00640 1 -- 172.21.15.60:0/2301943135 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5737+0+0 (secure 0 0 0) 0x7f10480ba470 con 0x7f105c0ccf00 2024-09-16T15:47:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.863+0000 7f105a800640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c079270 0x7f103c07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.863+0000 7f105a800640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c079270 0x7f103c07b730 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:50.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:50.867+0000 7f1052a00640 1 -- 172.21.15.60:0/2301943135 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1048084590 con 0x7f105c0ccf00 2024-09-16T15:47:51.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.099+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f105c0b2440 con 0x7f105c0ccf00 2024-09-16T15:47:51.102 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:47:51.100+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:47:51.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.102+0000 7f1052a00640 1 -- 172.21.15.60:0/2301943135 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) ==== 161+0+0 (secure 0 0 0) 0x7f1048083e90 con 0x7f105c0ccf00 2024-09-16T15:47:51.104 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:47:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c079270 msgr2=0x7f103c07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c079270 0x7f103c07b730 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c008040 comp rx=0 tx=0).stop 2024-09-16T15:47:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 msgr2=0x7f105c0cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 0x7f105c0cf5c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1048009950 tx=0x7f1048032c10 comp rx=0 tx=0).stop 2024-09-16T15:47:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 shutdown_connections 2024-09-16T15:47:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f103c079270 0x7f103c07b730 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15f5a0 0x7f105c1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0ccf00 0x7f105c0cf5c0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.105+0000 7f1061400640 1 --2- 172.21.15.60:0/2301943135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c073fb0 0x7f105c15a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.106+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 >> 172.21.15.60:0/2301943135 conn(0x7f105c0b1950 msgr2=0x7f105c09caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.106+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 shutdown_connections 2024-09-16T15:47:51.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.106+0000 7f1061400640 1 -- 172.21.15.60:0/2301943135 wait complete. 2024-09-16T15:47:51.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:47:51.174 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:47:51.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.285+0000 7f227b000640 1 -- 172.21.15.60:0/494178907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227406f870 msgr2=0x7f2274164ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:51.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.285+0000 7f227b000640 1 --2- 172.21.15.60:0/494178907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227406f870 0x7f2274164ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f226c00a970 tx=0x7f226c02b8d0 comp rx=0 tx=0).stop 2024-09-16T15:47:51.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.285+0000 7f227b000640 1 -- 172.21.15.60:0/494178907 shutdown_connections 2024-09-16T15:47:51.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.285+0000 7f227b000640 1 --2- 172.21.15.60:0/494178907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227406f870 0x7f2274164ee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:51.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.285+0000 7f227b000640 1 -- 172.21.15.60:0/494178907 >> 172.21.15.60:0/494178907 conn(0x7f22740b1840 msgr2=0x7f22740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:51.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.285+0000 7f227b000640 1 -- 172.21.15.60:0/494178907 shutdown_connections 2024-09-16T15:47:51.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.285+0000 7f227b000640 1 -- 172.21.15.60:0/494178907 wait complete. 2024-09-16T15:47:51.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.286+0000 7f227b000640 1 Processor -- start 2024-09-16T15:47:51.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.286+0000 7f227b000640 1 -- start start 2024-09-16T15:47:51.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.287+0000 7f227b000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227406f870 0x7f2274153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:51.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.287+0000 7f227b000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740b4bd0 0x7f22741642a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:51.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.287+0000 7f227b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22741437c0 0x7f227412dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:51.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.287+0000 7f227b000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f22740b47e0 con 0x7f22740b4bd0 2024-09-16T15:47:51.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.287+0000 7f227b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2274165420 con 0x7f22741437c0 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.287+0000 7f227b000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f22740ad200 con 0x7f227406f870 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740b4bd0 0x7f22741642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740b4bd0 0x7f22741642a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41688/0 (socket says 172.21.15.60:41688) 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 -- 172.21.15.60:0/3724839858 learned_addr learned my addr 172.21.15.60:0/3724839858 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f2279000640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22741437c0 0x7f227412dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f2273e00640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227406f870 0x7f2274153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 -- 172.21.15.60:0/3724839858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227406f870 msgr2=0x7f2274153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227406f870 0x7f2274153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 -- 172.21.15.60:0/3724839858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22741437c0 msgr2=0x7f227412dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22741437c0 0x7f227412dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 -- 172.21.15.60:0/3724839858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2274154320 con 0x7f22740b4bd0 2024-09-16T15:47:51.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.288+0000 7f226b400640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740b4bd0 0x7f22741642a0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f225c00c4c0 tx=0x7f225c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:51.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.289+0000 7f2270c00640 1 -- 172.21.15.60:0/3724839858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c0052d0 con 0x7f22740b4bd0 2024-09-16T15:47:51.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.289+0000 7f2270c00640 1 -- 172.21.15.60:0/3724839858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f225c005490 con 0x7f22740b4bd0 2024-09-16T15:47:51.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.289+0000 7f2270c00640 1 -- 172.21.15.60:0/3724839858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c012890 con 0x7f22740b4bd0 2024-09-16T15:47:51.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.289+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22740ac8d0 con 0x7f22740b4bd0 2024-09-16T15:47:51.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.289+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22740ace90 con 0x7f22740b4bd0 2024-09-16T15:47:51.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.290+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22740b3ec0 con 0x7f22740b4bd0 2024-09-16T15:47:51.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.291+0000 7f2270c00640 1 -- 172.21.15.60:0/3724839858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f225c012a30 con 0x7f22740b4bd0 2024-09-16T15:47:51.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.293+0000 7f2270c00640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2250079270 0x7f225007b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:51.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.293+0000 7f2270c00640 1 -- 172.21.15.60:0/3724839858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5823+0+0 (secure 0 0 0) 0x7f225c098ec0 con 0x7f22740b4bd0 2024-09-16T15:47:51.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.293+0000 7f2273e00640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2250079270 0x7f225007b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:51.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.294+0000 7f2273e00640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2250079270 0x7f225007b730 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f22640017a0 tx=0x7f2264001730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.297+0000 7f2270c00640 1 -- 172.21.15.60:0/3724839858 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f225c062710 con 0x7f22740b4bd0 2024-09-16T15:47:51.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:51.534+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f22740cd570 con 0x7f22740b4bd0 2024-09-16T15:47:52.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.104+0000 7f2270c00640 1 -- 172.21.15.60:0/3724839858 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) ==== 104+0+0 (secure 0 0 0) 0x7f225c067140 con 0x7f22740b4bd0 2024-09-16T15:47:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2250079270 msgr2=0x7f225007b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2250079270 0x7f225007b730 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f22640017a0 tx=0x7f2264001730 comp rx=0 tx=0).stop 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740b4bd0 msgr2=0x7f22741642a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740b4bd0 0x7f22741642a0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f225c00c4c0 tx=0x7f225c00c980 comp rx=0 tx=0).stop 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 shutdown_connections 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2250079270 0x7f225007b730 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22741437c0 0x7f227412dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22740b4bd0 0x7f22741642a0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 --2- 172.21.15.60:0/3724839858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f227406f870 0x7f2274153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 >> 172.21.15.60:0/3724839858 conn(0x7f22740b1840 msgr2=0x7f2274074080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 shutdown_connections 2024-09-16T15:47:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.107+0000 7f227b000640 1 -- 172.21.15.60:0/3724839858 wait complete. 2024-09-16T15:47:52.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:47:52.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.291+0000 7f7d18c00640 1 -- 172.21.15.60:0/2042197303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1406e9d0 msgr2=0x7f7d140b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.291+0000 7f7d18c00640 1 --2- 172.21.15.60:0/2042197303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1406e9d0 0x7f7d140b2be0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f7d140af820 tx=0x7f7cfc02a560 comp rx=0 tx=0).stop 2024-09-16T15:47:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.291+0000 7f7d18c00640 1 -- 172.21.15.60:0/2042197303 shutdown_connections 2024-09-16T15:47:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.291+0000 7f7d18c00640 1 --2- 172.21.15.60:0/2042197303 >> v2:172.21.15.100:3300/0 conn(0x7f7d140b3120 0x7f7d140cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.291+0000 7f7d18c00640 1 --2- 172.21.15.60:0/2042197303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d1406e9d0 0x7f7d140b2be0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.291+0000 7f7d18c00640 1 -- 172.21.15.60:0/2042197303 >> 172.21.15.60:0/2042197303 conn(0x7f7d140b18e0 msgr2=0x7f7d140b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.291+0000 7f7d18c00640 1 -- 172.21.15.60:0/2042197303 shutdown_connections 2024-09-16T15:47:52.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.291+0000 7f7d18c00640 1 -- 172.21.15.60:0/2042197303 wait complete. 2024-09-16T15:47:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.293+0000 7f7d18c00640 1 Processor -- start 2024-09-16T15:47:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.293+0000 7f7d18c00640 1 -- start start 2024-09-16T15:47:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d18c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1406e9d0 0x7f7d1415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d18c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d140b3120 0x7f7d14158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:52.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d18c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d14158900 0x7f7d14152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d18c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1406ee60 con 0x7f7d140b3120 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d18c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7d140ccf20 con 0x7f7d14158900 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d18c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7d14152c50 con 0x7f7d1406e9d0 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d09400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d140b3120 0x7f7d14158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d09400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d140b3120 0x7f7d14158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41702/0 (socket says 172.21.15.60:41702) 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d09400640 1 -- 172.21.15.60:0/3404858874 learned_addr learned my addr 172.21.15.60:0/3404858874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d11e00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1406e9d0 0x7f7d1415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.294+0000 7f7d12800640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d14158900 0x7f7d14152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d09400640 1 -- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1406e9d0 msgr2=0x7f7d1415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d09400640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1406e9d0 0x7f7d1415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d09400640 1 -- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d14158900 msgr2=0x7f7d14152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d09400640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d14158900 0x7f7d14152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d09400640 1 -- 172.21.15.60:0/3404858874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d140ca980 con 0x7f7d140b3120 2024-09-16T15:47:52.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d12800640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d14158900 0x7f7d14152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:52.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d09400640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d140b3120 0x7f7d14158eb0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009850 tx=0x7f7cfc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:52.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d0aa00640 1 -- 172.21.15.60:0/3404858874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc03f070 con 0x7f7d140b3120 2024-09-16T15:47:52.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d0aa00640 1 -- 172.21.15.60:0/3404858874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cfc0047a0 con 0x7f7d140b3120 2024-09-16T15:47:52.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d0aa00640 1 -- 172.21.15.60:0/3404858874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc035030 con 0x7f7d140b3120 2024-09-16T15:47:52.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d140cac60 con 0x7f7d140b3120 2024-09-16T15:47:52.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.295+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141f55e0 con 0x7f7d140b3120 2024-09-16T15:47:52.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.296+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d14069360 con 0x7f7d140b3120 2024-09-16T15:47:52.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.298+0000 7f7d0aa00640 1 -- 172.21.15.60:0/3404858874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f7cfc042070 con 0x7f7d140b3120 2024-09-16T15:47:52.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.299+0000 7f7d0aa00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7cf00792a0 0x7f7cf007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:52.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.299+0000 7f7d0aa00640 1 -- 172.21.15.60:0/3404858874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5823+0+0 (secure 0 0 0) 0x7f7cfc0ba180 con 0x7f7d140b3120 2024-09-16T15:47:52.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.299+0000 7f7d11e00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7cf00792a0 0x7f7cf007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:52.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.300+0000 7f7d11e00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7cf00792a0 0x7f7cf007b760 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7d1409df30 tx=0x7f7d04009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:52.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.303+0000 7f7d0aa00640 1 -- 172.21.15.60:0/3404858874 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7cfc0839d0 con 0x7f7d140b3120 2024-09-16T15:47:52.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:52.498+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d140cd3d0 con 0x7f7d140b3120 2024-09-16T15:47:53.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.109+0000 7f7d0aa00640 1 -- 172.21.15.60:0/3404858874 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v163) ==== 181+0+0 (secure 0 0 0) 0x7f7cfc088400 con 0x7f7d140b3120 2024-09-16T15:47:53.111 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:47:53.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7cf00792a0 msgr2=0x7f7cf007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7cf00792a0 0x7f7cf007b760 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7d1409df30 tx=0x7f7d04009040 comp rx=0 tx=0).stop 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d140b3120 msgr2=0x7f7d14158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d140b3120 0x7f7d14158eb0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009850 tx=0x7f7cfc003820 comp rx=0 tx=0).stop 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 shutdown_connections 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7cf00792a0 0x7f7cf007b760 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d14158900 0x7f7d14152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d140b3120 0x7f7d14158eb0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 --2- 172.21.15.60:0/3404858874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d1406e9d0 0x7f7d1415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.113+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 >> 172.21.15.60:0/3404858874 conn(0x7f7d140b18e0 msgr2=0x7f7d140af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.114+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 shutdown_connections 2024-09-16T15:47:53.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.114+0000 7f7d18c00640 1 -- 172.21.15.60:0/3404858874 wait complete. 2024-09-16T15:47:53.135 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:47:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.301+0000 7f5d26c00640 1 -- 172.21.15.60:0/100724600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d2006e6f0 msgr2=0x7f5d2006eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.301+0000 7f5d26c00640 1 --2- 172.21.15.60:0/100724600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d2006e6f0 0x7f5d2006eba0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d18005fd0 tx=0x7f5d1802a810 comp rx=0 tx=0).stop 2024-09-16T15:47:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.301+0000 7f5d26c00640 1 -- 172.21.15.60:0/100724600 shutdown_connections 2024-09-16T15:47:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.301+0000 7f5d26c00640 1 --2- 172.21.15.60:0/100724600 >> v2:172.21.15.100:3300/0 conn(0x7f5d20073de0 0x7f5d20074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.301+0000 7f5d26c00640 1 --2- 172.21.15.60:0/100724600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d2006e6f0 0x7f5d2006eba0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.301+0000 7f5d26c00640 1 -- 172.21.15.60:0/100724600 >> 172.21.15.60:0/100724600 conn(0x7f5d200b18e0 msgr2=0x7f5d200b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:53.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.301+0000 7f5d26c00640 1 -- 172.21.15.60:0/100724600 shutdown_connections 2024-09-16T15:47:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.301+0000 7f5d26c00640 1 -- 172.21.15.60:0/100724600 wait complete. 2024-09-16T15:47:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.303+0000 7f5d26c00640 1 Processor -- start 2024-09-16T15:47:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.303+0000 7f5d26c00640 1 -- start start 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d26c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2006e6f0 0x7f5d20159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d26c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d20073de0 0x7f5d201f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d26c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2015f9a0 0x7f5d20152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d26c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2006f0e0 con 0x7f5d2015f9a0 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d26c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5d200cd5e0 con 0x7f5d2006e6f0 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d26c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2006f260 con 0x7f5d20073de0 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d24c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2015f9a0 0x7f5d20152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d24c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2015f9a0 0x7f5d20152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41708/0 (socket says 172.21.15.60:41708) 2024-09-16T15:47:53.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d24c00640 1 -- 172.21.15.60:0/552958437 learned_addr learned my addr 172.21.15.60:0/552958437 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.304+0000 7f5d1fe00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2006e6f0 0x7f5d20159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.305+0000 7f5d17400640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d20073de0 0x7f5d201f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.305+0000 7f5d24c00640 1 -- 172.21.15.60:0/552958437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d20073de0 msgr2=0x7f5d201f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.305+0000 7f5d24c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d20073de0 0x7f5d201f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.305+0000 7f5d24c00640 1 -- 172.21.15.60:0/552958437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2006e6f0 msgr2=0x7f5d20159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.305+0000 7f5d24c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2006e6f0 0x7f5d20159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.305+0000 7f5d24c00640 1 -- 172.21.15.60:0/552958437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2015a820 con 0x7f5d2015f9a0 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.305+0000 7f5d24c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2015f9a0 0x7f5d20152170 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009b30 tx=0x7f5d180315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.305+0000 7f5d1cc00640 1 -- 172.21.15.60:0/552958437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18041020 con 0x7f5d2015f9a0 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.306+0000 7f5d1cc00640 1 -- 172.21.15.60:0/552958437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d18031ac0 con 0x7f5d2015f9a0 2024-09-16T15:47:53.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.306+0000 7f5d1cc00640 1 -- 172.21.15.60:0/552958437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18037040 con 0x7f5d2015f9a0 2024-09-16T15:47:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.306+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2015ade0 con 0x7f5d2015f9a0 2024-09-16T15:47:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.306+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2015b3a0 con 0x7f5d2015f9a0 2024-09-16T15:47:53.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.307+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d20069360 con 0x7f5d2015f9a0 2024-09-16T15:47:53.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.308+0000 7f5d1cc00640 1 -- 172.21.15.60:0/552958437 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f5d18044020 con 0x7f5d2015f9a0 2024-09-16T15:47:53.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.309+0000 7f5d1cc00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cfc079270 0x7f5cfc07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.310+0000 7f5d1cc00640 1 -- 172.21.15.60:0/552958437 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 5385+0+0 (secure 0 0 0) 0x7f5d180b9e30 con 0x7f5d2015f9a0 2024-09-16T15:47:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.310+0000 7f5d1fe00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cfc079270 0x7f5cfc07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.310+0000 7f5d1fe00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cfc079270 0x7f5cfc07b730 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5d10001720 tx=0x7f5d10007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:53.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.314+0000 7f5d1cc00640 1 -- 172.21.15.60:0/552958437 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5d1804b070 con 0x7f5d2015f9a0 2024-09-16T15:47:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:53.511+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5d2006eba0 con 0x7f5d2015f9a0 2024-09-16T15:47:54.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.111+0000 7f5d1cc00640 1 -- 172.21.15.60:0/552958437 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v164) ==== 169+0+0 (secure 0 0 0) 0x7f5d18083680 con 0x7f5d2015f9a0 2024-09-16T15:47:54.113 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:47:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cfc079270 msgr2=0x7f5cfc07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cfc079270 0x7f5cfc07b730 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5d10001720 tx=0x7f5d10007e90 comp rx=0 tx=0).stop 2024-09-16T15:47:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2015f9a0 msgr2=0x7f5d20152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2015f9a0 0x7f5d20152170 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009b30 tx=0x7f5d180315d0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 shutdown_connections 2024-09-16T15:47:54.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cfc079270 0x7f5cfc07b730 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d2015f9a0 0x7f5d20152170 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d20073de0 0x7f5d201f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 --2- 172.21.15.60:0/552958437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d2006e6f0 0x7f5d20159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.115+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 >> 172.21.15.60:0/552958437 conn(0x7f5d200b18e0 msgr2=0x7f5d2015ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:54.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.116+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 shutdown_connections 2024-09-16T15:47:54.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.117+0000 7f5d26c00640 1 -- 172.21.15.60:0/552958437 wait complete. 2024-09-16T15:47:54.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:47:54.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.302+0000 7fdf17200640 1 -- 172.21.15.60:0/2302996482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf100b5370 msgr2=0x7fdf100b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.302+0000 7fdf17200640 1 --2- 172.21.15.60:0/2302996482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf100b5370 0x7fdf100b3bc0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fdf100d18c0 tx=0x7fdf0002a560 comp rx=0 tx=0).stop 2024-09-16T15:47:54.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.302+0000 7fdf17200640 1 -- 172.21.15.60:0/2302996482 shutdown_connections 2024-09-16T15:47:54.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.303+0000 7fdf17200640 1 --2- 172.21.15.60:0/2302996482 >> v2:172.21.15.100:3300/0 conn(0x7fdf100b4100 0x7fdf100b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.303+0000 7fdf17200640 1 --2- 172.21.15.60:0/2302996482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf100b5370 0x7fdf100b3bc0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.303+0000 7fdf17200640 1 -- 172.21.15.60:0/2302996482 >> 172.21.15.60:0/2302996482 conn(0x7fdf100d3df0 msgr2=0x7fdf100d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:54.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.303+0000 7fdf17200640 1 -- 172.21.15.60:0/2302996482 shutdown_connections 2024-09-16T15:47:54.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.303+0000 7fdf17200640 1 -- 172.21.15.60:0/2302996482 wait complete. 2024-09-16T15:47:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.304+0000 7fdf17200640 1 Processor -- start 2024-09-16T15:47:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.304+0000 7fdf17200640 1 -- start start 2024-09-16T15:47:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf17200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100b4100 0x7fdf101578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf17200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf100b5370 0x7fdf101570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf17200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf10161440 0x7fdf100caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf17200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdf100b2040 con 0x7fdf10161440 2024-09-16T15:47:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf17200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdf100b1ec0 con 0x7fdf100b4100 2024-09-16T15:47:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf17200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdf100b21c0 con 0x7fdf100b5370 2024-09-16T15:47:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf16800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf10161440 0x7fdf100caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf16800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf10161440 0x7fdf100caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41722/0 (socket says 172.21.15.60:41722) 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.305+0000 7fdf16800640 1 -- 172.21.15.60:0/2766992040 learned_addr learned my addr 172.21.15.60:0/2766992040 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf15e00640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100b4100 0x7fdf101578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf15400640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf100b5370 0x7fdf101570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf16800640 1 -- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf100b5370 msgr2=0x7fdf101570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf16800640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf100b5370 0x7fdf101570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf16800640 1 -- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100b4100 msgr2=0x7fdf101578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf16800640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100b4100 0x7fdf101578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf16800640 1 -- 172.21.15.60:0/2766992040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf100b9220 con 0x7fdf10161440 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf15e00640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100b4100 0x7fdf101578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf16800640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf10161440 0x7fdf100caec0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00ead0 tx=0x7fdf0c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf0a000640 1 -- 172.21.15.60:0/2766992040 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00cde0 con 0x7fdf10161440 2024-09-16T15:47:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf0a000640 1 -- 172.21.15.60:0/2766992040 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf0c004c10 con 0x7fdf10161440 2024-09-16T15:47:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf0a000640 1 -- 172.21.15.60:0/2766992040 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c005020 con 0x7fdf10161440 2024-09-16T15:47:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.306+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf100b9500 con 0x7fdf10161440 2024-09-16T15:47:54.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.307+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf10157620 con 0x7fdf10161440 2024-09-16T15:47:54.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.308+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf100b1770 con 0x7fdf10161440 2024-09-16T15:47:54.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.309+0000 7fdf0a000640 1 -- 172.21.15.60:0/2766992040 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fdf0c0051c0 con 0x7fdf10161440 2024-09-16T15:47:54.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.310+0000 7fdf0a000640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdef4079270 0x7fdef407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.310+0000 7fdf0a000640 1 -- 172.21.15.60:0/2766992040 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7fdf0c099e30 con 0x7fdf10161440 2024-09-16T15:47:54.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.310+0000 7fdf15e00640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdef4079270 0x7fdef407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.310+0000 7fdf15e00640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdef4079270 0x7fdef407b730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf040016a0 tx=0x7fdf04009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.314+0000 7fdf0a000640 1 -- 172.21.15.60:0/2766992040 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdf0c016030 con 0x7fdf10161440 2024-09-16T15:47:54.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.519+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fdf100b96d0 con 0x7fdf10161440 2024-09-16T15:47:54.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.523+0000 7fdf0a000640 1 -- 172.21.15.60:0/2766992040 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7fdf0c063cd0 con 0x7fdf10161440 2024-09-16T15:47:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdef4079270 msgr2=0x7fdef407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdef4079270 0x7fdef407b730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf040016a0 tx=0x7fdf04009040 comp rx=0 tx=0).stop 2024-09-16T15:47:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf10161440 msgr2=0x7fdf100caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf10161440 0x7fdf100caec0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00ead0 tx=0x7fdf0c00ef90 comp rx=0 tx=0).stop 2024-09-16T15:47:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 shutdown_connections 2024-09-16T15:47:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdef4079270 0x7fdef407b730 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf10161440 0x7fdf100caec0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf100b5370 0x7fdf101570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 --2- 172.21.15.60:0/2766992040 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf100b4100 0x7fdf101578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.525+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 >> 172.21.15.60:0/2766992040 conn(0x7fdf100d3df0 msgr2=0x7fdf1006ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.526+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 shutdown_connections 2024-09-16T15:47:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.526+0000 7fdf17200640 1 -- 172.21.15.60:0/2766992040 wait complete. 2024-09-16T15:47:54.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-16T15:47:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.708+0000 7f2663000640 1 -- 172.21.15.60:0/4058970256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2640004d90 msgr2=0x7f2640005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.708+0000 7f2663000640 1 --2- 172.21.15.60:0/4058970256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2640004d90 0x7f2640005210 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f265400c610 tx=0x7f2640005830 comp rx=0 tx=0).stop 2024-09-16T15:47:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.708+0000 7f2661000640 1 -- 172.21.15.60:0/4058970256 reap_dead start 2024-09-16T15:47:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.708+0000 7f2663000640 1 -- 172.21.15.60:0/4058970256 shutdown_connections 2024-09-16T15:47:54.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.708+0000 7f2663000640 1 -- 172.21.15.60:0/4058970256 >> 172.21.15.60:0/4058970256 conn(0x7f265c0b2230 msgr2=0x7f265c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.708+0000 7f2663000640 1 -- 172.21.15.60:0/4058970256 shutdown_connections 2024-09-16T15:47:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.708+0000 7f2663000640 1 -- 172.21.15.60:0/4058970256 wait complete. 2024-09-16T15:47:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.709+0000 7f2663000640 1 Processor -- start 2024-09-16T15:47:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.710+0000 7f2663000640 1 -- start start 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.710+0000 7f2663000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2640004d90 0x7f265c13d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.710+0000 7f2663000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f265c13d580 0x7f265c1f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.710+0000 7f2663000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f265c1f8880 0x7f265c0d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.710+0000 7f2663000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f265c06d180 con 0x7f265c1f8880 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.710+0000 7f2663000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f265c06d000 con 0x7f2640004d90 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.710+0000 7f2663000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f265c06d300 con 0x7f265c13d580 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f265c1f8880 0x7f265c0d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f265c1f8880 0x7f265c0d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41738/0 (socket says 172.21.15.60:41738) 2024-09-16T15:47:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 -- 172.21.15.60:0/2094336321 learned_addr learned my addr 172.21.15.60:0/2094336321 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f265be00640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2640004d90 0x7f265c13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f265b400640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f265c13d580 0x7f265c1f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 -- 172.21.15.60:0/2094336321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f265c13d580 msgr2=0x7f265c1f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f265c13d580 0x7f265c1f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 -- 172.21.15.60:0/2094336321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2640004d90 msgr2=0x7f265c13d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2640004d90 0x7f265c13d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 -- 172.21.15.60:0/2094336321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c1f4720 con 0x7f265c1f8880 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.711+0000 7f2661000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f265c1f8880 0x7f265c0d0580 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f265400c700 tx=0x7f2654001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.712+0000 7f2653e00640 1 -- 172.21.15.60:0/2094336321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654039070 con 0x7f265c1f8880 2024-09-16T15:47:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.712+0000 7f2653e00640 1 -- 172.21.15.60:0/2094336321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2654034970 con 0x7f265c1f8880 2024-09-16T15:47:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.712+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c1f49a0 con 0x7f265c1f8880 2024-09-16T15:47:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.712+0000 7f2653e00640 1 -- 172.21.15.60:0/2094336321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654006d20 con 0x7f265c1f8880 2024-09-16T15:47:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.712+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c1f4f60 con 0x7f265c1f8880 2024-09-16T15:47:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.713+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c0cf3b0 con 0x7f265c1f8880 2024-09-16T15:47:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.714+0000 7f2653e00640 1 -- 172.21.15.60:0/2094336321 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f265402b670 con 0x7f265c1f8880 2024-09-16T15:47:54.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.715+0000 7f2653e00640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f263c079270 0x7f263c07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.715+0000 7f2653e00640 1 -- 172.21.15.60:0/2094336321 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7f26540b9320 con 0x7f265c1f8880 2024-09-16T15:47:54.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.715+0000 7f265be00640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f263c079270 0x7f263c07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.716+0000 7f265be00640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f263c079270 0x7f263c07b730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f264c000f60 tx=0x7f264c000ed0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.719+0000 7f2653e00640 1 -- 172.21.15.60:0/2094336321 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f265c1f4720 con 0x7f265c1f8880 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:54.986+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f265c13dc90 con 0x7f265c1f8880 2024-09-16T15:47:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.116+0000 7f2653e00640 1 -- 172.21.15.60:0/2094336321 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v513) ==== 145+0+0 (secure 0 0 0) 0x7f2654083130 con 0x7f265c1f8880 2024-09-16T15:47:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f263c079270 msgr2=0x7f263c07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f263c079270 0x7f263c07b730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f264c000f60 tx=0x7f264c000ed0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f265c1f8880 msgr2=0x7f265c0d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f265c1f8880 0x7f265c0d0580 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f265400c700 tx=0x7f2654001ac0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 shutdown_connections 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f263c079270 0x7f263c07b730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f265c1f8880 0x7f265c0d0580 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f265c13d580 0x7f265c1f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 --2- 172.21.15.60:0/2094336321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2640004d90 0x7f265c13d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 >> 172.21.15.60:0/2094336321 conn(0x7f265c0b2230 msgr2=0x7f265c15e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.119+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 shutdown_connections 2024-09-16T15:47:55.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.120+0000 7f2663000640 1 -- 172.21.15.60:0/2094336321 wait complete. 2024-09-16T15:47:55.137 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-16T15:47:55.137 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-16T15:47:55.138 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-16T15:47:55.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:47:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.296+0000 7fe7fc200640 1 -- 172.21.15.60:0/4283668910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ccf00 msgr2=0x7fe7f40acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.296+0000 7fe7fc200640 1 --2- 172.21.15.60:0/4283668910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ccf00 0x7fe7f40acd30 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0005fd0 tx=0x7fe7e002a540 comp rx=0 tx=0).stop 2024-09-16T15:47:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.296+0000 7fe7fc200640 1 -- 172.21.15.60:0/4283668910 shutdown_connections 2024-09-16T15:47:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.296+0000 7fe7fc200640 1 --2- 172.21.15.60:0/4283668910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ccf00 0x7fe7f40acd30 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.296+0000 7fe7fc200640 1 --2- 172.21.15.60:0/4283668910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f415f5a0 0x7fe7f40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.296+0000 7fe7fc200640 1 -- 172.21.15.60:0/4283668910 >> 172.21.15.60:0/4283668910 conn(0x7fe7f40b1950 msgr2=0x7fe7f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.296+0000 7fe7fc200640 1 -- 172.21.15.60:0/4283668910 shutdown_connections 2024-09-16T15:47:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.296+0000 7fe7fc200640 1 -- 172.21.15.60:0/4283668910 wait complete. 2024-09-16T15:47:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.298+0000 7fe7fc200640 1 Processor -- start 2024-09-16T15:47:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.298+0000 7fe7fc200640 1 -- start start 2024-09-16T15:47:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7fc200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ad270 0x7fe7f4145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7fc200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f40ccf00 0x7fe7f40ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7fc200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f415f5a0 0x7fe7f41f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7fc200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f406f2e0 con 0x7fe7f40ad270 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7fc200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f406f160 con 0x7fe7f415f5a0 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7fc200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f406f460 con 0x7fe7f40ccf00 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7f9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ad270 0x7fe7f4145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7f9800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ad270 0x7fe7f4145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41750/0 (socket says 172.21.15.60:41750) 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7f9800640 1 -- 172.21.15.60:0/1947682312 learned_addr learned my addr 172.21.15.60:0/1947682312 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7f8e00640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f40ccf00 0x7fe7f40ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.299+0000 7fe7fa200640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f415f5a0 0x7fe7f41f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f9800640 1 -- 172.21.15.60:0/1947682312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f40ccf00 msgr2=0x7fe7f40ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f9800640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f40ccf00 0x7fe7f40ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f9800640 1 -- 172.21.15.60:0/1947682312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f415f5a0 msgr2=0x7fe7f41f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f9800640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f415f5a0 0x7fe7f41f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f9800640 1 -- 172.21.15.60:0/1947682312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f40cef70 con 0x7fe7f40ad270 2024-09-16T15:47:55.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f9800640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ad270 0x7fe7f4145970 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800d770 tx=0x7fe7e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:55.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f1600640 1 -- 172.21.15.60:0/1947682312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8017070 con 0x7fe7f40ad270 2024-09-16T15:47:55.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f1600640 1 -- 172.21.15.60:0/1947682312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7e8009cc0 con 0x7fe7f40ad270 2024-09-16T15:47:55.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7f1600640 1 -- 172.21.15.60:0/1947682312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80106c0 con 0x7fe7f40ad270 2024-09-16T15:47:55.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f40cf250 con 0x7fe7f40ad270 2024-09-16T15:47:55.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.300+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f40cf810 con 0x7fe7f40ad270 2024-09-16T15:47:55.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.301+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f4147220 con 0x7fe7f40ad270 2024-09-16T15:47:55.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.302+0000 7fe7f1600640 1 -- 172.21.15.60:0/1947682312 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fe7e801a070 con 0x7fe7f40ad270 2024-09-16T15:47:55.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.303+0000 7fe7f1600640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe7d4079270 0x7fe7d407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.304+0000 7fe7f1600640 1 -- 172.21.15.60:0/1947682312 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7fe7e8099570 con 0x7fe7f40ad270 2024-09-16T15:47:55.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.304+0000 7fe7f8e00640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe7d4079270 0x7fe7d407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.304+0000 7fe7f8e00640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe7d4079270 0x7fe7d407b730 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0033000 tx=0x7fe7e0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:55.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.308+0000 7fe7f1600640 1 -- 172.21.15.60:0/1947682312 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe7e8063410 con 0x7fe7f40ad270 2024-09-16T15:47:55.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.501+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7f406f980 con 0x7fe7f40ad270 2024-09-16T15:47:55.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.502+0000 7fe7f1600640 1 -- 172.21.15.60:0/1947682312 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) ==== 74+0+13088 (secure 0 0 0) 0x7fe7e8062d10 con 0x7fe7f40ad270 2024-09-16T15:47:55.505 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:55.505 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":164,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:47:54.110886+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6835/3325202304":"2024-09-17T15:47:51.101778+0000","172.21.15.100:6801/3325202304":"2024-09-17T15:47:51.101778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:47:55.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe7d4079270 msgr2=0x7fe7d407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe7d4079270 0x7fe7d407b730 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0033000 tx=0x7fe7e0004730 comp rx=0 tx=0).stop 2024-09-16T15:47:55.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ad270 msgr2=0x7fe7f4145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ad270 0x7fe7f4145970 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800d770 tx=0x7fe7e800dc30 comp rx=0 tx=0).stop 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 shutdown_connections 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe7d4079270 0x7fe7d407b730 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f415f5a0 0x7fe7f41f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7f40ccf00 0x7fe7f40ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 --2- 172.21.15.60:0/1947682312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f40ad270 0x7fe7f4145970 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 >> 172.21.15.60:0/1947682312 conn(0x7fe7f40b1950 msgr2=0x7fe7f409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 shutdown_connections 2024-09-16T15:47:55.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.505+0000 7fe7fc200640 1 -- 172.21.15.60:0/1947682312 wait complete. 2024-09-16T15:47:55.523 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:38:23.733132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:47:55.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:47:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.684+0000 7fef6a600640 1 -- 172.21.15.60:0/1815481507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2000 msgr2=0x7fef640d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.684+0000 7fef6a600640 1 --2- 172.21.15.60:0/1815481507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2000 0x7fef640d23e0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fef58009900 tx=0x7fef5802a690 comp rx=0 tx=0).stop 2024-09-16T15:47:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.685+0000 7fef6a600640 1 -- 172.21.15.60:0/1815481507 shutdown_connections 2024-09-16T15:47:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.685+0000 7fef6a600640 1 --2- 172.21.15.60:0/1815481507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2000 0x7fef640d23e0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.685+0000 7fef6a600640 1 -- 172.21.15.60:0/1815481507 >> 172.21.15.60:0/1815481507 conn(0x7fef640d3e20 msgr2=0x7fef640d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.685+0000 7fef6a600640 1 -- 172.21.15.60:0/1815481507 shutdown_connections 2024-09-16T15:47:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.685+0000 7fef6a600640 1 -- 172.21.15.60:0/1815481507 wait complete. 2024-09-16T15:47:55.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.686+0000 7fef6a600640 1 Processor -- start 2024-09-16T15:47:55.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.686+0000 7fef6a600640 1 -- start start 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.687+0000 7fef6a600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef640b4e70 0x7fef6415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.687+0000 7fef6a600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640d2000 0x7fef641572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.687+0000 7fef6a600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2920 0x7fef6406d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.687+0000 7fef6a600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fef641692e0 con 0x7fef640d2920 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.687+0000 7fef6a600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef64169160 con 0x7fef640b4e70 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.687+0000 7fef6a600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fef64169460 con 0x7fef640d2000 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2920 0x7fef6406d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2920 0x7fef6406d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41768/0 (socket says 172.21.15.60:41768) 2024-09-16T15:47:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69c00640 1 -- 172.21.15.60:0/1555209653 learned_addr learned my addr 172.21.15.60:0/1555209653 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:55.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69200640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef640b4e70 0x7fef6415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef63e00640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640d2000 0x7fef641572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69c00640 1 -- 172.21.15.60:0/1555209653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640d2000 msgr2=0x7fef641572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69c00640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640d2000 0x7fef641572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69c00640 1 -- 172.21.15.60:0/1555209653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef640b4e70 msgr2=0x7fef6415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69c00640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef640b4e70 0x7fef6415e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.688+0000 7fef69c00640 1 -- 172.21.15.60:0/1555209653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6414c090 con 0x7fef640d2920 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.689+0000 7fef69c00640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2920 0x7fef6406d820 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c780 tx=0x7fef5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.689+0000 7fef60c00640 1 -- 172.21.15.60:0/1555209653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c004b60 con 0x7fef640d2920 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.689+0000 7fef60c00640 1 -- 172.21.15.60:0/1555209653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef5c004df0 con 0x7fef640d2920 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.689+0000 7fef60c00640 1 -- 172.21.15.60:0/1555209653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00aa70 con 0x7fef640d2920 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.689+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef641615d0 con 0x7fef640d2920 2024-09-16T15:47:55.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.689+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6414c330 con 0x7fef640d2920 2024-09-16T15:47:55.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.690+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef640d23e0 con 0x7fef640d2920 2024-09-16T15:47:55.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.691+0000 7fef60c00640 1 -- 172.21.15.60:0/1555209653 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fef5c00ac10 con 0x7fef640d2920 2024-09-16T15:47:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.692+0000 7fef60c00640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44079270 0x7fef4407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.692+0000 7fef60c00640 1 -- 172.21.15.60:0/1555209653 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7fef5c099600 con 0x7fef640d2920 2024-09-16T15:47:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.693+0000 7fef69200640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44079270 0x7fef4407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.693+0000 7fef69200640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44079270 0x7fef4407b730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fef58033000 tx=0x7fef58004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.696+0000 7fef60c00640 1 -- 172.21.15.60:0/1555209653 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fef5c020070 con 0x7fef640d2920 2024-09-16T15:47:55.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.891+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef64161bd0 con 0x7fef640d2920 2024-09-16T15:47:55.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.892+0000 7fef60c00640 1 -- 172.21.15.60:0/1555209653 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) ==== 93+0+10 (secure 0 0 0) 0x7fef5c062e50 con 0x7fef640d2920 2024-09-16T15:47:55.894 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-16T15:47:55.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44079270 msgr2=0x7fef4407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44079270 0x7fef4407b730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fef58033000 tx=0x7fef58004730 comp rx=0 tx=0).stop 2024-09-16T15:47:55.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2920 msgr2=0x7fef6406d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2920 0x7fef6406d820 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c780 tx=0x7fef5c00cc40 comp rx=0 tx=0).stop 2024-09-16T15:47:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 shutdown_connections 2024-09-16T15:47:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fef44079270 0x7fef4407b730 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef640d2920 0x7fef6406d820 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef640d2000 0x7fef641572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 --2- 172.21.15.60:0/1555209653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef640b4e70 0x7fef6415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 >> 172.21.15.60:0/1555209653 conn(0x7fef640d3e20 msgr2=0x7fef640b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 shutdown_connections 2024-09-16T15:47:55.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:55.894+0000 7fef6a600640 1 -- 172.21.15.60:0/1555209653 wait complete. 2024-09-16T15:47:55.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-16T15:47:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.071+0000 7f3b4ce00640 1 -- 172.21.15.60:0/544389406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b480ccf00 msgr2=0x7f3b480acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.071+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/544389406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b480ccf00 0x7f3b480acd30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3b30005fd0 tx=0x7f3b3002a540 comp rx=0 tx=0).stop 2024-09-16T15:47:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.071+0000 7f3b4ce00640 1 -- 172.21.15.60:0/544389406 shutdown_connections 2024-09-16T15:47:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.071+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/544389406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b480ccf00 0x7f3b480acd30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.071+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/544389406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b4815f5a0 0x7f3b480b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.071+0000 7f3b4ce00640 1 -- 172.21.15.60:0/544389406 >> 172.21.15.60:0/544389406 conn(0x7f3b480b1950 msgr2=0x7f3b480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.071+0000 7f3b4ce00640 1 -- 172.21.15.60:0/544389406 shutdown_connections 2024-09-16T15:47:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.071+0000 7f3b4ce00640 1 -- 172.21.15.60:0/544389406 wait complete. 2024-09-16T15:47:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.073+0000 7f3b4ce00640 1 Processor -- start 2024-09-16T15:47:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.073+0000 7f3b4ce00640 1 -- start start 2024-09-16T15:47:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b4ce00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b480ad270 0x7f3b481585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b4ce00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b480ccf00 0x7f3b48156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b4ce00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4815f5a0 0x7f3b4809caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b4ce00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4806f2e0 con 0x7f3b480ccf00 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b4ce00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4806f160 con 0x7f3b4815f5a0 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b4ce00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4806f460 con 0x7f3b480ad270 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b3d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b480ccf00 0x7f3b48156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b3d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b480ccf00 0x7f3b48156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41774/0 (socket says 172.21.15.60:41774) 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b3d400640 1 -- 172.21.15.60:0/2256215405 learned_addr learned my addr 172.21.15.60:0/2256215405 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b45e00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b480ad270 0x7f3b481585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b46800640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4815f5a0 0x7f3b4809caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b3d400640 1 -- 172.21.15.60:0/2256215405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b480ad270 msgr2=0x7f3b481585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b3d400640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b480ad270 0x7f3b481585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b3d400640 1 -- 172.21.15.60:0/2256215405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4815f5a0 msgr2=0x7f3b4809caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b3d400640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4815f5a0 0x7f3b4809caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.074+0000 7f3b3d400640 1 -- 172.21.15.60:0/2256215405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b48157540 con 0x7f3b480ccf00 2024-09-16T15:47:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.075+0000 7f3b3d400640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b480ccf00 0x7f3b48156df0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009850 tx=0x7f3b300038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.075+0000 7f3b3ea00640 1 -- 172.21.15.60:0/2256215405 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3003f070 con 0x7f3b480ccf00 2024-09-16T15:47:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.075+0000 7f3b3ea00640 1 -- 172.21.15.60:0/2256215405 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b300047a0 con 0x7f3b480ccf00 2024-09-16T15:47:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.075+0000 7f3b3ea00640 1 -- 172.21.15.60:0/2256215405 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30035030 con 0x7f3b480ccf00 2024-09-16T15:47:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.075+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b48157820 con 0x7f3b480ccf00 2024-09-16T15:47:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.075+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b48157de0 con 0x7f3b480ccf00 2024-09-16T15:47:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.076+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4806f980 con 0x7f3b480ccf00 2024-09-16T15:47:56.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.077+0000 7f3b3ea00640 1 -- 172.21.15.60:0/2256215405 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f3b30042070 con 0x7f3b480ccf00 2024-09-16T15:47:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.078+0000 7f3b3ea00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b24079270 0x7f3b2407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.078+0000 7f3b3ea00640 1 -- 172.21.15.60:0/2256215405 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7f3b300ba530 con 0x7f3b480ccf00 2024-09-16T15:47:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.079+0000 7f3b45e00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b24079270 0x7f3b2407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.079+0000 7f3b45e00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b24079270 0x7f3b2407b730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009b30 tx=0x7f3b38005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.082+0000 7f3b3ea00640 1 -- 172.21.15.60:0/2256215405 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b30083d80 con 0x7f3b480ccf00 2024-09-16T15:47:56.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:56.352+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f3b481581c0 con 0x7f3b480ccf00 2024-09-16T15:47:57.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.125+0000 7f3b3ea00640 1 -- 172.21.15.60:0/2256215405 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v515) ==== 166+0+0 (secure 0 0 0) 0x7f3b481581c0 con 0x7f3b480ccf00 2024-09-16T15:47:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b24079270 msgr2=0x7f3b2407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b24079270 0x7f3b2407b730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009b30 tx=0x7f3b38005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b480ccf00 msgr2=0x7f3b48156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b480ccf00 0x7f3b48156df0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009850 tx=0x7f3b300038a0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 shutdown_connections 2024-09-16T15:47:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f3b24079270 0x7f3b2407b730 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b4815f5a0 0x7f3b4809caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b480ccf00 0x7f3b48156df0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 --2- 172.21.15.60:0/2256215405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b480ad270 0x7f3b481585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 >> 172.21.15.60:0/2256215405 conn(0x7f3b480b1950 msgr2=0x7f3b48073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 shutdown_connections 2024-09-16T15:47:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.129+0000 7f3b4ce00640 1 -- 172.21.15.60:0/2256215405 wait complete. 2024-09-16T15:47:57.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:47:57.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.312+0000 7f15fe000640 1 -- 172.21.15.60:0/2540630495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 msgr2=0x7f15f806f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.312+0000 7f15fe000640 1 --2- 172.21.15.60:0/2540630495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 0x7f15f806f740 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f15f0005fd0 tx=0x7f15f002aa90 comp rx=0 tx=0).stop 2024-09-16T15:47:57.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.313+0000 7f15fe000640 1 -- 172.21.15.60:0/2540630495 shutdown_connections 2024-09-16T15:47:57.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.313+0000 7f15fe000640 1 --2- 172.21.15.60:0/2540630495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 0x7f15f806f740 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.313+0000 7f15fe000640 1 -- 172.21.15.60:0/2540630495 >> 172.21.15.60:0/2540630495 conn(0x7f15f80d0b30 msgr2=0x7f15f80d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:57.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.313+0000 7f15fe000640 1 -- 172.21.15.60:0/2540630495 shutdown_connections 2024-09-16T15:47:57.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.313+0000 7f15fe000640 1 -- 172.21.15.60:0/2540630495 wait complete. 2024-09-16T15:47:57.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.314+0000 7f15fe000640 1 Processor -- start 2024-09-16T15:47:57.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.315+0000 7f15fe000640 1 -- start start 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.315+0000 7f15fe000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f80cc8d0 0x7f15f81485a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.315+0000 7f15fe000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f80d1620 0x7f15f815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.315+0000 7f15fe000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 0x7f15f81552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.315+0000 7f15fe000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f15f806fc80 con 0x7f15f815cb40 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fe000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81565e0 con 0x7f15f80cc8d0 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fe000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f15f806fe00 con 0x7f15f80d1620 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fd600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 0x7f15f81552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fd600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 0x7f15f81552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41784/0 (socket says 172.21.15.60:41784) 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fd600640 1 -- 172.21.15.60:0/374794416 learned_addr learned my addr 172.21.15.60:0/374794416 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fcc00640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f80cc8d0 0x7f15f81485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15f7e00640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f80d1620 0x7f15f815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fd600640 1 -- 172.21.15.60:0/374794416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f80d1620 msgr2=0x7f15f815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fd600640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f80d1620 0x7f15f815b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fd600640 1 -- 172.21.15.60:0/374794416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f80cc8d0 msgr2=0x7f15f81485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fd600640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f80cc8d0 0x7f15f81485a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.316+0000 7f15fd600640 1 -- 172.21.15.60:0/374794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f815b8b0 con 0x7f15f815cb40 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.317+0000 7f15fd600640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 0x7f15f81552e0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009d90 tx=0x7f15f0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.317+0000 7f15f4c00640 1 -- 172.21.15.60:0/374794416 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0044070 con 0x7f15f815cb40 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.317+0000 7f15f4c00640 1 -- 172.21.15.60:0/374794416 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15f0004250 con 0x7f15f815cb40 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.317+0000 7f15f4c00640 1 -- 172.21.15.60:0/374794416 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004680 con 0x7f15f815cb40 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.317+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f80b3a80 con 0x7f15f815cb40 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.317+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f80c7d30 con 0x7f15f815cb40 2024-09-16T15:47:57.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.318+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f80703e0 con 0x7f15f815cb40 2024-09-16T15:47:57.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.319+0000 7f15f4c00640 1 -- 172.21.15.60:0/374794416 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f15f0047070 con 0x7f15f815cb40 2024-09-16T15:47:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.320+0000 7f15f4c00640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15d8079270 0x7f15d807b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.320+0000 7f15f4c00640 1 -- 172.21.15.60:0/374794416 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7f15f00ba390 con 0x7f15f815cb40 2024-09-16T15:47:57.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.321+0000 7f15fcc00640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15d8079270 0x7f15d807b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.321+0000 7f15fcc00640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15d8079270 0x7f15d807b730 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f15ec001680 tx=0x7f15ec009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:57.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.324+0000 7f15f4c00640 1 -- 172.21.15.60:0/374794416 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f15f0032030 con 0x7f15f815cb40 2024-09-16T15:47:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.522+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f15f80d11f0 con 0x7f15f815cb40 2024-09-16T15:47:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.522+0000 7f15f4c00640 1 -- 172.21.15.60:0/374794416 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7f15f0004820 con 0x7f15f815cb40 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3939.205749 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5797.855551 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3879.155944 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3769.377309 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4397.194279 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3973.773692 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6028.587574 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3946.818042 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds 20/20 2024-09-16T15:47:57.525 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced debug_ms 2/2 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_frag true 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:mds dev mds_verify_scatter true 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced client_mount_timeout 600 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_client 20/20 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced debug_ms 1/1 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:47:57.526 INFO:teuthology.orchestra.run.smithi060.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:47:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15d8079270 msgr2=0x7f15d807b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15d8079270 0x7f15d807b730 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f15ec001680 tx=0x7f15ec009040 comp rx=0 tx=0).stop 2024-09-16T15:47:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 msgr2=0x7f15f81552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 0x7f15f81552e0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009d90 tx=0x7f15f0003f10 comp rx=0 tx=0).stop 2024-09-16T15:47:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 shutdown_connections 2024-09-16T15:47:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f15d8079270 0x7f15d807b730 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15f815cb40 0x7f15f81552e0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15f80d1620 0x7f15f815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 --2- 172.21.15.60:0/374794416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15f80cc8d0 0x7f15f81485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 >> 172.21.15.60:0/374794416 conn(0x7f15f80d0b30 msgr2=0x7f15f806d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 shutdown_connections 2024-09-16T15:47:57.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.525+0000 7f15fe000640 1 -- 172.21.15.60:0/374794416 wait complete. 2024-09-16T15:47:57.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:47:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.706+0000 7f3372400640 1 -- 172.21.15.60:0/733465593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c0b3370 msgr2=0x7f336c0cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.706+0000 7f3372400640 1 --2- 172.21.15.60:0/733465593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c0b3370 0x7f336c0cebe0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f335c005fd0 tx=0x7f335c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.706+0000 7f3372400640 1 -- 172.21.15.60:0/733465593 shutdown_connections 2024-09-16T15:47:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.706+0000 7f3372400640 1 --2- 172.21.15.60:0/733465593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c0b3370 0x7f336c0cebe0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.706+0000 7f3372400640 1 -- 172.21.15.60:0/733465593 >> 172.21.15.60:0/733465593 conn(0x7f336c0b1840 msgr2=0x7f336c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.706+0000 7f3372400640 1 -- 172.21.15.60:0/733465593 shutdown_connections 2024-09-16T15:47:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.706+0000 7f3372400640 1 -- 172.21.15.60:0/733465593 wait complete. 2024-09-16T15:47:57.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.708+0000 7f3372400640 1 Processor -- start 2024-09-16T15:47:57.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.708+0000 7f3372400640 1 -- start start 2024-09-16T15:47:57.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f3372400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f336c0b3370 0x7f336c0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f3372400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15c080 0x7f336c1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f3372400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c15f090 0x7f336c1f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f3372400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f336c0cf590 con 0x7f336c15f090 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f3372400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f336c0cf410 con 0x7f336c15c080 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f3372400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f336c0cdad0 con 0x7f336c0b3370 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f336be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c15f090 0x7f336c1f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f336be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c15f090 0x7f336c1f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41796/0 (socket says 172.21.15.60:41796) 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.709+0000 7f336be00640 1 -- 172.21.15.60:0/347211366 learned_addr learned my addr 172.21.15.60:0/347211366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f336b400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f336c0b3370 0x7f336c0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f3362a00640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15c080 0x7f336c1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f336be00640 1 -- 172.21.15.60:0/347211366 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f336c0b3370 msgr2=0x7f336c0a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f336be00640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f336c0b3370 0x7f336c0a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f336be00640 1 -- 172.21.15.60:0/347211366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15c080 msgr2=0x7f336c1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f336be00640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15c080 0x7f336c1518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f336be00640 1 -- 172.21.15.60:0/347211366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c151e00 con 0x7f336c15f090 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f336be00640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c15f090 0x7f336c1f4f30 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f336400c6e0 tx=0x7f336400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.710+0000 7f3363e00640 1 -- 172.21.15.60:0/347211366 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364003b70 con 0x7f336c15f090 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.711+0000 7f3363e00640 1 -- 172.21.15.60:0/347211366 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3364003d30 con 0x7f336c15f090 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.711+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c1f5db0 con 0x7f336c15f090 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.711+0000 7f3363e00640 1 -- 172.21.15.60:0/347211366 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364005330 con 0x7f336c15f090 2024-09-16T15:47:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.711+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c1f67b0 con 0x7f336c15f090 2024-09-16T15:47:57.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.712+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c15e980 con 0x7f336c15f090 2024-09-16T15:47:57.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.713+0000 7f3363e00640 1 -- 172.21.15.60:0/347211366 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f33640054d0 con 0x7f336c15f090 2024-09-16T15:47:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.714+0000 7f3363e00640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f33480792a0 0x7f334807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.715+0000 7f3363e00640 1 -- 172.21.15.60:0/347211366 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7f3364099050 con 0x7f336c15f090 2024-09-16T15:47:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.715+0000 7f336b400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f33480792a0 0x7f334807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.715+0000 7f336b400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f33480792a0 0x7f334807b760 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f335c033000 tx=0x7f335c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.719+0000 7f3363e00640 1 -- 172.21.15.60:0/347211366 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f336c15b320 con 0x7f336c15f090 2024-09-16T15:47:57.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.914+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f336c15d250 con 0x7f336c15f090 2024-09-16T15:47:57.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.918+0000 7f3363e00640 1 -- 172.21.15.60:0/347211366 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f33640628a0 con 0x7f336c15f090 2024-09-16T15:47:57.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f33480792a0 msgr2=0x7f334807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f33480792a0 0x7f334807b760 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f335c033000 tx=0x7f335c004730 comp rx=0 tx=0).stop 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c15f090 msgr2=0x7f336c1f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c15f090 0x7f336c1f4f30 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f336400c6e0 tx=0x7f336400cba0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 shutdown_connections 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f33480792a0 0x7f334807b760 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f336c15f090 0x7f336c1f4f30 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f336c15c080 0x7f336c1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 --2- 172.21.15.60:0/347211366 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f336c0b3370 0x7f336c0a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 >> 172.21.15.60:0/347211366 conn(0x7f336c0b1840 msgr2=0x7f336c15c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.920+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 shutdown_connections 2024-09-16T15:47:57.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:57.921+0000 7f3372400640 1 -- 172.21.15.60:0/347211366 wait complete. 2024-09-16T15:47:57.939 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:47:57.939 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:47:58.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:47:58.128 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:47:58.128 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:58.128 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:58.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:47:58.165 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:58.165 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:58.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:47:58.202 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:47:58.202 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:58.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:47:58.239 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:47:58.239 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:58.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:47:58.278 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:47:58.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:47:58.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.437+0000 7f76bd200640 1 -- 172.21.15.60:0/3374275421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b80ccfd0 msgr2=0x7f76b80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.437+0000 7f76bd200640 1 --2- 172.21.15.60:0/3374275421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b80ccfd0 0x7f76b80730b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009be0 tx=0x7f76b002a970 comp rx=0 tx=0).stop 2024-09-16T15:47:58.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.437+0000 7f76bd200640 1 -- 172.21.15.60:0/3374275421 shutdown_connections 2024-09-16T15:47:58.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.437+0000 7f76bd200640 1 --2- 172.21.15.60:0/3374275421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b80ccfd0 0x7f76b80730b0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.437+0000 7f76bd200640 1 -- 172.21.15.60:0/3374275421 >> 172.21.15.60:0/3374275421 conn(0x7f76b80b1950 msgr2=0x7f76b80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:58.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.437+0000 7f76bd200640 1 -- 172.21.15.60:0/3374275421 shutdown_connections 2024-09-16T15:47:58.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.437+0000 7f76bd200640 1 -- 172.21.15.60:0/3374275421 wait complete. 2024-09-16T15:47:58.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.439+0000 7f76bd200640 1 Processor -- start 2024-09-16T15:47:58.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.439+0000 7f76bd200640 1 -- start start 2024-09-16T15:47:58.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76bd200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80ccfd0 0x7f76b80cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76bd200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b80cf300 0x7f76b814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76bd200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b81445d0 0x7f76b81e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76bd200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80735f0 con 0x7f76b81445d0 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76bd200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80b4f60 con 0x7f76b80ccfd0 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76bd200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f76b8073770 con 0x7f76b80cf300 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76b6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b81445d0 0x7f76b81e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76b6800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b81445d0 0x7f76b81e65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41814/0 (socket says 172.21.15.60:41814) 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.440+0000 7f76b6800640 1 -- 172.21.15.60:0/843253320 learned_addr learned my addr 172.21.15.60:0/843253320 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76b5e00640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80ccfd0 0x7f76b80cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76ad400640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b80cf300 0x7f76b814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76b6800640 1 -- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b80cf300 msgr2=0x7f76b814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76b6800640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b80cf300 0x7f76b814ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76b6800640 1 -- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80ccfd0 msgr2=0x7f76b80cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76b6800640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80ccfd0 0x7f76b80cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76b6800640 1 -- 172.21.15.60:0/843253320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b814e330 con 0x7f76b81445d0 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76ad400640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b80cf300 0x7f76b814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76b6800640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b81445d0 0x7f76b81e65e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006010 tx=0x7f76b0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.441+0000 7f76aea00640 1 -- 172.21.15.60:0/843253320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0003d60 con 0x7f76b81445d0 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.442+0000 7f76aea00640 1 -- 172.21.15.60:0/843253320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76b0004040 con 0x7f76b81445d0 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.442+0000 7f76aea00640 1 -- 172.21.15.60:0/843253320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0004490 con 0x7f76b81445d0 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.442+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b814b5d0 con 0x7f76b81445d0 2024-09-16T15:47:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.442+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b814bb90 con 0x7f76b81445d0 2024-09-16T15:47:58.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.443+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b8073c20 con 0x7f76b81445d0 2024-09-16T15:47:58.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.444+0000 7f76aea00640 1 -- 172.21.15.60:0/843253320 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f76b0004630 con 0x7f76b81445d0 2024-09-16T15:47:58.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.445+0000 7f76aea00640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f769407d680 0x7f769407fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.445+0000 7f76aea00640 1 -- 172.21.15.60:0/843253320 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7f76b00bad60 con 0x7f76b81445d0 2024-09-16T15:47:58.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.445+0000 7f76b5e00640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f769407d680 0x7f769407fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.446+0000 7f76b5e00640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f769407d680 0x7f769407fb40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f76a80046b0 tx=0x7f76a8007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:58.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.449+0000 7f76aea00640 1 -- 172.21.15.60:0/843253320 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f76b0084c00 con 0x7f76b81445d0 2024-09-16T15:47:58.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.680+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76b8070a20 con 0x7f76b81445d0 2024-09-16T15:47:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.681+0000 7f76aea00640 1 -- 172.21.15.60:0/843253320 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+3922 (secure 0 0 0) 0x7f76b0084500 con 0x7f76b81445d0 2024-09-16T15:47:58.683 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:47:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":115,"btime":"2024-09-16T15:47:52:104035+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7361,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1464936997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1464936997},{"type":"v1","addr":"172.21.15.60:6834","nonce":1464936997}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":7364,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1611348877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1611348877},{"type":"v1","addr":"172.21.15.60:6835","nonce":1611348877}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6834/2321854052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2321854052},{"type":"v1","addr":"172.21.15.100:6834","nonce":2321854052}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":107},{"gid":8515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1964107493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":1964107493},{"type":"v1","addr":"172.21.15.100:6835","nonce":1964107493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115}],"filesystems":[]} 2024-09-16T15:47:58.683 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 115 2024-09-16T15:47:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f769407d680 msgr2=0x7f769407fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f769407d680 0x7f769407fb40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f76a80046b0 tx=0x7f76a8007e90 comp rx=0 tx=0).stop 2024-09-16T15:47:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b81445d0 msgr2=0x7f76b81e65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b81445d0 0x7f76b81e65e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006010 tx=0x7f76b0003970 comp rx=0 tx=0).stop 2024-09-16T15:47:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 shutdown_connections 2024-09-16T15:47:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f769407d680 0x7f769407fb40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76b81445d0 0x7f76b81e65e0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76b80cf300 0x7f76b814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 --2- 172.21.15.60:0/843253320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76b80ccfd0 0x7f76b80cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 >> 172.21.15.60:0/843253320 conn(0x7f76b80b1950 msgr2=0x7f76b8073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 shutdown_connections 2024-09-16T15:47:58.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.683+0000 7f76bd200640 1 -- 172.21.15.60:0/843253320 wait complete. 2024-09-16T15:47:58.701 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:47:58.701 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:47:58.701 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:47:58.701 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:47:58.701 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:47:58.702 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:47:58.702 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:47:58.702 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:47:58.702 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:47:58.702 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:47:58.702 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:47:58.702 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:47:58.703 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:47:58.703 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:47:58.703 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:47:58.703 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:47:58.716 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:47:58.716 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:47:58.717 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:47:58.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:47:58.720 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:47:58.720 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:47:58.721 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:47:58.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:47:58.760 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:47:58.760 DEBUG:teuthology.parallel:result is None 2024-09-16T15:47:58.763 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:47:58.763 DEBUG:teuthology.parallel:result is None 2024-09-16T15:47:58.765 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:47:58.765 DEBUG:teuthology.parallel:result is None 2024-09-16T15:47:58.767 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:47:58.768 DEBUG:teuthology.parallel:result is None 2024-09-16T15:47:58.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:47:58.893 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-09-16T15:47:58.895 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:47:58.896 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-09-16T15:47:58.896 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-09-16T15:47:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.933+0000 7f4fa6000640 1 -- 172.21.15.60:0/3572080262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bac0 msgr2=0x7f4fa00cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.933+0000 7f4fa6000640 1 --2- 172.21.15.60:0/3572080262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bac0 0x7f4fa00cd8f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00cde30 tx=0x7f4f9002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.933+0000 7f4fa6000640 1 -- 172.21.15.60:0/3572080262 shutdown_connections 2024-09-16T15:47:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.933+0000 7f4fa6000640 1 --2- 172.21.15.60:0/3572080262 >> v2:172.21.15.100:3300/0 conn(0x7f4fa0071490 0x7f4fa015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.933+0000 7f4fa6000640 1 --2- 172.21.15.60:0/3572080262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bac0 0x7f4fa00cd8f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.933+0000 7f4fa6000640 1 -- 172.21.15.60:0/3572080262 >> 172.21.15.60:0/3572080262 conn(0x7f4fa00b2b90 msgr2=0x7f4fa00b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.933+0000 7f4fa6000640 1 -- 172.21.15.60:0/3572080262 shutdown_connections 2024-09-16T15:47:58.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.934+0000 7f4fa6000640 1 -- 172.21.15.60:0/3572080262 wait complete. 2024-09-16T15:47:58.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.935+0000 7f4fa6000640 1 Processor -- start 2024-09-16T15:47:58.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.935+0000 7f4fa6000640 1 -- start start 2024-09-16T15:47:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4fa6000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa0071490 0x7f4fa01f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4fa6000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa015bac0 0x7f4fa01f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4fa6000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bea0 0x7f4fa0073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4fa6000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01600a0 con 0x7f4fa015bea0 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4fa6000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa015ff20 con 0x7f4fa015bac0 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4fa6000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa006cb70 con 0x7f4fa0071490 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bea0 0x7f4fa0073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bea0 0x7f4fa0073d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45546/0 (socket says 172.21.15.60:45546) 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4f9fe00640 1 -- 172.21.15.60:0/398022248 learned_addr learned my addr 172.21.15.60:0/398022248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.936+0000 7f4f9f400640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa0071490 0x7f4fa01f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f97e00640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa015bac0 0x7f4fa01f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f9fe00640 1 -- 172.21.15.60:0/398022248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa0071490 msgr2=0x7f4fa01f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f9fe00640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa0071490 0x7f4fa01f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f9fe00640 1 -- 172.21.15.60:0/398022248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa015bac0 msgr2=0x7f4fa01f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f9fe00640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa015bac0 0x7f4fa01f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f9fe00640 1 -- 172.21.15.60:0/398022248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa013c7b0 con 0x7f4fa015bea0 2024-09-16T15:47:58.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f9fe00640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bea0 0x7f4fa0073d40 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800eaa0 tx=0x7f4f9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:58.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f97400640 1 -- 172.21.15.60:0/398022248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800cdb0 con 0x7f4fa015bea0 2024-09-16T15:47:58.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.937+0000 7f4f97400640 1 -- 172.21.15.60:0/398022248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f98004c10 con 0x7f4fa015bea0 2024-09-16T15:47:58.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.938+0000 7f4f97400640 1 -- 172.21.15.60:0/398022248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98005020 con 0x7f4fa015bea0 2024-09-16T15:47:58.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.938+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa013c1f0 con 0x7f4fa015bea0 2024-09-16T15:47:58.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.938+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa01f4260 con 0x7f4fa015bea0 2024-09-16T15:47:58.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.939+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa01f4750 con 0x7f4fa015bea0 2024-09-16T15:47:58.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.940+0000 7f4f97400640 1 -- 172.21.15.60:0/398022248 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f4f980051c0 con 0x7f4fa015bea0 2024-09-16T15:47:58.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.941+0000 7f4f97400640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4f7c079270 0x7f4f7c07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.941+0000 7f4f97400640 1 -- 172.21.15.60:0/398022248 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4971+0+0 (secure 0 0 0) 0x7f4f98099de0 con 0x7f4fa015bea0 2024-09-16T15:47:58.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.941+0000 7f4f9f400640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4f7c079270 0x7f4f7c07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.942+0000 7f4f9f400640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4f7c079270 0x7f4f7c07b730 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f90033000 tx=0x7f4f90004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:58.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:58.945+0000 7f4f97400640 1 -- 172.21.15.60:0/398022248 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4f98016030 con 0x7f4fa015bea0 2024-09-16T15:47:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:47:59.155+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4fa015f1c0 con 0x7f4fa015bea0 2024-09-16T15:48:00.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.152+0000 7f4f97400640 1 -- 172.21.15.60:0/398022248 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v169) ==== 97+0+0 (secure 0 0 0) 0x7f4f980635a0 con 0x7f4fa015bea0 2024-09-16T15:48:00.154 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-09-16T15:48:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4f7c079270 msgr2=0x7f4f7c07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4f7c079270 0x7f4f7c07b730 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f90033000 tx=0x7f4f90004730 comp rx=0 tx=0).stop 2024-09-16T15:48:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bea0 msgr2=0x7f4fa0073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bea0 0x7f4fa0073d40 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800eaa0 tx=0x7f4f9800ef60 comp rx=0 tx=0).stop 2024-09-16T15:48:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 shutdown_connections 2024-09-16T15:48:00.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4f7c079270 0x7f4f7c07b730 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fa015bea0 0x7f4fa0073d40 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fa015bac0 0x7f4fa01f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 --2- 172.21.15.60:0/398022248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fa0071490 0x7f4fa01f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 >> 172.21.15.60:0/398022248 conn(0x7f4fa00b2b90 msgr2=0x7f4fa00cb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 shutdown_connections 2024-09-16T15:48:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.156+0000 7f4fa6000640 1 -- 172.21.15.60:0/398022248 wait complete. 2024-09-16T15:48:00.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:48:00.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.341+0000 7f5e22800640 1 -- 172.21.15.60:0/2289748761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c0ccf00 msgr2=0x7f5e1c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.341+0000 7f5e22800640 1 --2- 172.21.15.60:0/2289748761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c0ccf00 0x7f5e1c0b2440 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005fd0 tx=0x7f5e0402a4a0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.341+0000 7f5e22800640 1 -- 172.21.15.60:0/2289748761 shutdown_connections 2024-09-16T15:48:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.341+0000 7f5e22800640 1 --2- 172.21.15.60:0/2289748761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c0ccf00 0x7f5e1c0b2440 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.341+0000 7f5e22800640 1 --2- 172.21.15.60:0/2289748761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e1c073fb0 0x7f5e1c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.341+0000 7f5e22800640 1 -- 172.21.15.60:0/2289748761 >> 172.21.15.60:0/2289748761 conn(0x7f5e1c0b1950 msgr2=0x7f5e1c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.341+0000 7f5e22800640 1 -- 172.21.15.60:0/2289748761 shutdown_connections 2024-09-16T15:48:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.341+0000 7f5e22800640 1 -- 172.21.15.60:0/2289748761 wait complete. 2024-09-16T15:48:00.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.343+0000 7f5e22800640 1 Processor -- start 2024-09-16T15:48:00.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.343+0000 7f5e22800640 1 -- start start 2024-09-16T15:48:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e22800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e1c073fb0 0x7f5e1c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e22800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e1c0ccf00 0x7f5e1c149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e22800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c15f5a0 0x7f5e1c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e22800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c06f2e0 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e22800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c06f160 con 0x7f5e1c0ccf00 2024-09-16T15:48:00.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e22800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c06f460 con 0x7f5e1c073fb0 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e1be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c15f5a0 0x7f5e1c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e1be00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c15f5a0 0x7f5e1c0cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45572/0 (socket says 172.21.15.60:45572) 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e1be00640 1 -- 172.21.15.60:0/2745905411 learned_addr learned my addr 172.21.15.60:0/2745905411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e12a00640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e1c0ccf00 0x7f5e1c149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.344+0000 7f5e1b400640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e1c073fb0 0x7f5e1c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e1be00640 1 -- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e1c073fb0 msgr2=0x7f5e1c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e1be00640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e1c073fb0 0x7f5e1c158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e1be00640 1 -- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e1c0ccf00 msgr2=0x7f5e1c149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e1be00640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e1c0ccf00 0x7f5e1c149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e1be00640 1 -- 172.21.15.60:0/2745905411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c1f3a40 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e1b400640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e1c073fb0 0x7f5e1c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:00.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e1be00640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c15f5a0 0x7f5e1c0cfcb0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b970 tx=0x7f5e1400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e13e00640 1 -- 172.21.15.60:0/2745905411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14019070 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e13e00640 1 -- 172.21.15.60:0/2745905411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e1400cbf0 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e13e00640 1 -- 172.21.15.60:0/2745905411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e14014440 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c1579a0 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.345+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c157f60 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.346+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1c06f980 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.348+0000 7f5e13e00640 1 -- 172.21.15.60:0/2745905411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f5e1401c070 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.348+0000 7f5e13e00640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5df80792a0 0x7f5df807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:00.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.349+0000 7f5e13e00640 1 -- 172.21.15.60:0/2745905411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4885+0+0 (secure 0 0 0) 0x7f5e140990b0 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.349+0000 7f5e1b400640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5df80792a0 0x7f5df807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:00.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.349+0000 7f5e1b400640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5df80792a0 0x7f5df807b760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c006fd0 tx=0x7f5e0c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:00.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.353+0000 7f5e13e00640 1 -- 172.21.15.60:0/2745905411 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5e14062900 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.619+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5e1c0acd30 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.620+0000 7f5e13e00640 1 -- 172.21.15.60:0/2745905411 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f5e14067330 con 0x7f5e1c15f5a0 2024-09-16T15:48:00.622 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:00.622 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:00.622 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-09-16T15:48:00.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.622 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm++DxCxAAD5cfTkbca/q8xsdAUn/LLw==", 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.623 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmuLufHxAAF7lQ9/7QF/zanGuaRs28kA==", 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-09-16T15:48:00.624 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmGqW+FRAAw2xMfJgsURkCkolb+FKodA==", 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm9lBcKRAAl8zCW1UsFPR1gVcGvhFjkw==", 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.625 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC/UOhmDfUmGxAAUQov/Iz5syyvqjoFb5N2FQ==", 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:00.626 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDCUOhmI7a2JhAAsd7KKrphBmiZUsz2bHlPVQ==", 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDGUOhmL7QpExAAzmkdwZVyujlP/apwXJEzBQ==", 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.628 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDJUOhmw88HHBAAJctuRU8Ub26AaTMJ0Ko+nw==", 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.629 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDPUOhmeM9EFhAAYRROM+PEvLJ2VwMYZRbt6g==", 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDSUOhmJ5bQLxAA0KT4V8dGGSSYgR5UM6nUdQ==", 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.630 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDVUOhmR6beOBAAFmCRHQpBd+Nkk3Kx6ZAZfQ==", 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.631 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQDZUOhmfg/dBxAAFRM05IW07YTXzz0FZSfChQ==", 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.632 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA==", 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhm5TtEAxAASYkCMdeIjqb07NM6LX+efw==", 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.633 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhmad9lOBAAsYM1+XBDCOWDfFYXY9lpIA==", 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:48:00.634 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC+UOhmrUquCRAAzl6oD3pRZeKVqC/Hu6m2nw==", 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-09-16T15:48:00.635 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmZAGUFxAA/RzU7mYLklmwfd4yW/96/Q==", 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC9UOhm6gpDBRAAeBIhWL8KucuYr8VSGT6aEQ==", 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-09-16T15:48:00.637 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQC8UOhmyP1yORAAqeoJoNSl8ByoeYPDEG0Zjw==", 2024-09-16T15:48:00.638 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-09-16T15:48:00.638 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-09-16T15:48:00.638 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-09-16T15:48:00.638 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-09-16T15:48:00.638 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.638 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:00.638 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-16T15:48:00.638 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5df80792a0 msgr2=0x7f5df807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5df80792a0 0x7f5df807b760 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c006fd0 tx=0x7f5e0c008040 comp rx=0 tx=0).stop 2024-09-16T15:48:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c15f5a0 msgr2=0x7f5e1c0cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c15f5a0 0x7f5e1c0cfcb0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b970 tx=0x7f5e1400be30 comp rx=0 tx=0).stop 2024-09-16T15:48:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 shutdown_connections 2024-09-16T15:48:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5df80792a0 0x7f5df807b760 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e1c15f5a0 0x7f5e1c0cfcb0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e1c0ccf00 0x7f5e1c149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 --2- 172.21.15.60:0/2745905411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e1c073fb0 0x7f5e1c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 >> 172.21.15.60:0/2745905411 conn(0x7f5e1c0b1950 msgr2=0x7f5e1c0ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 shutdown_connections 2024-09-16T15:48:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.622+0000 7f5e22800640 1 -- 172.21.15.60:0/2745905411 wait complete. 2024-09-16T15:48:00.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:48:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.801+0000 7fb159a00640 1 -- 172.21.15.60:0/2740658454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb154073fb0 msgr2=0x7fb154074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.801+0000 7fb159a00640 1 --2- 172.21.15.60:0/2740658454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb154073fb0 0x7fb154074440 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009830 tx=0x7fb14c02a480 comp rx=0 tx=0).stop 2024-09-16T15:48:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.801+0000 7fb159a00640 1 -- 172.21.15.60:0/2740658454 shutdown_connections 2024-09-16T15:48:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.801+0000 7fb159a00640 1 --2- 172.21.15.60:0/2740658454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb154073fb0 0x7fb154074440 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.801+0000 7fb159a00640 1 --2- 172.21.15.60:0/2740658454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1540ccf00 0x7fb1540b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.801+0000 7fb159a00640 1 -- 172.21.15.60:0/2740658454 >> 172.21.15.60:0/2740658454 conn(0x7fb1540b1950 msgr2=0x7fb1540b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.801+0000 7fb159a00640 1 -- 172.21.15.60:0/2740658454 shutdown_connections 2024-09-16T15:48:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.801+0000 7fb159a00640 1 -- 172.21.15.60:0/2740658454 wait complete. 2024-09-16T15:48:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.802+0000 7fb159a00640 1 Processor -- start 2024-09-16T15:48:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.803+0000 7fb159a00640 1 -- start start 2024-09-16T15:48:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.803+0000 7fb159a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb154073fb0 0x7fb154158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.803+0000 7fb159a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1540ccf00 0x7fb154159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.803+0000 7fb159a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb15415f5a0 0x7fb1541f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.803+0000 7fb159a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb15406f2e0 con 0x7fb15415f5a0 2024-09-16T15:48:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.803+0000 7fb159a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb15406f160 con 0x7fb1540ccf00 2024-09-16T15:48:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.803+0000 7fb159a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb15406f460 con 0x7fb154073fb0 2024-09-16T15:48:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb15415f5a0 0x7fb1541f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb15415f5a0 0x7fb1541f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45588/0 (socket says 172.21.15.60:45588) 2024-09-16T15:48:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 -- 172.21.15.60:0/1824401405 learned_addr learned my addr 172.21.15.60:0/1824401405 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 -- 172.21.15.60:0/1824401405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb154073fb0 msgr2=0x7fb154158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb154073fb0 0x7fb154158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 -- 172.21.15.60:0/1824401405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1540ccf00 msgr2=0x7fb154159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1540ccf00 0x7fb154159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 -- 172.21.15.60:0/1824401405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15415b540 con 0x7fb15415f5a0 2024-09-16T15:48:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.804+0000 7fb153400640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb15415f5a0 0x7fb1541f5ce0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009920 tx=0x7fb14c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.805+0000 7fb14aa00640 1 -- 172.21.15.60:0/1824401405 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c040070 con 0x7fb15415f5a0 2024-09-16T15:48:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.805+0000 7fb14aa00640 1 -- 172.21.15.60:0/1824401405 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb14c005310 con 0x7fb15415f5a0 2024-09-16T15:48:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.805+0000 7fb14aa00640 1 -- 172.21.15.60:0/1824401405 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c005c90 con 0x7fb15415f5a0 2024-09-16T15:48:00.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.805+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15415b820 con 0x7fb15415f5a0 2024-09-16T15:48:00.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.805+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb15415bde0 con 0x7fb15415f5a0 2024-09-16T15:48:00.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.806+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb15406f9f0 con 0x7fb15415f5a0 2024-09-16T15:48:00.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.807+0000 7fb14aa00640 1 -- 172.21.15.60:0/1824401405 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fb14c039040 con 0x7fb15415f5a0 2024-09-16T15:48:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.809+0000 7fb14aa00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb134079270 0x7fb13407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.809+0000 7fb14aa00640 1 -- 172.21.15.60:0/1824401405 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4885+0+0 (secure 0 0 0) 0x7fb14c0ba4e0 con 0x7fb15415f5a0 2024-09-16T15:48:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.809+0000 7fb152a00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb134079270 0x7fb13407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.809+0000 7fb152a00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb134079270 0x7fb13407b730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb144004a30 tx=0x7fb144001650 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:00.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:00.813+0000 7fb14aa00640 1 -- 172.21.15.60:0/1824401405 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb14c0845a0 con 0x7fb15415f5a0 2024-09-16T15:48:01.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.011+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb154071150 con 0x7fb15415f5a0 2024-09-16T15:48:01.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.012+0000 7fb14aa00640 1 -- 172.21.15.60:0/1824401405 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v169) ==== 84+0+65 (secure 0 0 0) 0x7fb14c083ea0 con 0x7fb15415f5a0 2024-09-16T15:48:01.014 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:01.014 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:48:01.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:01.014 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-09-16T15:48:01.014 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-09-16T15:48:01.014 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:01.014 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb134079270 msgr2=0x7fb13407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb134079270 0x7fb13407b730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb144004a30 tx=0x7fb144001650 comp rx=0 tx=0).stop 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb15415f5a0 msgr2=0x7fb1541f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb15415f5a0 0x7fb1541f5ce0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009920 tx=0x7fb14c032c10 comp rx=0 tx=0).stop 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 shutdown_connections 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb134079270 0x7fb13407b730 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb15415f5a0 0x7fb1541f5ce0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1540ccf00 0x7fb154159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 --2- 172.21.15.60:0/1824401405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb154073fb0 0x7fb154158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.014+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 >> 172.21.15.60:0/1824401405 conn(0x7fb1540b1950 msgr2=0x7fb15409ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.015+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 shutdown_connections 2024-09-16T15:48:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.015+0000 7fb159a00640 1 -- 172.21.15.60:0/1824401405 wait complete. 2024-09-16T15:48:01.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:48:01.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:48:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.197+0000 7f7ba5a00640 1 -- 172.21.15.60:0/899616837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba015bda0 msgr2=0x7f7ba0073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.197+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/899616837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba015bda0 0x7f7ba0073010 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009830 tx=0x7f7b9002a480 comp rx=0 tx=0).stop 2024-09-16T15:48:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.198+0000 7f7ba5a00640 1 -- 172.21.15.60:0/899616837 shutdown_connections 2024-09-16T15:48:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.198+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/899616837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba0070550 0x7f7ba00cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.198+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/899616837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba015bda0 0x7f7ba0073010 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.198+0000 7f7ba5a00640 1 -- 172.21.15.60:0/899616837 >> 172.21.15.60:0/899616837 conn(0x7f7ba00cfa70 msgr2=0x7f7ba00cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.198+0000 7f7ba5a00640 1 -- 172.21.15.60:0/899616837 shutdown_connections 2024-09-16T15:48:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.198+0000 7f7ba5a00640 1 -- 172.21.15.60:0/899616837 wait complete. 2024-09-16T15:48:01.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.199+0000 7f7ba5a00640 1 Processor -- start 2024-09-16T15:48:01.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.199+0000 7f7ba5a00640 1 -- start start 2024-09-16T15:48:01.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.200+0000 7f7ba5a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba006e590 0x7f7ba01f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:01.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.200+0000 7f7ba5a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba0070550 0x7f7ba00c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:01.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.200+0000 7f7ba5a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba015bda0 0x7f7ba01f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:01.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.200+0000 7f7ba5a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba00cba90 con 0x7f7ba0070550 2024-09-16T15:48:01.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.200+0000 7f7ba5a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba00cb910 con 0x7f7ba006e590 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.200+0000 7f7ba5a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba00cbc10 con 0x7f7ba015bda0 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba0070550 0x7f7ba00c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba0070550 0x7f7ba00c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45618/0 (socket says 172.21.15.60:45618) 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 -- 172.21.15.60:0/1373062081 learned_addr learned my addr 172.21.15.60:0/1373062081 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7ba5000640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba015bda0 0x7f7ba01f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9fe00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba006e590 0x7f7ba01f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 -- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba015bda0 msgr2=0x7f7ba01f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba015bda0 0x7f7ba01f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 -- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba006e590 msgr2=0x7f7ba01f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba006e590 0x7f7ba01f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 -- 172.21.15.60:0/1373062081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba00c88b0 con 0x7f7ba0070550 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7ba5000640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba015bda0 0x7f7ba01f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.201+0000 7f7b9f400640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba0070550 0x7f7ba00c8370 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c480 tx=0x7f7b8c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:01.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.202+0000 7f7b97e00640 1 -- 172.21.15.60:0/1373062081 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c016020 con 0x7f7ba0070550 2024-09-16T15:48:01.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.202+0000 7f7b97e00640 1 -- 172.21.15.60:0/1373062081 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b8c012920 con 0x7f7ba0070550 2024-09-16T15:48:01.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.202+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba01f8da0 con 0x7f7ba0070550 2024-09-16T15:48:01.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.202+0000 7f7b97e00640 1 -- 172.21.15.60:0/1373062081 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c012d70 con 0x7f7ba0070550 2024-09-16T15:48:01.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.202+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba01f5020 con 0x7f7ba0070550 2024-09-16T15:48:01.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.203+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba00cc1a0 con 0x7f7ba0070550 2024-09-16T15:48:01.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.204+0000 7f7b97e00640 1 -- 172.21.15.60:0/1373062081 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f7b8c019020 con 0x7f7ba0070550 2024-09-16T15:48:01.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.205+0000 7f7b97e00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b80079270 0x7f7b8007b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:01.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.205+0000 7f7b97e00640 1 -- 172.21.15.60:0/1373062081 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 4885+0+0 (secure 0 0 0) 0x7f7b8c099410 con 0x7f7ba0070550 2024-09-16T15:48:01.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.205+0000 7f7b9fe00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b80079270 0x7f7b8007b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:01.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.206+0000 7f7b9fe00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b80079270 0x7f7b8007b730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7b90034000 tx=0x7f7b90004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:01.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.209+0000 7f7b97e00640 1 -- 172.21.15.60:0/1373062081 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7b8c020020 con 0x7f7ba0070550 2024-09-16T15:48:01.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:01.405+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7ba015a170 con 0x7f7ba0070550 2024-09-16T15:48:01.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:48:01.890 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:48:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.160+0000 7f7b97e00640 1 -- 172.21.15.60:0/1373062081 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v170) ==== 138+0+0 (secure 0 0 0) 0x7f7b8c0632b0 con 0x7f7ba0070550 2024-09-16T15:48:02.163 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-09-16T15:48:02.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b80079270 msgr2=0x7f7b8007b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:02.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b80079270 0x7f7b8007b730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7b90034000 tx=0x7f7b90004730 comp rx=0 tx=0).stop 2024-09-16T15:48:02.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba0070550 msgr2=0x7f7ba00c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:02.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba0070550 0x7f7ba00c8370 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c480 tx=0x7f7b8c00c940 comp rx=0 tx=0).stop 2024-09-16T15:48:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 shutdown_connections 2024-09-16T15:48:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b80079270 0x7f7b8007b730 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ba015bda0 0x7f7ba01f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ba0070550 0x7f7ba00c8370 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 --2- 172.21.15.60:0/1373062081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ba006e590 0x7f7ba01f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 >> 172.21.15.60:0/1373062081 conn(0x7f7ba00cfa70 msgr2=0x7f7ba00b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.164+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 shutdown_connections 2024-09-16T15:48:02.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.165+0000 7f7ba5a00640 1 -- 172.21.15.60:0/1373062081 wait complete. 2024-09-16T15:48:02.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:48:02.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.357+0000 7fc99de00640 1 -- 172.21.15.60:0/3444798625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981612b0 msgr2=0x7fc9980b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:02.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.357+0000 7fc99de00640 1 --2- 172.21.15.60:0/3444798625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981612b0 0x7fc9980b3110 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fc988005fd0 tx=0x7fc98802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.357+0000 7fc99de00640 1 -- 172.21.15.60:0/3444798625 shutdown_connections 2024-09-16T15:48:02.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.357+0000 7fc99de00640 1 --2- 172.21.15.60:0/3444798625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc998161ab0 0x7fc998161f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.357+0000 7fc99de00640 1 --2- 172.21.15.60:0/3444798625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9981612b0 0x7fc9980b3110 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.357+0000 7fc99de00640 1 -- 172.21.15.60:0/3444798625 >> 172.21.15.60:0/3444798625 conn(0x7fc9980d1c20 msgr2=0x7fc9980d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:02.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.357+0000 7fc99de00640 1 -- 172.21.15.60:0/3444798625 shutdown_connections 2024-09-16T15:48:02.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.357+0000 7fc99de00640 1 -- 172.21.15.60:0/3444798625 wait complete. 2024-09-16T15:48:02.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.359+0000 7fc99de00640 1 Processor -- start 2024-09-16T15:48:02.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.359+0000 7fc99de00640 1 -- start start 2024-09-16T15:48:02.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99de00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9981612b0 0x7fc9980c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc998161ab0 0x7fc9981f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998162450 0x7fc9980709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99de00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc998161850 con 0x7fc998162450 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9980b25f0 con 0x7fc998161ab0 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99de00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc99806e390 con 0x7fc9981612b0 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998162450 0x7fc9980709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99ca00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9981612b0 0x7fc9980c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998162450 0x7fc9980709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45638/0 (socket says 172.21.15.60:45638) 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc998161ab0 0x7fc9981f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.360+0000 7fc99d400640 1 -- 172.21.15.60:0/2194834554 learned_addr learned my addr 172.21.15.60:0/2194834554 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc99d400640 1 -- 172.21.15.60:0/2194834554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9981612b0 msgr2=0x7fc9980c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc99d400640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9981612b0 0x7fc9980c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc99d400640 1 -- 172.21.15.60:0/2194834554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc998161ab0 msgr2=0x7fc9981f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc99d400640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc998161ab0 0x7fc9981f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc99d400640 1 -- 172.21.15.60:0/2194834554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9981f4a10 con 0x7fc998162450 2024-09-16T15:48:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc99d400640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998162450 0x7fc9980709f0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc99000c700 tx=0x7fc99000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:02.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc995600640 1 -- 172.21.15.60:0/2194834554 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990003b70 con 0x7fc998162450 2024-09-16T15:48:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc995600640 1 -- 172.21.15.60:0/2194834554 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc990003d10 con 0x7fc998162450 2024-09-16T15:48:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9981f4cf0 con 0x7fc998162450 2024-09-16T15:48:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc995600640 1 -- 172.21.15.60:0/2194834554 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99000aab0 con 0x7fc998162450 2024-09-16T15:48:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.361+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981f52b0 con 0x7fc998162450 2024-09-16T15:48:02.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.362+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99806e840 con 0x7fc998162450 2024-09-16T15:48:02.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.364+0000 7fc995600640 1 -- 172.21.15.60:0/2194834554 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fc99000ac50 con 0x7fc998162450 2024-09-16T15:48:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.365+0000 7fc995600640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc974079270 0x7fc97407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.365+0000 7fc995600640 1 -- 172.21.15.60:0/2194834554 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5255+0+0 (secure 0 0 0) 0x7fc990099c10 con 0x7fc998162450 2024-09-16T15:48:02.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.366+0000 7fc99ca00640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc974079270 0x7fc97407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.366+0000 7fc99ca00640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc974079270 0x7fc97407b730 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc988033000 tx=0x7fc988004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:02.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.369+0000 7fc995600640 1 -- 172.21.15.60:0/2194834554 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc990063460 con 0x7fc998162450 2024-09-16T15:48:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:02.567+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc9981f5620 con 0x7fc998162450 2024-09-16T15:48:03.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.168+0000 7fc995600640 1 -- 172.21.15.60:0/2194834554 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v171) ==== 170+0+0 (secure 0 0 0) 0x7fc990067e90 con 0x7fc998162450 2024-09-16T15:48:03.170 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-09-16T15:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc974079270 msgr2=0x7fc97407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc974079270 0x7fc97407b730 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc988033000 tx=0x7fc988004730 comp rx=0 tx=0).stop 2024-09-16T15:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998162450 msgr2=0x7fc9980709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:03.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998162450 0x7fc9980709f0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fc99000c700 tx=0x7fc99000cbc0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 shutdown_connections 2024-09-16T15:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc974079270 0x7fc97407b730 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc998162450 0x7fc9980709f0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc998161ab0 0x7fc9981f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 --2- 172.21.15.60:0/2194834554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9981612b0 0x7fc9980c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 >> 172.21.15.60:0/2194834554 conn(0x7fc9980d1c20 msgr2=0x7fc9980ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 shutdown_connections 2024-09-16T15:48:03.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.172+0000 7fc99de00640 1 -- 172.21.15.60:0/2194834554 wait complete. 2024-09-16T15:48:03.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:48:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.373+0000 7f635c400640 1 -- 172.21.15.60:0/1131668658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cec60 msgr2=0x7f63540cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.373+0000 7f635c400640 1 --2- 172.21.15.60:0/1131668658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cec60 0x7f63540cf070 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f634c005fd0 tx=0x7f634c02a940 comp rx=0 tx=0).stop 2024-09-16T15:48:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.373+0000 7f635c400640 1 -- 172.21.15.60:0/1131668658 shutdown_connections 2024-09-16T15:48:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.373+0000 7f635c400640 1 --2- 172.21.15.60:0/1131668658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540cec60 0x7f63540cf070 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.373+0000 7f635c400640 1 -- 172.21.15.60:0/1131668658 >> 172.21.15.60:0/1131668658 conn(0x7f63540b1840 msgr2=0x7f63540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.373+0000 7f635c400640 1 -- 172.21.15.60:0/1131668658 shutdown_connections 2024-09-16T15:48:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.373+0000 7f635c400640 1 -- 172.21.15.60:0/1131668658 wait complete. 2024-09-16T15:48:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.374+0000 7f635c400640 1 Processor -- start 2024-09-16T15:48:03.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.375+0000 7f635c400640 1 -- start start 2024-09-16T15:48:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.375+0000 7f635c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540af190 0x7f6354156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.375+0000 7f635c400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63540cec60 0x7f63541e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.375+0000 7f635c400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6354161760 0x7f63541e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.375+0000 7f635c400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f63541623a0 con 0x7f6354161760 2024-09-16T15:48:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6354070530 con 0x7f63540af190 2024-09-16T15:48:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635c400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6354162520 con 0x7f63540cec60 2024-09-16T15:48:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6354161760 0x7f63541e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6354161760 0x7f63541e8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45654/0 (socket says 172.21.15.60:45654) 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 -- 172.21.15.60:0/3033275417 learned_addr learned my addr 172.21.15.60:0/3033275417 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f6359a00640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540af190 0x7f6354156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f6359000640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63540cec60 0x7f63541e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 -- 172.21.15.60:0/3033275417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63540cec60 msgr2=0x7f63541e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63540cec60 0x7f63541e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 -- 172.21.15.60:0/3033275417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540af190 msgr2=0x7f6354156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540af190 0x7f6354156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 -- 172.21.15.60:0/3033275417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6354157160 con 0x7f6354161760 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.376+0000 7f635a400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6354161760 0x7f63541e8290 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f634c009cd0 tx=0x7f634c031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.377+0000 7f6351600640 1 -- 172.21.15.60:0/3033275417 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c030430 con 0x7f6354161760 2024-09-16T15:48:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.377+0000 7f6351600640 1 -- 172.21.15.60:0/3033275417 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f634c004260 con 0x7f6354161760 2024-09-16T15:48:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.377+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63541f4cc0 con 0x7f6354161760 2024-09-16T15:48:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.377+0000 7f6351600640 1 -- 172.21.15.60:0/3033275417 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c0046b0 con 0x7f6354161760 2024-09-16T15:48:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.377+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63541f5280 con 0x7f6354161760 2024-09-16T15:48:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.378+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63541629d0 con 0x7f6354161760 2024-09-16T15:48:03.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.379+0000 7f6351600640 1 -- 172.21.15.60:0/3033275417 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f634c039440 con 0x7f6354161760 2024-09-16T15:48:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.380+0000 7f6351600640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6334079270 0x7f633407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.380+0000 7f6351600640 1 -- 172.21.15.60:0/3033275417 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5637+0+0 (secure 0 0 0) 0x7f634c0bb060 con 0x7f6354161760 2024-09-16T15:48:03.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.381+0000 7f6359a00640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6334079270 0x7f633407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:03.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.381+0000 7f6359a00640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6334079270 0x7f633407b730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f63480015e0 tx=0x7f6348009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:03.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.384+0000 7f6351600640 1 -- 172.21.15.60:0/3033275417 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f634c038e70 con 0x7f6354161760 2024-09-16T15:48:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:03.621+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6354073550 con 0x7f6354161760 2024-09-16T15:48:04.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.170+0000 7f6351600640 1 -- 172.21.15.60:0/3033275417 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v118) ==== 174+0+0 (secure 0 0 0) 0x7f634c084f00 con 0x7f6354161760 2024-09-16T15:48:04.173 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 26 and data pool 27 2024-09-16T15:48:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.175+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6334079270 msgr2=0x7f633407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.175+0000 7f635c400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6334079270 0x7f633407b730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f63480015e0 tx=0x7f6348009040 comp rx=0 tx=0).stop 2024-09-16T15:48:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.175+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6354161760 msgr2=0x7f63541e8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.175+0000 7f635c400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6354161760 0x7f63541e8290 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f634c009cd0 tx=0x7f634c031ea0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.176+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 shutdown_connections 2024-09-16T15:48:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.176+0000 7f635c400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6334079270 0x7f633407b730 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.176+0000 7f635c400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6354161760 0x7f63541e8290 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.176+0000 7f635c400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63540cec60 0x7f63541e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.176+0000 7f635c400640 1 --2- 172.21.15.60:0/3033275417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63540af190 0x7f6354156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.176+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 >> 172.21.15.60:0/3033275417 conn(0x7f63540b1840 msgr2=0x7f6354072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.177+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 shutdown_connections 2024-09-16T15:48:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.177+0000 7f635c400640 1 -- 172.21.15.60:0/3033275417 wait complete. 2024-09-16T15:48:04.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:04.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.379+0000 7fd0cb200640 1 -- 172.21.15.60:0/228484085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1612d0 msgr2=0x7fd0cc0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.379+0000 7fd0cb200640 1 --2- 172.21.15.60:0/228484085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1612d0 0x7fd0cc0b4e00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009830 tx=0x7fd0bc02a480 comp rx=0 tx=0).stop 2024-09-16T15:48:04.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.379+0000 7fd0cb200640 1 -- 172.21.15.60:0/228484085 shutdown_connections 2024-09-16T15:48:04.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.379+0000 7fd0cb200640 1 --2- 172.21.15.60:0/228484085 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0cc0d4980 0x7fd0cc0d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.379+0000 7fd0cb200640 1 --2- 172.21.15.60:0/228484085 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1612d0 0x7fd0cc0b4e00 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.379+0000 7fd0cb200640 1 -- 172.21.15.60:0/228484085 >> 172.21.15.60:0/228484085 conn(0x7fd0cc0d3df0 msgr2=0x7fd0cc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.379+0000 7fd0cb200640 1 -- 172.21.15.60:0/228484085 shutdown_connections 2024-09-16T15:48:04.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.379+0000 7fd0cb200640 1 -- 172.21.15.60:0/228484085 wait complete. 2024-09-16T15:48:04.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.380+0000 7fd0cb200640 1 Processor -- start 2024-09-16T15:48:04.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.381+0000 7fd0cb200640 1 -- start start 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.381+0000 7fd0cb200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0cc0d4980 0x7fd0cc0c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.381+0000 7fd0cb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0cc1612d0 0x7fd0cc06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0cb200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1616b0 0x7fd0cc1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0cb200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc0b1c00 con 0x7fd0cc1616b0 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0cb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc0b1a80 con 0x7fd0cc1612d0 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0cb200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd0cc0b1d80 con 0x7fd0cc0d4980 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0ca800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1616b0 0x7fd0cc1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0ca800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1616b0 0x7fd0cc1578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45682/0 (socket says 172.21.15.60:45682) 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0ca800640 1 -- 172.21.15.60:0/1487267250 learned_addr learned my addr 172.21.15.60:0/1487267250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:04.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0c9400640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0cc1612d0 0x7fd0cc06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0c9e00640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0cc0d4980 0x7fd0cc0c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0ca800640 1 -- 172.21.15.60:0/1487267250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0cc0d4980 msgr2=0x7fd0cc0c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0ca800640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0cc0d4980 0x7fd0cc0c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0ca800640 1 -- 172.21.15.60:0/1487267250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0cc1612d0 msgr2=0x7fd0cc06d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0ca800640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0cc1612d0 0x7fd0cc06d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.382+0000 7fd0ca800640 1 -- 172.21.15.60:0/1487267250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc06de30 con 0x7fd0cc1616b0 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.383+0000 7fd0ca800640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1616b0 0x7fd0cc1578f0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400caa0 tx=0x7fd0c400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.383+0000 7fd0c2000640 1 -- 172.21.15.60:0/1487267250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4016020 con 0x7fd0cc1616b0 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.383+0000 7fd0c2000640 1 -- 172.21.15.60:0/1487267250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0c400acf0 con 0x7fd0cc1616b0 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.383+0000 7fd0c2000640 1 -- 172.21.15.60:0/1487267250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4012680 con 0x7fd0cc1616b0 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.383+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc06e0d0 con 0x7fd0cc1616b0 2024-09-16T15:48:04.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.383+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc15bae0 con 0x7fd0cc1616b0 2024-09-16T15:48:04.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.384+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0cc0b4e00 con 0x7fd0cc1616b0 2024-09-16T15:48:04.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.385+0000 7fd0c2000640 1 -- 172.21.15.60:0/1487267250 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fd0c4019020 con 0x7fd0cc1616b0 2024-09-16T15:48:04.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.386+0000 7fd0c2000640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0ac079270 0x7fd0ac07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.386+0000 7fd0c2000640 1 -- 172.21.15.60:0/1487267250 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 5737+0+0 (secure 0 0 0) 0x7fd0c40999f0 con 0x7fd0cc1616b0 2024-09-16T15:48:04.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.387+0000 7fd0c9e00640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0ac079270 0x7fd0ac07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.387+0000 7fd0c9e00640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0ac079270 0x7fd0ac07b730 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc034000 tx=0x7fd0bc004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:04.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.391+0000 7fd0c2000640 1 -- 172.21.15.60:0/1487267250 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd0c4020020 con 0x7fd0cc1616b0 2024-09-16T15:48:04.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.590+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0cc0d09a0 con 0x7fd0cc1616b0 2024-09-16T15:48:04.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.591+0000 7fd0c2000640 1 -- 172.21.15.60:0/1487267250 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) ==== 74+0+16800 (secure 0 0 0) 0x7fd0cc0d2cc0 con 0x7fd0cc1616b0 2024-09-16T15:48:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:04.593 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":172,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:48:03.623662+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:01.416561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:02.578764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.593+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0ac079270 msgr2=0x7fd0ac07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.593+0000 7fd0cb200640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0ac079270 0x7fd0ac07b730 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc034000 tx=0x7fd0bc004730 comp rx=0 tx=0).stop 2024-09-16T15:48:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1616b0 msgr2=0x7fd0cc1578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1616b0 0x7fd0cc1578f0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400caa0 tx=0x7fd0c400cf60 comp rx=0 tx=0).stop 2024-09-16T15:48:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 shutdown_connections 2024-09-16T15:48:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd0ac079270 0x7fd0ac07b730 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0cc1616b0 0x7fd0cc1578f0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0cc1612d0 0x7fd0cc06d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 --2- 172.21.15.60:0/1487267250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0cc0d4980 0x7fd0cc0c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 >> 172.21.15.60:0/1487267250 conn(0x7fd0cc0d3df0 msgr2=0x7fd0cc0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 shutdown_connections 2024-09-16T15:48:04.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.594+0000 7fd0cb200640 1 -- 172.21.15.60:0/1487267250 wait complete. 2024-09-16T15:48:04.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:04.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.771+0000 7f97b6c00640 1 -- 172.21.15.60:0/4101182165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9794004800 msgr2=0x7f9794004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.771+0000 7f97b6c00640 1 --2- 172.21.15.60:0/4101182165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9794004800 0x7f9794004c80 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f9798009b50 tx=0x7f9794005830 comp rx=0 tx=0).stop 2024-09-16T15:48:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.771+0000 7f97b4c00640 1 -- 172.21.15.60:0/4101182165 reap_dead start 2024-09-16T15:48:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.771+0000 7f97b6c00640 1 -- 172.21.15.60:0/4101182165 shutdown_connections 2024-09-16T15:48:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.771+0000 7f97b6c00640 1 -- 172.21.15.60:0/4101182165 >> 172.21.15.60:0/4101182165 conn(0x7f97b00b1950 msgr2=0x7f97b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.771+0000 7f97b6c00640 1 -- 172.21.15.60:0/4101182165 shutdown_connections 2024-09-16T15:48:04.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.771+0000 7f97b6c00640 1 -- 172.21.15.60:0/4101182165 wait complete. 2024-09-16T15:48:04.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.773+0000 7f97b6c00640 1 Processor -- start 2024-09-16T15:48:04.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.773+0000 7f97b6c00640 1 -- start start 2024-09-16T15:48:04.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97b6c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9794004800 0x7f97b00ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97b6c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b0154f10 0x7f97b014eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97b6c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b009c7d0 0x7f97b009cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97b6c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f97b006f9f0 con 0x7f97b0154f10 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97b6c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97b006f870 con 0x7f9794004800 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97b6c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f97b00b2440 con 0x7f97b009c7d0 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97a7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b0154f10 0x7f97b014eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97a7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b0154f10 0x7f97b014eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45700/0 (socket says 172.21.15.60:45700) 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97a7400640 1 -- 172.21.15.60:0/948626214 learned_addr learned my addr 172.21.15.60:0/948626214 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97afe00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9794004800 0x7f97b00ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.774+0000 7f97b4c00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b009c7d0 0x7f97b009cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97a7400640 1 -- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b009c7d0 msgr2=0x7f97b009cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97a7400640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b009c7d0 0x7f97b009cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97a7400640 1 -- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9794004800 msgr2=0x7f97b00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97a7400640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9794004800 0x7f97b00ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97a7400640 1 -- 172.21.15.60:0/948626214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b009d190 con 0x7f97b0154f10 2024-09-16T15:48:04.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97afe00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9794004800 0x7f97b00ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:04.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97a7400640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b0154f10 0x7f97b014eaa0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f9798036040 tx=0x7f9798033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:04.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97acc00640 1 -- 172.21.15.60:0/948626214 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798043070 con 0x7f97b0154f10 2024-09-16T15:48:04.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97acc00640 1 -- 172.21.15.60:0/948626214 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9798004320 con 0x7f97b0154f10 2024-09-16T15:48:04.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97acc00640 1 -- 172.21.15.60:0/948626214 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798038040 con 0x7f97b0154f10 2024-09-16T15:48:04.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b00ce9a0 con 0x7f97b0154f10 2024-09-16T15:48:04.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.775+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b00b0380 con 0x7f97b0154f10 2024-09-16T15:48:04.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.776+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b0074440 con 0x7f97b0154f10 2024-09-16T15:48:04.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.777+0000 7f97acc00640 1 -- 172.21.15.60:0/948626214 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f9798046070 con 0x7f97b0154f10 2024-09-16T15:48:04.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.778+0000 7f97acc00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f978c079270 0x7f978c07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.778+0000 7f97acc00640 1 -- 172.21.15.60:0/948626214 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 5737+0+0 (secure 0 0 0) 0x7f97980b99b0 con 0x7f97b0154f10 2024-09-16T15:48:04.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.779+0000 7f97afe00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f978c079270 0x7f978c07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.779+0000 7f97afe00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f978c079270 0x7f978c07b730 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f97a0000f80 tx=0x7f97a0006ef0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:04.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.782+0000 7f97acc00640 1 -- 172.21.15.60:0/948626214 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9798083200 con 0x7f97b0154f10 2024-09-16T15:48:04.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.978+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97b00cc6f0 con 0x7f97b0154f10 2024-09-16T15:48:04.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.980+0000 7f97acc00640 1 -- 172.21.15.60:0/948626214 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) ==== 74+0+16800 (secure 0 0 0) 0x7f9798087c30 con 0x7f97b0154f10 2024-09-16T15:48:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:04.982 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":172,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:48:03.623662+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:01.416561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:02.578764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f978c079270 msgr2=0x7f978c07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f978c079270 0x7f978c07b730 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f97a0000f80 tx=0x7f97a0006ef0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b0154f10 msgr2=0x7f97b014eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b0154f10 0x7f97b014eaa0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f9798036040 tx=0x7f9798033670 comp rx=0 tx=0).stop 2024-09-16T15:48:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 shutdown_connections 2024-09-16T15:48:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f978c079270 0x7f978c07b730 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b009c7d0 0x7f97b009cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b0154f10 0x7f97b014eaa0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 --2- 172.21.15.60:0/948626214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9794004800 0x7f97b00ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 >> 172.21.15.60:0/948626214 conn(0x7f97b00b1950 msgr2=0x7f97b00cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 shutdown_connections 2024-09-16T15:48:04.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:04.982+0000 7f97b6c00640 1 -- 172.21.15.60:0/948626214 wait complete. 2024-09-16T15:48:05.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:48:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.164+0000 7fabdbe00640 1 -- 172.21.15.60:0/44031033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 msgr2=0x7fabbc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.164+0000 7fabdbe00640 1 --2- 172.21.15.60:0/44031033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 0x7fabbc003e20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fabc800bb20 tx=0x7fabbc005830 comp rx=0 tx=0).stop 2024-09-16T15:48:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.164+0000 7fabd9e00640 1 -- 172.21.15.60:0/44031033 reap_dead start 2024-09-16T15:48:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.164+0000 7fabdbe00640 1 -- 172.21.15.60:0/44031033 shutdown_connections 2024-09-16T15:48:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.164+0000 7fabdbe00640 1 -- 172.21.15.60:0/44031033 >> 172.21.15.60:0/44031033 conn(0x7fabd40b1840 msgr2=0x7fabd40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.164+0000 7fabdbe00640 1 -- 172.21.15.60:0/44031033 shutdown_connections 2024-09-16T15:48:05.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.164+0000 7fabdbe00640 1 -- 172.21.15.60:0/44031033 wait complete. 2024-09-16T15:48:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.166+0000 7fabdbe00640 1 Processor -- start 2024-09-16T15:48:05.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.166+0000 7fabdbe00640 1 -- start start 2024-09-16T15:48:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 0x7fabd4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415c7a0 0x7fabd41586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd415cc20 0x7fabd41536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabdbe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fabd40ce4a0 con 0x7fabd415cc20 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabdbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fabd4070470 con 0x7fabbc003a00 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabdbe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fabd40ce620 con 0x7fabd415c7a0 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabd9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd415cc20 0x7fabd41536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabd9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 0x7fabd4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabd9400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 0x7fabd4160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44630/0 (socket says 172.21.15.60:44630) 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabd9400640 1 -- 172.21.15.60:0/2835734812 learned_addr learned my addr 172.21.15.60:0/2835734812 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.167+0000 7fabd8a00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415c7a0 0x7fabd41586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd9e00640 1 -- 172.21.15.60:0/2835734812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415c7a0 msgr2=0x7fabd41586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd9e00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415c7a0 0x7fabd41586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd9e00640 1 -- 172.21.15.60:0/2835734812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 msgr2=0x7fabd4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd9e00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 0x7fabd4160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd9e00640 1 -- 172.21.15.60:0/2835734812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd4158460 con 0x7fabd415cc20 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd9400640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 0x7fabd4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd9e00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd415cc20 0x7fabd41536e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fabd40cfba0 tx=0x7fabcc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:05.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd1600640 1 -- 172.21.15.60:0/2835734812 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc009b20 con 0x7fabd415cc20 2024-09-16T15:48:05.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd1600640 1 -- 172.21.15.60:0/2835734812 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabcc015030 con 0x7fabd415cc20 2024-09-16T15:48:05.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabd1600640 1 -- 172.21.15.60:0/2835734812 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc003e10 con 0x7fabd415cc20 2024-09-16T15:48:05.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd4157ea0 con 0x7fabd415cc20 2024-09-16T15:48:05.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.168+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd415fd40 con 0x7fabd415cc20 2024-09-16T15:48:05.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.170+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd40cf090 con 0x7fabd415cc20 2024-09-16T15:48:05.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.171+0000 7fabd1600640 1 -- 172.21.15.60:0/2835734812 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fabcc004030 con 0x7fabd415cc20 2024-09-16T15:48:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.172+0000 7fabd1600640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fabb4079270 0x7fabb407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.172+0000 7fabd1600640 1 -- 172.21.15.60:0/2835734812 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 5737+0+0 (secure 0 0 0) 0x7fabcc099ca0 con 0x7fabd415cc20 2024-09-16T15:48:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.172+0000 7fabd9400640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fabb4079270 0x7fabb407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.173+0000 7fabd9400640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fabb4079270 0x7fabb407b730 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fabc800b540 tx=0x7fabc803c040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:05.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.176+0000 7fabd1600640 1 -- 172.21.15.60:0/2835734812 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fabcc0634f0 con 0x7fabd415cc20 2024-09-16T15:48:05.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:05.415+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fabd40b07c0 con 0x7fabd415cc20 2024-09-16T15:48:06.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.201+0000 7fabd1600640 1 -- 172.21.15.60:0/2835734812 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v121) ==== 118+0+0 (secure 0 0 0) 0x7fabcc067f20 con 0x7fabd415cc20 2024-09-16T15:48:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fabb4079270 msgr2=0x7fabb407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fabb4079270 0x7fabb407b730 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fabc800b540 tx=0x7fabc803c040 comp rx=0 tx=0).stop 2024-09-16T15:48:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd415cc20 msgr2=0x7fabd41536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd415cc20 0x7fabd41536e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fabd40cfba0 tx=0x7fabcc00e720 comp rx=0 tx=0).stop 2024-09-16T15:48:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 shutdown_connections 2024-09-16T15:48:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fabb4079270 0x7fabb407b730 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fabd415cc20 0x7fabd41536e0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fabd415c7a0 0x7fabd41586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 --2- 172.21.15.60:0/2835734812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fabbc003a00 0x7fabd4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 >> 172.21.15.60:0/2835734812 conn(0x7fabd40b1840 msgr2=0x7fabd40ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 shutdown_connections 2024-09-16T15:48:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.205+0000 7fabdbe00640 1 -- 172.21.15.60:0/2835734812 wait complete. 2024-09-16T15:48:06.225 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:48:06.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:48:06.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.395+0000 7fbeea400640 1 -- 172.21.15.60:0/307835502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee40cccd0 msgr2=0x7fbee4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.395+0000 7fbeea400640 1 --2- 172.21.15.60:0/307835502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee40cccd0 0x7fbee4073440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbecc005fd0 tx=0x7fbecc02a540 comp rx=0 tx=0).stop 2024-09-16T15:48:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.395+0000 7fbeea400640 1 -- 172.21.15.60:0/307835502 shutdown_connections 2024-09-16T15:48:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.395+0000 7fbeea400640 1 --2- 172.21.15.60:0/307835502 >> v2:172.21.15.100:3300/0 conn(0x7fbee406efa0 0x7fbee40cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.395+0000 7fbeea400640 1 --2- 172.21.15.60:0/307835502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee40cccd0 0x7fbee4073440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.395+0000 7fbeea400640 1 -- 172.21.15.60:0/307835502 >> 172.21.15.60:0/307835502 conn(0x7fbee40b1840 msgr2=0x7fbee40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.395+0000 7fbeea400640 1 -- 172.21.15.60:0/307835502 shutdown_connections 2024-09-16T15:48:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.395+0000 7fbeea400640 1 -- 172.21.15.60:0/307835502 wait complete. 2024-09-16T15:48:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.397+0000 7fbeea400640 1 Processor -- start 2024-09-16T15:48:06.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.397+0000 7fbeea400640 1 -- start start 2024-09-16T15:48:06.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbeea400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee406efa0 0x7fbee415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbeea400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee40cccd0 0x7fbee413e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbeea400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee4145750 0x7fbee41f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbeea400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fbee40ce480 con 0x7fbee40cccd0 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbeea400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbee40ce300 con 0x7fbee406efa0 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbeea400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fbee40ce600 con 0x7fbee4145750 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbedaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee40cccd0 0x7fbee413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbedaa00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee40cccd0 0x7fbee413e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45712/0 (socket says 172.21.15.60:45712) 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbedaa00640 1 -- 172.21.15.60:0/2444153937 learned_addr learned my addr 172.21.15.60:0/2444153937 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbee3e00640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee4145750 0x7fbee41f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.398+0000 7fbee3400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee406efa0 0x7fbee415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedaa00640 1 -- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee4145750 msgr2=0x7fbee41f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedaa00640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee4145750 0x7fbee41f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedaa00640 1 -- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee406efa0 msgr2=0x7fbee415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedaa00640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee406efa0 0x7fbee415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedaa00640 1 -- 172.21.15.60:0/2444153937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee41f5140 con 0x7fbee40cccd0 2024-09-16T15:48:06.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbee3e00640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee4145750 0x7fbee41f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:06.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedaa00640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee40cccd0 0x7fbee413e940 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009850 tx=0x7fbecc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:06.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedbe00640 1 -- 172.21.15.60:0/2444153937 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc03f070 con 0x7fbee40cccd0 2024-09-16T15:48:06.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedbe00640 1 -- 172.21.15.60:0/2444153937 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbecc0047a0 con 0x7fbee40cccd0 2024-09-16T15:48:06.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbedbe00640 1 -- 172.21.15.60:0/2444153937 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc035030 con 0x7fbee40cccd0 2024-09-16T15:48:06.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee41f5420 con 0x7fbee40cccd0 2024-09-16T15:48:06.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.399+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee40c4fa0 con 0x7fbee40cccd0 2024-09-16T15:48:06.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.401+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee40c4ab0 con 0x7fbee40cccd0 2024-09-16T15:48:06.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.402+0000 7fbedbe00640 1 -- 172.21.15.60:0/2444153937 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fbecc042070 con 0x7fbee40cccd0 2024-09-16T15:48:06.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.403+0000 7fbedbe00640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbec0079270 0x7fbec007b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:06.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.403+0000 7fbedbe00640 1 -- 172.21.15.60:0/2444153937 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fbecc0ba150 con 0x7fbee40cccd0 2024-09-16T15:48:06.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.403+0000 7fbee3400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbec0079270 0x7fbec007b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:06.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.404+0000 7fbee3400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbec0079270 0x7fbec007b730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbed4001720 tx=0x7fbed4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:06.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.408+0000 7fbedbe00640 1 -- 172.21.15.60:0/2444153937 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbecc083a00 con 0x7fbee40cccd0 2024-09-16T15:48:06.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:06.643+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbee40cdbb0 con 0x7fbee40cccd0 2024-09-16T15:48:07.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.207+0000 7fbedbe00640 1 -- 172.21.15.60:0/2444153937 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v122) ==== 122+0+0 (secure 0 0 0) 0x7fbecc088430 con 0x7fbee40cccd0 2024-09-16T15:48:07.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbec0079270 msgr2=0x7fbec007b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbec0079270 0x7fbec007b730 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbed4001720 tx=0x7fbed4009040 comp rx=0 tx=0).stop 2024-09-16T15:48:07.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee40cccd0 msgr2=0x7fbee413e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee40cccd0 0x7fbee413e940 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009850 tx=0x7fbecc0038a0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 shutdown_connections 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fbec0079270 0x7fbec007b730 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbee4145750 0x7fbee41f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbee40cccd0 0x7fbee413e940 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 --2- 172.21.15.60:0/2444153937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbee406efa0 0x7fbee415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 >> 172.21.15.60:0/2444153937 conn(0x7fbee40b1840 msgr2=0x7fbee40b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 shutdown_connections 2024-09-16T15:48:07.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.210+0000 7fbeea400640 1 -- 172.21.15.60:0/2444153937 wait complete. 2024-09-16T15:48:07.231 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:48:07.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.394+0000 7f96fbe00640 1 -- 172.21.15.60:0/759877160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96fc1612d0 msgr2=0x7f96fc0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.394+0000 7f96fbe00640 1 --2- 172.21.15.60:0/759877160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96fc1612d0 0x7f96fc0b4e00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009860 tx=0x7f96ec02a530 comp rx=0 tx=0).stop 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.394+0000 7f96fbe00640 1 -- 172.21.15.60:0/759877160 shutdown_connections 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.394+0000 7f96fbe00640 1 --2- 172.21.15.60:0/759877160 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96fc1612d0 0x7f96fc0b4e00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.394+0000 7f96fbe00640 1 -- 172.21.15.60:0/759877160 >> 172.21.15.60:0/759877160 conn(0x7f96fc0d3df0 msgr2=0x7f96fc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.395+0000 7f96fbe00640 1 -- 172.21.15.60:0/759877160 shutdown_connections 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.395+0000 7f96fbe00640 1 -- 172.21.15.60:0/759877160 wait complete. 2024-09-16T15:48:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.396+0000 7f96fbe00640 1 Processor -- start 2024-09-16T15:48:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.396+0000 7f96fbe00640 1 -- start start 2024-09-16T15:48:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96fc0d4980 0x7f96fc14b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96fc1612d0 0x7f96fc1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fbe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1616b0 0x7f96fc148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fbe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc0b1ee0 con 0x7f96fc1612d0 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fbe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc0b1d60 con 0x7f96fc1616b0 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fbe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0b2060 con 0x7f96fc0d4980 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fa000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96fc1612d0 0x7f96fc1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fa000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96fc1612d0 0x7f96fc1f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45718/0 (socket says 172.21.15.60:45718) 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96fa000640 1 -- 172.21.15.60:0/3719382210 learned_addr learned my addr 172.21.15.60:0/3719382210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.397+0000 7f96faa00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96fc0d4980 0x7f96fc14b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fb400640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1616b0 0x7f96fc148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fa000640 1 -- 172.21.15.60:0/3719382210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96fc0d4980 msgr2=0x7f96fc14b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fa000640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96fc0d4980 0x7f96fc14b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fa000640 1 -- 172.21.15.60:0/3719382210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1616b0 msgr2=0x7f96fc148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fa000640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1616b0 0x7f96fc148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fa000640 1 -- 172.21.15.60:0/3719382210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc14bce0 con 0x7f96fc1612d0 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fa000640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96fc1612d0 0x7f96fc1f3f90 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c420 tx=0x7f96e800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:07.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96f2a00640 1 -- 172.21.15.60:0/3719382210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8017070 con 0x7f96fc1612d0 2024-09-16T15:48:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96f2a00640 1 -- 172.21.15.60:0/3719382210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96e80053c0 con 0x7f96fc1612d0 2024-09-16T15:48:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96f2a00640 1 -- 172.21.15.60:0/3719382210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8003850 con 0x7f96fc1612d0 2024-09-16T15:48:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc1f44d0 con 0x7f96fc1612d0 2024-09-16T15:48:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.398+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc1f8500 con 0x7f96fc1612d0 2024-09-16T15:48:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.400+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96fc0b25a0 con 0x7f96fc1612d0 2024-09-16T15:48:07.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.401+0000 7f96f2a00640 1 -- 172.21.15.60:0/3719382210 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f96e801a070 con 0x7f96fc1612d0 2024-09-16T15:48:07.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.402+0000 7f96f2a00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f96dc079270 0x7f96dc07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.402+0000 7f96f2a00640 1 -- 172.21.15.60:0/3719382210 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f96e80987c0 con 0x7f96fc1612d0 2024-09-16T15:48:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.402+0000 7f96faa00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f96dc079270 0x7f96dc07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.403+0000 7f96faa00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f96dc079270 0x7f96dc07b730 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f96ec034000 tx=0x7f96ec004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:07.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.406+0000 7f96f2a00640 1 -- 172.21.15.60:0/3719382210 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f96e8062010 con 0x7f96fc1612d0 2024-09-16T15:48:07.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.639+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96fc0d0450 con 0x7f96fc1612d0 2024-09-16T15:48:07.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.640+0000 7f96f2a00640 1 -- 172.21.15.60:0/3719382210 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5227 (secure 0 0 0) 0x7f96e8066a40 con 0x7f96fc1612d0 2024-09-16T15:48:07.642 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:07.642 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":122,"btime":"2024-09-16T15:48:07:203498+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/657477165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":657477165},{"type":"v1","addr":"172.21.15.100:6801","nonce":657477165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/293588121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":293588121},{"type":"v1","addr":"172.21.15.100:6835","nonce":293588121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/903832682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":903832682},{"type":"v1","addr":"172.21.15.60:6834","nonce":903832682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:03.623622+0000","modified":"2024-09-16T15:48:06.645819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/786870322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":786870322},{"type":"v1","addr":"172.21.15.60:6835","nonce":786870322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9222,"qdb_cluster":[9222]},"id":13}]} 2024-09-16T15:48:07.642 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 122 2024-09-16T15:48:07.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f96dc079270 msgr2=0x7f96dc07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f96dc079270 0x7f96dc07b730 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f96ec034000 tx=0x7f96ec004730 comp rx=0 tx=0).stop 2024-09-16T15:48:07.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96fc1612d0 msgr2=0x7f96fc1f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96fc1612d0 0x7f96fc1f3f90 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c420 tx=0x7f96e800c8e0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 shutdown_connections 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f96dc079270 0x7f96dc07b730 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96fc1616b0 0x7f96fc148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96fc1612d0 0x7f96fc1f3f90 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 --2- 172.21.15.60:0/3719382210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96fc0d4980 0x7f96fc14b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 >> 172.21.15.60:0/3719382210 conn(0x7f96fc0d3df0 msgr2=0x7f96fc0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 shutdown_connections 2024-09-16T15:48:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.642+0000 7f96fbe00640 1 -- 172.21.15.60:0/3719382210 wait complete. 2024-09-16T15:48:07.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.819+0000 7f9de0000640 1 -- 172.21.15.60:0/1152217360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd80d28c0 msgr2=0x7f9dd80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.819+0000 7f9de0000640 1 --2- 172.21.15.60:0/1152217360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd80d28c0 0x7f9dd80cebb0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4005fd0 tx=0x7f9dc402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.820+0000 7f9de0000640 1 -- 172.21.15.60:0/1152217360 shutdown_connections 2024-09-16T15:48:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.820+0000 7f9de0000640 1 --2- 172.21.15.60:0/1152217360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd80d28c0 0x7f9dd80cebb0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.820+0000 7f9de0000640 1 --2- 172.21.15.60:0/1152217360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dd80b1a00 0x7f9dd80b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.820+0000 7f9de0000640 1 -- 172.21.15.60:0/1152217360 >> 172.21.15.60:0/1152217360 conn(0x7f9dd80d1da0 msgr2=0x7f9dd80d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.820+0000 7f9de0000640 1 -- 172.21.15.60:0/1152217360 shutdown_connections 2024-09-16T15:48:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.820+0000 7f9de0000640 1 -- 172.21.15.60:0/1152217360 wait complete. 2024-09-16T15:48:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.821+0000 7f9de0000640 1 Processor -- start 2024-09-16T15:48:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.821+0000 7f9de0000640 1 -- start start 2024-09-16T15:48:07.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.822+0000 7f9de0000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd80b1a00 0x7f9dd80b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.822+0000 7f9de0000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dd80d28c0 0x7f9dd81f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.822+0000 7f9de0000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd815c040 0x7f9dd81f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.822+0000 7f9de0000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd80b23d0 con 0x7f9dd815c040 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.822+0000 7f9de0000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd80cf300 con 0x7f9dd80b1a00 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.822+0000 7f9de0000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd80b2550 con 0x7f9dd80d28c0 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd815c040 0x7f9dd81f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd815c040 0x7f9dd81f8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45728/0 (socket says 172.21.15.60:45728) 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 -- 172.21.15.60:0/16919293 learned_addr learned my addr 172.21.15.60:0/16919293 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9ddcc00640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dd80d28c0 0x7f9dd81f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9ddd600640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd80b1a00 0x7f9dd80b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 -- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dd80d28c0 msgr2=0x7f9dd81f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dd80d28c0 0x7f9dd81f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 -- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd80b1a00 msgr2=0x7f9dd80b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd80b1a00 0x7f9dd80b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 -- 172.21.15.60:0/16919293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd80c86c0 con 0x7f9dd815c040 2024-09-16T15:48:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9ddcc00640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dd80d28c0 0x7f9dd81f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.823+0000 7f9dde000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd815c040 0x7f9dd81f8430 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000e970 tx=0x7f9dd000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.824+0000 7f9dd5600640 1 -- 172.21.15.60:0/16919293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000ccc0 con 0x7f9dd815c040 2024-09-16T15:48:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.824+0000 7f9dd5600640 1 -- 172.21.15.60:0/16919293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd000ce60 con 0x7f9dd815c040 2024-09-16T15:48:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.824+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd80b4c70 con 0x7f9dd815c040 2024-09-16T15:48:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.824+0000 7f9dd5600640 1 -- 172.21.15.60:0/16919293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0005280 con 0x7f9dd815c040 2024-09-16T15:48:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.824+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd80b7370 con 0x7f9dd815c040 2024-09-16T15:48:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.825+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd80b71a0 con 0x7f9dd815c040 2024-09-16T15:48:07.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.826+0000 7f9dd5600640 1 -- 172.21.15.60:0/16919293 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f9dd0005420 con 0x7f9dd815c040 2024-09-16T15:48:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.827+0000 7f9dd5600640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9db8079270 0x7f9db807b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.827+0000 7f9dd5600640 1 -- 172.21.15.60:0/16919293 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f9dd0099e50 con 0x7f9dd815c040 2024-09-16T15:48:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.827+0000 7f9ddd600640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9db8079270 0x7f9db807b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.828+0000 7f9ddd600640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9db8079270 0x7f9db807b730 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc001e20 tx=0x7f9dcc009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:07.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:07.831+0000 7f9dd5600640 1 -- 172.21.15.60:0/16919293 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9dd0063cf0 con 0x7f9dd815c040 2024-09-16T15:48:08.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.032+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dd80b5190 con 0x7f9dd815c040 2024-09-16T15:48:08.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.033+0000 7f9dd5600640 1 -- 172.21.15.60:0/16919293 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16787 (secure 0 0 0) 0x7f9dd80b5190 con 0x7f9dd815c040 2024-09-16T15:48:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:08.035 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":173,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:48:05.189290+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:01.416561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:02.578764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:08.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9db8079270 msgr2=0x7f9db807b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9db8079270 0x7f9db807b730 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc001e20 tx=0x7f9dcc009040 comp rx=0 tx=0).stop 2024-09-16T15:48:08.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd815c040 msgr2=0x7f9dd81f8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd815c040 0x7f9dd81f8430 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000e970 tx=0x7f9dd000ed70 comp rx=0 tx=0).stop 2024-09-16T15:48:08.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 shutdown_connections 2024-09-16T15:48:08.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9db8079270 0x7f9db807b730 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dd815c040 0x7f9dd81f8430 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dd80d28c0 0x7f9dd81f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 --2- 172.21.15.60:0/16919293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd80b1a00 0x7f9dd80b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 >> 172.21.15.60:0/16919293 conn(0x7f9dd80d1da0 msgr2=0x7f9dd80d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 shutdown_connections 2024-09-16T15:48:08.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.036+0000 7f9de0000640 1 -- 172.21.15.60:0/16919293 wait complete. 2024-09-16T15:48:08.055 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:48:08.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:08.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.215+0000 7f665f200640 1 -- 172.21.15.60:0/1020039991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66580d1610 msgr2=0x7f66580cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.215+0000 7f665f200640 1 --2- 172.21.15.60:0/1020039991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66580d1610 0x7f66580cecc0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f66580d4f60 tx=0x7f665002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.215+0000 7f665f200640 1 -- 172.21.15.60:0/1020039991 shutdown_connections 2024-09-16T15:48:08.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.215+0000 7f665f200640 1 --2- 172.21.15.60:0/1020039991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66580d1610 0x7f66580cecc0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.215+0000 7f665f200640 1 --2- 172.21.15.60:0/1020039991 >> v2:172.21.15.100:3300/0 conn(0x7f66580d2050 0x7f66580d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.215+0000 7f665f200640 1 -- 172.21.15.60:0/1020039991 >> 172.21.15.60:0/1020039991 conn(0x7f66580d3fe0 msgr2=0x7f66580d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.215+0000 7f665f200640 1 -- 172.21.15.60:0/1020039991 shutdown_connections 2024-09-16T15:48:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.215+0000 7f665f200640 1 -- 172.21.15.60:0/1020039991 wait complete. 2024-09-16T15:48:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.216+0000 7f665f200640 1 Processor -- start 2024-09-16T15:48:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.216+0000 7f665f200640 1 -- start start 2024-09-16T15:48:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.217+0000 7f665f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66580d1610 0x7f66580cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.217+0000 7f665f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66580d2050 0x7f66581f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.217+0000 7f665f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f665815e0d0 0x7f66580b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.217+0000 7f665f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f66580cf610 con 0x7f665815e0d0 2024-09-16T15:48:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.217+0000 7f665f200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f66580cf490 con 0x7f66580d2050 2024-09-16T15:48:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.217+0000 7f665f200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f66580cf790 con 0x7f66580d1610 2024-09-16T15:48:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f665815e0d0 0x7f66580b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f665815e0d0 0x7f66580b86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45750/0 (socket says 172.21.15.60:45750) 2024-09-16T15:48:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 -- 172.21.15.60:0/518398819 learned_addr learned my addr 172.21.15.60:0/518398819 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f6657e00640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66580d1610 0x7f66580cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f6657400640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66580d2050 0x7f66581f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 -- 172.21.15.60:0/518398819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66580d1610 msgr2=0x7f66580cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66580d1610 0x7f66580cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 -- 172.21.15.60:0/518398819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66580d2050 msgr2=0x7f66581f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66580d2050 0x7f66581f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 -- 172.21.15.60:0/518398819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66580cb510 con 0x7f665815e0d0 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.218+0000 7f665d200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f665815e0d0 0x7f66580b86f0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6650009ca0 tx=0x7f6650031690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.219+0000 7f664fe00640 1 -- 172.21.15.60:0/518398819 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650040070 con 0x7f665815e0d0 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.219+0000 7f664fe00640 1 -- 172.21.15.60:0/518398819 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6650004040 con 0x7f665815e0d0 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.219+0000 7f664fe00640 1 -- 172.21.15.60:0/518398819 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650004470 con 0x7f665815e0d0 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.219+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66580cb780 con 0x7f665815e0d0 2024-09-16T15:48:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.219+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66580cbd40 con 0x7f665815e0d0 2024-09-16T15:48:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.220+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66580b9ed0 con 0x7f665815e0d0 2024-09-16T15:48:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.221+0000 7f664fe00640 1 -- 172.21.15.60:0/518398819 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f6650039040 con 0x7f665815e0d0 2024-09-16T15:48:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.222+0000 7f664fe00640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6638079270 0x7f663807b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.222+0000 7f664fe00640 1 -- 172.21.15.60:0/518398819 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f66500ba8f0 con 0x7f665815e0d0 2024-09-16T15:48:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.222+0000 7f6657e00640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6638079270 0x7f663807b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.223+0000 7f6657e00640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6638079270 0x7f663807b730 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f665815ffc0 tx=0x7f6648009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.226+0000 7f664fe00640 1 -- 172.21.15.60:0/518398819 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6650084790 con 0x7f665815e0d0 2024-09-16T15:48:08.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.420+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66580ce220 con 0x7f6638079270 2024-09-16T15:48:08.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.424+0000 7f664fe00640 1 -- 172.21.15.60:0/518398819 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f66580ce220 con 0x7f6638079270 2024-09-16T15:48:08.429 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:08.430 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:48:08.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6638079270 msgr2=0x7f663807b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6638079270 0x7f663807b730 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f665815ffc0 tx=0x7f6648009040 comp rx=0 tx=0).stop 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f665815e0d0 msgr2=0x7f66580b86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f665815e0d0 0x7f66580b86f0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6650009ca0 tx=0x7f6650031690 comp rx=0 tx=0).stop 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 shutdown_connections 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6638079270 0x7f663807b730 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f665815e0d0 0x7f66580b86f0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66580d2050 0x7f66581f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 --2- 172.21.15.60:0/518398819 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66580d1610 0x7f66580cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.428+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 >> 172.21.15.60:0/518398819 conn(0x7f66580d3fe0 msgr2=0x7f665806d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.429+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 shutdown_connections 2024-09-16T15:48:08.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.429+0000 7f665f200640 1 -- 172.21.15.60:0/518398819 wait complete. 2024-09-16T15:48:08.447 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-16T15:48:08.225123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284328,"kb_used_data":18168,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690392,"statfs":{"total":773094113280,"available":772802961408,"internally_reserved":0,"allocated":18604032,"data_stored":8850676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001076"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823153+0000","last_change":"2024-09-16T15:48:04.197922+0000","last_active":"2024-09-16T15:48:06.823153+0000","last_peered":"2024-09-16T15:48:06.823153+0000","last_clean":"2024-09-16T15:48:06.823153+0000","last_became_active":"2024-09-16T15:48:04.197638+0000","last_became_peered":"2024-09-16T15:48:04.197638+0000","last_unstale":"2024-09-16T15:48:06.823153+0000","last_undegraded":"2024-09-16T15:48:06.823153+0000","last_fullsized":"2024-09-16T15:48:06.823153+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:04:59.723888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823289+0000","last_change":"2024-09-16T15:48:04.198923+0000","last_active":"2024-09-16T15:48:06.823289+0000","last_peered":"2024-09-16T15:48:06.823289+0000","last_clean":"2024-09-16T15:48:06.823289+0000","last_became_active":"2024-09-16T15:48:04.198553+0000","last_became_peered":"2024-09-16T15:48:04.198553+0000","last_unstale":"2024-09-16T15:48:06.823289+0000","last_undegraded":"2024-09-16T15:48:06.823289+0000","last_fullsized":"2024-09-16T15:48:06.823289+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:48:13.533119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274228+0000","last_change":"2024-09-16T15:48:04.202180+0000","last_active":"2024-09-16T15:48:06.274228+0000","last_peered":"2024-09-16T15:48:06.274228+0000","last_clean":"2024-09-16T15:48:06.274228+0000","last_became_active":"2024-09-16T15:48:04.201914+0000","last_became_peered":"2024-09-16T15:48:04.201914+0000","last_unstale":"2024-09-16T15:48:06.274228+0000","last_undegraded":"2024-09-16T15:48:06.274228+0000","last_fullsized":"2024-09-16T15:48:06.274228+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:13.850947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182823+0000","last_change":"2024-09-16T15:48:04.200120+0000","last_active":"2024-09-16T15:48:06.182823+0000","last_peered":"2024-09-16T15:48:06.182823+0000","last_clean":"2024-09-16T15:48:06.182823+0000","last_became_active":"2024-09-16T15:48:04.199776+0000","last_became_peered":"2024-09-16T15:48:04.199776+0000","last_unstale":"2024-09-16T15:48:06.182823+0000","last_undegraded":"2024-09-16T15:48:06.182823+0000","last_fullsized":"2024-09-16T15:48:06.182823+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:09:56.020663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585528+0000","last_change":"2024-09-16T15:48:04.207577+0000","last_active":"2024-09-16T15:48:05.585528+0000","last_peered":"2024-09-16T15:48:05.585528+0000","last_clean":"2024-09-16T15:48:05.585528+0000","last_became_active":"2024-09-16T15:48:04.207290+0000","last_became_peered":"2024-09-16T15:48:04.207290+0000","last_unstale":"2024-09-16T15:48:05.585528+0000","last_undegraded":"2024-09-16T15:48:05.585528+0000","last_fullsized":"2024-09-16T15:48:05.585528+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:25:36.672954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585371+0000","last_change":"2024-09-16T15:48:04.203301+0000","last_active":"2024-09-16T15:48:05.585371+0000","last_peered":"2024-09-16T15:48:05.585371+0000","last_clean":"2024-09-16T15:48:05.585371+0000","last_became_active":"2024-09-16T15:48:04.202831+0000","last_became_peered":"2024-09-16T15:48:04.202831+0000","last_unstale":"2024-09-16T15:48:05.585371+0000","last_undegraded":"2024-09-16T15:48:05.585371+0000","last_fullsized":"2024-09-16T15:48:05.585371+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:14.620274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822492+0000","last_change":"2024-09-16T15:48:04.210825+0000","last_active":"2024-09-16T15:48:06.822492+0000","last_peered":"2024-09-16T15:48:06.822492+0000","last_clean":"2024-09-16T15:48:06.822492+0000","last_became_active":"2024-09-16T15:48:04.199466+0000","last_became_peered":"2024-09-16T15:48:04.199466+0000","last_unstale":"2024-09-16T15:48:06.822492+0000","last_undegraded":"2024-09-16T15:48:06.822492+0000","last_fullsized":"2024-09-16T15:48:06.822492+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:20.525297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585214+0000","last_change":"2024-09-16T15:48:04.190660+0000","last_active":"2024-09-16T15:48:05.585214+0000","last_peered":"2024-09-16T15:48:05.585214+0000","last_clean":"2024-09-16T15:48:05.585214+0000","last_became_active":"2024-09-16T15:48:04.190413+0000","last_became_peered":"2024-09-16T15:48:04.190413+0000","last_unstale":"2024-09-16T15:48:05.585214+0000","last_undegraded":"2024-09-16T15:48:05.585214+0000","last_fullsized":"2024-09-16T15:48:05.585214+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:58.822830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823533+0000","last_change":"2024-09-16T15:48:04.192862+0000","last_active":"2024-09-16T15:48:06.823533+0000","last_peered":"2024-09-16T15:48:06.823533+0000","last_clean":"2024-09-16T15:48:06.823533+0000","last_became_active":"2024-09-16T15:48:04.191793+0000","last_became_peered":"2024-09-16T15:48:04.191793+0000","last_unstale":"2024-09-16T15:48:06.823533+0000","last_undegraded":"2024-09-16T15:48:06.823533+0000","last_fullsized":"2024-09-16T15:48:06.823533+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:20.270919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198597+0000","last_change":"2024-09-16T15:48:04.194667+0000","last_active":"2024-09-16T15:48:05.198597+0000","last_peered":"2024-09-16T15:48:05.198597+0000","last_clean":"2024-09-16T15:48:05.198597+0000","last_became_active":"2024-09-16T15:48:04.194308+0000","last_became_peered":"2024-09-16T15:48:04.194308+0000","last_unstale":"2024-09-16T15:48:05.198597+0000","last_undegraded":"2024-09-16T15:48:05.198597+0000","last_fullsized":"2024-09-16T15:48:05.198597+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:18.240793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274365+0000","last_change":"2024-09-16T15:48:04.204567+0000","last_active":"2024-09-16T15:48:06.274365+0000","last_peered":"2024-09-16T15:48:06.274365+0000","last_clean":"2024-09-16T15:48:06.274365+0000","last_became_active":"2024-09-16T15:48:04.204305+0000","last_became_peered":"2024-09-16T15:48:04.204305+0000","last_unstale":"2024-09-16T15:48:06.274365+0000","last_undegraded":"2024-09-16T15:48:06.274365+0000","last_fullsized":"2024-09-16T15:48:06.274365+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:34.637763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200208+0000","last_change":"2024-09-16T15:48:04.197592+0000","last_active":"2024-09-16T15:48:05.200208+0000","last_peered":"2024-09-16T15:48:05.200208+0000","last_clean":"2024-09-16T15:48:05.200208+0000","last_became_active":"2024-09-16T15:48:04.197325+0000","last_became_peered":"2024-09-16T15:48:04.197325+0000","last_unstale":"2024-09-16T15:48:05.200208+0000","last_undegraded":"2024-09-16T15:48:05.200208+0000","last_fullsized":"2024-09-16T15:48:05.200208+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.852073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200290+0000","last_change":"2024-09-16T15:48:04.192138+0000","last_active":"2024-09-16T15:48:05.200290+0000","last_peered":"2024-09-16T15:48:05.200290+0000","last_clean":"2024-09-16T15:48:05.200290+0000","last_became_active":"2024-09-16T15:48:04.190454+0000","last_became_peered":"2024-09-16T15:48:04.190454+0000","last_unstale":"2024-09-16T15:48:05.200290+0000","last_undegraded":"2024-09-16T15:48:05.200290+0000","last_fullsized":"2024-09-16T15:48:05.200290+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:01.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822547+0000","last_change":"2024-09-16T15:48:04.205762+0000","last_active":"2024-09-16T15:48:06.822547+0000","last_peered":"2024-09-16T15:48:06.822547+0000","last_clean":"2024-09-16T15:48:06.822547+0000","last_became_active":"2024-09-16T15:48:04.205480+0000","last_became_peered":"2024-09-16T15:48:04.205480+0000","last_unstale":"2024-09-16T15:48:06.822547+0000","last_undegraded":"2024-09-16T15:48:06.822547+0000","last_fullsized":"2024-09-16T15:48:06.822547+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:34.503811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822489+0000","last_change":"2024-09-16T15:48:04.208965+0000","last_active":"2024-09-16T15:48:06.822489+0000","last_peered":"2024-09-16T15:48:06.822489+0000","last_clean":"2024-09-16T15:48:06.822489+0000","last_became_active":"2024-09-16T15:48:04.203160+0000","last_became_peered":"2024-09-16T15:48:04.203160+0000","last_unstale":"2024-09-16T15:48:06.822489+0000","last_undegraded":"2024-09-16T15:48:06.822489+0000","last_fullsized":"2024-09-16T15:48:06.822489+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:31.962754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.197860+0000","last_change":"2024-09-16T15:48:04.192434+0000","last_active":"2024-09-16T15:48:05.197860+0000","last_peered":"2024-09-16T15:48:05.197860+0000","last_clean":"2024-09-16T15:48:05.197860+0000","last_became_active":"2024-09-16T15:48:04.192147+0000","last_became_peered":"2024-09-16T15:48:04.192147+0000","last_unstale":"2024-09-16T15:48:05.197860+0000","last_undegraded":"2024-09-16T15:48:05.197860+0000","last_fullsized":"2024-09-16T15:48:05.197860+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:37.016457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823092+0000","last_change":"2024-09-16T15:48:04.194831+0000","last_active":"2024-09-16T15:48:06.823092+0000","last_peered":"2024-09-16T15:48:06.823092+0000","last_clean":"2024-09-16T15:48:06.823092+0000","last_became_active":"2024-09-16T15:48:04.194554+0000","last_became_peered":"2024-09-16T15:48:04.194554+0000","last_unstale":"2024-09-16T15:48:06.823092+0000","last_undegraded":"2024-09-16T15:48:06.823092+0000","last_fullsized":"2024-09-16T15:48:06.823092+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:24.793488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198166+0000","last_change":"2024-09-16T15:48:04.202550+0000","last_active":"2024-09-16T15:48:05.198166+0000","last_peered":"2024-09-16T15:48:05.198166+0000","last_clean":"2024-09-16T15:48:05.198166+0000","last_became_active":"2024-09-16T15:48:04.202267+0000","last_became_peered":"2024-09-16T15:48:04.202267+0000","last_unstale":"2024-09-16T15:48:05.198166+0000","last_undegraded":"2024-09-16T15:48:05.198166+0000","last_fullsized":"2024-09-16T15:48:05.198166+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:31.962754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198204+0000","last_change":"2024-09-16T15:48:04.201689+0000","last_active":"2024-09-16T15:48:05.198204+0000","last_peered":"2024-09-16T15:48:05.198204+0000","last_clean":"2024-09-16T15:48:05.198204+0000","last_became_active":"2024-09-16T15:48:04.201108+0000","last_became_peered":"2024-09-16T15:48:04.201108+0000","last_unstale":"2024-09-16T15:48:05.198204+0000","last_undegraded":"2024-09-16T15:48:05.198204+0000","last_fullsized":"2024-09-16T15:48:05.198204+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:28.133935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585620+0000","last_change":"2024-09-16T15:48:04.205783+0000","last_active":"2024-09-16T15:48:05.585620+0000","last_peered":"2024-09-16T15:48:05.585620+0000","last_clean":"2024-09-16T15:48:05.585620+0000","last_became_active":"2024-09-16T15:48:04.205044+0000","last_became_peered":"2024-09-16T15:48:04.205044+0000","last_unstale":"2024-09-16T15:48:05.585620+0000","last_undegraded":"2024-09-16T15:48:05.585620+0000","last_fullsized":"2024-09-16T15:48:05.585620+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:25.973298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182979+0000","last_change":"2024-09-16T15:48:04.212115+0000","last_active":"2024-09-16T15:48:06.182979+0000","last_peered":"2024-09-16T15:48:06.182979+0000","last_clean":"2024-09-16T15:48:06.182979+0000","last_became_active":"2024-09-16T15:48:04.211677+0000","last_became_peered":"2024-09-16T15:48:04.211677+0000","last_unstale":"2024-09-16T15:48:06.182979+0000","last_undegraded":"2024-09-16T15:48:06.182979+0000","last_fullsized":"2024-09-16T15:48:06.182979+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:24.963722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200620+0000","last_change":"2024-09-16T15:48:04.205057+0000","last_active":"2024-09-16T15:48:05.200620+0000","last_peered":"2024-09-16T15:48:05.200620+0000","last_clean":"2024-09-16T15:48:05.200620+0000","last_became_active":"2024-09-16T15:48:04.204768+0000","last_became_peered":"2024-09-16T15:48:04.204768+0000","last_unstale":"2024-09-16T15:48:05.200620+0000","last_undegraded":"2024-09-16T15:48:05.200620+0000","last_fullsized":"2024-09-16T15:48:05.200620+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:47:24.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200576+0000","last_change":"2024-09-16T15:48:04.195071+0000","last_active":"2024-09-16T15:48:05.200576+0000","last_peered":"2024-09-16T15:48:05.200576+0000","last_clean":"2024-09-16T15:48:05.200576+0000","last_became_active":"2024-09-16T15:48:04.190837+0000","last_became_peered":"2024-09-16T15:48:04.190837+0000","last_unstale":"2024-09-16T15:48:05.200576+0000","last_undegraded":"2024-09-16T15:48:05.200576+0000","last_fullsized":"2024-09-16T15:48:05.200576+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:31.497014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182759+0000","last_change":"2024-09-16T15:48:04.213153+0000","last_active":"2024-09-16T15:48:06.182759+0000","last_peered":"2024-09-16T15:48:06.182759+0000","last_clean":"2024-09-16T15:48:06.182759+0000","last_became_active":"2024-09-16T15:48:04.212863+0000","last_became_peered":"2024-09-16T15:48:04.212863+0000","last_unstale":"2024-09-16T15:48:06.182759+0000","last_undegraded":"2024-09-16T15:48:06.182759+0000","last_fullsized":"2024-09-16T15:48:06.182759+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:37.016457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823310+0000","last_change":"2024-09-16T15:48:04.208845+0000","last_active":"2024-09-16T15:48:06.823310+0000","last_peered":"2024-09-16T15:48:06.823310+0000","last_clean":"2024-09-16T15:48:06.823310+0000","last_became_active":"2024-09-16T15:48:04.208529+0000","last_became_peered":"2024-09-16T15:48:04.208529+0000","last_unstale":"2024-09-16T15:48:06.823310+0000","last_undegraded":"2024-09-16T15:48:06.823310+0000","last_fullsized":"2024-09-16T15:48:06.823310+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:10.867458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823132+0000","last_change":"2024-09-16T15:48:04.198585+0000","last_active":"2024-09-16T15:48:06.823132+0000","last_peered":"2024-09-16T15:48:06.823132+0000","last_clean":"2024-09-16T15:48:06.823132+0000","last_became_active":"2024-09-16T15:48:04.198323+0000","last_became_peered":"2024-09-16T15:48:04.198323+0000","last_unstale":"2024-09-16T15:48:06.823132+0000","last_undegraded":"2024-09-16T15:48:06.823132+0000","last_fullsized":"2024-09-16T15:48:06.823132+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:14:53.824138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200237+0000","last_change":"2024-09-16T15:48:04.196025+0000","last_active":"2024-09-16T15:48:05.200237+0000","last_peered":"2024-09-16T15:48:05.200237+0000","last_clean":"2024-09-16T15:48:05.200237+0000","last_became_active":"2024-09-16T15:48:04.195738+0000","last_became_peered":"2024-09-16T15:48:04.195738+0000","last_unstale":"2024-09-16T15:48:05.200237+0000","last_undegraded":"2024-09-16T15:48:05.200237+0000","last_fullsized":"2024-09-16T15:48:05.200237+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:17.050178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273863+0000","last_change":"2024-09-16T15:48:04.196230+0000","last_active":"2024-09-16T15:48:06.273863+0000","last_peered":"2024-09-16T15:48:06.273863+0000","last_clean":"2024-09-16T15:48:06.273863+0000","last_became_active":"2024-09-16T15:48:04.195976+0000","last_became_peered":"2024-09-16T15:48:04.195976+0000","last_unstale":"2024-09-16T15:48:06.273863+0000","last_undegraded":"2024-09-16T15:48:06.273863+0000","last_fullsized":"2024-09-16T15:48:06.273863+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:16.633554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200039+0000","last_change":"2024-09-16T15:48:04.196151+0000","last_active":"2024-09-16T15:48:05.200039+0000","last_peered":"2024-09-16T15:48:05.200039+0000","last_clean":"2024-09-16T15:48:05.200039+0000","last_became_active":"2024-09-16T15:48:04.195689+0000","last_became_peered":"2024-09-16T15:48:04.195689+0000","last_unstale":"2024-09-16T15:48:05.200039+0000","last_undegraded":"2024-09-16T15:48:05.200039+0000","last_fullsized":"2024-09-16T15:48:05.200039+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:40.002311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823110+0000","last_change":"2024-09-16T15:48:04.198675+0000","last_active":"2024-09-16T15:48:06.823110+0000","last_peered":"2024-09-16T15:48:06.823110+0000","last_clean":"2024-09-16T15:48:06.823110+0000","last_became_active":"2024-09-16T15:48:04.198074+0000","last_became_peered":"2024-09-16T15:48:04.198074+0000","last_unstale":"2024-09-16T15:48:06.823110+0000","last_undegraded":"2024-09-16T15:48:06.823110+0000","last_fullsized":"2024-09-16T15:48:06.823110+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:14.297001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198086+0000","last_change":"2024-09-16T15:48:04.213248+0000","last_active":"2024-09-16T15:48:05.198086+0000","last_peered":"2024-09-16T15:48:05.198086+0000","last_clean":"2024-09-16T15:48:05.198086+0000","last_became_active":"2024-09-16T15:48:04.203595+0000","last_became_peered":"2024-09-16T15:48:04.203595+0000","last_unstale":"2024-09-16T15:48:05.198086+0000","last_undegraded":"2024-09-16T15:48:05.198086+0000","last_fullsized":"2024-09-16T15:48:05.198086+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:00.748962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822443+0000","last_change":"2024-09-16T15:48:04.196041+0000","last_active":"2024-09-16T15:48:06.822443+0000","last_peered":"2024-09-16T15:48:06.822443+0000","last_clean":"2024-09-16T15:48:06.822443+0000","last_became_active":"2024-09-16T15:48:04.195742+0000","last_became_peered":"2024-09-16T15:48:04.195742+0000","last_unstale":"2024-09-16T15:48:06.822443+0000","last_undegraded":"2024-09-16T15:48:06.822443+0000","last_fullsized":"2024-09-16T15:48:06.822443+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:52.910495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823369+0000","last_change":"2024-09-16T15:48:04.197896+0000","last_active":"2024-09-16T15:48:06.823369+0000","last_peered":"2024-09-16T15:48:06.823369+0000","last_clean":"2024-09-16T15:48:06.823369+0000","last_became_active":"2024-09-16T15:48:04.197529+0000","last_became_peered":"2024-09-16T15:48:04.197529+0000","last_unstale":"2024-09-16T15:48:06.823369+0000","last_undegraded":"2024-09-16T15:48:06.823369+0000","last_fullsized":"2024-09-16T15:48:06.823369+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:40.034009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"172'4","reported_seq":19,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200465+0000","last_change":"2024-09-16T15:48:03.188074+0000","last_active":"2024-09-16T15:48:05.200465+0000","last_peered":"2024-09-16T15:48:05.200465+0000","last_clean":"2024-09-16T15:48:05.200465+0000","last_became_active":"2024-09-16T15:48:03.187768+0000","last_became_peered":"2024-09-16T15:48:03.187768+0000","last_unstale":"2024-09-16T15:48:05.200465+0000","last_undegraded":"2024-09-16T15:48:05.200465+0000","last_fullsized":"2024-09-16T15:48:05.200465+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:41.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200919+0000","last_change":"2024-09-16T15:48:04.196297+0000","last_active":"2024-09-16T15:48:05.200919+0000","last_peered":"2024-09-16T15:48:05.200919+0000","last_clean":"2024-09-16T15:48:05.200919+0000","last_became_active":"2024-09-16T15:48:04.196004+0000","last_became_peered":"2024-09-16T15:48:04.196004+0000","last_unstale":"2024-09-16T15:48:05.200919+0000","last_undegraded":"2024-09-16T15:48:05.200919+0000","last_fullsized":"2024-09-16T15:48:05.200919+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:42.602478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823537+0000","last_change":"2024-09-16T15:48:03.191407+0000","last_active":"2024-09-16T15:48:06.823537+0000","last_peered":"2024-09-16T15:48:06.823537+0000","last_clean":"2024-09-16T15:48:06.823537+0000","last_became_active":"2024-09-16T15:48:03.190533+0000","last_became_peered":"2024-09-16T15:48:03.190533+0000","last_unstale":"2024-09-16T15:48:06.823537+0000","last_undegraded":"2024-09-16T15:48:06.823537+0000","last_fullsized":"2024-09-16T15:48:06.823537+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:54.227271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822828+0000","last_change":"2024-09-16T15:48:04.203696+0000","last_active":"2024-09-16T15:48:06.822828+0000","last_peered":"2024-09-16T15:48:06.822828+0000","last_clean":"2024-09-16T15:48:06.822828+0000","last_became_active":"2024-09-16T15:48:04.200494+0000","last_became_peered":"2024-09-16T15:48:04.200494+0000","last_unstale":"2024-09-16T15:48:06.822828+0000","last_undegraded":"2024-09-16T15:48:06.822828+0000","last_fullsized":"2024-09-16T15:48:06.822828+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:58.976149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274282+0000","last_change":"2024-09-16T15:48:03.195571+0000","last_active":"2024-09-16T15:48:06.274282+0000","last_peered":"2024-09-16T15:48:06.274282+0000","last_clean":"2024-09-16T15:48:06.274282+0000","last_became_active":"2024-09-16T15:48:03.195294+0000","last_became_peered":"2024-09-16T15:48:03.195294+0000","last_unstale":"2024-09-16T15:48:06.274282+0000","last_undegraded":"2024-09-16T15:48:06.274282+0000","last_fullsized":"2024-09-16T15:48:06.274282+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:02.063117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.197978+0000","last_change":"2024-09-16T15:48:04.201365+0000","last_active":"2024-09-16T15:48:05.197978+0000","last_peered":"2024-09-16T15:48:05.197978+0000","last_clean":"2024-09-16T15:48:05.197978+0000","last_became_active":"2024-09-16T15:48:04.200751+0000","last_became_peered":"2024-09-16T15:48:04.200751+0000","last_unstale":"2024-09-16T15:48:05.197978+0000","last_undegraded":"2024-09-16T15:48:05.197978+0000","last_fullsized":"2024-09-16T15:48:05.197978+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:52.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182941+0000","last_change":"2024-09-16T15:48:03.200040+0000","last_active":"2024-09-16T15:48:06.182941+0000","last_peered":"2024-09-16T15:48:06.182941+0000","last_clean":"2024-09-16T15:48:06.182941+0000","last_became_active":"2024-09-16T15:48:03.199650+0000","last_became_peered":"2024-09-16T15:48:03.199650+0000","last_unstale":"2024-09-16T15:48:06.182941+0000","last_undegraded":"2024-09-16T15:48:06.182941+0000","last_fullsized":"2024-09-16T15:48:06.182941+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:29.770414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182828+0000","last_change":"2024-09-16T15:48:04.201062+0000","last_active":"2024-09-16T15:48:06.182828+0000","last_peered":"2024-09-16T15:48:06.182828+0000","last_clean":"2024-09-16T15:48:06.182828+0000","last_became_active":"2024-09-16T15:48:04.200724+0000","last_became_peered":"2024-09-16T15:48:04.200724+0000","last_unstale":"2024-09-16T15:48:06.182828+0000","last_undegraded":"2024-09-16T15:48:06.182828+0000","last_fullsized":"2024-09-16T15:48:06.182828+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:07:08.806828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585521+0000","last_change":"2024-09-16T15:48:03.181659+0000","last_active":"2024-09-16T15:48:05.585521+0000","last_peered":"2024-09-16T15:48:05.585521+0000","last_clean":"2024-09-16T15:48:05.585521+0000","last_became_active":"2024-09-16T15:48:03.181337+0000","last_became_peered":"2024-09-16T15:48:03.181337+0000","last_unstale":"2024-09-16T15:48:05.585521+0000","last_undegraded":"2024-09-16T15:48:05.585521+0000","last_fullsized":"2024-09-16T15:48:05.585521+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:36.908344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585296+0000","last_change":"2024-09-16T15:48:04.192582+0000","last_active":"2024-09-16T15:48:05.585296+0000","last_peered":"2024-09-16T15:48:05.585296+0000","last_clean":"2024-09-16T15:48:05.585296+0000","last_became_active":"2024-09-16T15:48:04.192299+0000","last_became_peered":"2024-09-16T15:48:04.192299+0000","last_unstale":"2024-09-16T15:48:05.585296+0000","last_undegraded":"2024-09-16T15:48:05.585296+0000","last_fullsized":"2024-09-16T15:48:05.585296+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:33:17.441611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585118+0000","last_change":"2024-09-16T15:48:03.185920+0000","last_active":"2024-09-16T15:48:05.585118+0000","last_peered":"2024-09-16T15:48:05.585118+0000","last_clean":"2024-09-16T15:48:05.585118+0000","last_became_active":"2024-09-16T15:48:03.185643+0000","last_became_peered":"2024-09-16T15:48:03.185643+0000","last_unstale":"2024-09-16T15:48:05.585118+0000","last_undegraded":"2024-09-16T15:48:05.585118+0000","last_fullsized":"2024-09-16T15:48:05.585118+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:08.674483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182684+0000","last_change":"2024-09-16T15:48:04.202736+0000","last_active":"2024-09-16T15:48:06.182684+0000","last_peered":"2024-09-16T15:48:06.182684+0000","last_clean":"2024-09-16T15:48:06.182684+0000","last_became_active":"2024-09-16T15:48:04.202330+0000","last_became_peered":"2024-09-16T15:48:04.202330+0000","last_unstale":"2024-09-16T15:48:06.182684+0000","last_undegraded":"2024-09-16T15:48:06.182684+0000","last_fullsized":"2024-09-16T15:48:06.182684+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:22.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.199009+0000","last_change":"2024-09-16T15:48:03.182323+0000","last_active":"2024-09-16T15:48:05.199009+0000","last_peered":"2024-09-16T15:48:05.199009+0000","last_clean":"2024-09-16T15:48:05.199009+0000","last_became_active":"2024-09-16T15:48:03.182062+0000","last_became_peered":"2024-09-16T15:48:03.182062+0000","last_unstale":"2024-09-16T15:48:05.199009+0000","last_undegraded":"2024-09-16T15:48:05.199009+0000","last_fullsized":"2024-09-16T15:48:05.199009+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:26.164248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.183232+0000","last_change":"2024-09-16T15:48:04.200373+0000","last_active":"2024-09-16T15:48:06.183232+0000","last_peered":"2024-09-16T15:48:06.183232+0000","last_clean":"2024-09-16T15:48:06.183232+0000","last_became_active":"2024-09-16T15:48:04.200041+0000","last_became_peered":"2024-09-16T15:48:04.200041+0000","last_unstale":"2024-09-16T15:48:06.183232+0000","last_undegraded":"2024-09-16T15:48:06.183232+0000","last_fullsized":"2024-09-16T15:48:06.183232+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:31.962754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585602+0000","last_change":"2024-09-16T15:48:03.191241+0000","last_active":"2024-09-16T15:48:05.585602+0000","last_peered":"2024-09-16T15:48:05.585602+0000","last_clean":"2024-09-16T15:48:05.585602+0000","last_became_active":"2024-09-16T15:48:03.190791+0000","last_became_peered":"2024-09-16T15:48:03.190791+0000","last_unstale":"2024-09-16T15:48:05.585602+0000","last_undegraded":"2024-09-16T15:48:05.585602+0000","last_fullsized":"2024-09-16T15:48:05.585602+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:59.334295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274301+0000","last_change":"2024-09-16T15:48:04.207673+0000","last_active":"2024-09-16T15:48:06.274301+0000","last_peered":"2024-09-16T15:48:06.274301+0000","last_clean":"2024-09-16T15:48:06.274301+0000","last_became_active":"2024-09-16T15:48:04.207320+0000","last_became_peered":"2024-09-16T15:48:04.207320+0000","last_unstale":"2024-09-16T15:48:06.274301+0000","last_undegraded":"2024-09-16T15:48:06.274301+0000","last_fullsized":"2024-09-16T15:48:06.274301+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:44.180969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.199725+0000","last_change":"2024-09-16T15:48:03.190471+0000","last_active":"2024-09-16T15:48:05.199725+0000","last_peered":"2024-09-16T15:48:05.199725+0000","last_clean":"2024-09-16T15:48:05.199725+0000","last_became_active":"2024-09-16T15:48:03.190083+0000","last_became_peered":"2024-09-16T15:48:03.190083+0000","last_unstale":"2024-09-16T15:48:05.199725+0000","last_undegraded":"2024-09-16T15:48:05.199725+0000","last_fullsized":"2024-09-16T15:48:05.199725+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:08.009241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182974+0000","last_change":"2024-09-16T15:48:04.202413+0000","last_active":"2024-09-16T15:48:06.182974+0000","last_peered":"2024-09-16T15:48:06.182974+0000","last_clean":"2024-09-16T15:48:06.182974+0000","last_became_active":"2024-09-16T15:48:04.202126+0000","last_became_peered":"2024-09-16T15:48:04.202126+0000","last_unstale":"2024-09-16T15:48:06.182974+0000","last_undegraded":"2024-09-16T15:48:06.182974+0000","last_fullsized":"2024-09-16T15:48:06.182974+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:00.748962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"172'3","reported_seq":18,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198240+0000","last_change":"2024-09-16T15:48:03.200450+0000","last_active":"2024-09-16T15:48:05.198240+0000","last_peered":"2024-09-16T15:48:05.198240+0000","last_clean":"2024-09-16T15:48:05.198240+0000","last_became_active":"2024-09-16T15:48:03.200189+0000","last_became_peered":"2024-09-16T15:48:03.200189+0000","last_unstale":"2024-09-16T15:48:05.198240+0000","last_undegraded":"2024-09-16T15:48:05.198240+0000","last_fullsized":"2024-09-16T15:48:05.198240+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.106382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822963+0000","last_change":"2024-09-16T15:48:04.206128+0000","last_active":"2024-09-16T15:48:06.822963+0000","last_peered":"2024-09-16T15:48:06.822963+0000","last_clean":"2024-09-16T15:48:06.822963+0000","last_became_active":"2024-09-16T15:48:04.205541+0000","last_became_peered":"2024-09-16T15:48:04.205541+0000","last_unstale":"2024-09-16T15:48:06.822963+0000","last_undegraded":"2024-09-16T15:48:06.822963+0000","last_fullsized":"2024-09-16T15:48:06.822963+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:16.209822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"172'6","reported_seq":21,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.183071+0000","last_change":"2024-09-16T15:48:03.186871+0000","last_active":"2024-09-16T15:48:06.183071+0000","last_peered":"2024-09-16T15:48:06.183071+0000","last_clean":"2024-09-16T15:48:06.183071+0000","last_became_active":"2024-09-16T15:48:03.186623+0000","last_became_peered":"2024-09-16T15:48:03.186623+0000","last_unstale":"2024-09-16T15:48:06.183071+0000","last_undegraded":"2024-09-16T15:48:06.183071+0000","last_fullsized":"2024-09-16T15:48:06.183071+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:27.633580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585368+0000","last_change":"2024-09-16T15:48:04.193702+0000","last_active":"2024-09-16T15:48:05.585368+0000","last_peered":"2024-09-16T15:48:05.585368+0000","last_clean":"2024-09-16T15:48:05.585368+0000","last_became_active":"2024-09-16T15:48:04.193255+0000","last_became_peered":"2024-09-16T15:48:04.193255+0000","last_unstale":"2024-09-16T15:48:05.585368+0000","last_undegraded":"2024-09-16T15:48:05.585368+0000","last_fullsized":"2024-09-16T15:48:05.585368+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:05.880148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200023+0000","last_change":"2024-09-16T15:48:03.178628+0000","last_active":"2024-09-16T15:48:05.200023+0000","last_peered":"2024-09-16T15:48:05.200023+0000","last_clean":"2024-09-16T15:48:05.200023+0000","last_became_active":"2024-09-16T15:48:03.178042+0000","last_became_peered":"2024-09-16T15:48:03.178042+0000","last_unstale":"2024-09-16T15:48:05.200023+0000","last_undegraded":"2024-09-16T15:48:05.200023+0000","last_fullsized":"2024-09-16T15:48:05.200023+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:55.426329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198339+0000","last_change":"2024-09-16T15:48:04.198273+0000","last_active":"2024-09-16T15:48:05.198339+0000","last_peered":"2024-09-16T15:48:05.198339+0000","last_clean":"2024-09-16T15:48:05.198339+0000","last_became_active":"2024-09-16T15:48:04.197994+0000","last_became_peered":"2024-09-16T15:48:04.197994+0000","last_unstale":"2024-09-16T15:48:05.198339+0000","last_undegraded":"2024-09-16T15:48:05.198339+0000","last_fullsized":"2024-09-16T15:48:05.198339+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:34.503811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.199377+0000","last_change":"2024-09-16T15:48:03.200321+0000","last_active":"2024-09-16T15:48:05.199377+0000","last_peered":"2024-09-16T15:48:05.199377+0000","last_clean":"2024-09-16T15:48:05.199377+0000","last_became_active":"2024-09-16T15:48:03.199960+0000","last_became_peered":"2024-09-16T15:48:03.199960+0000","last_unstale":"2024-09-16T15:48:05.199377+0000","last_undegraded":"2024-09-16T15:48:05.199377+0000","last_fullsized":"2024-09-16T15:48:05.199377+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:20.634568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273817+0000","last_change":"2024-09-16T15:48:04.195065+0000","last_active":"2024-09-16T15:48:06.273817+0000","last_peered":"2024-09-16T15:48:06.273817+0000","last_clean":"2024-09-16T15:48:06.273817+0000","last_became_active":"2024-09-16T15:48:04.194756+0000","last_became_peered":"2024-09-16T15:48:04.194756+0000","last_unstale":"2024-09-16T15:48:06.273817+0000","last_undegraded":"2024-09-16T15:48:06.273817+0000","last_fullsized":"2024-09-16T15:48:06.273817+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:06.340253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198156+0000","last_change":"2024-09-16T15:48:03.187302+0000","last_active":"2024-09-16T15:48:05.198156+0000","last_peered":"2024-09-16T15:48:05.198156+0000","last_clean":"2024-09-16T15:48:05.198156+0000","last_became_active":"2024-09-16T15:48:03.186956+0000","last_became_peered":"2024-09-16T15:48:03.186956+0000","last_unstale":"2024-09-16T15:48:05.198156+0000","last_undegraded":"2024-09-16T15:48:05.198156+0000","last_fullsized":"2024-09-16T15:48:05.198156+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:14.588599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822703+0000","last_change":"2024-09-16T15:48:04.203544+0000","last_active":"2024-09-16T15:48:06.822703+0000","last_peered":"2024-09-16T15:48:06.822703+0000","last_clean":"2024-09-16T15:48:06.822703+0000","last_became_active":"2024-09-16T15:48:04.202810+0000","last_became_peered":"2024-09-16T15:48:04.202810+0000","last_unstale":"2024-09-16T15:48:06.822703+0000","last_undegraded":"2024-09-16T15:48:06.822703+0000","last_fullsized":"2024-09-16T15:48:06.822703+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:45.434356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"172'8","reported_seq":23,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273882+0000","last_change":"2024-09-16T15:48:03.194143+0000","last_active":"2024-09-16T15:48:06.273882+0000","last_peered":"2024-09-16T15:48:06.273882+0000","last_clean":"2024-09-16T15:48:06.273882+0000","last_became_active":"2024-09-16T15:48:03.193838+0000","last_became_peered":"2024-09-16T15:48:03.193838+0000","last_unstale":"2024-09-16T15:48:06.273882+0000","last_undegraded":"2024-09-16T15:48:06.273882+0000","last_fullsized":"2024-09-16T15:48:06.273882+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:11.223325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823543+0000","last_change":"2024-09-16T15:48:04.198193+0000","last_active":"2024-09-16T15:48:06.823543+0000","last_peered":"2024-09-16T15:48:06.823543+0000","last_clean":"2024-09-16T15:48:06.823543+0000","last_became_active":"2024-09-16T15:48:04.197401+0000","last_became_peered":"2024-09-16T15:48:04.197401+0000","last_unstale":"2024-09-16T15:48:06.823543+0000","last_undegraded":"2024-09-16T15:48:06.823543+0000","last_fullsized":"2024-09-16T15:48:06.823543+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:24.450618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585646+0000","last_change":"2024-09-16T15:48:03.192985+0000","last_active":"2024-09-16T15:48:05.585646+0000","last_peered":"2024-09-16T15:48:05.585646+0000","last_clean":"2024-09-16T15:48:05.585646+0000","last_became_active":"2024-09-16T15:48:03.192025+0000","last_became_peered":"2024-09-16T15:48:03.192025+0000","last_unstale":"2024-09-16T15:48:05.585646+0000","last_undegraded":"2024-09-16T15:48:05.585646+0000","last_fullsized":"2024-09-16T15:48:05.585646+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:30.742172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823342+0000","last_change":"2024-09-16T15:48:04.208147+0000","last_active":"2024-09-16T15:48:06.823342+0000","last_peered":"2024-09-16T15:48:06.823342+0000","last_clean":"2024-09-16T15:48:06.823342+0000","last_became_active":"2024-09-16T15:48:04.207884+0000","last_became_peered":"2024-09-16T15:48:04.207884+0000","last_unstale":"2024-09-16T15:48:06.823342+0000","last_undegraded":"2024-09-16T15:48:06.823342+0000","last_fullsized":"2024-09-16T15:48:06.823342+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:35.050040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823607+0000","last_change":"2024-09-16T15:48:03.197016+0000","last_active":"2024-09-16T15:48:06.823607+0000","last_peered":"2024-09-16T15:48:06.823607+0000","last_clean":"2024-09-16T15:48:06.823607+0000","last_became_active":"2024-09-16T15:48:03.196735+0000","last_became_peered":"2024-09-16T15:48:03.196735+0000","last_unstale":"2024-09-16T15:48:06.823607+0000","last_undegraded":"2024-09-16T15:48:06.823607+0000","last_fullsized":"2024-09-16T15:48:06.823607+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:54.808565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274011+0000","last_change":"2024-09-16T15:48:04.203315+0000","last_active":"2024-09-16T15:48:06.274011+0000","last_peered":"2024-09-16T15:48:06.274011+0000","last_clean":"2024-09-16T15:48:06.274011+0000","last_became_active":"2024-09-16T15:48:04.203102+0000","last_became_peered":"2024-09-16T15:48:04.203102+0000","last_unstale":"2024-09-16T15:48:06.274011+0000","last_undegraded":"2024-09-16T15:48:06.274011+0000","last_fullsized":"2024-09-16T15:48:06.274011+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:13.329855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198295+0000","last_change":"2024-09-16T15:48:03.190121+0000","last_active":"2024-09-16T15:48:05.198295+0000","last_peered":"2024-09-16T15:48:05.198295+0000","last_clean":"2024-09-16T15:48:05.198295+0000","last_became_active":"2024-09-16T15:48:03.189679+0000","last_became_peered":"2024-09-16T15:48:03.189679+0000","last_unstale":"2024-09-16T15:48:05.198295+0000","last_undegraded":"2024-09-16T15:48:05.198295+0000","last_fullsized":"2024-09-16T15:48:05.198295+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:29.770414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198096+0000","last_change":"2024-09-16T15:48:04.203493+0000","last_active":"2024-09-16T15:48:05.198096+0000","last_peered":"2024-09-16T15:48:05.198096+0000","last_clean":"2024-09-16T15:48:05.198096+0000","last_became_active":"2024-09-16T15:48:04.196079+0000","last_became_peered":"2024-09-16T15:48:04.196079+0000","last_unstale":"2024-09-16T15:48:05.198096+0000","last_undegraded":"2024-09-16T15:48:05.198096+0000","last_fullsized":"2024-09-16T15:48:05.198096+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:52.910495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273959+0000","last_change":"2024-09-16T15:48:03.198905+0000","last_active":"2024-09-16T15:48:06.273959+0000","last_peered":"2024-09-16T15:48:06.273959+0000","last_clean":"2024-09-16T15:48:06.273959+0000","last_became_active":"2024-09-16T15:48:03.198503+0000","last_became_peered":"2024-09-16T15:48:03.198503+0000","last_unstale":"2024-09-16T15:48:06.273959+0000","last_undegraded":"2024-09-16T15:48:06.273959+0000","last_fullsized":"2024-09-16T15:48:06.273959+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:54.895294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.197971+0000","last_change":"2024-09-16T15:48:04.202439+0000","last_active":"2024-09-16T15:48:05.197971+0000","last_peered":"2024-09-16T15:48:05.197971+0000","last_clean":"2024-09-16T15:48:05.197971+0000","last_became_active":"2024-09-16T15:48:04.201996+0000","last_became_peered":"2024-09-16T15:48:04.201996+0000","last_unstale":"2024-09-16T15:48:05.197971+0000","last_undegraded":"2024-09-16T15:48:05.197971+0000","last_fullsized":"2024-09-16T15:48:05.197971+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:09:56.020663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198456+0000","last_change":"2024-09-16T15:48:03.196365+0000","last_active":"2024-09-16T15:48:05.198456+0000","last_peered":"2024-09-16T15:48:05.198456+0000","last_clean":"2024-09-16T15:48:05.198456+0000","last_became_active":"2024-09-16T15:48:03.196069+0000","last_became_peered":"2024-09-16T15:48:03.196069+0000","last_unstale":"2024-09-16T15:48:05.198456+0000","last_undegraded":"2024-09-16T15:48:05.198456+0000","last_fullsized":"2024-09-16T15:48:05.198456+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:34.865882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273961+0000","last_change":"2024-09-16T15:48:04.196696+0000","last_active":"2024-09-16T15:48:06.273961+0000","last_peered":"2024-09-16T15:48:06.273961+0000","last_clean":"2024-09-16T15:48:06.273961+0000","last_became_active":"2024-09-16T15:48:04.196424+0000","last_became_peered":"2024-09-16T15:48:04.196424+0000","last_unstale":"2024-09-16T15:48:06.273961+0000","last_undegraded":"2024-09-16T15:48:06.273961+0000","last_fullsized":"2024-09-16T15:48:06.273961+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:30.653715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822744+0000","last_change":"2024-09-16T15:48:03.201031+0000","last_active":"2024-09-16T15:48:06.822744+0000","last_peered":"2024-09-16T15:48:06.822744+0000","last_clean":"2024-09-16T15:48:06.822744+0000","last_became_active":"2024-09-16T15:48:03.200772+0000","last_became_peered":"2024-09-16T15:48:03.200772+0000","last_unstale":"2024-09-16T15:48:06.822744+0000","last_undegraded":"2024-09-16T15:48:06.822744+0000","last_fullsized":"2024-09-16T15:48:06.822744+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:09.154644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822929+0000","last_change":"2024-09-16T15:48:04.190203+0000","last_active":"2024-09-16T15:48:06.822929+0000","last_peered":"2024-09-16T15:48:06.822929+0000","last_clean":"2024-09-16T15:48:06.822929+0000","last_became_active":"2024-09-16T15:48:04.189887+0000","last_became_peered":"2024-09-16T15:48:04.189887+0000","last_unstale":"2024-09-16T15:48:06.822929+0000","last_undegraded":"2024-09-16T15:48:06.822929+0000","last_fullsized":"2024-09-16T15:48:06.822929+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:07.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822665+0000","last_change":"2024-09-16T15:48:03.194768+0000","last_active":"2024-09-16T15:48:06.822665+0000","last_peered":"2024-09-16T15:48:06.822665+0000","last_clean":"2024-09-16T15:48:06.822665+0000","last_became_active":"2024-09-16T15:48:03.194473+0000","last_became_peered":"2024-09-16T15:48:03.194473+0000","last_unstale":"2024-09-16T15:48:06.822665+0000","last_undegraded":"2024-09-16T15:48:06.822665+0000","last_fullsized":"2024-09-16T15:48:06.822665+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:46.612458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200135+0000","last_change":"2024-09-16T15:48:04.187882+0000","last_active":"2024-09-16T15:48:05.200135+0000","last_peered":"2024-09-16T15:48:05.200135+0000","last_clean":"2024-09-16T15:48:05.200135+0000","last_became_active":"2024-09-16T15:48:04.187539+0000","last_became_peered":"2024-09-16T15:48:04.187539+0000","last_unstale":"2024-09-16T15:48:05.200135+0000","last_undegraded":"2024-09-16T15:48:05.200135+0000","last_fullsized":"2024-09-16T15:48:05.200135+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:17.307654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585287+0000","last_change":"2024-09-16T15:48:03.195487+0000","last_active":"2024-09-16T15:48:05.585287+0000","last_peered":"2024-09-16T15:48:05.585287+0000","last_clean":"2024-09-16T15:48:05.585287+0000","last_became_active":"2024-09-16T15:48:03.195220+0000","last_became_peered":"2024-09-16T15:48:03.195220+0000","last_unstale":"2024-09-16T15:48:05.585287+0000","last_undegraded":"2024-09-16T15:48:05.585287+0000","last_fullsized":"2024-09-16T15:48:05.585287+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:45.093277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200680+0000","last_change":"2024-09-16T15:48:04.198829+0000","last_active":"2024-09-16T15:48:05.200680+0000","last_peered":"2024-09-16T15:48:05.200680+0000","last_clean":"2024-09-16T15:48:05.200680+0000","last_became_active":"2024-09-16T15:48:04.196541+0000","last_became_peered":"2024-09-16T15:48:04.196541+0000","last_unstale":"2024-09-16T15:48:05.200680+0000","last_undegraded":"2024-09-16T15:48:05.200680+0000","last_fullsized":"2024-09-16T15:48:05.200680+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:31.354447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274227+0000","last_change":"2024-09-16T15:48:03.194210+0000","last_active":"2024-09-16T15:48:06.274227+0000","last_peered":"2024-09-16T15:48:06.274227+0000","last_clean":"2024-09-16T15:48:06.274227+0000","last_became_active":"2024-09-16T15:48:03.193942+0000","last_became_peered":"2024-09-16T15:48:03.193942+0000","last_unstale":"2024-09-16T15:48:06.274227+0000","last_undegraded":"2024-09-16T15:48:06.274227+0000","last_fullsized":"2024-09-16T15:48:06.274227+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:39.201068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823208+0000","last_change":"2024-09-16T15:48:04.205111+0000","last_active":"2024-09-16T15:48:06.823208+0000","last_peered":"2024-09-16T15:48:06.823208+0000","last_clean":"2024-09-16T15:48:06.823208+0000","last_became_active":"2024-09-16T15:48:04.204835+0000","last_became_peered":"2024-09-16T15:48:04.204835+0000","last_unstale":"2024-09-16T15:48:06.823208+0000","last_undegraded":"2024-09-16T15:48:06.823208+0000","last_fullsized":"2024-09-16T15:48:06.823208+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:53.954030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"173'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.183243+0000","last_change":"2024-09-16T15:48:03.185058+0000","last_active":"2024-09-16T15:48:06.183243+0000","last_peered":"2024-09-16T15:48:06.183243+0000","last_clean":"2024-09-16T15:48:06.183243+0000","last_became_active":"2024-09-16T15:48:03.184767+0000","last_became_peered":"2024-09-16T15:48:03.184767+0000","last_unstale":"2024-09-16T15:48:06.183243+0000","last_undegraded":"2024-09-16T15:48:06.183243+0000","last_fullsized":"2024-09-16T15:48:06.183243+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:34.865882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823198+0000","last_change":"2024-09-16T15:48:04.212157+0000","last_active":"2024-09-16T15:48:06.823198+0000","last_peered":"2024-09-16T15:48:06.823198+0000","last_clean":"2024-09-16T15:48:06.823198+0000","last_became_active":"2024-09-16T15:48:04.211392+0000","last_became_peered":"2024-09-16T15:48:04.211392+0000","last_unstale":"2024-09-16T15:48:06.823198+0000","last_undegraded":"2024-09-16T15:48:06.823198+0000","last_fullsized":"2024-09-16T15:48:06.823198+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:47.353119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585531+0000","last_change":"2024-09-16T15:48:03.192452+0000","last_active":"2024-09-16T15:48:05.585531+0000","last_peered":"2024-09-16T15:48:05.585531+0000","last_clean":"2024-09-16T15:48:05.585531+0000","last_became_active":"2024-09-16T15:48:03.192139+0000","last_became_peered":"2024-09-16T15:48:03.192139+0000","last_unstale":"2024-09-16T15:48:05.585531+0000","last_undegraded":"2024-09-16T15:48:05.585531+0000","last_fullsized":"2024-09-16T15:48:05.585531+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:08.925056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200776+0000","last_change":"2024-09-16T15:48:04.196200+0000","last_active":"2024-09-16T15:48:05.200776+0000","last_peered":"2024-09-16T15:48:05.200776+0000","last_clean":"2024-09-16T15:48:05.200776+0000","last_became_active":"2024-09-16T15:48:04.194266+0000","last_became_peered":"2024-09-16T15:48:04.194266+0000","last_unstale":"2024-09-16T15:48:05.200776+0000","last_undegraded":"2024-09-16T15:48:05.200776+0000","last_fullsized":"2024-09-16T15:48:05.200776+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:54.260034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274082+0000","last_change":"2024-09-16T15:48:03.196561+0000","last_active":"2024-09-16T15:48:06.274082+0000","last_peered":"2024-09-16T15:48:06.274082+0000","last_clean":"2024-09-16T15:48:06.274082+0000","last_became_active":"2024-09-16T15:48:03.195181+0000","last_became_peered":"2024-09-16T15:48:03.195181+0000","last_unstale":"2024-09-16T15:48:06.274082+0000","last_undegraded":"2024-09-16T15:48:06.274082+0000","last_fullsized":"2024-09-16T15:48:06.274082+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:22.597636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822746+0000","last_change":"2024-09-16T15:48:04.198037+0000","last_active":"2024-09-16T15:48:06.822746+0000","last_peered":"2024-09-16T15:48:06.822746+0000","last_clean":"2024-09-16T15:48:06.822746+0000","last_became_active":"2024-09-16T15:48:04.197719+0000","last_became_peered":"2024-09-16T15:48:04.197719+0000","last_unstale":"2024-09-16T15:48:06.822746+0000","last_undegraded":"2024-09-16T15:48:06.822746+0000","last_fullsized":"2024-09-16T15:48:06.822746+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:42:16.552612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182919+0000","last_change":"2024-09-16T15:48:03.192744+0000","last_active":"2024-09-16T15:48:06.182919+0000","last_peered":"2024-09-16T15:48:06.182919+0000","last_clean":"2024-09-16T15:48:06.182919+0000","last_became_active":"2024-09-16T15:48:03.192390+0000","last_became_peered":"2024-09-16T15:48:03.192390+0000","last_unstale":"2024-09-16T15:48:06.182919+0000","last_undegraded":"2024-09-16T15:48:06.182919+0000","last_fullsized":"2024-09-16T15:48:06.182919+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.106382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200075+0000","last_change":"2024-09-16T15:48:04.209980+0000","last_active":"2024-09-16T15:48:05.200075+0000","last_peered":"2024-09-16T15:48:05.200075+0000","last_clean":"2024-09-16T15:48:05.200075+0000","last_became_active":"2024-09-16T15:48:04.209602+0000","last_became_peered":"2024-09-16T15:48:04.209602+0000","last_unstale":"2024-09-16T15:48:05.200075+0000","last_undegraded":"2024-09-16T15:48:05.200075+0000","last_fullsized":"2024-09-16T15:48:05.200075+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:27.132201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585598+0000","last_change":"2024-09-16T15:48:03.190882+0000","last_active":"2024-09-16T15:48:05.585598+0000","last_peered":"2024-09-16T15:48:05.585598+0000","last_clean":"2024-09-16T15:48:05.585598+0000","last_became_active":"2024-09-16T15:48:03.190598+0000","last_became_peered":"2024-09-16T15:48:03.190598+0000","last_unstale":"2024-09-16T15:48:05.585598+0000","last_undegraded":"2024-09-16T15:48:05.585598+0000","last_fullsized":"2024-09-16T15:48:05.585598+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:39.810393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273914+0000","last_change":"2024-09-16T15:48:04.188698+0000","last_active":"2024-09-16T15:48:06.273914+0000","last_peered":"2024-09-16T15:48:06.273914+0000","last_clean":"2024-09-16T15:48:06.273914+0000","last_became_active":"2024-09-16T15:48:04.188416+0000","last_became_peered":"2024-09-16T15:48:04.188416+0000","last_unstale":"2024-09-16T15:48:06.273914+0000","last_undegraded":"2024-09-16T15:48:06.273914+0000","last_fullsized":"2024-09-16T15:48:06.273914+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:28.390298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200199+0000","last_change":"2024-09-16T15:48:03.181891+0000","last_active":"2024-09-16T15:48:05.200199+0000","last_peered":"2024-09-16T15:48:05.200199+0000","last_clean":"2024-09-16T15:48:05.200199+0000","last_became_active":"2024-09-16T15:48:03.179526+0000","last_became_peered":"2024-09-16T15:48:03.179526+0000","last_unstale":"2024-09-16T15:48:05.200199+0000","last_undegraded":"2024-09-16T15:48:05.200199+0000","last_fullsized":"2024-09-16T15:48:05.200199+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:26:45.313588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822967+0000","last_change":"2024-09-16T15:48:04.190251+0000","last_active":"2024-09-16T15:48:06.822967+0000","last_peered":"2024-09-16T15:48:06.822967+0000","last_clean":"2024-09-16T15:48:06.822967+0000","last_became_active":"2024-09-16T15:48:04.189706+0000","last_became_peered":"2024-09-16T15:48:04.189706+0000","last_unstale":"2024-09-16T15:48:06.822967+0000","last_undegraded":"2024-09-16T15:48:06.822967+0000","last_fullsized":"2024-09-16T15:48:06.822967+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:07.019797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.199449+0000","last_change":"2024-09-16T15:48:03.187201+0000","last_active":"2024-09-16T15:48:05.199449+0000","last_peered":"2024-09-16T15:48:05.199449+0000","last_clean":"2024-09-16T15:48:05.199449+0000","last_became_active":"2024-09-16T15:48:03.184533+0000","last_became_peered":"2024-09-16T15:48:03.184533+0000","last_unstale":"2024-09-16T15:48:05.199449+0000","last_undegraded":"2024-09-16T15:48:05.199449+0000","last_fullsized":"2024-09-16T15:48:05.199449+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:17.844388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822589+0000","last_change":"2024-09-16T15:48:04.200907+0000","last_active":"2024-09-16T15:48:06.822589+0000","last_peered":"2024-09-16T15:48:06.822589+0000","last_clean":"2024-09-16T15:48:06.822589+0000","last_became_active":"2024-09-16T15:48:04.199532+0000","last_became_peered":"2024-09-16T15:48:04.199532+0000","last_unstale":"2024-09-16T15:48:06.822589+0000","last_undegraded":"2024-09-16T15:48:06.822589+0000","last_fullsized":"2024-09-16T15:48:06.822589+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:18.240793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200149+0000","last_change":"2024-09-16T15:48:03.188117+0000","last_active":"2024-09-16T15:48:05.200149+0000","last_peered":"2024-09-16T15:48:05.200149+0000","last_clean":"2024-09-16T15:48:05.200149+0000","last_became_active":"2024-09-16T15:48:03.187786+0000","last_became_peered":"2024-09-16T15:48:03.187786+0000","last_unstale":"2024-09-16T15:48:05.200149+0000","last_undegraded":"2024-09-16T15:48:05.200149+0000","last_fullsized":"2024-09-16T15:48:05.200149+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.415177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":416,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182532+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:48:06.182532+0000","last_peered":"2024-09-16T15:48:06.182532+0000","last_clean":"2024-09-16T15:48:06.182532+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:48:06.182532+0000","last_undegraded":"2024-09-16T15:48:06.182532+0000","last_fullsized":"2024-09-16T15:48:06.182532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509561,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33892,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2768896,"data_stored":1546956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2179072,"data_stored":961886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2166784,"data_stored":955558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2195456,"data_stored":963020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2187264,"data_stored":961486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37460,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334380,"statfs":{"total":96636764160,"available":96598405120,"internally_reserved":0,"allocated":2162688,"data_stored":954976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542265,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2183168,"data_stored":961360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509561,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38044,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333796,"statfs":{"total":96636764160,"available":96597807104,"internally_reserved":0,"allocated":2760704,"data_stored":1545434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:08.451 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:08.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.647+0000 7f7b1fe00640 1 -- 172.21.15.60:0/4243532808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 msgr2=0x7f7b200adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.647+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/4243532808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 0x7f7b200adcd0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f7b200cd570 tx=0x7f7b0802a540 comp rx=0 tx=0).stop 2024-09-16T15:48:08.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.647+0000 7f7b1fe00640 1 -- 172.21.15.60:0/4243532808 shutdown_connections 2024-09-16T15:48:08.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.647+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/4243532808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 0x7f7b200adcd0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.647+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/4243532808 >> v2:172.21.15.100:3300/0 conn(0x7f7b2006fb90 0x7f7b2006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.647+0000 7f7b1fe00640 1 -- 172.21.15.60:0/4243532808 >> 172.21.15.60:0/4243532808 conn(0x7f7b200cfa20 msgr2=0x7f7b200cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.647+0000 7f7b1fe00640 1 -- 172.21.15.60:0/4243532808 shutdown_connections 2024-09-16T15:48:08.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.647+0000 7f7b1fe00640 1 -- 172.21.15.60:0/4243532808 wait complete. 2024-09-16T15:48:08.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.649+0000 7f7b1fe00640 1 Processor -- start 2024-09-16T15:48:08.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.649+0000 7f7b1fe00640 1 -- start start 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b1fe00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2006fb90 0x7f7b2015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b1fe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 0x7f7b201f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b1fe00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b200cd5f0 0x7f7b201f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b1fe00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2015ef00 con 0x7f7b200ad880 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b1fe00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2015ed80 con 0x7f7b200cd5f0 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b1fe00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2015f080 con 0x7f7b2006fb90 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b16000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 0x7f7b201f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b16000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 0x7f7b201f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45764/0 (socket says 172.21.15.60:45764) 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b16000640 1 -- 172.21.15.60:0/1313428469 learned_addr learned my addr 172.21.15.60:0/1313428469 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:08.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b1f400640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b200cd5f0 0x7f7b201f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.650+0000 7f7b1ea00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2006fb90 0x7f7b2015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b16000640 1 -- 172.21.15.60:0/1313428469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2006fb90 msgr2=0x7f7b2015ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b16000640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2006fb90 0x7f7b2015ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b16000640 1 -- 172.21.15.60:0/1313428469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b200cd5f0 msgr2=0x7f7b201f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b16000640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b200cd5f0 0x7f7b201f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b16000640 1 -- 172.21.15.60:0/1313428469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b201f52f0 con 0x7f7b200ad880 2024-09-16T15:48:08.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b16000640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 0x7f7b201f5b90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009780 tx=0x7f7b08032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b17400640 1 -- 172.21.15.60:0/1313428469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08040070 con 0x7f7b200ad880 2024-09-16T15:48:08.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b17400640 1 -- 172.21.15.60:0/1313428469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b08005310 con 0x7f7b200ad880 2024-09-16T15:48:08.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b201f55d0 con 0x7f7b200ad880 2024-09-16T15:48:08.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.651+0000 7f7b17400640 1 -- 172.21.15.60:0/1313428469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080047a0 con 0x7f7b200ad880 2024-09-16T15:48:08.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.652+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2015b2a0 con 0x7f7b200ad880 2024-09-16T15:48:08.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.653+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2006ff70 con 0x7f7b200ad880 2024-09-16T15:48:08.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.654+0000 7f7b17400640 1 -- 172.21.15.60:0/1313428469 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f7b08039040 con 0x7f7b200ad880 2024-09-16T15:48:08.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.655+0000 7f7b17400640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7afc0792a0 0x7f7afc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.655+0000 7f7b17400640 1 -- 172.21.15.60:0/1313428469 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f7b080ba420 con 0x7f7b200ad880 2024-09-16T15:48:08.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.655+0000 7f7b1ea00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7afc0792a0 0x7f7afc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.656+0000 7f7b1ea00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7afc0792a0 0x7f7afc07b760 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009b30 tx=0x7f7b10005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.659+0000 7f7b17400640 1 -- 172.21.15.60:0/1313428469 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7b08083e90 con 0x7f7b200ad880 2024-09-16T15:48:08.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.852+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b20154e10 con 0x7f7afc0792a0 2024-09-16T15:48:08.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.856+0000 7f7b17400640 1 -- 172.21.15.60:0/1313428469 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f7b20154e10 con 0x7f7afc0792a0 2024-09-16T15:48:08.862 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:48:08.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7afc0792a0 msgr2=0x7f7afc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7afc0792a0 0x7f7afc07b760 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009b30 tx=0x7f7b10005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 msgr2=0x7f7b201f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 0x7f7b201f5b90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009780 tx=0x7f7b08032c10 comp rx=0 tx=0).stop 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 shutdown_connections 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7afc0792a0 0x7f7afc07b760 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b200cd5f0 0x7f7b201f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b200ad880 0x7f7b201f5b90 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 --2- 172.21.15.60:0/1313428469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2006fb90 0x7f7b2015ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 >> 172.21.15.60:0/1313428469 conn(0x7f7b200cfa20 msgr2=0x7f7b20160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 shutdown_connections 2024-09-16T15:48:08.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:08.861+0000 7f7b1fe00640 1 -- 172.21.15.60:0/1313428469 wait complete. 2024-09-16T15:48:08.880 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-16T15:48:08.225123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284328,"kb_used_data":18168,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690392,"statfs":{"total":773094113280,"available":772802961408,"internally_reserved":0,"allocated":18604032,"data_stored":8850676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001076"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823153+0000","last_change":"2024-09-16T15:48:04.197922+0000","last_active":"2024-09-16T15:48:06.823153+0000","last_peered":"2024-09-16T15:48:06.823153+0000","last_clean":"2024-09-16T15:48:06.823153+0000","last_became_active":"2024-09-16T15:48:04.197638+0000","last_became_peered":"2024-09-16T15:48:04.197638+0000","last_unstale":"2024-09-16T15:48:06.823153+0000","last_undegraded":"2024-09-16T15:48:06.823153+0000","last_fullsized":"2024-09-16T15:48:06.823153+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:04:59.723888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823289+0000","last_change":"2024-09-16T15:48:04.198923+0000","last_active":"2024-09-16T15:48:06.823289+0000","last_peered":"2024-09-16T15:48:06.823289+0000","last_clean":"2024-09-16T15:48:06.823289+0000","last_became_active":"2024-09-16T15:48:04.198553+0000","last_became_peered":"2024-09-16T15:48:04.198553+0000","last_unstale":"2024-09-16T15:48:06.823289+0000","last_undegraded":"2024-09-16T15:48:06.823289+0000","last_fullsized":"2024-09-16T15:48:06.823289+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:48:13.533119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274228+0000","last_change":"2024-09-16T15:48:04.202180+0000","last_active":"2024-09-16T15:48:06.274228+0000","last_peered":"2024-09-16T15:48:06.274228+0000","last_clean":"2024-09-16T15:48:06.274228+0000","last_became_active":"2024-09-16T15:48:04.201914+0000","last_became_peered":"2024-09-16T15:48:04.201914+0000","last_unstale":"2024-09-16T15:48:06.274228+0000","last_undegraded":"2024-09-16T15:48:06.274228+0000","last_fullsized":"2024-09-16T15:48:06.274228+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:13.850947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182823+0000","last_change":"2024-09-16T15:48:04.200120+0000","last_active":"2024-09-16T15:48:06.182823+0000","last_peered":"2024-09-16T15:48:06.182823+0000","last_clean":"2024-09-16T15:48:06.182823+0000","last_became_active":"2024-09-16T15:48:04.199776+0000","last_became_peered":"2024-09-16T15:48:04.199776+0000","last_unstale":"2024-09-16T15:48:06.182823+0000","last_undegraded":"2024-09-16T15:48:06.182823+0000","last_fullsized":"2024-09-16T15:48:06.182823+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:09:56.020663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585528+0000","last_change":"2024-09-16T15:48:04.207577+0000","last_active":"2024-09-16T15:48:05.585528+0000","last_peered":"2024-09-16T15:48:05.585528+0000","last_clean":"2024-09-16T15:48:05.585528+0000","last_became_active":"2024-09-16T15:48:04.207290+0000","last_became_peered":"2024-09-16T15:48:04.207290+0000","last_unstale":"2024-09-16T15:48:05.585528+0000","last_undegraded":"2024-09-16T15:48:05.585528+0000","last_fullsized":"2024-09-16T15:48:05.585528+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:25:36.672954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585371+0000","last_change":"2024-09-16T15:48:04.203301+0000","last_active":"2024-09-16T15:48:05.585371+0000","last_peered":"2024-09-16T15:48:05.585371+0000","last_clean":"2024-09-16T15:48:05.585371+0000","last_became_active":"2024-09-16T15:48:04.202831+0000","last_became_peered":"2024-09-16T15:48:04.202831+0000","last_unstale":"2024-09-16T15:48:05.585371+0000","last_undegraded":"2024-09-16T15:48:05.585371+0000","last_fullsized":"2024-09-16T15:48:05.585371+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:14.620274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822492+0000","last_change":"2024-09-16T15:48:04.210825+0000","last_active":"2024-09-16T15:48:06.822492+0000","last_peered":"2024-09-16T15:48:06.822492+0000","last_clean":"2024-09-16T15:48:06.822492+0000","last_became_active":"2024-09-16T15:48:04.199466+0000","last_became_peered":"2024-09-16T15:48:04.199466+0000","last_unstale":"2024-09-16T15:48:06.822492+0000","last_undegraded":"2024-09-16T15:48:06.822492+0000","last_fullsized":"2024-09-16T15:48:06.822492+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:20.525297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585214+0000","last_change":"2024-09-16T15:48:04.190660+0000","last_active":"2024-09-16T15:48:05.585214+0000","last_peered":"2024-09-16T15:48:05.585214+0000","last_clean":"2024-09-16T15:48:05.585214+0000","last_became_active":"2024-09-16T15:48:04.190413+0000","last_became_peered":"2024-09-16T15:48:04.190413+0000","last_unstale":"2024-09-16T15:48:05.585214+0000","last_undegraded":"2024-09-16T15:48:05.585214+0000","last_fullsized":"2024-09-16T15:48:05.585214+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:58.822830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823533+0000","last_change":"2024-09-16T15:48:04.192862+0000","last_active":"2024-09-16T15:48:06.823533+0000","last_peered":"2024-09-16T15:48:06.823533+0000","last_clean":"2024-09-16T15:48:06.823533+0000","last_became_active":"2024-09-16T15:48:04.191793+0000","last_became_peered":"2024-09-16T15:48:04.191793+0000","last_unstale":"2024-09-16T15:48:06.823533+0000","last_undegraded":"2024-09-16T15:48:06.823533+0000","last_fullsized":"2024-09-16T15:48:06.823533+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:20.270919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198597+0000","last_change":"2024-09-16T15:48:04.194667+0000","last_active":"2024-09-16T15:48:05.198597+0000","last_peered":"2024-09-16T15:48:05.198597+0000","last_clean":"2024-09-16T15:48:05.198597+0000","last_became_active":"2024-09-16T15:48:04.194308+0000","last_became_peered":"2024-09-16T15:48:04.194308+0000","last_unstale":"2024-09-16T15:48:05.198597+0000","last_undegraded":"2024-09-16T15:48:05.198597+0000","last_fullsized":"2024-09-16T15:48:05.198597+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:18.240793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274365+0000","last_change":"2024-09-16T15:48:04.204567+0000","last_active":"2024-09-16T15:48:06.274365+0000","last_peered":"2024-09-16T15:48:06.274365+0000","last_clean":"2024-09-16T15:48:06.274365+0000","last_became_active":"2024-09-16T15:48:04.204305+0000","last_became_peered":"2024-09-16T15:48:04.204305+0000","last_unstale":"2024-09-16T15:48:06.274365+0000","last_undegraded":"2024-09-16T15:48:06.274365+0000","last_fullsized":"2024-09-16T15:48:06.274365+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:41:34.637763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200208+0000","last_change":"2024-09-16T15:48:04.197592+0000","last_active":"2024-09-16T15:48:05.200208+0000","last_peered":"2024-09-16T15:48:05.200208+0000","last_clean":"2024-09-16T15:48:05.200208+0000","last_became_active":"2024-09-16T15:48:04.197325+0000","last_became_peered":"2024-09-16T15:48:04.197325+0000","last_unstale":"2024-09-16T15:48:05.200208+0000","last_undegraded":"2024-09-16T15:48:05.200208+0000","last_fullsized":"2024-09-16T15:48:05.200208+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.852073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200290+0000","last_change":"2024-09-16T15:48:04.192138+0000","last_active":"2024-09-16T15:48:05.200290+0000","last_peered":"2024-09-16T15:48:05.200290+0000","last_clean":"2024-09-16T15:48:05.200290+0000","last_became_active":"2024-09-16T15:48:04.190454+0000","last_became_peered":"2024-09-16T15:48:04.190454+0000","last_unstale":"2024-09-16T15:48:05.200290+0000","last_undegraded":"2024-09-16T15:48:05.200290+0000","last_fullsized":"2024-09-16T15:48:05.200290+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:01.298044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822547+0000","last_change":"2024-09-16T15:48:04.205762+0000","last_active":"2024-09-16T15:48:06.822547+0000","last_peered":"2024-09-16T15:48:06.822547+0000","last_clean":"2024-09-16T15:48:06.822547+0000","last_became_active":"2024-09-16T15:48:04.205480+0000","last_became_peered":"2024-09-16T15:48:04.205480+0000","last_unstale":"2024-09-16T15:48:06.822547+0000","last_undegraded":"2024-09-16T15:48:06.822547+0000","last_fullsized":"2024-09-16T15:48:06.822547+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:34.503811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822489+0000","last_change":"2024-09-16T15:48:04.208965+0000","last_active":"2024-09-16T15:48:06.822489+0000","last_peered":"2024-09-16T15:48:06.822489+0000","last_clean":"2024-09-16T15:48:06.822489+0000","last_became_active":"2024-09-16T15:48:04.203160+0000","last_became_peered":"2024-09-16T15:48:04.203160+0000","last_unstale":"2024-09-16T15:48:06.822489+0000","last_undegraded":"2024-09-16T15:48:06.822489+0000","last_fullsized":"2024-09-16T15:48:06.822489+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:31.962754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.197860+0000","last_change":"2024-09-16T15:48:04.192434+0000","last_active":"2024-09-16T15:48:05.197860+0000","last_peered":"2024-09-16T15:48:05.197860+0000","last_clean":"2024-09-16T15:48:05.197860+0000","last_became_active":"2024-09-16T15:48:04.192147+0000","last_became_peered":"2024-09-16T15:48:04.192147+0000","last_unstale":"2024-09-16T15:48:05.197860+0000","last_undegraded":"2024-09-16T15:48:05.197860+0000","last_fullsized":"2024-09-16T15:48:05.197860+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:37.016457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823092+0000","last_change":"2024-09-16T15:48:04.194831+0000","last_active":"2024-09-16T15:48:06.823092+0000","last_peered":"2024-09-16T15:48:06.823092+0000","last_clean":"2024-09-16T15:48:06.823092+0000","last_became_active":"2024-09-16T15:48:04.194554+0000","last_became_peered":"2024-09-16T15:48:04.194554+0000","last_unstale":"2024-09-16T15:48:06.823092+0000","last_undegraded":"2024-09-16T15:48:06.823092+0000","last_fullsized":"2024-09-16T15:48:06.823092+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:24.793488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198166+0000","last_change":"2024-09-16T15:48:04.202550+0000","last_active":"2024-09-16T15:48:05.198166+0000","last_peered":"2024-09-16T15:48:05.198166+0000","last_clean":"2024-09-16T15:48:05.198166+0000","last_became_active":"2024-09-16T15:48:04.202267+0000","last_became_peered":"2024-09-16T15:48:04.202267+0000","last_unstale":"2024-09-16T15:48:05.198166+0000","last_undegraded":"2024-09-16T15:48:05.198166+0000","last_fullsized":"2024-09-16T15:48:05.198166+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:31.962754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198204+0000","last_change":"2024-09-16T15:48:04.201689+0000","last_active":"2024-09-16T15:48:05.198204+0000","last_peered":"2024-09-16T15:48:05.198204+0000","last_clean":"2024-09-16T15:48:05.198204+0000","last_became_active":"2024-09-16T15:48:04.201108+0000","last_became_peered":"2024-09-16T15:48:04.201108+0000","last_unstale":"2024-09-16T15:48:05.198204+0000","last_undegraded":"2024-09-16T15:48:05.198204+0000","last_fullsized":"2024-09-16T15:48:05.198204+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:19:28.133935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585620+0000","last_change":"2024-09-16T15:48:04.205783+0000","last_active":"2024-09-16T15:48:05.585620+0000","last_peered":"2024-09-16T15:48:05.585620+0000","last_clean":"2024-09-16T15:48:05.585620+0000","last_became_active":"2024-09-16T15:48:04.205044+0000","last_became_peered":"2024-09-16T15:48:04.205044+0000","last_unstale":"2024-09-16T15:48:05.585620+0000","last_undegraded":"2024-09-16T15:48:05.585620+0000","last_fullsized":"2024-09-16T15:48:05.585620+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:57:25.973298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182979+0000","last_change":"2024-09-16T15:48:04.212115+0000","last_active":"2024-09-16T15:48:06.182979+0000","last_peered":"2024-09-16T15:48:06.182979+0000","last_clean":"2024-09-16T15:48:06.182979+0000","last_became_active":"2024-09-16T15:48:04.211677+0000","last_became_peered":"2024-09-16T15:48:04.211677+0000","last_unstale":"2024-09-16T15:48:06.182979+0000","last_undegraded":"2024-09-16T15:48:06.182979+0000","last_fullsized":"2024-09-16T15:48:06.182979+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:24.963722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200620+0000","last_change":"2024-09-16T15:48:04.205057+0000","last_active":"2024-09-16T15:48:05.200620+0000","last_peered":"2024-09-16T15:48:05.200620+0000","last_clean":"2024-09-16T15:48:05.200620+0000","last_became_active":"2024-09-16T15:48:04.204768+0000","last_became_peered":"2024-09-16T15:48:04.204768+0000","last_unstale":"2024-09-16T15:48:05.200620+0000","last_undegraded":"2024-09-16T15:48:05.200620+0000","last_fullsized":"2024-09-16T15:48:05.200620+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:47:24.968464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200576+0000","last_change":"2024-09-16T15:48:04.195071+0000","last_active":"2024-09-16T15:48:05.200576+0000","last_peered":"2024-09-16T15:48:05.200576+0000","last_clean":"2024-09-16T15:48:05.200576+0000","last_became_active":"2024-09-16T15:48:04.190837+0000","last_became_peered":"2024-09-16T15:48:04.190837+0000","last_unstale":"2024-09-16T15:48:05.200576+0000","last_undegraded":"2024-09-16T15:48:05.200576+0000","last_fullsized":"2024-09-16T15:48:05.200576+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:23:31.497014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182759+0000","last_change":"2024-09-16T15:48:04.213153+0000","last_active":"2024-09-16T15:48:06.182759+0000","last_peered":"2024-09-16T15:48:06.182759+0000","last_clean":"2024-09-16T15:48:06.182759+0000","last_became_active":"2024-09-16T15:48:04.212863+0000","last_became_peered":"2024-09-16T15:48:04.212863+0000","last_unstale":"2024-09-16T15:48:06.182759+0000","last_undegraded":"2024-09-16T15:48:06.182759+0000","last_fullsized":"2024-09-16T15:48:06.182759+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:37.016457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823310+0000","last_change":"2024-09-16T15:48:04.208845+0000","last_active":"2024-09-16T15:48:06.823310+0000","last_peered":"2024-09-16T15:48:06.823310+0000","last_clean":"2024-09-16T15:48:06.823310+0000","last_became_active":"2024-09-16T15:48:04.208529+0000","last_became_peered":"2024-09-16T15:48:04.208529+0000","last_unstale":"2024-09-16T15:48:06.823310+0000","last_undegraded":"2024-09-16T15:48:06.823310+0000","last_fullsized":"2024-09-16T15:48:06.823310+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:10.867458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823132+0000","last_change":"2024-09-16T15:48:04.198585+0000","last_active":"2024-09-16T15:48:06.823132+0000","last_peered":"2024-09-16T15:48:06.823132+0000","last_clean":"2024-09-16T15:48:06.823132+0000","last_became_active":"2024-09-16T15:48:04.198323+0000","last_became_peered":"2024-09-16T15:48:04.198323+0000","last_unstale":"2024-09-16T15:48:06.823132+0000","last_undegraded":"2024-09-16T15:48:06.823132+0000","last_fullsized":"2024-09-16T15:48:06.823132+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:14:53.824138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200237+0000","last_change":"2024-09-16T15:48:04.196025+0000","last_active":"2024-09-16T15:48:05.200237+0000","last_peered":"2024-09-16T15:48:05.200237+0000","last_clean":"2024-09-16T15:48:05.200237+0000","last_became_active":"2024-09-16T15:48:04.195738+0000","last_became_peered":"2024-09-16T15:48:04.195738+0000","last_unstale":"2024-09-16T15:48:05.200237+0000","last_undegraded":"2024-09-16T15:48:05.200237+0000","last_fullsized":"2024-09-16T15:48:05.200237+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:17.050178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273863+0000","last_change":"2024-09-16T15:48:04.196230+0000","last_active":"2024-09-16T15:48:06.273863+0000","last_peered":"2024-09-16T15:48:06.273863+0000","last_clean":"2024-09-16T15:48:06.273863+0000","last_became_active":"2024-09-16T15:48:04.195976+0000","last_became_peered":"2024-09-16T15:48:04.195976+0000","last_unstale":"2024-09-16T15:48:06.273863+0000","last_undegraded":"2024-09-16T15:48:06.273863+0000","last_fullsized":"2024-09-16T15:48:06.273863+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:16.633554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200039+0000","last_change":"2024-09-16T15:48:04.196151+0000","last_active":"2024-09-16T15:48:05.200039+0000","last_peered":"2024-09-16T15:48:05.200039+0000","last_clean":"2024-09-16T15:48:05.200039+0000","last_became_active":"2024-09-16T15:48:04.195689+0000","last_became_peered":"2024-09-16T15:48:04.195689+0000","last_unstale":"2024-09-16T15:48:05.200039+0000","last_undegraded":"2024-09-16T15:48:05.200039+0000","last_fullsized":"2024-09-16T15:48:05.200039+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:40.002311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823110+0000","last_change":"2024-09-16T15:48:04.198675+0000","last_active":"2024-09-16T15:48:06.823110+0000","last_peered":"2024-09-16T15:48:06.823110+0000","last_clean":"2024-09-16T15:48:06.823110+0000","last_became_active":"2024-09-16T15:48:04.198074+0000","last_became_peered":"2024-09-16T15:48:04.198074+0000","last_unstale":"2024-09-16T15:48:06.823110+0000","last_undegraded":"2024-09-16T15:48:06.823110+0000","last_fullsized":"2024-09-16T15:48:06.823110+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:14.297001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198086+0000","last_change":"2024-09-16T15:48:04.213248+0000","last_active":"2024-09-16T15:48:05.198086+0000","last_peered":"2024-09-16T15:48:05.198086+0000","last_clean":"2024-09-16T15:48:05.198086+0000","last_became_active":"2024-09-16T15:48:04.203595+0000","last_became_peered":"2024-09-16T15:48:04.203595+0000","last_unstale":"2024-09-16T15:48:05.198086+0000","last_undegraded":"2024-09-16T15:48:05.198086+0000","last_fullsized":"2024-09-16T15:48:05.198086+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:00.748962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822443+0000","last_change":"2024-09-16T15:48:04.196041+0000","last_active":"2024-09-16T15:48:06.822443+0000","last_peered":"2024-09-16T15:48:06.822443+0000","last_clean":"2024-09-16T15:48:06.822443+0000","last_became_active":"2024-09-16T15:48:04.195742+0000","last_became_peered":"2024-09-16T15:48:04.195742+0000","last_unstale":"2024-09-16T15:48:06.822443+0000","last_undegraded":"2024-09-16T15:48:06.822443+0000","last_fullsized":"2024-09-16T15:48:06.822443+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:52.910495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823369+0000","last_change":"2024-09-16T15:48:04.197896+0000","last_active":"2024-09-16T15:48:06.823369+0000","last_peered":"2024-09-16T15:48:06.823369+0000","last_clean":"2024-09-16T15:48:06.823369+0000","last_became_active":"2024-09-16T15:48:04.197529+0000","last_became_peered":"2024-09-16T15:48:04.197529+0000","last_unstale":"2024-09-16T15:48:06.823369+0000","last_undegraded":"2024-09-16T15:48:06.823369+0000","last_fullsized":"2024-09-16T15:48:06.823369+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:40.034009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"172'4","reported_seq":19,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200465+0000","last_change":"2024-09-16T15:48:03.188074+0000","last_active":"2024-09-16T15:48:05.200465+0000","last_peered":"2024-09-16T15:48:05.200465+0000","last_clean":"2024-09-16T15:48:05.200465+0000","last_became_active":"2024-09-16T15:48:03.187768+0000","last_became_peered":"2024-09-16T15:48:03.187768+0000","last_unstale":"2024-09-16T15:48:05.200465+0000","last_undegraded":"2024-09-16T15:48:05.200465+0000","last_fullsized":"2024-09-16T15:48:05.200465+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:57:41.638780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200919+0000","last_change":"2024-09-16T15:48:04.196297+0000","last_active":"2024-09-16T15:48:05.200919+0000","last_peered":"2024-09-16T15:48:05.200919+0000","last_clean":"2024-09-16T15:48:05.200919+0000","last_became_active":"2024-09-16T15:48:04.196004+0000","last_became_peered":"2024-09-16T15:48:04.196004+0000","last_unstale":"2024-09-16T15:48:05.200919+0000","last_undegraded":"2024-09-16T15:48:05.200919+0000","last_fullsized":"2024-09-16T15:48:05.200919+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:55:42.602478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823537+0000","last_change":"2024-09-16T15:48:03.191407+0000","last_active":"2024-09-16T15:48:06.823537+0000","last_peered":"2024-09-16T15:48:06.823537+0000","last_clean":"2024-09-16T15:48:06.823537+0000","last_became_active":"2024-09-16T15:48:03.190533+0000","last_became_peered":"2024-09-16T15:48:03.190533+0000","last_unstale":"2024-09-16T15:48:06.823537+0000","last_undegraded":"2024-09-16T15:48:06.823537+0000","last_fullsized":"2024-09-16T15:48:06.823537+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:54.227271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822828+0000","last_change":"2024-09-16T15:48:04.203696+0000","last_active":"2024-09-16T15:48:06.822828+0000","last_peered":"2024-09-16T15:48:06.822828+0000","last_clean":"2024-09-16T15:48:06.822828+0000","last_became_active":"2024-09-16T15:48:04.200494+0000","last_became_peered":"2024-09-16T15:48:04.200494+0000","last_unstale":"2024-09-16T15:48:06.822828+0000","last_undegraded":"2024-09-16T15:48:06.822828+0000","last_fullsized":"2024-09-16T15:48:06.822828+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:58.976149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274282+0000","last_change":"2024-09-16T15:48:03.195571+0000","last_active":"2024-09-16T15:48:06.274282+0000","last_peered":"2024-09-16T15:48:06.274282+0000","last_clean":"2024-09-16T15:48:06.274282+0000","last_became_active":"2024-09-16T15:48:03.195294+0000","last_became_peered":"2024-09-16T15:48:03.195294+0000","last_unstale":"2024-09-16T15:48:06.274282+0000","last_undegraded":"2024-09-16T15:48:06.274282+0000","last_fullsized":"2024-09-16T15:48:06.274282+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:51:02.063117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.197978+0000","last_change":"2024-09-16T15:48:04.201365+0000","last_active":"2024-09-16T15:48:05.197978+0000","last_peered":"2024-09-16T15:48:05.197978+0000","last_clean":"2024-09-16T15:48:05.197978+0000","last_became_active":"2024-09-16T15:48:04.200751+0000","last_became_peered":"2024-09-16T15:48:04.200751+0000","last_unstale":"2024-09-16T15:48:05.197978+0000","last_undegraded":"2024-09-16T15:48:05.197978+0000","last_fullsized":"2024-09-16T15:48:05.197978+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:19:52.253452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182941+0000","last_change":"2024-09-16T15:48:03.200040+0000","last_active":"2024-09-16T15:48:06.182941+0000","last_peered":"2024-09-16T15:48:06.182941+0000","last_clean":"2024-09-16T15:48:06.182941+0000","last_became_active":"2024-09-16T15:48:03.199650+0000","last_became_peered":"2024-09-16T15:48:03.199650+0000","last_unstale":"2024-09-16T15:48:06.182941+0000","last_undegraded":"2024-09-16T15:48:06.182941+0000","last_fullsized":"2024-09-16T15:48:06.182941+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:29.770414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182828+0000","last_change":"2024-09-16T15:48:04.201062+0000","last_active":"2024-09-16T15:48:06.182828+0000","last_peered":"2024-09-16T15:48:06.182828+0000","last_clean":"2024-09-16T15:48:06.182828+0000","last_became_active":"2024-09-16T15:48:04.200724+0000","last_became_peered":"2024-09-16T15:48:04.200724+0000","last_unstale":"2024-09-16T15:48:06.182828+0000","last_undegraded":"2024-09-16T15:48:06.182828+0000","last_fullsized":"2024-09-16T15:48:06.182828+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:07:08.806828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585521+0000","last_change":"2024-09-16T15:48:03.181659+0000","last_active":"2024-09-16T15:48:05.585521+0000","last_peered":"2024-09-16T15:48:05.585521+0000","last_clean":"2024-09-16T15:48:05.585521+0000","last_became_active":"2024-09-16T15:48:03.181337+0000","last_became_peered":"2024-09-16T15:48:03.181337+0000","last_unstale":"2024-09-16T15:48:05.585521+0000","last_undegraded":"2024-09-16T15:48:05.585521+0000","last_fullsized":"2024-09-16T15:48:05.585521+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:36.908344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585296+0000","last_change":"2024-09-16T15:48:04.192582+0000","last_active":"2024-09-16T15:48:05.585296+0000","last_peered":"2024-09-16T15:48:05.585296+0000","last_clean":"2024-09-16T15:48:05.585296+0000","last_became_active":"2024-09-16T15:48:04.192299+0000","last_became_peered":"2024-09-16T15:48:04.192299+0000","last_unstale":"2024-09-16T15:48:05.585296+0000","last_undegraded":"2024-09-16T15:48:05.585296+0000","last_fullsized":"2024-09-16T15:48:05.585296+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:33:17.441611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585118+0000","last_change":"2024-09-16T15:48:03.185920+0000","last_active":"2024-09-16T15:48:05.585118+0000","last_peered":"2024-09-16T15:48:05.585118+0000","last_clean":"2024-09-16T15:48:05.585118+0000","last_became_active":"2024-09-16T15:48:03.185643+0000","last_became_peered":"2024-09-16T15:48:03.185643+0000","last_unstale":"2024-09-16T15:48:05.585118+0000","last_undegraded":"2024-09-16T15:48:05.585118+0000","last_fullsized":"2024-09-16T15:48:05.585118+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:59:08.674483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182684+0000","last_change":"2024-09-16T15:48:04.202736+0000","last_active":"2024-09-16T15:48:06.182684+0000","last_peered":"2024-09-16T15:48:06.182684+0000","last_clean":"2024-09-16T15:48:06.182684+0000","last_became_active":"2024-09-16T15:48:04.202330+0000","last_became_peered":"2024-09-16T15:48:04.202330+0000","last_unstale":"2024-09-16T15:48:06.182684+0000","last_undegraded":"2024-09-16T15:48:06.182684+0000","last_fullsized":"2024-09-16T15:48:06.182684+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:01:22.672492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.199009+0000","last_change":"2024-09-16T15:48:03.182323+0000","last_active":"2024-09-16T15:48:05.199009+0000","last_peered":"2024-09-16T15:48:05.199009+0000","last_clean":"2024-09-16T15:48:05.199009+0000","last_became_active":"2024-09-16T15:48:03.182062+0000","last_became_peered":"2024-09-16T15:48:03.182062+0000","last_unstale":"2024-09-16T15:48:05.199009+0000","last_undegraded":"2024-09-16T15:48:05.199009+0000","last_fullsized":"2024-09-16T15:48:05.199009+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:26.164248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.183232+0000","last_change":"2024-09-16T15:48:04.200373+0000","last_active":"2024-09-16T15:48:06.183232+0000","last_peered":"2024-09-16T15:48:06.183232+0000","last_clean":"2024-09-16T15:48:06.183232+0000","last_became_active":"2024-09-16T15:48:04.200041+0000","last_became_peered":"2024-09-16T15:48:04.200041+0000","last_unstale":"2024-09-16T15:48:06.183232+0000","last_undegraded":"2024-09-16T15:48:06.183232+0000","last_fullsized":"2024-09-16T15:48:06.183232+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:31.962754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585602+0000","last_change":"2024-09-16T15:48:03.191241+0000","last_active":"2024-09-16T15:48:05.585602+0000","last_peered":"2024-09-16T15:48:05.585602+0000","last_clean":"2024-09-16T15:48:05.585602+0000","last_became_active":"2024-09-16T15:48:03.190791+0000","last_became_peered":"2024-09-16T15:48:03.190791+0000","last_unstale":"2024-09-16T15:48:05.585602+0000","last_undegraded":"2024-09-16T15:48:05.585602+0000","last_fullsized":"2024-09-16T15:48:05.585602+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:59.334295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274301+0000","last_change":"2024-09-16T15:48:04.207673+0000","last_active":"2024-09-16T15:48:06.274301+0000","last_peered":"2024-09-16T15:48:06.274301+0000","last_clean":"2024-09-16T15:48:06.274301+0000","last_became_active":"2024-09-16T15:48:04.207320+0000","last_became_peered":"2024-09-16T15:48:04.207320+0000","last_unstale":"2024-09-16T15:48:06.274301+0000","last_undegraded":"2024-09-16T15:48:06.274301+0000","last_fullsized":"2024-09-16T15:48:06.274301+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:44.180969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.199725+0000","last_change":"2024-09-16T15:48:03.190471+0000","last_active":"2024-09-16T15:48:05.199725+0000","last_peered":"2024-09-16T15:48:05.199725+0000","last_clean":"2024-09-16T15:48:05.199725+0000","last_became_active":"2024-09-16T15:48:03.190083+0000","last_became_peered":"2024-09-16T15:48:03.190083+0000","last_unstale":"2024-09-16T15:48:05.199725+0000","last_undegraded":"2024-09-16T15:48:05.199725+0000","last_fullsized":"2024-09-16T15:48:05.199725+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:08.009241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182974+0000","last_change":"2024-09-16T15:48:04.202413+0000","last_active":"2024-09-16T15:48:06.182974+0000","last_peered":"2024-09-16T15:48:06.182974+0000","last_clean":"2024-09-16T15:48:06.182974+0000","last_became_active":"2024-09-16T15:48:04.202126+0000","last_became_peered":"2024-09-16T15:48:04.202126+0000","last_unstale":"2024-09-16T15:48:06.182974+0000","last_undegraded":"2024-09-16T15:48:06.182974+0000","last_fullsized":"2024-09-16T15:48:06.182974+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:00.748962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"172'3","reported_seq":18,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198240+0000","last_change":"2024-09-16T15:48:03.200450+0000","last_active":"2024-09-16T15:48:05.198240+0000","last_peered":"2024-09-16T15:48:05.198240+0000","last_clean":"2024-09-16T15:48:05.198240+0000","last_became_active":"2024-09-16T15:48:03.200189+0000","last_became_peered":"2024-09-16T15:48:03.200189+0000","last_unstale":"2024-09-16T15:48:05.198240+0000","last_undegraded":"2024-09-16T15:48:05.198240+0000","last_fullsized":"2024-09-16T15:48:05.198240+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.106382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822963+0000","last_change":"2024-09-16T15:48:04.206128+0000","last_active":"2024-09-16T15:48:06.822963+0000","last_peered":"2024-09-16T15:48:06.822963+0000","last_clean":"2024-09-16T15:48:06.822963+0000","last_became_active":"2024-09-16T15:48:04.205541+0000","last_became_peered":"2024-09-16T15:48:04.205541+0000","last_unstale":"2024-09-16T15:48:06.822963+0000","last_undegraded":"2024-09-16T15:48:06.822963+0000","last_fullsized":"2024-09-16T15:48:06.822963+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:16.209822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"172'6","reported_seq":21,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.183071+0000","last_change":"2024-09-16T15:48:03.186871+0000","last_active":"2024-09-16T15:48:06.183071+0000","last_peered":"2024-09-16T15:48:06.183071+0000","last_clean":"2024-09-16T15:48:06.183071+0000","last_became_active":"2024-09-16T15:48:03.186623+0000","last_became_peered":"2024-09-16T15:48:03.186623+0000","last_unstale":"2024-09-16T15:48:06.183071+0000","last_undegraded":"2024-09-16T15:48:06.183071+0000","last_fullsized":"2024-09-16T15:48:06.183071+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:27.633580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585368+0000","last_change":"2024-09-16T15:48:04.193702+0000","last_active":"2024-09-16T15:48:05.585368+0000","last_peered":"2024-09-16T15:48:05.585368+0000","last_clean":"2024-09-16T15:48:05.585368+0000","last_became_active":"2024-09-16T15:48:04.193255+0000","last_became_peered":"2024-09-16T15:48:04.193255+0000","last_unstale":"2024-09-16T15:48:05.585368+0000","last_undegraded":"2024-09-16T15:48:05.585368+0000","last_fullsized":"2024-09-16T15:48:05.585368+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:05.880148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200023+0000","last_change":"2024-09-16T15:48:03.178628+0000","last_active":"2024-09-16T15:48:05.200023+0000","last_peered":"2024-09-16T15:48:05.200023+0000","last_clean":"2024-09-16T15:48:05.200023+0000","last_became_active":"2024-09-16T15:48:03.178042+0000","last_became_peered":"2024-09-16T15:48:03.178042+0000","last_unstale":"2024-09-16T15:48:05.200023+0000","last_undegraded":"2024-09-16T15:48:05.200023+0000","last_fullsized":"2024-09-16T15:48:05.200023+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:55.426329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198339+0000","last_change":"2024-09-16T15:48:04.198273+0000","last_active":"2024-09-16T15:48:05.198339+0000","last_peered":"2024-09-16T15:48:05.198339+0000","last_clean":"2024-09-16T15:48:05.198339+0000","last_became_active":"2024-09-16T15:48:04.197994+0000","last_became_peered":"2024-09-16T15:48:04.197994+0000","last_unstale":"2024-09-16T15:48:05.198339+0000","last_undegraded":"2024-09-16T15:48:05.198339+0000","last_fullsized":"2024-09-16T15:48:05.198339+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:34.503811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.199377+0000","last_change":"2024-09-16T15:48:03.200321+0000","last_active":"2024-09-16T15:48:05.199377+0000","last_peered":"2024-09-16T15:48:05.199377+0000","last_clean":"2024-09-16T15:48:05.199377+0000","last_became_active":"2024-09-16T15:48:03.199960+0000","last_became_peered":"2024-09-16T15:48:03.199960+0000","last_unstale":"2024-09-16T15:48:05.199377+0000","last_undegraded":"2024-09-16T15:48:05.199377+0000","last_fullsized":"2024-09-16T15:48:05.199377+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:20.634568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273817+0000","last_change":"2024-09-16T15:48:04.195065+0000","last_active":"2024-09-16T15:48:06.273817+0000","last_peered":"2024-09-16T15:48:06.273817+0000","last_clean":"2024-09-16T15:48:06.273817+0000","last_became_active":"2024-09-16T15:48:04.194756+0000","last_became_peered":"2024-09-16T15:48:04.194756+0000","last_unstale":"2024-09-16T15:48:06.273817+0000","last_undegraded":"2024-09-16T15:48:06.273817+0000","last_fullsized":"2024-09-16T15:48:06.273817+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:06.340253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198156+0000","last_change":"2024-09-16T15:48:03.187302+0000","last_active":"2024-09-16T15:48:05.198156+0000","last_peered":"2024-09-16T15:48:05.198156+0000","last_clean":"2024-09-16T15:48:05.198156+0000","last_became_active":"2024-09-16T15:48:03.186956+0000","last_became_peered":"2024-09-16T15:48:03.186956+0000","last_unstale":"2024-09-16T15:48:05.198156+0000","last_undegraded":"2024-09-16T15:48:05.198156+0000","last_fullsized":"2024-09-16T15:48:05.198156+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:14.588599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822703+0000","last_change":"2024-09-16T15:48:04.203544+0000","last_active":"2024-09-16T15:48:06.822703+0000","last_peered":"2024-09-16T15:48:06.822703+0000","last_clean":"2024-09-16T15:48:06.822703+0000","last_became_active":"2024-09-16T15:48:04.202810+0000","last_became_peered":"2024-09-16T15:48:04.202810+0000","last_unstale":"2024-09-16T15:48:06.822703+0000","last_undegraded":"2024-09-16T15:48:06.822703+0000","last_fullsized":"2024-09-16T15:48:06.822703+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:45.434356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"172'8","reported_seq":23,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273882+0000","last_change":"2024-09-16T15:48:03.194143+0000","last_active":"2024-09-16T15:48:06.273882+0000","last_peered":"2024-09-16T15:48:06.273882+0000","last_clean":"2024-09-16T15:48:06.273882+0000","last_became_active":"2024-09-16T15:48:03.193838+0000","last_became_peered":"2024-09-16T15:48:03.193838+0000","last_unstale":"2024-09-16T15:48:06.273882+0000","last_undegraded":"2024-09-16T15:48:06.273882+0000","last_fullsized":"2024-09-16T15:48:06.273882+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:11.223325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823543+0000","last_change":"2024-09-16T15:48:04.198193+0000","last_active":"2024-09-16T15:48:06.823543+0000","last_peered":"2024-09-16T15:48:06.823543+0000","last_clean":"2024-09-16T15:48:06.823543+0000","last_became_active":"2024-09-16T15:48:04.197401+0000","last_became_peered":"2024-09-16T15:48:04.197401+0000","last_unstale":"2024-09-16T15:48:06.823543+0000","last_undegraded":"2024-09-16T15:48:06.823543+0000","last_fullsized":"2024-09-16T15:48:06.823543+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:24.450618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585646+0000","last_change":"2024-09-16T15:48:03.192985+0000","last_active":"2024-09-16T15:48:05.585646+0000","last_peered":"2024-09-16T15:48:05.585646+0000","last_clean":"2024-09-16T15:48:05.585646+0000","last_became_active":"2024-09-16T15:48:03.192025+0000","last_became_peered":"2024-09-16T15:48:03.192025+0000","last_unstale":"2024-09-16T15:48:05.585646+0000","last_undegraded":"2024-09-16T15:48:05.585646+0000","last_fullsized":"2024-09-16T15:48:05.585646+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:30.742172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823342+0000","last_change":"2024-09-16T15:48:04.208147+0000","last_active":"2024-09-16T15:48:06.823342+0000","last_peered":"2024-09-16T15:48:06.823342+0000","last_clean":"2024-09-16T15:48:06.823342+0000","last_became_active":"2024-09-16T15:48:04.207884+0000","last_became_peered":"2024-09-16T15:48:04.207884+0000","last_unstale":"2024-09-16T15:48:06.823342+0000","last_undegraded":"2024-09-16T15:48:06.823342+0000","last_fullsized":"2024-09-16T15:48:06.823342+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:35.050040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823607+0000","last_change":"2024-09-16T15:48:03.197016+0000","last_active":"2024-09-16T15:48:06.823607+0000","last_peered":"2024-09-16T15:48:06.823607+0000","last_clean":"2024-09-16T15:48:06.823607+0000","last_became_active":"2024-09-16T15:48:03.196735+0000","last_became_peered":"2024-09-16T15:48:03.196735+0000","last_unstale":"2024-09-16T15:48:06.823607+0000","last_undegraded":"2024-09-16T15:48:06.823607+0000","last_fullsized":"2024-09-16T15:48:06.823607+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:54.808565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274011+0000","last_change":"2024-09-16T15:48:04.203315+0000","last_active":"2024-09-16T15:48:06.274011+0000","last_peered":"2024-09-16T15:48:06.274011+0000","last_clean":"2024-09-16T15:48:06.274011+0000","last_became_active":"2024-09-16T15:48:04.203102+0000","last_became_peered":"2024-09-16T15:48:04.203102+0000","last_unstale":"2024-09-16T15:48:06.274011+0000","last_undegraded":"2024-09-16T15:48:06.274011+0000","last_fullsized":"2024-09-16T15:48:06.274011+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:13.329855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198295+0000","last_change":"2024-09-16T15:48:03.190121+0000","last_active":"2024-09-16T15:48:05.198295+0000","last_peered":"2024-09-16T15:48:05.198295+0000","last_clean":"2024-09-16T15:48:05.198295+0000","last_became_active":"2024-09-16T15:48:03.189679+0000","last_became_peered":"2024-09-16T15:48:03.189679+0000","last_unstale":"2024-09-16T15:48:05.198295+0000","last_undegraded":"2024-09-16T15:48:05.198295+0000","last_fullsized":"2024-09-16T15:48:05.198295+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:58:29.770414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198096+0000","last_change":"2024-09-16T15:48:04.203493+0000","last_active":"2024-09-16T15:48:05.198096+0000","last_peered":"2024-09-16T15:48:05.198096+0000","last_clean":"2024-09-16T15:48:05.198096+0000","last_became_active":"2024-09-16T15:48:04.196079+0000","last_became_peered":"2024-09-16T15:48:04.196079+0000","last_unstale":"2024-09-16T15:48:05.198096+0000","last_undegraded":"2024-09-16T15:48:05.198096+0000","last_fullsized":"2024-09-16T15:48:05.198096+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:52.910495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273959+0000","last_change":"2024-09-16T15:48:03.198905+0000","last_active":"2024-09-16T15:48:06.273959+0000","last_peered":"2024-09-16T15:48:06.273959+0000","last_clean":"2024-09-16T15:48:06.273959+0000","last_became_active":"2024-09-16T15:48:03.198503+0000","last_became_peered":"2024-09-16T15:48:03.198503+0000","last_unstale":"2024-09-16T15:48:06.273959+0000","last_undegraded":"2024-09-16T15:48:06.273959+0000","last_fullsized":"2024-09-16T15:48:06.273959+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:54.895294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.197971+0000","last_change":"2024-09-16T15:48:04.202439+0000","last_active":"2024-09-16T15:48:05.197971+0000","last_peered":"2024-09-16T15:48:05.197971+0000","last_clean":"2024-09-16T15:48:05.197971+0000","last_became_active":"2024-09-16T15:48:04.201996+0000","last_became_peered":"2024-09-16T15:48:04.201996+0000","last_unstale":"2024-09-16T15:48:05.197971+0000","last_undegraded":"2024-09-16T15:48:05.197971+0000","last_fullsized":"2024-09-16T15:48:05.197971+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:09:56.020663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.198456+0000","last_change":"2024-09-16T15:48:03.196365+0000","last_active":"2024-09-16T15:48:05.198456+0000","last_peered":"2024-09-16T15:48:05.198456+0000","last_clean":"2024-09-16T15:48:05.198456+0000","last_became_active":"2024-09-16T15:48:03.196069+0000","last_became_peered":"2024-09-16T15:48:03.196069+0000","last_unstale":"2024-09-16T15:48:05.198456+0000","last_undegraded":"2024-09-16T15:48:05.198456+0000","last_fullsized":"2024-09-16T15:48:05.198456+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:34.865882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273961+0000","last_change":"2024-09-16T15:48:04.196696+0000","last_active":"2024-09-16T15:48:06.273961+0000","last_peered":"2024-09-16T15:48:06.273961+0000","last_clean":"2024-09-16T15:48:06.273961+0000","last_became_active":"2024-09-16T15:48:04.196424+0000","last_became_peered":"2024-09-16T15:48:04.196424+0000","last_unstale":"2024-09-16T15:48:06.273961+0000","last_undegraded":"2024-09-16T15:48:06.273961+0000","last_fullsized":"2024-09-16T15:48:06.273961+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:09:30.653715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822744+0000","last_change":"2024-09-16T15:48:03.201031+0000","last_active":"2024-09-16T15:48:06.822744+0000","last_peered":"2024-09-16T15:48:06.822744+0000","last_clean":"2024-09-16T15:48:06.822744+0000","last_became_active":"2024-09-16T15:48:03.200772+0000","last_became_peered":"2024-09-16T15:48:03.200772+0000","last_unstale":"2024-09-16T15:48:06.822744+0000","last_undegraded":"2024-09-16T15:48:06.822744+0000","last_fullsized":"2024-09-16T15:48:06.822744+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:09.154644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822929+0000","last_change":"2024-09-16T15:48:04.190203+0000","last_active":"2024-09-16T15:48:06.822929+0000","last_peered":"2024-09-16T15:48:06.822929+0000","last_clean":"2024-09-16T15:48:06.822929+0000","last_became_active":"2024-09-16T15:48:04.189887+0000","last_became_peered":"2024-09-16T15:48:04.189887+0000","last_unstale":"2024-09-16T15:48:06.822929+0000","last_undegraded":"2024-09-16T15:48:06.822929+0000","last_fullsized":"2024-09-16T15:48:06.822929+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:07.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822665+0000","last_change":"2024-09-16T15:48:03.194768+0000","last_active":"2024-09-16T15:48:06.822665+0000","last_peered":"2024-09-16T15:48:06.822665+0000","last_clean":"2024-09-16T15:48:06.822665+0000","last_became_active":"2024-09-16T15:48:03.194473+0000","last_became_peered":"2024-09-16T15:48:03.194473+0000","last_unstale":"2024-09-16T15:48:06.822665+0000","last_undegraded":"2024-09-16T15:48:06.822665+0000","last_fullsized":"2024-09-16T15:48:06.822665+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:46.612458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200135+0000","last_change":"2024-09-16T15:48:04.187882+0000","last_active":"2024-09-16T15:48:05.200135+0000","last_peered":"2024-09-16T15:48:05.200135+0000","last_clean":"2024-09-16T15:48:05.200135+0000","last_became_active":"2024-09-16T15:48:04.187539+0000","last_became_peered":"2024-09-16T15:48:04.187539+0000","last_unstale":"2024-09-16T15:48:05.200135+0000","last_undegraded":"2024-09-16T15:48:05.200135+0000","last_fullsized":"2024-09-16T15:48:05.200135+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:17.307654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585287+0000","last_change":"2024-09-16T15:48:03.195487+0000","last_active":"2024-09-16T15:48:05.585287+0000","last_peered":"2024-09-16T15:48:05.585287+0000","last_clean":"2024-09-16T15:48:05.585287+0000","last_became_active":"2024-09-16T15:48:03.195220+0000","last_became_peered":"2024-09-16T15:48:03.195220+0000","last_unstale":"2024-09-16T15:48:05.585287+0000","last_undegraded":"2024-09-16T15:48:05.585287+0000","last_fullsized":"2024-09-16T15:48:05.585287+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:45.093277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200680+0000","last_change":"2024-09-16T15:48:04.198829+0000","last_active":"2024-09-16T15:48:05.200680+0000","last_peered":"2024-09-16T15:48:05.200680+0000","last_clean":"2024-09-16T15:48:05.200680+0000","last_became_active":"2024-09-16T15:48:04.196541+0000","last_became_peered":"2024-09-16T15:48:04.196541+0000","last_unstale":"2024-09-16T15:48:05.200680+0000","last_undegraded":"2024-09-16T15:48:05.200680+0000","last_fullsized":"2024-09-16T15:48:05.200680+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:31.354447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"172'2","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274227+0000","last_change":"2024-09-16T15:48:03.194210+0000","last_active":"2024-09-16T15:48:06.274227+0000","last_peered":"2024-09-16T15:48:06.274227+0000","last_clean":"2024-09-16T15:48:06.274227+0000","last_became_active":"2024-09-16T15:48:03.193942+0000","last_became_peered":"2024-09-16T15:48:03.193942+0000","last_unstale":"2024-09-16T15:48:06.274227+0000","last_undegraded":"2024-09-16T15:48:06.274227+0000","last_fullsized":"2024-09-16T15:48:06.274227+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:39.201068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823208+0000","last_change":"2024-09-16T15:48:04.205111+0000","last_active":"2024-09-16T15:48:06.823208+0000","last_peered":"2024-09-16T15:48:06.823208+0000","last_clean":"2024-09-16T15:48:06.823208+0000","last_became_active":"2024-09-16T15:48:04.204835+0000","last_became_peered":"2024-09-16T15:48:04.204835+0000","last_unstale":"2024-09-16T15:48:06.823208+0000","last_undegraded":"2024-09-16T15:48:06.823208+0000","last_fullsized":"2024-09-16T15:48:06.823208+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:53.954030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"173'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.183243+0000","last_change":"2024-09-16T15:48:03.185058+0000","last_active":"2024-09-16T15:48:06.183243+0000","last_peered":"2024-09-16T15:48:06.183243+0000","last_clean":"2024-09-16T15:48:06.183243+0000","last_became_active":"2024-09-16T15:48:03.184767+0000","last_became_peered":"2024-09-16T15:48:03.184767+0000","last_unstale":"2024-09-16T15:48:06.183243+0000","last_undegraded":"2024-09-16T15:48:06.183243+0000","last_fullsized":"2024-09-16T15:48:06.183243+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:34.865882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.823198+0000","last_change":"2024-09-16T15:48:04.212157+0000","last_active":"2024-09-16T15:48:06.823198+0000","last_peered":"2024-09-16T15:48:06.823198+0000","last_clean":"2024-09-16T15:48:06.823198+0000","last_became_active":"2024-09-16T15:48:04.211392+0000","last_became_peered":"2024-09-16T15:48:04.211392+0000","last_unstale":"2024-09-16T15:48:06.823198+0000","last_undegraded":"2024-09-16T15:48:06.823198+0000","last_fullsized":"2024-09-16T15:48:06.823198+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:47.353119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585531+0000","last_change":"2024-09-16T15:48:03.192452+0000","last_active":"2024-09-16T15:48:05.585531+0000","last_peered":"2024-09-16T15:48:05.585531+0000","last_clean":"2024-09-16T15:48:05.585531+0000","last_became_active":"2024-09-16T15:48:03.192139+0000","last_became_peered":"2024-09-16T15:48:03.192139+0000","last_unstale":"2024-09-16T15:48:05.585531+0000","last_undegraded":"2024-09-16T15:48:05.585531+0000","last_fullsized":"2024-09-16T15:48:05.585531+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:08.925056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200776+0000","last_change":"2024-09-16T15:48:04.196200+0000","last_active":"2024-09-16T15:48:05.200776+0000","last_peered":"2024-09-16T15:48:05.200776+0000","last_clean":"2024-09-16T15:48:05.200776+0000","last_became_active":"2024-09-16T15:48:04.194266+0000","last_became_peered":"2024-09-16T15:48:04.194266+0000","last_unstale":"2024-09-16T15:48:05.200776+0000","last_undegraded":"2024-09-16T15:48:05.200776+0000","last_fullsized":"2024-09-16T15:48:05.200776+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:36:54.260034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.274082+0000","last_change":"2024-09-16T15:48:03.196561+0000","last_active":"2024-09-16T15:48:06.274082+0000","last_peered":"2024-09-16T15:48:06.274082+0000","last_clean":"2024-09-16T15:48:06.274082+0000","last_became_active":"2024-09-16T15:48:03.195181+0000","last_became_peered":"2024-09-16T15:48:03.195181+0000","last_unstale":"2024-09-16T15:48:06.274082+0000","last_undegraded":"2024-09-16T15:48:06.274082+0000","last_fullsized":"2024-09-16T15:48:06.274082+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:22.597636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822746+0000","last_change":"2024-09-16T15:48:04.198037+0000","last_active":"2024-09-16T15:48:06.822746+0000","last_peered":"2024-09-16T15:48:06.822746+0000","last_clean":"2024-09-16T15:48:06.822746+0000","last_became_active":"2024-09-16T15:48:04.197719+0000","last_became_peered":"2024-09-16T15:48:04.197719+0000","last_unstale":"2024-09-16T15:48:06.822746+0000","last_undegraded":"2024-09-16T15:48:06.822746+0000","last_fullsized":"2024-09-16T15:48:06.822746+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:42:16.552612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182919+0000","last_change":"2024-09-16T15:48:03.192744+0000","last_active":"2024-09-16T15:48:06.182919+0000","last_peered":"2024-09-16T15:48:06.182919+0000","last_clean":"2024-09-16T15:48:06.182919+0000","last_became_active":"2024-09-16T15:48:03.192390+0000","last_became_peered":"2024-09-16T15:48:03.192390+0000","last_unstale":"2024-09-16T15:48:06.182919+0000","last_undegraded":"2024-09-16T15:48:06.182919+0000","last_fullsized":"2024-09-16T15:48:06.182919+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.106382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200075+0000","last_change":"2024-09-16T15:48:04.209980+0000","last_active":"2024-09-16T15:48:05.200075+0000","last_peered":"2024-09-16T15:48:05.200075+0000","last_clean":"2024-09-16T15:48:05.200075+0000","last_became_active":"2024-09-16T15:48:04.209602+0000","last_became_peered":"2024-09-16T15:48:04.209602+0000","last_unstale":"2024-09-16T15:48:05.200075+0000","last_undegraded":"2024-09-16T15:48:05.200075+0000","last_fullsized":"2024-09-16T15:48:05.200075+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:27.132201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.585598+0000","last_change":"2024-09-16T15:48:03.190882+0000","last_active":"2024-09-16T15:48:05.585598+0000","last_peered":"2024-09-16T15:48:05.585598+0000","last_clean":"2024-09-16T15:48:05.585598+0000","last_became_active":"2024-09-16T15:48:03.190598+0000","last_became_peered":"2024-09-16T15:48:03.190598+0000","last_unstale":"2024-09-16T15:48:05.585598+0000","last_undegraded":"2024-09-16T15:48:05.585598+0000","last_fullsized":"2024-09-16T15:48:05.585598+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:39.810393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.273914+0000","last_change":"2024-09-16T15:48:04.188698+0000","last_active":"2024-09-16T15:48:06.273914+0000","last_peered":"2024-09-16T15:48:06.273914+0000","last_clean":"2024-09-16T15:48:06.273914+0000","last_became_active":"2024-09-16T15:48:04.188416+0000","last_became_peered":"2024-09-16T15:48:04.188416+0000","last_unstale":"2024-09-16T15:48:06.273914+0000","last_undegraded":"2024-09-16T15:48:06.273914+0000","last_fullsized":"2024-09-16T15:48:06.273914+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:28.390298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200199+0000","last_change":"2024-09-16T15:48:03.181891+0000","last_active":"2024-09-16T15:48:05.200199+0000","last_peered":"2024-09-16T15:48:05.200199+0000","last_clean":"2024-09-16T15:48:05.200199+0000","last_became_active":"2024-09-16T15:48:03.179526+0000","last_became_peered":"2024-09-16T15:48:03.179526+0000","last_unstale":"2024-09-16T15:48:05.200199+0000","last_undegraded":"2024-09-16T15:48:05.200199+0000","last_fullsized":"2024-09-16T15:48:05.200199+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:26:45.313588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822967+0000","last_change":"2024-09-16T15:48:04.190251+0000","last_active":"2024-09-16T15:48:06.822967+0000","last_peered":"2024-09-16T15:48:06.822967+0000","last_clean":"2024-09-16T15:48:06.822967+0000","last_became_active":"2024-09-16T15:48:04.189706+0000","last_became_peered":"2024-09-16T15:48:04.189706+0000","last_unstale":"2024-09-16T15:48:06.822967+0000","last_undegraded":"2024-09-16T15:48:06.822967+0000","last_fullsized":"2024-09-16T15:48:06.822967+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:07.019797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.199449+0000","last_change":"2024-09-16T15:48:03.187201+0000","last_active":"2024-09-16T15:48:05.199449+0000","last_peered":"2024-09-16T15:48:05.199449+0000","last_clean":"2024-09-16T15:48:05.199449+0000","last_became_active":"2024-09-16T15:48:03.184533+0000","last_became_peered":"2024-09-16T15:48:03.184533+0000","last_unstale":"2024-09-16T15:48:05.199449+0000","last_undegraded":"2024-09-16T15:48:05.199449+0000","last_fullsized":"2024-09-16T15:48:05.199449+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:17.844388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.822589+0000","last_change":"2024-09-16T15:48:04.200907+0000","last_active":"2024-09-16T15:48:06.822589+0000","last_peered":"2024-09-16T15:48:06.822589+0000","last_clean":"2024-09-16T15:48:06.822589+0000","last_became_active":"2024-09-16T15:48:04.199532+0000","last_became_peered":"2024-09-16T15:48:04.199532+0000","last_unstale":"2024-09-16T15:48:06.822589+0000","last_undegraded":"2024-09-16T15:48:06.822589+0000","last_fullsized":"2024-09-16T15:48:06.822589+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:03.162924+0000","last_clean_scrub_stamp":"2024-09-16T15:48:03.162924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:18.240793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"172'1","reported_seq":16,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:05.200149+0000","last_change":"2024-09-16T15:48:03.188117+0000","last_active":"2024-09-16T15:48:05.200149+0000","last_peered":"2024-09-16T15:48:05.200149+0000","last_clean":"2024-09-16T15:48:05.200149+0000","last_became_active":"2024-09-16T15:48:03.187786+0000","last_became_peered":"2024-09-16T15:48:03.187786+0000","last_unstale":"2024-09-16T15:48:05.200149+0000","last_undegraded":"2024-09-16T15:48:05.200149+0000","last_fullsized":"2024-09-16T15:48:05.200149+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:02.155612+0000","last_clean_scrub_stamp":"2024-09-16T15:48:02.155612+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:17:58.415177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"75'79","reported_seq":416,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182532+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:48:06.182532+0000","last_peered":"2024-09-16T15:48:06.182532+0000","last_clean":"2024-09-16T15:48:06.182532+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:48:06.182532+0000","last_undegraded":"2024-09-16T15:48:06.182532+0000","last_fullsized":"2024-09-16T15:48:06.182532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509561,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33892,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2768896,"data_stored":1546956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2179072,"data_stored":961886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2166784,"data_stored":955558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2195456,"data_stored":963020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2187264,"data_stored":961486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37460,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334380,"statfs":{"total":96636764160,"available":96598405120,"internally_reserved":0,"allocated":2162688,"data_stored":954976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542265,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37484,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334356,"statfs":{"total":96636764160,"available":96598380544,"internally_reserved":0,"allocated":2183168,"data_stored":961360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509561,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38044,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333796,"statfs":{"total":96636764160,"available":96597807104,"internally_reserved":0,"allocated":2760704,"data_stored":1545434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:08.883 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:48:08.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:48:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.087+0000 7f298dc00640 1 -- 172.21.15.60:0/3529902265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 msgr2=0x7f298806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.087+0000 7f298dc00640 1 --2- 172.21.15.60:0/3529902265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 0x7f298806f250 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f29880d1560 tx=0x7f298002aa10 comp rx=0 tx=0).stop 2024-09-16T15:48:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.087+0000 7f298dc00640 1 -- 172.21.15.60:0/3529902265 shutdown_connections 2024-09-16T15:48:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.087+0000 7f298dc00640 1 --2- 172.21.15.60:0/3529902265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 0x7f298806f250 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.087+0000 7f298dc00640 1 --2- 172.21.15.60:0/3529902265 >> v2:172.21.15.100:3300/0 conn(0x7f2988151cb0 0x7f29880b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.087+0000 7f298dc00640 1 -- 172.21.15.60:0/3529902265 >> 172.21.15.60:0/3529902265 conn(0x7f29880b2130 msgr2=0x7f29880b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.087+0000 7f298dc00640 1 -- 172.21.15.60:0/3529902265 shutdown_connections 2024-09-16T15:48:09.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.087+0000 7f298dc00640 1 -- 172.21.15.60:0/3529902265 wait complete. 2024-09-16T15:48:09.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.088+0000 7f298dc00640 1 Processor -- start 2024-09-16T15:48:09.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.089+0000 7f298dc00640 1 -- start start 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f298dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 0x7f29881f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f298dc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29880b3d10 0x7f298814b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f298dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2988151cb0 0x7f2988142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f298dc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f298806fba0 con 0x7f2988073740 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f298dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f298806fa20 con 0x7f2988151cb0 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f298dc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f298806fd20 con 0x7f29880b3d10 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 0x7f29881f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 0x7f29881f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53146/0 (socket says 172.21.15.60:53146) 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986a00640 1 -- 172.21.15.60:0/144955812 learned_addr learned my addr 172.21.15.60:0/144955812 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:09.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2987400640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2988151cb0 0x7f2988142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986000640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29880b3d10 0x7f298814b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986a00640 1 -- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29880b3d10 msgr2=0x7f298814b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986a00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29880b3d10 0x7f298814b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986a00640 1 -- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2988151cb0 msgr2=0x7f2988142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986a00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2988151cb0 0x7f2988142d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986a00640 1 -- 172.21.15.60:0/144955812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298814b7e0 con 0x7f2988073740 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.090+0000 7f2986000640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29880b3d10 0x7f298814b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.091+0000 7f2986a00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 0x7f29881f4c40 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f297800d6d0 tx=0x7f297800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.091+0000 7f297ea00640 1 -- 172.21.15.60:0/144955812 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978017070 con 0x7f2988073740 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.091+0000 7f297ea00640 1 -- 172.21.15.60:0/144955812 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2978005330 con 0x7f2988073740 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.091+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298814bac0 con 0x7f2988073740 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.091+0000 7f297ea00640 1 -- 172.21.15.60:0/144955812 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978009820 con 0x7f2988073740 2024-09-16T15:48:09.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.091+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298814c080 con 0x7f2988073740 2024-09-16T15:48:09.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.092+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2988070240 con 0x7f2988073740 2024-09-16T15:48:09.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.093+0000 7f297ea00640 1 -- 172.21.15.60:0/144955812 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f2978009bd0 con 0x7f2988073740 2024-09-16T15:48:09.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.094+0000 7f297ea00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f29680792a0 0x7f296807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.094+0000 7f297ea00640 1 -- 172.21.15.60:0/144955812 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f29780994c0 con 0x7f2988073740 2024-09-16T15:48:09.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.095+0000 7f2986000640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f29680792a0 0x7f296807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.095+0000 7f2986000640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f29680792a0 0x7f296807b760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2988148590 tx=0x7f2974009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:09.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.099+0000 7f297ea00640 1 -- 172.21.15.60:0/144955812 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2978062f30 con 0x7f2988073740 2024-09-16T15:48:09.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.372+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f29880cdb70 con 0x7f2988073740 2024-09-16T15:48:09.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.374+0000 7f297ea00640 1 -- 172.21.15.60:0/144955812 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7f2978067960 con 0x7f2988073740 2024-09-16T15:48:09.377 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-16T15:48:09.377 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQC9UOhmHPHuMRAARNd+N6VpnovAWHojhfFAuA== 2024-09-16T15:48:09.377 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-09-16T15:48:09.377 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-09-16T15:48:09.377 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:48:09.377 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.377+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f29680792a0 msgr2=0x7f296807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.377+0000 7f298dc00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f29680792a0 0x7f296807b760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2988148590 tx=0x7f2974009040 comp rx=0 tx=0).stop 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.377+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 msgr2=0x7f29881f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.377+0000 7f298dc00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 0x7f29881f4c40 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f297800d6d0 tx=0x7f297800db90 comp rx=0 tx=0).stop 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.378+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 shutdown_connections 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.378+0000 7f298dc00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f29680792a0 0x7f296807b760 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.378+0000 7f298dc00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2988151cb0 0x7f2988142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.378+0000 7f298dc00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29880b3d10 0x7f298814b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.378+0000 7f298dc00640 1 --2- 172.21.15.60:0/144955812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2988073740 0x7f29881f4c40 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.378+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 >> 172.21.15.60:0/144955812 conn(0x7f29880b2130 msgr2=0x7f29880b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.378+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 shutdown_connections 2024-09-16T15:48:09.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.378+0000 7f298dc00640 1 -- 172.21.15.60:0/144955812 wait complete. 2024-09-16T15:48:09.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:09.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.553+0000 7fd9a8800640 1 -- 172.21.15.60:0/2402235780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00b2440 msgr2=0x7fd9a006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.553+0000 7fd9a8800640 1 --2- 172.21.15.60:0/2402235780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00b2440 0x7fd9a006f160 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005fd0 tx=0x7fd99c02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.553+0000 7fd9a8800640 1 -- 172.21.15.60:0/2402235780 shutdown_connections 2024-09-16T15:48:09.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.553+0000 7fd9a8800640 1 --2- 172.21.15.60:0/2402235780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00b2440 0x7fd9a006f160 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.553+0000 7fd9a8800640 1 -- 172.21.15.60:0/2402235780 >> 172.21.15.60:0/2402235780 conn(0x7fd9a00b1950 msgr2=0x7fd9a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:09.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.553+0000 7fd9a8800640 1 -- 172.21.15.60:0/2402235780 shutdown_connections 2024-09-16T15:48:09.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.553+0000 7fd9a8800640 1 -- 172.21.15.60:0/2402235780 wait complete. 2024-09-16T15:48:09.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.555+0000 7fd9a8800640 1 Processor -- start 2024-09-16T15:48:09.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.555+0000 7fd9a8800640 1 -- start start 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a8800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9a00b2440 0x7fd9a00cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a8800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00ccf00 0x7fd9a00b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a8800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9a015f5a0 0x7fd9a015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a8800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a0074840 con 0x7fd9a00ccf00 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a8800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a00746c0 con 0x7fd9a00b2440 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a8800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a006f6a0 con 0x7fd9a015f5a0 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00ccf00 0x7fd9a00b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a5400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00ccf00 0x7fd9a00b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53160/0 (socket says 172.21.15.60:53160) 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a5400640 1 -- 172.21.15.60:0/4278213840 learned_addr learned my addr 172.21.15.60:0/4278213840 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a5e00640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9a00b2440 0x7fd9a00cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.556+0000 7fd9a6800640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9a015f5a0 0x7fd9a015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd9a5400640 1 -- 172.21.15.60:0/4278213840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9a015f5a0 msgr2=0x7fd9a015ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd9a5400640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9a015f5a0 0x7fd9a015ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd9a5400640 1 -- 172.21.15.60:0/4278213840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9a00b2440 msgr2=0x7fd9a00cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd9a5400640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9a00b2440 0x7fd9a00cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd9a5400640 1 -- 172.21.15.60:0/4278213840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a00cbab0 con 0x7fd9a00ccf00 2024-09-16T15:48:09.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd9a5400640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00ccf00 0x7fd9a00b08d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c500 tx=0x7fd99000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd99a000640 1 -- 172.21.15.60:0/4278213840 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9900052d0 con 0x7fd9a00ccf00 2024-09-16T15:48:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd99a000640 1 -- 172.21.15.60:0/4278213840 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd990005470 con 0x7fd9a00ccf00 2024-09-16T15:48:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd99a000640 1 -- 172.21.15.60:0/4278213840 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990012870 con 0x7fd9a00ccf00 2024-09-16T15:48:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a00b13b0 con 0x7fd9a00ccf00 2024-09-16T15:48:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.557+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a01f4f50 con 0x7fd9a00ccf00 2024-09-16T15:48:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.559+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a006f160 con 0x7fd9a00ccf00 2024-09-16T15:48:09.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.560+0000 7fd99a000640 1 -- 172.21.15.60:0/4278213840 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fd9900035f0 con 0x7fd9a00ccf00 2024-09-16T15:48:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.561+0000 7fd99a000640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd984079270 0x7fd98407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.561+0000 7fd99a000640 1 -- 172.21.15.60:0/4278213840 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fd990098f60 con 0x7fd9a00ccf00 2024-09-16T15:48:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.561+0000 7fd9a5e00640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd984079270 0x7fd98407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.562+0000 7fd9a5e00640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd984079270 0x7fd98407b730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00b4f30 tx=0x7fd994009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.565+0000 7fd99a000640 1 -- 172.21.15.60:0/4278213840 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd990062730 con 0x7fd9a00ccf00 2024-09-16T15:48:09.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.796+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9a01f5120 con 0x7fd9a00ccf00 2024-09-16T15:48:09.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.797+0000 7fd99a000640 1 -- 172.21.15.60:0/4278213840 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5227 (secure 0 0 0) 0x7fd990067160 con 0x7fd9a00ccf00 2024-09-16T15:48:09.799 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:09.799 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":122,"btime":"2024-09-16T15:48:07:203498+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/657477165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":657477165},{"type":"v1","addr":"172.21.15.100:6801","nonce":657477165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/293588121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":293588121},{"type":"v1","addr":"172.21.15.100:6835","nonce":293588121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/903832682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":903832682},{"type":"v1","addr":"172.21.15.60:6834","nonce":903832682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:03.623622+0000","modified":"2024-09-16T15:48:06.645819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/786870322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":786870322},{"type":"v1","addr":"172.21.15.60:6835","nonce":786870322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9222,"qdb_cluster":[9222]},"id":13}]} 2024-09-16T15:48:09.799 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 122 2024-09-16T15:48:09.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd984079270 msgr2=0x7fd98407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd984079270 0x7fd98407b730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00b4f30 tx=0x7fd994009040 comp rx=0 tx=0).stop 2024-09-16T15:48:09.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00ccf00 msgr2=0x7fd9a00b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00ccf00 0x7fd9a00b08d0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c500 tx=0x7fd99000c9c0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 shutdown_connections 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd984079270 0x7fd98407b730 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9a015f5a0 0x7fd9a015ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9a00ccf00 0x7fd9a00b08d0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 --2- 172.21.15.60:0/4278213840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9a00b2440 0x7fd9a00cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 >> 172.21.15.60:0/4278213840 conn(0x7fd9a00b1950 msgr2=0x7fd9a00ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 shutdown_connections 2024-09-16T15:48:09.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:09.799+0000 7fd9a8800640 1 -- 172.21.15.60:0/4278213840 wait complete. 2024-09-16T15:48:10.151 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-09-16T15:48:10.150+0000 7f87c6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:48:10.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 122, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:48:03.623622+0000', 'modified': '2024-09-16T15:48:06.645819+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9222}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9222': {'gid': 9222, 'name': 'c', 'rank': 0, 'incarnation': 118, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/786870322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 786870322}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 786870322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9222, 'qdb_cluster': [9222]} 2024-09-16T15:48:10.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:48:10.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:48:10.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.384+0000 7fb965400640 1 -- 172.21.15.60:0/3084960876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9600af3d0 msgr2=0x7fb96006f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.384+0000 7fb965400640 1 --2- 172.21.15.60:0/3084960876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9600af3d0 0x7fb96006f490 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fb96006d9e0 tx=0x7fb94c02a560 comp rx=0 tx=0).stop 2024-09-16T15:48:10.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.384+0000 7fb965400640 1 -- 172.21.15.60:0/3084960876 shutdown_connections 2024-09-16T15:48:10.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.384+0000 7fb965400640 1 --2- 172.21.15.60:0/3084960876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9600af3d0 0x7fb96006f490 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.384+0000 7fb965400640 1 --2- 172.21.15.60:0/3084960876 >> v2:172.21.15.100:3300/0 conn(0x7fb960161550 0x7fb96006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.384+0000 7fb965400640 1 -- 172.21.15.60:0/3084960876 >> 172.21.15.60:0/3084960876 conn(0x7fb9600b1950 msgr2=0x7fb9600b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.385+0000 7fb965400640 1 -- 172.21.15.60:0/3084960876 shutdown_connections 2024-09-16T15:48:10.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.385+0000 7fb965400640 1 -- 172.21.15.60:0/3084960876 wait complete. 2024-09-16T15:48:10.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.386+0000 7fb965400640 1 Processor -- start 2024-09-16T15:48:10.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.386+0000 7fb965400640 1 -- start start 2024-09-16T15:48:10.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb965400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb96006f9d0 0x7fb9601547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb965400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9600af3d0 0x7fb960155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb965400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb960161550 0x7fb96015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb965400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb960073f80 con 0x7fb960161550 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb965400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb960073e00 con 0x7fb96006f9d0 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb965400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb960074100 con 0x7fb9600af3d0 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb95f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb960161550 0x7fb96015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb95f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb960161550 0x7fb96015a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53186/0 (socket says 172.21.15.60:53186) 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.387+0000 7fb95f200640 1 -- 172.21.15.60:0/1788518489 learned_addr learned my addr 172.21.15.60:0/1788518489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb95e800640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb96006f9d0 0x7fb9601547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb95de00640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9600af3d0 0x7fb960155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb95f200640 1 -- 172.21.15.60:0/1788518489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9600af3d0 msgr2=0x7fb960155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb95f200640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9600af3d0 0x7fb960155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb95f200640 1 -- 172.21.15.60:0/1788518489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb96006f9d0 msgr2=0x7fb9601547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb95f200640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb96006f9d0 0x7fb9601547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb95f200640 1 -- 172.21.15.60:0/1788518489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96009bf90 con 0x7fb960161550 2024-09-16T15:48:10.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb95f200640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb960161550 0x7fb96015a010 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c7d0 tx=0x7fb95800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb956a00640 1 -- 172.21.15.60:0/1788518489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958003b70 con 0x7fb960161550 2024-09-16T15:48:10.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb956a00640 1 -- 172.21.15.60:0/1788518489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb958003d30 con 0x7fb960161550 2024-09-16T15:48:10.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb956a00640 1 -- 172.21.15.60:0/1788518489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95800aab0 con 0x7fb960161550 2024-09-16T15:48:10.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.388+0000 7fb965400640 1 -- 172.21.15.60:0/1788518489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96009c270 con 0x7fb960161550 2024-09-16T15:48:10.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.389+0000 7fb965400640 1 -- 172.21.15.60:0/1788518489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96009c830 con 0x7fb960161550 2024-09-16T15:48:10.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.392+0000 7fb956a00640 1 -- 172.21.15.60:0/1788518489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fb95800ac50 con 0x7fb960161550 2024-09-16T15:48:10.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.393+0000 7fb956a00640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb9400792a0 0x7fb94007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.393+0000 7fb9682ac740 1 Processor -- start 2024-09-16T15:48:10.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.393+0000 7fb956a00640 1 -- 172.21.15.60:0/1788518489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fb9580990d0 con 0x7fb960161550 2024-09-16T15:48:10.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.393+0000 7fb9682ac740 1 -- start start 2024-09-16T15:48:10.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.393+0000 7fb95e800640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb9400792a0 0x7fb94007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb95e800640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb9400792a0 0x7fb94007b760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb950009b30 tx=0x7fb950005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 0x559282269c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224a20 0x559282268390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x5592822688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb9682ac740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x5592821a7430 con 0x559282224fa0 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb9682ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5592821975d0 con 0x5592822698b0 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb9682ac740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x559282186a00 con 0x559282224a20 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb95f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x5592822688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb95f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x5592822688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53190/0 (socket says 172.21.15.60:53190) 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.394+0000 7fb95f200640 1 -- 172.21.15.60:0/30436396 learned_addr learned my addr 172.21.15.60:0/30436396 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb95de00640 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224a20 0x559282268390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb95e800640 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 0x559282269c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb95f200640 1 -- 172.21.15.60:0/30436396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224a20 msgr2=0x559282268390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb95f200640 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224a20 0x559282268390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb95f200640 1 -- 172.21.15.60:0/30436396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 msgr2=0x559282269c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:10.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb95f200640 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 0x559282269c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb95f200640 1 -- 172.21.15.60:0/30436396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928214f520 con 0x559282224fa0 2024-09-16T15:48:10.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb95f200640 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x5592822688d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c7a0 tx=0x7fb95801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc7006304ba701b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb94b000640 1 -- 172.21.15.60:0/30436396 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95806bf70 con 0x559282224fa0 2024-09-16T15:48:10.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb94b000640 1 -- 172.21.15.60:0/30436396 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb958067120 con 0x559282224fa0 2024-09-16T15:48:10.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.395+0000 7fb94b000640 1 -- 172.21.15.60:0/30436396 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958063e00 con 0x559282224fa0 2024-09-16T15:48:10.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 -- 172.21.15.60:0/30436396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 msgr2=0x5592822688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x5592822688d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c7a0 tx=0x7fb95801b000 comp rx=0 tx=0).stop 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 -- 172.21.15.60:0/30436396 shutdown_connections 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x5592822688d0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224a20 0x559282268390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 --2- 172.21.15.60:0/30436396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 0x559282269c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 -- 172.21.15.60:0/30436396 >> 172.21.15.60:0/30436396 conn(0x55928221fff0 msgr2=0x559282223a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 -- 172.21.15.60:0/30436396 shutdown_connections 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.396+0000 7fb9682ac740 1 -- 172.21.15.60:0/30436396 wait complete. 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.397+0000 7fb9682ac740 1 -- start start 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.397+0000 7fb9682ac740 1 Processor -- start 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x559282237ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 0x559282263c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55928224e9d0 0x559282264150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb9682ac740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x55928212d330 con 0x559282224fa0 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb9682ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55928218a810 con 0x5592822698b0 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb9682ac740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x559281ef7620 con 0x55928224e9d0 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb9682ac740 10 client.0 _ll_register_callbacks cb 0x559282132420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb95e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x559282237ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb95e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x559282237ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53200/0 (socket says 172.21.15.60:53200) 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.398+0000 7fb95e800640 1 -- 172.21.15.60:0/488683771 learned_addr learned my addr 172.21.15.60:0/488683771 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb95f200640 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55928224e9d0 0x559282264150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb95de00640 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 0x559282263c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb95e800640 1 -- 172.21.15.60:0/488683771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55928224e9d0 msgr2=0x559282264150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb95e800640 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55928224e9d0 0x559282264150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb95e800640 1 -- 172.21.15.60:0/488683771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 msgr2=0x559282263c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb95e800640 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 0x559282263c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb95e800640 1 -- 172.21.15.60:0/488683771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559281ee8850 con 0x559282224fa0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb95e800640 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x559282237ba0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fb950004600 tx=0x7fb950004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed54144aa0ce6f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb93f400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb93f400640 1 -- 172.21.15.60:0/488683771 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950005220 con 0x559282224fa0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb93f400640 1 -- 172.21.15.60:0/488683771 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fb9500053c0 con 0x559282224fa0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.399+0000 7fb9682ac740 1 -- 172.21.15.60:0/488683771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb958067120 con 0x559282224fa0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb93f400640 1 -- 172.21.15.60:0/488683771 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) ==== 24+0+0 (secure 0 0 0) 0x7fb950005600 con 0x559282224fa0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb9682ac740 10 client.9348 fetch_fsmap learned FSMap version 122 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb9682ac740 1 -- 172.21.15.60:0/488683771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) -- 0x559281f5f6b0 con 0x559282224fa0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb93f400640 1 -- 172.21.15.60:0/488683771 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb958067120 con 0x559282224fa0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb93f400640 1 -- 172.21.15.60:0/488683771 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 122) ==== 3502+0+0 (secure 0 0 0) 0x7fb95002f050 con 0x559282224fa0 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb9682ac740 10 client.9348 fetch_fsmap finished waiting for FSMap version 122 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb9682ac740 10 client.9348 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:48:10.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb9682ac740 20 client.9348 populate_metadata read hostname 'smithi060' 2024-09-16T15:48:10.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb9682ac740 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] conn(0x5592822394d0 0x559282285f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb9682ac740 4 client.9348 mds_command: new command op to 9222 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:48:10.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb9682ac740 1 -- 172.21.15.60:0/488683771 --> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559282199520 con 0x5592822394d0 2024-09-16T15:48:10.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.400+0000 7fb95de00640 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] conn(0x5592822394d0 0x559282285f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.401+0000 7fb95de00640 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] conn(0x5592822394d0 0x559282285f00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4210e7f9bfcc906 server_cookie=ea44544daee7c956 in_seq=0 out_seq=0 2024-09-16T15:48:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.401+0000 7fb93f400640 10 client.9348 ms_handle_connect on v2:172.21.15.60:6833/786870322 2024-09-16T15:48:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb93f400640 1 -- 172.21.15.60:0/488683771 <== mds.0 v2:172.21.15.60:6833/786870322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559282199520 con 0x5592822394d0 2024-09-16T15:48:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb93f400640 10 client.9348 handle_command_reply: tid=0 2024-09-16T15:48:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 client.9348 shutdown 2024-09-16T15:48:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 -- 172.21.15.60:0/488683771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 msgr2=0x559282237ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x559282237ba0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fb950004600 tx=0x7fb950004730 comp rx=0 tx=0).stop 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 -- 172.21.15.60:0/488683771 shutdown_connections 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] conn(0x5592822394d0 0x559282285f00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55928224e9d0 0x559282264150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5592822698b0 0x559282263c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 --2- 172.21.15.60:0/488683771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x559282237ba0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 -- 172.21.15.60:0/488683771 >> 172.21.15.60:0/488683771 conn(0x559282224a20 msgr2=0x55928221d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb93f400640 0 client.9348 ms_handle_reset on v2:172.21.15.60:6833/786870322 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 -- 172.21.15.60:0/488683771 shutdown_connections 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 1 -- 172.21.15.60:0/488683771 wait complete. 2024-09-16T15:48:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.402+0000 7fb9682ac740 20 client.9348 trim_cache size 0 max 16384 2024-09-16T15:48:10.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.422+0000 7fb9682ac740 1 Processor -- start 2024-09-16T15:48:10.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.422+0000 7fb9682ac740 1 -- start start 2024-09-16T15:48:10.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224fa0 0x55928226b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x55928226bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5592822698b0 0x55928224ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb9682ac740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb9601f8ee0 con 0x5592822698b0 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb9682ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb9601f8240 con 0x55928224e9d0 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb9682ac740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb9601f8890 con 0x559282224fa0 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5592822698b0 0x55928224ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5592822698b0 0x55928224ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53216/0 (socket says 172.21.15.60:53216) 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95f200640 1 -- 172.21.15.60:0/3086988283 learned_addr learned my addr 172.21.15.60:0/3086988283 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95de00640 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x55928226bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95e800640 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224fa0 0x55928226b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95f200640 1 -- 172.21.15.60:0/3086988283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224fa0 msgr2=0x55928226b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95f200640 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224fa0 0x55928226b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95f200640 1 -- 172.21.15.60:0/3086988283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 msgr2=0x55928226bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:10.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95f200640 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x55928226bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.423+0000 7fb95f200640 1 -- 172.21.15.60:0/3086988283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928226a780 con 0x5592822698b0 2024-09-16T15:48:10.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.424+0000 7fb95f200640 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5592822698b0 0x55928224ce60 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb958098f80 tx=0x7fb958000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571cfa74828b6321 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.424+0000 7fb94a600640 1 -- 172.21.15.60:0/3086988283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958067120 con 0x5592822698b0 2024-09-16T15:48:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.424+0000 7fb94a600640 1 -- 172.21.15.60:0/3086988283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb958062920 con 0x5592822698b0 2024-09-16T15:48:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.424+0000 7fb94a600640 1 -- 172.21.15.60:0/3086988283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95806bfd0 con 0x5592822698b0 2024-09-16T15:48:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 -- 172.21.15.60:0/3086988283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5592822698b0 msgr2=0x55928224ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5592822698b0 0x55928224ce60 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb958098f80 tx=0x7fb958000f40 comp rx=0 tx=0).stop 2024-09-16T15:48:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 -- 172.21.15.60:0/3086988283 shutdown_connections 2024-09-16T15:48:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5592822698b0 0x55928224ce60 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x55928226bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 --2- 172.21.15.60:0/3086988283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559282224fa0 0x55928226b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 -- 172.21.15.60:0/3086988283 >> 172.21.15.60:0/3086988283 conn(0x559282224a20 msgr2=0x559282237620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 -- 172.21.15.60:0/3086988283 shutdown_connections 2024-09-16T15:48:10.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 -- 172.21.15.60:0/3086988283 wait complete. 2024-09-16T15:48:10.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.425+0000 7fb9682ac740 1 -- start start 2024-09-16T15:48:10.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.426+0000 7fb9682ac740 1 Processor -- start 2024-09-16T15:48:10.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.426+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x55928227d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.426+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x559282248f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb9682ac740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5592822698b0 0x5592822de060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb9682ac740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x559282239350 con 0x559282224fa0 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb9682ac740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559282238fa0 con 0x55928224e9d0 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb9682ac740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x5592822399a0 con 0x5592822698b0 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb9682ac740 10 client.0 _ll_register_callbacks cb 0x7fb9500043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x55928227d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x55928227d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53228/0 (socket says 172.21.15.60:53228) 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 -- 172.21.15.60:0/1931167960 learned_addr learned my addr 172.21.15.60:0/1931167960 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95f200640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5592822698b0 0x5592822de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95de00640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x559282248f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 -- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5592822698b0 msgr2=0x5592822de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:10.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5592822698b0 0x5592822de060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 -- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 msgr2=0x559282248f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x559282248f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 -- 172.21.15.60:0/1931167960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592822822a0 con 0x559282224fa0 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95de00640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x559282248f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95f200640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5592822698b0 0x5592822de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb95e800640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x55928227d880 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb950031570 tx=0x7fb9500315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da223782904f4376 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.427+0000 7fb93f400640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb93f400640 1 -- 172.21.15.60:0/1931167960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500319d0 con 0x559282224fa0 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb93f400640 1 -- 172.21.15.60:0/1931167960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fb950009040 con 0x559282224fa0 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 1 -- 172.21.15.60:0/1931167960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb958062920 con 0x559282224fa0 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb93f400640 1 -- 172.21.15.60:0/1931167960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) ==== 24+0+0 (secure 0 0 0) 0x7fb95003f050 con 0x559282224fa0 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 10 client.9354 fetch_fsmap learned FSMap version 122 2024-09-16T15:48:10.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 1 -- 172.21.15.60:0/1931167960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) -- 0x55928227f6e0 con 0x559282224fa0 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb93f400640 1 -- 172.21.15.60:0/1931167960 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb958062920 con 0x559282224fa0 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb93f400640 1 -- 172.21.15.60:0/1931167960 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 122) ==== 3502+0+0 (secure 0 0 0) 0x7fb950042020 con 0x559282224fa0 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 10 client.9354 fetch_fsmap finished waiting for FSMap version 122 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 10 client.9354 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 20 client.9354 populate_metadata read hostname 'smithi060' 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] conn(0x55928224d3e0 0x55928221ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 4 client.9354 mds_command: new command op to 9222 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.428+0000 7fb9682ac740 1 -- 172.21.15.60:0/1931167960 --> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] -- command(tid 0: {"prefix": "status"}) -- 0x559282194fd0 con 0x55928224d3e0 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.429+0000 7fb95de00640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] conn(0x55928224d3e0 0x55928221ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.429+0000 7fb95de00640 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] conn(0x55928224d3e0 0x55928221ce40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab00f4bcebaba2f server_cookie=54eba7a1a2e776e3 in_seq=0 out_seq=0 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.429+0000 7fb93f400640 10 client.9354 ms_handle_connect on v2:172.21.15.60:6833/786870322 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.429+0000 7fb93f400640 1 -- 172.21.15.60:0/1931167960 <== mds.0 v2:172.21.15.60:6833/786870322 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x559282194fd0 con 0x55928224d3e0 2024-09-16T15:48:10.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.429+0000 7fb93f400640 10 client.9354 handle_command_reply: tid=0 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.429+0000 7fb9682ac740 1 client.9354 shutdown 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 -- 172.21.15.60:0/1931167960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 msgr2=0x55928227d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x55928227d880 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb950031570 tx=0x7fb9500315a0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 -- 172.21.15.60:0/1931167960 shutdown_connections 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.60:6833/786870322,v1:172.21.15.60:6835/786870322] conn(0x55928224d3e0 0x55928221ce40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5592822698b0 0x5592822de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55928224e9d0 0x559282248f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 --2- 172.21.15.60:0/1931167960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559282224fa0 0x55928227d880 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb93f400640 0 client.9354 ms_handle_reset on v2:172.21.15.60:6833/786870322 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 -- 172.21.15.60:0/1931167960 >> 172.21.15.60:0/1931167960 conn(0x559282224a20 msgr2=0x55928224f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 -- 172.21.15.60:0/1931167960 shutdown_connections 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 1 -- 172.21.15.60:0/1931167960 wait complete. 2024-09-16T15:48:10.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.430+0000 7fb9682ac740 20 client.9354 trim_cache size 0 max 16384 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "b577f92e-7c09-4d32-a34c-b870818f4d85", 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9222, 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 6.2578781689999996, 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 122, 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 173, 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 173, 2024-09-16T15:48:10.437 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 11.531632747 2024-09-16T15:48:10.438 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 -- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb9400792a0 msgr2=0x7fb94007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb9400792a0 0x7fb94007b760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb950009b30 tx=0x7fb950005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:10.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 -- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb960161550 msgr2=0x7fb96015a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb960161550 0x7fb96015a010 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c7d0 tx=0x7fb95800cc90 comp rx=0 tx=0).stop 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 -- 172.21.15.60:0/1788518489 shutdown_connections 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb9400792a0 0x7fb94007b760 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb960161550 0x7fb96015a010 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9600af3d0 0x7fb960155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 --2- 172.21.15.60:0/1788518489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb96006f9d0 0x7fb9601547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 -- 172.21.15.60:0/1788518489 >> 172.21.15.60:0/1788518489 conn(0x7fb9600b1950 msgr2=0x7fb96006ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 -- 172.21.15.60:0/1788518489 shutdown_connections 2024-09-16T15:48:10.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.431+0000 7fb965400640 1 -- 172.21.15.60:0/1788518489 wait complete. 2024-09-16T15:48:10.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:48:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.606+0000 7f5cdb200640 1 -- 172.21.15.60:0/2705240392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 msgr2=0x7f5cdc0d1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.606+0000 7f5cdb200640 1 --2- 172.21.15.60:0/2705240392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 0x7f5cdc0d1c60 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005fd0 tx=0x7f5cc802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.606+0000 7f5cdb200640 1 -- 172.21.15.60:0/2705240392 shutdown_connections 2024-09-16T15:48:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.606+0000 7f5cdb200640 1 --2- 172.21.15.60:0/2705240392 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 0x7f5cdc0d1c60 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.606+0000 7f5cdb200640 1 --2- 172.21.15.60:0/2705240392 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cdc1612d0 0x7f5cdc0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.606+0000 7f5cdb200640 1 -- 172.21.15.60:0/2705240392 >> 172.21.15.60:0/2705240392 conn(0x7f5cdc0d3df0 msgr2=0x7f5cdc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.606+0000 7f5cdb200640 1 -- 172.21.15.60:0/2705240392 shutdown_connections 2024-09-16T15:48:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.606+0000 7f5cdb200640 1 -- 172.21.15.60:0/2705240392 wait complete. 2024-09-16T15:48:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.607+0000 7f5cdb200640 1 Processor -- start 2024-09-16T15:48:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.608+0000 7f5cdb200640 1 -- start start 2024-09-16T15:48:10.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.608+0000 7f5cdb200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 0x7f5cdc1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.608+0000 7f5cdb200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cdc1612d0 0x7f5cdc06dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cdb200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cdc1616b0 0x7f5cdc0b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cdb200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc0b1c00 con 0x7f5cdc0d4980 2024-09-16T15:48:10.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cdb200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc0b1a80 con 0x7f5cdc1612d0 2024-09-16T15:48:10.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cdb200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5cdc0b1d80 con 0x7f5cdc1616b0 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 0x7f5cdc1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 0x7f5cdc1f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53246/0 (socket says 172.21.15.60:53246) 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 -- 172.21.15.60:0/931371135 learned_addr learned my addr 172.21.15.60:0/931371135 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cda800640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cdc1616b0 0x7f5cdc0b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9400640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cdc1612d0 0x7f5cdc06dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 -- 172.21.15.60:0/931371135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cdc1616b0 msgr2=0x7f5cdc0b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cdc1616b0 0x7f5cdc0b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 -- 172.21.15.60:0/931371135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cdc1612d0 msgr2=0x7f5cdc06dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cdc1612d0 0x7f5cdc06dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 -- 172.21.15.60:0/931371135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc0b7b80 con 0x7f5cdc0d4980 2024-09-16T15:48:10.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.609+0000 7f5cd9e00640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 0x7f5cdc1f5080 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c920 tx=0x7f5ccc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.610+0000 7f5cd2000640 1 -- 172.21.15.60:0/931371135 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc007c70 con 0x7f5cdc0d4980 2024-09-16T15:48:10.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.610+0000 7f5cd2000640 1 -- 172.21.15.60:0/931371135 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ccc00f450 con 0x7f5cdc0d4980 2024-09-16T15:48:10.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.610+0000 7f5cd2000640 1 -- 172.21.15.60:0/931371135 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00f8a0 con 0x7f5cdc0d4980 2024-09-16T15:48:10.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.610+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc0b6dc0 con 0x7f5cdc0d4980 2024-09-16T15:48:10.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.610+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc0b7380 con 0x7f5cdc0d4980 2024-09-16T15:48:10.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.611+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cdc0b4e00 con 0x7f5cdc0d4980 2024-09-16T15:48:10.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.612+0000 7f5cd2000640 1 -- 172.21.15.60:0/931371135 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f5ccc00fa40 con 0x7f5cdc0d4980 2024-09-16T15:48:10.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.613+0000 7f5cd2000640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cbc0792a0 0x7f5cbc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.613+0000 7f5cd2000640 1 -- 172.21.15.60:0/931371135 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f5ccc099c80 con 0x7f5cdc0d4980 2024-09-16T15:48:10.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.613+0000 7f5cd9400640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cbc0792a0 0x7f5cbc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.614+0000 7f5cd9400640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cbc0792a0 0x7f5cbc07b760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8033000 tx=0x7f5cc8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.617+0000 7f5cd2000640 1 -- 172.21.15.60:0/931371135 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5ccc014030 con 0x7f5cdc0d4980 2024-09-16T15:48:10.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.853+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5cdc0b3a00 con 0x7f5cdc0d4980 2024-09-16T15:48:10.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.854+0000 7f5cd2000640 1 -- 172.21.15.60:0/931371135 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v122) ==== 53+0+73 (secure 0 0 0) 0x7f5ccc063b20 con 0x7f5cdc0d4980 2024-09-16T15:48:10.856 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:48:10.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cbc0792a0 msgr2=0x7f5cbc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cbc0792a0 0x7f5cbc07b760 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8033000 tx=0x7f5cc8004730 comp rx=0 tx=0).stop 2024-09-16T15:48:10.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 msgr2=0x7f5cdc1f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 0x7f5cdc1f5080 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c920 tx=0x7f5ccc00cde0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 shutdown_connections 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5cbc0792a0 0x7f5cbc07b760 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cdc1616b0 0x7f5cdc0b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cdc1612d0 0x7f5cdc06dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 --2- 172.21.15.60:0/931371135 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cdc0d4980 0x7f5cdc1f5080 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 >> 172.21.15.60:0/931371135 conn(0x7f5cdc0d3df0 msgr2=0x7f5cdc0cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 shutdown_connections 2024-09-16T15:48:10.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:10.856+0000 7f5cdb200640 1 -- 172.21.15.60:0/931371135 wait complete. 2024-09-16T15:48:10.873 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:48:10.873 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:48:10.874 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-09-16T15:48:10.874 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:48:10.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:10.874 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:48:10.874 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:48:10.874 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:48:10.874 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:48:10.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:10.874 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-09-16T15:48:10.910 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:48:10.910 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:48:10.910 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:48:10.910 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:10.910 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-09-16T15:48:10.910 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42158sec preferred_lft 42158sec 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-16T15:48:10.911 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::68e0:18ff:fe06:b0af/64 scope link 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5e:91:8f:4b:ea:c2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::5c91:8fff:fe4b:eac2/64 scope link 2024-09-16T15:48:10.912 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:10.913 INFO:teuthology.orchestra.run.smithi060.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:48:10.913 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 02:72:95:49:e2:88 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:48:10.913 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::72:95ff:fe49:e288/64 scope link 2024-09-16T15:48:10.913 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:10.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:10.913 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:48:11.004 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-16T15:48:11.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:11.007 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-09-16T15:48:11.043 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-09-16T15:48:11.043 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:48:11.043 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T15:48:11.043 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T15:48:11.043 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:48:11.043 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:48:11.043 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-09-16T15:48:11.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T15:48:11.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T15:48:11.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T15:48:11.044 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-09-16T15:48:11.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:48:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:48:11.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:11.244 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:48:11.279 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:48:11.279 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:48:11.279 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:48:11.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:11.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-09-16T15:48:11.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:48:11.376 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:11.411 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:48:11.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:48:11.412 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:11.487 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-16T15:48:11.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:11.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:48:11.616 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:48:11.616 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:48:11.645 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:48:11.645 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:48:11.645 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:48:11.645 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with new device syntax: 0@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:48:11.645 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:48:11.647 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:48:11.647 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:48:11.648 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:48:11.648 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:48:11.648 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:48:11.651 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:48:11.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:11.651 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:11.707 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:48:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:48:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:48:11.804 INFO:teuthology.orchestra.run.smithi060.stdout:client7694 2024-09-16T15:48:11.804 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:48:11.804 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:11.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:48:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.037+0000 7f7c6ec00640 1 -- 172.21.15.60:0/2540335014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c680d0670 msgr2=0x7f7c680d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.038+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/2540335014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c680d0670 0x7f7c680d2760 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7c680cc220 tx=0x7f7c5402a5f0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.038+0000 7f7c6ec00640 1 -- 172.21.15.60:0/2540335014 shutdown_connections 2024-09-16T15:48:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.038+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/2540335014 >> v2:172.21.15.100:3300/0 conn(0x7f7c680b1680 0x7f7c680b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.038+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/2540335014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c680d0670 0x7f7c680d2760 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.038+0000 7f7c6ec00640 1 -- 172.21.15.60:0/2540335014 >> 172.21.15.60:0/2540335014 conn(0x7f7c680d1c40 msgr2=0x7f7c680d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.038+0000 7f7c6ec00640 1 -- 172.21.15.60:0/2540335014 shutdown_connections 2024-09-16T15:48:12.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.038+0000 7f7c6ec00640 1 -- 172.21.15.60:0/2540335014 wait complete. 2024-09-16T15:48:12.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.039+0000 7f7c6ec00640 1 Processor -- start 2024-09-16T15:48:12.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.039+0000 7f7c6ec00640 1 -- start start 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.040+0000 7f7c6ec00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c680b1680 0x7f7c68152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.040+0000 7f7c6ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c680d0670 0x7f7c680b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.040+0000 7f7c6ec00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6815bd30 0x7f7c6814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.040+0000 7f7c6ec00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7c680b21f0 con 0x7f7c6815bd30 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.040+0000 7f7c6ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7c680b2070 con 0x7f7c680d0670 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.040+0000 7f7c6ec00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7c680b2370 con 0x7f7c680b1680 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6815bd30 0x7f7c6814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6815bd30 0x7f7c6814c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53258/0 (socket says 172.21.15.60:53258) 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 -- 172.21.15.60:0/388138403 learned_addr learned my addr 172.21.15.60:0/388138403 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6d800640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c680b1680 0x7f7c68152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:12.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6ce00640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c680d0670 0x7f7c680b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 -- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c680b1680 msgr2=0x7f7c68152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c680b1680 0x7f7c68152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 -- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c680d0670 msgr2=0x7f7c680b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c680d0670 0x7f7c680b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 -- 172.21.15.60:0/388138403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c680b6650 con 0x7f7c6815bd30 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6ce00640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c680d0670 0x7f7c680b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6d800640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c680b1680 0x7f7c68152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.041+0000 7f7c6e200640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6815bd30 0x7f7c6814c110 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000eaa0 tx=0x7f7c6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.042+0000 7f7c65600640 1 -- 172.21.15.60:0/388138403 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000cdb0 con 0x7f7c6815bd30 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.042+0000 7f7c65600640 1 -- 172.21.15.60:0/388138403 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c60004c10 con 0x7f7c6815bd30 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.042+0000 7f7c65600640 1 -- 172.21.15.60:0/388138403 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60005020 con 0x7f7c6815bd30 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.042+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c680b6930 con 0x7f7c6815bd30 2024-09-16T15:48:12.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.042+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c680b6ef0 con 0x7f7c6815bd30 2024-09-16T15:48:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.043+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c680b2820 con 0x7f7c6815bd30 2024-09-16T15:48:12.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.044+0000 7f7c65600640 1 -- 172.21.15.60:0/388138403 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f7c600051c0 con 0x7f7c6815bd30 2024-09-16T15:48:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.045+0000 7f7c65600640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c48079270 0x7f7c4807b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.045+0000 7f7c65600640 1 -- 172.21.15.60:0/388138403 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f7c60099ec0 con 0x7f7c6815bd30 2024-09-16T15:48:12.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.045+0000 7f7c6d800640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c48079270 0x7f7c4807b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:12.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.046+0000 7f7c6d800640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c48079270 0x7f7c4807b730 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0016a0 tx=0x7f7c5c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:12.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.049+0000 7f7c65600640 1 -- 172.21.15.60:0/388138403 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7c60016030 con 0x7f7c6815bd30 2024-09-16T15:48:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.287+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7c680cde20 con 0x7f7c6815bd30 2024-09-16T15:48:12.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.288+0000 7f7c65600640 1 -- 172.21.15.60:0/388138403 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v122) ==== 53+0+73 (secure 0 0 0) 0x7f7c60063cd0 con 0x7f7c6815bd30 2024-09-16T15:48:12.290 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c48079270 msgr2=0x7f7c4807b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c48079270 0x7f7c4807b730 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0016a0 tx=0x7f7c5c009040 comp rx=0 tx=0).stop 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6815bd30 msgr2=0x7f7c6814c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6815bd30 0x7f7c6814c110 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000eaa0 tx=0x7f7c6000ef60 comp rx=0 tx=0).stop 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 shutdown_connections 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7c48079270 0x7f7c4807b730 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c6815bd30 0x7f7c6814c110 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7c680d0670 0x7f7c680b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 --2- 172.21.15.60:0/388138403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7c680b1680 0x7f7c68152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 >> 172.21.15.60:0/388138403 conn(0x7f7c680d1c40 msgr2=0x7f7c680b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 shutdown_connections 2024-09-16T15:48:12.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:12.290+0000 7f7c6ec00640 1 -- 172.21.15.60:0/388138403 wait complete. 2024-09-16T15:48:12.309 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:48:12.309 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:48:12.309 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-16T15:48:12.309 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:48:12.309 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:48:12.309 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:48:12.309 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:48:12.309 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:48:12.310 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:48:12.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:12.310 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:12.349 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42157sec preferred_lft 42157sec 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:48:12.350 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::b890:18ff:febb:5e7f/64 scope link 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:29:82:51:00:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::429:82ff:fe51:c4/64 scope link 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether f6:b7:6d:58:65:de brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::f4b7:6dff:fe58:65de/64 scope link 2024-09-16T15:48:12.351 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:12.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:12.352 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:48:12.445 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-16T15:48:12.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:12.448 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-16T15:48:12.485 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-16T15:48:12.485 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:48:12.485 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:48:12.485 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:48:12.485 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:48:12.486 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:48:12.486 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-16T15:48:12.486 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:48:12.486 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:48:12.486 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:48:12.486 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-16T15:48:12.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:48:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:48:12.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:48:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:48:12.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:12.682 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:48:12.717 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:48:12.717 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:48:12.717 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:48:12.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:12.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-16T15:48:12.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:48:12.813 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:48:12.847 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:48:12.848 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:48:12.848 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:48:12.922 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-16T15:48:12.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:12.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:48:13.049 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:48:13.050 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:48:13.076 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-16T15:48:13.076 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-16T15:48:13.077 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-09-16T15:48:13.077 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 1@b577f92e-7c09-4d32-a34c-b870818f4d85.cephfs=/ 2024-09-16T15:48:13.077 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.60:3300/172.21.15.100:3300/172.21.15.100:3301" will pass to kernel 2024-09-16T15:48:13.078 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:48:13.078 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:48:13.078 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:48:13.078 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:48:13.078 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:48:13.081 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:48:13.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:13.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:48:13.136 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:48:13.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:48:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:48:13.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-16T15:48:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:48:13.225 INFO:teuthology.orchestra.run.smithi100.stdout:client7697 2024-09-16T15:48:13.226 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:48:13.226 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:13.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:48:13.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.437+0000 7f94cea00640 1 -- 172.21.15.60:0/858105353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c8159aa0 msgr2=0x7f94c80741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.437+0000 7f94cea00640 1 --2- 172.21.15.60:0/858105353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c8159aa0 0x7f94c80741a0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f94b8005fd0 tx=0x7f94b802a670 comp rx=0 tx=0).stop 2024-09-16T15:48:13.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.437+0000 7f94cea00640 1 -- 172.21.15.60:0/858105353 shutdown_connections 2024-09-16T15:48:13.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.437+0000 7f94cea00640 1 --2- 172.21.15.60:0/858105353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c8159aa0 0x7f94c80741a0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.437+0000 7f94cea00640 1 -- 172.21.15.60:0/858105353 >> 172.21.15.60:0/858105353 conn(0x7f94c80b18e0 msgr2=0x7f94c80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.437+0000 7f94cea00640 1 -- 172.21.15.60:0/858105353 shutdown_connections 2024-09-16T15:48:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.437+0000 7f94cea00640 1 -- 172.21.15.60:0/858105353 wait complete. 2024-09-16T15:48:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.438+0000 7f94cea00640 1 Processor -- start 2024-09-16T15:48:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.438+0000 7f94cea00640 1 -- start start 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.438+0000 7f94cea00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94c806f220 0x7f94c809d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cea00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94c8159aa0 0x7f94c8157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c815cff0 0x7f94c8157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cea00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f94c806f6b0 con 0x7f94c815cff0 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cea00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94c80ccc50 con 0x7f94c806f220 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cea00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f94c806f830 con 0x7f94c8159aa0 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c815cff0 0x7f94c8157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c815cff0 0x7f94c8157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53272/0 (socket says 172.21.15.60:53272) 2024-09-16T15:48:13.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 -- 172.21.15.60:0/375368334 learned_addr learned my addr 172.21.15.60:0/375368334 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94c7e00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94c806f220 0x7f94c809d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94c7400640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94c8159aa0 0x7f94c8157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 -- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94c8159aa0 msgr2=0x7f94c8157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94c8159aa0 0x7f94c8157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 -- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94c806f220 msgr2=0x7f94c809d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94c806f220 0x7f94c809d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 -- 172.21.15.60:0/375368334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c8156ec0 con 0x7f94c815cff0 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94c7e00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94c806f220 0x7f94c809d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.439+0000 7f94cca00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c815cff0 0x7f94c8157f80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f94c000c7a0 tx=0x7f94c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.440+0000 7f94bfe00640 1 -- 172.21.15.60:0/375368334 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0003b70 con 0x7f94c815cff0 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.440+0000 7f94bfe00640 1 -- 172.21.15.60:0/375368334 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94c0003d10 con 0x7f94c815cff0 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.440+0000 7f94bfe00640 1 -- 172.21.15.60:0/375368334 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c000aa50 con 0x7f94c815cff0 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.440+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c8156100 con 0x7f94c815cff0 2024-09-16T15:48:13.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.440+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81566c0 con 0x7f94c815cff0 2024-09-16T15:48:13.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.441+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c8069360 con 0x7f94c815cff0 2024-09-16T15:48:13.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.442+0000 7f94bfe00640 1 -- 172.21.15.60:0/375368334 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f94c000abf0 con 0x7f94c815cff0 2024-09-16T15:48:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.443+0000 7f94bfe00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f94a8079270 0x7f94a807b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.443+0000 7f94bfe00640 1 -- 172.21.15.60:0/375368334 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f94c0099280 con 0x7f94c815cff0 2024-09-16T15:48:13.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.443+0000 7f94c7e00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f94a8079270 0x7f94a807b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.444+0000 7f94c7e00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f94a8079270 0x7f94a807b730 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f94b8033000 tx=0x7f94b8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:13.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.447+0000 7f94bfe00640 1 -- 172.21.15.60:0/375368334 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f94c0063120 con 0x7f94c815cff0 2024-09-16T15:48:13.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:13.684+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f94c8155cd0 con 0x7f94c815cff0 2024-09-16T15:48:14.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.288+0000 7f94bfe00640 1 -- 172.21.15.60:0/375368334 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v123) ==== 136+0+0 (secure 0 0 0) 0x7f94c0062a20 con 0x7f94c815cff0 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f94a8079270 msgr2=0x7f94a807b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f94a8079270 0x7f94a807b730 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f94b8033000 tx=0x7f94b8004730 comp rx=0 tx=0).stop 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c815cff0 msgr2=0x7f94c8157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c815cff0 0x7f94c8157f80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f94c000c7a0 tx=0x7f94c000cc60 comp rx=0 tx=0).stop 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 shutdown_connections 2024-09-16T15:48:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f94a8079270 0x7f94a807b730 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94c815cff0 0x7f94c8157f80 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94c8159aa0 0x7f94c8157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 --2- 172.21.15.60:0/375368334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94c806f220 0x7f94c809d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 >> 172.21.15.60:0/375368334 conn(0x7f94c80b18e0 msgr2=0x7f94c80ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 shutdown_connections 2024-09-16T15:48:14.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.291+0000 7f94cea00640 1 -- 172.21.15.60:0/375368334 wait complete. 2024-09-16T15:48:14.311 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.477+0000 7f9147200640 1 -- 172.21.15.60:0/2872163042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140072df0 msgr2=0x7f91400732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.477+0000 7f9147200640 1 --2- 172.21.15.60:0/2872163042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140072df0 0x7f91400732a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9138005fd0 tx=0x7f913802aa10 comp rx=0 tx=0).stop 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.477+0000 7f9147200640 1 -- 172.21.15.60:0/2872163042 shutdown_connections 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.477+0000 7f9147200640 1 --2- 172.21.15.60:0/2872163042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140072df0 0x7f91400732a0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.477+0000 7f9147200640 1 --2- 172.21.15.60:0/2872163042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9140161550 0x7f914006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.477+0000 7f9147200640 1 -- 172.21.15.60:0/2872163042 >> 172.21.15.60:0/2872163042 conn(0x7f91400b1950 msgr2=0x7f91400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.478+0000 7f9147200640 1 -- 172.21.15.60:0/2872163042 shutdown_connections 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.478+0000 7f9147200640 1 -- 172.21.15.60:0/2872163042 wait complete. 2024-09-16T15:48:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.479+0000 7f9147200640 1 Processor -- start 2024-09-16T15:48:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.479+0000 7f9147200640 1 -- start start 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9147200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9140072df0 0x7f91400c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9147200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91400af3d0 0x7f91400cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9147200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140161550 0x7f9140151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9147200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f91400737e0 con 0x7f9140161550 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9147200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f914006fdb0 con 0x7f9140072df0 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9147200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9140073960 con 0x7f91400af3d0 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9145200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140161550 0x7f9140151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9145200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140161550 0x7f9140151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53294/0 (socket says 172.21.15.60:53294) 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.480+0000 7f9145200640 1 -- 172.21.15.60:0/983202843 learned_addr learned my addr 172.21.15.60:0/983202843 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f913fe00640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9140072df0 0x7f91400c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f9137400640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91400af3d0 0x7f91400cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f9145200640 1 -- 172.21.15.60:0/983202843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91400af3d0 msgr2=0x7f91400cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f9145200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91400af3d0 0x7f91400cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f9145200640 1 -- 172.21.15.60:0/983202843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9140072df0 msgr2=0x7f91400c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f9145200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9140072df0 0x7f91400c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f9145200640 1 -- 172.21.15.60:0/983202843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91400cc150 con 0x7f9140161550 2024-09-16T15:48:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f9145200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140161550 0x7f9140151c60 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9138006010 tx=0x7f9138003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.481+0000 7f913cc00640 1 -- 172.21.15.60:0/983202843 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138047070 con 0x7f9140161550 2024-09-16T15:48:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.482+0000 7f913cc00640 1 -- 172.21.15.60:0/983202843 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9138004250 con 0x7f9140161550 2024-09-16T15:48:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.482+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91400cc430 con 0x7f9140161550 2024-09-16T15:48:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.482+0000 7f913cc00640 1 -- 172.21.15.60:0/983202843 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138033050 con 0x7f9140161550 2024-09-16T15:48:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.482+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91400cc9f0 con 0x7f9140161550 2024-09-16T15:48:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.483+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91400b3030 con 0x7f9140161550 2024-09-16T15:48:14.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.484+0000 7f913cc00640 1 -- 172.21.15.60:0/983202843 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f913804a070 con 0x7f9140161550 2024-09-16T15:48:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.485+0000 7f913cc00640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f911c079270 0x7f911c07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.485+0000 7f913cc00640 1 -- 172.21.15.60:0/983202843 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f91380ba620 con 0x7f9140161550 2024-09-16T15:48:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.485+0000 7f913fe00640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f911c079270 0x7f911c07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.486+0000 7f913fe00640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f911c079270 0x7f911c07b730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9130005fd0 tx=0x7f9130005850 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.489+0000 7f913cc00640 1 -- 172.21.15.60:0/983202843 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f91400b3030 con 0x7f9140161550 2024-09-16T15:48:14.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.708+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f9140073ec0 con 0x7f911c079270 2024-09-16T15:48:14.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.710+0000 7f913cc00640 1 -- 172.21.15.60:0/983202843 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f9140073ec0 con 0x7f911c079270 2024-09-16T15:48:14.711 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-09-16T15:48:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-16T15:48:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "cephfs" 2024-09-16T15:48:14.711 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-16T15:48:14.711 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-09-16T15:48:14.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f911c079270 msgr2=0x7f911c07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f911c079270 0x7f911c07b730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9130005fd0 tx=0x7f9130005850 comp rx=0 tx=0).stop 2024-09-16T15:48:14.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140161550 msgr2=0x7f9140151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140161550 0x7f9140151c60 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9138006010 tx=0x7f9138003f10 comp rx=0 tx=0).stop 2024-09-16T15:48:14.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 shutdown_connections 2024-09-16T15:48:14.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f911c079270 0x7f911c07b730 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9140161550 0x7f9140151c60 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91400af3d0 0x7f91400cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 --2- 172.21.15.60:0/983202843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9140072df0 0x7f91400c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 >> 172.21.15.60:0/983202843 conn(0x7f91400b1950 msgr2=0x7f914006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 shutdown_connections 2024-09-16T15:48:14.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.712+0000 7f9147200640 1 -- 172.21.15.60:0/983202843 wait complete. 2024-09-16T15:48:14.730 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:48:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.893+0000 7fb5c7000640 1 -- 172.21.15.60:0/4048101436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c0145750 msgr2=0x7fb5c0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.893+0000 7fb5c7000640 1 --2- 172.21.15.60:0/4048101436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c0145750 0x7fb5c0073440 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00af750 tx=0x7fb5b002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.893+0000 7fb5c7000640 1 -- 172.21.15.60:0/4048101436 shutdown_connections 2024-09-16T15:48:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.893+0000 7fb5c7000640 1 --2- 172.21.15.60:0/4048101436 >> v2:172.21.15.100:3300/0 conn(0x7fb5c006efa0 0x7fb5c00cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.893+0000 7fb5c7000640 1 --2- 172.21.15.60:0/4048101436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c0145750 0x7fb5c0073440 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.893+0000 7fb5c7000640 1 -- 172.21.15.60:0/4048101436 >> 172.21.15.60:0/4048101436 conn(0x7fb5c00b1840 msgr2=0x7fb5c00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.893+0000 7fb5c7000640 1 -- 172.21.15.60:0/4048101436 shutdown_connections 2024-09-16T15:48:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.893+0000 7fb5c7000640 1 -- 172.21.15.60:0/4048101436 wait complete. 2024-09-16T15:48:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.894+0000 7fb5c7000640 1 Processor -- start 2024-09-16T15:48:14.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.895+0000 7fb5c7000640 1 -- start start 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.895+0000 7fb5c7000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c006efa0 0x7fb5c00ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.895+0000 7fb5c7000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c00ccb00 0x7fb5c00ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.895+0000 7fb5c7000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c0145750 0x7fb5c00a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.895+0000 7fb5c7000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00ced40 con 0x7fb5c00ccb00 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.895+0000 7fb5c7000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c00cebc0 con 0x7fb5c0145750 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.895+0000 7fb5c7000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c00ceec0 con 0x7fb5c006efa0 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c00ccb00 0x7fb5c00ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c00ccb00 0x7fb5c00ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53316/0 (socket says 172.21.15.60:53316) 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 -- 172.21.15.60:0/3302227326 learned_addr learned my addr 172.21.15.60:0/3302227326 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:14.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5c5000640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c0145750 0x7fb5c00a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bfe00640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c006efa0 0x7fb5c00ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 -- 172.21.15.60:0/3302227326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c006efa0 msgr2=0x7fb5c00ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c006efa0 0x7fb5c00ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 -- 172.21.15.60:0/3302227326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c0145750 msgr2=0x7fb5c00a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c0145750 0x7fb5c00a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 -- 172.21.15.60:0/3302227326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c00ad7b0 con 0x7fb5c00ccb00 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.896+0000 7fb5bf400640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c00ccb00 0x7fb5c00ab710 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00d770 tx=0x7fb5ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.897+0000 7fb5b7e00640 1 -- 172.21.15.60:0/3302227326 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac017070 con 0x7fb5c00ccb00 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.897+0000 7fb5b7e00640 1 -- 172.21.15.60:0/3302227326 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5ac009d20 con 0x7fb5c00ccb00 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.897+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c00ada90 con 0x7fb5c00ccb00 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.897+0000 7fb5b7e00640 1 -- 172.21.15.60:0/3302227326 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac005280 con 0x7fb5c00ccb00 2024-09-16T15:48:14.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.897+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c00ae050 con 0x7fb5c00ccb00 2024-09-16T15:48:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.898+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c00abc50 con 0x7fb5c00ccb00 2024-09-16T15:48:14.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.899+0000 7fb5b7e00640 1 -- 172.21.15.60:0/3302227326 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fb5ac005420 con 0x7fb5c00ccb00 2024-09-16T15:48:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.900+0000 7fb5b7e00640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5a0079270 0x7fb5a007b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.900+0000 7fb5b7e00640 1 -- 172.21.15.60:0/3302227326 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fb5ac099640 con 0x7fb5c00ccb00 2024-09-16T15:48:14.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.900+0000 7fb5bfe00640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5a0079270 0x7fb5a007b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.901+0000 7fb5bfe00640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5a0079270 0x7fb5a007b730 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0033000 tx=0x7fb5b0004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:14.904+0000 7fb5b7e00640 1 -- 172.21.15.60:0/3302227326 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5ac0634e0 con 0x7fb5c00ccb00 2024-09-16T15:48:15.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.105+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb5c00b0a50 con 0x7fb5c00ccb00 2024-09-16T15:48:15.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.106+0000 7fb5b7e00640 1 -- 172.21.15.60:0/3302227326 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7fb5ac062de0 con 0x7fb5c00ccb00 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5a0079270 msgr2=0x7fb5a007b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5a0079270 0x7fb5a007b730 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0033000 tx=0x7fb5b0004730 comp rx=0 tx=0).stop 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c00ccb00 msgr2=0x7fb5c00ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c00ccb00 0x7fb5c00ab710 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00d770 tx=0x7fb5ac00dc30 comp rx=0 tx=0).stop 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 shutdown_connections 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5a0079270 0x7fb5a007b730 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5c0145750 0x7fb5c00a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5c00ccb00 0x7fb5c00ab710 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 --2- 172.21.15.60:0/3302227326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5c006efa0 0x7fb5c00ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 >> 172.21.15.60:0/3302227326 conn(0x7fb5c00b1840 msgr2=0x7fb5c00a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.108+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 shutdown_connections 2024-09-16T15:48:15.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.109+0000 7fb5c7000640 1 -- 172.21.15.60:0/3302227326 wait complete. 2024-09-16T15:48:15.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:15.126 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp91350/subvol79140) 2024-09-16T15:48:15.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:15.188 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-16T15:48:15.225 INFO:teuthology.orchestra.run.smithi060.stdout:cephfs_data 2024-09-16T15:48:15.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:15.225 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp91350/subvol79140) 2024-09-16T15:48:15.328 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:48:15.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp91350/subvol79140/.meta 2024-09-16T15:48:15.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol79140 subvol_grp91350 2024-09-16T15:48:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.590+0000 7fea2c400640 1 -- 172.21.15.60:0/3453540607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 msgr2=0x7fea2406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.590+0000 7fea2c400640 1 --2- 172.21.15.60:0/3453540607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 0x7fea2406f6b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fea24070850 tx=0x7fea1c02aac0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.591+0000 7fea2c400640 1 -- 172.21.15.60:0/3453540607 shutdown_connections 2024-09-16T15:48:15.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.591+0000 7fea2c400640 1 --2- 172.21.15.60:0/3453540607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 0x7fea2406f6b0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.591+0000 7fea2c400640 1 --2- 172.21.15.60:0/3453540607 >> v2:172.21.15.100:3300/0 conn(0x7fea2415cff0 0x7fea2406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.591+0000 7fea2c400640 1 -- 172.21.15.60:0/3453540607 >> 172.21.15.60:0/3453540607 conn(0x7fea240b18e0 msgr2=0x7fea240b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.591+0000 7fea2c400640 1 -- 172.21.15.60:0/3453540607 shutdown_connections 2024-09-16T15:48:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.591+0000 7fea2c400640 1 -- 172.21.15.60:0/3453540607 wait complete. 2024-09-16T15:48:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.592+0000 7fea2c400640 1 Processor -- start 2024-09-16T15:48:15.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.592+0000 7fea2c400640 1 -- start start 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.593+0000 7fea2c400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 0x7fea241f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.593+0000 7fea2c400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea24159aa0 0x7fea241585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.593+0000 7fea2c400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea2415cff0 0x7fea240ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.593+0000 7fea2c400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fea240cc520 con 0x7fea2406f220 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.593+0000 7fea2c400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea240cc3a0 con 0x7fea24159aa0 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.593+0000 7fea2c400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fea240cc6a0 con 0x7fea2415cff0 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.593+0000 7fea29a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 0x7fea241f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.593+0000 7fea29a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 0x7fea241f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53344/0 (socket says 172.21.15.60:53344) 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea29a00640 1 -- 172.21.15.60:0/93296013 learned_addr learned my addr 172.21.15.60:0/93296013 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea2a400640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea2415cff0 0x7fea240ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:15.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea29000640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea24159aa0 0x7fea241585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea29a00640 1 -- 172.21.15.60:0/93296013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea2415cff0 msgr2=0x7fea240ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea29a00640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea2415cff0 0x7fea240ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea29a00640 1 -- 172.21.15.60:0/93296013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea24159aa0 msgr2=0x7fea241585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea29a00640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea24159aa0 0x7fea241585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea29a00640 1 -- 172.21.15.60:0/93296013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea24157520 con 0x7fea2406f220 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea29a00640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 0x7fea241f5460 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c480 tx=0x7fea1800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea21600640 1 -- 172.21.15.60:0/93296013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea180052d0 con 0x7fea2406f220 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea21600640 1 -- 172.21.15.60:0/93296013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea18005470 con 0x7fea2406f220 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea21600640 1 -- 172.21.15.60:0/93296013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18012870 con 0x7fea2406f220 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.594+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea24157800 con 0x7fea2406f220 2024-09-16T15:48:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.595+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea241580a0 con 0x7fea2406f220 2024-09-16T15:48:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.596+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea241582a0 con 0x7fea2406f220 2024-09-16T15:48:15.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.597+0000 7fea21600640 1 -- 172.21.15.60:0/93296013 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fea180035f0 con 0x7fea2406f220 2024-09-16T15:48:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.598+0000 7fea21600640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fea04079270 0x7fea0407b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.598+0000 7fea21600640 1 -- 172.21.15.60:0/93296013 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fea18098ea0 con 0x7fea2406f220 2024-09-16T15:48:15.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.598+0000 7fea29000640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fea04079270 0x7fea0407b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:15.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.599+0000 7fea29000640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fea04079270 0x7fea0407b730 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea24159400 tx=0x7fea10005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:15.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.602+0000 7fea21600640 1 -- 172.21.15.60:0/93296013 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fea241582a0 con 0x7fea2406f220 2024-09-16T15:48:15.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.826+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol79140", "group_name": "subvol_grp91350", "target": ["mon-mgr", ""]}) -- 0x7fea24070980 con 0x7fea04079270 2024-09-16T15:48:15.833 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:15.832+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:15.833 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:15.832+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:15.833 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:15.832+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:15.833 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:15.832+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:15.833 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:15.832+0000 7fcb85a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:15.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.870+0000 7fea21600640 1 -- 172.21.15.60:0/93296013 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fea24070980 con 0x7fea04079270 2024-09-16T15:48:15.872 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/subvol_grp91350/subvol79140 2024-09-16T15:48:15.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fea04079270 msgr2=0x7fea0407b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fea04079270 0x7fea0407b730 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea24159400 tx=0x7fea10005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:15.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 msgr2=0x7fea241f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 0x7fea241f5460 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c480 tx=0x7fea1800c940 comp rx=0 tx=0).stop 2024-09-16T15:48:15.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 shutdown_connections 2024-09-16T15:48:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fea04079270 0x7fea0407b730 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea2415cff0 0x7fea240ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea24159aa0 0x7fea241585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 --2- 172.21.15.60:0/93296013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea2406f220 0x7fea241f5460 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 >> 172.21.15.60:0/93296013 conn(0x7fea240b18e0 msgr2=0x7fea24159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 shutdown_connections 2024-09-16T15:48:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:15.873+0000 7fea2c400640 1 -- 172.21.15.60:0/93296013 wait complete. 2024-09-16T15:48:15.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol79140 subvol_grp91350 2024-09-16T15:48:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.054+0000 7fec69200640 1 -- 172.21.15.60:0/1415595535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6406e6f0 msgr2=0x7fec6406eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.054+0000 7fec69200640 1 --2- 172.21.15.60:0/1415595535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6406e6f0 0x7fec6406eba0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fec5c005fd0 tx=0x7fec5c02aa50 comp rx=0 tx=0).stop 2024-09-16T15:48:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.054+0000 7fec69200640 1 -- 172.21.15.60:0/1415595535 shutdown_connections 2024-09-16T15:48:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.054+0000 7fec69200640 1 --2- 172.21.15.60:0/1415595535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6406e6f0 0x7fec6406eba0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.054+0000 7fec69200640 1 --2- 172.21.15.60:0/1415595535 >> v2:172.21.15.100:3300/0 conn(0x7fec64073de0 0x7fec64074240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.054+0000 7fec69200640 1 -- 172.21.15.60:0/1415595535 >> 172.21.15.60:0/1415595535 conn(0x7fec640b18e0 msgr2=0x7fec640b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.054+0000 7fec69200640 1 -- 172.21.15.60:0/1415595535 shutdown_connections 2024-09-16T15:48:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.054+0000 7fec69200640 1 -- 172.21.15.60:0/1415595535 wait complete. 2024-09-16T15:48:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.055+0000 7fec69200640 1 Processor -- start 2024-09-16T15:48:16.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.056+0000 7fec69200640 1 -- start start 2024-09-16T15:48:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.056+0000 7fec69200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6406e6f0 0x7fec640b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.056+0000 7fec69200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec64073de0 0x7fec640cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.056+0000 7fec69200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415f9a0 0x7fec640d0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.056+0000 7fec69200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fec640ccc20 con 0x7fec64073de0 2024-09-16T15:48:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.056+0000 7fec69200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec640cd5e0 con 0x7fec6415f9a0 2024-09-16T15:48:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.056+0000 7fec69200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fec6406f0e0 con 0x7fec6406e6f0 2024-09-16T15:48:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec64073de0 0x7fec640cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec64073de0 0x7fec640cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53382/0 (socket says 172.21.15.60:53382) 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 -- 172.21.15.60:0/404948850 learned_addr learned my addr 172.21.15.60:0/404948850 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec62800640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415f9a0 0x7fec640d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec61e00640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6406e6f0 0x7fec640b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 -- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6406e6f0 msgr2=0x7fec640b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6406e6f0 0x7fec640b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 -- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415f9a0 msgr2=0x7fec640d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415f9a0 0x7fec640d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 -- 172.21.15.60:0/404948850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec640b45e0 con 0x7fec64073de0 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec62800640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415f9a0 0x7fec640d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.057+0000 7fec59400640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec64073de0 0x7fec640cfcc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00e9c0 tx=0x7fec4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.058+0000 7fec5aa00640 1 -- 172.21.15.60:0/404948850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00cd80 con 0x7fec64073de0 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.058+0000 7fec5aa00640 1 -- 172.21.15.60:0/404948850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec4c004020 con 0x7fec64073de0 2024-09-16T15:48:16.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.058+0000 7fec5aa00640 1 -- 172.21.15.60:0/404948850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c004450 con 0x7fec64073de0 2024-09-16T15:48:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.058+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec640b3820 con 0x7fec64073de0 2024-09-16T15:48:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.058+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec640b3de0 con 0x7fec64073de0 2024-09-16T15:48:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.059+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec64069360 con 0x7fec64073de0 2024-09-16T15:48:16.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.060+0000 7fec5aa00640 1 -- 172.21.15.60:0/404948850 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7fec4c0035f0 con 0x7fec64073de0 2024-09-16T15:48:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.061+0000 7fec5aa00640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec40079270 0x7fec4007b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.061+0000 7fec5aa00640 1 -- 172.21.15.60:0/404948850 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fec4c099f80 con 0x7fec64073de0 2024-09-16T15:48:16.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.061+0000 7fec61e00640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec40079270 0x7fec4007b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.062+0000 7fec61e00640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec40079270 0x7fec4007b730 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fec540015e0 tx=0x7fec54009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:16.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.065+0000 7fec5aa00640 1 -- 172.21.15.60:0/404948850 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fec4c063e20 con 0x7fec64073de0 2024-09-16T15:48:16.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.291+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol79140", "group_name": "subvol_grp91350", "target": ["mon-mgr", ""]}) -- 0x7fec640cd7b0 con 0x7fec40079270 2024-09-16T15:48:16.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.296+0000 7fec5aa00640 1 -- 172.21.15.60:0/404948850 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fec640cd7b0 con 0x7fec40079270 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi060.stdout:/volumes/subvol_grp91350/subvol79140 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec40079270 msgr2=0x7fec4007b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec40079270 0x7fec4007b730 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fec540015e0 tx=0x7fec54009040 comp rx=0 tx=0).stop 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec64073de0 msgr2=0x7fec640cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec64073de0 0x7fec640cfcc0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00e9c0 tx=0x7fec4c00ee80 comp rx=0 tx=0).stop 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 shutdown_connections 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fec40079270 0x7fec4007b730 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6415f9a0 0x7fec640d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec64073de0 0x7fec640cfcc0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 --2- 172.21.15.60:0/404948850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6406e6f0 0x7fec640b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 >> 172.21.15.60:0/404948850 conn(0x7fec640b18e0 msgr2=0x7fec6415de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 shutdown_connections 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.298+0000 7fec69200640 1 -- 172.21.15.60:0/404948850 wait complete. 2024-09-16T15:48:16.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:16.315 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/656c13b9af1a8e55f3a204ad853fd5e0.meta) 2024-09-16T15:48:16.375 INFO:teuthology.orchestra.run.smithi060.stdout:[GLOBAL] 2024-09-16T15:48:16.375 INFO:teuthology.orchestra.run.smithi060.stdout:version = 1 2024-09-16T15:48:16.375 INFO:teuthology.orchestra.run.smithi060.stdout:type = subvolume 2024-09-16T15:48:16.375 INFO:teuthology.orchestra.run.smithi060.stdout:path = /volumes/subvol_grp91350/subvol79140 2024-09-16T15:48:16.375 INFO:teuthology.orchestra.run.smithi060.stdout:state = complete 2024-09-16T15:48:16.375 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:16.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol79140 subvol_grp91350 2024-09-16T15:48:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.541+0000 7f09a7a00640 1 -- 172.21.15.60:0/3731451790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 msgr2=0x7f09a006f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.541+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3731451790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 0x7f09a006f6b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f09a00b2680 tx=0x7f099802aac0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.541+0000 7f09a7a00640 1 -- 172.21.15.60:0/3731451790 shutdown_connections 2024-09-16T15:48:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.541+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3731451790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 0x7f09a006f6b0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.541+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3731451790 >> v2:172.21.15.100:3300/0 conn(0x7f09a0159aa0 0x7f09a00741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.541+0000 7f09a7a00640 1 -- 172.21.15.60:0/3731451790 >> 172.21.15.60:0/3731451790 conn(0x7f09a00b18e0 msgr2=0x7f09a00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.541+0000 7f09a7a00640 1 -- 172.21.15.60:0/3731451790 shutdown_connections 2024-09-16T15:48:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.541+0000 7f09a7a00640 1 -- 172.21.15.60:0/3731451790 wait complete. 2024-09-16T15:48:16.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.543+0000 7f09a7a00640 1 Processor -- start 2024-09-16T15:48:16.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.543+0000 7f09a7a00640 1 -- start start 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a7a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 0x7f09a01445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a7a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0159aa0 0x7f09a0156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a7a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09a015cff0 0x7f09a00cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a7a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00cc520 con 0x7f09a006f220 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a7a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00cc3a0 con 0x7f09a0159aa0 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a7a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f09a00cc6a0 con 0x7f09a015cff0 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 0x7f09a01445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a5000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 0x7f09a01445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53402/0 (socket says 172.21.15.60:53402) 2024-09-16T15:48:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.544+0000 7f09a5000640 1 -- 172.21.15.60:0/3230999825 learned_addr learned my addr 172.21.15.60:0/3230999825 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f09a5a00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09a015cff0 0x7f09a00cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f0997e00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0159aa0 0x7f09a0156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f09a5000640 1 -- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09a015cff0 msgr2=0x7f09a00cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f09a5000640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09a015cff0 0x7f09a00cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f09a5000640 1 -- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0159aa0 msgr2=0x7f09a0156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f09a5000640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0159aa0 0x7f09a0156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f09a5000640 1 -- 172.21.15.60:0/3230999825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a0158b20 con 0x7f09a006f220 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f0997e00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0159aa0 0x7f09a0156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:16.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f09a5000640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 0x7f09a01445c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f099000e9f0 tx=0x7f099000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:16.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f099d600640 1 -- 172.21.15.60:0/3230999825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099000cd80 con 0x7f09a006f220 2024-09-16T15:48:16.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f099d600640 1 -- 172.21.15.60:0/3230999825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0990010430 con 0x7f09a006f220 2024-09-16T15:48:16.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.545+0000 7f099d600640 1 -- 172.21.15.60:0/3230999825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990010840 con 0x7f09a006f220 2024-09-16T15:48:16.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.546+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a01501a0 con 0x7f09a006f220 2024-09-16T15:48:16.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.546+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a0159080 con 0x7f09a006f220 2024-09-16T15:48:16.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.547+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a00cf680 con 0x7f09a006f220 2024-09-16T15:48:16.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.548+0000 7f099d600640 1 -- 172.21.15.60:0/3230999825 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f09900109e0 con 0x7f09a006f220 2024-09-16T15:48:16.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.550+0000 7f099d600640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f097c079270 0x7f097c07b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.550+0000 7f099d600640 1 -- 172.21.15.60:0/3230999825 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f099009a7f0 con 0x7f09a006f220 2024-09-16T15:48:16.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.550+0000 7f0997e00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f097c079270 0x7f097c07b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.551+0000 7f0997e00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f097c079270 0x7f097c07b730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0988007fd0 tx=0x7f0988009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:16.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.553+0000 7f099d600640 1 -- 172.21.15.60:0/3230999825 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f09a0158b20 con 0x7f09a006f220 2024-09-16T15:48:16.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.777+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol79140", "group_name": "subvol_grp91350", "target": ["mon-mgr", ""]}) -- 0x7f09a0070980 con 0x7f097c079270 2024-09-16T15:48:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.803+0000 7f099d600640 1 -- 172.21.15.60:0/3230999825 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f09a0070980 con 0x7f097c079270 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.805+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f097c079270 msgr2=0x7f097c07b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.805+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f097c079270 0x7f097c07b730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0988007fd0 tx=0x7f0988009040 comp rx=0 tx=0).stop 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.805+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 msgr2=0x7f09a01445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.805+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 0x7f09a01445c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f099000e9f0 tx=0x7f099000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.806+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 shutdown_connections 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.806+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f097c079270 0x7f097c07b730 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.806+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09a015cff0 0x7f09a00cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.806+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09a0159aa0 0x7f09a0156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.806+0000 7f09a7a00640 1 --2- 172.21.15.60:0/3230999825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09a006f220 0x7f09a01445c0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.806+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 >> 172.21.15.60:0/3230999825 conn(0x7f09a00b18e0 msgr2=0x7f09a0154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:16.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.806+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 shutdown_connections 2024-09-16T15:48:16.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16.806+0000 7f09a7a00640 1 -- 172.21.15.60:0/3230999825 wait complete. 2024-09-16T15:48:16.809 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.808+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.809 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.808+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.809 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.808+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.810 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.808+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.810 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.808+0000 7fcb88c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.824 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:48:16.828 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.826+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.828 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.826+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.828 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.826+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.829 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.826+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.829 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-09-16T15:48:16.826+0000 7fcb89600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:16.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:48:16.913 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-16T15:48:16.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:48:16.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp91350 2024-09-16T15:48:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.085+0000 7f42d9a00640 1 -- 172.21.15.60:0/1069472146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d4145750 msgr2=0x7f42d4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.085+0000 7f42d9a00640 1 --2- 172.21.15.60:0/1069472146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d4145750 0x7f42d4073440 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009860 tx=0x7f42c402a510 comp rx=0 tx=0).stop 2024-09-16T15:48:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.085+0000 7f42d9a00640 1 -- 172.21.15.60:0/1069472146 shutdown_connections 2024-09-16T15:48:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.085+0000 7f42d9a00640 1 --2- 172.21.15.60:0/1069472146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d4145750 0x7f42d4073440 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.085+0000 7f42d9a00640 1 -- 172.21.15.60:0/1069472146 >> 172.21.15.60:0/1069472146 conn(0x7f42d40b1840 msgr2=0x7f42d40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.085+0000 7f42d9a00640 1 -- 172.21.15.60:0/1069472146 shutdown_connections 2024-09-16T15:48:17.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.085+0000 7f42d9a00640 1 -- 172.21.15.60:0/1069472146 wait complete. 2024-09-16T15:48:17.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.087+0000 7f42d9a00640 1 Processor -- start 2024-09-16T15:48:17.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.087+0000 7f42d9a00640 1 -- start start 2024-09-16T15:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d9a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d406efa0 0x7f42d4142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d9a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d40cccd0 0x7f42d41f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d9a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d4145750 0x7f42d41f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d9a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40ce6d0 con 0x7f42d406efa0 2024-09-16T15:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d9a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40ce550 con 0x7f42d4145750 2024-09-16T15:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d9a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f42d40ce850 con 0x7f42d40cccd0 2024-09-16T15:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d406efa0 0x7f42d4142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d406efa0 0x7f42d4142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53454/0 (socket says 172.21.15.60:53454) 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.088+0000 7f42d2a00640 1 -- 172.21.15.60:0/1982796742 learned_addr learned my addr 172.21.15.60:0/1982796742 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42ca000640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d40cccd0 0x7f42d41f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42d3400640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d4145750 0x7f42d41f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42d2a00640 1 -- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d40cccd0 msgr2=0x7f42d41f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42d2a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d40cccd0 0x7f42d41f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42d2a00640 1 -- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d4145750 msgr2=0x7f42d41f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42d2a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d4145750 0x7f42d41f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42d2a00640 1 -- 172.21.15.60:0/1982796742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d415b380 con 0x7f42d406efa0 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42d3400640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d4145750 0x7f42d41f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:17.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42d2a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d406efa0 0x7f42d4142eb0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009980 tx=0x7f42c4033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.089+0000 7f42cb400640 1 -- 172.21.15.60:0/1982796742 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4031b50 con 0x7f42d406efa0 2024-09-16T15:48:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.090+0000 7f42cb400640 1 -- 172.21.15.60:0/1982796742 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42c4031de0 con 0x7f42d406efa0 2024-09-16T15:48:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.090+0000 7f42cb400640 1 -- 172.21.15.60:0/1982796742 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4033330 con 0x7f42d406efa0 2024-09-16T15:48:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.090+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d415aae0 con 0x7f42d406efa0 2024-09-16T15:48:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.090+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d415b0a0 con 0x7f42d406efa0 2024-09-16T15:48:17.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.091+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d40ced00 con 0x7f42d406efa0 2024-09-16T15:48:17.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.093+0000 7f42cb400640 1 -- 172.21.15.60:0/1982796742 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 25) ==== 102205+0+0 (secure 0 0 0) 0x7f42c40334d0 con 0x7f42d406efa0 2024-09-16T15:48:17.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.094+0000 7f42cb400640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f42b0079270 0x7f42b007b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.094+0000 7f42cb400640 1 -- 172.21.15.60:0/1982796742 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f42c40baa90 con 0x7f42d406efa0 2024-09-16T15:48:17.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.094+0000 7f42ca000640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f42b0079270 0x7f42b007b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.095+0000 7f42ca000640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f42b0079270 0x7f42b007b730 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f42d4140660 tx=0x7f42bc009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:17.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.097+0000 7f42cb400640 1 -- 172.21.15.60:0/1982796742 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f42c4052080 con 0x7f42d406efa0 2024-09-16T15:48:17.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.300+0000 7f42cb400640 1 -- 172.21.15.60:0/1982796742 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 26) ==== 102256+0+0 (secure 0 0 0) 0x7f42c40805c0 con 0x7f42d406efa0 2024-09-16T15:48:17.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.324+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp91350", "target": ["mon-mgr", ""]}) -- 0x7f42d40aff70 con 0x7f42b0079270 2024-09-16T15:48:17.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.328+0000 7f42cb400640 1 -- 172.21.15.60:0/1982796742 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42d40aff70 con 0x7f42b0079270 2024-09-16T15:48:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f42b0079270 msgr2=0x7f42b007b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f42b0079270 0x7f42b007b730 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f42d4140660 tx=0x7f42bc009040 comp rx=0 tx=0).stop 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d406efa0 msgr2=0x7f42d4142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d406efa0 0x7f42d4142eb0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009980 tx=0x7f42c4033ba0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 shutdown_connections 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f42b0079270 0x7f42b007b730 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42d4145750 0x7f42d41f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42d40cccd0 0x7f42d41f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 --2- 172.21.15.60:0/1982796742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42d406efa0 0x7f42d4142eb0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 >> 172.21.15.60:0/1982796742 conn(0x7f42d40b1840 msgr2=0x7f42d40cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 shutdown_connections 2024-09-16T15:48:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.330+0000 7f42d9a00640 1 -- 172.21.15.60:0/1982796742 wait complete. 2024-09-16T15:48:17.346 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:48:17.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:48:17.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:48:17.524 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:48:17.524 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:48:17.524 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:48:17.524 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:17.560 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:48:17.560 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:17.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:48:17.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.835+0000 7fd4fa400640 1 -- 172.21.15.60:0/352555980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40725d0 msgr2=0x7fd4f40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.835+0000 7fd4fa400640 1 --2- 172.21.15.60:0/352555980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40725d0 0x7fd4f40b28f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009830 tx=0x7fd4e002a510 comp rx=0 tx=0).stop 2024-09-16T15:48:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.835+0000 7fd4fa400640 1 -- 172.21.15.60:0/352555980 shutdown_connections 2024-09-16T15:48:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.835+0000 7fd4fa400640 1 --2- 172.21.15.60:0/352555980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40725d0 0x7fd4f40b28f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.835+0000 7fd4fa400640 1 --2- 172.21.15.60:0/352555980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4f40d0540 0x7fd4f406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.835+0000 7fd4fa400640 1 -- 172.21.15.60:0/352555980 >> 172.21.15.60:0/352555980 conn(0x7fd4f40cfa20 msgr2=0x7fd4f40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.835+0000 7fd4fa400640 1 -- 172.21.15.60:0/352555980 shutdown_connections 2024-09-16T15:48:17.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.835+0000 7fd4fa400640 1 -- 172.21.15.60:0/352555980 wait complete. 2024-09-16T15:48:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.837+0000 7fd4fa400640 1 Processor -- start 2024-09-16T15:48:17.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.837+0000 7fd4fa400640 1 -- start start 2024-09-16T15:48:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4fa400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40d0540 0x7fd4f4151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4fa400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f415bc30 0x7fd4f40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4fa400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4f40c9400 0x7fd4f40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4fa400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4074660 con 0x7fd4f415bc30 2024-09-16T15:48:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4fa400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f406f860 con 0x7fd4f40d0540 2024-09-16T15:48:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4fa400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f4152490 con 0x7fd4f40c9400 2024-09-16T15:48:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4f3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f415bc30 0x7fd4f40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4f3e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f415bc30 0x7fd4f40b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53464/0 (socket says 172.21.15.60:53464) 2024-09-16T15:48:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.838+0000 7fd4f3e00640 1 -- 172.21.15.60:0/603896419 learned_addr learned my addr 172.21.15.60:0/603896419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f9000640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40d0540 0x7fd4f4151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f9a00640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4f40c9400 0x7fd4f40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f3e00640 1 -- 172.21.15.60:0/603896419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4f40c9400 msgr2=0x7fd4f40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f3e00640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4f40c9400 0x7fd4f40b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f3e00640 1 -- 172.21.15.60:0/603896419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40d0540 msgr2=0x7fd4f4151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f3e00640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40d0540 0x7fd4f4151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f3e00640 1 -- 172.21.15.60:0/603896419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f41f8200 con 0x7fd4f415bc30 2024-09-16T15:48:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f3e00640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f415bc30 0x7fd4f40b3f20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0000c00 tx=0x7fd4e0032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f0c00640 1 -- 172.21.15.60:0/603896419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0040070 con 0x7fd4f415bc30 2024-09-16T15:48:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.839+0000 7fd4f0c00640 1 -- 172.21.15.60:0/603896419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4e00038e0 con 0x7fd4f415bc30 2024-09-16T15:48:17.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.840+0000 7fd4f0c00640 1 -- 172.21.15.60:0/603896419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0005c90 con 0x7fd4f415bc30 2024-09-16T15:48:17.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.840+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f41f84e0 con 0x7fd4f415bc30 2024-09-16T15:48:17.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.840+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f41f8aa0 con 0x7fd4f415bc30 2024-09-16T15:48:17.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.841+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4158eb0 con 0x7fd4f415bc30 2024-09-16T15:48:17.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.842+0000 7fd4f0c00640 1 -- 172.21.15.60:0/603896419 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 26) ==== 102256+0+0 (secure 0 0 0) 0x7fd4e0039040 con 0x7fd4f415bc30 2024-09-16T15:48:17.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.843+0000 7fd4f0c00640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd4d4079300 0x7fd4d407b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.843+0000 7fd4f0c00640 1 -- 172.21.15.60:0/603896419 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fd4e00bd6b0 con 0x7fd4f415bc30 2024-09-16T15:48:17.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.843+0000 7fd4f9000640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd4d4079300 0x7fd4d407b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.844+0000 7fd4f9000640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd4d4079300 0x7fd4d407b7c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8007fd0 tx=0x7fd4e8001370 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:17.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:17.847+0000 7fd4f0c00640 1 -- 172.21.15.60:0/603896419 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd4f4158eb0 con 0x7fd4f415bc30 2024-09-16T15:48:18.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.053+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd4f406fdf0 con 0x7fd4f415bc30 2024-09-16T15:48:18.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.054+0000 7fd4f0c00640 1 -- 172.21.15.60:0/603896419 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v173) ==== 80+0+0 (secure 0 0 0) 0x7fd4e00848e0 con 0x7fd4f415bc30 2024-09-16T15:48:18.055 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:48:18.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd4d4079300 msgr2=0x7fd4d407b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd4d4079300 0x7fd4d407b7c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8007fd0 tx=0x7fd4e8001370 comp rx=0 tx=0).stop 2024-09-16T15:48:18.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f415bc30 msgr2=0x7fd4f40b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f415bc30 0x7fd4f40b3f20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0000c00 tx=0x7fd4e0032c10 comp rx=0 tx=0).stop 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 shutdown_connections 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd4d4079300 0x7fd4d407b7c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4f40c9400 0x7fd4f40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4f415bc30 0x7fd4f40b3f20 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 --2- 172.21.15.60:0/603896419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4f40d0540 0x7fd4f4151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 >> 172.21.15.60:0/603896419 conn(0x7fd4f40cfa20 msgr2=0x7fd4f4156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 shutdown_connections 2024-09-16T15:48:18.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.056+0000 7fd4fa400640 1 -- 172.21.15.60:0/603896419 wait complete. 2024-09-16T15:48:18.073 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:48:18.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:18.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:18.176 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:48:18.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:18.177 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:48:18.222 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:48:18.222 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:48:18.223 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:18.260 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:48:18.260 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:18.337 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:48:18.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.497+0000 7fd293200640 1 -- 172.21.15.60:0/1297465861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2940725d0 msgr2=0x7fd2940b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.497+0000 7fd293200640 1 --2- 172.21.15.60:0/1297465861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2940725d0 0x7fd2940b28f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009830 tx=0x7fd27c02a5b0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.497+0000 7fd293200640 1 -- 172.21.15.60:0/1297465861 shutdown_connections 2024-09-16T15:48:18.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.497+0000 7fd293200640 1 --2- 172.21.15.60:0/1297465861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2940725d0 0x7fd2940b28f0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.497+0000 7fd293200640 1 -- 172.21.15.60:0/1297465861 >> 172.21.15.60:0/1297465861 conn(0x7fd2940cfa20 msgr2=0x7fd2940cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:18.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.498+0000 7fd293200640 1 -- 172.21.15.60:0/1297465861 shutdown_connections 2024-09-16T15:48:18.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.498+0000 7fd293200640 1 -- 172.21.15.60:0/1297465861 wait complete. 2024-09-16T15:48:18.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.499+0000 7fd293200640 1 Processor -- start 2024-09-16T15:48:18.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.500+0000 7fd293200640 1 -- start start 2024-09-16T15:48:18.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.500+0000 7fd293200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2940d0540 0x7fd2940aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.500+0000 7fd293200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29415bc30 0x7fd2940cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.500+0000 7fd293200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd29414da60 0x7fd2940cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.500+0000 7fd293200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fd294074660 con 0x7fd29415bc30 2024-09-16T15:48:18.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.500+0000 7fd293200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd29406fa30 con 0x7fd2940d0540 2024-09-16T15:48:18.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.500+0000 7fd293200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fd2940cf630 con 0x7fd29414da60 2024-09-16T15:48:18.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd289400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29415bc30 0x7fd2940cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd289400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29415bc30 0x7fd2940cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53472/0 (socket says 172.21.15.60:53472) 2024-09-16T15:48:18.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd289400640 1 -- 172.21.15.60:0/3540607715 learned_addr learned my addr 172.21.15.60:0/3540607715 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:18.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd289400640 1 -- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd29414da60 msgr2=0x7fd2940cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd292800640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd29414da60 0x7fd2940cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd291e00640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2940d0540 0x7fd2940aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd289400640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd29414da60 0x7fd2940cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd289400640 1 -- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2940d0540 msgr2=0x7fd2940aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd289400640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2940d0540 0x7fd2940aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd289400640 1 -- 172.21.15.60:0/3540607715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2940ab120 con 0x7fd29415bc30 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd291e00640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2940d0540 0x7fd2940aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.501+0000 7fd292800640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd29414da60 0x7fd2940cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.502+0000 7fd289400640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29415bc30 0x7fd2940cf0f0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000c00 tx=0x7fd27c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.502+0000 7fd28aa00640 1 -- 172.21.15.60:0/3540607715 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c040070 con 0x7fd29415bc30 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.502+0000 7fd28aa00640 1 -- 172.21.15.60:0/3540607715 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd27c003af0 con 0x7fd29415bc30 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.502+0000 7fd28aa00640 1 -- 172.21.15.60:0/3540607715 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c045460 con 0x7fd29415bc30 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.502+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2940ab400 con 0x7fd29415bc30 2024-09-16T15:48:18.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.502+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2940cd5f0 con 0x7fd29415bc30 2024-09-16T15:48:18.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.503+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2940b28f0 con 0x7fd29415bc30 2024-09-16T15:48:18.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.504+0000 7fd28aa00640 1 -- 172.21.15.60:0/3540607715 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 26) ==== 102256+0+0 (secure 0 0 0) 0x7fd27c039040 con 0x7fd29415bc30 2024-09-16T15:48:18.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.506+0000 7fd28aa00640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd270079300 0x7fd27007b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.506+0000 7fd28aa00640 1 -- 172.21.15.60:0/3540607715 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fd27c0ba8d0 con 0x7fd29415bc30 2024-09-16T15:48:18.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.506+0000 7fd291e00640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd270079300 0x7fd27007b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.507+0000 7fd291e00640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd270079300 0x7fd27007b7c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd2840046b0 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:18.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.510+0000 7fd28aa00640 1 -- 172.21.15.60:0/3540607715 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd27c084990 con 0x7fd29415bc30 2024-09-16T15:48:18.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.716+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd2941515d0 con 0x7fd29415bc30 2024-09-16T15:48:18.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.717+0000 7fd28aa00640 1 -- 172.21.15.60:0/3540607715 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v173) ==== 80+0+0 (secure 0 0 0) 0x7fd27c084290 con 0x7fd29415bc30 2024-09-16T15:48:18.718 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-09-16T15:48:18.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd270079300 msgr2=0x7fd27007b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd270079300 0x7fd27007b7c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd2840046b0 tx=0x7fd284009040 comp rx=0 tx=0).stop 2024-09-16T15:48:18.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29415bc30 msgr2=0x7fd2940cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29415bc30 0x7fd2940cf0f0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000c00 tx=0x7fd27c032d50 comp rx=0 tx=0).stop 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 shutdown_connections 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fd270079300 0x7fd27007b7c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd29414da60 0x7fd2940cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd29415bc30 0x7fd2940cf0f0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 --2- 172.21.15.60:0/3540607715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2940d0540 0x7fd2940aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 >> 172.21.15.60:0/3540607715 conn(0x7fd2940cfa20 msgr2=0x7fd2940b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 shutdown_connections 2024-09-16T15:48:18.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:18.719+0000 7fd293200640 1 -- 172.21.15.60:0/3540607715 wait complete. 2024-09-16T15:48:18.735 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:48:18.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:18.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T15:48:18.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:48:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:18.799 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:48:18.835 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:48:18.835 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:48:18.835 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:18.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:48:18.871 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:48:18.871 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:48:18.871 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:18.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:48:18.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.070+0000 7f87dbc00640 1 -- 172.21.15.60:0/3148282922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b8003a00 msgr2=0x7f87b8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.070+0000 7f87dbc00640 1 --2- 172.21.15.60:0/3148282922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87b8003a00 0x7f87b8003e20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f87c400c2e0 tx=0x7f87b8005830 comp rx=0 tx=0).stop 2024-09-16T15:48:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.070+0000 7f87d9c00640 1 -- 172.21.15.60:0/3148282922 reap_dead start 2024-09-16T15:48:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.070+0000 7f87dbc00640 1 -- 172.21.15.60:0/3148282922 shutdown_connections 2024-09-16T15:48:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.070+0000 7f87dbc00640 1 -- 172.21.15.60:0/3148282922 >> 172.21.15.60:0/3148282922 conn(0x7f87d40b1950 msgr2=0x7f87d40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.070+0000 7f87dbc00640 1 -- 172.21.15.60:0/3148282922 shutdown_connections 2024-09-16T15:48:19.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.071+0000 7f87dbc00640 1 -- 172.21.15.60:0/3148282922 wait complete. 2024-09-16T15:48:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.072+0000 7f87dbc00640 1 Processor -- start 2024-09-16T15:48:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.072+0000 7f87dbc00640 1 -- start start 2024-09-16T15:48:19.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.073+0000 7f87dbc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87b8003a00 0x7f87d40b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.073+0000 7f87dbc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87d40b0860 0x7f87d41f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.073+0000 7f87dbc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87d41f86a0 0x7f87d41595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.073+0000 7f87dbc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4161a70 con 0x7f87d40b0860 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.073+0000 7f87dbc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87d406fdb0 con 0x7f87d41f86a0 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.073+0000 7f87dbc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f87d4073280 con 0x7f87b8003a00 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.073+0000 7f87cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87d40b0860 0x7f87d41f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87cbe00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87d40b0860 0x7f87d41f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47158/0 (socket says 172.21.15.60:47158) 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87cbe00640 1 -- 172.21.15.60:0/2226974883 learned_addr learned my addr 172.21.15.60:0/2226974883 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87d9200640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87b8003a00 0x7f87d40b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87d9c00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87d41f86a0 0x7f87d41595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87cbe00640 1 -- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87b8003a00 msgr2=0x7f87d40b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87cbe00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87b8003a00 0x7f87d40b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87cbe00640 1 -- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87d41f86a0 msgr2=0x7f87d41595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87cbe00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87d41f86a0 0x7f87d41595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87cbe00640 1 -- 172.21.15.60:0/2226974883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d41f4c10 con 0x7f87d40b0860 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87d9c00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87d41f86a0 0x7f87d41595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87cbe00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87d40b0860 0x7f87d41f8160 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f87bc011ab0 tx=0x7f87bc011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87d1600640 1 -- 172.21.15.60:0/2226974883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc01e070 con 0x7f87d40b0860 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87d1600640 1 -- 172.21.15.60:0/2226974883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87bc00fc70 con 0x7f87d40b0860 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87d1600640 1 -- 172.21.15.60:0/2226974883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc019670 con 0x7f87d40b0860 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.074+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d41f4ef0 con 0x7f87d40b0860 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.075+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d41f54b0 con 0x7f87d40b0860 2024-09-16T15:48:19.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.076+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d41568e0 con 0x7f87d40b0860 2024-09-16T15:48:19.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.077+0000 7f87d1600640 1 -- 172.21.15.60:0/2226974883 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 26) ==== 102256+0+0 (secure 0 0 0) 0x7f87bc021070 con 0x7f87d40b0860 2024-09-16T15:48:19.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.078+0000 7f87d1600640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f87b0079330 0x7f87b007b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.078+0000 7f87d1600640 1 -- 172.21.15.60:0/2226974883 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f87bc099c10 con 0x7f87d40b0860 2024-09-16T15:48:19.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.078+0000 7f87d9200640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f87b0079330 0x7f87b007b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.079+0000 7f87d9200640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f87b0079330 0x7f87b007b7f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f87c4033000 tx=0x7f87c4038040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:19.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.082+0000 7f87d1600640 1 -- 172.21.15.60:0/2226974883 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f87bc00fe60 con 0x7f87d40b0860 2024-09-16T15:48:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.321+0000 7f87d1600640 1 -- 172.21.15.60:0/2226974883 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f87bc05f6c0 con 0x7f87d40b0860 2024-09-16T15:48:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.327+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87d40737e0 con 0x7f87d40b0860 2024-09-16T15:48:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.328+0000 7f87d1600640 1 -- 172.21.15.60:0/2226974883 <== mon.0 v2:172.21.15.60:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5227 (secure 0 0 0) 0x7f87bc082160 con 0x7f87d40b0860 2024-09-16T15:48:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:19.329 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"btime":"2024-09-16T15:48:14:280493+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/657477165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":657477165},{"type":"v1","addr":"172.21.15.100:6801","nonce":657477165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/293588121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":293588121},{"type":"v1","addr":"172.21.15.100:6835","nonce":293588121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/903832682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":903832682},{"type":"v1","addr":"172.21.15.60:6834","nonce":903832682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:03.623622+0000","modified":"2024-09-16T15:48:06.645819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/786870322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":786870322},{"type":"v1","addr":"172.21.15.60:6835","nonce":786870322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9222,"qdb_cluster":[9222]},"id":13}]} 2024-09-16T15:48:19.329 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 123 2024-09-16T15:48:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f87b0079330 msgr2=0x7f87b007b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f87b0079330 0x7f87b007b7f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f87c4033000 tx=0x7f87c4038040 comp rx=0 tx=0).stop 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87d40b0860 msgr2=0x7f87d41f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87d40b0860 0x7f87d41f8160 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f87bc011ab0 tx=0x7f87bc011f70 comp rx=0 tx=0).stop 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 shutdown_connections 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f87b0079330 0x7f87b007b7f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87d41f86a0 0x7f87d41595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87d40b0860 0x7f87d41f8160 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 --2- 172.21.15.60:0/2226974883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87b8003a00 0x7f87d40b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 >> 172.21.15.60:0/2226974883 conn(0x7f87d40b1950 msgr2=0x7f87d40acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 shutdown_connections 2024-09-16T15:48:19.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.330+0000 7f87dbc00640 1 -- 172.21.15.60:0/2226974883 wait complete. 2024-09-16T15:48:19.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:19.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.506+0000 7f9155e00640 1 -- 172.21.15.60:0/3168651057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f915015f5a0 msgr2=0x7f91500b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.506+0000 7f9155e00640 1 --2- 172.21.15.60:0/3168651057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f915015f5a0 0x7f91500b2440 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f91500cfbb0 tx=0x7f914002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.506+0000 7f9155e00640 1 -- 172.21.15.60:0/3168651057 shutdown_connections 2024-09-16T15:48:19.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.506+0000 7f9155e00640 1 --2- 172.21.15.60:0/3168651057 >> v2:172.21.15.100:3300/0 conn(0x7f91500ccf00 0x7f91500acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.506+0000 7f9155e00640 1 --2- 172.21.15.60:0/3168651057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f915015f5a0 0x7f91500b2440 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.506+0000 7f9155e00640 1 -- 172.21.15.60:0/3168651057 >> 172.21.15.60:0/3168651057 conn(0x7f91500b1950 msgr2=0x7f91500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:19.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.506+0000 7f9155e00640 1 -- 172.21.15.60:0/3168651057 shutdown_connections 2024-09-16T15:48:19.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.506+0000 7f9155e00640 1 -- 172.21.15.60:0/3168651057 wait complete. 2024-09-16T15:48:19.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.508+0000 7f9155e00640 1 Processor -- start 2024-09-16T15:48:19.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.508+0000 7f9155e00640 1 -- start start 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.509+0000 7f9155e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91500ad270 0x7f915014c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.509+0000 7f9155e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91500ccf00 0x7f9150152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.509+0000 7f9155e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f915015f5a0 0x7f9150152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.509+0000 7f9155e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f915006f160 con 0x7f91500ad270 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.509+0000 7f9155e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f915006f2e0 con 0x7f91500ccf00 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.509+0000 7f9155e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f915006f460 con 0x7f915015f5a0 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.509+0000 7f914ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91500ad270 0x7f915014c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914ea00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91500ad270 0x7f915014c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47170/0 (socket says 172.21.15.60:47170) 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914ea00640 1 -- 172.21.15.60:0/3299520695 learned_addr learned my addr 172.21.15.60:0/3299520695 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:19.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f9146000640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91500ccf00 0x7f9150152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914f400640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f915015f5a0 0x7f9150152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914ea00640 1 -- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f915015f5a0 msgr2=0x7f9150152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914ea00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f915015f5a0 0x7f9150152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914ea00640 1 -- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91500ccf00 msgr2=0x7f9150152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914ea00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91500ccf00 0x7f9150152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914ea00640 1 -- 172.21.15.60:0/3299520695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91501f2c40 con 0x7f91500ad270 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f9146000640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91500ccf00 0x7f9150152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f914ea00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91500ad270 0x7f915014c8e0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f9140009780 tx=0x7f9140033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.510+0000 7f9147e00640 1 -- 172.21.15.60:0/3299520695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140031d60 con 0x7f91500ad270 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.511+0000 7f9147e00640 1 -- 172.21.15.60:0/3299520695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9140033e10 con 0x7f91500ad270 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.511+0000 7f9147e00640 1 -- 172.21.15.60:0/3299520695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140005260 con 0x7f91500ad270 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.511+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9150150990 con 0x7f91500ad270 2024-09-16T15:48:19.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.511+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91501f2ee0 con 0x7f91500ad270 2024-09-16T15:48:19.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.512+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915006f9f0 con 0x7f91500ad270 2024-09-16T15:48:19.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.513+0000 7f9147e00640 1 -- 172.21.15.60:0/3299520695 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f9140005400 con 0x7f91500ad270 2024-09-16T15:48:19.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.514+0000 7f9147e00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f912c079420 0x7f912c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.514+0000 7f9147e00640 1 -- 172.21.15.60:0/3299520695 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f91400ba700 con 0x7f91500ad270 2024-09-16T15:48:19.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.514+0000 7f9146000640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f912c079420 0x7f912c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.515+0000 7f9146000640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f912c079420 0x7f912c07b8e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f91380058c0 tx=0x7f9138005850 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:19.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.518+0000 7f9147e00640 1 -- 172.21.15.60:0/3299520695 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f914003f030 con 0x7f91500ad270 2024-09-16T15:48:19.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.753+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f915015e060 con 0x7f91500ad270 2024-09-16T15:48:19.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.753+0000 7f9147e00640 1 -- 172.21.15.60:0/3299520695 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5227 (secure 0 0 0) 0x7f9140084560 con 0x7f91500ad270 2024-09-16T15:48:19.754 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:19.754 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"btime":"2024-09-16T15:48:14:280493+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/657477165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":657477165},{"type":"v1","addr":"172.21.15.100:6801","nonce":657477165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/293588121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":293588121},{"type":"v1","addr":"172.21.15.100:6835","nonce":293588121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/903832682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":903832682},{"type":"v1","addr":"172.21.15.60:6834","nonce":903832682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:03.623622+0000","modified":"2024-09-16T15:48:06.645819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/786870322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":786870322},{"type":"v1","addr":"172.21.15.60:6835","nonce":786870322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9222,"qdb_cluster":[9222]},"id":13}]} 2024-09-16T15:48:19.755 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 123 2024-09-16T15:48:19.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f912c079420 msgr2=0x7f912c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f912c079420 0x7f912c07b8e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f91380058c0 tx=0x7f9138005850 comp rx=0 tx=0).stop 2024-09-16T15:48:19.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91500ad270 msgr2=0x7f915014c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91500ad270 0x7f915014c8e0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f9140009780 tx=0x7f9140033c20 comp rx=0 tx=0).stop 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 shutdown_connections 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f912c079420 0x7f912c07b8e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f915015f5a0 0x7f9150152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91500ccf00 0x7f9150152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 --2- 172.21.15.60:0/3299520695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91500ad270 0x7f915014c8e0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 >> 172.21.15.60:0/3299520695 conn(0x7f91500b1950 msgr2=0x7f915015e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 shutdown_connections 2024-09-16T15:48:19.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.756+0000 7f9155e00640 1 -- 172.21.15.60:0/3299520695 wait complete. 2024-09-16T15:48:19.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.932+0000 7f622d400640 1 -- 172.21.15.60:0/2147752811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f622815bc30 msgr2=0x7f62280b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.932+0000 7f622d400640 1 --2- 172.21.15.60:0/2147752811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f622815bc30 0x7f62280b28f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f6218005fd0 tx=0x7f621802aa90 comp rx=0 tx=0).stop 2024-09-16T15:48:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.933+0000 7f622d400640 1 -- 172.21.15.60:0/2147752811 shutdown_connections 2024-09-16T15:48:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.933+0000 7f622d400640 1 --2- 172.21.15.60:0/2147752811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62280d0540 0x7f622806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.933+0000 7f622d400640 1 --2- 172.21.15.60:0/2147752811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f622815bc30 0x7f62280b28f0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.933+0000 7f622d400640 1 -- 172.21.15.60:0/2147752811 >> 172.21.15.60:0/2147752811 conn(0x7f62280cfa20 msgr2=0x7f62280cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.933+0000 7f622d400640 1 -- 172.21.15.60:0/2147752811 shutdown_connections 2024-09-16T15:48:19.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.933+0000 7f622d400640 1 -- 172.21.15.60:0/2147752811 wait complete. 2024-09-16T15:48:19.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.934+0000 7f622d400640 1 Processor -- start 2024-09-16T15:48:19.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.934+0000 7f622d400640 1 -- start start 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.935+0000 7f622d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62280d0540 0x7f62281f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.935+0000 7f622d400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f622815bc30 0x7f6228154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.935+0000 7f622d400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f622815d170 0x7f6228154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.935+0000 7f622d400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6228074660 con 0x7f62280d0540 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.935+0000 7f622d400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f622806f860 con 0x7f622815bc30 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.935+0000 7f622d400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f62280caf30 con 0x7f622815d170 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.935+0000 7f6227e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62280d0540 0x7f62281f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6227e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62280d0540 0x7f62281f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47182/0 (socket says 172.21.15.60:47182) 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6227e00640 1 -- 172.21.15.60:0/1492836874 learned_addr learned my addr 172.21.15.60:0/1492836874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f621f400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f622815bc30 0x7f6228154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f622ca00640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f622815d170 0x7f6228154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6227e00640 1 -- 172.21.15.60:0/1492836874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f622815d170 msgr2=0x7f6228154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6227e00640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f622815d170 0x7f6228154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6227e00640 1 -- 172.21.15.60:0/1492836874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f622815bc30 msgr2=0x7f6228154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6227e00640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f622815bc30 0x7f6228154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6227e00640 1 -- 172.21.15.60:0/1492836874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622815c3b0 con 0x7f62280d0540 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6227e00640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62280d0540 0x7f62281f2f40 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f6218006010 tx=0x7f6218003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.936+0000 7f6224c00640 1 -- 172.21.15.60:0/1492836874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218044070 con 0x7f62280d0540 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.937+0000 7f6224c00640 1 -- 172.21.15.60:0/1492836874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6218003e00 con 0x7f62280d0540 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.937+0000 7f6224c00640 1 -- 172.21.15.60:0/1492836874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218004490 con 0x7f62280d0540 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.937+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622815c690 con 0x7f62280d0540 2024-09-16T15:48:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.937+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281f3480 con 0x7f62280d0540 2024-09-16T15:48:19.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.938+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622806fdf0 con 0x7f62280d0540 2024-09-16T15:48:19.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.939+0000 7f6224c00640 1 -- 172.21.15.60:0/1492836874 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f6218047070 con 0x7f62280d0540 2024-09-16T15:48:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.940+0000 7f6224c00640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6204079420 0x7f620407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.940+0000 7f6224c00640 1 -- 172.21.15.60:0/1492836874 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f62180ba1d0 con 0x7f62280d0540 2024-09-16T15:48:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.940+0000 7f621f400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6204079420 0x7f620407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:19.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.941+0000 7f621f400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6204079420 0x7f620407b8e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6228153da0 tx=0x7f6210009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:19.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:19.944+0000 7f6224c00640 1 -- 172.21.15.60:0/1492836874 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6218032030 con 0x7f62280d0540 2024-09-16T15:48:20.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.138+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6228151560 con 0x7f62280d0540 2024-09-16T15:48:20.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.140+0000 7f6224c00640 1 -- 172.21.15.60:0/1492836874 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16787 (secure 0 0 0) 0x7f62180839e0 con 0x7f62280d0540 2024-09-16T15:48:20.141 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:20.141 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":173,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:48:05.189290+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:01.416561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:02.578764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:20.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6204079420 msgr2=0x7f620407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6204079420 0x7f620407b8e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6228153da0 tx=0x7f6210009040 comp rx=0 tx=0).stop 2024-09-16T15:48:20.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62280d0540 msgr2=0x7f62281f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62280d0540 0x7f62281f2f40 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f6218006010 tx=0x7f6218003c10 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 shutdown_connections 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6204079420 0x7f620407b8e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f622815d170 0x7f6228154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f622815bc30 0x7f6228154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 --2- 172.21.15.60:0/1492836874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f62280d0540 0x7f62281f2f40 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 >> 172.21.15.60:0/1492836874 conn(0x7f62280cfa20 msgr2=0x7f62280b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 shutdown_connections 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.142+0000 7f622d400640 1 -- 172.21.15.60:0/1492836874 wait complete. 2024-09-16T15:48:20.159 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:48:20.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.322+0000 7faa4e600640 1 -- 172.21.15.60:0/229999473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b4100 msgr2=0x7faa480b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.322+0000 7faa4e600640 1 --2- 172.21.15.60:0/229999473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b4100 0x7faa480b1770 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7faa480d4ea0 tx=0x7faa4002a7e0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.322+0000 7faa4e600640 1 -- 172.21.15.60:0/229999473 shutdown_connections 2024-09-16T15:48:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.322+0000 7faa4e600640 1 --2- 172.21.15.60:0/229999473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b4100 0x7faa480b1770 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.322+0000 7faa4e600640 1 --2- 172.21.15.60:0/229999473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa480b5370 0x7faa480b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.322+0000 7faa4e600640 1 -- 172.21.15.60:0/229999473 >> 172.21.15.60:0/229999473 conn(0x7faa480d3df0 msgr2=0x7faa480d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.323+0000 7faa4e600640 1 -- 172.21.15.60:0/229999473 shutdown_connections 2024-09-16T15:48:20.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.323+0000 7faa4e600640 1 -- 172.21.15.60:0/229999473 wait complete. 2024-09-16T15:48:20.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.324+0000 7faa4e600640 1 Processor -- start 2024-09-16T15:48:20.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.324+0000 7faa4e600640 1 -- start start 2024-09-16T15:48:20.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.325+0000 7faa4e600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa480b4100 0x7faa4806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.325+0000 7faa4e600640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b5370 0x7faa48155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.325+0000 7faa4e600640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa48161440 0x7faa48155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.325+0000 7faa4e600640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faa480b2040 con 0x7faa480b4100 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.325+0000 7faa4e600640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa480b1ec0 con 0x7faa480b5370 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.325+0000 7faa4e600640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7faa480b21c0 con 0x7faa48161440 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa4d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa480b4100 0x7faa4806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa4d200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa480b4100 0x7faa4806e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47194/0 (socket says 172.21.15.60:47194) 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa4d200640 1 -- 172.21.15.60:0/2411250987 learned_addr learned my addr 172.21.15.60:0/2411250987 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa47e00640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b5370 0x7faa48155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa4dc00640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa48161440 0x7faa48155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa47e00640 1 -- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa48161440 msgr2=0x7faa48155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa47e00640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa48161440 0x7faa48155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa47e00640 1 -- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa480b4100 msgr2=0x7faa4806e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa47e00640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa480b4100 0x7faa4806e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa47e00640 1 -- 172.21.15.60:0/2411250987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa481f4430 con 0x7faa480b5370 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.326+0000 7faa4d200640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa480b4100 0x7faa4806e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:48:20.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.327+0000 7faa47e00640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b5370 0x7faa48155680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e9f0 tx=0x7faa3400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:20.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.327+0000 7faa44c00640 1 -- 172.21.15.60:0/2411250987 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3400ccc0 con 0x7faa480b5370 2024-09-16T15:48:20.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.327+0000 7faa44c00640 1 -- 172.21.15.60:0/2411250987 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa34010430 con 0x7faa480b5370 2024-09-16T15:48:20.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.327+0000 7faa44c00640 1 -- 172.21.15.60:0/2411250987 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34010860 con 0x7faa480b5370 2024-09-16T15:48:20.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.327+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa481f4710 con 0x7faa480b5370 2024-09-16T15:48:20.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.327+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa481f4cd0 con 0x7faa480b5370 2024-09-16T15:48:20.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.328+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa480b1770 con 0x7faa480b5370 2024-09-16T15:48:20.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.329+0000 7faa44c00640 1 -- 172.21.15.60:0/2411250987 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7faa34010a00 con 0x7faa480b5370 2024-09-16T15:48:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.330+0000 7faa44c00640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faa28079420 0x7faa2807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.330+0000 7faa44c00640 1 -- 172.21.15.60:0/2411250987 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7faa34099d00 con 0x7faa480b5370 2024-09-16T15:48:20.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.330+0000 7faa4d200640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faa28079420 0x7faa2807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.331+0000 7faa4d200640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faa28079420 0x7faa2807b8e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa3c007b10 tx=0x7faa3c007a60 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:20.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.335+0000 7faa44c00640 1 -- 172.21.15.60:0/2411250987 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faa34063770 con 0x7faa480b5370 2024-09-16T15:48:20.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.569+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa480760b0 con 0x7faa480b5370 2024-09-16T15:48:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.570+0000 7faa44c00640 1 -- 172.21.15.60:0/2411250987 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5227 (secure 0 0 0) 0x7faa340681a0 con 0x7faa480b5370 2024-09-16T15:48:20.571 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:20.571 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"btime":"2024-09-16T15:48:14:280493+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/657477165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":657477165},{"type":"v1","addr":"172.21.15.100:6801","nonce":657477165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/293588121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":293588121},{"type":"v1","addr":"172.21.15.100:6835","nonce":293588121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/903832682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":903832682},{"type":"v1","addr":"172.21.15.60:6834","nonce":903832682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:03.623622+0000","modified":"2024-09-16T15:48:06.645819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/786870322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":786870322},{"type":"v1","addr":"172.21.15.60:6835","nonce":786870322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9222,"qdb_cluster":[9222]},"id":13}]} 2024-09-16T15:48:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 123 2024-09-16T15:48:20.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faa28079420 msgr2=0x7faa2807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faa28079420 0x7faa2807b8e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa3c007b10 tx=0x7faa3c007a60 comp rx=0 tx=0).stop 2024-09-16T15:48:20.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b5370 msgr2=0x7faa48155680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b5370 0x7faa48155680 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e9f0 tx=0x7faa3400eeb0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 shutdown_connections 2024-09-16T15:48:20.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7faa28079420 0x7faa2807b8e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa48161440 0x7faa48155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa480b5370 0x7faa48155680 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 --2- 172.21.15.60:0/2411250987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa480b4100 0x7faa4806e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 >> 172.21.15.60:0/2411250987 conn(0x7faa480d3df0 msgr2=0x7faa480d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 shutdown_connections 2024-09-16T15:48:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.573+0000 7faa4e600640 1 -- 172.21.15.60:0/2411250987 wait complete. 2024-09-16T15:48:20.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.750+0000 7f2883e00640 1 -- 172.21.15.60:0/1190590462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 msgr2=0x7f28840b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.750+0000 7f2883e00640 1 --2- 172.21.15.60:0/1190590462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 0x7f28840b28f0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2884073b00 tx=0x7f287402a530 comp rx=0 tx=0).stop 2024-09-16T15:48:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.750+0000 7f2883e00640 1 -- 172.21.15.60:0/1190590462 shutdown_connections 2024-09-16T15:48:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.750+0000 7f2883e00640 1 --2- 172.21.15.60:0/1190590462 >> v2:172.21.15.100:3300/0 conn(0x7f28840725d0 0x7f2884073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.750+0000 7f2883e00640 1 --2- 172.21.15.60:0/1190590462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 0x7f28840b28f0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.750+0000 7f2883e00640 1 -- 172.21.15.60:0/1190590462 >> 172.21.15.60:0/1190590462 conn(0x7f28840cfa20 msgr2=0x7f28840cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.750+0000 7f2883e00640 1 -- 172.21.15.60:0/1190590462 shutdown_connections 2024-09-16T15:48:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.750+0000 7f2883e00640 1 -- 172.21.15.60:0/1190590462 wait complete. 2024-09-16T15:48:20.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.752+0000 7f2883e00640 1 Processor -- start 2024-09-16T15:48:20.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.752+0000 7f2883e00640 1 -- start start 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2883e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28840d0540 0x7f28841f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2883e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 0x7f28840cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2883e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28840cee10 0x7f2884157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2883e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f2884074610 con 0x7f288415bc30 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2883e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2884074490 con 0x7f28840cee10 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2883e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f2884074790 con 0x7f28840d0540 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2882000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 0x7f28840cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2882000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 0x7f28840cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47212/0 (socket says 172.21.15.60:47212) 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2882000640 1 -- 172.21.15.60:0/3013014377 learned_addr learned my addr 172.21.15.60:0/3013014377 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2883400640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28840cee10 0x7f2884157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.753+0000 7f2882a00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28840d0540 0x7f28841f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2882000640 1 -- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28840d0540 msgr2=0x7f28841f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2882000640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28840d0540 0x7f28841f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2882000640 1 -- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28840cee10 msgr2=0x7f2884157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2882000640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28840cee10 0x7f2884157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2882000640 1 -- 172.21.15.60:0/3013014377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28840cdd90 con 0x7f288415bc30 2024-09-16T15:48:20.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2882a00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28840d0540 0x7f28841f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:20.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2882000640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 0x7f28840cd5f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f287000d770 tx=0x7f287000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:20.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f287aa00640 1 -- 172.21.15.60:0/3013014377 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870017070 con 0x7f288415bc30 2024-09-16T15:48:20.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f287aa00640 1 -- 172.21.15.60:0/3013014377 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2870009d20 con 0x7f288415bc30 2024-09-16T15:48:20.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f287aa00640 1 -- 172.21.15.60:0/3013014377 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28700106c0 con 0x7f288415bc30 2024-09-16T15:48:20.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28840ce070 con 0x7f288415bc30 2024-09-16T15:48:20.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.754+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28840ce910 con 0x7f288415bc30 2024-09-16T15:48:20.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.755+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288406fd20 con 0x7f288415bc30 2024-09-16T15:48:20.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.757+0000 7f287aa00640 1 -- 172.21.15.60:0/3013014377 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f287001a070 con 0x7f288415bc30 2024-09-16T15:48:20.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.758+0000 7f287aa00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2864079450 0x7f286407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.758+0000 7f287aa00640 1 -- 172.21.15.60:0/3013014377 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7f2870099670 con 0x7f288415bc30 2024-09-16T15:48:20.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.758+0000 7f2882a00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2864079450 0x7f286407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.759+0000 7f2882a00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2864079450 0x7f286407b910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2874034000 tx=0x7f2874004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:20.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.762+0000 7f287aa00640 1 -- 172.21.15.60:0/3013014377 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2870062e80 con 0x7f288415bc30 2024-09-16T15:48:20.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.997+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28840cff30 con 0x7f288415bc30 2024-09-16T15:48:20.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:20.997+0000 7f287aa00640 1 -- 172.21.15.60:0/3013014377 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) ==== 95+0+5227 (secure 0 0 0) 0x7f28700678b0 con 0x7f288415bc30 2024-09-16T15:48:20.998 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:20.998 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"btime":"2024-09-16T15:48:14:280493+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/657477165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":657477165},{"type":"v1","addr":"172.21.15.100:6801","nonce":657477165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/293588121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":293588121},{"type":"v1","addr":"172.21.15.100:6835","nonce":293588121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/903832682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":903832682},{"type":"v1","addr":"172.21.15.60:6834","nonce":903832682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:03.623622+0000","modified":"2024-09-16T15:48:06.645819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9222},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9222":{"gid":9222,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/786870322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":786870322},{"type":"v1","addr":"172.21.15.60:6835","nonce":786870322}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9222,"qdb_cluster":[9222]},"id":13}]} 2024-09-16T15:48:20.999 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 123 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2864079450 msgr2=0x7f286407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2864079450 0x7f286407b910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2874034000 tx=0x7f2874004730 comp rx=0 tx=0).stop 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 msgr2=0x7f28840cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 0x7f28840cd5f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f287000d770 tx=0x7f287000dc30 comp rx=0 tx=0).stop 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 shutdown_connections 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f2864079450 0x7f286407b910 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28840cee10 0x7f2884157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f288415bc30 0x7f28840cd5f0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 --2- 172.21.15.60:0/3013014377 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28840d0540 0x7f28841f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 >> 172.21.15.60:0/3013014377 conn(0x7f28840cfa20 msgr2=0x7f2884152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:21.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 shutdown_connections 2024-09-16T15:48:21.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.000+0000 7f2883e00640 1 -- 172.21.15.60:0/3013014377 wait complete. 2024-09-16T15:48:21.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:21.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.176+0000 7fa74c800640 1 -- 172.21.15.60:0/2656516872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72c004800 msgr2=0x7fa72c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.176+0000 7fa74c800640 1 --2- 172.21.15.60:0/2656516872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72c004800 0x7fa72c004c80 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa734006fa0 tx=0x7fa72c005830 comp rx=0 tx=0).stop 2024-09-16T15:48:21.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.176+0000 7fa74c800640 1 -- 172.21.15.60:0/2656516872 shutdown_connections 2024-09-16T15:48:21.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.176+0000 7fa74c800640 1 --2- 172.21.15.60:0/2656516872 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72c004800 0x7fa72c004c80 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.176+0000 7fa74c800640 1 -- 172.21.15.60:0/2656516872 >> 172.21.15.60:0/2656516872 conn(0x7fa7440b1950 msgr2=0x7fa7440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:21.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.176+0000 7fa74c800640 1 -- 172.21.15.60:0/2656516872 shutdown_connections 2024-09-16T15:48:21.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.176+0000 7fa74c800640 1 -- 172.21.15.60:0/2656516872 wait complete. 2024-09-16T15:48:21.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.178+0000 7fa74c800640 1 Processor -- start 2024-09-16T15:48:21.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.178+0000 7fa74c800640 1 -- start start 2024-09-16T15:48:21.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa74c800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72c004800 0x7fa744151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa74c800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa744156310 0x7fa74414b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa74c800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa744157c00 0x7fa744158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa74c800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fa744074420 con 0x7fa744156310 2024-09-16T15:48:21.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa74c800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7440b26a0 con 0x7fa744157c00 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa74c800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fa7440745a0 con 0x7fa72c004800 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa749400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa744156310 0x7fa74414b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa749400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa744156310 0x7fa74414b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47230/0 (socket says 172.21.15.60:47230) 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa749400640 1 -- 172.21.15.60:0/967376775 learned_addr learned my addr 172.21.15.60:0/967376775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa749e00640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72c004800 0x7fa744151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.179+0000 7fa74a800640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa744157c00 0x7fa744158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa749400640 1 -- 172.21.15.60:0/967376775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72c004800 msgr2=0x7fa744151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa749400640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72c004800 0x7fa744151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa749400640 1 -- 172.21.15.60:0/967376775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa744157c00 msgr2=0x7fa744158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa749400640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa744157c00 0x7fa744158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa749400640 1 -- 172.21.15.60:0/967376775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7440d0570 con 0x7fa744156310 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa749400640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa744156310 0x7fa74414b460 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c320 tx=0x7fa734006d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa73e000640 1 -- 172.21.15.60:0/967376775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7340037f0 con 0x7fa744156310 2024-09-16T15:48:21.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa73e000640 1 -- 172.21.15.60:0/967376775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa734003a80 con 0x7fa744156310 2024-09-16T15:48:21.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa73e000640 1 -- 172.21.15.60:0/967376775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73402e450 con 0x7fa744156310 2024-09-16T15:48:21.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa744156770 con 0x7fa744156310 2024-09-16T15:48:21.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.180+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7440d0ad0 con 0x7fa744156310 2024-09-16T15:48:21.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.182+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7440acd30 con 0x7fa744156310 2024-09-16T15:48:21.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.183+0000 7fa73e000640 1 -- 172.21.15.60:0/967376775 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7fa734035020 con 0x7fa744156310 2024-09-16T15:48:21.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.184+0000 7fa73e000640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa728079450 0x7fa72807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.184+0000 7fa73e000640 1 -- 172.21.15.60:0/967376775 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fa7340b96e0 con 0x7fa744156310 2024-09-16T15:48:21.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.184+0000 7fa749e00640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa728079450 0x7fa72807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.185+0000 7fa749e00640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa728079450 0x7fa72807b910 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa744052890 tx=0x7fa738005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.188+0000 7fa73e000640 1 -- 172.21.15.60:0/967376775 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa734047030 con 0x7fa744156310 2024-09-16T15:48:21.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.386+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7440afc30 con 0x7fa744156310 2024-09-16T15:48:21.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.388+0000 7fa73e000640 1 -- 172.21.15.60:0/967376775 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+16787 (secure 0 0 0) 0x7fa7440d0570 con 0x7fa744156310 2024-09-16T15:48:21.389 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:21.389 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":173,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:48:05.189290+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:01.416561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:02.578764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:21.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa728079450 msgr2=0x7fa72807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa728079450 0x7fa72807b910 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa744052890 tx=0x7fa738005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa744156310 msgr2=0x7fa74414b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa744156310 0x7fa74414b460 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c320 tx=0x7fa734006d20 comp rx=0 tx=0).stop 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 shutdown_connections 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fa728079450 0x7fa72807b910 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa744157c00 0x7fa744158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa744156310 0x7fa74414b460 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 --2- 172.21.15.60:0/967376775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72c004800 0x7fa744151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 >> 172.21.15.60:0/967376775 conn(0x7fa7440b1950 msgr2=0x7fa74406f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 shutdown_connections 2024-09-16T15:48:21.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.390+0000 7fa74c800640 1 -- 172.21.15.60:0/967376775 wait complete. 2024-09-16T15:48:21.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:48:21.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.568+0000 7fe080a00640 1 -- 172.21.15.60:0/1139977220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b34b0 msgr2=0x7fe07c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.568+0000 7fe080a00640 1 --2- 172.21.15.60:0/1139977220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b34b0 0x7fe07c0b3940 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0b6930 tx=0x7fe06802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.568+0000 7fe080a00640 1 -- 172.21.15.60:0/1139977220 shutdown_connections 2024-09-16T15:48:21.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.568+0000 7fe080a00640 1 --2- 172.21.15.60:0/1139977220 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b34b0 0x7fe07c0b3940 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.568+0000 7fe080a00640 1 --2- 172.21.15.60:0/1139977220 >> v2:172.21.15.100:3300/0 conn(0x7fe07c161440 0x7fe07c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.568+0000 7fe080a00640 1 -- 172.21.15.60:0/1139977220 >> 172.21.15.60:0/1139977220 conn(0x7fe07c0d3df0 msgr2=0x7fe07c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:21.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.568+0000 7fe080a00640 1 -- 172.21.15.60:0/1139977220 shutdown_connections 2024-09-16T15:48:21.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.568+0000 7fe080a00640 1 -- 172.21.15.60:0/1139977220 wait complete. 2024-09-16T15:48:21.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.569+0000 7fe080a00640 1 Processor -- start 2024-09-16T15:48:21.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.570+0000 7fe080a00640 1 -- start start 2024-09-16T15:48:21.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe080a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe07c0b34b0 0x7fe07c0b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe080a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b3e80 0x7fe07c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe080a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c161440 0x7fe07c0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe080a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0b2900 con 0x7fe07c0b3e80 2024-09-16T15:48:21.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe080a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0b2780 con 0x7fe07c161440 2024-09-16T15:48:21.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe080a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fe07c0b2a80 con 0x7fe07c0b34b0 2024-09-16T15:48:21.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe079e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b3e80 0x7fe07c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe079e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b3e80 0x7fe07c1559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47254/0 (socket says 172.21.15.60:47254) 2024-09-16T15:48:21.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe079e00640 1 -- 172.21.15.60:0/170652673 learned_addr learned my addr 172.21.15.60:0/170652673 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:21.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe07b200640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c161440 0x7fe07c0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.571+0000 7fe07a800640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe07c0b34b0 0x7fe07c0b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe079e00640 1 -- 172.21.15.60:0/170652673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe07c0b34b0 msgr2=0x7fe07c0b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe079e00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe07c0b34b0 0x7fe07c0b94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe079e00640 1 -- 172.21.15.60:0/170652673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c161440 msgr2=0x7fe07c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe079e00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c161440 0x7fe07c0b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe079e00640 1 -- 172.21.15.60:0/170652673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c1f7ea0 con 0x7fe07c0b3e80 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe079e00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b3e80 0x7fe07c1559c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fe06802aa90 tx=0x7fe068032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe072a00640 1 -- 172.21.15.60:0/170652673 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068040070 con 0x7fe07c0b3e80 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe072a00640 1 -- 172.21.15.60:0/170652673 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe068005310 con 0x7fe07c0b3e80 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c1f8180 con 0x7fe07c0b3e80 2024-09-16T15:48:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe072a00640 1 -- 172.21.15.60:0/170652673 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0680047a0 con 0x7fe07c0b3e80 2024-09-16T15:48:21.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.572+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c1f8740 con 0x7fe07c0b3e80 2024-09-16T15:48:21.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.573+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c0b3940 con 0x7fe07c0b3e80 2024-09-16T15:48:21.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.575+0000 7fe072a00640 1 -- 172.21.15.60:0/170652673 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7fe068039040 con 0x7fe07c0b3e80 2024-09-16T15:48:21.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.575+0000 7fe072a00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe05c079420 0x7fe05c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.576+0000 7fe072a00640 1 -- 172.21.15.60:0/170652673 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5737+0+0 (secure 0 0 0) 0x7fe0680ba8c0 con 0x7fe07c0b3e80 2024-09-16T15:48:21.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.576+0000 7fe07a800640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe05c079420 0x7fe05c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.576+0000 7fe07a800640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe05c079420 0x7fe05c07b8e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00bb50 tx=0x7fe06c00b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:21.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.580+0000 7fe072a00640 1 -- 172.21.15.60:0/170652673 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe068084010 con 0x7fe07c0b3e80 2024-09-16T15:48:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.811+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe07c0b55d0 con 0x7fe07c0b3e80 2024-09-16T15:48:21.813 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-09-16T15:48:21.812+0000 7f2a11000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:48:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.814+0000 7fe072a00640 1 -- 172.21.15.60:0/170652673 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v124) ==== 161+0+0 (secure 0 0 0) 0x7fe068088a40 con 0x7fe07c0b3e80 2024-09-16T15:48:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:48:21.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.817+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe05c079420 msgr2=0x7fe05c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.817+0000 7fe080a00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe05c079420 0x7fe05c07b8e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00bb50 tx=0x7fe06c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.817+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b3e80 msgr2=0x7fe07c1559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.817+0000 7fe080a00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b3e80 0x7fe07c1559c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fe06802aa90 tx=0x7fe068032c10 comp rx=0 tx=0).stop 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.818+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 shutdown_connections 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.818+0000 7fe080a00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fe05c079420 0x7fe05c07b8e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.818+0000 7fe080a00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07c161440 0x7fe07c0b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.818+0000 7fe080a00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe07c0b3e80 0x7fe07c1559c0 secure :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fe06802aa90 tx=0x7fe068032c10 comp rx=0 tx=0).stop 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.818+0000 7fe080a00640 1 --2- 172.21.15.60:0/170652673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe07c0b34b0 0x7fe07c0b94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.818+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 >> 172.21.15.60:0/170652673 conn(0x7fe07c0d3df0 msgr2=0x7fe07c1605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:21.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.818+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 shutdown_connections 2024-09-16T15:48:21.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:21.818+0000 7fe080a00640 1 -- 172.21.15.60:0/170652673 wait complete. 2024-09-16T15:48:21.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:48:21.887 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-09-16T15:48:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 -- 172.21.15.60:0/2892814622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 msgr2=0x7f993c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 --2- 172.21.15.60:0/2892814622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 0x7f993c06fce0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f993c0724f0 tx=0x7f993002a560 comp rx=0 tx=0).stop 2024-09-16T15:48:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 -- 172.21.15.60:0/2892814622 shutdown_connections 2024-09-16T15:48:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 --2- 172.21.15.60:0/2892814622 >> v2:172.21.15.100:3300/0 conn(0x7f993c0cb4f0 0x7f993c0cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 --2- 172.21.15.60:0/2892814622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f993c072560 0x7f993c073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 --2- 172.21.15.60:0/2892814622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 0x7f993c06fce0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 -- 172.21.15.60:0/2892814622 >> 172.21.15.60:0/2892814622 conn(0x7f993c0cf9f0 msgr2=0x7f993c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 -- 172.21.15.60:0/2892814622 shutdown_connections 2024-09-16T15:48:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.002+0000 7f9942000640 1 -- 172.21.15.60:0/2892814622 wait complete. 2024-09-16T15:48:22.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.004+0000 7f9942000640 1 Processor -- start 2024-09-16T15:48:22.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.004+0000 7f9942000640 1 -- start start 2024-09-16T15:48:22.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9942000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f993c072560 0x7f993c14a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9942000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f993c0cb4f0 0x7f993c0ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9942000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 0x7f993c0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9942000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f993c0ae850 con 0x7f993c15bec0 2024-09-16T15:48:22.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9942000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f993c0ae6d0 con 0x7f993c072560 2024-09-16T15:48:22.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9942000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f993c0ae9d0 con 0x7f993c0cb4f0 2024-09-16T15:48:22.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f993be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f993c0cb4f0 0x7f993c0ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9940c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f993c072560 0x7f993c14a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9941600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 0x7f993c0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9940c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f993c072560 0x7f993c14a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:59708/0 (socket says 172.21.15.60:59708) 2024-09-16T15:48:22.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f993be00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f993c0cb4f0 0x7f993c0ac380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:33874/0 (socket says 172.21.15.60:33874) 2024-09-16T15:48:22.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.005+0000 7f9940c00640 1 -- 172.21.15.60:0/4141075927 learned_addr learned my addr 172.21.15.60:0/4141075927 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f993be00640 1 -- 172.21.15.60:0/4141075927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f993c072560 msgr2=0x7f993c14a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f993be00640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f993c072560 0x7f993c14a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f993be00640 1 -- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 msgr2=0x7f993c0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f993be00640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 0x7f993c0ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f993be00640 1 -- 172.21.15.60:0/4141075927 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c15e1b0 con 0x7f993c0cb4f0 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f9940c00640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f993c072560 0x7f993c14a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f9941600640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 0x7f993c0ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f993be00640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f993c0cb4f0 0x7f993c0ac380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9928009900 tx=0x7f9928009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f9938c00640 1 -- 172.21.15.60:0/4141075927 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928004030 con 0x7f993c0cb4f0 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f9938c00640 1 -- 172.21.15.60:0/4141075927 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99280041f0 con 0x7f993c0cb4f0 2024-09-16T15:48:22.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f9938c00640 1 -- 172.21.15.60:0/4141075927 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928007860 con 0x7f993c0cb4f0 2024-09-16T15:48:22.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c15e490 con 0x7f993c0cb4f0 2024-09-16T15:48:22.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.006+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c15ea50 con 0x7f993c0cb4f0 2024-09-16T15:48:22.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.007+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c0b0000 con 0x7f993c0cb4f0 2024-09-16T15:48:22.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.009+0000 7f9938c00640 1 -- 172.21.15.60:0/4141075927 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f9928007a00 con 0x7f993c0cb4f0 2024-09-16T15:48:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.010+0000 7f9938c00640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f991c079420 0x7f991c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.010+0000 7f9938c00640 1 -- 172.21.15.60:0/4141075927 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(174..174 src has 1..174) ==== 5823+0+0 (secure 0 0 0) 0x7f992809d330 con 0x7f993c0cb4f0 2024-09-16T15:48:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.010+0000 7f9940c00640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f991c079420 0x7f991c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.011+0000 7f9940c00640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f991c079420 0x7f991c07b8e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f99300096c0 tx=0x7f9930004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:22.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.015+0000 7f9938c00640 1 -- 172.21.15.60:0/4141075927 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9928066b40 con 0x7f993c0cb4f0 2024-09-16T15:48:22.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.250+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f993c074050 con 0x7f993c0cb4f0 2024-09-16T15:48:22.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.827+0000 7f9938c00640 1 -- 172.21.15.60:0/4141075927 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v125) ==== 104+0+0 (secure 0 0 0) 0x7f992806b570 con 0x7f993c0cb4f0 2024-09-16T15:48:22.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f991c079420 msgr2=0x7f991c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f991c079420 0x7f991c07b8e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f99300096c0 tx=0x7f9930004730 comp rx=0 tx=0).stop 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f993c0cb4f0 msgr2=0x7f993c0ac380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f993c0cb4f0 0x7f993c0ac380 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9928009900 tx=0x7f9928009dc0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 shutdown_connections 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f991c079420 0x7f991c07b8e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f993c15bec0 0x7f993c0ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f993c0cb4f0 0x7f993c0ac380 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 --2- 172.21.15.60:0/4141075927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f993c072560 0x7f993c14a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 >> 172.21.15.60:0/4141075927 conn(0x7f993c0cf9f0 msgr2=0x7f993c0b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 shutdown_connections 2024-09-16T15:48:22.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:22.829+0000 7f9942000640 1 -- 172.21.15.60:0/4141075927 wait complete. 2024-09-16T15:48:22.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:48:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.013+0000 7f6a28e00640 1 -- 172.21.15.60:0/3447981453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 msgr2=0x7f6a08004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.013+0000 7f6a28e00640 1 --2- 172.21.15.60:0/3447981453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 0x7f6a08004800 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6a10006f70 tx=0x7f6a08005830 comp rx=0 tx=0).stop 2024-09-16T15:48:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.013+0000 7f6a22800640 1 -- 172.21.15.60:0/3447981453 reap_dead start 2024-09-16T15:48:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.013+0000 7f6a28e00640 1 -- 172.21.15.60:0/3447981453 shutdown_connections 2024-09-16T15:48:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.013+0000 7f6a28e00640 1 -- 172.21.15.60:0/3447981453 >> 172.21.15.60:0/3447981453 conn(0x7f6a240b18e0 msgr2=0x7f6a240b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.013+0000 7f6a28e00640 1 -- 172.21.15.60:0/3447981453 shutdown_connections 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.013+0000 7f6a28e00640 1 -- 172.21.15.60:0/3447981453 wait complete. 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.014+0000 7f6a28e00640 1 Processor -- start 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.015+0000 7f6a28e00640 1 -- start start 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.015+0000 7f6a28e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 0x7f6a2415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a28e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a24155e50 0x7f6a2415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a28e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24157780 0x7f6a24157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a28e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f6a240adde0 con 0x7f6a080041c0 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a28e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a240b4b70 con 0x7f6a24157780 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a28e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6a240adf60 con 0x7f6a24155e50 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 0x7f6a2415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 0x7f6a2415ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47286/0 (socket says 172.21.15.60:47286) 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 -- 172.21.15.60:0/2968861758 learned_addr learned my addr 172.21.15.60:0/2968861758 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21400640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a24155e50 0x7f6a2415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a22800640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24157780 0x7f6a24157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 -- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a24155e50 msgr2=0x7f6a2415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a24155e50 0x7f6a2415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 -- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24157780 msgr2=0x7f6a24157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24157780 0x7f6a24157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 -- 172.21.15.60:0/2968861758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a241f8870 con 0x7f6a080041c0 2024-09-16T15:48:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21400640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a24155e50 0x7f6a2415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.016+0000 7f6a21e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 0x7f6a2415ae40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003d80 tx=0x7f6a140017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.017+0000 7f6a1a000640 1 -- 172.21.15.60:0/2968861758 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14011640 con 0x7f6a080041c0 2024-09-16T15:48:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.017+0000 7f6a1a000640 1 -- 172.21.15.60:0/2968861758 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a14016020 con 0x7f6a080041c0 2024-09-16T15:48:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.017+0000 7f6a1a000640 1 -- 172.21.15.60:0/2968861758 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14011a80 con 0x7f6a080041c0 2024-09-16T15:48:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.017+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241f8b50 con 0x7f6a080041c0 2024-09-16T15:48:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.017+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241f9110 con 0x7f6a080041c0 2024-09-16T15:48:23.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.018+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a24069360 con 0x7f6a080041c0 2024-09-16T15:48:23.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.019+0000 7f6a1a000640 1 -- 172.21.15.60:0/2968861758 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f6a14011c20 con 0x7f6a080041c0 2024-09-16T15:48:23.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.020+0000 7f6a1a000640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a04079420 0x7f6a0407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:23.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.020+0000 7f6a1a000640 1 -- 172.21.15.60:0/2968861758 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 5823+0+0 (secure 0 0 0) 0x7f6a14099d80 con 0x7f6a080041c0 2024-09-16T15:48:23.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.020+0000 7f6a21400640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a04079420 0x7f6a0407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:23.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.021+0000 7f6a21400640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a04079420 0x7f6a0407b8e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6a10033000 tx=0x7f6a2406fbd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:23.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.024+0000 7f6a1a000640 1 -- 172.21.15.60:0/2968861758 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6a14020030 con 0x7f6a080041c0 2024-09-16T15:48:23.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.223+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6a241f5020 con 0x7f6a080041c0 2024-09-16T15:48:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.822+0000 7f6a1a000640 1 -- 172.21.15.60:0/2968861758 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) ==== 181+0+0 (secure 0 0 0) 0x7f6a14063e40 con 0x7f6a080041c0 2024-09-16T15:48:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:48:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a04079420 msgr2=0x7f6a0407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a04079420 0x7f6a0407b8e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6a10033000 tx=0x7f6a2406fbd0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 msgr2=0x7f6a2415ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 0x7f6a2415ae40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003d80 tx=0x7f6a140017a0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 shutdown_connections 2024-09-16T15:48:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6a04079420 0x7f6a0407b8e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a24157780 0x7f6a24157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a24155e50 0x7f6a2415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 --2- 172.21.15.60:0/2968861758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a080041c0 0x7f6a2415ae40 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 >> 172.21.15.60:0/2968861758 conn(0x7f6a240b18e0 msgr2=0x7f6a240b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 shutdown_connections 2024-09-16T15:48:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:23.826+0000 7f6a28e00640 1 -- 172.21.15.60:0/2968861758 wait complete. 2024-09-16T15:48:23.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:48:24.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.013+0000 7f5ed1c00640 1 -- 172.21.15.60:0/4155841474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc070380 msgr2=0x7f5ecc0cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.013+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/4155841474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc070380 0x7f5ecc0cb1c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc06f8d0 tx=0x7f5ec402a8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.013+0000 7f5ed1c00640 1 -- 172.21.15.60:0/4155841474 shutdown_connections 2024-09-16T15:48:24.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.013+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/4155841474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc070380 0x7f5ecc0cb1c0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.013+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/4155841474 >> v2:172.21.15.100:3300/0 conn(0x7f5ecc06e590 0x7f5ecc06fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.013+0000 7f5ed1c00640 1 -- 172.21.15.60:0/4155841474 >> 172.21.15.60:0/4155841474 conn(0x7f5ecc0cfa70 msgr2=0x7f5ecc0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:24.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.013+0000 7f5ed1c00640 1 -- 172.21.15.60:0/4155841474 shutdown_connections 2024-09-16T15:48:24.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.013+0000 7f5ed1c00640 1 -- 172.21.15.60:0/4155841474 wait complete. 2024-09-16T15:48:24.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.015+0000 7f5ed1c00640 1 Processor -- start 2024-09-16T15:48:24.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.015+0000 7f5ed1c00640 1 -- start start 2024-09-16T15:48:24.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1c00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ecc06e590 0x7f5ecc1f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:24.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1c00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ecc070380 0x7f5ecc1f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:24.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1c00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc15bda0 0x7f5ecc15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:24.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1c00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0cba90 con 0x7f5ecc15bda0 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1c00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0cb910 con 0x7f5ecc070380 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1c00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc0cbc10 con 0x7f5ecc06e590 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc15bda0 0x7f5ecc15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc15bda0 0x7f5ecc15cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47302/0 (socket says 172.21.15.60:47302) 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ed1200640 1 -- 172.21.15.60:0/1506720670 learned_addr learned my addr 172.21.15.60:0/1506720670 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ec3400640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ecc070380 0x7f5ecc1f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.016+0000 7f5ecbe00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ecc06e590 0x7f5ecc1f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ed1200640 1 -- 172.21.15.60:0/1506720670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ecc06e590 msgr2=0x7f5ecc1f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ed1200640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ecc06e590 0x7f5ecc1f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ed1200640 1 -- 172.21.15.60:0/1506720670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ecc070380 msgr2=0x7f5ecc1f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ed1200640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ecc070380 0x7f5ecc1f8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ed1200640 1 -- 172.21.15.60:0/1506720670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc15d690 con 0x7f5ecc15bda0 2024-09-16T15:48:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ed1200640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc15bda0 0x7f5ecc15cf20 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009ca0 tx=0x7f5ec4031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ec8c00640 1 -- 172.21.15.60:0/1506720670 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4040070 con 0x7f5ecc15bda0 2024-09-16T15:48:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ec8c00640 1 -- 172.21.15.60:0/1506720670 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec4004040 con 0x7f5ecc15bda0 2024-09-16T15:48:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc15d970 con 0x7f5ecc15bda0 2024-09-16T15:48:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ec8c00640 1 -- 172.21.15.60:0/1506720670 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4004470 con 0x7f5ecc15bda0 2024-09-16T15:48:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.017+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc0cddc0 con 0x7f5ecc15bda0 2024-09-16T15:48:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.018+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc0af950 con 0x7f5ecc15bda0 2024-09-16T15:48:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.019+0000 7f5ec8c00640 1 -- 172.21.15.60:0/1506720670 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f5ec4039040 con 0x7f5ecc15bda0 2024-09-16T15:48:24.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.020+0000 7f5ec8c00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ea8079420 0x7f5ea807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.021+0000 7f5ec8c00640 1 -- 172.21.15.60:0/1506720670 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5385+0+0 (secure 0 0 0) 0x7f5ec40baab0 con 0x7f5ecc15bda0 2024-09-16T15:48:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.021+0000 7f5ecbe00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ea8079420 0x7f5ea807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:24.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.021+0000 7f5ecbe00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ea8079420 0x7f5ea807b8e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc007fd0 tx=0x7f5ebc001370 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:24.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.025+0000 7f5ec8c00640 1 -- 172.21.15.60:0/1506720670 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5ec40842c0 con 0x7f5ecc15bda0 2024-09-16T15:48:24.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.220+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5ecc0af670 con 0x7f5ecc15bda0 2024-09-16T15:48:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.828+0000 7f5ec8c00640 1 -- 172.21.15.60:0/1506720670 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) ==== 169+0+0 (secure 0 0 0) 0x7f5ec4088cf0 con 0x7f5ecc15bda0 2024-09-16T15:48:24.830 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ea8079420 msgr2=0x7f5ea807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ea8079420 0x7f5ea807b8e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc007fd0 tx=0x7f5ebc001370 comp rx=0 tx=0).stop 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc15bda0 msgr2=0x7f5ecc15cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc15bda0 0x7f5ecc15cf20 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009ca0 tx=0x7f5ec4031720 comp rx=0 tx=0).stop 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 shutdown_connections 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5ea8079420 0x7f5ea807b8e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ecc15bda0 0x7f5ecc15cf20 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ecc070380 0x7f5ecc1f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 --2- 172.21.15.60:0/1506720670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ecc06e590 0x7f5ecc1f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.833+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 >> 172.21.15.60:0/1506720670 conn(0x7f5ecc0cfa70 msgr2=0x7f5ecc159450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.834+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 shutdown_connections 2024-09-16T15:48:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:24.834+0000 7f5ed1c00640 1 -- 172.21.15.60:0/1506720670 wait complete. 2024-09-16T15:48:24.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:48:25.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.016+0000 7fdf2a800640 1 -- 172.21.15.60:0/1492003355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24072df0 msgr2=0x7fdf240732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.016+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1492003355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24072df0 0x7fdf240732a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005fd0 tx=0x7fdf1c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:48:25.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.016+0000 7fdf2a800640 1 -- 172.21.15.60:0/1492003355 shutdown_connections 2024-09-16T15:48:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.016+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1492003355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24072df0 0x7fdf240732a0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.016+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1492003355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf24161550 0x7fdf2406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.016+0000 7fdf2a800640 1 -- 172.21.15.60:0/1492003355 >> 172.21.15.60:0/1492003355 conn(0x7fdf240b1950 msgr2=0x7fdf240b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.016+0000 7fdf2a800640 1 -- 172.21.15.60:0/1492003355 shutdown_connections 2024-09-16T15:48:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.016+0000 7fdf2a800640 1 -- 172.21.15.60:0/1492003355 wait complete. 2024-09-16T15:48:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.018+0000 7fdf2a800640 1 Processor -- start 2024-09-16T15:48:25.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.018+0000 7fdf2a800640 1 -- start start 2024-09-16T15:48:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf2a800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf24072df0 0x7fdf240c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf2a800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf240af3d0 0x7fdf240cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf2a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24161550 0x7fdf24151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf2a800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fdf240737e0 con 0x7fdf24161550 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf2a800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2406fdb0 con 0x7fdf240af3d0 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf2a800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fdf24073960 con 0x7fdf24072df0 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24161550 0x7fdf24151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24161550 0x7fdf24151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47326/0 (socket says 172.21.15.60:47326) 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23e00640 1 -- 172.21.15.60:0/1624044267 learned_addr learned my addr 172.21.15.60:0/1624044267 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf1aa00640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf240af3d0 0x7fdf240cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23400640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf24072df0 0x7fdf240c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23e00640 1 -- 172.21.15.60:0/1624044267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf24072df0 msgr2=0x7fdf240c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23e00640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf24072df0 0x7fdf240c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23e00640 1 -- 172.21.15.60:0/1624044267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf240af3d0 msgr2=0x7fdf240cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23e00640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf240af3d0 0x7fdf240cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.019+0000 7fdf23e00640 1 -- 172.21.15.60:0/1624044267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf240cc150 con 0x7fdf24161550 2024-09-16T15:48:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.020+0000 7fdf23e00640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24161550 0x7fdf24151c60 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006010 tx=0x7fdf1c003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.020+0000 7fdf1be00640 1 -- 172.21.15.60:0/1624044267 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c047070 con 0x7fdf24161550 2024-09-16T15:48:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.020+0000 7fdf1be00640 1 -- 172.21.15.60:0/1624044267 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf1c02f670 con 0x7fdf24161550 2024-09-16T15:48:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.020+0000 7fdf1be00640 1 -- 172.21.15.60:0/1624044267 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c033050 con 0x7fdf24161550 2024-09-16T15:48:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.020+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf240cc430 con 0x7fdf24161550 2024-09-16T15:48:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.020+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf240cc9f0 con 0x7fdf24161550 2024-09-16T15:48:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.021+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf240b3030 con 0x7fdf24161550 2024-09-16T15:48:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.022+0000 7fdf1be00640 1 -- 172.21.15.60:0/1624044267 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7fdf1c04a070 con 0x7fdf24161550 2024-09-16T15:48:25.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.023+0000 7fdf1be00640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf00079420 0x7fdf0007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.023+0000 7fdf1be00640 1 -- 172.21.15.60:0/1624044267 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7fdf1c0ba3a0 con 0x7fdf24161550 2024-09-16T15:48:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.024+0000 7fdf23400640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf00079420 0x7fdf0007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.024+0000 7fdf23400640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf00079420 0x7fdf0007b8e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009b30 tx=0x7fdf14005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.028+0000 7fdf1be00640 1 -- 172.21.15.60:0/1624044267 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdf240b3030 con 0x7fdf24161550 2024-09-16T15:48:25.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.226+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fdf24074390 con 0x7fdf24161550 2024-09-16T15:48:25.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.230+0000 7fdf1be00640 1 -- 172.21.15.60:0/1624044267 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v49) ==== 71+0+0 (secure 0 0 0) 0x7fdf1c083e10 con 0x7fdf24161550 2024-09-16T15:48:25.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf00079420 msgr2=0x7fdf0007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf00079420 0x7fdf0007b8e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009b30 tx=0x7fdf14005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24161550 msgr2=0x7fdf24151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24161550 0x7fdf24151c60 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006010 tx=0x7fdf1c003ef0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 shutdown_connections 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fdf00079420 0x7fdf0007b8e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf24161550 0x7fdf24151c60 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf240af3d0 0x7fdf240cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 --2- 172.21.15.60:0/1624044267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf24072df0 0x7fdf240c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 >> 172.21.15.60:0/1624044267 conn(0x7fdf240b1950 msgr2=0x7fdf2406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 shutdown_connections 2024-09-16T15:48:25.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.233+0000 7fdf2a800640 1 -- 172.21.15.60:0/1624044267 wait complete. 2024-09-16T15:48:25.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-16T15:48:25.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.411+0000 7f7b91000640 1 -- 172.21.15.60:0/2446524045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b70004800 msgr2=0x7f7b70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.411+0000 7f7b91000640 1 --2- 172.21.15.60:0/2446524045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b70004800 0x7f7b70004c80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400c290 tx=0x7f7b70005830 comp rx=0 tx=0).stop 2024-09-16T15:48:25.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.411+0000 7f7b8a800640 1 -- 172.21.15.60:0/2446524045 reap_dead start 2024-09-16T15:48:25.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.411+0000 7f7b91000640 1 -- 172.21.15.60:0/2446524045 shutdown_connections 2024-09-16T15:48:25.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.411+0000 7f7b91000640 1 -- 172.21.15.60:0/2446524045 >> 172.21.15.60:0/2446524045 conn(0x7f7b8c0b1950 msgr2=0x7f7b8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.411+0000 7f7b91000640 1 -- 172.21.15.60:0/2446524045 shutdown_connections 2024-09-16T15:48:25.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.411+0000 7f7b91000640 1 -- 172.21.15.60:0/2446524045 wait complete. 2024-09-16T15:48:25.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.412+0000 7f7b91000640 1 Processor -- start 2024-09-16T15:48:25.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.413+0000 7f7b91000640 1 -- start start 2024-09-16T15:48:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.413+0000 7f7b91000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b70004800 0x7f7b8c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.413+0000 7f7b91000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8c1f6260 0x7f7b8c1f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.413+0000 7f7b91000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8c15e630 0x7f7b8c15ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.413+0000 7f7b91000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c0cf130 con 0x7f7b8c15e630 2024-09-16T15:48:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.413+0000 7f7b91000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c073bc0 con 0x7f7b70004800 2024-09-16T15:48:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.413+0000 7f7b91000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8c0cf2b0 con 0x7f7b8c1f6260 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b8a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8c15e630 0x7f7b8c15ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b8a800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8c15e630 0x7f7b8c15ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47334/0 (socket says 172.21.15.60:47334) 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b8a800640 1 -- 172.21.15.60:0/2121123107 learned_addr learned my addr 172.21.15.60:0/2121123107 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b89e00640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b70004800 0x7f7b8c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b81400640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8c1f6260 0x7f7b8c1f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b8a800640 1 -- 172.21.15.60:0/2121123107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8c1f6260 msgr2=0x7f7b8c1f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b8a800640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8c1f6260 0x7f7b8c1f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b8a800640 1 -- 172.21.15.60:0/2121123107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b70004800 msgr2=0x7f7b8c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b8a800640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b70004800 0x7f7b8c0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.414+0000 7f7b8a800640 1 -- 172.21.15.60:0/2121123107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8c13d690 con 0x7f7b8c15e630 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.415+0000 7f7b8a800640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8c15e630 0x7f7b8c15ea50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c074850 tx=0x7f7b84011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.415+0000 7f7b82a00640 1 -- 172.21.15.60:0/2121123107 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400f8f0 con 0x7f7b8c15e630 2024-09-16T15:48:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.415+0000 7f7b82a00640 1 -- 172.21.15.60:0/2121123107 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b8400fab0 con 0x7f7b8c15e630 2024-09-16T15:48:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.415+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8c13dc00 con 0x7f7b8c15e630 2024-09-16T15:48:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.415+0000 7f7b82a00640 1 -- 172.21.15.60:0/2121123107 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84005020 con 0x7f7b8c15e630 2024-09-16T15:48:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.415+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8c13e1c0 con 0x7f7b8c15e630 2024-09-16T15:48:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.416+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8c0cebf0 con 0x7f7b8c15e630 2024-09-16T15:48:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.417+0000 7f7b82a00640 1 -- 172.21.15.60:0/2121123107 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f7b840051c0 con 0x7f7b8c15e630 2024-09-16T15:48:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.418+0000 7f7b82a00640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b68079420 0x7f7b6807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.418+0000 7f7b82a00640 1 -- 172.21.15.60:0/2121123107 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f7b84099c40 con 0x7f7b8c15e630 2024-09-16T15:48:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.418+0000 7f7b89e00640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b68079420 0x7f7b6807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.419+0000 7f7b89e00640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b68079420 0x7f7b6807b8e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0097f0 tx=0x7f7b7c009500 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.422+0000 7f7b82a00640 1 -- 172.21.15.60:0/2121123107 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7b84011e20 con 0x7f7b8c15e630 2024-09-16T15:48:25.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.688+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f7b8c13e390 con 0x7f7b8c15e630 2024-09-16T15:48:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.830+0000 7f7b82a00640 1 -- 172.21.15.60:0/2121123107 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v541) ==== 163+0+0 (secure 0 0 0) 0x7f7b84063450 con 0x7f7b8c15e630 2024-09-16T15:48:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b68079420 msgr2=0x7f7b6807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b68079420 0x7f7b6807b8e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0097f0 tx=0x7f7b7c009500 comp rx=0 tx=0).stop 2024-09-16T15:48:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8c15e630 msgr2=0x7f7b8c15ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8c15e630 0x7f7b8c15ea50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c074850 tx=0x7f7b84011bd0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 shutdown_connections 2024-09-16T15:48:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f7b68079420 0x7f7b6807b8e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8c15e630 0x7f7b8c15ea50 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8c1f6260 0x7f7b8c1f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 --2- 172.21.15.60:0/2121123107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b70004800 0x7f7b8c0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 >> 172.21.15.60:0/2121123107 conn(0x7f7b8c0b1950 msgr2=0x7f7b8c0aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 shutdown_connections 2024-09-16T15:48:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:25.833+0000 7f7b91000640 1 -- 172.21.15.60:0/2121123107 wait complete. 2024-09-16T15:48:25.850 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-16T15:48:25.850 INFO:tasks.cephfs_test_runner: 2024-09-16T15:48:25.850 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-16T15:48:25.850 INFO:tasks.cephfs_test_runner:Ran 11 tests in 560.302s 2024-09-16T15:48:25.850 INFO:tasks.cephfs_test_runner: 2024-09-16T15:48:25.850 INFO:tasks.cephfs_test_runner:OK 2024-09-16T15:48:25.851 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-16T15:48:25.860 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T15:48:25.880 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T15:48:25.880 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:48:25.880 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:25.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:48:25.915 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-16T15:48:25.915 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:25.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:48:25.999 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:48:25.999 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:26.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:48:26.039 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-16T15:48:26.039 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:26.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052587,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248952k,nr_inodes=812238,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497908k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:48:26.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:26.117 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:48:26.153 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:48:26.153 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:48:26.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:26.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:48:26.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:26.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:48:26.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:26.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-09-16T15:48:26.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:26.819 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-09-16T15:48:26.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:26.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-09-16T15:48:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2024-09-16T15:48:26.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:48:26.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:26.958 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:48:26.998 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:48:26.998 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:48:26.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:26.999 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:48:27.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:27.098 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:48:27.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:27.337 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-16T15:48:27.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:27.644 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-16T15:48:27.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:27.679 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-16T15:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-09-16T15:48:27.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:48:27.781 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T15:48:27.790 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:48:27.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:27.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.957+0000 7f5809e00640 1 -- 172.21.15.60:0/1468061899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 msgr2=0x7f57e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.957+0000 7f5809e00640 1 --2- 172.21.15.60:0/1468061899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 0x7f57e800b9c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e3d0 tx=0x7f57e8005830 comp rx=0 tx=0).stop 2024-09-16T15:48:27.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.957+0000 7f5803400640 1 -- 172.21.15.60:0/1468061899 reap_dead start 2024-09-16T15:48:27.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.957+0000 7f5809e00640 1 -- 172.21.15.60:0/1468061899 shutdown_connections 2024-09-16T15:48:27.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.957+0000 7f5809e00640 1 -- 172.21.15.60:0/1468061899 >> 172.21.15.60:0/1468061899 conn(0x7f58040b1950 msgr2=0x7f58040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.957+0000 7f5809e00640 1 -- 172.21.15.60:0/1468061899 shutdown_connections 2024-09-16T15:48:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.957+0000 7f5809e00640 1 -- 172.21.15.60:0/1468061899 wait complete. 2024-09-16T15:48:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.958+0000 7f5809e00640 1 Processor -- start 2024-09-16T15:48:27.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.959+0000 7f5809e00640 1 -- start start 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.959+0000 7f5809e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 0x7f58040c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.959+0000 7f5809e00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58041f7c70 0x7f58041f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.959+0000 7f5809e00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58040cadd0 0x7f58040cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5809e00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f58040b25c0 con 0x7f57e80051c0 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5809e00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58040b2440 con 0x7f58040cadd0 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5809e00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f58040b2740 con 0x7f58041f7c70 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5802a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 0x7f58040c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5802a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 0x7f58040c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47356/0 (socket says 172.21.15.60:47356) 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5802a00640 1 -- 172.21.15.60:0/3271638650 learned_addr learned my addr 172.21.15.60:0/3271638650 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f57fa000640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58041f7c70 0x7f58041f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5803400640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58040cadd0 0x7f58040cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5802a00640 1 -- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58041f7c70 msgr2=0x7f58041f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5802a00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58041f7c70 0x7f58041f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5802a00640 1 -- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58040cadd0 msgr2=0x7f58040cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5802a00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58040cadd0 0x7f58040cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5802a00640 1 -- 172.21.15.60:0/3271638650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58040cbae0 con 0x7f57e80051c0 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.960+0000 7f5803400640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58040cadd0 0x7f58040cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.961+0000 7f5802a00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 0x7f58040c9470 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f57f4011ab0 tx=0x7f57f4011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.961+0000 7f57fbe00640 1 -- 172.21.15.60:0/3271638650 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f401e070 con 0x7f57e80051c0 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.961+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58040cbdc0 con 0x7f57e80051c0 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.961+0000 7f57fbe00640 1 -- 172.21.15.60:0/3271638650 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57f400fdc0 con 0x7f57e80051c0 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.961+0000 7f57fbe00640 1 -- 172.21.15.60:0/3271638650 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4019650 con 0x7f57e80051c0 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.961+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580409baf0 con 0x7f57e80051c0 2024-09-16T15:48:27.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.962+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5804074460 con 0x7f57e80051c0 2024-09-16T15:48:27.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.963+0000 7f57fbe00640 1 -- 172.21.15.60:0/3271638650 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f57f4021070 con 0x7f57e80051c0 2024-09-16T15:48:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.964+0000 7f57fbe00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f57e0079420 0x7f57e007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.964+0000 7f57fbe00640 1 -- 172.21.15.60:0/3271638650 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f57f409a0e0 con 0x7f57e80051c0 2024-09-16T15:48:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.964+0000 7f57fa000640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f57e0079420 0x7f57e007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.965+0000 7f57fa000640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f57e0079420 0x7f57e007b8e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f58040cabc0 tx=0x7f57ec001cb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:27.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:27.968+0000 7f57fbe00640 1 -- 172.21.15.60:0/3271638650 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f57f4063870 con 0x7f57e80051c0 2024-09-16T15:48:28.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.159+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58040b4940 con 0x7f57e0079420 2024-09-16T15:48:28.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.160+0000 7f57fbe00640 1 -- 172.21.15.60:0/3271638650 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13504 (secure 0 0 0) 0x7f58040b4940 con 0x7f57e0079420 2024-09-16T15:48:28.161 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:28.162 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f57e0079420 msgr2=0x7f57e007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f57e0079420 0x7f57e007b8e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f58040cabc0 tx=0x7f57ec001cb0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 msgr2=0x7f58040c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 0x7f58040c9470 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f57f4011ab0 tx=0x7f57f4011f70 comp rx=0 tx=0).stop 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 shutdown_connections 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f57e0079420 0x7f57e007b8e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58040cadd0 0x7f58040cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58041f7c70 0x7f58041f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 --2- 172.21.15.60:0/3271638650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57e80051c0 0x7f58040c9470 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.162+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 >> 172.21.15.60:0/3271638650 conn(0x7f58040b1950 msgr2=0x7f58040b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.163+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 shutdown_connections 2024-09-16T15:48:28.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.163+0000 7f5809e00640 1 -- 172.21.15.60:0/3271638650 wait complete. 2024-09-16T15:48:28.180 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-16T15:48:26.228884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284456,"kb_used_data":18340,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690264,"statfs":{"total":773094113280,"available":772802830336,"internally_reserved":0,"allocated":18780160,"data_stored":8988791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-51301,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-65,"num_write_kb":-73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001190"},"pg_stats":[{"pgid":"1.0","version":"75'79","reported_seq":416,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-16T15:48:06.182532+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:48:06.182532+0000","last_peered":"2024-09-16T15:48:06.182532+0000","last_clean":"2024-09-16T15:48:06.182532+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:48:06.182532+0000","last_undegraded":"2024-09-16T15:48:06.182532+0000","last_fullsized":"2024-09-16T15:48:06.182532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33952,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337888,"statfs":{"total":96636764160,"available":96601997312,"internally_reserved":0,"allocated":2834432,"data_stored":1606328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2195456,"data_stored":968298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509562,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":961886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509562,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33384,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338456,"statfs":{"total":96636764160,"available":96602578944,"internally_reserved":0,"allocated":2252800,"data_stored":1016367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37480,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334360,"statfs":{"total":96636764160,"available":96598384640,"internally_reserved":0,"allocated":2187264,"data_stored":961486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509562,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37468,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334372,"statfs":{"total":96636764160,"available":96598396928,"internally_reserved":0,"allocated":2174976,"data_stored":961304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542268,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37476,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334364,"statfs":{"total":96636764160,"available":96598388736,"internally_reserved":0,"allocated":2183168,"data_stored":961360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509564,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2772992,"data_stored":1551762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:28.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.341+0000 7faca5800640 1 -- 172.21.15.60:0/632664386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca00ccf00 msgr2=0x7faca00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.341+0000 7faca5800640 1 --2- 172.21.15.60:0/632664386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca00ccf00 0x7faca00b2440 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fac8c005fd0 tx=0x7fac8c02a540 comp rx=0 tx=0).stop 2024-09-16T15:48:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.341+0000 7faca5800640 1 -- 172.21.15.60:0/632664386 shutdown_connections 2024-09-16T15:48:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.341+0000 7faca5800640 1 --2- 172.21.15.60:0/632664386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca00ccf00 0x7faca00b2440 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.341+0000 7faca5800640 1 --2- 172.21.15.60:0/632664386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca0073fb0 0x7faca0074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.341+0000 7faca5800640 1 -- 172.21.15.60:0/632664386 >> 172.21.15.60:0/632664386 conn(0x7faca00b1950 msgr2=0x7faca00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.341+0000 7faca5800640 1 -- 172.21.15.60:0/632664386 shutdown_connections 2024-09-16T15:48:28.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.341+0000 7faca5800640 1 -- 172.21.15.60:0/632664386 wait complete. 2024-09-16T15:48:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.342+0000 7faca5800640 1 Processor -- start 2024-09-16T15:48:28.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.343+0000 7faca5800640 1 -- start start 2024-09-16T15:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7faca5800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca0073fb0 0x7faca00cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7faca5800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca00ccf00 0x7faca01585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7faca5800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca015f5a0 0x7faca0158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7faca5800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7faca006f2e0 con 0x7faca015f5a0 2024-09-16T15:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7faca5800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faca006f160 con 0x7faca00ccf00 2024-09-16T15:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7faca5800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7faca006f460 con 0x7faca0073fb0 2024-09-16T15:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca015f5a0 0x7faca0158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:28.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9f400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca015f5a0 0x7faca0158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47380/0 (socket says 172.21.15.60:47380) 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9f400640 1 -- 172.21.15.60:0/3631869306 learned_addr learned my addr 172.21.15.60:0/3631869306 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9ea00640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca0073fb0 0x7faca00cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9e000640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca00ccf00 0x7faca01585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9f400640 1 -- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca0073fb0 msgr2=0x7faca00cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9f400640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca0073fb0 0x7faca00cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9f400640 1 -- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca00ccf00 msgr2=0x7faca01585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9f400640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca00ccf00 0x7faca01585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9f400640 1 -- 172.21.15.60:0/3631869306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca00cfdf0 con 0x7faca015f5a0 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.344+0000 7fac9e000640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca00ccf00 0x7faca01585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.345+0000 7fac9f400640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca015f5a0 0x7faca0158b10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fac9800eb10 tx=0x7fac9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.345+0000 7fac96a00640 1 -- 172.21.15.60:0/3631869306 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98004b20 con 0x7faca015f5a0 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.345+0000 7fac96a00640 1 -- 172.21.15.60:0/3631869306 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac98004ce0 con 0x7faca015f5a0 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.345+0000 7fac96a00640 1 -- 172.21.15.60:0/3631869306 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98005280 con 0x7faca015f5a0 2024-09-16T15:48:28.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.345+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca00d00d0 con 0x7faca015f5a0 2024-09-16T15:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.345+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca00cf7e0 con 0x7faca015f5a0 2024-09-16T15:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.346+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca006f980 con 0x7faca015f5a0 2024-09-16T15:48:28.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.347+0000 7fac96a00640 1 -- 172.21.15.60:0/3631869306 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7fac98005420 con 0x7faca015f5a0 2024-09-16T15:48:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.349+0000 7fac96a00640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac80079420 0x7fac8007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.349+0000 7fac96a00640 1 -- 172.21.15.60:0/3631869306 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7fac98099fa0 con 0x7faca015f5a0 2024-09-16T15:48:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.349+0000 7fac9ea00640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac80079420 0x7fac8007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:28.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.350+0000 7fac9ea00640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac80079420 0x7fac8007b8e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac90001720 tx=0x7fac90009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:28.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.353+0000 7fac96a00640 1 -- 172.21.15.60:0/3631869306 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fac98064060 con 0x7faca015f5a0 2024-09-16T15:48:28.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.547+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faca00cfc40 con 0x7fac80079420 2024-09-16T15:48:28.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.547+0000 7fac96a00640 1 -- 172.21.15.60:0/3631869306 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7faca00cfc40 con 0x7fac80079420 2024-09-16T15:48:28.549 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:48:28.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac80079420 msgr2=0x7fac8007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac80079420 0x7fac8007b8e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac90001720 tx=0x7fac90009040 comp rx=0 tx=0).stop 2024-09-16T15:48:28.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca015f5a0 msgr2=0x7faca0158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca015f5a0 0x7faca0158b10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fac9800eb10 tx=0x7fac9800c2d0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 shutdown_connections 2024-09-16T15:48:28.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fac80079420 0x7fac8007b8e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca015f5a0 0x7faca0158b10 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca00ccf00 0x7faca01585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 --2- 172.21.15.60:0/3631869306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca0073fb0 0x7faca00cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 >> 172.21.15.60:0/3631869306 conn(0x7faca00b1950 msgr2=0x7faca015e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:28.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 shutdown_connections 2024-09-16T15:48:28.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.550+0000 7faca5800640 1 -- 172.21.15.60:0/3631869306 wait complete. 2024-09-16T15:48:28.566 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-16T15:48:28.229158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":284672,"kb_used_data":18504,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690048,"statfs":{"total":773094113280,"available":772802609152,"internally_reserved":0,"allocated":18948096,"data_stored":9013032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-51301,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-65,"num_write_kb":-73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.413119"},"pg_stats":[{"pgid":"1.0","version":"75'79","reported_seq":421,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-16T15:48:24.882592+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:48:24.882592+0000","last_peered":"2024-09-16T15:48:24.882592+0000","last_clean":"2024-09-16T15:48:24.882592+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:48:24.882592+0000","last_undegraded":"2024-09-16T15:48:24.882592+0000","last_fullsized":"2024-09-16T15:48:24.882592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33936,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337904,"statfs":{"total":96636764160,"available":96602013696,"internally_reserved":0,"allocated":2813952,"data_stored":1569405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509563,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33356,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338484,"statfs":{"total":96636764160,"available":96602607616,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509565,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38100,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333740,"statfs":{"total":96636764160,"available":96597749760,"internally_reserved":0,"allocated":2813952,"data_stored":1569405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:28.567 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:48:28.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.726+0000 7f40eee00640 1 -- 172.21.15.60:0/774068943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e815f5a0 msgr2=0x7f40e80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.726+0000 7f40eee00640 1 --2- 172.21.15.60:0/774068943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e815f5a0 0x7f40e80b2440 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f40e806dac0 tx=0x7f40d802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.726+0000 7f40eee00640 1 -- 172.21.15.60:0/774068943 shutdown_connections 2024-09-16T15:48:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.726+0000 7f40eee00640 1 --2- 172.21.15.60:0/774068943 >> v2:172.21.15.100:3300/0 conn(0x7f40e8073fb0 0x7f40e8074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.726+0000 7f40eee00640 1 --2- 172.21.15.60:0/774068943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e815f5a0 0x7f40e80b2440 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.726+0000 7f40eee00640 1 -- 172.21.15.60:0/774068943 >> 172.21.15.60:0/774068943 conn(0x7f40e80b1950 msgr2=0x7f40e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.726+0000 7f40eee00640 1 -- 172.21.15.60:0/774068943 shutdown_connections 2024-09-16T15:48:28.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.726+0000 7f40eee00640 1 -- 172.21.15.60:0/774068943 wait complete. 2024-09-16T15:48:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.728+0000 7f40eee00640 1 Processor -- start 2024-09-16T15:48:28.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.728+0000 7f40eee00640 1 -- start start 2024-09-16T15:48:28.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40eee00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e8073fb0 0x7f40e80ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40eee00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e80ccf00 0x7f40e814e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40eee00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815f5a0 0x7f40e814e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40eee00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f40e806f2e0 con 0x7f40e80ccf00 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40eee00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40e806f160 con 0x7f40e815f5a0 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40eee00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f40e806f460 con 0x7f40e8073fb0 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40e7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e80ccf00 0x7f40e814e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40e7400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e80ccf00 0x7f40e814e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47392/0 (socket says 172.21.15.60:47392) 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40e7400640 1 -- 172.21.15.60:0/3886661612 learned_addr learned my addr 172.21.15.60:0/3886661612 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40e7e00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e8073fb0 0x7f40e80ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.729+0000 7f40ece00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815f5a0 0x7f40e814e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40e7400640 1 -- 172.21.15.60:0/3886661612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e8073fb0 msgr2=0x7f40e80ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40e7400640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e8073fb0 0x7f40e80ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40e7400640 1 -- 172.21.15.60:0/3886661612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815f5a0 msgr2=0x7f40e814e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40e7400640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815f5a0 0x7f40e814e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40e7400640 1 -- 172.21.15.60:0/3886661612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e81f5680 con 0x7f40e80ccf00 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40e7400640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e80ccf00 0x7f40e814e2b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c480 tx=0x7f40d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40dfe00640 1 -- 172.21.15.60:0/3886661612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4016020 con 0x7f40e80ccf00 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40dfe00640 1 -- 172.21.15.60:0/3886661612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40d4012920 con 0x7f40e80ccf00 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40dfe00640 1 -- 172.21.15.60:0/3886661612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4012d70 con 0x7f40e80ccf00 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.730+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e81f5c60 con 0x7f40e80ccf00 2024-09-16T15:48:28.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.731+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e81f6220 con 0x7f40e80ccf00 2024-09-16T15:48:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.732+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e806f9f0 con 0x7f40e80ccf00 2024-09-16T15:48:28.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.733+0000 7f40dfe00640 1 -- 172.21.15.60:0/3886661612 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f40d4019020 con 0x7f40e80ccf00 2024-09-16T15:48:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.734+0000 7f40dfe00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c8079420 0x7f40c807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.734+0000 7f40dfe00640 1 -- 172.21.15.60:0/3886661612 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f40d40994b0 con 0x7f40e80ccf00 2024-09-16T15:48:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.734+0000 7f40e7e00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c8079420 0x7f40c807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:28.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.735+0000 7f40e7e00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c8079420 0x7f40c807b8e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f40d8034000 tx=0x7f40d8004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:28.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.738+0000 7f40dfe00640 1 -- 172.21.15.60:0/3886661612 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f40d4020020 con 0x7f40e80ccf00 2024-09-16T15:48:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.930+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40e80cf210 con 0x7f40c8079420 2024-09-16T15:48:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.931+0000 7f40dfe00640 1 -- 172.21.15.60:0/3886661612 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7f40e80cf210 con 0x7f40c8079420 2024-09-16T15:48:28.932 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:48:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c8079420 msgr2=0x7f40c807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c8079420 0x7f40c807b8e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f40d8034000 tx=0x7f40d8004730 comp rx=0 tx=0).stop 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e80ccf00 msgr2=0x7f40e814e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e80ccf00 0x7f40e814e2b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f40d400c480 tx=0x7f40d400c940 comp rx=0 tx=0).stop 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 shutdown_connections 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f40c8079420 0x7f40c807b8e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40e815f5a0 0x7f40e814e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40e80ccf00 0x7f40e814e2b0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 --2- 172.21.15.60:0/3886661612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40e8073fb0 0x7f40e80ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 >> 172.21.15.60:0/3886661612 conn(0x7f40e80b1950 msgr2=0x7f40e80ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 shutdown_connections 2024-09-16T15:48:28.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:28.933+0000 7f40eee00640 1 -- 172.21.15.60:0/3886661612 wait complete. 2024-09-16T15:48:28.950 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-16T15:48:28.229158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":284672,"kb_used_data":18504,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690048,"statfs":{"total":773094113280,"available":772802609152,"internally_reserved":0,"allocated":18948096,"data_stored":9013032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-51301,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-65,"num_write_kb":-73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.413119"},"pg_stats":[{"pgid":"1.0","version":"75'79","reported_seq":421,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-16T15:48:24.882592+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:48:24.882592+0000","last_peered":"2024-09-16T15:48:24.882592+0000","last_clean":"2024-09-16T15:48:24.882592+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:48:24.882592+0000","last_undegraded":"2024-09-16T15:48:24.882592+0000","last_fullsized":"2024-09-16T15:48:24.882592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509565,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33936,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337904,"statfs":{"total":96636764160,"available":96602013696,"internally_reserved":0,"allocated":2813952,"data_stored":1569405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509563,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33356,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338484,"statfs":{"total":96636764160,"available":96602607616,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542269,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509565,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38100,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333740,"statfs":{"total":96636764160,"available":96597749760,"internally_reserved":0,"allocated":2813952,"data_stored":1569405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:28.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.109+0000 7ff7ed000640 1 -- 172.21.15.60:0/2870083255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e815f5a0 msgr2=0x7ff7e80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.109+0000 7ff7ed000640 1 --2- 172.21.15.60:0/2870083255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e815f5a0 0x7ff7e80b2440 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009830 tx=0x7ff7d802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.110+0000 7ff7ed000640 1 -- 172.21.15.60:0/2870083255 shutdown_connections 2024-09-16T15:48:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.110+0000 7ff7ed000640 1 --2- 172.21.15.60:0/2870083255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e815f5a0 0x7ff7e80b2440 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.110+0000 7ff7ed000640 1 -- 172.21.15.60:0/2870083255 >> 172.21.15.60:0/2870083255 conn(0x7ff7e80b1950 msgr2=0x7ff7e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.110+0000 7ff7ed000640 1 -- 172.21.15.60:0/2870083255 shutdown_connections 2024-09-16T15:48:29.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.110+0000 7ff7ed000640 1 -- 172.21.15.60:0/2870083255 wait complete. 2024-09-16T15:48:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.111+0000 7ff7ed000640 1 Processor -- start 2024-09-16T15:48:29.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.111+0000 7ff7ed000640 1 -- start start 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.112+0000 7ff7ed000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e8073fb0 0x7ff7e8143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.112+0000 7ff7ed000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e80ccf00 0x7ff7e809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.112+0000 7ff7ed000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7e815f5a0 0x7ff7e809bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.112+0000 7ff7ed000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e806f9f0 con 0x7ff7e80ccf00 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.112+0000 7ff7ed000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e806f870 con 0x7ff7e8073fb0 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.112+0000 7ff7ed000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e8074420 con 0x7ff7e815f5a0 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e80ccf00 0x7ff7e809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e80ccf00 0x7ff7e809ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52750/0 (socket says 172.21.15.60:52750) 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 -- 172.21.15.60:0/139138661 learned_addr learned my addr 172.21.15.60:0/139138661 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7e5e00640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e8073fb0 0x7ff7e8143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7e6800640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7e815f5a0 0x7ff7e809bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 -- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7e815f5a0 msgr2=0x7ff7e809bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7e815f5a0 0x7ff7e809bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 -- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e8073fb0 msgr2=0x7ff7e8143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e8073fb0 0x7ff7e8143fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 -- 172.21.15.60:0/139138661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e80cc070 con 0x7ff7e80ccf00 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7e5e00640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e8073fb0 0x7ff7e8143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.113+0000 7ff7dd400640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e80ccf00 0x7ff7e809ba50 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c700 tx=0x7ff7d000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.114+0000 7ff7dea00640 1 -- 172.21.15.60:0/139138661 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0016020 con 0x7ff7e80ccf00 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.114+0000 7ff7dea00640 1 -- 172.21.15.60:0/139138661 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7d000a930 con 0x7ff7e80ccf00 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.114+0000 7ff7dea00640 1 -- 172.21.15.60:0/139138661 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000ad40 con 0x7ff7e80ccf00 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.114+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e8144a40 con 0x7ff7e80ccf00 2024-09-16T15:48:29.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.114+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e8145000 con 0x7ff7e80ccf00 2024-09-16T15:48:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.115+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e80b2440 con 0x7ff7e80ccf00 2024-09-16T15:48:29.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.116+0000 7ff7dea00640 1 -- 172.21.15.60:0/139138661 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7ff7d0019020 con 0x7ff7e80ccf00 2024-09-16T15:48:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.117+0000 7ff7dea00640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7c4079420 0x7ff7c407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.117+0000 7ff7dea00640 1 -- 172.21.15.60:0/139138661 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7ff7d00996e0 con 0x7ff7e80ccf00 2024-09-16T15:48:29.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.117+0000 7ff7e5e00640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7c4079420 0x7ff7c407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:29.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.118+0000 7ff7e5e00640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7c4079420 0x7ff7c407b8e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8034000 tx=0x7ff7d8033040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:29.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.121+0000 7ff7dea00640 1 -- 172.21.15.60:0/139138661 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff7d0020020 con 0x7ff7e80ccf00 2024-09-16T15:48:29.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.318+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7e81451d0 con 0x7ff7e80ccf00 2024-09-16T15:48:29.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.319+0000 7ff7dea00640 1 -- 172.21.15.60:0/139138661 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) ==== 74+0+13084 (secure 0 0 0) 0x7ff7d0063540 con 0x7ff7e80ccf00 2024-09-16T15:48:29.320 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:29.320 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":176,"fsid":"b577f92e-7c09-4d32-a34c-b870818f4d85","created":"2024-09-16T15:38:08.527485+0000","modified":"2024-09-16T15:48:24.823110+0000","last_up_change":"2024-09-16T15:38:22.500021+0000","last_in_change":"2024-09-16T15:38:12.722418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:38:23.733132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0de59e14-e6eb-4c11-8c21-84177ef27af2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6817","nonce":841472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6819","nonce":841472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6823","nonce":841472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":841472238},{"type":"v1","addr":"172.21.15.60:6821","nonce":841472238}]},"public_addr":"172.21.15.60:6817/841472238","cluster_addr":"172.21.15.60:6819/841472238","heartbeat_back_addr":"172.21.15.60:6823/841472238","heartbeat_front_addr":"172.21.15.60:6821/841472238","state":["exists","up"]},{"osd":1,"uuid":"8a091819-c752-4c9d-87dd-3df656f8eb0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6801","nonce":910651751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6803","nonce":910651751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6807","nonce":910651751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":910651751},{"type":"v1","addr":"172.21.15.60:6805","nonce":910651751}]},"public_addr":"172.21.15.60:6801/910651751","cluster_addr":"172.21.15.60:6803/910651751","heartbeat_back_addr":"172.21.15.60:6807/910651751","heartbeat_front_addr":"172.21.15.60:6805/910651751","state":["exists","up"]},{"osd":2,"uuid":"cd71dc0c-6837-4b8c-8380-6000e5cff822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6809","nonce":3414864047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6811","nonce":3414864047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6815","nonce":3414864047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":3414864047},{"type":"v1","addr":"172.21.15.60:6813","nonce":3414864047}]},"public_addr":"172.21.15.60:6809/3414864047","cluster_addr":"172.21.15.60:6811/3414864047","heartbeat_back_addr":"172.21.15.60:6815/3414864047","heartbeat_front_addr":"172.21.15.60:6813/3414864047","state":["exists","up"]},{"osd":3,"uuid":"5daf1e38-aa78-4ca0-b91e-36e4d0a8ccb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6825","nonce":3085378704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6827","nonce":3085378704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6831","nonce":3085378704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":3085378704},{"type":"v1","addr":"172.21.15.60:6829","nonce":3085378704}]},"public_addr":"172.21.15.60:6825/3085378704","cluster_addr":"172.21.15.60:6827/3085378704","heartbeat_back_addr":"172.21.15.60:6831/3085378704","heartbeat_front_addr":"172.21.15.60:6829/3085378704","state":["exists","up"]},{"osd":4,"uuid":"a60837b5-a8b6-4837-827a-53041d9b4d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6811","nonce":3116664612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6813","nonce":3116664612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6817","nonce":3116664612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3116664612},{"type":"v1","addr":"172.21.15.100:6815","nonce":3116664612}]},"public_addr":"172.21.15.100:6811/3116664612","cluster_addr":"172.21.15.100:6813/3116664612","heartbeat_back_addr":"172.21.15.100:6817/3116664612","heartbeat_front_addr":"172.21.15.100:6815/3116664612","state":["exists","up"]},{"osd":5,"uuid":"e6b32c2e-58f9-4cca-b0fe-efb31f753ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6819","nonce":1966336275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6821","nonce":1966336275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6825","nonce":1966336275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1966336275},{"type":"v1","addr":"172.21.15.100:6823","nonce":1966336275}]},"public_addr":"172.21.15.100:6819/1966336275","cluster_addr":"172.21.15.100:6821/1966336275","heartbeat_back_addr":"172.21.15.100:6825/1966336275","heartbeat_front_addr":"172.21.15.100:6823/1966336275","state":["exists","up"]},{"osd":6,"uuid":"b57116a4-56ce-4fc4-87b1-30c4dd0e4827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6827","nonce":3702271988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6829","nonce":3702271988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6833","nonce":3702271988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":3702271988},{"type":"v1","addr":"172.21.15.100:6831","nonce":3702271988}]},"public_addr":"172.21.15.100:6827/3702271988","cluster_addr":"172.21.15.100:6829/3702271988","heartbeat_back_addr":"172.21.15.100:6833/3702271988","heartbeat_front_addr":"172.21.15.100:6831/3702271988","state":["exists","up"]},{"osd":7,"uuid":"beb26a63-23f6-424a-b72c-d48de6bc032b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6803","nonce":1334072453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6805","nonce":1334072453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6809","nonce":1334072453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1334072453},{"type":"v1","addr":"172.21.15.100:6807","nonce":1334072453}]},"public_addr":"172.21.15.100:6803/1334072453","cluster_addr":"172.21.15.100:6805/1334072453","heartbeat_back_addr":"172.21.15.100:6809/1334072453","heartbeat_front_addr":"172.21.15.100:6807/1334072453","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.488038+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:18.898387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.536819+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.082673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.345102+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.758894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:20.095951+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:38:19.017886+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/786870322":"2024-09-17T15:48:21.813060+0000","172.21.15.60:6833/786870322":"2024-09-17T15:48:21.813060+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.321+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7c4079420 msgr2=0x7ff7c407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.321+0000 7ff7ed000640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7c4079420 0x7ff7c407b8e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8034000 tx=0x7ff7d8033040 comp rx=0 tx=0).stop 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.321+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e80ccf00 msgr2=0x7ff7e809ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.321+0000 7ff7ed000640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e80ccf00 0x7ff7e809ba50 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c700 tx=0x7ff7d000cbc0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.322+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 shutdown_connections 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.322+0000 7ff7ed000640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7ff7c4079420 0x7ff7c407b8e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.322+0000 7ff7ed000640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff7e815f5a0 0x7ff7e809bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.322+0000 7ff7ed000640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff7e80ccf00 0x7ff7e809ba50 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.322+0000 7ff7ed000640 1 --2- 172.21.15.60:0/139138661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7e8073fb0 0x7ff7e8143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.322+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 >> 172.21.15.60:0/139138661 conn(0x7ff7e80b1950 msgr2=0x7ff7e80ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.322+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 shutdown_connections 2024-09-16T15:48:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:29.322+0000 7ff7ed000640 1 -- 172.21.15.60:0/139138661 wait complete. 2024-09-16T15:48:30.340 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T15:48:30.341 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:48:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.515+0000 7f4924200640 1 -- 172.21.15.60:0/517688299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c0733e0 msgr2=0x7f491c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.515+0000 7f4924200640 1 --2- 172.21.15.60:0/517688299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c0733e0 0x7f491c0cce50 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f491c070cb0 tx=0x7f490802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.516+0000 7f4924200640 1 -- 172.21.15.60:0/517688299 shutdown_connections 2024-09-16T15:48:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.516+0000 7f4924200640 1 --2- 172.21.15.60:0/517688299 >> v2:172.21.15.100:3300/0 conn(0x7f491c15c350 0x7f491c15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.516+0000 7f4924200640 1 --2- 172.21.15.60:0/517688299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c0733e0 0x7f491c0cce50 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.516+0000 7f4924200640 1 --2- 172.21.15.60:0/517688299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15f910 0x7f491c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.516+0000 7f4924200640 1 -- 172.21.15.60:0/517688299 >> 172.21.15.60:0/517688299 conn(0x7f491c0b18e0 msgr2=0x7f491c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.516+0000 7f4924200640 1 -- 172.21.15.60:0/517688299 shutdown_connections 2024-09-16T15:48:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.516+0000 7f4924200640 1 -- 172.21.15.60:0/517688299 wait complete. 2024-09-16T15:48:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.517+0000 7f4924200640 1 Processor -- start 2024-09-16T15:48:30.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.517+0000 7f4924200640 1 -- start start 2024-09-16T15:48:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.518+0000 7f4924200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15c350 0x7f491c09d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.518+0000 7f4924200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15f910 0x7f491c1f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.518+0000 7f4924200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c15d960 0x7f491c09b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.518+0000 7f4924200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f491c074a60 con 0x7f491c15d960 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.518+0000 7f4924200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f491c0cd5a0 con 0x7f491c15c350 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.518+0000 7f4924200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f491c15cd40 con 0x7f491c15f910 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.518+0000 7f4922200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c15d960 0x7f491c09b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.518+0000 7f4922200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c15d960 0x7f491c09b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52760/0 (socket says 172.21.15.60:52760) 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4922200640 1 -- 172.21.15.60:0/347990706 learned_addr learned my addr 172.21.15.60:0/347990706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4920e00640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15f910 0x7f491c1f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4921800640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15c350 0x7f491c09d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4922200640 1 -- 172.21.15.60:0/347990706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15f910 msgr2=0x7f491c1f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4922200640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f491c15f910 0x7f491c1f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4922200640 1 -- 172.21.15.60:0/347990706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15c350 msgr2=0x7f491c09d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4922200640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f491c15c350 0x7f491c09d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4922200640 1 -- 172.21.15.60:0/347990706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c09da10 con 0x7f491c15d960 2024-09-16T15:48:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4922200640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c15d960 0x7f491c09b660 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f491400ead0 tx=0x7f491400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491400cde0 con 0x7f491c15d960 2024-09-16T15:48:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4914004c10 con 0x7f491c15d960 2024-09-16T15:48:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c1572d0 con 0x7f491c15d960 2024-09-16T15:48:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.519+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914005020 con 0x7f491c15d960 2024-09-16T15:48:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.520+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c157890 con 0x7f491c15d960 2024-09-16T15:48:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.520+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f491c1f4080 con 0x7f491c15d960 2024-09-16T15:48:30.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.522+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f49140051c0 con 0x7f491c15d960 2024-09-16T15:48:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.523+0000 7f4919600640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f48fc079420 0x7f48fc07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.523+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f4914099f60 con 0x7f491c15d960 2024-09-16T15:48:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.523+0000 7f4919600640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7f48fc07f2e0 0x7f48fc081700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.523+0000 7f4921800640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f48fc079420 0x7f48fc07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.523+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 --> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4914009050 con 0x7f48fc07f2e0 2024-09-16T15:48:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.523+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) ==== 24+0+0 (secure 0 0 0) 0x7f49140a12a0 con 0x7f491c15d960 2024-09-16T15:48:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.523+0000 7f4920e00640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7f48fc07f2e0 0x7f48fc081700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.524+0000 7f4921800640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f48fc079420 0x7f48fc07b8e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f491c1f65f0 tx=0x7f4910005c90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.524+0000 7f4920e00640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7f48fc07f2e0 0x7f48fc081700 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.525+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 <== osd.0 v2:172.21.15.60:6816/841472238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4914009050 con 0x7f48fc07f2e0 2024-09-16T15:48:30.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.538+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 --> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f491c15c800 con 0x7f48fc07f2e0 2024-09-16T15:48:30.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.538+0000 7f4919600640 1 -- 172.21.15.60:0/347990706 <== osd.0 v2:172.21.15.60:6816/841472238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f491c15c800 con 0x7f48fc07f2e0 2024-09-16T15:48:30.539 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:30.540 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:48:30.540 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7f48fc07f2e0 msgr2=0x7f48fc081700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6816/841472238,v1:172.21.15.60:6817/841472238] conn(0x7f48fc07f2e0 0x7f48fc081700 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f48fc079420 msgr2=0x7f48fc07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f48fc079420 0x7f48fc07b8e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f491c1f65f0 tx=0x7f4910005c90 comp rx=0 tx=0).stop 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c15d960 msgr2=0x7f491c09b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 --2- 172.21.15.60:0/347990706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f491c15d960 0x7f491c09b660 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f491400ead0 tx=0x7f491400ef90 comp rx=0 tx=0).stop 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4922200640 1 -- 172.21.15.60:0/347990706 reap_dead start 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 shutdown_connections 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 >> 172.21.15.60:0/347990706 conn(0x7f491c0b18e0 msgr2=0x7f491c0cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 shutdown_connections 2024-09-16T15:48:30.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.539+0000 7f4924200640 1 -- 172.21.15.60:0/347990706 wait complete. 2024-09-16T15:48:30.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T15:48:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.715+0000 7f9c0f200640 1 -- 172.21.15.60:0/802655114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c101616b0 msgr2=0x7f9c100d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.715+0000 7f9c0f200640 1 --2- 172.21.15.60:0/802655114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c101616b0 0x7f9c100d2370 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9c08005fd0 tx=0x7f9c0802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.716+0000 7f9c0f200640 1 -- 172.21.15.60:0/802655114 shutdown_connections 2024-09-16T15:48:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.716+0000 7f9c0f200640 1 --2- 172.21.15.60:0/802655114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c101616b0 0x7f9c100d2370 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.716+0000 7f9c0f200640 1 -- 172.21.15.60:0/802655114 >> 172.21.15.60:0/802655114 conn(0x7f9c100d3df0 msgr2=0x7f9c100d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.716+0000 7f9c0f200640 1 -- 172.21.15.60:0/802655114 shutdown_connections 2024-09-16T15:48:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.716+0000 7f9c0f200640 1 -- 172.21.15.60:0/802655114 wait complete. 2024-09-16T15:48:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.717+0000 7f9c0f200640 1 Processor -- start 2024-09-16T15:48:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.717+0000 7f9c0f200640 1 -- start start 2024-09-16T15:48:30.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.718+0000 7f9c0f200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c100d4980 0x7f9c10160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.718+0000 7f9c0f200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c101612d0 0x7f9c10156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.718+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c101612d0 0x7f9c10156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c101612d0 0x7f9c10156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52772/0 (socket says 172.21.15.60:52772) 2024-09-16T15:48:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c100d4980 0x7f9c10160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0f200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c101616b0 0x7f9c1015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c100d4980 0x7f9c10160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42482/0 (socket says 172.21.15.60:42482) 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0f200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101690d0 con 0x7f9c101612d0 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10168f50 con 0x7f9c100d4980 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9c10169250 con 0x7f9c101616b0 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0d400640 1 -- 172.21.15.60:0/3342400535 learned_addr learned my addr 172.21.15.60:0/3342400535 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0e800640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c101616b0 0x7f9c1015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0d400640 1 -- 172.21.15.60:0/3342400535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c101616b0 msgr2=0x7f9c1015bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0d400640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c101616b0 0x7f9c1015bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0d400640 1 -- 172.21.15.60:0/3342400535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c100d4980 msgr2=0x7f9c10160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.719+0000 7f9c0d400640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c100d4980 0x7f9c10160530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.720+0000 7f9c0d400640 1 -- 172.21.15.60:0/3342400535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c100b7e20 con 0x7f9c101612d0 2024-09-16T15:48:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.720+0000 7f9c0e800640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c101616b0 0x7f9c1015bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.720+0000 7f9c0d400640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c101612d0 0x7f9c10156290 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c730 tx=0x7f9bfc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.720+0000 7f9c06000640 1 -- 172.21.15.60:0/3342400535 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0050d0 con 0x7f9c101612d0 2024-09-16T15:48:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.720+0000 7f9c06000640 1 -- 172.21.15.60:0/3342400535 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bfc005290 con 0x7f9c101612d0 2024-09-16T15:48:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.720+0000 7f9c06000640 1 -- 172.21.15.60:0/3342400535 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00a8a0 con 0x7f9c101612d0 2024-09-16T15:48:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.720+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c100b8100 con 0x7f9c101612d0 2024-09-16T15:48:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.720+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c100c7b40 con 0x7f9c101612d0 2024-09-16T15:48:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.721+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c100b4e00 con 0x7f9c101612d0 2024-09-16T15:48:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.723+0000 7f9c06000640 1 -- 172.21.15.60:0/3342400535 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f9bfc00aa40 con 0x7f9c101612d0 2024-09-16T15:48:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.724+0000 7f9c06000640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9bf0079420 0x7f9bf007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.724+0000 7f9c06000640 1 -- 172.21.15.60:0/3342400535 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f9bfc098d70 con 0x7f9c101612d0 2024-09-16T15:48:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.724+0000 7f9c0de00640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9bf0079420 0x7f9bf007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.725+0000 7f9c0de00640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9bf0079420 0x7f9bf007b8e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9c00000c00 tx=0x7f9c00001640 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:30.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.728+0000 7f9c06000640 1 -- 172.21.15.60:0/3342400535 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c100b5510 con 0x7f9c101612d0 2024-09-16T15:48:30.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.924+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f9c101567d0 con 0x7f9bf0079420 2024-09-16T15:48:30.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.925+0000 7f9c06000640 1 -- 172.21.15.60:0/3342400535 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9c00001c50 con 0x7f9bf0079420 2024-09-16T15:48:30.926 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T15:48:30.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9bf0079420 msgr2=0x7f9bf007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9bf0079420 0x7f9bf007b8e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9c00000c00 tx=0x7f9c00001640 comp rx=0 tx=0).stop 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c101612d0 msgr2=0x7f9c10156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c101612d0 0x7f9c10156290 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c730 tx=0x7f9bfc00cbf0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 shutdown_connections 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9bf0079420 0x7f9bf007b8e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c101616b0 0x7f9c1015bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c101612d0 0x7f9c10156290 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 --2- 172.21.15.60:0/3342400535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c100d4980 0x7f9c10160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 >> 172.21.15.60:0/3342400535 conn(0x7f9c100d3df0 msgr2=0x7f9c100d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 shutdown_connections 2024-09-16T15:48:30.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:30.927+0000 7f9c0f200640 1 -- 172.21.15.60:0/3342400535 wait complete. 2024-09-16T15:48:30.945 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T15:48:30.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:48:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.110+0000 7f8c6b000640 1 -- 172.21.15.60:0/687270275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c480053c0 msgr2=0x7f8c4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.110+0000 7f8c6b000640 1 --2- 172.21.15.60:0/687270275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c480053c0 0x7f8c4800b9c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00c590 tx=0x7f8c48005830 comp rx=0 tx=0).stop 2024-09-16T15:48:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.110+0000 7f8c6b000640 1 -- 172.21.15.60:0/687270275 shutdown_connections 2024-09-16T15:48:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.110+0000 7f8c6b000640 1 --2- 172.21.15.60:0/687270275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c480053c0 0x7f8c4800b9c0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.110+0000 7f8c6b000640 1 -- 172.21.15.60:0/687270275 >> 172.21.15.60:0/687270275 conn(0x7f8c6406dc80 msgr2=0x7f8c6406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.110+0000 7f8c6b000640 1 -- 172.21.15.60:0/687270275 shutdown_connections 2024-09-16T15:48:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.110+0000 7f8c6b000640 1 -- 172.21.15.60:0/687270275 wait complete. 2024-09-16T15:48:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.111+0000 7f8c6b000640 1 Processor -- start 2024-09-16T15:48:31.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.112+0000 7f8c6b000640 1 -- start start 2024-09-16T15:48:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.112+0000 7f8c6b000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c480053c0 0x7f8c6413f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.112+0000 7f8c6b000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6413f8a0 0x7f8c6413fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.112+0000 7f8c6b000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64141890 0x7f8c64141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.112+0000 7f8c6b000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64155610 con 0x7f8c64141890 2024-09-16T15:48:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.112+0000 7f8c6b000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64155490 con 0x7f8c6413f8a0 2024-09-16T15:48:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.112+0000 7f8c6b000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6406fc10 con 0x7f8c480053c0 2024-09-16T15:48:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64141890 0x7f8c64141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64141890 0x7f8c64141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52784/0 (socket says 172.21.15.60:52784) 2024-09-16T15:48:31.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 -- 172.21.15.60:0/3654410205 learned_addr learned my addr 172.21.15.60:0/3654410205 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:31.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c63e00640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c480053c0 0x7f8c6413f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c5b400640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6413f8a0 0x7f8c6413fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 -- 172.21.15.60:0/3654410205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c480053c0 msgr2=0x7f8c6413f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c480053c0 0x7f8c6413f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 -- 172.21.15.60:0/3654410205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6413f8a0 msgr2=0x7f8c6413fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6413f8a0 0x7f8c6413fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 -- 172.21.15.60:0/3654410205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6413f5b0 con 0x7f8c64141890 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c63e00640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c480053c0 0x7f8c6413f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c5b400640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c6413f8a0 0x7f8c6413fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.113+0000 7f8c69000640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64141890 0x7f8c64141d10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c02b410 tx=0x7f8c5c003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.114+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c039070 con 0x7f8c64141890 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.114+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c5c02ba50 con 0x7f8c64141890 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.114+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c02be60 con 0x7f8c64141890 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.114+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c64142380 con 0x7f8c64141890 2024-09-16T15:48:31.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.114+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6414e7f0 con 0x7f8c64141890 2024-09-16T15:48:31.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.115+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c641428a0 con 0x7f8c64141890 2024-09-16T15:48:31.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.116+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f8c5c034680 con 0x7f8c64141890 2024-09-16T15:48:31.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.117+0000 7f8c60c00640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c40079450 0x7f8c4007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.117+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f8c5c0b9720 con 0x7f8c64141890 2024-09-16T15:48:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.117+0000 7f8c60c00640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7f8c4007f310 0x7f8c40081730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.117+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 --> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c40081e00 con 0x7f8c4007f310 2024-09-16T15:48:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.117+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) ==== 24+0+0 (secure 0 0 0) 0x7f8c5c0be5e0 con 0x7f8c64141890 2024-09-16T15:48:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.118+0000 7f8c63e00640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c40079450 0x7f8c4007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.118+0000 7f8c5b400640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7f8c4007f310 0x7f8c40081730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.118+0000 7f8c63e00640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c40079450 0x7f8c4007b910 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400aa70 tx=0x7f8c54005d40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.118+0000 7f8c5b400640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7f8c4007f310 0x7f8c40081730 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.119+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 <== osd.1 v2:172.21.15.60:6800/910651751 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c40081e00 con 0x7f8c4007f310 2024-09-16T15:48:31.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.132+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 --> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8c640703a0 con 0x7f8c4007f310 2024-09-16T15:48:31.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c60c00640 1 -- 172.21.15.60:0/3654410205 <== osd.1 v2:172.21.15.60:6800/910651751 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8c640703a0 con 0x7f8c4007f310 2024-09-16T15:48:31.134 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:31.134 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:48:31.134 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7f8c4007f310 msgr2=0x7f8c40081730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6800/910651751,v1:172.21.15.60:6801/910651751] conn(0x7f8c4007f310 0x7f8c40081730 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c40079450 msgr2=0x7f8c4007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c40079450 0x7f8c4007b910 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400aa70 tx=0x7f8c54005d40 comp rx=0 tx=0).stop 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64141890 msgr2=0x7f8c64141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 --2- 172.21.15.60:0/3654410205 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c64141890 0x7f8c64141d10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c02b410 tx=0x7f8c5c003b80 comp rx=0 tx=0).stop 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c69000640 1 -- 172.21.15.60:0/3654410205 reap_dead start 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 shutdown_connections 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 >> 172.21.15.60:0/3654410205 conn(0x7f8c6406dc80 msgr2=0x7f8c64153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 shutdown_connections 2024-09-16T15:48:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.133+0000 7f8c6b000640 1 -- 172.21.15.60:0/3654410205 wait complete. 2024-09-16T15:48:31.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T15:48:31.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.312+0000 7f4fdba00640 1 -- 172.21.15.60:0/4033240985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 msgr2=0x7f4fd40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.312+0000 7f4fdba00640 1 --2- 172.21.15.60:0/4033240985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 0x7f4fd40b2440 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009830 tx=0x7f4fc802a480 comp rx=0 tx=0).stop 2024-09-16T15:48:31.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.312+0000 7f4fdba00640 1 -- 172.21.15.60:0/4033240985 shutdown_connections 2024-09-16T15:48:31.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.312+0000 7f4fdba00640 1 --2- 172.21.15.60:0/4033240985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd4073fb0 0x7f4fd4074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.312+0000 7f4fdba00640 1 --2- 172.21.15.60:0/4033240985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 0x7f4fd40b2440 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.312+0000 7f4fdba00640 1 -- 172.21.15.60:0/4033240985 >> 172.21.15.60:0/4033240985 conn(0x7f4fd40b1950 msgr2=0x7f4fd40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.312+0000 7f4fdba00640 1 -- 172.21.15.60:0/4033240985 shutdown_connections 2024-09-16T15:48:31.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.312+0000 7f4fdba00640 1 -- 172.21.15.60:0/4033240985 wait complete. 2024-09-16T15:48:31.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.314+0000 7f4fdba00640 1 Processor -- start 2024-09-16T15:48:31.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.314+0000 7f4fdba00640 1 -- start start 2024-09-16T15:48:31.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fdba00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4073fb0 0x7f4fd41f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fdba00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd40ccf00 0x7f4fd41eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fdba00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 0x7f4fd41f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fdba00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd406f2e0 con 0x7f4fd4073fb0 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fdba00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd406f160 con 0x7f4fd415f5a0 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fdba00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd406f460 con 0x7f4fd40ccf00 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fd9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4073fb0 0x7f4fd41f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fd9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4073fb0 0x7f4fd41f8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52786/0 (socket says 172.21.15.60:52786) 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fd9000640 1 -- 172.21.15.60:0/2716389207 learned_addr learned my addr 172.21.15.60:0/2716389207 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fd9a00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 0x7f4fd41f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.315+0000 7f4fd3e00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd40ccf00 0x7f4fd41eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd9000640 1 -- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd40ccf00 msgr2=0x7f4fd41eec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd9000640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd40ccf00 0x7f4fd41eec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd9000640 1 -- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 msgr2=0x7f4fd41f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd9000640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 0x7f4fd41f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd9000640 1 -- 172.21.15.60:0/2716389207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd41f4dc0 con 0x7f4fd4073fb0 2024-09-16T15:48:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd9a00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 0x7f4fd41f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:31.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd9000640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4073fb0 0x7f4fd41f8dc0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009950 tx=0x7f4fc8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd0c00640 1 -- 172.21.15.60:0/2716389207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8005c90 con 0x7f4fd4073fb0 2024-09-16T15:48:31.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd0c00640 1 -- 172.21.15.60:0/2716389207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fc8031e50 con 0x7f4fd4073fb0 2024-09-16T15:48:31.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd4153620 con 0x7f4fd4073fb0 2024-09-16T15:48:31.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fd0c00640 1 -- 172.21.15.60:0/2716389207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8005260 con 0x7f4fd4073fb0 2024-09-16T15:48:31.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.316+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd4153be0 con 0x7f4fd4073fb0 2024-09-16T15:48:31.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.317+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd40ad200 con 0x7f4fd4073fb0 2024-09-16T15:48:31.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.319+0000 7f4fd0c00640 1 -- 172.21.15.60:0/2716389207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f4fc8005400 con 0x7f4fd4073fb0 2024-09-16T15:48:31.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.319+0000 7f4fd0c00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4fb4079420 0x7f4fb407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.320+0000 7f4fd0c00640 1 -- 172.21.15.60:0/2716389207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f4fc80ba530 con 0x7f4fd4073fb0 2024-09-16T15:48:31.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.320+0000 7f4fd3e00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4fb4079420 0x7f4fb407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.320+0000 7f4fd3e00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4fb4079420 0x7f4fb407b8e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4fd41f8310 tx=0x7f4fc0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.324+0000 7f4fd0c00640 1 -- 172.21.15.60:0/2716389207 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4fc803f030 con 0x7f4fd4073fb0 2024-09-16T15:48:31.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.519+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f4fd4154a00 con 0x7f4fb4079420 2024-09-16T15:48:31.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.519+0000 7f4fd0c00640 1 -- 172.21.15.60:0/2716389207 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4fd4154a00 con 0x7f4fb4079420 2024-09-16T15:48:31.520 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.521+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4fb4079420 msgr2=0x7f4fb407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.521+0000 7f4fdba00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4fb4079420 0x7f4fb407b8e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4fd41f8310 tx=0x7f4fc0009040 comp rx=0 tx=0).stop 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.521+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4073fb0 msgr2=0x7f4fd41f8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.521+0000 7f4fdba00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4073fb0 0x7f4fd41f8dc0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009950 tx=0x7f4fc8033c20 comp rx=0 tx=0).stop 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.521+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 shutdown_connections 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.521+0000 7f4fdba00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4fb4079420 0x7f4fb407b8e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.521+0000 7f4fdba00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fd415f5a0 0x7f4fd41f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.521+0000 7f4fdba00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fd40ccf00 0x7f4fd41eec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.522+0000 7f4fdba00640 1 --2- 172.21.15.60:0/2716389207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fd4073fb0 0x7f4fd41f8dc0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.522+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 >> 172.21.15.60:0/2716389207 conn(0x7f4fd40b1950 msgr2=0x7f4fd40af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.522+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 shutdown_connections 2024-09-16T15:48:31.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.522+0000 7f4fdba00640 1 -- 172.21.15.60:0/2716389207 wait complete. 2024-09-16T15:48:31.538 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T15:48:31.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:48:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.702+0000 7fc4c9000640 1 -- 172.21.15.60:0/3716210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4c4070550 msgr2=0x7fc4c40cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.702+0000 7fc4c9000640 1 --2- 172.21.15.60:0/3716210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4c4070550 0x7fc4c40cb1c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc005fd0 tx=0x7fc4bc02a820 comp rx=0 tx=0).stop 2024-09-16T15:48:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.702+0000 7fc4c9000640 1 -- 172.21.15.60:0/3716210 shutdown_connections 2024-09-16T15:48:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.702+0000 7fc4c9000640 1 --2- 172.21.15.60:0/3716210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4c4070550 0x7fc4c40cb1c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.702+0000 7fc4c9000640 1 --2- 172.21.15.60:0/3716210 >> v2:172.21.15.100:3300/0 conn(0x7fc4c415bda0 0x7fc4c4073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.702+0000 7fc4c9000640 1 -- 172.21.15.60:0/3716210 >> 172.21.15.60:0/3716210 conn(0x7fc4c40cfa70 msgr2=0x7fc4c40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.702+0000 7fc4c9000640 1 -- 172.21.15.60:0/3716210 shutdown_connections 2024-09-16T15:48:31.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.702+0000 7fc4c9000640 1 -- 172.21.15.60:0/3716210 wait complete. 2024-09-16T15:48:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.704+0000 7fc4c9000640 1 Processor -- start 2024-09-16T15:48:31.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.704+0000 7fc4c9000640 1 -- start start 2024-09-16T15:48:31.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4c406e590 0x7fc4c415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c4070550 0x7fc4c415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c415bda0 0x7fc4c40cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c9000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c40cba90 con 0x7fc4c4070550 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c9000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c40cb910 con 0x7fc4c415bda0 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c9000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c40cbc10 con 0x7fc4c406e590 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c4070550 0x7fc4c415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c2a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c4070550 0x7fc4c415ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52802/0 (socket says 172.21.15.60:52802) 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c2a00640 1 -- 172.21.15.60:0/1166478255 learned_addr learned my addr 172.21.15.60:0/1166478255 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c3400640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4c406e590 0x7fc4c415dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.705+0000 7fc4c3e00640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c415bda0 0x7fc4c40cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4c2a00640 1 -- 172.21.15.60:0/1166478255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4c406e590 msgr2=0x7fc4c415dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4c2a00640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4c406e590 0x7fc4c415dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4c2a00640 1 -- 172.21.15.60:0/1166478255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c415bda0 msgr2=0x7fc4c40cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4c2a00640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c415bda0 0x7fc4c40cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4c2a00640 1 -- 172.21.15.60:0/1166478255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c41f2540 con 0x7fc4c4070550 2024-09-16T15:48:31.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4c2a00640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c4070550 0x7fc4c415ec20 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c490 tx=0x7fc4b000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0016020 con 0x7fc4c4070550 2024-09-16T15:48:31.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b0012940 con 0x7fc4c4070550 2024-09-16T15:48:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0012d90 con 0x7fc4c4070550 2024-09-16T15:48:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c41f6080 con 0x7fc4c4070550 2024-09-16T15:48:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.706+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c41f6640 con 0x7fc4c4070550 2024-09-16T15:48:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.707+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4c406fa20 con 0x7fc4c4070550 2024-09-16T15:48:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.709+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7fc4b0019020 con 0x7fc4c4070550 2024-09-16T15:48:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.709+0000 7fc4bb400640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc4a4079420 0x7fc4a407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.710+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7fc4b0098e00 con 0x7fc4c4070550 2024-09-16T15:48:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.710+0000 7fc4c3400640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc4a4079420 0x7fc4a407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.710+0000 7fc4bb400640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fc4a407f2e0 0x7fc4a4081700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.710+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 --> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4b0004470 con 0x7fc4a407f2e0 2024-09-16T15:48:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.710+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) ==== 24+0+0 (secure 0 0 0) 0x7fc4b00a2050 con 0x7fc4c4070550 2024-09-16T15:48:31.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.710+0000 7fc4c3e00640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fc4a407f2e0 0x7fc4a4081700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.710+0000 7fc4c3400640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc4a4079420 0x7fc4a407b8e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4159610 tx=0x7fc4b4005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.711+0000 7fc4c3e00640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fc4a407f2e0 0x7fc4a4081700 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.711+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 <== osd.2 v2:172.21.15.60:6808/3414864047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4b0004470 con 0x7fc4a407f2e0 2024-09-16T15:48:31.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 --> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc4c40cc1a0 con 0x7fc4a407f2e0 2024-09-16T15:48:31.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4bb400640 1 -- 172.21.15.60:0/1166478255 <== osd.2 v2:172.21.15.60:6808/3414864047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc4c40cc1a0 con 0x7fc4a407f2e0 2024-09-16T15:48:31.726 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:48:31.726 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fc4a407f2e0 msgr2=0x7fc4a4081700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4c9000640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6808/3414864047,v1:172.21.15.60:6809/3414864047] conn(0x7fc4a407f2e0 0x7fc4a4081700 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc4a4079420 msgr2=0x7fc4a407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4c9000640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc4a4079420 0x7fc4a407b8e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4159610 tx=0x7fc4b4005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c4070550 msgr2=0x7fc4c415ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4c9000640 1 --2- 172.21.15.60:0/1166478255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4c4070550 0x7fc4c415ec20 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c490 tx=0x7fc4b000c950 comp rx=0 tx=0).stop 2024-09-16T15:48:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.725+0000 7fc4c3e00640 1 -- 172.21.15.60:0/1166478255 reap_dead start 2024-09-16T15:48:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.726+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 shutdown_connections 2024-09-16T15:48:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.726+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 >> 172.21.15.60:0/1166478255 conn(0x7fc4c40cfa70 msgr2=0x7fc4c40b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.726+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 shutdown_connections 2024-09-16T15:48:31.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.726+0000 7fc4c9000640 1 -- 172.21.15.60:0/1166478255 wait complete. 2024-09-16T15:48:31.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T15:48:31.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.903+0000 7f1b90a00640 1 -- 172.21.15.60:0/2106278564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 msgr2=0x7f1b70003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.903+0000 7f1b90a00640 1 --2- 172.21.15.60:0/2106278564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 0x7f1b70003dc0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0017c0 tx=0x7f1b70005830 comp rx=0 tx=0).stop 2024-09-16T15:48:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.903+0000 7f1b8a800640 1 -- 172.21.15.60:0/2106278564 reap_dead start 2024-09-16T15:48:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.903+0000 7f1b90a00640 1 -- 172.21.15.60:0/2106278564 shutdown_connections 2024-09-16T15:48:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.903+0000 7f1b90a00640 1 -- 172.21.15.60:0/2106278564 >> 172.21.15.60:0/2106278564 conn(0x7f1b8c0b1950 msgr2=0x7f1b8c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.903+0000 7f1b90a00640 1 -- 172.21.15.60:0/2106278564 shutdown_connections 2024-09-16T15:48:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.903+0000 7f1b90a00640 1 -- 172.21.15.60:0/2106278564 wait complete. 2024-09-16T15:48:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.905+0000 7f1b90a00640 1 Processor -- start 2024-09-16T15:48:31.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.905+0000 7f1b90a00640 1 -- start start 2024-09-16T15:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b90a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 0x7f1b8c1e4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b90a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c15e9a0 0x7f1b8c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b90a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8c1e8d60 0x7f1b8c1e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b90a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c074840 con 0x7f1b700039e0 2024-09-16T15:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b90a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c0746c0 con 0x7f1b8c15e9a0 2024-09-16T15:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b90a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8c06f160 con 0x7f1b8c1e8d60 2024-09-16T15:48:31.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b89e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 0x7f1b8c1e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b89e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 0x7f1b8c1e4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52816/0 (socket says 172.21.15.60:52816) 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b89e00640 1 -- 172.21.15.60:0/3638856625 learned_addr learned my addr 172.21.15.60:0/3638856625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b8a800640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8c1e8d60 0x7f1b8c1e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b81400640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c15e9a0 0x7f1b8c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.906+0000 7f1b89e00640 1 -- 172.21.15.60:0/3638856625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8c1e8d60 msgr2=0x7f1b8c1e91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b89e00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8c1e8d60 0x7f1b8c1e91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b89e00640 1 -- 172.21.15.60:0/3638856625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c15e9a0 msgr2=0x7f1b8c1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b89e00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c15e9a0 0x7f1b8c1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b89e00640 1 -- 172.21.15.60:0/3638856625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c1e4e30 con 0x7f1b700039e0 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b89e00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 0x7f1b8c1e4730 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c02aa80 tx=0x7f1b7c0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b82a00640 1 -- 172.21.15.60:0/3638856625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c042070 con 0x7f1b700039e0 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b82a00640 1 -- 172.21.15.60:0/3638856625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b7c0037f0 con 0x7f1b700039e0 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b82a00640 1 -- 172.21.15.60:0/3638856625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c037040 con 0x7f1b700039e0 2024-09-16T15:48:31.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8c1e5110 con 0x7f1b700039e0 2024-09-16T15:48:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.907+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8c1e56d0 con 0x7f1b700039e0 2024-09-16T15:48:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.908+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8c06f6c0 con 0x7f1b700039e0 2024-09-16T15:48:31.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.909+0000 7f1b82a00640 1 -- 172.21.15.60:0/3638856625 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f1b7c003be0 con 0x7f1b700039e0 2024-09-16T15:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.910+0000 7f1b82a00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b68079420 0x7f1b6807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.911+0000 7f1b82a00640 1 -- 172.21.15.60:0/3638856625 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f1b7c0b9010 con 0x7f1b700039e0 2024-09-16T15:48:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.911+0000 7f1b81400640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b68079420 0x7f1b6807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.911+0000 7f1b81400640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b68079420 0x7f1b6807b8e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b74001240 tx=0x7f1b74000ed0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:31.915+0000 7f1b82a00640 1 -- 172.21.15.60:0/3638856625 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1b7c082820 con 0x7f1b700039e0 2024-09-16T15:48:32.029 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:48:32.033 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:48:32.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.111+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f1b8c0b3830 con 0x7f1b68079420 2024-09-16T15:48:32.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.112+0000 7f1b82a00640 1 -- 172.21.15.60:0/3638856625 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1b8c0b3830 con 0x7f1b68079420 2024-09-16T15:48:32.113 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T15:48:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b68079420 msgr2=0x7f1b6807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b68079420 0x7f1b6807b8e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b74001240 tx=0x7f1b74000ed0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 msgr2=0x7f1b8c1e4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 0x7f1b8c1e4730 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c02aa80 tx=0x7f1b7c0060e0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 shutdown_connections 2024-09-16T15:48:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1b68079420 0x7f1b6807b8e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b8c1e8d60 0x7f1b8c1e91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b8c15e9a0 0x7f1b8c1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 --2- 172.21.15.60:0/3638856625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b700039e0 0x7f1b8c1e4730 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 >> 172.21.15.60:0/3638856625 conn(0x7f1b8c0b1950 msgr2=0x7f1b8c09d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 shutdown_connections 2024-09-16T15:48:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.114+0000 7f1b90a00640 1 -- 172.21.15.60:0/3638856625 wait complete. 2024-09-16T15:48:32.132 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T15:48:32.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:48:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.336+0000 7f24e3200640 1 -- 172.21.15.60:0/3481671230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e4070550 msgr2=0x7f24e4166840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.336+0000 7f24e3200640 1 --2- 172.21.15.60:0/3481671230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e4070550 0x7f24e4166840 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f24dc005fd0 tx=0x7f24dc02a940 comp rx=0 tx=0).stop 2024-09-16T15:48:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.336+0000 7f24e3200640 1 -- 172.21.15.60:0/3481671230 shutdown_connections 2024-09-16T15:48:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.336+0000 7f24e3200640 1 --2- 172.21.15.60:0/3481671230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e4070550 0x7f24e4166840 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.336+0000 7f24e3200640 1 -- 172.21.15.60:0/3481671230 >> 172.21.15.60:0/3481671230 conn(0x7f24e40cfa70 msgr2=0x7f24e40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.336+0000 7f24e3200640 1 -- 172.21.15.60:0/3481671230 shutdown_connections 2024-09-16T15:48:32.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.336+0000 7f24e3200640 1 -- 172.21.15.60:0/3481671230 wait complete. 2024-09-16T15:48:32.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.337+0000 7f24e3200640 1 Processor -- start 2024-09-16T15:48:32.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.338+0000 7f24e3200640 1 -- start start 2024-09-16T15:48:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e3200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e406e590 0x7f24e4151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e3200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24e4070550 0x7f24e40ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e3200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e415bda0 0x7f24e41f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e3200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4166f00 con 0x7f24e406e590 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e3200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4166d80 con 0x7f24e4070550 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e3200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f24e4167080 con 0x7f24e415bda0 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e406e590 0x7f24e4151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e1e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e406e590 0x7f24e4151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52836/0 (socket says 172.21.15.60:52836) 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e1e00640 1 -- 172.21.15.60:0/2072911029 learned_addr learned my addr 172.21.15.60:0/2072911029 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24d9400640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24e4070550 0x7f24e40ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e2800640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e415bda0 0x7f24e41f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e1e00640 1 -- 172.21.15.60:0/2072911029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e415bda0 msgr2=0x7f24e41f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e1e00640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e415bda0 0x7f24e41f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e1e00640 1 -- 172.21.15.60:0/2072911029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24e4070550 msgr2=0x7f24e40ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e1e00640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24e4070550 0x7f24e40ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.339+0000 7f24e1e00640 1 -- 172.21.15.60:0/2072911029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e41f8c40 con 0x7f24e406e590 2024-09-16T15:48:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.340+0000 7f24e1e00640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e406e590 0x7f24e4151940 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f24d400c3a0 tx=0x7f24d400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.340+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4017070 con 0x7f24e406e590 2024-09-16T15:48:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.340+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24d40053c0 con 0x7f24e406e590 2024-09-16T15:48:32.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.340+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4003850 con 0x7f24e406e590 2024-09-16T15:48:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.340+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e41f8f20 con 0x7f24e406e590 2024-09-16T15:48:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.340+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e4151e80 con 0x7f24e406e590 2024-09-16T15:48:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.341+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24e406fe40 con 0x7f24e406e590 2024-09-16T15:48:32.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.342+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f24d401a070 con 0x7f24e406e590 2024-09-16T15:48:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.343+0000 7f24daa00640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24c0079420 0x7f24c007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.343+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f24d40988c0 con 0x7f24e406e590 2024-09-16T15:48:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.344+0000 7f24d9400640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24c0079420 0x7f24c007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.344+0000 7f24daa00640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7f24c007f2e0 0x7f24c0081700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.344+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 --> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24c0081dd0 con 0x7f24c007f2e0 2024-09-16T15:48:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.344+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) ==== 24+0+0 (secure 0 0 0) 0x7f24d40a2050 con 0x7f24e406e590 2024-09-16T15:48:32.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.344+0000 7f24e2800640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7f24c007f2e0 0x7f24c0081700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.344+0000 7f24d9400640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24c0079420 0x7f24c007b8e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f24e4166d50 tx=0x7f24cc007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.345+0000 7f24e2800640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7f24c007f2e0 0x7f24c0081700 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.345+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 <== osd.3 v2:172.21.15.60:6824/3085378704 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24c0081dd0 con 0x7f24c007f2e0 2024-09-16T15:48:32.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.358+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 --> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f24e4075080 con 0x7f24c007f2e0 2024-09-16T15:48:32.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24daa00640 1 -- 172.21.15.60:0/2072911029 <== osd.3 v2:172.21.15.60:6824/3085378704 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f24e4075080 con 0x7f24c007f2e0 2024-09-16T15:48:32.360 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:48:32.360 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7f24c007f2e0 msgr2=0x7f24c0081700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6824/3085378704,v1:172.21.15.60:6825/3085378704] conn(0x7f24c007f2e0 0x7f24c0081700 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24c0079420 msgr2=0x7f24c007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f24c0079420 0x7f24c007b8e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f24e4166d50 tx=0x7f24cc007e90 comp rx=0 tx=0).stop 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e406e590 msgr2=0x7f24e4151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 --2- 172.21.15.60:0/2072911029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e406e590 0x7f24e4151940 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f24d400c3a0 tx=0x7f24d400c860 comp rx=0 tx=0).stop 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e2800640 1 -- 172.21.15.60:0/2072911029 reap_dead start 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 shutdown_connections 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 >> 172.21.15.60:0/2072911029 conn(0x7f24e40cfa70 msgr2=0x7f24e41546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.359+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 shutdown_connections 2024-09-16T15:48:32.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.360+0000 7f24e3200640 1 -- 172.21.15.60:0/2072911029 wait complete. 2024-09-16T15:48:32.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T15:48:32.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.534+0000 7fc62dc00640 1 -- 172.21.15.60:0/1442141253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6280ccf00 msgr2=0x7fc6280b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.534+0000 7fc62dc00640 1 --2- 172.21.15.60:0/1442141253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6280ccf00 0x7fc6280b2440 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fc62806dac0 tx=0x7fc61002a580 comp rx=0 tx=0).stop 2024-09-16T15:48:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.534+0000 7fc62dc00640 1 -- 172.21.15.60:0/1442141253 shutdown_connections 2024-09-16T15:48:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.534+0000 7fc62dc00640 1 --2- 172.21.15.60:0/1442141253 >> v2:172.21.15.100:3300/0 conn(0x7fc628073fb0 0x7fc628074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.534+0000 7fc62dc00640 1 --2- 172.21.15.60:0/1442141253 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6280ccf00 0x7fc6280b2440 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.534+0000 7fc62dc00640 1 -- 172.21.15.60:0/1442141253 >> 172.21.15.60:0/1442141253 conn(0x7fc6280b1950 msgr2=0x7fc6280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.534+0000 7fc62dc00640 1 -- 172.21.15.60:0/1442141253 shutdown_connections 2024-09-16T15:48:32.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.534+0000 7fc62dc00640 1 -- 172.21.15.60:0/1442141253 wait complete. 2024-09-16T15:48:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.536+0000 7fc62dc00640 1 Processor -- start 2024-09-16T15:48:32.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.536+0000 7fc62dc00640 1 -- start start 2024-09-16T15:48:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc62dc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc628073fb0 0x7fc6280b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc62dc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6280ccf00 0x7fc6281f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc62dc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62815f5a0 0x7fc6280cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc62dc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fc62806f2e0 con 0x7fc628073fb0 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc62dc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc62806f160 con 0x7fc6280ccf00 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc62dc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fc62806f460 con 0x7fc62815f5a0 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc626a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc628073fb0 0x7fc6280b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc626a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc628073fb0 0x7fc6280b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52854/0 (socket says 172.21.15.60:52854) 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc626a00640 1 -- 172.21.15.60:0/2699327599 learned_addr learned my addr 172.21.15.60:0/2699327599 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc61e000640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6280ccf00 0x7fc6281f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.537+0000 7fc627400640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62815f5a0 0x7fc6280cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc626a00640 1 -- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62815f5a0 msgr2=0x7fc6280cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc626a00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62815f5a0 0x7fc6280cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc626a00640 1 -- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6280ccf00 msgr2=0x7fc6281f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc626a00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6280ccf00 0x7fc6281f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc626a00640 1 -- 172.21.15.60:0/2699327599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62815a570 con 0x7fc628073fb0 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc627400640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62815f5a0 0x7fc6280cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc626a00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc628073fb0 0x7fc6280b4010 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc61800c730 tx=0x7fc61800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc61f400640 1 -- 172.21.15.60:0/2699327599 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618003b70 con 0x7fc628073fb0 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc61f400640 1 -- 172.21.15.60:0/2699327599 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc618003d30 con 0x7fc628073fb0 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc61f400640 1 -- 172.21.15.60:0/2699327599 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61800aab0 con 0x7fc628073fb0 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62815a7e0 con 0x7fc628073fb0 2024-09-16T15:48:32.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.538+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62815ada0 con 0x7fc628073fb0 2024-09-16T15:48:32.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.540+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62806f9f0 con 0x7fc628073fb0 2024-09-16T15:48:32.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.541+0000 7fc61f400640 1 -- 172.21.15.60:0/2699327599 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7fc61800ac50 con 0x7fc628073fb0 2024-09-16T15:48:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.542+0000 7fc61f400640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc604079420 0x7fc60407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.542+0000 7fc61f400640 1 -- 172.21.15.60:0/2699327599 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7fc618099260 con 0x7fc628073fb0 2024-09-16T15:48:32.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.542+0000 7fc61e000640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc604079420 0x7fc60407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.543+0000 7fc61e000640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc604079420 0x7fc60407b8e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc610033000 tx=0x7fc610004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.546+0000 7fc61f400640 1 -- 172.21.15.60:0/2699327599 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc618063120 con 0x7fc628073fb0 2024-09-16T15:48:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.747+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fc62815afe0 con 0x7fc604079420 2024-09-16T15:48:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.748+0000 7fc61f400640 1 -- 172.21.15.60:0/2699327599 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc62815afe0 con 0x7fc604079420 2024-09-16T15:48:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc604079420 msgr2=0x7fc60407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc604079420 0x7fc60407b8e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc610033000 tx=0x7fc610004730 comp rx=0 tx=0).stop 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc628073fb0 msgr2=0x7fc6280b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc628073fb0 0x7fc6280b4010 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc61800c730 tx=0x7fc61800cbf0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 shutdown_connections 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fc604079420 0x7fc60407b8e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc62815f5a0 0x7fc6280cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6280ccf00 0x7fc6281f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 --2- 172.21.15.60:0/2699327599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc628073fb0 0x7fc6280b4010 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.750+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 >> 172.21.15.60:0/2699327599 conn(0x7fc6280b1950 msgr2=0x7fc62809cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.751+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 shutdown_connections 2024-09-16T15:48:32.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.751+0000 7fc62dc00640 1 -- 172.21.15.60:0/2699327599 wait complete. 2024-09-16T15:48:32.768 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T15:48:32.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:48:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.931+0000 7f4961400640 1 -- 172.21.15.60:0/3185006738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bda0 msgr2=0x7f495c073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.931+0000 7f4961400640 1 --2- 172.21.15.60:0/3185006738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bda0 0x7f495c073010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f494c009860 tx=0x7f494c02a530 comp rx=0 tx=0).stop 2024-09-16T15:48:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.932+0000 7f4961400640 1 -- 172.21.15.60:0/3185006738 shutdown_connections 2024-09-16T15:48:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.932+0000 7f4961400640 1 --2- 172.21.15.60:0/3185006738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c15bda0 0x7f495c073010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.932+0000 7f4961400640 1 -- 172.21.15.60:0/3185006738 >> 172.21.15.60:0/3185006738 conn(0x7f495c0cfa70 msgr2=0x7f495c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.932+0000 7f4961400640 1 -- 172.21.15.60:0/3185006738 shutdown_connections 2024-09-16T15:48:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.932+0000 7f4961400640 1 -- 172.21.15.60:0/3185006738 wait complete. 2024-09-16T15:48:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.933+0000 7f4961400640 1 Processor -- start 2024-09-16T15:48:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.933+0000 7f4961400640 1 -- start start 2024-09-16T15:48:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.934+0000 7f4961400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c06e590 0x7f495c15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.934+0000 7f4961400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c070550 0x7f495c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.934+0000 7f4961400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c15bda0 0x7f495c0c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.934+0000 7f4961400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f495c0cbc60 con 0x7f495c15bda0 2024-09-16T15:48:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.934+0000 7f4961400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f495c0cbae0 con 0x7f495c06e590 2024-09-16T15:48:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.934+0000 7f4961400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f495c0cbde0 con 0x7f495c070550 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f4960a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c15bda0 0x7f495c0c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f4960a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c15bda0 0x7f495c0c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52864/0 (socket says 172.21.15.60:52864) 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f4960a00640 1 -- 172.21.15.60:0/270071727 learned_addr learned my addr 172.21.15.60:0/270071727 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f495be00640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c06e590 0x7f495c15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f495b400640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c070550 0x7f495c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f4960a00640 1 -- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c070550 msgr2=0x7f495c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f4960a00640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f495c070550 0x7f495c0c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f4960a00640 1 -- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c06e590 msgr2=0x7f495c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f4960a00640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f495c06e590 0x7f495c15eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.935+0000 7f4960a00640 1 -- 172.21.15.60:0/270071727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c0c8500 con 0x7f495c15bda0 2024-09-16T15:48:32.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.936+0000 7f4960a00640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c15bda0 0x7f495c0c5bb0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f495400c780 tx=0x7f495400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.936+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954003b70 con 0x7f495c15bda0 2024-09-16T15:48:32.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.936+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4954003d10 con 0x7f495c15bda0 2024-09-16T15:48:32.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.936+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c0c87e0 con 0x7f495c15bda0 2024-09-16T15:48:32.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.936+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495400aab0 con 0x7f495c15bda0 2024-09-16T15:48:32.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.936+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c0c5690 con 0x7f495c15bda0 2024-09-16T15:48:32.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.937+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f495c073010 con 0x7f495c15bda0 2024-09-16T15:48:32.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.938+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f495400ac50 con 0x7f495c15bda0 2024-09-16T15:48:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.939+0000 7f4953e00640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f493c079420 0x7f493c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.939+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f4954099290 con 0x7f495c15bda0 2024-09-16T15:48:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.939+0000 7f495be00640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f493c079420 0x7f493c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.940+0000 7f4953e00640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7f493c07f2e0 0x7f493c081700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.940+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 --> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f493c081dd0 con 0x7f493c07f2e0 2024-09-16T15:48:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.940+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) ==== 24+0+0 (secure 0 0 0) 0x7f49540a0220 con 0x7f495c15bda0 2024-09-16T15:48:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.940+0000 7f495b400640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7f493c07f2e0 0x7f493c081700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.940+0000 7f495be00640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f493c079420 0x7f493c07b8e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f494c034000 tx=0x7f494c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.941+0000 7f495b400640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7f493c07f2e0 0x7f493c081700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.942+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 <== osd.4 v2:172.21.15.100:6810/3116664612 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f493c081dd0 con 0x7f493c07f2e0 2024-09-16T15:48:32.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.955+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 --> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f495c072b90 con 0x7f493c07f2e0 2024-09-16T15:48:32.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4953e00640 1 -- 172.21.15.60:0/270071727 <== osd.4 v2:172.21.15.100:6810/3116664612 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f495c072b90 con 0x7f493c07f2e0 2024-09-16T15:48:32.957 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:48:32.957 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7f493c07f2e0 msgr2=0x7f493c081700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.100:6810/3116664612,v1:172.21.15.100:6811/3116664612] conn(0x7f493c07f2e0 0x7f493c081700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f493c079420 msgr2=0x7f493c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f493c079420 0x7f493c07b8e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f494c034000 tx=0x7f494c004730 comp rx=0 tx=0).stop 2024-09-16T15:48:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c15bda0 msgr2=0x7f495c0c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 --2- 172.21.15.60:0/270071727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f495c15bda0 0x7f495c0c5bb0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f495400c780 tx=0x7f495400cc40 comp rx=0 tx=0).stop 2024-09-16T15:48:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4960a00640 1 -- 172.21.15.60:0/270071727 reap_dead start 2024-09-16T15:48:32.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 shutdown_connections 2024-09-16T15:48:32.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 >> 172.21.15.60:0/270071727 conn(0x7f495c0cfa70 msgr2=0x7f495c0afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 shutdown_connections 2024-09-16T15:48:32.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:32.956+0000 7f4961400640 1 -- 172.21.15.60:0/270071727 wait complete. 2024-09-16T15:48:32.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T15:48:33.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.132+0000 7f1769000640 1 -- 172.21.15.60:0/3706078658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640ccf00 msgr2=0x7f17640b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.132+0000 7f1769000640 1 --2- 172.21.15.60:0/3706078658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640ccf00 0x7f17640b2440 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f174c005fd0 tx=0x7f174c02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.132+0000 7f1769000640 1 -- 172.21.15.60:0/3706078658 shutdown_connections 2024-09-16T15:48:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.132+0000 7f1769000640 1 --2- 172.21.15.60:0/3706078658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640ccf00 0x7f17640b2440 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.132+0000 7f1769000640 1 --2- 172.21.15.60:0/3706078658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764073fb0 0x7f1764074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.132+0000 7f1769000640 1 -- 172.21.15.60:0/3706078658 >> 172.21.15.60:0/3706078658 conn(0x7f17640b1950 msgr2=0x7f17640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.132+0000 7f1769000640 1 -- 172.21.15.60:0/3706078658 shutdown_connections 2024-09-16T15:48:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.132+0000 7f1769000640 1 -- 172.21.15.60:0/3706078658 wait complete. 2024-09-16T15:48:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.134+0000 7f1769000640 1 Processor -- start 2024-09-16T15:48:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.134+0000 7f1769000640 1 -- start start 2024-09-16T15:48:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1769000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1764073fb0 0x7f17640cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1769000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17640ccf00 0x7f17640b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1769000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f176415f5a0 0x7f17640b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1769000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f176406f2e0 con 0x7f1764073fb0 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1769000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f176406f160 con 0x7f176415f5a0 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1769000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f176406f460 con 0x7f17640ccf00 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1761e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1764073fb0 0x7f17640cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1761e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1764073fb0 0x7f17640cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52894/0 (socket says 172.21.15.60:52894) 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1761e00640 1 -- 172.21.15.60:0/2938919858 learned_addr learned my addr 172.21.15.60:0/2938919858 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1762800640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f176415f5a0 0x7f17640b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.135+0000 7f1759400640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17640ccf00 0x7f17640b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1761e00640 1 -- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17640ccf00 msgr2=0x7f17640b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1761e00640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17640ccf00 0x7f17640b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1761e00640 1 -- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f176415f5a0 msgr2=0x7f17640b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1761e00640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f176415f5a0 0x7f17640b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1761e00640 1 -- 172.21.15.60:0/2938919858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176409aaa0 con 0x7f1764073fb0 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1759400640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17640ccf00 0x7f17640b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1761e00640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1764073fb0 0x7f17640cb4a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f175400b3f0 tx=0x7f175400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f175aa00640 1 -- 172.21.15.60:0/2938919858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754015020 con 0x7f1764073fb0 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f175aa00640 1 -- 172.21.15.60:0/2938919858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1754011920 con 0x7f1764073fb0 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f175aa00640 1 -- 172.21.15.60:0/2938919858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754011d70 con 0x7f1764073fb0 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176409ad80 con 0x7f1764073fb0 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.136+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176409b340 con 0x7f1764073fb0 2024-09-16T15:48:33.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.137+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176406f990 con 0x7f1764073fb0 2024-09-16T15:48:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.139+0000 7f175aa00640 1 -- 172.21.15.60:0/2938919858 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f1754018020 con 0x7f1764073fb0 2024-09-16T15:48:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.139+0000 7f175aa00640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1740079420 0x7f174007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.140+0000 7f175aa00640 1 -- 172.21.15.60:0/2938919858 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f1754099180 con 0x7f1764073fb0 2024-09-16T15:48:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.140+0000 7f1759400640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1740079420 0x7f174007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.140+0000 7f1759400640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1740079420 0x7f174007b8e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f174c033000 tx=0x7f174c004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:33.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.144+0000 7f175aa00640 1 -- 172.21.15.60:0/2938919858 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f175401f030 con 0x7f1764073fb0 2024-09-16T15:48:33.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.338+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f176409b740 con 0x7f1740079420 2024-09-16T15:48:33.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.338+0000 7f175aa00640 1 -- 172.21.15.60:0/2938919858 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f176409b740 con 0x7f1740079420 2024-09-16T15:48:33.339 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.340+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1740079420 msgr2=0x7f174007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.340+0000 7f1769000640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1740079420 0x7f174007b8e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f174c033000 tx=0x7f174c004730 comp rx=0 tx=0).stop 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.340+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1764073fb0 msgr2=0x7f17640cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.340+0000 7f1769000640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1764073fb0 0x7f17640cb4a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f175400b3f0 tx=0x7f175400b8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.341+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 shutdown_connections 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.341+0000 7f1769000640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1740079420 0x7f174007b8e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.341+0000 7f1769000640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f176415f5a0 0x7f17640b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.341+0000 7f1769000640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17640ccf00 0x7f17640b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.341+0000 7f1769000640 1 --2- 172.21.15.60:0/2938919858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1764073fb0 0x7f17640cb4a0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.341+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 >> 172.21.15.60:0/2938919858 conn(0x7f17640b1950 msgr2=0x7f17640b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:33.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.341+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 shutdown_connections 2024-09-16T15:48:33.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.341+0000 7f1769000640 1 -- 172.21.15.60:0/2938919858 wait complete. 2024-09-16T15:48:33.358 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T15:48:33.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:48:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.521+0000 7f4d64800640 1 -- 172.21.15.60:0/1606636444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c159aa0 msgr2=0x7f4d5c0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.521+0000 7f4d64800640 1 --2- 172.21.15.60:0/1606636444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c159aa0 0x7f4d5c0741a0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009900 tx=0x7f4d5002a610 comp rx=0 tx=0).stop 2024-09-16T15:48:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.521+0000 7f4d64800640 1 -- 172.21.15.60:0/1606636444 shutdown_connections 2024-09-16T15:48:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.521+0000 7f4d64800640 1 --2- 172.21.15.60:0/1606636444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5c15cff0 0x7f4d5c06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.521+0000 7f4d64800640 1 --2- 172.21.15.60:0/1606636444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c159aa0 0x7f4d5c0741a0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.521+0000 7f4d64800640 1 -- 172.21.15.60:0/1606636444 >> 172.21.15.60:0/1606636444 conn(0x7f4d5c0b18e0 msgr2=0x7f4d5c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.522+0000 7f4d64800640 1 -- 172.21.15.60:0/1606636444 shutdown_connections 2024-09-16T15:48:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.522+0000 7f4d64800640 1 -- 172.21.15.60:0/1606636444 wait complete. 2024-09-16T15:48:33.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.523+0000 7f4d64800640 1 Processor -- start 2024-09-16T15:48:33.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.523+0000 7f4d64800640 1 -- start start 2024-09-16T15:48:33.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d64800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c06f220 0x7f4d5c1f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d64800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c159aa0 0x7f4d5c0cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d64800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5c15cff0 0x7f4d5c1f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d64800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c06fa60 con 0x7f4d5c06f220 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d64800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c06f8e0 con 0x7f4d5c159aa0 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d64800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f4d5c0cc8a0 con 0x7f4d5c15cff0 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d61e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c06f220 0x7f4d5c1f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d61e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c06f220 0x7f4d5c1f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52918/0 (socket says 172.21.15.60:52918) 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.524+0000 7f4d61e00640 1 -- 172.21.15.60:0/3403380675 learned_addr learned my addr 172.21.15.60:0/3403380675 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d61400640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c159aa0 0x7f4d5c0cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d62800640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5c15cff0 0x7f4d5c1f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d61e00640 1 -- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5c15cff0 msgr2=0x7f4d5c1f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d61e00640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d5c15cff0 0x7f4d5c1f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d61e00640 1 -- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c159aa0 msgr2=0x7f4d5c0cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d61e00640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5c159aa0 0x7f4d5c0cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d61e00640 1 -- 172.21.15.60:0/3403380675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5c09ba50 con 0x7f4d5c06f220 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d61e00640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c06f220 0x7f4d5c1f2af0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f4d50000c00 tx=0x7f4d50003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5003f070 con 0x7f4d5c06f220 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d50005600 con 0x7f4d5c06f220 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50035030 con 0x7f4d5c06f220 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.525+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c158680 con 0x7f4d5c06f220 2024-09-16T15:48:33.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.526+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c158ec0 con 0x7f4d5c06f220 2024-09-16T15:48:33.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.526+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d5c0741a0 con 0x7f4d5c06f220 2024-09-16T15:48:33.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.528+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f4d50042070 con 0x7f4d5c06f220 2024-09-16T15:48:33.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.529+0000 7f4d56000640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d40079420 0x7f4d4007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.529+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f4d500ba0b0 con 0x7f4d5c06f220 2024-09-16T15:48:33.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.529+0000 7f4d56000640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7f4d4007f2e0 0x7f4d40081700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.529+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 --> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d5003fe70 con 0x7f4d4007f2e0 2024-09-16T15:48:33.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.529+0000 7f4d61400640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d40079420 0x7f4d4007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.529+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) ==== 24+0+0 (secure 0 0 0) 0x7f4d500c12a0 con 0x7f4d5c06f220 2024-09-16T15:48:33.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.529+0000 7f4d62800640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7f4d4007f2e0 0x7f4d40081700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.530+0000 7f4d61400640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d40079420 0x7f4d4007b8e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0d0030 tx=0x7f4d4c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:33.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.530+0000 7f4d62800640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7f4d4007f2e0 0x7f4d40081700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:33.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.531+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 <== osd.5 v2:172.21.15.100:6818/1966336275 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d5003fe70 con 0x7f4d4007f2e0 2024-09-16T15:48:33.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.545+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 --> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4d5c070980 con 0x7f4d4007f2e0 2024-09-16T15:48:33.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.545+0000 7f4d56000640 1 -- 172.21.15.60:0/3403380675 <== osd.5 v2:172.21.15.100:6818/1966336275 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4d5c070980 con 0x7f4d4007f2e0 2024-09-16T15:48:33.547 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:48:33.547 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:33.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7f4d4007f2e0 msgr2=0x7f4d40081700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.100:6818/1966336275,v1:172.21.15.100:6819/1966336275] conn(0x7f4d4007f2e0 0x7f4d40081700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d40079420 msgr2=0x7f4d4007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f4d40079420 0x7f4d4007b8e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0d0030 tx=0x7f4d4c005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:33.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c06f220 msgr2=0x7f4d5c1f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 --2- 172.21.15.60:0/3403380675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d5c06f220 0x7f4d5c1f2af0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f4d50000c00 tx=0x7f4d50003820 comp rx=0 tx=0).stop 2024-09-16T15:48:33.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d62800640 1 -- 172.21.15.60:0/3403380675 reap_dead start 2024-09-16T15:48:33.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 shutdown_connections 2024-09-16T15:48:33.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 >> 172.21.15.60:0/3403380675 conn(0x7f4d5c0b18e0 msgr2=0x7f4d5c0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:33.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 shutdown_connections 2024-09-16T15:48:33.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.546+0000 7f4d64800640 1 -- 172.21.15.60:0/3403380675 wait complete. 2024-09-16T15:48:33.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T15:48:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.722+0000 7f8c2de00640 1 -- 172.21.15.60:0/2844338498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c2815f5a0 msgr2=0x7f8c280b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.722+0000 7f8c2de00640 1 --2- 172.21.15.60:0/2844338498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c2815f5a0 0x7f8c280b2440 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8c18005fd0 tx=0x7f8c1802a540 comp rx=0 tx=0).stop 2024-09-16T15:48:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.722+0000 7f8c2de00640 1 -- 172.21.15.60:0/2844338498 shutdown_connections 2024-09-16T15:48:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.722+0000 7f8c2de00640 1 --2- 172.21.15.60:0/2844338498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c280ccf00 0x7f8c280acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.722+0000 7f8c2de00640 1 --2- 172.21.15.60:0/2844338498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c2815f5a0 0x7f8c280b2440 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.722+0000 7f8c2de00640 1 -- 172.21.15.60:0/2844338498 >> 172.21.15.60:0/2844338498 conn(0x7f8c280b1950 msgr2=0x7f8c280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.722+0000 7f8c2de00640 1 -- 172.21.15.60:0/2844338498 shutdown_connections 2024-09-16T15:48:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.722+0000 7f8c2de00640 1 -- 172.21.15.60:0/2844338498 wait complete. 2024-09-16T15:48:33.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.724+0000 7f8c2de00640 1 Processor -- start 2024-09-16T15:48:33.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.724+0000 7f8c2de00640 1 -- start start 2024-09-16T15:48:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c2de00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c280ad270 0x7f8c28152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c2de00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c280ccf00 0x7f8c281531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c2de00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c2815f5a0 0x7f8c281f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c2de00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2806f2e0 con 0x7f8c280ad270 2024-09-16T15:48:33.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c2de00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2806f160 con 0x7f8c280ccf00 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c2de00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2806f460 con 0x7f8c2815f5a0 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c26a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c280ad270 0x7f8c28152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c26a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c280ad270 0x7f8c28152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52946/0 (socket says 172.21.15.60:52946) 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c26a00640 1 -- 172.21.15.60:0/1887524397 learned_addr learned my addr 172.21.15.60:0/1887524397 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c27400640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c2815f5a0 0x7f8c281f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c1e000640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c280ccf00 0x7f8c281531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c26a00640 1 -- 172.21.15.60:0/1887524397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c2815f5a0 msgr2=0x7f8c281f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c26a00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c2815f5a0 0x7f8c281f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c26a00640 1 -- 172.21.15.60:0/1887524397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c280ccf00 msgr2=0x7f8c281531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c26a00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c280ccf00 0x7f8c281531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.725+0000 7f8c26a00640 1 -- 172.21.15.60:0/1887524397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c28152040 con 0x7f8c280ad270 2024-09-16T15:48:33.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.726+0000 7f8c26a00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c280ad270 0x7f8c28152c60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f8c18034040 tx=0x7f8c180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:33.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.726+0000 7f8c1fe00640 1 -- 172.21.15.60:0/1887524397 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1803e030 con 0x7f8c280ad270 2024-09-16T15:48:33.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.726+0000 7f8c1fe00640 1 -- 172.21.15.60:0/1887524397 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c18003d90 con 0x7f8c280ad270 2024-09-16T15:48:33.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.726+0000 7f8c1fe00640 1 -- 172.21.15.60:0/1887524397 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18035040 con 0x7f8c280ad270 2024-09-16T15:48:33.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.726+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c28150fa0 con 0x7f8c280ad270 2024-09-16T15:48:33.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.726+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c28151560 con 0x7f8c280ad270 2024-09-16T15:48:33.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.727+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c28072f50 con 0x7f8c280ad270 2024-09-16T15:48:33.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.728+0000 7f8c1fe00640 1 -- 172.21.15.60:0/1887524397 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f8c18041030 con 0x7f8c280ad270 2024-09-16T15:48:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.730+0000 7f8c1fe00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c04079450 0x7f8c0407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.730+0000 7f8c1fe00640 1 -- 172.21.15.60:0/1887524397 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f8c180b9f60 con 0x7f8c280ad270 2024-09-16T15:48:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.730+0000 7f8c1e000640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c04079450 0x7f8c0407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.731+0000 7f8c1e000640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c04079450 0x7f8c0407b910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8c10005fd0 tx=0x7f8c10005850 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:33.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.734+0000 7f8c1fe00640 1 -- 172.21.15.60:0/1887524397 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8c28072f50 con 0x7f8c280ad270 2024-09-16T15:48:33.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.933+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f8c2806f9f0 con 0x7f8c04079450 2024-09-16T15:48:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.934+0000 7f8c1fe00640 1 -- 172.21.15.60:0/1887524397 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8c2806f9f0 con 0x7f8c04079450 2024-09-16T15:48:33.935 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T15:48:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c04079450 msgr2=0x7f8c0407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c04079450 0x7f8c0407b910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8c10005fd0 tx=0x7f8c10005850 comp rx=0 tx=0).stop 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c280ad270 msgr2=0x7f8c28152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c280ad270 0x7f8c28152c60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f8c18034040 tx=0x7f8c180038a0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 shutdown_connections 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f8c04079450 0x7f8c0407b910 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8c2815f5a0 0x7f8c281f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c280ccf00 0x7f8c281531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 --2- 172.21.15.60:0/1887524397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c280ad270 0x7f8c28152c60 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 >> 172.21.15.60:0/1887524397 conn(0x7f8c280b1950 msgr2=0x7f8c2809b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 shutdown_connections 2024-09-16T15:48:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:33.936+0000 7f8c2de00640 1 -- 172.21.15.60:0/1887524397 wait complete. 2024-09-16T15:48:33.954 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T15:48:33.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:48:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.118+0000 7f985bc00640 1 -- 172.21.15.60:0/2993899678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f985406f220 msgr2=0x7f985406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.118+0000 7f985bc00640 1 --2- 172.21.15.60:0/2993899678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f985406f220 0x7f985406f6b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f984c005fd0 tx=0x7f984c02aaa0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.118+0000 7f985bc00640 1 -- 172.21.15.60:0/2993899678 shutdown_connections 2024-09-16T15:48:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.118+0000 7f985bc00640 1 --2- 172.21.15.60:0/2993899678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f985406f220 0x7f985406f6b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.118+0000 7f985bc00640 1 --2- 172.21.15.60:0/2993899678 >> v2:172.21.15.100:3300/0 conn(0x7f9854159aa0 0x7f98540741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.118+0000 7f985bc00640 1 -- 172.21.15.60:0/2993899678 >> 172.21.15.60:0/2993899678 conn(0x7f98540b18e0 msgr2=0x7f98540b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.118+0000 7f985bc00640 1 -- 172.21.15.60:0/2993899678 shutdown_connections 2024-09-16T15:48:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.118+0000 7f985bc00640 1 -- 172.21.15.60:0/2993899678 wait complete. 2024-09-16T15:48:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.120+0000 7f985bc00640 1 Processor -- start 2024-09-16T15:48:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.120+0000 7f985bc00640 1 -- start start 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f985bc00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f985406f220 0x7f98540b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f985bc00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9854159aa0 0x7f98540add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f985bc00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f985415cff0 0x7f98540ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f985bc00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f98540cc520 con 0x7f9854159aa0 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f985bc00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98540cc3a0 con 0x7f985406f220 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f985bc00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f98540cc6a0 con 0x7f985415cff0 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f9853e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9854159aa0 0x7f98540add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f9853e00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9854159aa0 0x7f98540add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52950/0 (socket says 172.21.15.60:52950) 2024-09-16T15:48:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f9853e00640 1 -- 172.21.15.60:0/590783670 learned_addr learned my addr 172.21.15.60:0/590783670 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f9859200640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f985406f220 0x7f98540b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.121+0000 7f9859c00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f985415cff0 0x7f98540ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9853e00640 1 -- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f985415cff0 msgr2=0x7f98540ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9853e00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f985415cff0 0x7f98540ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9853e00640 1 -- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f985406f220 msgr2=0x7f98540b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9853e00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f985406f220 0x7f98540b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9853e00640 1 -- 172.21.15.60:0/590783670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98540b4cc0 con 0x7f9854159aa0 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9859200640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f985406f220 0x7f98540b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:48:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9853e00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9854159aa0 0x7f98540add80 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f984000c3a0 tx=0x7f984000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840017070 con 0x7f9854159aa0 2024-09-16T15:48:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98400053c0 con 0x7f9854159aa0 2024-09-16T15:48:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98541f8d50 con 0x7f9854159aa0 2024-09-16T15:48:34.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840003850 con 0x7f9854159aa0 2024-09-16T15:48:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.122+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98541f4fd0 con 0x7f9854159aa0 2024-09-16T15:48:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.124+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98540ccba0 con 0x7f9854159aa0 2024-09-16T15:48:34.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.125+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f984001a070 con 0x7f9854159aa0 2024-09-16T15:48:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.126+0000 7f9850c00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9834079450 0x7f983407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.126+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f9840098800 con 0x7f9854159aa0 2024-09-16T15:48:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.126+0000 7f9859200640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9834079450 0x7f983407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.126+0000 7f9850c00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f983407f310 0x7f9834081730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.126+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 --> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9840016ec0 con 0x7f983407f310 2024-09-16T15:48:34.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.126+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) ==== 24+0+0 (secure 0 0 0) 0x7f98400a2050 con 0x7f9854159aa0 2024-09-16T15:48:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.127+0000 7f9859200640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9834079450 0x7f983407b910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9848009fc0 tx=0x7f9848005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.127+0000 7f9859c00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f983407f310 0x7f9834081730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.127+0000 7f9859c00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f983407f310 0x7f9834081730 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.128+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 <== osd.6 v2:172.21.15.100:6826/3702271988 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9840016ec0 con 0x7f983407f310 2024-09-16T15:48:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.141+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 --> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9854070980 con 0x7f983407f310 2024-09-16T15:48:34.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.142+0000 7f9850c00640 1 -- 172.21.15.60:0/590783670 <== osd.6 v2:172.21.15.100:6826/3702271988 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9854070980 con 0x7f983407f310 2024-09-16T15:48:34.144 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:48:34.144 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f983407f310 msgr2=0x7f9834081730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.100:6826/3702271988,v1:172.21.15.100:6827/3702271988] conn(0x7f983407f310 0x7f9834081730 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9834079450 msgr2=0x7f983407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9834079450 0x7f983407b910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9848009fc0 tx=0x7f9848005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9854159aa0 msgr2=0x7f98540add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 --2- 172.21.15.60:0/590783670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9854159aa0 0x7f98540add80 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f984000c3a0 tx=0x7f984000c860 comp rx=0 tx=0).stop 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f9859c00640 1 -- 172.21.15.60:0/590783670 reap_dead start 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 shutdown_connections 2024-09-16T15:48:34.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 >> 172.21.15.60:0/590783670 conn(0x7f98540b18e0 msgr2=0x7f985415b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 shutdown_connections 2024-09-16T15:48:34.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.143+0000 7f985bc00640 1 -- 172.21.15.60:0/590783670 wait complete. 2024-09-16T15:48:34.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T15:48:34.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.319+0000 7f9ed3000640 1 -- 172.21.15.60:0/824677697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b5370 msgr2=0x7f9ecc0b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.319+0000 7f9ed3000640 1 --2- 172.21.15.60:0/824677697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b5370 0x7f9ecc0b3bc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0d4ea0 tx=0x7f9eb802a480 comp rx=0 tx=0).stop 2024-09-16T15:48:34.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.319+0000 7f9ed3000640 1 -- 172.21.15.60:0/824677697 shutdown_connections 2024-09-16T15:48:34.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.319+0000 7f9ed3000640 1 --2- 172.21.15.60:0/824677697 >> v2:172.21.15.100:3300/0 conn(0x7f9ecc0b4100 0x7f9ecc0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.319+0000 7f9ed3000640 1 --2- 172.21.15.60:0/824677697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b5370 0x7f9ecc0b3bc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.319+0000 7f9ed3000640 1 -- 172.21.15.60:0/824677697 >> 172.21.15.60:0/824677697 conn(0x7f9ecc0d3df0 msgr2=0x7f9ecc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:34.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.319+0000 7f9ed3000640 1 -- 172.21.15.60:0/824677697 shutdown_connections 2024-09-16T15:48:34.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.319+0000 7f9ed3000640 1 -- 172.21.15.60:0/824677697 wait complete. 2024-09-16T15:48:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.321+0000 7f9ed3000640 1 Processor -- start 2024-09-16T15:48:34.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.321+0000 7f9ed3000640 1 -- start start 2024-09-16T15:48:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed3000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b4100 0x7f9ecc1f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed3000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0b5370 0x7f9ecc0cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed3000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc161440 0x7f9ecc1f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed3000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0b2040 con 0x7f9ecc161440 2024-09-16T15:48:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed3000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0b1ec0 con 0x7f9ecc0b5370 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed3000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f9ecc0b21c0 con 0x7f9ecc0b4100 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed2600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc161440 0x7f9ecc1f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed2600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc161440 0x7f9ecc1f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52956/0 (socket says 172.21.15.60:52956) 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed2600640 1 -- 172.21.15.60:0/2942629147 learned_addr learned my addr 172.21.15.60:0/2942629147 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed1200640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0b5370 0x7f9ecc0cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.322+0000 7f9ed1c00640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b4100 0x7f9ecc1f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ed2600640 1 -- 172.21.15.60:0/2942629147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b4100 msgr2=0x7f9ecc1f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ed2600640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b4100 0x7f9ecc1f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ed2600640 1 -- 172.21.15.60:0/2942629147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0b5370 msgr2=0x7f9ecc0cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ed2600640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0b5370 0x7f9ecc0cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ed2600640 1 -- 172.21.15.60:0/2942629147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc1f5980 con 0x7f9ecc161440 2024-09-16T15:48:34.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ed2600640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc161440 0x7f9ecc1f48e0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400eb40 tx=0x7f9ec400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ec9600640 1 -- 172.21.15.60:0/2942629147 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4004b20 con 0x7f9ecc161440 2024-09-16T15:48:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ec9600640 1 -- 172.21.15.60:0/2942629147 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ec4004ce0 con 0x7f9ecc161440 2024-09-16T15:48:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ec9600640 1 -- 172.21.15.60:0/2942629147 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4005280 con 0x7f9ecc161440 2024-09-16T15:48:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc0cbe10 con 0x7f9ecc161440 2024-09-16T15:48:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.323+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc0cc370 con 0x7f9ecc161440 2024-09-16T15:48:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.325+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ecc160c00 con 0x7f9ecc161440 2024-09-16T15:48:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.326+0000 7f9ec9600640 1 -- 172.21.15.60:0/2942629147 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f9ec4005420 con 0x7f9ecc161440 2024-09-16T15:48:34.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.327+0000 7f9ec9600640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9eac079420 0x7f9eac07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.327+0000 7f9ec9600640 1 -- 172.21.15.60:0/2942629147 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f9ec4099ea0 con 0x7f9ecc161440 2024-09-16T15:48:34.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.327+0000 7f9ed1c00640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9eac079420 0x7f9eac07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.328+0000 7f9ed1c00640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9eac079420 0x7f9eac07b8e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc06d7c0 tx=0x7f9ec0007e90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.331+0000 7f9ec9600640 1 -- 172.21.15.60:0/2942629147 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9ecc160c00 con 0x7f9ecc161440 2024-09-16T15:48:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.527+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f9ecc0b1770 con 0x7f9eac079420 2024-09-16T15:48:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.528+0000 7f9ec9600640 1 -- 172.21.15.60:0/2942629147 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9ecc0b1770 con 0x7f9eac079420 2024-09-16T15:48:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T15:48:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.530+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9eac079420 msgr2=0x7f9eac07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.530+0000 7f9ed3000640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9eac079420 0x7f9eac07b8e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc06d7c0 tx=0x7f9ec0007e90 comp rx=0 tx=0).stop 2024-09-16T15:48:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.530+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc161440 msgr2=0x7f9ecc1f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.530+0000 7f9ed3000640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc161440 0x7f9ecc1f48e0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400eb40 tx=0x7f9ec400c2d0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.531+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 shutdown_connections 2024-09-16T15:48:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.531+0000 7f9ed3000640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f9eac079420 0x7f9eac07b8e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.531+0000 7f9ed3000640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc161440 0x7f9ecc1f48e0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.531+0000 7f9ed3000640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0b5370 0x7f9ecc0cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.531+0000 7f9ed3000640 1 --2- 172.21.15.60:0/2942629147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b4100 0x7f9ecc1f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.531+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 >> 172.21.15.60:0/2942629147 conn(0x7f9ecc0d3df0 msgr2=0x7f9ecc0b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:34.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.531+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 shutdown_connections 2024-09-16T15:48:34.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.531+0000 7f9ed3000640 1 -- 172.21.15.60:0/2942629147 wait complete. 2024-09-16T15:48:34.547 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T15:48:34.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:48:34.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.710+0000 7f185d000640 1 -- 172.21.15.60:0/2402093633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185815cff0 msgr2=0x7f185806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.710+0000 7f185d000640 1 --2- 172.21.15.60:0/2402093633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185815cff0 0x7f185806e6f0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f1840005fd0 tx=0x7f184002a540 comp rx=0 tx=0).stop 2024-09-16T15:48:34.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.710+0000 7f185d000640 1 -- 172.21.15.60:0/2402093633 shutdown_connections 2024-09-16T15:48:34.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.710+0000 7f185d000640 1 --2- 172.21.15.60:0/2402093633 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f185815cff0 0x7f185806e6f0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.710+0000 7f185d000640 1 --2- 172.21.15.60:0/2402093633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f185806f220 0x7f185806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.710+0000 7f185d000640 1 -- 172.21.15.60:0/2402093633 >> 172.21.15.60:0/2402093633 conn(0x7f18580b18e0 msgr2=0x7f18580b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:34.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.710+0000 7f185d000640 1 -- 172.21.15.60:0/2402093633 shutdown_connections 2024-09-16T15:48:34.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.710+0000 7f185d000640 1 -- 172.21.15.60:0/2402093633 wait complete. 2024-09-16T15:48:34.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.712+0000 7f185d000640 1 Processor -- start 2024-09-16T15:48:34.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.712+0000 7f185d000640 1 -- start start 2024-09-16T15:48:34.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f185d000640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f185806f220 0x7f185814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f185d000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1858159aa0 0x7f1858153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f185d000640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185815cff0 0x7f1858154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f185d000640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f18580cc520 con 0x7f1858159aa0 2024-09-16T15:48:34.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f185d000640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18580cc3a0 con 0x7f185815cff0 2024-09-16T15:48:34.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f185d000640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f18580cc6a0 con 0x7f185806f220 2024-09-16T15:48:34.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f184d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1858159aa0 0x7f1858153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f184d400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1858159aa0 0x7f1858153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52974/0 (socket says 172.21.15.60:52974) 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f184d400640 1 -- 172.21.15.60:0/2515826643 learned_addr learned my addr 172.21.15.60:0/2515826643 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f1855e00640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f185806f220 0x7f185814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f1856800640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185815cff0 0x7f1858154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f184d400640 1 -- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f185806f220 msgr2=0x7f185814b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f184d400640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f185806f220 0x7f185814b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f184d400640 1 -- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185815cff0 msgr2=0x7f1858154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f184d400640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185815cff0 0x7f1858154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f184d400640 1 -- 172.21.15.60:0/2515826643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580adab0 con 0x7f1858159aa0 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.713+0000 7f1856800640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f185815cff0 0x7f1858154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.714+0000 7f184d400640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1858159aa0 0x7f1858153be0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f1840034040 tx=0x7f18400039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.714+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184003e030 con 0x7f1858159aa0 2024-09-16T15:48:34.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.714+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18400047a0 con 0x7f1858159aa0 2024-09-16T15:48:34.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.714+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840035040 con 0x7f1858159aa0 2024-09-16T15:48:34.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.714+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18580add90 con 0x7f1858159aa0 2024-09-16T15:48:34.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.714+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18580b2f30 con 0x7f1858159aa0 2024-09-16T15:48:34.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.715+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18580b2d30 con 0x7f1858159aa0 2024-09-16T15:48:34.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.716+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f1840041030 con 0x7f1858159aa0 2024-09-16T15:48:34.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.717+0000 7f184ea00640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1834079420 0x7f183407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.717+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f18400ba2e0 con 0x7f1858159aa0 2024-09-16T15:48:34.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.718+0000 7f184ea00640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7f183407f2e0 0x7f1834081700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.718+0000 7f1855e00640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1834079420 0x7f183407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.718+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 --> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1840001d30 con 0x7f183407f2e0 2024-09-16T15:48:34.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.718+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=176) ==== 24+0+0 (secure 0 0 0) 0x7f18400bcb40 con 0x7f1858159aa0 2024-09-16T15:48:34.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.718+0000 7f1856800640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7f183407f2e0 0x7f1834081700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.718+0000 7f1855e00640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1834079420 0x7f183407b8e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1848004ea0 tx=0x7f1848009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.719+0000 7f1856800640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7f183407f2e0 0x7f1834081700 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.720+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 <== osd.7 v2:172.21.15.100:6802/1334072453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1840001d30 con 0x7f183407f2e0 2024-09-16T15:48:34.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.733+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 --> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f18580ccbc0 con 0x7f183407f2e0 2024-09-16T15:48:34.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f184ea00640 1 -- 172.21.15.60:0/2515826643 <== osd.7 v2:172.21.15.100:6802/1334072453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f18580ccbc0 con 0x7f183407f2e0 2024-09-16T15:48:34.735 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-16T15:48:34.735 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:48:34.735 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7f183407f2e0 msgr2=0x7f1834081700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.100:6802/1334072453,v1:172.21.15.100:6803/1334072453] conn(0x7f183407f2e0 0x7f1834081700 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1834079420 msgr2=0x7f183407b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f1834079420 0x7f183407b8e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1848004ea0 tx=0x7f1848009040 comp rx=0 tx=0).stop 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1858159aa0 msgr2=0x7f1858153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 --2- 172.21.15.60:0/2515826643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1858159aa0 0x7f1858153be0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f1840034040 tx=0x7f18400039e0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f1856800640 1 -- 172.21.15.60:0/2515826643 reap_dead start 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 shutdown_connections 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 >> 172.21.15.60:0/2515826643 conn(0x7f18580b18e0 msgr2=0x7f185815d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 shutdown_connections 2024-09-16T15:48:34.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.734+0000 7f185d000640 1 -- 172.21.15.60:0/2515826643 wait complete. 2024-09-16T15:48:34.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T15:48:34.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.910+0000 7f0f65200640 1 -- 172.21.15.60:0/3876008349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 msgr2=0x7f0f600b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.910+0000 7f0f65200640 1 --2- 172.21.15.60:0/3876008349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 0x7f0f600b2440 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f0f600cfbb0 tx=0x7f0f5002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.911+0000 7f0f65200640 1 -- 172.21.15.60:0/3876008349 shutdown_connections 2024-09-16T15:48:34.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.911+0000 7f0f65200640 1 --2- 172.21.15.60:0/3876008349 >> v2:172.21.15.100:3300/0 conn(0x7f0f600ccf00 0x7f0f600acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.911+0000 7f0f65200640 1 --2- 172.21.15.60:0/3876008349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 0x7f0f600b2440 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.911+0000 7f0f65200640 1 -- 172.21.15.60:0/3876008349 >> 172.21.15.60:0/3876008349 conn(0x7f0f600b1950 msgr2=0x7f0f600b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:34.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.911+0000 7f0f65200640 1 -- 172.21.15.60:0/3876008349 shutdown_connections 2024-09-16T15:48:34.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.911+0000 7f0f65200640 1 -- 172.21.15.60:0/3876008349 wait complete. 2024-09-16T15:48:34.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.912+0000 7f0f65200640 1 Processor -- start 2024-09-16T15:48:34.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.912+0000 7f0f65200640 1 -- start start 2024-09-16T15:48:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.913+0000 7f0f65200640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f600ad270 0x7f0f60143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.913+0000 7f0f65200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f600ccf00 0x7f0f60150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.913+0000 7f0f65200640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 0x7f0f60152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.913+0000 7f0f65200640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6006f160 con 0x7f0f6015f5a0 2024-09-16T15:48:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.913+0000 7f0f65200640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6006f2e0 con 0x7f0f600ccf00 2024-09-16T15:48:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.913+0000 7f0f65200640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6006f460 con 0x7f0f600ad270 2024-09-16T15:48:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 0x7f0f60152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 0x7f0f60152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52998/0 (socket says 172.21.15.60:52998) 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 -- 172.21.15.60:0/2383637771 learned_addr learned my addr 172.21.15.60:0/2383637771 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5de00640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f600ad270 0x7f0f60143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f57e00640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f600ccf00 0x7f0f60150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 -- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f600ad270 msgr2=0x7f0f60143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f600ad270 0x7f0f60143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 -- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f600ccf00 msgr2=0x7f0f60150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f600ccf00 0x7f0f60150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 -- 172.21.15.60:0/2383637771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f60151720 con 0x7f0f6015f5a0 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f57e00640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f600ccf00 0x7f0f60150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.914+0000 7f0f5e800640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 0x7f0f60152e70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800c780 tx=0x7f0f5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.915+0000 7f0f56000640 1 -- 172.21.15.60:0/2383637771 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58017070 con 0x7f0f6015f5a0 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.915+0000 7f0f56000640 1 -- 172.21.15.60:0/2383637771 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f58005070 con 0x7f0f6015f5a0 2024-09-16T15:48:34.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.915+0000 7f0f56000640 1 -- 172.21.15.60:0/2383637771 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58005500 con 0x7f0f6015f5a0 2024-09-16T15:48:34.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.915+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f60151a00 con 0x7f0f6015f5a0 2024-09-16T15:48:34.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.915+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f60151fc0 con 0x7f0f6015f5a0 2024-09-16T15:48:34.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.916+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6006fa10 con 0x7f0f6015f5a0 2024-09-16T15:48:34.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.917+0000 7f0f56000640 1 -- 172.21.15.60:0/2383637771 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f0f5801a070 con 0x7f0f6015f5a0 2024-09-16T15:48:34.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.918+0000 7f0f56000640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f3c079420 0x7f0f3c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.918+0000 7f0f56000640 1 -- 172.21.15.60:0/2383637771 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f0f58098bc0 con 0x7f0f6015f5a0 2024-09-16T15:48:34.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.918+0000 7f0f5de00640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f3c079420 0x7f0f3c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.919+0000 7f0f5de00640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f3c079420 0x7f0f3c07b8e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0f50033000 tx=0x7f0f50004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:34.922+0000 7f0f56000640 1 -- 172.21.15.60:0/2383637771 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0f58062a20 con 0x7f0f6015f5a0 2024-09-16T15:48:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.116+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f0f6006aaf0 con 0x7f0f3c079420 2024-09-16T15:48:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.116+0000 7f0f56000640 1 -- 172.21.15.60:0/2383637771 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0f6006aaf0 con 0x7f0f3c079420 2024-09-16T15:48:35.117 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f3c079420 msgr2=0x7f0f3c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f3c079420 0x7f0f3c07b8e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0f50033000 tx=0x7f0f50004730 comp rx=0 tx=0).stop 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 msgr2=0x7f0f60152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 0x7f0f60152e70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800c780 tx=0x7f0f5800cc40 comp rx=0 tx=0).stop 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 shutdown_connections 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f0f3c079420 0x7f0f3c07b8e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0f6015f5a0 0x7f0f60152e70 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f600ccf00 0x7f0f60150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 --2- 172.21.15.60:0/2383637771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0f600ad270 0x7f0f60143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.118+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 >> 172.21.15.60:0/2383637771 conn(0x7f0f600b1950 msgr2=0x7f0f6009c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.119+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 shutdown_connections 2024-09-16T15:48:35.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.119+0000 7f0f65200640 1 -- 172.21.15.60:0/2383637771 wait complete. 2024-09-16T15:48:35.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:35.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.296+0000 7fda64400640 1 -- 172.21.15.60:0/971949310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c15f5a0 msgr2=0x7fda5c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.296+0000 7fda64400640 1 --2- 172.21.15.60:0/971949310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c15f5a0 0x7fda5c0b2440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fda50009860 tx=0x7fda5002a510 comp rx=0 tx=0).stop 2024-09-16T15:48:35.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.296+0000 7fda64400640 1 -- 172.21.15.60:0/971949310 shutdown_connections 2024-09-16T15:48:35.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.296+0000 7fda64400640 1 --2- 172.21.15.60:0/971949310 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c15f5a0 0x7fda5c0b2440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.296+0000 7fda64400640 1 -- 172.21.15.60:0/971949310 >> 172.21.15.60:0/971949310 conn(0x7fda5c0b1950 msgr2=0x7fda5c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.296+0000 7fda64400640 1 -- 172.21.15.60:0/971949310 shutdown_connections 2024-09-16T15:48:35.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.296+0000 7fda64400640 1 -- 172.21.15.60:0/971949310 wait complete. 2024-09-16T15:48:35.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.297+0000 7fda64400640 1 Processor -- start 2024-09-16T15:48:35.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.297+0000 7fda64400640 1 -- start start 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.298+0000 7fda64400640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c073fb0 0x7fda5c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.298+0000 7fda64400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda5c0ccf00 0x7fda5c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.298+0000 7fda64400640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda5c15f5a0 0x7fda5c1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.298+0000 7fda64400640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c06ef30 con 0x7fda5c0ccf00 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.298+0000 7fda64400640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c06fa80 con 0x7fda5c15f5a0 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.298+0000 7fda64400640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fda5c074460 con 0x7fda5c073fb0 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda5c0ccf00 0x7fda5c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda5c0ccf00 0x7fda5c0cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53006/0 (socket says 172.21.15.60:53006) 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 -- 172.21.15.60:0/61374203 learned_addr learned my addr 172.21.15.60:0/61374203 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61a00640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c073fb0 0x7fda5c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 -- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c073fb0 msgr2=0x7fda5c1555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c073fb0 0x7fda5c1555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 -- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda5c15f5a0 msgr2=0x7fda5c1f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda62400640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda5c15f5a0 0x7fda5c1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda5c15f5a0 0x7fda5c1f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 -- 172.21.15.60:0/61374203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c0d0b40 con 0x7fda5c0ccf00 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61a00640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c073fb0 0x7fda5c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda62400640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda5c15f5a0 0x7fda5c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.299+0000 7fda61000640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda5c0ccf00 0x7fda5c0cc660 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fda4800c460 tx=0x7fda4800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.300+0000 7fda59600640 1 -- 172.21.15.60:0/61374203 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48016020 con 0x7fda5c0ccf00 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.300+0000 7fda59600640 1 -- 172.21.15.60:0/61374203 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda48012920 con 0x7fda5c0ccf00 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.300+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c0d0580 con 0x7fda5c0ccf00 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.300+0000 7fda59600640 1 -- 172.21.15.60:0/61374203 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48012d50 con 0x7fda5c0ccf00 2024-09-16T15:48:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.300+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c1e6cc0 con 0x7fda5c0ccf00 2024-09-16T15:48:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.301+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5c1e4730 con 0x7fda5c0ccf00 2024-09-16T15:48:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.302+0000 7fda59600640 1 -- 172.21.15.60:0/61374203 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7fda48019020 con 0x7fda5c0ccf00 2024-09-16T15:48:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.303+0000 7fda59600640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fda3c079420 0x7fda3c07b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.303+0000 7fda59600640 1 -- 172.21.15.60:0/61374203 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7fda48099290 con 0x7fda5c0ccf00 2024-09-16T15:48:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.304+0000 7fda61a00640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fda3c079420 0x7fda3c07b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.304+0000 7fda61a00640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fda3c079420 0x7fda3c07b8e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda50034000 tx=0x7fda50004730 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.308+0000 7fda59600640 1 -- 172.21.15.60:0/61374203 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fda48020030 con 0x7fda5c0ccf00 2024-09-16T15:48:35.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.499+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda5c155af0 con 0x7fda3c079420 2024-09-16T15:48:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.500+0000 7fda59600640 1 -- 172.21.15.60:0/61374203 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13367 (secure 0 0 0) 0x7fda5c155af0 con 0x7fda3c079420 2024-09-16T15:48:35.501 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:35.501 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.502+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fda3c079420 msgr2=0x7fda3c07b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.502+0000 7fda64400640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fda3c079420 0x7fda3c07b8e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda50034000 tx=0x7fda50004730 comp rx=0 tx=0).stop 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.502+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda5c0ccf00 msgr2=0x7fda5c0cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.502+0000 7fda64400640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda5c0ccf00 0x7fda5c0cc660 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fda4800c460 tx=0x7fda4800c920 comp rx=0 tx=0).stop 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.503+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 shutdown_connections 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.503+0000 7fda64400640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fda3c079420 0x7fda3c07b8e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.503+0000 7fda64400640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda5c15f5a0 0x7fda5c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.503+0000 7fda64400640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda5c0ccf00 0x7fda5c0cc660 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.503+0000 7fda64400640 1 --2- 172.21.15.60:0/61374203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda5c073fb0 0x7fda5c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.503+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 >> 172.21.15.60:0/61374203 conn(0x7fda5c0b1950 msgr2=0x7fda5c0afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.503+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 shutdown_connections 2024-09-16T15:48:35.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:35.503+0000 7fda64400640 1 -- 172.21.15.60:0/61374203 wait complete. 2024-09-16T15:48:35.519 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-16T15:48:34.230139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":284672,"kb_used_data":18504,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690048,"statfs":{"total":773094113280,"available":772802609152,"internally_reserved":0,"allocated":18948096,"data_stored":9013032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-4,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001560"},"pg_stats":[{"pgid":"1.0","version":"75'79","reported_seq":422,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-16T15:48:24.882649+0000","last_change":"2024-09-16T15:38:25.659390+0000","last_active":"2024-09-16T15:48:24.882649+0000","last_peered":"2024-09-16T15:48:24.882649+0000","last_clean":"2024-09-16T15:48:24.882649+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:48:24.882649+0000","last_undegraded":"2024-09-16T15:48:24.882649+0000","last_fullsized":"2024-09-16T15:48:24.882649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:38:24.505217+0000","last_clean_scrub_stamp":"2024-09-16T15:38:24.505217+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:41.487086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33936,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337904,"statfs":{"total":96636764160,"available":96602013696,"internally_reserved":0,"allocated":2813952,"data_stored":1569405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509566,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33356,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338484,"statfs":{"total":96636764160,"available":96602607616,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542270,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38100,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333740,"statfs":{"total":96636764160,"available":96597749760,"internally_reserved":0,"allocated":2813952,"data_stored":1569405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:35.522 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T15:38:24.505217+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=38, tm_sec=24, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=48, tm_sec=29, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:48:35.522 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T15:48:55.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:55.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.694+0000 7fb5d8a00640 1 -- 172.21.15.60:0/2900455593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d4073fb0 msgr2=0x7fb5d4074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.694+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/2900455593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d4073fb0 0x7fb5d4074420 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4075130 tx=0x7fb5cc02aa30 comp rx=0 tx=0).stop 2024-09-16T15:48:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.694+0000 7fb5d8a00640 1 -- 172.21.15.60:0/2900455593 shutdown_connections 2024-09-16T15:48:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.694+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/2900455593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d4073fb0 0x7fb5d4074420 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.694+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/2900455593 >> v2:172.21.15.100:3300/0 conn(0x7fb5d415f5a0 0x7fb5d40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.694+0000 7fb5d8a00640 1 -- 172.21.15.60:0/2900455593 >> 172.21.15.60:0/2900455593 conn(0x7fb5d40b1950 msgr2=0x7fb5d40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.694+0000 7fb5d8a00640 1 -- 172.21.15.60:0/2900455593 shutdown_connections 2024-09-16T15:48:55.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.694+0000 7fb5d8a00640 1 -- 172.21.15.60:0/2900455593 wait complete. 2024-09-16T15:48:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.696+0000 7fb5d8a00640 1 Processor -- start 2024-09-16T15:48:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.696+0000 7fb5d8a00640 1 -- start start 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5d4073fb0 0x7fb5d4145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40ccf00 0x7fb5d4158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d8a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d415f5a0 0x7fb5d4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d8a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d406f2e0 con 0x7fb5d415f5a0 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d8a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d406f160 con 0x7fb5d40ccf00 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d8a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d406f460 con 0x7fb5d4073fb0 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d415f5a0 0x7fb5d4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d2800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d415f5a0 0x7fb5d4159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48842/0 (socket says 172.21.15.60:48842) 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d2800640 1 -- 172.21.15.60:0/1180026414 learned_addr learned my addr 172.21.15.60:0/1180026414 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5d1e00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5d4073fb0 0x7fb5d4145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:55.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.697+0000 7fb5c9400640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40ccf00 0x7fb5d4158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:55.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5d2800640 1 -- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5d4073fb0 msgr2=0x7fb5d4145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:55.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5d2800640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5d4073fb0 0x7fb5d4145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5d2800640 1 -- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40ccf00 msgr2=0x7fb5d4158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:55.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5d2800640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40ccf00 0x7fb5d4158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5d2800640 1 -- 172.21.15.60:0/1180026414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d409bc60 con 0x7fb5d415f5a0 2024-09-16T15:48:55.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5c9400640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40ccf00 0x7fb5d4158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:55.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5d2800640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d415f5a0 0x7fb5d4159d90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc02b020 tx=0x7fb5cc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5caa00640 1 -- 172.21.15.60:0/1180026414 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc046070 con 0x7fb5d415f5a0 2024-09-16T15:48:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5caa00640 1 -- 172.21.15.60:0/1180026414 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5cc004250 con 0x7fb5d415f5a0 2024-09-16T15:48:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5caa00640 1 -- 172.21.15.60:0/1180026414 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc033050 con 0x7fb5d415f5a0 2024-09-16T15:48:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d409bf40 con 0x7fb5d415f5a0 2024-09-16T15:48:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.698+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d409c500 con 0x7fb5d415f5a0 2024-09-16T15:48:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.700+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d409c700 con 0x7fb5d415f5a0 2024-09-16T15:48:55.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.701+0000 7fb5caa00640 1 -- 172.21.15.60:0/1180026414 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7fb5cc049070 con 0x7fb5d415f5a0 2024-09-16T15:48:55.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.702+0000 7fb5caa00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5b0079420 0x7fb5b007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:55.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.702+0000 7fb5caa00640 1 -- 172.21.15.60:0/1180026414 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7fb5cc0ba440 con 0x7fb5d415f5a0 2024-09-16T15:48:55.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.702+0000 7fb5d1e00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5b0079420 0x7fb5b007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:55.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.703+0000 7fb5d1e00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5b0079420 0x7fb5b007b8e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4004630 tx=0x7fb5c400b2e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:55.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.707+0000 7fb5caa00640 1 -- 172.21.15.60:0/1180026414 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5d409c700 con 0x7fb5d415f5a0 2024-09-16T15:48:55.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.899+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 --> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5d40cc0f0 con 0x7fb5b0079420 2024-09-16T15:48:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.900+0000 7fb5caa00640 1 -- 172.21.15.60:0/1180026414 <== mgr.4129 v2:172.21.15.60:6836/4219958426 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13370 (secure 0 0 0) 0x7fb5d40cc0f0 con 0x7fb5b0079420 2024-09-16T15:48:55.901 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:55.902 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-16T15:48:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5b0079420 msgr2=0x7fb5b007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5b0079420 0x7fb5b007b8e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4004630 tx=0x7fb5c400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d415f5a0 msgr2=0x7fb5d4159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:55.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d415f5a0 0x7fb5d4159d90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc02b020 tx=0x7fb5cc003f10 comp rx=0 tx=0).stop 2024-09-16T15:48:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 shutdown_connections 2024-09-16T15:48:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7fb5b0079420 0x7fb5b007b8e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5d415f5a0 0x7fb5d4159d90 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5d40ccf00 0x7fb5d4158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 --2- 172.21.15.60:0/1180026414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5d4073fb0 0x7fb5d4145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 >> 172.21.15.60:0/1180026414 conn(0x7fb5d40b1950 msgr2=0x7fb5d40cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 shutdown_connections 2024-09-16T15:48:55.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:55.903+0000 7fb5d8a00640 1 -- 172.21.15.60:0/1180026414 wait complete. 2024-09-16T15:48:55.919 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-16T15:48:54.234674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":284660,"kb_used_data":18504,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754690060,"statfs":{"total":773094113280,"available":772802621440,"internally_reserved":0,"allocated":18948096,"data_stored":9013032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002901"},"pg_stats":[{"pgid":"1.0","version":"176'81","reported_seq":432,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-16T15:48:35.525948+0000","last_change":"2024-09-16T15:48:35.525948+0000","last_active":"2024-09-16T15:48:35.525948+0000","last_peered":"2024-09-16T15:48:35.525948+0000","last_clean":"2024-09-16T15:48:35.525948+0000","last_became_active":"2024-09-16T15:38:25.658077+0000","last_became_peered":"2024-09-16T15:38:25.658077+0000","last_unstale":"2024-09-16T15:48:35.525948+0000","last_undegraded":"2024-09-16T15:48:35.525948+0000","last_fullsized":"2024-09-16T15:48:35.525948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'81","last_scrub_stamp":"2024-09-16T15:48:35.525829+0000","last_deep_scrub":"176'81","last_deep_scrub_stamp":"2024-09-16T15:48:35.525829+0000","last_clean_scrub_stamp":"2024-09-16T15:48:35.525829+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:05.980915+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509570,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33932,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337908,"statfs":{"total":96636764160,"available":96602017792,"internally_reserved":0,"allocated":2813952,"data_stored":1569405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33356,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338484,"statfs":{"total":96636764160,"available":96602607616,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33360,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509569,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509568,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37520,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":2220032,"data_stored":979037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509570,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38092,"kb_used_data":2748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333748,"statfs":{"total":96636764160,"available":96597757952,"internally_reserved":0,"allocated":2813952,"data_stored":1569405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:55.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T15:48:56.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.075+0000 7f615ec00640 1 -- 172.21.15.60:0/190630590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f615806efa0 msgr2=0x7f61580cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.075+0000 7f615ec00640 1 --2- 172.21.15.60:0/190630590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f615806efa0 0x7f61580cc5c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f6144005fd0 tx=0x7f614402a540 comp rx=0 tx=0).stop 2024-09-16T15:48:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.075+0000 7f615ec00640 1 -- 172.21.15.60:0/190630590 shutdown_connections 2024-09-16T15:48:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.075+0000 7f615ec00640 1 --2- 172.21.15.60:0/190630590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f615806efa0 0x7f61580cc5c0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.075+0000 7f615ec00640 1 --2- 172.21.15.60:0/190630590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6158145750 0x7f6158073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.075+0000 7f615ec00640 1 -- 172.21.15.60:0/190630590 >> 172.21.15.60:0/190630590 conn(0x7f61580b1840 msgr2=0x7f61580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.075+0000 7f615ec00640 1 -- 172.21.15.60:0/190630590 shutdown_connections 2024-09-16T15:48:56.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.075+0000 7f615ec00640 1 -- 172.21.15.60:0/190630590 wait complete. 2024-09-16T15:48:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.077+0000 7f615ec00640 1 Processor -- start 2024-09-16T15:48:56.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.077+0000 7f615ec00640 1 -- start start 2024-09-16T15:48:56.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f615ec00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f615806efa0 0x7f615815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:56.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f615ec00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61580ccb00 0x7f61580cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f615ec00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6158145750 0x7f61580adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f615ec00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f61580b26b0 con 0x7f61580ccb00 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f615ec00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f61580cedd0 con 0x7f615806efa0 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f615ec00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f6158160100 con 0x7f6158145750 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f6157400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61580ccb00 0x7f61580cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f6157400640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61580ccb00 0x7f61580cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48870/0 (socket says 172.21.15.60:48870) 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f6157400640 1 -- 172.21.15.60:0/3444453112 learned_addr learned my addr 172.21.15.60:0/3444453112 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f615cc00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6158145750 0x7f61580adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.078+0000 7f6157e00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f615806efa0 0x7f615815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f6157400640 1 -- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6158145750 msgr2=0x7f61580adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f6157400640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6158145750 0x7f61580adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f6157400640 1 -- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f615806efa0 msgr2=0x7f615815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f6157400640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f615806efa0 0x7f615815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f6157400640 1 -- 172.21.15.60:0/3444453112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61581f8210 con 0x7f61580ccb00 2024-09-16T15:48:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f615cc00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6158145750 0x7f61580adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f6157e00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f615806efa0 0x7f615815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f6157400640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61580ccb00 0x7f61580cbc20 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6144009850 tx=0x7f61440039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f614fe00640 1 -- 172.21.15.60:0/3444453112 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614403f070 con 0x7f61580ccb00 2024-09-16T15:48:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f614fe00640 1 -- 172.21.15.60:0/3444453112 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6144004890 con 0x7f61580ccb00 2024-09-16T15:48:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f614fe00640 1 -- 172.21.15.60:0/3444453112 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144035030 con 0x7f61580ccb00 2024-09-16T15:48:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61581f84f0 con 0x7f61580ccb00 2024-09-16T15:48:56.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.079+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581f8ab0 con 0x7f61580ccb00 2024-09-16T15:48:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.081+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61581605b0 con 0x7f61580ccb00 2024-09-16T15:48:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.082+0000 7f614fe00640 1 -- 172.21.15.60:0/3444453112 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f6144042070 con 0x7f61580ccb00 2024-09-16T15:48:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.083+0000 7f614fe00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6138079450 0x7f613807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.083+0000 7f614fe00640 1 -- 172.21.15.60:0/3444453112 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f61440ba4c0 con 0x7f61580ccb00 2024-09-16T15:48:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.083+0000 7f6157e00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6138079450 0x7f613807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:56.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.084+0000 7f6157e00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6138079450 0x7f613807b910 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6148009890 tx=0x7f6148008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:56.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.087+0000 7f614fe00640 1 -- 172.21.15.60:0/3444453112 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f61440c0030 con 0x7f61580ccb00 2024-09-16T15:48:56.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.281+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f61580a9700 con 0x7f61580ccb00 2024-09-16T15:48:56.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.285+0000 7f614fe00640 1 -- 172.21.15.60:0/3444453112 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v50)=0 v50) ==== 123+0+0 (secure 0 0 0) 0x7f6144084320 con 0x7f61580ccb00 2024-09-16T15:48:56.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.286+0000 7f614fe00640 1 -- 172.21.15.60:0/3444453112 <== mon.0 v2:172.21.15.60:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f6144083730 con 0x7f61580ccb00 2024-09-16T15:48:56.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6138079450 msgr2=0x7f613807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6138079450 0x7f613807b910 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6148009890 tx=0x7f6148008040 comp rx=0 tx=0).stop 2024-09-16T15:48:56.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61580ccb00 msgr2=0x7f61580cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61580ccb00 0x7f61580cbc20 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6144009850 tx=0x7f61440039e0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 shutdown_connections 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f6138079450 0x7f613807b910 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6158145750 0x7f61580adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61580ccb00 0x7f61580cbc20 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 --2- 172.21.15.60:0/3444453112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f615806efa0 0x7f615815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 >> 172.21.15.60:0/3444453112 conn(0x7f61580b1840 msgr2=0x7f61580b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 shutdown_connections 2024-09-16T15:48:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.288+0000 7f615ec00640 1 -- 172.21.15.60:0/3444453112 wait complete. 2024-09-16T15:48:56.307 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:48:56.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.467+0000 7f5186a00640 1 -- 172.21.15.60:0/1151057005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5180161430 msgr2=0x7f51800b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.467+0000 7f5186a00640 1 --2- 172.21.15.60:0/1151057005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5180161430 0x7f51800b5540 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f51800d03a0 tx=0x7f517402a8f0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.468+0000 7f5186a00640 1 -- 172.21.15.60:0/1151057005 shutdown_connections 2024-09-16T15:48:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.468+0000 7f5186a00640 1 --2- 172.21.15.60:0/1151057005 >> v2:172.21.15.100:3300/0 conn(0x7f51800cff50 0x7f51800b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.468+0000 7f5186a00640 1 --2- 172.21.15.60:0/1151057005 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5180161430 0x7f51800b5540 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.468+0000 7f5186a00640 1 -- 172.21.15.60:0/1151057005 >> 172.21.15.60:0/1151057005 conn(0x7f51800d3dc0 msgr2=0x7f51800d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.468+0000 7f5186a00640 1 -- 172.21.15.60:0/1151057005 shutdown_connections 2024-09-16T15:48:56.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.468+0000 7f5186a00640 1 -- 172.21.15.60:0/1151057005 wait complete. 2024-09-16T15:48:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.469+0000 7f5186a00640 1 Processor -- start 2024-09-16T15:48:56.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.469+0000 7f5186a00640 1 -- start start 2024-09-16T15:48:56.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.470+0000 7f5186a00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51800b4aa0 0x7f51801f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.470+0000 7f5186a00640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51800cff50 0x7f51801f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.470+0000 7f5186a00640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180161430 0x7f51801f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.470+0000 7f5186a00640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 -- 0x7f51800b3df0 con 0x7f51800b4aa0 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.470+0000 7f5186a00640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51800b3c70 con 0x7f5180161430 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.470+0000 7f5186a00640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 -- 0x7f51800b3f70 con 0x7f51800cff50 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.470+0000 7f5185600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51800b4aa0 0x7f51801f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5185600640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51800b4aa0 0x7f51801f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:48890/0 (socket says 172.21.15.60:48890) 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5185600640 1 -- 172.21.15.60:0/1179828520 learned_addr learned my addr 172.21.15.60:0/1179828520 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5186000640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180161430 0x7f51801f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5184c00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51800cff50 0x7f51801f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5185600640 1 -- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51800cff50 msgr2=0x7f51801f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5185600640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51800cff50 0x7f51801f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5185600640 1 -- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180161430 msgr2=0x7f51801f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5185600640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180161430 0x7f51801f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5185600640 1 -- 172.21.15.60:0/1179828520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51800cd530 con 0x7f51800b4aa0 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5186000640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180161430 0x7f51801f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5184c00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51800cff50 0x7f51801f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f5185600640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51800b4aa0 0x7f51801f8490 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5174004af0 tx=0x7f5174003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.471+0000 7f517d600640 1 -- 172.21.15.60:0/1179828520 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174004240 con 0x7f51800b4aa0 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.472+0000 7f517d600640 1 -- 172.21.15.60:0/1179828520 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f517402d070 con 0x7f51800b4aa0 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.472+0000 7f517d600640 1 -- 172.21.15.60:0/1179828520 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174004680 con 0x7f51800b4aa0 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.472+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51800cd7b0 con 0x7f51800b4aa0 2024-09-16T15:48:56.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.472+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51800cdd30 con 0x7f51800b4aa0 2024-09-16T15:48:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.473+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51800b3030 con 0x7f51800b4aa0 2024-09-16T15:48:56.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.474+0000 7f517d600640 1 -- 172.21.15.60:0/1179828520 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 27) ==== 102358+0+0 (secure 0 0 0) 0x7f5174031480 con 0x7f51800b4aa0 2024-09-16T15:48:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.475+0000 7f517d600640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5160079420 0x7f516007b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.475+0000 7f517d600640 1 -- 172.21.15.60:0/1179828520 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 4971+0+0 (secure 0 0 0) 0x7f51740bb0d0 con 0x7f51800b4aa0 2024-09-16T15:48:56.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.475+0000 7f5184c00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5160079420 0x7f516007b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:56.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.476+0000 7f5184c00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5160079420 0x7f516007b8e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f518015b520 tx=0x7f516c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:56.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.480+0000 7f517d600640 1 -- 172.21.15.60:0/1179828520 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51800b3030 con 0x7f51800b4aa0 2024-09-16T15:48:56.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.711+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51800b4570 con 0x7f51800b4aa0 2024-09-16T15:48:56.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.712+0000 7f517d600640 1 -- 172.21.15.60:0/1179828520 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+3913 (secure 0 0 0) 0x7f51800cd530 con 0x7f51800b4aa0 2024-09-16T15:48:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-16T15:48:56.713 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":125,"btime":"2024-09-16T15:48:22:815587+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7598,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6801/657477165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":657477165},{"type":"v1","addr":"172.21.15.100:6801","nonce":657477165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":7607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/293588121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":293588121},{"type":"v1","addr":"172.21.15.100:6835","nonce":293588121}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":8596,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/903832682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":903832682},{"type":"v1","addr":"172.21.15.60:6834","nonce":903832682}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":9498,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1176419520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1176419520},{"type":"v1","addr":"172.21.15.60:6835","nonce":1176419520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":125}],"filesystems":[]} 2024-09-16T15:48:56.714 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 125 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5160079420 msgr2=0x7f516007b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5160079420 0x7f516007b8e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f518015b520 tx=0x7f516c009040 comp rx=0 tx=0).stop 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51800b4aa0 msgr2=0x7f51801f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51800b4aa0 0x7f51801f8490 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5174004af0 tx=0x7f5174003c20 comp rx=0 tx=0).stop 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 shutdown_connections 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:6836/4219958426,v1:172.21.15.60:6837/4219958426] conn(0x7f5160079420 0x7f516007b8e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5180161430 0x7f51801f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51800cff50 0x7f51801f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 --2- 172.21.15.60:0/1179828520 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51800b4aa0 0x7f51801f8490 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.714+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 >> 172.21.15.60:0/1179828520 conn(0x7f51800d3dc0 msgr2=0x7f5180149b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.715+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 shutdown_connections 2024-09-16T15:48:56.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-16T15:48:56.715+0000 7f5186a00640 1 -- 172.21.15.60:0/1179828520 wait complete. 2024-09-16T15:48:56.731 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-16T15:48:56.731 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T15:48:56.731 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:48:56.731 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:56.745 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:48:56.745 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:48:56.745 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:56.801 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:48:56.801 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:48:56.801 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:56.818 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:48:56.818 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:48:56.818 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:56.874 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:48:56.874 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T15:48:56.875 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T15:48:56.875 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:56.916 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T15:48:56.917 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T15:48:56.917 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:56.978 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T15:48:56.978 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T15:48:56.978 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.041 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T15:48:57.041 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T15:48:57.041 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.093 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T15:48:57.094 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T15:48:57.094 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.142 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T15:48:57.142 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T15:48:57.142 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.192 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T15:48:57.192 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T15:48:57.192 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.256 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T15:48:57.256 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T15:48:57.256 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.313 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T15:48:57.314 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T15:48:57.314 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T15:48:57.314 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.388 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T15:48:57.388 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T15:48:57.389 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.453 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T15:48:57.453 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T15:48:57.453 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T15:48:57.453 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.475 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T15:48:57.475 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T15:48:57.475 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.494 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T15:48:57.494 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T15:48:57.494 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:57.554 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T15:48:57.555 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T15:48:57.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-16T15:48:57.644 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:48:57.644 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T15:48:57.875 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:48:57.875 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T15:48:57.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:48:57.981 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T15:48:58.122 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:48:58.122 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T15:48:58.225 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:48:58.225 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T15:48:58.486 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:48:58.487 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T15:48:58.589 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:48:58.589 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T15:48:58.687 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:48:58.687 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T15:48:58.790 INFO:tasks.ceph:Archiving mon data... 2024-09-16T15:48:58.791 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/data/mon.a.tgz 2024-09-16T15:48:58.792 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-09-16T15:48:58.829 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.AXFv4qQLPV 2024-09-16T15:48:58.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.AXFv4qQLPV 2024-09-16T15:48:59.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.AXFv4qQLPV 2024-09-16T15:48:59.245 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.AXFv4qQLPV is 1MB 2024-09-16T15:48:59.345 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.AXFv4qQLPV 2024-09-16T15:48:59.380 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/data/mon.b.tgz 2024-09-16T15:48:59.380 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-16T15:48:59.415 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.z4IeXB7arI 2024-09-16T15:48:59.415 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.z4IeXB7arI 2024-09-16T15:48:59.751 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.z4IeXB7arI 2024-09-16T15:48:59.845 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.z4IeXB7arI is 1MB 2024-09-16T15:48:59.947 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.z4IeXB7arI 2024-09-16T15:48:59.984 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/data/mon.c.tgz 2024-09-16T15:48:59.984 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-16T15:49:00.059 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.3JbdbKkRpk 2024-09-16T15:49:00.059 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.3JbdbKkRpk 2024-09-16T15:49:00.372 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.3JbdbKkRpk 2024-09-16T15:49:00.466 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.3JbdbKkRpk is 1MB 2024-09-16T15:49:00.564 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.3JbdbKkRpk 2024-09-16T15:49:00.600 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T15:49:00.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T15:49:00.604 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T15:49:00.732 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T15:49:00.767 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T15:49:00.767 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T15:49:00.768 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/crash to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/remote/smithi060/crash 2024-09-16T15:49:00.769 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T15:49:00.828 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/remote/smithi100/crash 2024-09-16T15:49:00.829 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T15:49:00.883 INFO:tasks.ceph:Shutting down logrotate 2024-09-16T15:49:00.883 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:49:00.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:49:00.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T15:49:01.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T15:49:01.083 INFO:tasks.ceph:Compressing logs... 2024-09-16T15:49:01.084 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T15:49:01.088 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T15:49:01.138 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T15:49:01.139 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49829.log 2024-09-16T15:49:01.140 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68086.log 2024-09-16T15:49:01.141 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42385.log 2024-09-16T15:49:01.141 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49829.log.gz 2024-09-16T15:49:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36266.log 2024-09-16T15:49:01.142 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80140.log 2024-09-16T15:49:01.142 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68086.log.gz 2024-09-16T15:49:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73127.log 2024-09-16T15:49:01.144 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42385.log.gz 2024-09-16T15:49:01.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52949.log 2024-09-16T15:49:01.144 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80140.log.gz 2024-09-16T15:49:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36146.log 2024-09-16T15:49:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36266.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36266.log.gz 2024-09-16T15:49:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43937.log 2024-09-16T15:49:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36146.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36146.log.gz 2024-09-16T15:49:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43403.log 2024-09-16T15:49:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.43937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43937.log.gz 2024-09-16T15:49:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T15:49:01.149 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.73127.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.73127.log.gz 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55221.log 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52949.log: gzip 87.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.52949.log.gz -- 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.66315.log 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49350.log 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55221.log.gz 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54023.log 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66315.log.gz 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.49350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49350.log.gz 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64236.log 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54023.log.gz 2024-09-16T15:49:01.150 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52196.log 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.43403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43403.log.gz 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47017.log 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41704.log 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.47017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47017.log.gz 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44501.log 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.3.41704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41704.log.gz 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.44501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44501.log.gz 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42829.log 2024-09-16T15:49:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47515.log 2024-09-16T15:49:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.42829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42829.log.gz 2024-09-16T15:49:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T15:49:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.47515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47515.log.gz 2024-09-16T15:49:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T15:49:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36306.log 2024-09-16T15:49:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36026.log 2024-09-16T15:49:01.155 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45535.log 2024-09-16T15:49:01.155 INFO:teuthology.orchestra.run.smithi100.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36306.log.gz 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64236.log.gz 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55698.log 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52196.log.gz 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45842.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65695.log 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61683.log 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55698.log.gz 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66167.log 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65695.log.gz 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44793.log 2024-09-16T15:49:01.157 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61683.log.gz 2024-09-16T15:49:01.158 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66167.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66167.log.gz 2024-09-16T15:49:01.158 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75563.log 2024-09-16T15:49:01.158 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55892.log 2024-09-16T15:49:01.158 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44793.log.gz 2024-09-16T15:49:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36026.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T15:49:01.161 INFO:teuthology.orchestra.run.smithi100.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36026.log.gz 2024-09-16T15:49:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.45535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45535.log.gz 2024-09-16T15:49:01.161 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44994.log 2024-09-16T15:49:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T15:49:01.162 INFO:teuthology.orchestra.run.smithi100.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T15:49:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.44994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42299.log 2024-09-16T15:49:01.162 INFO:teuthology.orchestra.run.smithi100.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44994.log.gz 2024-09-16T15:49:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T15:49:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.42299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42299.log.gz 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65264.log 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75563.log.gz 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54881.log 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55892.log.gz 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65264.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65264.log.gz 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47898.log 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53527.log 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54881.log.gz 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78362.log 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47898.log.gz 2024-09-16T15:49:01.163 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66984.log 2024-09-16T15:49:01.164 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53527.log.gz 2024-09-16T15:49:01.164 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78362.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78362.log.gz 2024-09-16T15:49:01.164 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64970.log 2024-09-16T15:49:01.166 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T15:49:01.166 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36106.log 2024-09-16T15:49:01.166 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.36106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41289.log 2024-09-16T15:49:01.167 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51756.log 2024-09-16T15:49:01.167 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66984.log.gz 2024-09-16T15:49:01.167 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80452.log 2024-09-16T15:49:01.167 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64970.log.gz 2024-09-16T15:49:01.167 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70362.log 2024-09-16T15:49:01.167 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51756.log.gz 2024-09-16T15:49:01.167 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67711.log 2024-09-16T15:49:01.167 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80452.log.gz 2024-09-16T15:49:01.168 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70362.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.70362.log.gz 2024-09-16T15:49:01.168 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47356.log 2024-09-16T15:49:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.41289.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36106.log.gz 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi100.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi100.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41289.log.gz 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69948.log 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67711.log.gz 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59567.log 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47356.log.gz 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70608.log 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69948.log.gz 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59567.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59567.log.gz 2024-09-16T15:49:01.171 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61792.log 2024-09-16T15:49:01.173 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49096.log 2024-09-16T15:49:01.173 INFO:teuthology.orchestra.run.smithi060.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.70608.log.gz 2024-09-16T15:49:01.173 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78629.log 2024-09-16T15:49:01.173 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61792.log.gz 2024-09-16T15:49:01.173 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74005.log 2024-09-16T15:49:01.174 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78629.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49096.log.gz 2024-09-16T15:49:01.174 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78629.log.gz 2024-09-16T15:49:01.174 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52154.log 2024-09-16T15:49:01.174 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44101.log 2024-09-16T15:49:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46027.log 2024-09-16T15:49:01.176 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36186.log 2024-09-16T15:49:01.176 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74005.log.gz 2024-09-16T15:49:01.176 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54746.log 2024-09-16T15:49:01.176 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52154.log.gz 2024-09-16T15:49:01.176 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80410.log 2024-09-16T15:49:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T15:49:01.178 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36186.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36186.log.gz 2024-09-16T15:49:01.178 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44101.log.gz 2024-09-16T15:49:01.178 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54746.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54746.log.gz 2024-09-16T15:49:01.178 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55638.log 2024-09-16T15:49:01.178 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-09-16T15:49:01.178 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80410.log.gz 2024-09-16T15:49:01.179 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77469.log 2024-09-16T15:49:01.179 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55638.log.gz 2024-09-16T15:49:01.180 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49871.log 2024-09-16T15:49:01.180 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-09-16T15:49:01.181 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.46027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46027.log.gz 2024-09-16T15:49:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77469.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77469.log.gz 2024-09-16T15:49:01.181 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44422.log 2024-09-16T15:49:01.182 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-09-16T15:49:01.182 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49871.log.gz 2024-09-16T15:49:01.182 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70239.log 2024-09-16T15:49:01.183 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44422.log.gz 2024-09-16T15:49:01.183 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40773.log 2024-09-16T15:49:01.183 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-09-16T15:49:01.184 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36066.log 2024-09-16T15:49:01.184 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70239.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.70239.log.gz 2024-09-16T15:49:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46052.log 2024-09-16T15:49:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42586.log 2024-09-16T15:49:01.185 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40773.log.gz 2024-09-16T15:49:01.186 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50243.log 2024-09-16T15:49:01.186 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46052.log.gz 2024-09-16T15:49:01.187 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36226.log 2024-09-16T15:49:01.187 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74402.log 2024-09-16T15:49:01.188 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42586.log.gz 2024-09-16T15:49:01.188 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50243.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50243.log.gz -5 2024-09-16T15:49:01.188 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41116.log 2024-09-16T15:49:01.188 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36066.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36066.log.gz 2024-09-16T15:49:01.189 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55475.log 2024-09-16T15:49:01.189 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74402.log.gz 2024-09-16T15:49:01.190 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73290.log 2024-09-16T15:49:01.190 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41116.log.gz 2024-09-16T15:49:01.190 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.55475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55475.log.gz 2024-09-16T15:49:01.191 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57969.log 2024-09-16T15:49:01.192 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46506.log 2024-09-16T15:49:01.192 INFO:teuthology.orchestra.run.smithi100.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36226.log.gz 2024-09-16T15:49:01.192 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80056.log 2024-09-16T15:49:01.193 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73290.log.gz 2024-09-16T15:49:01.193 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75125.log 2024-09-16T15:49:01.193 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57969.log.gz 2024-09-16T15:49:01.194 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54702.log 2024-09-16T15:49:01.194 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80056.log.gz 2024-09-16T15:49:01.194 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75125.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.75125.log.gz 2024-09-16T15:49:01.194 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47398.log 2024-09-16T15:49:01.195 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43365.log 2024-09-16T15:49:01.195 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54702.log.gz 2024-09-16T15:49:01.196 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2024-09-16T15:49:01.196 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47398.log.gz 2024-09-16T15:49:01.197 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.46506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46506.log.gz 2024-09-16T15:49:01.197 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.43365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43365.log.gz 2024-09-16T15:49:01.197 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46136.log 2024-09-16T15:49:01.198 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64571.log 2024-09-16T15:49:01.198 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2024-09-16T15:49:01.198 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70866.log 2024-09-16T15:49:01.198 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46136.log.gz 2024-09-16T15:49:01.199 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45045.log 2024-09-16T15:49:01.200 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64571.log.gz 2024-09-16T15:49:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70866.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.70866.log.gz 2024-09-16T15:49:01.200 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68567.log 2024-09-16T15:49:01.201 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59704.log 2024-09-16T15:49:01.201 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45045.log.gz 2024-09-16T15:49:01.202 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70743.log 2024-09-16T15:49:01.202 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68567.log.gz 2024-09-16T15:49:01.203 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65390.log 2024-09-16T15:49:01.203 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59704.log.gz 2024-09-16T15:49:01.204 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57364.log 2024-09-16T15:49:01.204 INFO:teuthology.orchestra.run.smithi060.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.70743.log.gz 2024-09-16T15:49:01.205 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54797.log 86.9% 2024-09-16T15:49:01.205 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65390.log.gz 2024-09-16T15:49:01.205 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56855.log 2024-09-16T15:49:01.205 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57364.log.gz 2024-09-16T15:49:01.206 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60435.log 2024-09-16T15:49:01.206 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54797.log.gz 2024-09-16T15:49:01.207 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64679.log 2024-09-16T15:49:01.207 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56855.log.gz 2024-09-16T15:49:01.208 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60435.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60435.log.gz 2024-09-16T15:49:01.208 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45501.log 2024-09-16T15:49:01.209 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54927.log 87.1% 2024-09-16T15:49:01.209 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64679.log.gz 2024-09-16T15:49:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45501.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45501.log.gz 2024-09-16T15:49:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65474.log 2024-09-16T15:49:01.211 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56058.log 2024-09-16T15:49:01.211 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54927.log.gz 2024-09-16T15:49:01.212 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52028.log 2024-09-16T15:49:01.212 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65474.log.gz 2024-09-16T15:49:01.213 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50385.log 2024-09-16T15:49:01.213 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56058.log.gz 2024-09-16T15:49:01.213 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50890.log 2024-09-16T15:49:01.214 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52028.log.gz 2024-09-16T15:49:01.214 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47524.log 2024-09-16T15:49:01.214 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50385.log.gz 2024-09-16T15:49:01.215 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74486.log 2024-09-16T15:49:01.216 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50890.log.gz 2024-09-16T15:49:01.216 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47524.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.47524.log.gz --verbose 2024-09-16T15:49:01.216 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.48634.log 2024-09-16T15:49:01.217 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52280.log 2024-09-16T15:49:01.217 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74486.log.gz 2024-09-16T15:49:01.218 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62453.log 2024-09-16T15:49:01.218 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48634.log.gz 2024-09-16T15:49:01.219 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52280.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T15:49:01.219 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52280.log.gz 2024-09-16T15:49:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64322.log 2024-09-16T15:49:01.220 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62453.log.gz 2024-09-16T15:49:01.221 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46920.log 2024-09-16T15:49:01.222 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52406.log 2024-09-16T15:49:01.222 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64322.log.gz 2024-09-16T15:49:01.222 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76912.log 2024-09-16T15:49:01.223 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2024-09-16T15:49:01.224 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41242.log 2024-09-16T15:49:01.224 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52406.log.gz 2024-09-16T15:49:01.225 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52112.log 2024-09-16T15:49:01.225 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76912.log.gz 2024-09-16T15:49:01.226 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41242.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.41242.log.gz 2024-09-16T15:49:01.226 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45884.log 2024-09-16T15:49:01.227 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52112.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52112.log.gzgzip 2024-09-16T15:49:01.227 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69489.log 2024-09-16T15:49:01.228 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48592.log 2024-09-16T15:49:01.228 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45884.log.gz 2024-09-16T15:49:01.229 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57023.log 2024-09-16T15:49:01.229 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69489.log.gz 2024-09-16T15:49:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48592.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48592.log.gz 2024-09-16T15:49:01.230 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45926.log 2024-09-16T15:49:01.231 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69275.log 2024-09-16T15:49:01.231 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57023.log.gz 2024-09-16T15:49:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45926.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45926.log.gz 2024-09-16T15:49:01.232 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78128.log 2024-09-16T15:49:01.233 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71124.log 2024-09-16T15:49:01.233 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69275.log.gz 2024-09-16T15:49:01.234 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78128.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78128.log.gz 2024-09-16T15:49:01.234 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71843.log 2024-09-16T15:49:01.235 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68044.log 2024-09-16T15:49:01.235 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71124.log.gz 2024-09-16T15:49:01.236 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67753.log 87.4% 2024-09-16T15:49:01.237 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71843.log.gz 2024-09-16T15:49:01.237 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61209.log 2024-09-16T15:49:01.238 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68044.log.gz 2024-09-16T15:49:01.238 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67753.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67753.log.gz 2024-09-16T15:49:01.239 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71801.log 2024-09-16T15:49:01.240 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72533.log 2024-09-16T15:49:01.240 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61209.log.gz 2024-09-16T15:49:01.240 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71801.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71801.log.gz 2024-09-16T15:49:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79655.log 2024-09-16T15:49:01.242 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56447.log 2024-09-16T15:49:01.242 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72533.log.gz 2024-09-16T15:49:01.243 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79655.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79655.log.gz 2024-09-16T15:49:01.243 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74215.log 2024-09-16T15:49:01.244 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75980.log 2024-09-16T15:49:01.244 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56447.log.gz 2024-09-16T15:49:01.245 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74215.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74215.log.gz 2024-09-16T15:49:01.245 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54532.log 2024-09-16T15:49:01.246 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56100.log 2024-09-16T15:49:01.247 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75980.log.gz 2024-09-16T15:49:01.247 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54532.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54532.log.gz 2024-09-16T15:49:01.247 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46763.log 2024-09-16T15:49:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60519.log 2024-09-16T15:49:01.248 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56100.log.gz 2024-09-16T15:49:01.249 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46763.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46763.log.gz 2024-09-16T15:49:01.249 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57494.log 2024-09-16T15:49:01.250 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79510.log 2024-09-16T15:49:01.251 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57494.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60519.log.gz 2024-09-16T15:49:01.252 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77715.log 87.1% 2024-09-16T15:49:01.252 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57494.log.gz 2024-09-16T15:49:01.252 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75167.log 2024-09-16T15:49:01.253 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79510.log.gz 2024-09-16T15:49:01.253 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79300.log 2024-09-16T15:49:01.254 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77715.log.gz 2024-09-16T15:49:01.254 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75503.log 2024-09-16T15:49:01.255 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75167.log.gz 2024-09-16T15:49:01.255 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62014.log 2024-09-16T15:49:01.256 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79300.log.gz 2024-09-16T15:49:01.257 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75503.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75503.log.gz 2024-09-16T15:49:01.257 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47626.log 2024-09-16T15:49:01.258 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40725.log 2024-09-16T15:49:01.258 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62014.log.gz 2024-09-16T15:49:01.259 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40484.log 2024-09-16T15:49:01.259 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47626.log.gz 2024-09-16T15:49:01.259 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79468.log 2024-09-16T15:49:01.260 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40725.log.gz 2024-09-16T15:49:01.261 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40484.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40484.log.gz 2024-09-16T15:49:01.261 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41536.log 2024-09-16T15:49:01.262 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76996.log 2024-09-16T15:49:01.262 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79468.log.gz 2024-09-16T15:49:01.263 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41536.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41536.log.gz 2024-09-16T15:49:01.263 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45543.log 2024-09-16T15:49:01.264 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70403.log 2024-09-16T15:49:01.264 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76996.log.gz 2024-09-16T15:49:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45543.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45543.log.gz 2024-09-16T15:49:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73714.log 2024-09-16T15:49:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80350.log 2024-09-16T15:49:01.266 INFO:teuthology.orchestra.run.smithi060.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.70403.log.gz 2024-09-16T15:49:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73714.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73714.log.gz 2024-09-16T15:49:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53981.log 2024-09-16T15:49:01.268 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60080.log 2024-09-16T15:49:01.268 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.80350.log.gz 2024-09-16T15:49:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53981.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53981.log.gz 2024-09-16T15:49:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78254.log 2024-09-16T15:49:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42838.log 2024-09-16T15:49:01.270 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60080.log.gz 2024-09-16T15:49:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78254.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78254.log.gz 2024-09-16T15:49:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41368.log 2024-09-16T15:49:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2024-09-16T15:49:01.272 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42838.log.gz 2024-09-16T15:49:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41368.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.41368.log.gz 2024-09-16T15:49:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42670.log 2024-09-16T15:49:01.274 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77673.log 2024-09-16T15:49:01.274 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2024-09-16T15:49:01.275 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42670.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42670.log.gz 2024-09-16T15:49:01.275 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61377.log 2024-09-16T15:49:01.276 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64870.log 2024-09-16T15:49:01.277 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77673.log.gz 2024-09-16T15:49:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61377.log: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.61377.log.gz 2024-09-16T15:49:01.277 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65306.log 2024-09-16T15:49:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72310.log 2024-09-16T15:49:01.278 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64870.log.gz 2024-09-16T15:49:01.279 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65306.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.65306.log.gz 2024-09-16T15:49:01.280 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68399.log 2024-09-16T15:49:01.280 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46600.log 2024-09-16T15:49:01.281 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72310.log.gz 2024-09-16T15:49:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68399.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68399.log.gz 2024-09-16T15:49:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56405.log 2024-09-16T15:49:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.46600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46600.log.gz 2024-09-16T15:49:01.283 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51371.log 2024-09-16T15:49:01.283 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58446.log 2024-09-16T15:49:01.283 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56405.log.gz 2024-09-16T15:49:01.284 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59994.log 2024-09-16T15:49:01.285 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51371.log.gz 2024-09-16T15:49:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58446.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58446.log.gz 2024-09-16T15:49:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73412.log 2024-09-16T15:49:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48323.log 2024-09-16T15:49:01.287 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59994.log.gz 2024-09-16T15:49:01.288 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73412.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73412.log.gz 2024-09-16T15:49:01.288 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44835.log 2024-09-16T15:49:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75293.log 2024-09-16T15:49:01.289 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48323.log.gz 2024-09-16T15:49:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44835.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44835.log.gz 2024-09-16T15:49:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58404.log 2024-09-16T15:49:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68483.log 2024-09-16T15:49:01.291 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75293.log.gz 2024-09-16T15:49:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58404.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58404.log.gz 2024-09-16T15:49:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59044.log 2024-09-16T15:49:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77777.log 2024-09-16T15:49:01.293 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68483.log.gz 2024-09-16T15:49:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59044.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59044.log.gz 2024-09-16T15:49:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74047.log 2024-09-16T15:49:01.295 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69096.log 2024-09-16T15:49:01.295 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77777.log.gz 2024-09-16T15:49:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74047.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74047.log.gz 2024-09-16T15:49:01.296 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45459.log 2024-09-16T15:49:01.297 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52448.log 2024-09-16T15:49:01.298 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69096.log.gz 2024-09-16T15:49:01.298 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45459.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45459.log.gz 2024-09-16T15:49:01.298 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50057.log 2024-09-16T15:49:01.299 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47566.log 2024-09-16T15:49:01.299 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52448.log.gz 2024-09-16T15:49:01.300 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50057.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50057.log.gz 2024-09-16T15:49:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75461.log 2024-09-16T15:49:01.301 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49513.log 2024-09-16T15:49:01.302 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47566.log.gz 2024-09-16T15:49:01.303 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75461.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75461.log.gz 2024-09-16T15:49:01.303 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47940.log 2024-09-16T15:49:01.304 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42050.log 2024-09-16T15:49:01.304 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49513.log.gz 2024-09-16T15:49:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47940.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47940.log.gz 2024-09-16T15:49:01.305 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62766.log 2024-09-16T15:49:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72621.log 2024-09-16T15:49:01.306 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42050.log.gz 2024-09-16T15:49:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62766.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62766.log.gz 2024-09-16T15:49:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75037.log 2024-09-16T15:49:01.308 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49555.log 2024-09-16T15:49:01.308 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72621.log.gz 2024-09-16T15:49:01.309 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75037.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75037.log.gz 2024-09-16T15:49:01.309 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56939.log 2024-09-16T15:49:01.310 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74654.log 2024-09-16T15:49:01.310 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49555.log.gz 2024-09-16T15:49:01.311 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56939.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56939.log.gz 2024-09-16T15:49:01.311 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46010.log 2024-09-16T15:49:01.312 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76702.log 2024-09-16T15:49:01.312 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74654.log.gz 2024-09-16T15:49:01.313 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46010.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46010.log.gz 2024-09-16T15:49:01.313 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69615.log 2024-09-16T15:49:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65944.log 2024-09-16T15:49:01.314 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76702.log.gz 2024-09-16T15:49:01.315 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52490.log 2024-09-16T15:49:01.316 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69615.log.gz 2024-09-16T15:49:01.316 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77821.log 2024-09-16T15:49:01.316 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65944.log.gz 2024-09-16T15:49:01.317 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77549.log 2024-09-16T15:49:01.317 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52490.log.gz 2024-09-16T15:49:01.318 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45968.log 2024-09-16T15:49:01.318 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77821.log.gz 2024-09-16T15:49:01.319 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77549.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77549.log.gz 2024-09-16T15:49:01.320 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56584.log 2024-09-16T15:49:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60038.log 2024-09-16T15:49:01.321 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45968.log.gz 2024-09-16T15:49:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56584.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56584.log.gz 2024-09-16T15:49:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49995.log 2024-09-16T15:49:01.322 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79164.log 2024-09-16T15:49:01.323 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60038.log.gz 2024-09-16T15:49:01.323 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49995.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.49995.log.gz 2024-09-16T15:49:01.324 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54969.log 2024-09-16T15:49:01.324 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70907.log 2024-09-16T15:49:01.325 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79164.log.gz 2024-09-16T15:49:01.326 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54969.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54969.log.gz 2024-09-16T15:49:01.326 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48928.log 2024-09-16T15:49:01.327 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36385.log 2024-09-16T15:49:01.327 INFO:teuthology.orchestra.run.smithi060.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.70907.log.gz 2024-09-16T15:49:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48928.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48928.log.gz 2024-09-16T15:49:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77374.log 2024-09-16T15:49:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61888.log 2024-09-16T15:49:01.329 INFO:teuthology.orchestra.run.smithi060.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36385.log.gz 2024-09-16T15:49:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77374.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77374.log.gz 2024-09-16T15:49:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43823.log 2024-09-16T15:49:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41716.log 2024-09-16T15:49:01.331 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.61888.log.gz 2024-09-16T15:49:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.2.43823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43823.log.gz 2024-09-16T15:49:01.332 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74131.log 2024-09-16T15:49:01.333 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71530.log 2024-09-16T15:49:01.333 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41716.log.gz 2024-09-16T15:49:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74131.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74131.log.gz 2024-09-16T15:49:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54149.log 2024-09-16T15:49:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78296.log 2024-09-16T15:49:01.335 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71530.log.gz 2024-09-16T15:49:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40948.log 2024-09-16T15:49:01.336 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54149.log.gz 2024-09-16T15:49:01.337 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56542.log 2024-09-16T15:49:01.338 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40948.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78296.log.gz 2024-09-16T15:49:01.338 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40948.log.gz 2024-09-16T15:49:01.338 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77914.log 2024-09-16T15:49:01.339 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50713.log 2024-09-16T15:49:01.339 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56542.log.gz 2024-09-16T15:49:01.340 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77914.log: gzip -5 --verbose 87.2% -- -- replaced with /var/log/ceph/ceph-client.admin.77914.log.gz 2024-09-16T15:49:01.340 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.80308.log 2024-09-16T15:49:01.341 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51497.log 2024-09-16T15:49:01.341 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50713.log.gz 2024-09-16T15:49:01.342 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77080.log 2024-09-16T15:49:01.342 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80308.log.gz 2024-09-16T15:49:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74953.log 2024-09-16T15:49:01.343 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51497.log.gz 2024-09-16T15:49:01.344 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42754.log 2024-09-16T15:49:01.344 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77080.log.gz 2024-09-16T15:49:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74953.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74953.log.gz 2024-09-16T15:49:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73332.log 2024-09-16T15:49:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45333.log 2024-09-16T15:49:01.347 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42754.log.gz 2024-09-16T15:49:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73332.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73332.log.gz 2024-09-16T15:49:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54574.log 2024-09-16T15:49:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70649.log 2024-09-16T15:49:01.348 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45333.log.gz 2024-09-16T15:49:01.349 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54574.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54574.log.gz 2024-09-16T15:49:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79552.log 2024-09-16T15:49:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55287.log 2024-09-16T15:49:01.351 INFO:teuthology.orchestra.run.smithi060.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.70649.log.gz 2024-09-16T15:49:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79552.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79552.log.gz 2024-09-16T15:49:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55772.log 2024-09-16T15:49:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53897.log 2024-09-16T15:49:01.353 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55287.log.gz 2024-09-16T15:49:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55772.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55772.log.gz 2024-09-16T15:49:01.354 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-09-16T15:49:01.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65432.log 2024-09-16T15:49:01.355 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53897.log.gz 2024-09-16T15:49:01.355 INFO:teuthology.orchestra.run.smithi100.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-16T15:49:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36386.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-09-16T15:49:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70485.log 2024-09-16T15:49:01.357 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40403.log 2024-09-16T15:49:01.357 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65432.log.gz 2024-09-16T15:49:01.358 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70485.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.70485.log.gz 2024-09-16T15:49:01.358 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2024-09-16T15:49:01.359 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36209.log 2024-09-16T15:49:01.359 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40403.log.gz 2024-09-16T15:49:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49429.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2024-09-16T15:49:01.360 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76954.log 2024-09-16T15:49:01.361 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-09-16T15:49:01.361 INFO:teuthology.orchestra.run.smithi060.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36209.log.gz 2024-09-16T15:49:01.362 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76954.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76954.log.gz 2024-09-16T15:49:01.362 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49933.log 2024-09-16T15:49:01.363 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62284.log 2024-09-16T15:49:01.363 INFO:teuthology.orchestra.run.smithi100.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T15:49:01.364 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-09-16T15:49:01.364 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49933.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49933.log.gz 2024-09-16T15:49:01.364 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51581.log 2024-09-16T15:49:01.365 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42049.log 2024-09-16T15:49:01.365 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62284.log.gz 2024-09-16T15:49:01.366 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51581.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51581.log.gz 2024-09-16T15:49:01.366 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58320.log 2024-09-16T15:49:01.367 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-09-16T15:49:01.368 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42049.log.gz 2024-09-16T15:49:01.368 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57587.log 2024-09-16T15:49:01.369 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58320.log.gz 2024-09-16T15:49:01.369 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48199.log 2024-09-16T15:49:01.370 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2024-09-16T15:49:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57587.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57587.log.gz 2024-09-16T15:49:01.371 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59525.log 2024-09-16T15:49:01.371 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66942.log 2024-09-16T15:49:01.372 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48199.log.gz 2024-09-16T15:49:01.373 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59525.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59525.log.gz 2024-09-16T15:49:01.373 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53046.log 2024-09-16T15:49:01.374 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80746.log 2024-09-16T15:49:01.374 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66942.log.gz 2024-09-16T15:49:01.375 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53046.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53046.log.gz 2024-09-16T15:49:01.375 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T15:49:01.376 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66441.log 2024-09-16T15:49:01.376 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80746.log.gz 2024-09-16T15:49:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-09-16T15:49:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66441.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66441.log.gz 2024-09-16T15:49:01.378 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57240.log 2024-09-16T15:49:01.379 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51287.log 2024-09-16T15:49:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44709.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz 2024-09-16T15:49:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57240.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57240.log.gz 2024-09-16T15:49:01.380 INFO:teuthology.orchestra.run.smithi060.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T15:49:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70702.log 2024-09-16T15:49:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56771.log 2024-09-16T15:49:01.381 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51287.log.gz 2024-09-16T15:49:01.382 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55053.log 2024-09-16T15:49:01.382 INFO:teuthology.orchestra.run.smithi060.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.70702.log.gz 2024-09-16T15:49:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72705.log 2024-09-16T15:49:01.383 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56771.log.gz 2024-09-16T15:49:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79972.log 2024-09-16T15:49:01.384 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55053.log.gz 2024-09-16T15:49:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72705.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72705.log.gz 2024-09-16T15:49:01.385 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80912.log 2024-09-16T15:49:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47688.log 2024-09-16T15:49:01.386 INFO:teuthology.orchestra.run.smithi060.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.79972.log.gz 2024-09-16T15:49:01.387 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80912.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80912.log.gz 2024-09-16T15:49:01.387 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61972.log 2024-09-16T15:49:01.388 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67944.log 2024-09-16T15:49:01.388 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47688.log.gz 2024-09-16T15:49:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61972.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.61972.log.gz 2024-09-16T15:49:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77872.log 2024-09-16T15:49:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68212.log 2024-09-16T15:49:01.390 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67944.log.gz 2024-09-16T15:49:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77872.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77872.log.gz 2024-09-16T15:49:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41032.log 2024-09-16T15:49:01.392 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71220.log 2024-09-16T15:49:01.392 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68212.log.gz 2024-09-16T15:49:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41032.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.41032.log.gz 2024-09-16T15:49:01.393 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48718.log 2024-09-16T15:49:01.394 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72789.log 2024-09-16T15:49:01.395 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71220.log.gz 2024-09-16T15:49:01.395 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48718.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48718.log.gz 2024-09-16T15:49:01.396 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76386.log 2024-09-16T15:49:01.397 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41158.log 2024-09-16T15:49:01.397 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72789.log.gz 2024-09-16T15:49:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76386.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76386.log.gz 2024-09-16T15:49:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68609.log 2024-09-16T15:49:01.398 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61293.log 2024-09-16T15:49:01.399 INFO:teuthology.orchestra.run.smithi060.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.41158.log.gz 2024-09-16T15:49:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68609.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68609.log.gz 2024-09-16T15:49:01.400 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44877.log 2024-09-16T15:49:01.401 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42387.log 2024-09-16T15:49:01.401 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61293.log.gz 2024-09-16T15:49:01.402 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66591.log 2024-09-16T15:49:01.402 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44877.log.gz 2024-09-16T15:49:01.403 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46304.log 2024-09-16T15:49:01.403 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42387.log.gz 2024-09-16T15:49:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66591.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66591.log.gz 2024-09-16T15:49:01.404 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76644.log 2024-09-16T15:49:01.405 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62495.log 2024-09-16T15:49:01.405 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gz 2024-09-16T15:49:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76644.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76644.log.gz 2024-09-16T15:49:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72747.log 2024-09-16T15:49:01.407 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71633.log 2024-09-16T15:49:01.407 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62495.log.gz 2024-09-16T15:49:01.408 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72747.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72747.log.gz 2024-09-16T15:49:01.408 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62140.log 2024-09-16T15:49:01.409 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45417.log 2024-09-16T15:49:01.409 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71633.log.gz 2024-09-16T15:49:01.410 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62140.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62140.log.gz 2024-09-16T15:49:01.410 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72268.log 2024-09-16T15:49:01.411 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55095.log 2024-09-16T15:49:01.411 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45417.log.gz 2024-09-16T15:49:01.412 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72268.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.72268.log.gz 2024-09-16T15:49:01.413 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36306.log 2024-09-16T15:49:01.413 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47314.log 2024-09-16T15:49:01.413 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55095.log.gz 2024-09-16T15:49:01.414 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36306.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36306.log.gz 2024-09-16T15:49:01.414 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80788.log 2024-09-16T15:49:01.415 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59784.log 2024-09-16T15:49:01.416 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47314.log.gz 2024-09-16T15:49:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80788.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80788.log.gz 2024-09-16T15:49:01.417 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77631.log 2024-09-16T15:49:01.417 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71362.log 2024-09-16T15:49:01.418 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59784.log.gz 2024-09-16T15:49:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77631.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77631.log.gz 2024-09-16T15:49:01.419 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69447.log 2024-09-16T15:49:01.419 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60122.log 2024-09-16T15:49:01.420 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71362.log.gz 2024-09-16T15:49:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69447.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69447.log.gz 2024-09-16T15:49:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36032.log 2024-09-16T15:49:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51142.log 2024-09-16T15:49:01.422 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60122.log.gz 2024-09-16T15:49:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36032.log: 80.0% -- replaced with /var/log/ceph/ceph-client.admin.36032.log.gz 2024-09-16T15:49:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53449.log 2024-09-16T15:49:01.424 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79069.log 2024-09-16T15:49:01.424 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51142.log.gz 2024-09-16T15:49:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53449.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.53449.log.gz 2024-09-16T15:49:01.425 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74444.log 2024-09-16T15:49:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75209.log 2024-09-16T15:49:01.426 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79069.log.gz 2024-09-16T15:49:01.427 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74444.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74444.log.gzgzip 2024-09-16T15:49:01.427 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59609.log 2024-09-16T15:49:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58488.log 2024-09-16T15:49:01.428 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75209.log.gz 2024-09-16T15:49:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59609.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59609.log.gz 2024-09-16T15:49:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68314.log 2024-09-16T15:49:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80182.log 2024-09-16T15:49:01.430 INFO:teuthology.orchestra.run.smithi060.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.58488.log.gz 2024-09-16T15:49:01.431 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68314.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68314.log.gz 2024-09-16T15:49:01.431 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77332.log 2024-09-16T15:49:01.432 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78985.log 2024-09-16T15:49:01.432 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80182.log.gz 2024-09-16T15:49:01.433 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77332.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77332.log.gz 2024-09-16T15:49:01.433 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62808.log 2024-09-16T15:49:01.434 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75251.log 2024-09-16T15:49:01.435 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78985.log.gz 2024-09-16T15:49:01.435 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62808.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.62808.log.gz 2024-09-16T15:49:01.435 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46346.log 2024-09-16T15:49:01.436 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66525.log 2024-09-16T15:49:01.436 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75251.log.gz 2024-09-16T15:49:01.437 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80620.log 2024-09-16T15:49:01.438 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46346.log.gz 2024-09-16T15:49:01.438 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66858.log 2024-09-16T15:49:01.439 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80620.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66525.log.gz 2024-09-16T15:49:01.440 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80620.log.gz 2024-09-16T15:49:01.440 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70825.log 2024-09-16T15:49:01.440 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55137.log 2024-09-16T15:49:01.441 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66858.log.gz 2024-09-16T15:49:01.441 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69363.log 2024-09-16T15:49:01.442 INFO:teuthology.orchestra.run.smithi060.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.70825.log.gz 2024-09-16T15:49:01.443 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55137.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55137.log.gz 2024-09-16T15:49:01.443 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48546.log 2024-09-16T15:49:01.444 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41284.log 2024-09-16T15:49:01.444 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69363.log.gz 2024-09-16T15:49:01.445 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48546.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48546.log.gz 2024-09-16T15:49:01.445 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51455.log 2024-09-16T15:49:01.446 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59002.log 2024-09-16T15:49:01.446 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41284.log.gz 2024-09-16T15:49:01.447 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51455.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51455.log.gz 2024-09-16T15:49:01.447 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74995.log 2024-09-16T15:49:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74089.log 2024-09-16T15:49:01.448 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59002.log.gz 2024-09-16T15:49:01.449 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74995.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74995.log.gz 2024-09-16T15:49:01.449 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47023.log 2024-09-16T15:49:01.450 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69140.log 2024-09-16T15:49:01.450 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74089.log.gz 2024-09-16T15:49:01.451 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47023.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47023.log.gz 2024-09-16T15:49:01.451 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78550.log 2024-09-16T15:49:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41713.log 2024-09-16T15:49:01.452 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69140.log.gz 2024-09-16T15:49:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.78550.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.78550.log.gz 2024-09-16T15:49:01.453 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67283.log 2024-09-16T15:49:01.454 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45249.log 2024-09-16T15:49:01.454 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41713.log.gz 2024-09-16T15:49:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66125.log 2024-09-16T15:49:01.456 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67283.log.gz 2024-09-16T15:49:01.456 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58528.log 2024-09-16T15:49:01.457 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45249.log.gz 2024-09-16T15:49:01.458 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66125.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66125.log.gz 2024-09-16T15:49:01.458 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72831.log 2024-09-16T15:49:01.459 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73947.log 2024-09-16T15:49:01.459 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.58528.log.gz 2024-09-16T15:49:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72831.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72831.log.gz 2024-09-16T15:49:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71320.log 2024-09-16T15:49:01.461 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55996.log 2024-09-16T15:49:01.461 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73947.log.gz 2024-09-16T15:49:01.462 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71320.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71320.log.gz 2024-09-16T15:49:01.462 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48676.log 2024-09-16T15:49:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50974.log 2024-09-16T15:49:01.463 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55996.log: /var/log/ceph/ceph-client.admin.48676.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55996.log.gz 2024-09-16T15:49:01.464 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48676.log.gz 2024-09-16T15:49:01.464 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77956.log 2024-09-16T15:49:01.465 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46721.log 2024-09-16T15:49:01.465 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50974.log.gz 2024-09-16T15:49:01.466 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56686.log 2024-09-16T15:49:01.466 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77956.log.gz 2024-09-16T15:49:01.467 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46721.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46721.log.gz 2024-09-16T15:49:01.467 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59170.log 2024-09-16T15:49:01.468 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68651.log 2024-09-16T15:49:01.468 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56686.log.gz 2024-09-16T15:49:01.469 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59170.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59170.log.gz 2024-09-16T15:49:01.469 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61167.log 2024-09-16T15:49:01.470 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80494.log 2024-09-16T15:49:01.470 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68651.log.gz 2024-09-16T15:49:01.471 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61167.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61167.log.gz 2024-09-16T15:49:01.471 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55596.log 2024-09-16T15:49:01.472 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57406.log 2024-09-16T15:49:01.472 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80494.log.gz 2024-09-16T15:49:01.473 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55596.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55596.log.gz -5 2024-09-16T15:49:01.473 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52823.log 2024-09-16T15:49:01.474 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67421.log 2024-09-16T15:49:01.474 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57406.log.gz 2024-09-16T15:49:01.475 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52823.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52823.log.gz 2024-09-16T15:49:01.475 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67463.log 2024-09-16T15:49:01.476 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65222.log 2024-09-16T15:49:01.477 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.67421.log.gz 2024-09-16T15:49:01.477 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67463.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.67463.log.gz 2024-09-16T15:49:01.477 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51838.log 2024-09-16T15:49:01.478 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69321.log 2024-09-16T15:49:01.478 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65222.log.gz 2024-09-16T15:49:01.479 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51838.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51838.log.gz 2024-09-16T15:49:01.480 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65988.log 2024-09-16T15:49:01.481 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74528.log 2024-09-16T15:49:01.481 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69321.log.gz 2024-09-16T15:49:01.482 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65988.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65988.log.gz 2024-09-16T15:49:01.482 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75671.log 2024-09-16T15:49:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48119.log 2024-09-16T15:49:01.483 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74528.log.gz 2024-09-16T15:49:01.484 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75671.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75671.log.gzgzip 2024-09-16T15:49:01.484 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70948.log 2024-09-16T15:49:01.485 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35951.log 2024-09-16T15:49:01.485 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48119.log.gz 2024-09-16T15:49:01.486 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70948.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.70948.log.gz 2024-09-16T15:49:01.486 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76744.log 2024-09-16T15:49:01.487 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65600.log 2024-09-16T15:49:01.487 INFO:teuthology.orchestra.run.smithi060.stderr: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.35951.log.gz 2024-09-16T15:49:01.488 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76744.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76744.log.gz 2024-09-16T15:49:01.488 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53669.log 2024-09-16T15:49:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66273.log 2024-09-16T15:49:01.489 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65600.log.gz 2024-09-16T15:49:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53669.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53669.log.gz 2024-09-16T15:49:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69233.log 2024-09-16T15:49:01.491 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50775.log 2024-09-16T15:49:01.491 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66273.log.gz 2024-09-16T15:49:01.492 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62537.log 2024-09-16T15:49:01.492 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69233.log.gz 2024-09-16T15:49:01.493 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36176.log 2024-09-16T15:49:01.493 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50775.log.gz 2024-09-16T15:49:01.494 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62537.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62537.log.gz 2024-09-16T15:49:01.494 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69573.log 2024-09-16T15:49:01.495 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56897.log 2024-09-16T15:49:01.495 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36176.log.gz 2024-09-16T15:49:01.496 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69573.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69573.log.gz 2024-09-16T15:49:01.496 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44961.log 2024-09-16T15:49:01.497 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80266.log 2024-09-16T15:49:01.497 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56897.log.gz 2024-09-16T15:49:01.498 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44961.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.44961.log.gz 2024-09-16T15:49:01.498 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78870.log 2024-09-16T15:49:01.499 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73569.log 2024-09-16T15:49:01.499 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80266.log.gz 2024-09-16T15:49:01.500 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78870.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78870.log.gz 2024-09-16T15:49:01.501 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73630.log 2024-09-16T15:49:01.501 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57065.log 2024-09-16T15:49:01.501 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73569.log.gz 2024-09-16T15:49:01.503 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73630.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73630.log.gz 2024-09-16T15:49:01.503 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74612.log 2024-09-16T15:49:01.504 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42389.log 2024-09-16T15:49:01.504 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57065.log.gz 2024-09-16T15:49:01.505 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74612.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74612.log.gz 2024-09-16T15:49:01.505 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64425.log 2024-09-16T15:49:01.506 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53711.log 2024-09-16T15:49:01.506 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42389.log.gz 2024-09-16T15:49:01.507 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64425.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64425.log.gz 2024-09-16T15:49:01.507 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49162.log 2024-09-16T15:49:01.507 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53711.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-16T15:49:01.508 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53711.log.gz 2024-09-16T15:49:01.509 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49162.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49162.log.gz 2024-09-16T15:49:01.509 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60773.log 2024-09-16T15:49:01.510 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73206.log 2024-09-16T15:49:01.510 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.60773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60773.log.gz 2024-09-16T15:49:01.511 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73466.log 2024-09-16T15:49:01.512 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73206.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73206.log.gz 2024-09-16T15:49:01.512 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36643.log 2024-09-16T15:49:01.513 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80872.log 2024-09-16T15:49:01.514 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73466.log.gz 2024-09-16T15:49:01.514 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36643.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36643.log.gz 2024-09-16T15:49:01.514 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44751.log 2024-09-16T15:49:01.515 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80872.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80872.log.gz 2024-09-16T15:49:01.516 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59273.log 2024-09-16T15:49:01.516 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44590.log 2024-09-16T15:49:01.517 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44751.log.gz 2024-09-16T15:49:01.518 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59273.log: gzip -5 --verbose 87.6% -- -- replaced with /var/log/ceph/ceph-client.admin.59273.log.gz /var/log/ceph/ceph-client.admin.44667.log 2024-09-16T15:49:01.518 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-16T15:49:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77038.log 2024-09-16T15:49:01.519 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44590.log.gz 2024-09-16T15:49:01.520 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44667.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2024-09-16T15:49:01.520 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44017.log 2024-09-16T15:49:01.521 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51966.log 2024-09-16T15:49:01.521 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77038.log.gz 2024-09-16T15:49:01.522 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44017.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.44017.log.gz --verbose 2024-09-16T15:49:01.522 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.58782.log 2024-09-16T15:49:01.523 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42281.log 2024-09-16T15:49:01.524 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51966.log.gz 2024-09-16T15:49:01.525 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58782.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.58782.log.gz 2024-09-16T15:49:01.525 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47256.log 2024-09-16T15:49:01.526 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72491.log 2024-09-16T15:49:01.526 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42281.log.gz 2024-09-16T15:49:01.527 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47256.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47256.log.gz 2024-09-16T15:49:01.527 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69681.log 2024-09-16T15:49:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77206.log 2024-09-16T15:49:01.528 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72491.log.gz 2024-09-16T15:49:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69681.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69681.log.gz 2024-09-16T15:49:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68525.log 2024-09-16T15:49:01.530 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41074.log 2024-09-16T15:49:01.531 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77206.log.gz 2024-09-16T15:49:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68525.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68525.log.gz 2024-09-16T15:49:01.531 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76452.log 2024-09-16T15:49:01.533 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41074.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41074.log.gz 2024-09-16T15:49:01.533 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57450.log 2024-09-16T15:49:01.533 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.76452.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.70784.log 2024-09-16T15:49:01.534 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.76452.log.gz 2024-09-16T15:49:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57450.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57450.log.gz 2024-09-16T15:49:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53294.log 2024-09-16T15:49:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59086.log 2024-09-16T15:49:01.536 INFO:teuthology.orchestra.run.smithi060.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.70784.log.gz 2024-09-16T15:49:01.537 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79612.log 2024-09-16T15:49:01.537 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53294.log.gz 2024-09-16T15:49:01.538 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59086.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59086.log.gz 2024-09-16T15:49:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62056.log 2024-09-16T15:49:01.539 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55179.log 2024-09-16T15:49:01.540 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79612.log.gz 2024-09-16T15:49:01.541 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62056.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62056.log.gz 2024-09-16T15:49:01.541 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68357.log 2024-09-16T15:49:01.542 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50509.log 2024-09-16T15:49:01.542 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55179.log.gz 2024-09-16T15:49:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68357.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68357.log.gz 2024-09-16T15:49:01.543 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58664.log 2024-09-16T15:49:01.544 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50119.log 2024-09-16T15:49:01.545 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50509.log.gz 2024-09-16T15:49:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58664.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.58664.log.gz 2024-09-16T15:49:01.545 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71030.log 2024-09-16T15:49:01.546 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50119.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50119.log.gz 2024-09-16T15:49:01.547 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44500.log 2024-09-16T15:49:01.547 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51880.log 2024-09-16T15:49:01.548 INFO:teuthology.orchestra.run.smithi060.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.71030.log.gz 2024-09-16T15:49:01.549 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44500.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44500.log.gz 2024-09-16T15:49:01.549 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40580.log 2024-09-16T15:49:01.550 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69869.log 2024-09-16T15:49:01.550 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51880.log.gz 2024-09-16T15:49:01.551 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58584.log 2024-09-16T15:49:01.551 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40580.log.gz 2024-09-16T15:49:01.552 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.69869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69869.log.gz 2024-09-16T15:49:01.552 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36444.log 2024-09-16T15:49:01.553 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47730.log 2024-09-16T15:49:01.554 INFO:teuthology.orchestra.run.smithi060.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.58584.log.gz 2024-09-16T15:49:01.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48024.log 2024-09-16T15:49:01.555 INFO:teuthology.orchestra.run.smithi060.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36444.log.gz 2024-09-16T15:49:01.555 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47730.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47730.log.gz 2024-09-16T15:49:01.556 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65180.log 2024-09-16T15:49:01.557 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57629.log 2024-09-16T15:49:01.557 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48024.log.gz 2024-09-16T15:49:01.558 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65180.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.65180.log.gz 2024-09-16T15:49:01.558 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35860.log 2024-09-16T15:49:01.559 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57629.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57629.log.gz 2024-09-16T15:49:01.559 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66483.log 2024-09-16T15:49:01.560 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74570.log 2024-09-16T15:49:01.560 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35860.log.gz 2024-09-16T15:49:01.561 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66483.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66483.log.gz 2024-09-16T15:49:01.561 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69052.log 2024-09-16T15:49:01.562 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74570.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74570.log.gz 2024-09-16T15:49:01.563 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-09-16T15:49:01.563 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79242.log 2024-09-16T15:49:01.564 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69052.log.gz 2024-09-16T15:49:01.565 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50651.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-09-16T15:49:01.565 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62579.log 2024-09-16T15:49:01.566 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79242.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79242.log.gz 2024-09-16T15:49:01.566 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36407.log 2024-09-16T15:49:01.567 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78212.log 2024-09-16T15:49:01.567 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62579.log.gz 2024-09-16T15:49:01.568 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36407.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36407.log.gz 2024-09-16T15:49:01.568 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51016.log 2024-09-16T15:49:01.569 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72663.log 2024-09-16T15:49:01.570 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78212.log.gz 2024-09-16T15:49:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51016.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51016.log.gz 2024-09-16T15:49:01.570 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80536.log 2024-09-16T15:49:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72663.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72663.log.gz 2024-09-16T15:49:01.571 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41705.log 2024-09-16T15:49:01.573 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50571.log 2024-09-16T15:49:01.573 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80536.log.gz 2024-09-16T15:49:01.574 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58860.log 2024-09-16T15:49:01.574 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41705.log.gz 2024-09-16T15:49:01.575 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50571.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50571.log.gz 2024-09-16T15:49:01.575 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54470.log 2024-09-16T15:49:01.576 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53252.log 2024-09-16T15:49:01.576 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58860.log.gz 2024-09-16T15:49:01.577 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68788.log 2024-09-16T15:49:01.577 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54470.log.gz 2024-09-16T15:49:01.578 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53252.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53252.log.gz 2024-09-16T15:49:01.578 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57322.log 2024-09-16T15:49:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68128.log 2024-09-16T15:49:01.580 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68788.log.gz 2024-09-16T15:49:01.581 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57322.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.57322.log.gz --verbose 2024-09-16T15:49:01.581 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.71404.log 2024-09-16T15:49:01.582 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41712.log 2024-09-16T15:49:01.582 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68128.log.gz 2024-09-16T15:49:01.583 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71404.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71404.log.gz 2024-09-16T15:49:01.583 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-09-16T15:49:01.584 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58223.log 2024-09-16T15:49:01.584 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41712.log.gz 2024-09-16T15:49:01.585 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48281.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-09-16T15:49:01.585 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53585.log 2024-09-16T15:49:01.586 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.58223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58223.log.gz 2024-09-16T15:49:01.586 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61725.log 2024-09-16T15:49:01.587 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43018.log 2024-09-16T15:49:01.587 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53585.log.gz 2024-09-16T15:49:01.588 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61725.log: 86.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.61725.log.gz 2024-09-16T15:49:01.588 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71759.log 2024-09-16T15:49:01.590 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36350.log 2024-09-16T15:49:01.590 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43018.log.gz 2024-09-16T15:49:01.591 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71759.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71759.log.gz 2024-09-16T15:49:01.591 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72064.log 2024-09-16T15:49:01.592 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36350.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T15:49:01.592 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36350.log.gz 2024-09-16T15:49:01.593 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72064.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72064.log.gz 2024-09-16T15:49:01.593 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51413.log 2024-09-16T15:49:01.593 INFO:teuthology.orchestra.run.smithi100.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T15:49:01.594 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64509.log 2024-09-16T15:49:01.595 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44185.log 2024-09-16T15:49:01.595 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51413.log.gz 2024-09-16T15:49:01.596 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57801.log 2024-09-16T15:49:01.597 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.64509.log.gz 2024-09-16T15:49:01.598 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48844.log 2024-09-16T15:49:01.598 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44185.log.gz 2024-09-16T15:49:01.599 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48504.log 2024-09-16T15:49:01.599 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57801.log.gz 2024-09-16T15:49:01.600 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42712.log 2024-09-16T15:49:01.600 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48844.log.gz 2024-09-16T15:49:01.602 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74317.log 2024-09-16T15:49:01.602 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48504.log.gz 2024-09-16T15:49:01.603 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69531.log 2024-09-16T15:49:01.603 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42712.log.gz 2024-09-16T15:49:01.604 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52238.log 2024-09-16T15:49:01.604 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74317.log.gz 2024-09-16T15:49:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-09-16T15:49:01.605 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69531.log.gz 2024-09-16T15:49:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52238.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-16T15:49:01.606 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79781.log 2024-09-16T15:49:01.606 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52238.log.gz 2024-09-16T15:49:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.42043.log: --verbose -- /var/log/ceph/ceph-client.admin.54658.log 2024-09-16T15:49:01.608 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gz 2024-09-16T15:49:01.608 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79342.log 2024-09-16T15:49:01.608 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79781.log.gz 2024-09-16T15:49:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54658.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54658.log.gz 2024-09-16T15:49:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72398.log 2024-09-16T15:49:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69405.log 2024-09-16T15:49:01.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79342.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79342.log.gz 2024-09-16T15:49:01.611 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72398.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72398.log.gz 2024-09-16T15:49:01.611 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45800.log 2024-09-16T15:49:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76870.log 2024-09-16T15:49:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69405.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69405.log.gz 2024-09-16T15:49:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45800.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45800.log.gz 2024-09-16T15:49:01.614 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53795.log 2024-09-16T15:49:01.614 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56813.log 2024-09-16T15:49:01.615 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76870.log.gz 2024-09-16T15:49:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53795.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53795.log.gz 2024-09-16T15:49:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62705.log 2024-09-16T15:49:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.56813.log: --verbose -- /var/log/ceph/ceph-client.0.52744.log 2024-09-16T15:49:01.617 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56813.log.gz 2024-09-16T15:49:01.618 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62705.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62705.log.gz 2024-09-16T15:49:01.618 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76064.log 2024-09-16T15:49:01.619 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.0.52744.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62242.log 2024-09-16T15:49:01.619 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52744.log.gz 2024-09-16T15:49:01.620 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76064.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76064.log.gz 2024-09-16T15:49:01.620 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58362.log 2024-09-16T15:49:01.621 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62242.log: -- /var/log/ceph/ceph-client.admin.35987.log 2024-09-16T15:49:01.621 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62242.log.gz 2024-09-16T15:49:01.622 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58362.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58362.log.gz 2024-09-16T15:49:01.622 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54275.log 2024-09-16T15:49:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68170.log 2024-09-16T15:49:01.623 INFO:teuthology.orchestra.run.smithi060.stderr: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.35987.log.gz 2024-09-16T15:49:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54275.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54275.log.gz 2024-09-16T15:49:01.624 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75377.log 2024-09-16T15:49:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42382.log 2024-09-16T15:49:01.625 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68170.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68170.log.gz 2024-09-16T15:49:01.626 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75377.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75377.log.gz 2024-09-16T15:49:01.626 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2024-09-16T15:49:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56500.log 2024-09-16T15:49:01.627 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42382.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42382.log.gz 2024-09-16T15:49:01.628 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47482.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz 2024-09-16T15:49:01.628 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71278.log 2024-09-16T15:49:01.629 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41711.log 2024-09-16T15:49:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56500.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56500.log.gz 2024-09-16T15:49:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65775.log 2024-09-16T15:49:01.630 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71278.log.gz 2024-09-16T15:49:01.631 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41711.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.41711.log.gz 2024-09-16T15:49:01.631 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42796.log 2024-09-16T15:49:01.632 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53149.log 2024-09-16T15:49:01.633 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65775.log.gz 2024-09-16T15:49:01.633 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42796.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.42796.log.gz -5 2024-09-16T15:49:01.634 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.71488.log 2024-09-16T15:49:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53149.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53149.log.gz 2024-09-16T15:49:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T15:49:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68968.log 2024-09-16T15:49:01.636 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71488.log.gz 2024-09-16T15:49:01.637 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60393.log 2024-09-16T15:49:01.638 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66357.log 2024-09-16T15:49:01.638 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68968.log.gz 2024-09-16T15:49:01.639 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79426.log 2024-09-16T15:49:01.640 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60393.log.gz 2024-09-16T15:49:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57885.log 2024-09-16T15:49:01.641 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66357.log.gz 2024-09-16T15:49:01.642 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47772.log 2024-09-16T15:49:01.642 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79426.log.gz 2024-09-16T15:49:01.643 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53336.log 2024-09-16T15:49:01.643 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57885.log.gz 2024-09-16T15:49:01.644 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50932.log 2024-09-16T15:49:01.644 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47772.log.gz 2024-09-16T15:49:01.646 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59866.log 2024-09-16T15:49:01.646 INFO:teuthology.orchestra.run.smithi060.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53336.log.gz 2024-09-16T15:49:01.647 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2024-09-16T15:49:01.647 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50932.log.gz 2024-09-16T15:49:01.648 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66083.log 2024-09-16T15:49:01.648 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59866.log.gz 2024-09-16T15:49:01.649 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50181.log 2024-09-16T15:49:01.649 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2024-09-16T15:49:01.650 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68693.log 2024-09-16T15:49:01.650 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66083.log.gz 2024-09-16T15:49:01.652 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74257.log 2024-09-16T15:49:01.652 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-09-16T15:49:01.654 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70321.log 2024-09-16T15:49:01.654 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68693.log.gz 2024-09-16T15:49:01.654 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57160.log 2024-09-16T15:49:01.654 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74257.log.gz 2024-09-16T15:49:01.655 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54370.log 2024-09-16T15:49:01.655 INFO:teuthology.orchestra.run.smithi060.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.70321.log.gz 2024-09-16T15:49:01.656 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57717.log 2024-09-16T15:49:01.656 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57160.log.gz 2024-09-16T15:49:01.658 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74871.log 2024-09-16T15:49:01.658 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54370.log.gz 2024-09-16T15:49:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59908.log 2024-09-16T15:49:01.659 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57717.log.gz 2024-09-16T15:49:01.660 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42053.log 2024-09-16T15:49:01.660 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74871.log.gz 2024-09-16T15:49:01.661 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51676.log 2024-09-16T15:49:01.661 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59908.log.gz 2024-09-16T15:49:01.663 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61930.log 2024-09-16T15:49:01.663 INFO:teuthology.orchestra.run.smithi060.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.42053.log.gz 2024-09-16T15:49:01.663 INFO:teuthology.orchestra.run.smithi100.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T15:49:01.664 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48453.log 2024-09-16T15:49:01.664 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51676.log.gz 2024-09-16T15:49:01.665 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42046.log 2024-09-16T15:49:01.665 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61930.log.gz 2024-09-16T15:49:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59315.log 2024-09-16T15:49:01.666 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48453.log.gz 2024-09-16T15:49:01.667 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56981.log 2024-09-16T15:49:01.667 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42046.log.gz 2024-09-16T15:49:01.669 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44311.log 2024-09-16T15:49:01.669 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59315.log.gz 2024-09-16T15:49:01.669 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59128.log 2024-09-16T15:49:01.670 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56981.log.gz 2024-09-16T15:49:01.670 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76306.log 2024-09-16T15:49:01.671 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44311.log.gz 2024-09-16T15:49:01.672 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52322.log 2024-09-16T15:49:01.672 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59128.log.gz 2024-09-16T15:49:01.673 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77164.log 2024-09-16T15:49:01.673 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76306.log.gz 2024-09-16T15:49:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53210.log 2024-09-16T15:49:01.674 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52322.log.gz 2024-09-16T15:49:01.675 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75605.log 2024-09-16T15:49:01.676 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77164.log.gz 2024-09-16T15:49:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72579.log 2024-09-16T15:49:01.677 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53210.log.gz 2024-09-16T15:49:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75335.log 2024-09-16T15:49:01.679 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.75605.log.gz 2024-09-16T15:49:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51202.log 2024-09-16T15:49:01.679 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72579.log.gz 2024-09-16T15:49:01.681 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75335.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-16T15:49:01.681 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75335.log.gz 2024-09-16T15:49:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70116.log 2024-09-16T15:49:01.682 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51202.log.gz 2024-09-16T15:49:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80578.log 2024-09-16T15:49:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47178.log 2024-09-16T15:49:01.684 INFO:teuthology.orchestra.run.smithi060.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.70116.log.gz 2024-09-16T15:49:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80578.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80578.log.gz 2024-09-16T15:49:01.686 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41326.log 2024-09-16T15:49:01.687 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47178.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47178.log.gz 2024-09-16T15:49:01.688 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66779.log 2024-09-16T15:49:01.688 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41326.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41326.log.gz 2024-09-16T15:49:01.689 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70032.log 2024-09-16T15:49:01.690 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.66779.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.0.66779.log.gz -- 2024-09-16T15:49:01.690 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.42032.log 2024-09-16T15:49:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36074.log 2024-09-16T15:49:01.692 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70032.log.gz 2024-09-16T15:49:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45187.log 2024-09-16T15:49:01.693 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42032.log.gz 2024-09-16T15:49:01.694 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58918.log 2024-09-16T15:49:01.694 INFO:teuthology.orchestra.run.smithi060.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36074.log.gz 2024-09-16T15:49:01.695 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64792.log 2024-09-16T15:49:01.695 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45187.log.gz 2024-09-16T15:49:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58918.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58918.log.gz 2024-09-16T15:49:01.697 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60351.log 2024-09-16T15:49:01.698 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64792.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64792.log.gz 2024-09-16T15:49:01.699 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77248.log 2024-09-16T15:49:01.699 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60351.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.60351.log.gz 2024-09-16T15:49:01.700 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58035.log 2024-09-16T15:49:01.701 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36416.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77248.log.gz 2024-09-16T15:49:01.701 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-16T15:49:01.702 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58035.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58035.log.gz 2024-09-16T15:49:01.703 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80014.log 2024-09-16T15:49:01.704 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45003.log 2024-09-16T15:49:01.704 INFO:teuthology.orchestra.run.smithi060.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.36416.log.gz 2024-09-16T15:49:01.705 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80014.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80014.log.gz 2024-09-16T15:49:01.706 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67543.log 2024-09-16T15:49:01.706 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45003.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45003.log.gz 2024-09-16T15:49:01.708 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45087.log 2024-09-16T15:49:01.708 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67543.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.67543.log.gz 2024-09-16T15:49:01.709 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54616.log 2024-09-16T15:49:01.710 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65138.log 2024-09-16T15:49:01.710 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45087.log.gz 2024-09-16T15:49:01.712 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54616.log: gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54616.log.gz 2024-09-16T15:49:01.712 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.67866.log 2024-09-16T15:49:01.712 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65138.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65138.log.gz 2024-09-16T15:49:01.713 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T15:49:01.714 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67379.log 2024-09-16T15:49:01.714 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67866.log.gz 2024-09-16T15:49:01.716 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42031.log 2024-09-16T15:49:01.717 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67379.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67379.log.gz 2024-09-16T15:49:01.718 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42031.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42031.log.gz 2024-09-16T15:49:01.718 INFO:teuthology.orchestra.run.smithi060.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T15:49:01.718 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69990.log 2024-09-16T15:49:01.720 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44143.log 2024-09-16T15:49:01.721 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69990.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69990.log.gz 2024-09-16T15:49:01.721 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62200.log 2024-09-16T15:49:01.722 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62621.log 2024-09-16T15:49:01.723 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44143.log.gz 2024-09-16T15:49:01.724 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67337.log 2024-09-16T15:49:01.724 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62200.log.gz 2024-09-16T15:49:01.725 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73508.log 2024-09-16T15:49:01.725 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62621.log.gz 2024-09-16T15:49:01.726 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59950.log 2024-09-16T15:49:01.726 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.67337.log.gz 2024-09-16T15:49:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73508.log: gzip -5 --verbose -- 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73508.log.gz 2024-09-16T15:49:01.728 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.55554.log 2024-09-16T15:49:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59950.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59950.log.gz 2024-09-16T15:49:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80098.log 2024-09-16T15:49:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55554.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55554.log.gz 2024-09-16T15:49:01.731 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.73869.log 2024-09-16T15:49:01.732 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80098.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80098.log.gz 2024-09-16T15:49:01.732 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72873.log 2024-09-16T15:49:01.733 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78170.log 2024-09-16T15:49:01.734 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73869.log.gz 2024-09-16T15:49:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42054.log 2024-09-16T15:49:01.735 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72873.log.gz 2024-09-16T15:49:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78170.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78170.log.gz 2024-09-16T15:49:01.736 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58624.log 2024-09-16T15:49:01.737 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42054.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42054.log.gz 2024-09-16T15:49:01.737 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70280.log 2024-09-16T15:49:01.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58624.log: 85.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64928.log 2024-09-16T15:49:01.739 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58624.log.gz 2024-09-16T15:49:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70280.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.70280.log.gz 2024-09-16T15:49:01.740 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48970.log 2024-09-16T15:49:01.741 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64928.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64928.log.gz 2024-09-16T15:49:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72449.log 2024-09-16T15:49:01.743 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40820.log 2024-09-16T15:49:01.743 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48970.log.gz 2024-09-16T15:49:01.744 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72449.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72449.log.gz 2024-09-16T15:49:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55011.log 2024-09-16T15:49:01.745 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40820.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40820.log.gz 2024-09-16T15:49:01.746 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78002.log 2024-09-16T15:49:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78086.log 2024-09-16T15:49:01.747 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55011.log.gz 2024-09-16T15:49:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66231.log 2024-09-16T15:49:01.748 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78002.log.gz 2024-09-16T15:49:01.749 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57545.log 2024-09-16T15:49:01.750 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78086.log.gz 2024-09-16T15:49:01.751 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75081.log 2024-09-16T15:49:01.751 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66231.log.gz 2024-09-16T15:49:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57545.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57545.log.gz 2024-09-16T15:49:01.752 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73248.log 2024-09-16T15:49:01.753 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75081.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75081.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71717.log 2024-09-16T15:49:01.753 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-16T15:49:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74791.log 2024-09-16T15:49:01.755 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73248.log.gz 2024-09-16T15:49:01.756 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64279.log 2024-09-16T15:49:01.756 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71717.log.gz 2024-09-16T15:49:01.758 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60979.log 2024-09-16T15:49:01.758 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74791.log.gz 2024-09-16T15:49:01.758 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64279.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64279.log.gz 2024-09-16T15:49:01.759 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80954.log 2024-09-16T15:49:01.760 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60979.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60979.log.gz 2024-09-16T15:49:01.760 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36844.log 2024-09-16T15:49:01.761 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44269.log 2024-09-16T15:49:01.762 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80954.log.gz 2024-09-16T15:49:01.763 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36844.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36844.log.gz 2024-09-16T15:49:01.763 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60895.log 2024-09-16T15:49:01.764 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44269.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44269.log.gz 2024-09-16T15:49:01.765 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72354.log 2024-09-16T15:49:01.766 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65858.log 2024-09-16T15:49:01.766 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60895.log.gz 2024-09-16T15:49:01.767 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72354.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78755.log 2024-09-16T15:49:01.767 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72354.log.gz 2024-09-16T15:49:01.768 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59441.log 2024-09-16T15:49:01.769 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65858.log.gz 2024-09-16T15:49:01.770 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78755.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78755.log.gz 2024-09-16T15:49:01.770 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57843.log 2024-09-16T15:49:01.772 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59441.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59441.log.gz 2024-09-16T15:49:01.773 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68441.log 2024-09-16T15:49:01.774 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62411.log 2024-09-16T15:49:01.774 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57843.log.gz 2024-09-16T15:49:01.775 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68441.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68441.log.gz 2024-09-16T15:49:01.776 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62411.log: -- /var/log/ceph/ceph-client.admin.60224.log 2024-09-16T15:49:01.776 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62411.log.gz 2024-09-16T15:49:01.777 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66032.log 2024-09-16T15:49:01.779 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80704.log 2024-09-16T15:49:01.779 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60224.log.gz 2024-09-16T15:49:01.780 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48886.log 2024-09-16T15:49:01.780 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66032.log.gz 2024-09-16T15:49:01.781 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69010.log 2024-09-16T15:49:01.781 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80704.log.gz 2024-09-16T15:49:01.783 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64467.log 2024-09-16T15:49:01.783 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48886.log.gz 2024-09-16T15:49:01.784 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45585.log 2024-09-16T15:49:01.784 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69010.log.gz 2024-09-16T15:49:01.786 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64467.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64467.log.gz 2024-09-16T15:49:01.786 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.45585.log: --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2024-09-16T15:49:01.786 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45585.log.gz 2024-09-16T15:49:01.787 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75859.log 2024-09-16T15:49:01.789 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54107.log 2024-09-16T15:49:01.789 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2024-09-16T15:49:01.789 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.75859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75859.log.gz 2024-09-16T15:49:01.790 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78671.log 2024-09-16T15:49:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54107.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54107.log.gz -5 2024-09-16T15:49:01.792 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56626.log 2024-09-16T15:49:01.793 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45754.log 2024-09-16T15:49:01.793 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78671.log.gz 2024-09-16T15:49:01.794 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56626.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56626.log.gz 2024-09-16T15:49:01.794 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49707.log 2024-09-16T15:49:01.795 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45754.log.gz 2024-09-16T15:49:01.795 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65096.log 2024-09-16T15:49:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41410.log 2024-09-16T15:49:01.797 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49707.log.gz 2024-09-16T15:49:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65096.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65096.log.gz 2024-09-16T15:49:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59483.log 2024-09-16T15:49:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41410.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41410.log.gz 2024-09-16T15:49:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46094.log 2024-09-16T15:49:01.801 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60852.log 2024-09-16T15:49:01.801 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59483.log.gz 2024-09-16T15:49:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46094.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46094.log.gz 2024-09-16T15:49:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47982.log 2024-09-16T15:49:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60852.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60852.log.gz 2024-09-16T15:49:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46262.log 2024-09-16T15:49:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41718.log 2024-09-16T15:49:01.806 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47982.log.gz 2024-09-16T15:49:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-09-16T15:49:01.807 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46262.log.gz 2024-09-16T15:49:01.808 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56220.log 2024-09-16T15:49:01.808 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41718.log.gz 2024-09-16T15:49:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46412.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-09-16T15:49:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57675.log 2024-09-16T15:49:01.810 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56220.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.56220.log.gz 2024-09-16T15:49:01.811 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76786.log 2024-09-16T15:49:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79739.log 2024-09-16T15:49:01.812 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57675.log.gz 2024-09-16T15:49:01.813 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76786.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76786.log.gz 2024-09-16T15:49:01.814 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2024-09-16T15:49:01.815 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79739.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79739.log.gz 2024-09-16T15:49:01.816 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62663.log 2024-09-16T15:49:01.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42628.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2024-09-16T15:49:01.817 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80830.log 2024-09-16T15:49:01.818 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79697.log 2024-09-16T15:49:01.818 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62663.log.gz 2024-09-16T15:49:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80830.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80830.log.gz 2024-09-16T15:49:01.820 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60585.log 2024-09-16T15:49:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79697.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79697.log.gz 2024-09-16T15:49:01.821 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57927.log 2024-09-16T15:49:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60585.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60585.log.gz 2024-09-16T15:49:01.823 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40864.log 2024-09-16T15:49:01.824 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75938.log 2024-09-16T15:49:01.825 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57927.log.gz 2024-09-16T15:49:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40864.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40864.log.gz 2024-09-16T15:49:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41714.log 2024-09-16T15:49:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75938.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75938.log.gz 2024-09-16T15:49:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54839.log 2024-09-16T15:49:01.828 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74360.log 2024-09-16T15:49:01.828 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41714.log.gz 2024-09-16T15:49:01.830 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76828.log 2024-09-16T15:49:01.830 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54839.log.gz 2024-09-16T15:49:01.831 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74360.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74360.log.gz 2024-09-16T15:49:01.831 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46220.log 2024-09-16T15:49:01.832 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53939.log 2024-09-16T15:49:01.833 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76828.log.gz 2024-09-16T15:49:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80662.log 2024-09-16T15:49:01.834 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46220.log.gz 2024-09-16T15:49:01.835 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60164.log 2024-09-16T15:49:01.835 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53939.log.gz 2024-09-16T15:49:01.836 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40906.log 2024-09-16T15:49:01.836 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80662.log.gz 2024-09-16T15:49:01.838 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70526.log 2024-09-16T15:49:01.838 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60164.log.gz 2024-09-16T15:49:01.839 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80224.log 2024-09-16T15:49:01.839 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40906.log.gz 2024-09-16T15:49:01.840 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76252.log 2024-09-16T15:49:01.840 INFO:teuthology.orchestra.run.smithi060.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.70526.log.gz 2024-09-16T15:49:01.841 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79384.log 2024-09-16T15:49:01.842 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80224.log.gz 2024-09-16T15:49:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65348.log 2024-09-16T15:49:01.843 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76252.log.gz 2024-09-16T15:49:01.844 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74696.log 2024-09-16T15:49:01.844 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79384.log.gz 2024-09-16T15:49:01.845 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78044.log 2024-09-16T15:49:01.845 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65348.log.gz 2024-09-16T15:49:01.846 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74696.log: gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74696.log.gz 2024-09-16T15:49:01.846 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.70444.log 2024-09-16T15:49:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78044.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78044.log.gz 2024-09-16T15:49:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72939.log 2024-09-16T15:49:01.849 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-09-16T15:49:01.849 INFO:teuthology.orchestra.run.smithi060.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.70444.log.gz 2024-09-16T15:49:01.850 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72939.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72939.log.gz 2024-09-16T15:49:01.851 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35880.log 2024-09-16T15:49:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45658.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-09-16T15:49:01.852 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40990.log 2024-09-16T15:49:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.35880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35880.log.gz 2024-09-16T15:49:01.853 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51539.log 2024-09-16T15:49:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36137.log 2024-09-16T15:49:01.855 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40990.log.gz 2024-09-16T15:49:01.856 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51539.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51539.log.gz 2024-09-16T15:49:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61335.log 2024-09-16T15:49:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36137.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36137.log.gz 2024-09-16T15:49:01.857 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68926.log 2024-09-16T15:49:01.859 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69191.log 2024-09-16T15:49:01.859 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61335.log.gz 2024-09-16T15:49:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68926.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68926.log.gz 2024-09-16T15:49:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51329.log 2024-09-16T15:49:01.861 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.69191.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69191.log.gz 2024-09-16T15:49:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54233.log 2024-09-16T15:49:01.863 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45291.log 2024-09-16T15:49:01.864 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51329.log.gz 2024-09-16T15:49:01.865 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54233.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54233.log.gz 2024-09-16T15:49:01.866 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55954.log 2024-09-16T15:49:01.866 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45291.log.gz 2024-09-16T15:49:01.867 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70157.log 2024-09-16T15:49:01.868 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65516.log 2024-09-16T15:49:01.868 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55954.log.gz 2024-09-16T15:49:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70157.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.70157.log.gz 2024-09-16T15:49:01.870 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74173.log 2024-09-16T15:49:01.871 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42880.log 2024-09-16T15:49:01.871 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65516.log.gz 2024-09-16T15:49:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74173.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74173.log.gz 2024-09-16T15:49:01.873 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2024-09-16T15:49:01.874 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76022.log 2024-09-16T15:49:01.874 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42880.log.gz 2024-09-16T15:49:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46962.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2024-09-16T15:49:01.875 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41200.log 2024-09-16T15:49:01.877 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53837.log 2024-09-16T15:49:01.877 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76022.log.gz 2024-09-16T15:49:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41200.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41200.log.gz 2024-09-16T15:49:01.878 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49054.log 2024-09-16T15:49:01.879 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53837.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53837.log.gz 2024-09-16T15:49:01.880 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71927.log 2024-09-16T15:49:01.881 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49054.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29828.log 2024-09-16T15:49:01.881 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49054.log.gz 2024-09-16T15:49:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71927.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71927.log.gz 2024-09-16T15:49:01.882 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42388.log 2024-09-16T15:49:01.883 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T15:49:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.tmp-client.admin.29828.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29828.log.gz 2024-09-16T15:49:01.884 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50323.log 2024-09-16T15:49:01.885 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42384.log 2024-09-16T15:49:01.885 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42388.log.gz 2024-09-16T15:49:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50323.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50323.log.gz 2024-09-16T15:49:01.886 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61113.log 2024-09-16T15:49:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42384.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42384.log.gz 2024-09-16T15:49:01.888 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71969.log 2024-09-16T15:49:01.889 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2024-09-16T15:49:01.889 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61113.log.gz 2024-09-16T15:49:01.891 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71969.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71969.log.gz 2024-09-16T15:49:01.891 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78713.log 2024-09-16T15:49:01.891 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48365.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2024-09-16T15:49:01.892 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68254.log 2024-09-16T15:49:01.894 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52865.log 2024-09-16T15:49:01.894 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78713.log.gz 2024-09-16T15:49:01.895 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50447.log 2024-09-16T15:49:01.895 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68254.log.gz 2024-09-16T15:49:01.896 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52865.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52865.log.gz 2024-09-16T15:49:01.896 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70198.log 2024-09-16T15:49:01.897 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.50447.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76566.log 2024-09-16T15:49:01.897 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50447.log.gz 2024-09-16T15:49:01.899 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52556.log 2024-09-16T15:49:01.899 INFO:teuthology.orchestra.run.smithi060.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.70198.log.gz 2024-09-16T15:49:01.900 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.76566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67609.log 2024-09-16T15:49:01.900 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76566.log.gz 2024-09-16T15:49:01.901 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59399.log 2024-09-16T15:49:01.901 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52556.log.gz 2024-09-16T15:49:01.902 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67609.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67609.log.gz 2024-09-16T15:49:01.903 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55850.log 2024-09-16T15:49:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59399.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59399.log.gz 2024-09-16T15:49:01.904 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77122.log 2024-09-16T15:49:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60309.log 2024-09-16T15:49:01.906 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55850.log.gz 2024-09-16T15:49:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47814.log 2024-09-16T15:49:01.907 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77122.log.gz 2024-09-16T15:49:01.908 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60309.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60309.log.gz -5 2024-09-16T15:49:01.908 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66900.log 2024-09-16T15:49:01.909 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52364.log 2024-09-16T15:49:01.909 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47814.log.gz 2024-09-16T15:49:01.910 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56729.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66900.log.gz 2024-09-16T15:49:01.911 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-16T15:49:01.911 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52364.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52364.log.gz 2024-09-16T15:49:01.912 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59230.log 2024-09-16T15:49:01.913 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71590.log 2024-09-16T15:49:01.913 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56729.log.gz 2024-09-16T15:49:01.914 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46679.log 2024-09-16T15:49:01.915 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59230.log.gz 2024-09-16T15:49:01.915 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71590.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71590.log.gz 2024-09-16T15:49:01.916 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36196.log 2024-09-16T15:49:01.917 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46679.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.65054.log -- replaced with /var/log/ceph/ceph-client.admin.46679.log.gz 2024-09-16T15:49:01.918 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-16T15:49:01.919 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65902.log 2024-09-16T15:49:01.919 INFO:teuthology.orchestra.run.smithi060.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36196.log.gz 2024-09-16T15:49:01.920 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65054.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65054.log.gz 2024-09-16T15:49:01.920 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62098.log 2024-09-16T15:49:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65902.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65902.log.gz 2024-09-16T15:49:01.921 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64637.log 2024-09-16T15:49:01.922 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61251.log 2024-09-16T15:49:01.923 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62098.log.gz 2024-09-16T15:49:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64637.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64637.log.gz 2024-09-16T15:49:01.924 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72144.log 2024-09-16T15:49:01.925 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61251.log.gz 2024-09-16T15:49:01.926 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73774.log 2024-09-16T15:49:01.928 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.72144.log: --verbose -- /var/log/ceph/ceph-client.admin.70074.log 2024-09-16T15:49:01.928 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6%/var/log/ceph/ceph-client.admin.73774.log: -- replaced with /var/log/ceph/ceph-client.admin.72144.log.gz 2024-09-16T15:49:01.928 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73774.log.gz 2024-09-16T15:49:01.929 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64383.log 2024-09-16T15:49:01.930 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56363.log 2024-09-16T15:49:01.930 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70074.log.gz 2024-09-16T15:49:01.931 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53753.log 2024-09-16T15:49:01.931 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.64383.log.gz 2024-09-16T15:49:01.933 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60477.log 2024-09-16T15:49:01.933 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56363.log.gz 2024-09-16T15:49:01.934 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57759.log 2024-09-16T15:49:01.934 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53753.log.gz 2024-09-16T15:49:01.935 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60477.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60477.log.gz 2024-09-16T15:49:01.935 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53107.log 2024-09-16T15:49:01.936 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57759.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62369.log 2024-09-16T15:49:01.937 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57759.log.gz 2024-09-16T15:49:01.938 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48409.log 2024-09-16T15:49:01.938 INFO:teuthology.orchestra.run.smithi060.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53107.log.gz 2024-09-16T15:49:01.939 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62369.log: gzip -5 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62369.log.gz 2024-09-16T15:49:01.939 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68002.log 2024-09-16T15:49:01.940 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48409.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48409.log.gz 2024-09-16T15:49:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59357.log 2024-09-16T15:49:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.68002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60937.log 2024-09-16T15:49:01.942 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68002.log.gz 2024-09-16T15:49:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71446.log 2024-09-16T15:49:01.943 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59357.log.gz 2024-09-16T15:49:01.944 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60937.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60937.log.gz 2024-09-16T15:49:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51245.log 2024-09-16T15:49:01.946 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71446.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71446.log.gz 2024-09-16T15:49:01.946 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67651.log 2024-09-16T15:49:01.947 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66399.log 2024-09-16T15:49:01.947 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51245.log.gz 2024-09-16T15:49:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54065.log 2024-09-16T15:49:01.948 INFO:teuthology.orchestra.run.smithi060.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.67651.log.gz 2024-09-16T15:49:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66399.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66399.log.gz 2024-09-16T15:49:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54065.log: -- /var/log/ceph/ceph-client.admin.47856.log 2024-09-16T15:49:01.951 INFO:teuthology.orchestra.run.smithi060.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54065.log.gz 2024-09-16T15:49:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36394.log 2024-09-16T15:49:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51058.log 2024-09-16T15:49:01.953 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47856.log.gz 2024-09-16T15:49:01.954 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36394.log: gzip -5 --verbose -- 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36394.log.gz 2024-09-16T15:49:01.955 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.58960.log 2024-09-16T15:49:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51058.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51058.log.gz 2024-09-16T15:49:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77290.log 2024-09-16T15:49:01.957 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-09-16T15:49:01.957 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58960.log.gz 2024-09-16T15:49:01.958 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41494.log 2024-09-16T15:49:01.959 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77290.log.gz 2024-09-16T15:49:01.960 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40681.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-09-16T15:49:01.960 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-09-16T15:49:01.961 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41494.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41494.log.gz 2024-09-16T15:49:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49629.log 2024-09-16T15:49:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70989.log 2024-09-16T15:49:01.963 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-09-16T15:49:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49629.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49629.log.gz 2024-09-16T15:49:01.965 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.70989.log: --verbose -- /var/log/ceph/ceph-client.admin.65558.log 2024-09-16T15:49:01.965 INFO:teuthology.orchestra.run.smithi060.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.70989.log.gz 2024-09-16T15:49:01.966 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78924.log 2024-09-16T15:49:01.968 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65558.log: gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65558.log.gz 2024-09-16T15:49:01.968 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.45375.log 2024-09-16T15:49:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78924.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78924.log.gz 2024-09-16T15:49:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T15:49:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51100.log 2024-09-16T15:49:01.971 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45375.log.gz 2024-09-16T15:49:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52070.log 2024-09-16T15:49:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51100.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51100.log.gz 2024-09-16T15:49:01.974 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62326.log 2024-09-16T15:49:01.975 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52070.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52070.log.gz 2024-09-16T15:49:01.976 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75419.log 2024-09-16T15:49:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62326.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62326.log.gz 2024-09-16T15:49:01.978 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.75419.log: -- /var/log/ceph/ceph-client.admin.61846.log 2024-09-16T15:49:01.979 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.75419.log.gz 2024-09-16T15:49:01.980 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46805.log 2024-09-16T15:49:01.981 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61846.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61846.log.gz 2024-09-16T15:49:01.982 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-16T15:49:01.982 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46805.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46805.log.gz 2024-09-16T15:49:01.983 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49471.log 2024-09-16T15:49:01.986 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.49471.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49471.log.gz 2024-09-16T15:49:01.987 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47440.log 2024-09-16T15:49:01.990 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36939.log 2024-09-16T15:49:01.991 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47440.log.gz 2024-09-16T15:49:01.992 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65012.log 2024-09-16T15:49:01.995 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.36939.log: gzip -5 86.7%/var/log/ceph/ceph-client.admin.65012.log: --verbose -- /var/log/ceph/ceph-client.admin.51922.log 2024-09-16T15:49:01.995 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65012.log.gz 2024-09-16T15:49:01.995 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36939.log.gz 2024-09-16T15:49:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46178.log 2024-09-16T15:49:02.001 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.51922.log: /var/log/ceph/ceph-client.admin.46178.log: 87.6%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.51922.log.gz -- 2024-09-16T15:49:02.001 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.60267.log 2024-09-16T15:49:02.002 INFO:teuthology.orchestra.run.smithi060.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46178.log.gz 2024-09-16T15:49:02.004 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62884.log 2024-09-16T15:49:02.004 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60267.log.gz 2024-09-16T15:49:02.006 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71885.log 2024-09-16T15:49:02.008 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62884.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.62884.log.gz 2024-09-16T15:49:02.010 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41699.log 2024-09-16T15:49:02.012 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71885.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71885.log.gz 2024-09-16T15:49:02.015 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71675.log 2024-09-16T15:49:02.018 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.71675.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41699.log.gz 2024-09-16T15:49:02.018 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52907.log 2024-09-16T15:49:02.019 INFO:teuthology.orchestra.run.smithi060.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T15:49:02.019 INFO:teuthology.orchestra.run.smithi060.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.71675.log.gz 2024-09-16T15:49:02.020 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53627.log 2024-09-16T15:49:02.022 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.52907.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52907.log.gz 2024-09-16T15:49:02.022 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72226.log 2024-09-16T15:49:02.024 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-16T15:49:02.024 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.53627.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53627.log.gz 2024-09-16T15:49:02.024 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45703.log 2024-09-16T15:49:02.025 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.72226.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72226.log.gz 2024-09-16T15:49:02.026 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43957.log 2024-09-16T15:49:02.027 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45703.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45703.log.gz 2024-09-16T15:49:02.028 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54191.log 2024-09-16T15:49:02.030 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43957.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43957.log.gz 2024-09-16T15:49:02.030 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44919.log 2024-09-16T15:49:02.031 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.54191.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54191.log.gz 2024-09-16T15:49:02.032 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70567.log 2024-09-16T15:49:02.033 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44919.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44919.log.gz 2024-09-16T15:49:02.034 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.70567.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.70567.log.gz 2024-09-16T15:49:02.035 INFO:teuthology.orchestra.run.smithi100.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T15:49:02.044 INFO:teuthology.orchestra.run.smithi100.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T15:49:02.065 INFO:teuthology.orchestra.run.smithi060.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T15:49:02.146 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T15:49:02.149 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-16T15:49:02.149 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m1.029s 2024-09-16T15:49:02.149 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m5.080s 2024-09-16T15:49:02.149 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m0.514s 2024-09-16T15:49:02.445 INFO:teuthology.orchestra.run.smithi060.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T15:49:02.524 INFO:teuthology.orchestra.run.smithi060.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi060.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T15:49:02.560 INFO:teuthology.orchestra.run.smithi060.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T15:49:02.753 INFO:teuthology.orchestra.run.smithi060.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T15:49:02.756 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-16T15:49:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m1.640s 2024-09-16T15:49:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m5.522s 2024-09-16T15:49:02.756 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m2.290s 2024-09-16T15:49:02.757 INFO:tasks.ceph:Archiving logs... 2024-09-16T15:49:02.757 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/remote/smithi060/log 2024-09-16T15:49:02.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T15:49:04.042 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/remote/smithi100/log 2024-09-16T15:49:04.042 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T15:49:04.708 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T15:49:04.722 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T15:49:04.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T15:49:04.727 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T15:49:04.847 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T15:49:04.858 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T15:49:04.858 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:49:04.863 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:49:04.894 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:49:04.896 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:49:04.902 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:49:04.902 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-09-16T15:49:04.902 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 -53us[ -61us] +/- 36ms 2024-09-16T15:49:04.902 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -256us[ -263us] +/- 85ms 2024-09-16T15:49:04.903 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +375us[ +367us] +/- 88ms 2024-09-16T15:49:04.903 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:49:04.903 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:49:04.903 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-16T15:49:04.903 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 -52us[ -43us] +/- 36ms 2024-09-16T15:49:04.903 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -222us[ -213us] +/- 85ms 2024-09-16T15:49:04.903 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +374us[ +383us] +/- 88ms 2024-09-16T15:49:04.903 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:49:04.905 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T15:49:04.916 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T15:49:04.916 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T15:49:04.931 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T15:49:05.090 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T15:49:05.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T15:49:05.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:49:05.271 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T15:49:05.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:49:05.331 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T15:49:05.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T15:49:05.357 INFO:teuthology.task.internal:Duration was 1092.052021 seconds 2024-09-16T15:49:05.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T15:49:05.373 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T15:49:05.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T15:49:05.377 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T15:49:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:49:05.464 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:49:05.926 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T15:49:05.927 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-09-16T15:49:05.927 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T15:49:05.977 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-09-16T15:49:05.977 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T15:49:06.026 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T15:49:06.027 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:49:06.031 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:49:06.089 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T15:49:06.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T15:49:06.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T15:49:06.268 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T15:49:06.277 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T15:49:06.277 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T15:49:06.333 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T15:49:06.389 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T15:49:06.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:49:06.405 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:49:06.471 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-09-16T15:49:06.475 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-09-16T15:49:06.504 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:49:06.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:49:06.538 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:49:06.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:49:06.571 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T15:49:06.583 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T15:49:06.583 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/remote/smithi060 2024-09-16T15:49:06.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T15:49:06.649 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907357/remote/smithi100 2024-09-16T15:49:06.650 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T15:49:06.712 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T15:49:06.712 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T15:49:06.716 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T15:49:06.750 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T15:49:06.761 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T15:49:06.761 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T15:49:06.770 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T15:49:06.770 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T15:49:06.792 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T15:49:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: 393253 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 15:49 /home/ubuntu/cephtest 2024-09-16T15:49:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: 393253 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 15:49 /home/ubuntu/cephtest 2024-09-16T15:49:06.829 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T15:49:06.841 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T15:49:06.889 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1092.05202126503 flavor: default owner: scheduled_avan@teuthology success: true 2024-09-16T15:49:06.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T15:49:06.969 INFO:teuthology.run:pass